AN EVALUATION OF A LEADERSHIP DEVELOPMENT PROGRAM FOR
INTERCOLLEGIATE WRESTLING COACHES
By
Eric Adrian Bean

A DISSERTATION
Submitted to
Michigan State University
in partial fulfillment of the requirements
for the degree of
DOCTOR OF PHILOSOPHY
KINESIOLOGY
2011

ABSTRACT
AN EVALUATION OF A LEADERSHIP DEVELOPMENT PROGRAM FOR
INTERCOLLEGIATE WRESTLING COACHES
By
Eric Adrian Bean
The overarching purpose of this study was to investigate the efficacy of two online
modules, comprising part of a coaching leadership course, in achieving the course
objectives as defined by key stakeholders. In addition the study aimed to examine
coaches‘ opinions regarding the ease and effectiveness of an online learning environment.
This was accomplished in three phases. First, in Phase One the researcher conducted
qualitative interviews with key stakeholders to identify the outcomes and objectives of
the intervention broadly, and of two of the online modules, specifically. The second
phase aimed to examine the effectiveness of the two online modules in increasing
knowledge, intended use, and actual use of material covered in the modules. Due to
unforeseen circumstances in obtaining and retaining a large enough control or
comparison group, a single group longitudinal design had to be employed in Phase Two.
Phase three was designed to examine the intervention group coaches‘ perspectives
regarding the ease and utility of the online video format. The stakeholders in Phase One
reported that their program goals were of both a macro and micro nature. For example,
they hoped that the coaches would, on a macro level: develop a broader awareness of all
that their coaching role includes, approach the coaching role similar to that of a CEO, be
willing to reflect on their strengths and weaknesses, etcetera. On a micro level, for
example, the Phase One participants hoped that the coaches would: improve
communication skills, develop a vision for their program, and bring in assistant coaches

or interns to supplement their weaknesses. In Phase Two, causality determinations were
limited, however, results indicated that the online modules effectively increased
knowledge and intended use of the material covered. Finally, Phase Three demonstrated
that the coaches perceived that the online modules met their needs, were easy to use, and
were an effective way of delivering that style of leadership content.

ACKNOWLEDGMENTS
I would like to first acknowledge and thank my mother, Alice, for not only
supporting and believing in me but also continuously challenging me to stretch beyond
self-imposed limits and weaknesses. In addition, my mother instilled in me a passion for
learning that will never cease. Thank you Mom.
I would like to extend a very special thank you to my committee members for
guiding me on this journey. There were many times where I struggled to see the light at
the end of the tunnel (or believed that the light belonged to a train) and yet you managed
to help me maintain some semblance of confidence that I would see this through to the
end. Not only were you patient with me but also you really helped me expand my
viewing angles and see the golden nuggets within this study that I was sure didn‘t exist.
Dr. Campbell, your knowledge and experience in evaluation research impacted the
quality of this evaluation immeasurably. Not only did you impact, and perhaps save, this
study but you have given me the confidence to continue to engage in evaluation research
beyond MSU. Dr. Ewing, thank you for everything. It was clear to me from the beginning
that you cared about me not only completing my dissertation but more importantly
learning from the journey. Your feedback and guidance throughout allowed me to see
how this experience was more than just another step in completing my doctorate. Dr.
Lauer, thank you for challenging me to broaden my scope and perspective with regard to
this dissertation. Your ability to apply information and knowledge from seemingly
unrelated sources helped enrich this dissertation and will help me improve my future
research and consulting.

iv

I would like to thank my advisor and committee chair Dr. Dan Gould. Dan, I have
always been in awe of your ability to guide and lead from the side or from behind. While
at times frustrating, you resisted in just giving me the answer and truly challenged and
guided me to find it on my own. I do not believe that I would have been able to complete
this dissertation without you. Lastly, your commitment to expanding knowledge in the
field of sport psychology and love of learning is infectious and I am so grateful to have
had the opportunity to be mentored by you. Thank you again to my committee, I will take
all of your collective guidance, lessons, and energy with me beyond MSU.
I would also like to thank the director of the NWCA, Mike Moyer, for providing
me unprecedented access to collegiate wrestling coaches around the country and for
providing me insight into the inner workings of collegiate wrestling. Thank you to the
coaches who voluntarily participated in this study.
Finally, I like to extend a special thank you to my wife and best friend, Nicole.
You never once stopped believing in me (even though I did a number of times). You
never once fell short in providing me what I needed to complete this journey. You were
my foundation and backbone that I leaned on a number of times. I am so blessed to have
you in my life and my only hope is that I can impact you the way you have and continue
to impact me. I love you.

v

TABLE OF CONTENTS

LIST OF FIGURES ......................................................................................................... viii
LIST OF TABLES ............................................................................................................. ix
CHAPTER 1
INTRODUCTION
Nature of the Problem .............................................................................................1
National Wrestling Coaches Association Leadership Academy .............................4
Evaluation Methodology .........................................................................................7
Purpose of the Study..............................................................................................10
Hypotheses ............................................................................................................11
CHAPTER 2
REVIEW OF LITERATURE
Responsibilities of Today‘s College Coach ..........................................................12
National Wrestling Coaches Association Leadership Academy ...........................13
Evaluation Literature .............................................................................................16
Types of Evaluation ...............................................................................................17
Evaluation Theory. ................................................................................................22
Leadership Literature.............................................................................................27
Conceptualizing and Defining Leadership ...........................................................27
Leadership Development and Evaluation. .............................................................29
Leadership in Sport................................................................................................32
CHAPTER 3
METHOD
Phase One: Qualitative Interviews of Key Stakeholders .......................................35
Phase Two: Evaluation of Online Module‘s 1 and 2 Intervention ........................37
Phase Three: Coaches‘ Opinions Regarding Ease and Utility of the Online
Video Format ........................................................................................................42
Flow Chart of Participant Experience Phases Two and Three ..............................43
Data Analysis .........................................................................................................44
CHAPTER 4
RESULTS
Phase One ..............................................................................................................62
Phase Two .............................................................................................................66
Phase Three ...........................................................................................................71
CHAPTER 5
DISCUSSION
vi

Phase One: Qualitative Interviews of Key Stakeholders .......................................74
Phase Two: Evaluation of Online Module‘s 1 and 2 Intervention ........................78
Phase Three: Coaches‘ Opinions Regarding Ease and Utility of the Online
Video Format .........................................................................................................82
Implications and Future Directions for the National Wrestling Coaches
Association .............................................................................................................84
Implications and Future Directions for Evaluation ................................................89
Implications for Coaching Education ....................................................................92
Strengths and Limitations ......................................................................................93
APPENDICES
Appendix A. Phase One Participants Demographic Questionnaire .......................98
Appendix B. Interview Guide ..............................................................................101
Appendix C. Phase Two Participants Demographic Questionnaire ....................104
Appendix D. Effectiveness of NWCA Leadership Academy Modules 1 and 2
(Time 1) ................................................................................................................107
Appendix E. Effectiveness of NWCA Leadership Academy Modules 1 and 2
(Time 2) ...............................................................................................................112
Appendix F. Follow-up Survey for Intervention Group 2 ...................................117
Appendix G. Retention and Use of NWCA Leadership Academy Modules
1 and 2 (Time 3) ...................................................................................................119
Appendix H. Control Group Recruitment Email .................................................125
Appendix I. Participants by Time ........................................................................127
Appendix J. Subscales at each time point ............................................................128
REFERENCES .........................................................................................................134

vii

LIST OF FIGURES

Figure 1. Flow Chart of Participant Experience Phases Two and Three ...................43

viii

LIST OF TABLES

Table 1. Objectives of Stakeholders in Phase One .....................................................49
Table 2. Intervention and Control Group Demographics … ......................................56
Table 3. Independent Sample t-test on Survey 2 ........................................................58
Table 4. Paired Sample t-test Intervention Group Time 1 to Time 2 .........................62
Table 5. Paired Sample t-test on the Subscales of Knowledge and Intention
Time 1 to Time 2 ........................................................................................................65
Table 6. Paired Sample t-test Intervention Group Time 2 to Time 3 .........................68
Table 7. Paired Sample t-test on the Subscales of Knowledge and Intention
Time 2 to Time 3 ........................................................................................................69
Table 8. Paired Sample t-test Intervention Group Time 1 to Time 3 .........................69
Table 9. Means and Standard Deviations for select Survey 3 items ..........................70
Table 10. Impact of Each Teaching Component in Modules 1 and 2 ........................71

ix

CHAPTER 1
INTRODUCTION
Nature of the problem
Wrestling is one of the oldest sports in the world with a history dating back to
antiquity. Over the centuries it has provided millions of young people not only a vehicle to
develop physically but also the enjoyment of participating in an exciting sport with
opportunities to learn important life lessons through their involvement. For example, in
addition to technical skills and tactics, wrestling coaches can use wrestling to teach their
athletes about the value of hard work and how to handle failure.
In the United States wrestling has had a rich history as a collegiate sport helpin g
colleges and universities meet their educational athletic objectives while allowing many
young men and women to further develop their wrestling skills and capabilities. However,
over the last 30 years collegiate wrestling has come under attack. Programs are being
eliminated as the costs of sponsoring intercollegiate athletic programs have skyrocketed.
With the NCAA only requiring universities at the Division I level to field a minimum of six
men‘s sports wrestling competes with the other 15 Olympic sports for a limited number of
varsity programs. Program elimination has occurred despite the fact that the NCAA Division
I Wrestling Championships is one of the most successful championships of all NCAA
sports—ranking between fourth and fifth in terms of revenue generated (NCAA, 2008).
Considering that there are a variety of reasons that programs are being cut it is
important to examine specific programs that have been cut. For example, Portland State
University and Delaware State University had their wrestling programs dropped due to poor
academic performance (Brown, 2009; Prince, 2009). While both programs were said to have

1

deficiencies in a variety of areas it was stated that the lack of improvement with the
Academic Progress Report (APR) was an area of major concern (Brown, 2009). Essentially,
the academic issues not only affected the wrestling team, but the athletic department as well
and therefore the administration at Delaware State University felt that the NCAA penalties,
as well as the economic factors, would be too great to absorb, and decided to cut the
wrestling program. In addition to academic concerns, some programs have been cut due to
low roster numbers. For example, Lawrence University in Wisconsin, despite a long and
successful history, suspended their program due to the fact that they did not have enough
participants to sustain the program (Acker, 2008). Finally, despite the tremendous presence
of high school wrestlers in the surrounding community and the fact that the Oregon wrestling
program was in good academic standing, the Oregon wrestling program was cut in order to
reinstate baseball at the university (Miller, 2007).
In addition to these publicized reasons for cutting programs, some believe that there
are other factors that athletic departments evaluate when determining whether or not to
eliminate a sport program. These factors include: (1) is the sport recognized as part of the
multi-conference the program belongs to; (2) ability to be nationally competitive; (3)
indigenous to the region from where the school recruits students; (4) is the sport funded at a
level consistent with the other programs in the conference or region; (5) small, enrollment
conscious schools, look for inexpensive sports that have large roster sizes; and (6) are there
many off the field incidences (M. Moyer, personal communication, December 7, 2009).
Finally, Mike Moyer, Executive Director of the National Wrestling Coaches Association
stated, ―many athletic directors have told [him] behind closed doors…that the tendency is to
eliminate teams that are high maintenance, either the players and or the coach‖ (M. Moyer,

2

personal communication, December 7, 2009). In summation, the publicized factors for
eliminating programs include economic hindrances, Title IX compliance, academic
shortcomings, and low roster numbers, whereas a less publicized factor for eliminating a
sport program is when either the coach, athletes, or both are high maintenance and disruptive
to the overarching mission of the athletic department.
While much of the stress placed on collegiate wrestling programs results from the
weakened economy and educational policies, there is a strong feeling that many programs
that have been eliminated, or face the threat of elimination, are in that position because of the
wrestling coaches. The coaches, while well-meaning and strong tacticians, lack strong
leadership and student-athlete development skills and capabilities. In particular, having
studied eliminated programs over the last 30 years the National Wrestling Coaches
Association (NWCA) has come to believe that a lack of off-the-mat leadership is a
significant issue. Coaches have not successfully garnered campus and community support for
their programs likely because these coaches were lacking in communication skills and have
not been successful at cultivating alumni involvement and donor support. Furthermore, they
have had wrestlers who demonstrated poor citizenship by getting into trouble off the mat or
failing to live up to their academic responsibilities. A need exists to address these issues by
providing leadership training to America‘s collegiate wrestling coaches with the goal of
maintaining and then growing the number of intercollegiate programs nationally.
Finally, it should be noted that while there are major threats to collegiate wrestling,
opportunities for growth within the sport exist. Many small colleges and universities are
enrollment driven and offer collegiate sports to help attract students to their campuses and
because fewer males than females apply to college today having a collegiate wrestling

3

program can bring more male students to campus. To accomplish this, however, coaches
must have the skills to ensure that their athletes develop academically and as people and
know how to raise funds to support their programs while linking them and their athletes to
academic and local communities.
National Wrestling Coaches Association Leadership Academy
In consideration of this threat the NWCA partnered with the Institute for the Study of
Youth Sports (ISYS) to develop a state-of-the art leadership development program for
America‘s collegiate wrestling coaches. Building from sport psychology, sport science and
business leadership research and best practices, and what is known about effective coaching
leadership, the program will address a number of important issues. These issues include: (1)
an understanding of educational athletics and the role wrestling plays in the lager academic
community; (2) program vision and mission; (3) the coach as a leader and principle s of
leadership; (4) public relations, marketing and fund raising skills; (5) effective campus and
community relations; and (6) strategies for developing wrestlers as people and the promotion
of life skills through wrestling involvement.
Over the last 10 years a transformation has taken place in education in that technology is
changing the way we are educating individuals (Coyner & McCann, 2004; Larreamendy-Joerns
& Leinhardt, 2006; Li & Irby, 2008). Perhaps the biggest technological change is in online
instructional formats. Some examples of sport organizations that utilize online education are the
National Federation of State High School Associations and the American Sport Education
Program, as well as numerous higher education institutions. Educational programs and
universities, including Michigan State University, are increasingly utilizing online education
both in Kinesiology and general education programs (Singh & Pan, 2004). The logical reasons

4

for utilizing online courses are decreased costs, convenience and accessibility to the consumer
(Coyner & McCann, 2004), and the ability to reach a larger market of potential learners due to
the fact that learners can complete the course from nearly any location. Therefore, the NWCA
Coaching Leadership Academy will have a strong online presence of lectures and interactive
material for the coach to work through, but will also include readings, an in-person coaching
leadership institute, then an individualized coaching plan for enhancing one‘s program, and a
process of peer mentoring. Thus, the program design is a three-pronged approach. The first
part of the leadership academy is 10 online video modules. Each module will take between
60 and 120 minutes to complete and include narrated PowerPoint‘s, interviews with
successful coaches, interviews with experts in various arenas of leadership, and interactive
exercises designed to aid the coach in developing his or her specific leadership needs.
Module 1 is designed to achieve five objectives: (1) increase the coach‘s awareness of threats
to the sport of wrestling; (2) create a sense of urgency for the coach to take responsibility for
saving wrestling by making his or her own program all it is capable of being; (3) help the
coach formulate a plan for taking his or her program to the next level; (4) motivate the coach
to improve his or her leadership skills and act to improve his or her program; and (5) have a
coach develop a vision for his or her program. Module 2 is designed to achieve four
objectives: (1) help the coach understand the importance of effective leadership; (2) increase
the coaches understanding of components of effective and ineffective leadership; (3) help the
coach understand his or her leadership style; and (4) help the coach develop effective leader
communication skills. Modules 3 to10 focus on developing all aspects of the wrestling
program, public and community relations, marketing and fundraising, ensuring wrestlers
develop academically, developing the wrestler as a person, helping wrestlers stay healt hy,

5

coaching ethics, and career building for coaches, but will not be discussed in detail because
they are not the focus of this investigation nor were they completed and available for
evaluation at the start of the investigation.
Finally, each coach is being asked to complete a 360-degree evaluation of his or her
coaching leadership ability. The general purpose of a 360-degree evaluation is to provide
leaders an opportunity to compare their self-perceptions with the observations of the
individuals that they are leading (or coaching), peers, and a supervisor (Kets de Vries,
Vrignaud, Florent-Treacy, 2004). In conjunction with the ISYS, the NWCA created a 360degree evaluation to examine a coach‘s ability to lead in a number of areas including: vision
and goals, communication skills, motivation and feedback, working relationships with
athletic administrators and supporting staff, marketing, recruiting and retention, fund raising,
and the athletes‘ academic development.
Following completion of the 10 online modules the coaches will then partake in an inperson leadership workshop. The in-person portion of the academy will include a small
group of coaches interacting with successful, veteran coaches for two-days engaging in indepth leadership development. Following this portion of the academy the coaches will be
matched to a successful coach mentor, and will engage in a traditional mentor-mentee
relationship.
The NWCA leaders believe that behavioral change will occur over time through the
combination of online educational materials, interaction and education at a leadership
summit, and through successful coaches mentoring other coaches to develop their leadership
abilities. The overarching goal of the Leadership Academy is to improve the leadership
ability of the coaches and in so doing decrease the amount of collegiate wrestling programs

6

that are cut in the United States and hopefully regain some of those that were lost. Currently
this program is the early stages of deployment and as such an opportunity exists t o evaluate
two of the online modules.
Evaluation Methodology
Evaluation is the ―use of social research methods to systematically investigate the
effectiveness of social intervention programs in ways that are adapted to their political and
organizational environments and are designed to inform social action to improve social
conditions‖ (Rossi, Lipsey, & Freeman, 2004, p. 16). Scriven (1991) offered an alternative
definition that evaluation is aimed at determining the merit, worth, and value of programs or
interventions. Given these definitions program evaluators utilize established research methods to
measure the effectiveness or value of a program or intervention. There are many different
methods of program evaluation that occur at different points along the developmental timeline of
a particular program and the research methods are determined by the type of evaluation questions
being asked. A brief overview of the different methods of program evaluation follows.
Many educational or social programs are designed to fill a specific void or alleviate a
social problem (Rossi, Lipsey, & Freeman, 2004). Therefore, it is critical to evaluate the extent
to which there is a need for a particular program. This form of evaluation is commonly referred
to as a needs assessment, which is ―a systematic set of procedures undertaken for the purpose of
setting priorities and making decisions about program or organizational improvement and
allocation of resources. The priorities are based on identified needs‖ (Witkin & Altshuld, 1995,
p. 4). A need is often defined as a gap between what is and what should be (Witkin & Altshuld,
1995). There are many reasons for conducting a needs assessment particularly that people within
a given population can perceive their needs as being quite different. Needs assessments ―are

7

conducted to derive information and perceptions of values as a guide to making policy and
program decisions that will benefit specific groups of people‖ (Witkin & Altshuld, 1995, p. 5).
Needs assessments can be undertaken for a variety of reasons, but typically are intended to lead
to action, change, and improvement.
Another series of evaluation questions focus on how a program is designed and
conceptualized. Rarely do program stakeholders have a clear plan for the process of how their
program is supposed to work and therefore an assessment of program theory is needed (Rossi,
Lipsey, & Freeman, 2004). One method within this type of evaluation is evaluability
assessments. An evaluability assessment is a process that helps one identify which evaluations
would be useful and feasible, and begin the process of designing evaluations (Wholey, Hatry, &
Newcomer, 2004). Essentially an evaluability assessment examines the program model by
clearly defining goals and objectives of the program, and if a program is ready to be assessed or
is evaluable. Typically, a program is not ready to be assessed when there is internal disagreement
on program goals and performance criteria, goals do not fit the activity and or resources,
performance indicators cannot be collected at a reasonable cost, or program managers are
unwilling or unable to create change. However, a program is evaluable when goals are well
defined, goals are plausible, an evaluator can obtain data at a reasonable cost, and program
stakeholders are willing to act on the data obtained from the evaluation.
Evaluators conducting evaluability assessments are attempting to understand the program
methods, objectives, delivery process, and the social reality through interviews, observations,
and existing data. Evaluability assessments are aimed at getting program stakeholders to clearly
define and articulate the programs design and goals. Additionally, some evaluators conducting an
evaluability assessment will, from the data collected, create a logic model that describes logical

8

connections between activities and outcomes (Wholey, Hatry, & Newcomer, 2004). Logic
models can identify the resources or inputs that lead to program outputs and outcomes.
Another common set of evaluation questions examines the impact or outcomes of a
program. Outcome evaluation is an assessment of the degree of change in the outcomes over
time (before, during, after, and following the conclusion of the program). An outcome evaluation
does not necessarily involve a basis of comparison. Usually outcome evaluation is more small
scale, time limited, and within the scope of a well-resourced community organization (Rossi,
Lipsey, & Freeman, 2004). An impact evaluation is inherently comparative (randomized
experiments or field experiments); looks at both program outcomes and also longer-term impacts
or impacts at other levels of analysis. These are larger scale, resource intensive, done for
programs for which it makes sense to invest this kind of effort. A program is ready for impact
evaluation when there is an existence of program theory; a decent amount of process data has
already been collected and or is readily accessible, timelines and expectations have been
reviewed, and resources are available for the evaluation.
One major challenge in impact assessment is to identify measurable goals. The
terminology of goals and objectives can be more burdensome than facilitative for program
staff (Patton, 2008). Rather one should focus on what program stakeholders are trying t o
achieve and the differences that they will observe within the program participants. The key is
to have them identify meaningful goals rather than precise goals that they are not truly
interested in.
Evaluation studies are typically performed to evaluate social intervention programs that
are aimed at improving a social condition, such as decreasing homelessness in a particular
community through educational programs. However, these methods could be adapted to

9

evaluating online educational programs such as the NWCA Coaches Leadership Academy
outlined above. Considering that the Leadership Academy is in its infancy and is still being
developed a unique opportunity exists to evaluate both the implementation and the outcomes of
the leadership academy.
Throughout the development of this Leadership Academy, the key stakeholders have
examined a variety of educational methods (in-person, one-one, online, etcetera) that could be
used to meet the leadership development needs of collegiate coaches. Rather than just choosing
one approach, the stakeholders decided that the most effective approach would be to involve a
combination of all three educational approaches. The program stakeholders are very much
concerned with the outcomes of the intervention; however, it is also essential to know what
occurred in the program that can be reasonably connected to those outcomes (Patton, 2008).
Because the program is still being designed, an evaluation should not only examine outcomes,
but also the process and implementation of the program.
Patton suggested that interventions or programs could fail for two likely reasons: (1) the
intervention was not implemented as designed or (2) the theory was not effective. By simply
evaluating outcomes one might miss what specific factors of the program led to, or hindered,
those outcomes. Therefore, the current evaluation will examine preliminary outcomes by
measuring knowledge gained, intended use, and actual use of the material covered in the first
two online modules, but will also evaluate implementation by examining how the coaches
interacted with the program (i.e. how much time they spent in the modules, whether or not they
completed the recommended exercises, etcetera) and what aspects of the modules best fit their
needs and were the most impactful.
Purpose of the Study

10

There are three main purposes of this study. The first is to clearly identify the
outcome and implementation objectives of the first two online modules as defined by
program stakeholders. The second is to examine the impact of the first two online modules
on the participating coaches with regard to the objectives identified by the stakeholders. For
the use of this study impact will be defined by examining changes in knowledge, use, and
intended use of the principles in the first two Modules. Finally, an evaluation of user
preferences and effectiveness regarding the delivery of the program will take place.
Hypotheses
Between-Group Hypotheses
1. The intervention group will experience significant increases in knowledge, use, and
intended use of the principles of the first two Modules compared to the control group.
2. The control group will experience no significant increases in knowledge, use, and
intended use of the principles of the first two Modules.
Within-Group Hypotheses
3. The intervention group will experience significant increases in knowledge, use, and
intended use from Time 1 to Time 2
4. No specific prediction was made from Time 2 to Time 3 for the intervention group.

11

CHAPTER 2
REVIEW OF LITERATURE
The purpose of this chapter is to provide a review of the literature that is relevant to this
study. The chapter begins by providing a rationale for developing an online leadership course for
intercollegiate wrestling coaches. Next, research pertaining to methodology in evaluation
research, including theories and challenges of evaluation, is presented. The evaluation literature
is followed by a brief summary of relevant leadership literature that is aimed at providing the
reader with an adequate amount of information relative to the leadership strategies presented in
the online course. Finally, research pertaining to leadership development will be discussed.
Responsibilities of Today’s College Coach
The college coach of today is required to do much more than just provide technical and
tactical instruction to his or her athletes (Baber & Eckrich, 1998; MacLean & Zakrajsek, 1996;
Mallett & Côté, 2006). The college coach of today has to effectively recruit and retain athletes,
maintain effective alumni relations, organize camps and clinics, actively engage in fundraising
opportunities, and establish a quality relationship with the surrounding community. Furthermore,
the performance of intercollegiate coaches is measured in multiple dimensions including team
outcomes, team academic performance, and recruiting quality (Cunningham & Dixon, 2003).
Although the coach is required to do much more than just coach, few are formally educated in
the areas that are required of a coach beyond the technical and tactical aspects.
The focus of current coaching education programs is aimed at high school and youth
sport coaches. For example, the National Federation of State High School Associations (NFHS)
provides online training to interscholastic coaches in teaching and modeling behavior, sport
specific technique, and general fundamentals of coaching (National Federation of State High

12

School Associations, 2009). Similarly, the American Sport Education Program provides
coaching education courses to over 25,000 coaches a year and offers courses on coaching
principles, sport physiology, sport psychology, and first aid in addition to other topics (American
Sport Education Program, 2009). Both programs offer courses within an online instructional
format and are mostly utilized by high school coaches.
Considering that most coaching educational programs are geared towards high school and
youth sport coaches and few aim to educate the coach beyond player safety and development, a
need exists then to provide intercollegiate coaches with an educational program that is designed
to develop the myriad of responsibilities required of a college coach. In addition, a need exists to
offer intercollegiate coaches the opportunity to develop skills beyond player development and
safety; particularly because they are being evaluated in other areas (MacLean & Zakrajsek,
1996).
National Wrestling Coaches Association Leadership Academy
Collegiate wrestling in the United States has a long and rich history helping universities
meet their educational athletic objectives while allowing many young men and women
opportunities to further develop their wrestling skills and capabilities. However, over the last 30
years programs have been eliminated as the costs of sponsoring intercollegiate athletic programs
have skyrocketed and because the NCAA only requires universities at the Division I level to
field a minimum of six men‘s sports. Therefore, wrestling has to compete with 15 other Olympic
sports, as well as Title IX requirements, for a limited number of varsity programs. Despite the
fact that the NCAA Division I Wrestling Championships is one of the most successful
championships of all NCAA sports, ranking between fourth and fifth in terms of revenue
generated (NCAA, 2008), programs are continuing to be eliminated.

13

The National Wrestling Coaches Association (NWCA) has studied eliminated programs
over the last 30 years, and while much of the stress placed on collegiate wrestling programs is a
result of the financial stress of universities, some believe program elimination is also a result of
the coaches. The coaches, while well meaning, lack strong leadership and student-athlete
development skills and capabilities. The NWCA has come to believe that collegiate wrestling
coaches are lacking the necessary leadership skills required of them outside of the wrestling
room. For example, coaches have not successfully garnered campus and community support for
their programs likely because these coaches were lacking communication, fundraising, and
public relations skills. Additionally, some programs have had wrestlers getting into trouble off
the mat or failing to live up to their academic responsibilities. The NWCA wishes to halt this
alarming trend of program elimination, and, ideally, reverse it, by providing leadership training
to America‘s collegiate coaches to address these issues.
In consideration of this threat the NWCA partnered with the Institute for the Study of
Youth Sports (ISYS) to develop a state-of-the art leadership development program for America‘s
collegiate wrestling coaches. The team that was assembled to create the leadership development
program included the director of the NWCA who was the engineer behind expressing the need
for such a program and has roughly 10 years of working with every coach in the country which
allowed him to create the vision for why this program was needed. The second member of this
team included the director of the ISYS who was selected because he has over 30 years of
developing and implementing coaches education programs. The third member of the team
included a collegiate Associate Athletic Director who was selected because he could offer
another perspective on the needs of college coaches. Building from business leadership research
and best practices, and what is known about effective coaching leadership, the development team

14

created a program that addresses a number of important issues. These issues include: (1) an
understanding of educational athletics and the role wrestling plays in the lager academic
community; (2) program vision and mission; (3) the coach as a leader and principles of
leadership; (4) public relations, marketing and fund raising skills; (5) effective campus and
community relations; and (6) strategies for developing wrestlers as people and the promotion of
life skills through wrestling involvement.
The program will have a strong online presence of lectures and interactive material for
the coach to work through, but will also include an in-person coaching leadership workshop, then
an individualized coaching plan for enhancing one‘s program, and a process of peer mentoring
from coaches who have been successful both on and off the mat. The NWCA leaders believe that
change will occur over time through the combination of online educational materials, interaction
and education at a leadership summit, and through successful coaches working with other
coaches to develop their leadership abilities. The overarching goal is to improve the leadership
ability of the coaches and in so doing decrease the amount of collegiate wrestling programs that
are cut in the United States and hopefully regain some of those programs that were lost.
Currently this program is the early stages of deployment and as such an opportunity exists to
evaluate some of the Modules.
Considering that the program is in the early stages of deployment it is useful to evaluate
the process and preliminary outcomes of the first two modules amongst a sample of collegiate
wrestling coaches. Therefore, it is critical to gain an understanding of evaluation literature and
methodology in order to appropriately examine, and define, ‗effectiveness.‘

15

Evaluation Literature
Considering that the current economic climate is weak and there are budget cuts being
made in nearly every industry, the era of accountability is likely to reach new heights as more
organizations, foundations, government programs, and universities will need statistical evidence
that their money is being spent appropriately and effectively. A need exists within the field of
coaching education online courses to implement more evaluation research.
Some evaluations have been conducted in the past, for example, Gould, Petlichkoff,
Hodge, and Simons (1990) evaluated the effectiveness of a psychological skills workshop in two
studies, with a group of senior elite wrestlers (Study 1) and a group of junior wrestlers (Study 2).
The participants completed a pre-camp test, post-camp test, and a 3-month follow-up test. An
analysis of the results of their study indicated that the program was effective in facilitating
change in the wrestlers‘ knowledge, perceived importance, and use of psychological skills
covered in the workshop. However, the results should be interpreted with caution because neither
study included a control group due to ethical and logistical issues. In one of the few studies to
evaluate online education in Kinesiology, Mary Barron (2006) examined the impact of an injury
prevention program on injury rates, coaches proficiency in areas of first aid and injury
prevention, coaches opinions of the program, and if change occurred within coaches decision
making ability following completion of the injury prevention program. Using quantitative
measures to assess the effectiveness of the course, Barron found positive findings regarding the
impact of the injury prevention program, however, a quarter of the coaches failed the follow-up
examination three months after completing the program. One finding of particular interest
indicated that coaches preferred the web-based presentation method to a traditional lecture style
course. It was reported that the coaches preferred to take the course online as opposed to in a

16

lecture because they were able to take the program when it suited them best, rather than having
to set aside a couple of hours on a weekend or a week night.
Gould, et al. (1990) recommended that evaluation research of interventions should
continue to be conducted. Additionally, in a paper on program planning in positive youth
development programs, Petitpas, Cornelius, van Raalte, and Jones (2005) called for the
evaluation of sport programs to include process, implementation, and outcome evaluation
studies. It should be noted that evaluation research is often conducted, for internal purposes or
otherwise, but is not published in peer-reviewed journals.
Types of Evaluation
Evaluation research is commonly used to evaluate programs that are designed to impact a
social issue, such as homelessness, disease prevention, or underage drinking (Rossi, Lipsey, &
Freeman, 2004). Evaluation is the ―use of social research methods to systematically investigate
the effectiveness of social intervention programs in ways that are adapted to their political and
organizational environments and are designed to inform social action to improve social
conditions‖ (Rossi, Lipsey, & Freeman, p. 16). Scriven (1991) offered an alternative definition
that evaluation is aimed at determining the merit, worth, and value of programs or interventions.
Given the definition and aim of evaluation, it is critical to examine the various types of
evaluations, major theories, and challenges in evaluation research.
Rossi, Lipsey, and Freeman (2004) indicated that every evaluation situation is unique and
there is no one evaluation method that can be used for all evaluations. Each evaluation design
must be tailored to fit the circumstances ―while yielding credible and useful answers to the
questions that motivate [the evaluation]‖ (Rossi, Lipsey, & Freeman, p. 32). Therefore, Rossi
and colleagues recommended that a critical component of evaluation design begins with an

17

understanding of the purpose of the evaluation and what questions the evaluation aims to answer.
After an appropriate set of questions has been determined, which itself can be an arduous
process, evaluation design can begin. Furthermore, questions typically fall into recognizable
categories or types of evaluations.
Considering that many social programs are designed to fill a specific void or alleviate a
social problem (Rossi, Lipsey, & Freeman, 2004) it is important to determine the extent of the
need for a particular program. This form of evaluation is commonly referred to as a needs
assessment. A needs assessment is an evaluation designed to make decisions about program
improvement or the allocation of resources. Thus, priorities are determined on identified needs
(Witkin & Altshuld, 1995). A need is often defined as a gap between what is and what should be
(Witkin & Altshuld, 1995). It is important to conduct a needs assessment because there may be
variation in the perceived needs among people in a given population. Needs assessments ―are
conducted to derive information and perceptions of values as a guide to making policy and
program decisions that will benefit specific groups of people‖ (Witkin & Altshuld, 1995, p. 5).
Needs assessments can be undertaken for a variety of reasons, but typically are intended to lead
to action, change, and improvement.
An example of a needs assessment was conducted by Gould, Carson, Fifer, Lauer, and
Benham (2009) and was designed to identify issues and concerns that may influence educational
and developmental objectives through the high school sport experience. The researchers
conducted focus group interviews with key stakeholders that included parents, coaches, athletic
directors, student athletes, and principals. Their analyses revealed issues that potentially could
have a negative impact on life skills or positive youth development that could normally result
from sport participation, which could justify needs for coaching or parental education.

18

Furthermore, their needs assessment, as recommended in the evaluation literature, examined the
perceived needs of various groups within a given population. This needs assessment allowed the
researchers to identify unique needs specific to each subgroup as well as to triangulate findings
across groups. Based on the identified needs (e.g. sport parent problems) follow up studies and
projects are being pursued (e.g. sport parent education programs are being developed) by the
authors.
Another series of evaluation questions focus on how a program is designed and
conceptualized. Rarely do program stakeholders have a clear plan for the process of how their
program is supposed to work and therefore an assessment of program theory is needed (Rossi,
Lipsey, & Freeman, 2004). One method within this type of evaluation is evaluability
assessments. Evaluability assessment is a process that helps one identify which evaluations
would be useful and feasible, and begin the process of designing evaluations (Wholey, Hatry, &
Newcomer, 2004). Essentially, an evaluability assessment examines the program model by
clearly defining goals and objectives of the program, and if a program is ready to be assessed or
is evaluable. Programs that are not ready to be assessed commonly fit one of the four following
criteria: there is internal disagreement on program goals and performance criteria; goals do not
fit the activity and or resources; performance indicators cannot be collected at a reasonable cost;
or program managers are unwilling or unable to create change as a result of the data collected.
However, a program is evaluable when goals are well defined, goals are plausible, an evaluator
can obtain data at a reasonable cost, and program stakeholders are willing to, and have clearly
outlined how they will, act on the data obtained from the evaluation.
Evaluators conducting evaluability assessments are attempting to understand the program
methods, objectives, delivery process, and the social reality through interviews, observations,

19

and existing data. Evaluability assessments are aimed at getting program stakeholders to clearly
define and articulate the program‘s design and goals. Additionally, some evaluators conducting
an evaluability assessment will, from the data collected, create a logic model that describes
logical connections between activities and outcomes (Wholey, Hatry, & Newcomer, 2004).
Logic models can identify the resources or inputs that lead to program outputs and outcomes.
One example of an evaluability assessment examined three programs of the Juvenile
Justice Commission (JCC) of the state of New Jersey (Finckenauer, Margaryan, & Sullivan,
2005). The JCC was created to centralize reform efforts, which include planning, policy efforts,
and service provision, in the juvenile justice system in New Jersey. The evaluability assessment
was performed to answer questions about each programs readiness for evaluation, whether it was
worthwhile to commit resources to an outcome evaluation, and whether the data collected from a
formal evaluation would be used. The researchers collected data through interviews, site visits,
meetings with staff, and through reviewing the record keeping systems. The researchers
concluded that the evaluability assessment provided useful answers that enabled them to tailor
evaluation designs for two of the programs (Finckenauer, Margaryan, & Sullivan, 2005).
Another common set of evaluation questions examines the impact or outcomes of a
program. Outcome evaluation is an assessment of the degree of change in the outcomes over
time (before, during, after, and following the conclusion of the program). An outcome evaluation
does not necessarily involve a basis of comparison. Usually outcome evaluation is more small
scale, time limited, and within the scope of a well-resourced community organization (Rossi,
Lipsey, & Freeman, 2004). An example of an outcome evaluation was undertaken to examine a
pilot program of the Play It Smart program, which was designed by the National Football
Foundation to develop life skills and meet the needs of football players in underserved

20

communities (Petitpas, van Raalte, Cornelius, & Presbrey, 2004). The evaluation was designed
to examine the goals of the Play It Smart program, which included improving academic
standards (e.g. GPA, SAT or ACT scores, and graduation rates), increased community service
activities, and knowledge and use of health enhancing behaviors. Data were collected from each
operating site and quarterly academic updates were used to track progress of the academic goals
of the Play It Smart program. Results revealed that the participants‘ grade point averages
increased and 83% of the participants went on to higher education. Although the results are
promising they must be interpreted with caution due to the lack of a control group.
An impact evaluation, on the other hand, is inherently comparative (randomized
experiments or field experiments); looks at both program outcomes, but also longer-term impacts
or impacts at other levels of analysis. These are larger scale, resource intensive, done for
programs for which it makes sense to invest this kind of effort. A program is ready for impact
evaluation when there is existence of program theory; a decent amount of process data has
already been collected and or is readily accessible, timelines and expectations have been
reviewed, and resources are available for the evaluation.
Weiss, Price, Bhalla, and Bolter (2006) conducted an impact evaluation that was
designed to evaluate a positive youth development program. The First Tee uses golf as a context
to promote personal life skills such as self-regulation, social responsibility, and effective
decision-making. Using quantitative measures, the researchers compared youth participants in
the First Tee program with youth in other organized activities, such as sports and band, on life
skills transfer, usage, and developmental outcomes. As is recommended in impact evaluation, the
researchers had both an experimental group and a control group. The researchers found

21

significant differences among the First Tee participants and the control group in life skill
transfer, developmental outcomes, and general life skill usage.
One major challenge in outcome or impact assessment is to identify measurable goals, in
order to determine effectiveness of a program. The terminology of goals and objectives ―have
become daunting weights that a program staff feel around their necks, burdening them, slowing
their efforts, and impeding rather than advancing their progress‖ (Patton, 2008, p. 239). Rather
one should focus on what program stakeholders are trying to achieve and the differences that
they will observe within the program participants. The key is to have them identify meaningful
goals rather than precise goals that they are not truly interested in. Specifically, evaluating
leadership development programs can be challenging if the program‘s objectives are not set out
clearly beforehand (De Vries, Florent-Treacy, Ramo, & Korotov, 2008). Therefore, it is
important to identify and define effectiveness which Schweigert (2006), in a review of evaluation
reports, posited that there are three different meanings of effectiveness in regard to the outcomes
or impact of a program: increased understanding of the material covered in a program,
accountability with regard to whether the actions taken and results achieved correspond to the
expectations of the intervention, and causality which examines the cause and effect relationship
between the intervention and the acquired behaviors. Given this perspective of evaluation it is
then important to examine theoretical perspectives on the process of determining effectiveness.
Evaluation Theory
To truly understand evaluation research it is critical to examine a few prevailing theories
in the field. Space limitations prevent a detailed examination of all theories of evaluation;
therefore only a few key theories will be reviewed. Alkin (2004) stated that evaluation theories
typically inform the reader about methodology, value, and use. Value, in regard to evaluation

22

theory, is the manner in which the data will be judged, whereas use is how the data will be used.
Alkin and Christie (2004) presented an evaluation theory tree that had three primary branches.
Each branch represented an orientation of evaluation theory and the branches were methods, use,
and valuing. The ‗methods‘ branch deals with obtaining generalizabiltiy and getting the right
method for the evaluation. Alkin and Christie recognized that while most evaluation theorists
discuss methodology, only a few theorists have been dedicated to examining methodology, and
those theorists represent the methods branch.
Theorists who believe that evaluation should be focused on making value judgments of
an evaluand (what or who an evaluator is evaluating) represent the ‗value‘ branch (Alkin &
Christie, 2004). The most prominent evaluation theorist in this branch is Michael Scriven, who is
often called the ―father of value.‖ Scriven (1991) believed that evaluation is the process of
determining the merit, worth, or value of something, which involves determining the standards of
merit, worth, or value and the performance of the evaluands on these standards. Scriven also
believed that the evaluator should integrate the data to arrive at an overall evaluation, which may
be both judgmental and a result of complex calculation. According to Scriven, merit is the
intrinsic value of evaluands in regard to how well they performed. Worth, on the other hand,
refers to the extent to which the evaluand met the needs of their target audience and whether or
not it was cost-effective. Finally, the value of a program, according to Scriven, is measured by its
importance in terms of the greater social good.
A limitation of Scriven‘s approach is that the evaluator remains external to the program
that is being evaluated and does not work with the evaluand to determine methods and the
general process of the evaluation. It is not the objective of this current study to evaluate the
project from a distance, rather, to work with program stakeholders to evaluate certain aspects of

23

the program and use that data to inform the continued development of the remaining aspects of
the program. Therefore, the current study will be grounded in utilization-focused theory.
The final branch of the evaluation theory tree, referred to as the ‗use‘ branch, focuses on
theorists who feel it is critical to conduct evaluations that are designed to help program
stakeholders make decisions (Alkin & Christie, 2004). Utilization theorists design evaluations
where the evaluator and the program stakeholders are solely focused on using the findings of
an evaluation. Although, there are many prominent theorists that would fall within this
branch, one of particular interest is Michael Patton.
Patton (2008) began his definition of utilization-focused evaluation as an ―evaluation
done for and with specific intended primary users for specific, intended uses‖ (p. 37).
Because the focus of evaluation, in this definition, is use, it is clear that the design and
process of evaluation should be permeated with an intentional focus on use from the
beginning of the evaluation all the way through to the end. Patton (2008) continued that
evaluations should be judged by their usefulness—for the intended users—and whether or
not they are actually used. Patton (2008) concluded, ―the focus of utilization-focused
evaluation is on intended use by intended users‖ (p. 37). Throughout this theory, Patton
(2000) emphasized that use is not some abstract construct; rather use ―concerns how real
people in the real world apply evaluation findings and experience the evaluation process‖ (p.
425). Finally, because utilization-focused evaluation is specifically used to create knowledge
that will be used by the intended users, the theory is quite flexible and adaptable to the
situation with regards to having the users work alongside the evaluator to select the methods,
content, and focus of the evaluation. Considering this definition of utilization -focused

24

evaluation, with regards to the emphasis on utility and use, the role of the evaluator and the
process of the evaluation are critical.
Patton (2008) offered an alternative method to Scriven with regard to determining value
or rendering judgment. Scriven proposed that the evaluator is external to the program being
evaluated and therefore independently renders judgment regarding the value of the program.
Patton, alternatively, stated that the evaluator could offer interpretations, and judgments, if that is
what the evaluation users want. However, Patton stated that even if the evaluators request him to
render judgments he will only do so after providing the stakeholders ―an opportunity to arrive at
their own conclusions unencumbered by [his] perspective‖ (p. 500). Finally, as is typical in
utilization-focused evaluations, Patton recommended that the evaluator should take on the role of
evaluation facilitator and work with the decision makers and intended users during the process of
rendering judgment and making recommendations based off of the data.
Because the program is still being designed, an evaluation should not only examine
outcomes, but also the process and implementation of the program. Patton suggested that
interventions or programs could fail for two likely reasons: (1) the intervention was not
implemented as designed or (2) the theory was not effective. By simply evaluating outcomes one
might miss what specific factors of the program led to, or hindered, those outcomes. Patton
recommended that an implementation evaluation aimed at program improvement would examine
a programs strengths and weaknesses. Furthermore, Patton discussed different types of an
implementation evaluation, two of which are of particular interest to the current evaluation. The
first is process evaluation that ―focuses on the internal dynamics and actual operations of a
program in an attempt to understand its strengths and weaknesses‖ (Patton, 2008, p. 324). A
process evaluation will include questions about how the participants experience and perceive the

25

program and search for explanations of what resulted in successes, failures or changes.
Therefore, a process evaluation aims to examine how an outcome was produced, rather than
simply just examining the outcome.
Another type of an implementation evaluation is a component evaluation. This type of
evaluation examines one or two pieces of a program rather than the entire program. For instance,
the NWCA Leadership Academy has several components including the in-person workshop,
mentoring, and 10-online modules. Rather than examining the impact or implementation of all of
the components the current evaluation will focus only on two of the online modules. Patton
(2008) argued a component evaluation can useful to determine potential improvements to a
component, or if the component merits continuation. For the purposes of this study, by
examining two of the online modules, the stakeholders can identify what aspects of the modules
are most and least impactful and then use that information to improve those modules further and
to inform the development and content of the modules that are yet to be developed. On the other
hand, should the evaluation findings be interpreted to demonstrate that the online modules are
not as useful as the stakeholders had hoped; the stakeholders could then refocus their efforts onto
the other components (in-person workshop and mentorship) of the program.
Therefore, the current evaluation will examine outcomes by measuring knowledge
gained, intended use, and actual use of the information covered in the first two online modules,
but will also evaluate implementation by examining how the coaches interacted with the program
(i.e. how much time they spent in the modules, whether or not they completed the recommended
exercises, etcetera) and what aspects of the modules best fit their needs and were the most
impactful.

26

Using utilization theory with a focus on outcome and implementation evaluation, this
evaluation will examine the implementation and impact of the first two online Modules of the
NWCA Leadership Academy. Considering that some of the objectives in the first two Modules
are to develop leadership abilities within the coaches, a brief review of the literature on
leadership is important.
Leadership Literature
This review of leadership literature is not a detailed review; rather it is intended to
provide enough information for the reader to obtain a working knowledge of leadership as it is
being applied to the online coaching Leadership Academy being evaluated in this study.
Therefore, it is critical to first examine general leadership research, briefly discuss factors
affecting today‘s leaders, and leadership development and evaluation. Sport psychology
leadership research and theory will also be briefly examined.
Conceptualizing and Defining Leadership
Leadership has been extensively studied in a myriad of industries including business (e.g.
Cohn, Khruana, & Reeves, 2005), management (e.g. Griffin, 2003) and sport (e.g. Case, 1987;
Chelladurai & Arnott, 1985; Chelladurai & Saleh, 1980; Mondello & Janelle, 2001). Even
though leadership has been extensively studied, there is not a single universally accepted
definition of leadership (Bennis, 2007) or an accepted characterization of leadership (Mondello
& Janelle, 2001). Despite these differences Northouse (2004) has indicated that four components
seem to be involved in all conceptualizations of leadership. These include: viewing leadership as
a process; recognizing that leadership involves influence; understanding that leadership occurs
within groups; and realizing that leadership is about achieving goals. Based on these components

27

Northouse (2004, p. 3) operationally defined leadership as ―a process whereby an individual
influences a group of individuals to achieve a common goal.‖
In his widely used leadership text, Northouse (2004) goes on to present the major
approaches to studying leadership. Included in these were the trait, skills, style, situational,
contingency, path-goal, leader-member exchange, transformational, team and psychodynamic
approaches. While each leadership approach has pros and cons, it is clear that to truly understand
contemporary leadership, personal aspects, characteristics, and skills of the leader, the situation
and their interaction are all critical to understand.
Coming to the same conclusion, Hackman and Wageman (2007) reframed common
leadership questions that typically have hindered the growth of leadership research. Hackman
and Wageman suggested that researchers examine the interaction of a leaders‘ personal attributes
and the situational properties, rather than just focusing on the traits of a leader. Examining the
interaction between a leaders‘ personal attributes and the situational properties, encompasses the
beliefs that a successful, or unsuccessful for that matter, leader is not always a direct result of his
or her specific behaviors. Therefore, one must also examine the conditions a leader is faced with
as well as his or her personal characteristics and attributes. In regard to this study, intercollegiate
coaches are being asked to lead in unfavorable economic conditions and are competing with
sports, such as football and basketball that are virtually recession proof. Furthermore, because of
the economic stress placed on many universities support staff cuts are being made, which result
in the coach having to do much more than just coach. Therefore, college coaches of today must
have skills in public relations, marketing, recruiting, fund raising, and alumni relations. They
must also adopt a broader view of the goals of college wrestling leadership to include success on

28

the mat as well as off whether that be in the academic success of their athletes, community
service, public relations and fund raising.
The NWCA Leadership Academy was designed by combining research in business and
leadership to provide education to collegiate coaches in an effort to develop their leadership
abilities beyond technical and tactical instruction. In a special issue of the American Psychologist
devoted to leadership, Bennis (2007) professed that leaders do not exist in a vacuum and one
must not overlook the situational impact on leadership. Vroom and Jago (2007, p. 23) define
leadership as ―a process of motivating others to work together collaboratively to accomplish
great things.‖ Vroom and Jago continued that organizational or leadership effectiveness is
affected by situational factors that are not always within one‘s control, the environment affects
how leaders behave, and situations influence the consequences of the leader‘s behavior.
Therefore, views about leadership that are strictly trait-centered or purely situational do not
encompass the phenomenon that is leadership. Furthermore, leadership development must adapt
to this new age of leadership and begin to ask the right questions about leadership, which,
ideally, will lead to more effective leadership development.
Leadership Development and Evaluation
Leaders are developed both informally and formally. Informally, individuals will engage
in personal leadership development by reading the latest book on the topic in an effort to pick up
some tips on how they can become a better leader. In a formal manner, leaders, or potential
leaders, will engage in leadership development training programs or workshops, feedback
programs, or will hire an executive coach. Riggio (2008) argued that the reason why there is a
significant amount of published material on leadership and so much money is spent on leadership
development is because people believe that leaders can have a big impact on an organization and

29

that the development of leadership is an abstract and complex phenomenon. Essentially,
organizations and individuals ―believe that leadership development is important and worth the
investment of resources and their personal time‖ (Riggio, p. 384).
Before discussing program design, it is first important to examine perspectives on
whether leaders are born or made. Avolio and Hannah (2008) cited behavioral geneticists and
research with twins to make the argument that there is significant evidence that one‘s ability for
leadership is more a result of their environment than a result of heredity. Rosenbach and Taylor
(2001) stated that although there is general agreement that leadership could be developed; some
individuals find it easier to grasp and successfully demonstrate the qualities of leadership.
Riggio (2008) recommended that models of employee training and development could be
applied to leadership development. Riggio suggested that the model begins with a needs
assessment that is driven by the organization‘s mission. In regard to the current study, the
NWCA has extensively researched the current state of wrestling and the requirements of the
coach in order to recognize the needs of the college coach of today. From this point, Riggio
suggested that organizations take into account their leaders‘ readiness, desire to learn, and
capacity to become a better leader. Following this step an organization can develop leadership
training programs that are specified to the needs and capacities of the leaders in training. The
next step, evaluation, is an often overlooked process that is either limited in scope or not
undertaken at all (Riggio, 2008).
Although evaluation research was previously reviewed it is important to examine aspects
of leadership development evaluation that are unique. Riggio (2008) stated that evaluations of
leadership development are commonly focused on perceptions of success, measures of retention,
behavioral criteria (are the lessons being put into action), and finally results criteria, which

30

examines whether or not there was a direct link between increased revenue or better performance
and the leadership development course. De Vries, Florent-Treacy, Ramo, and Korotov (2008)
recommended that leadership development programs should be constructed on a framework that
lends itself to an integrated evaluation. The authors recommended that the framework include:
clear outcome goals of the intervention, theoretical basis that matches the organizational context,
specific leadership dimensions that the program is targeted toward, and appropriate teaching
tools that fit the focus of the program. One benefit of this design is that an evaluator can evaluate
the program within each aspect of the recommended framework.
More specifically to coaching education programs, Gilbert and Trudel (1999)
recommended the strategy of measuring if the course was delivered as it was designed, if the
coach acquired new knowledge, and if the coach actually applied the knowledge exhibited by his
or her coaching behaviors. The authors collected data through participant observation, semistructured interviews, stimulated recall interviews, and systematic observation. They concluded
that although the evaluation strategy was time consuming, it was successful. However, one
limitation was that the study was not designed to evaluate the program; rather it was designed to
test the evaluation strategy. Therefore, the authors only studied one coach involved in the coach
education program.
In summary, leadership is a complex phenomenon, so much so that a single, all
encompassing definition of leadership has not yet been discovered (Bennis, 2007; Mondello &
Janelle, 2001); leadership development is also a complex topic. The college coach of today is
required to demonstrate leadership qualities of both an instructional nature directed toward their
athletes, as well as leadership skills unrelated to coaching technique (Barber & Eckrich, 1998;
MacLean & Zakrajsek, 1996; Mallett & Côté, 2006). Therefore, coaching education programs

31

must be designed to develop assets that are required of a coach. Furthermore, coaching education
programs must be evaluated to assure that they are meeting the needs of coaches as well as
having an impact on their leadership abilities. For example, the NWCA Coaching Leadership
Academy was based on the assumption that today‘s wrestling coach is a leader that has the goal
of influencing those he or she leads to achieve key team and organizational goals. Historically,
the goal was simply to enhance wrestler and team performance on the mat. Today, however,
leadership goals must be viewed more broadly to include the academic performance of the
student athlete, fund raising, community relations and enhanced athlete health and safety. Thus,
the goals of today‘s coach as a leader are multifaceted. For this reason the NWCA program is
designed to enhance the coach‘s knowledge and awareness in each of these areas.
Regardless of the goals a leader pursues, modern leadership involves helping one‘s
organization define their goals and then facilitate members‘ efforts for goal achievement. Given
this fact, the NWCA focuses on leader influence skills like creating a compelling program vision,
setting goals, building trust and credibility, effectively communicating and providing feedback.
Leadership in Sport
Thus far, the general leadership research has been summarized because the NWCA
Leadership Academy is aimed at developing coaches as general leaders, much like a corporate
Chief Executive Officer (M. Moyer, personal communication, December 1, 2008). However, this
does not mean that sport psychology researchers have not studied leadership. Leadership has
been extensively studied (see Weinberg & Gould, 2007 for a general overview) and parallels the
general leadership research. The dominant model guiding sport psychology research is
Chelladurai‘s (1990) multidimensional model of leadership. This model contends that effective
leadership is influenced by the situation, leader characteristics and member characteristics.

32

Player performance and satisfaction are also a function of the leaders actual behavior, the
behavior required by the situation, and preferred behaviors. Maximum motivation, performance
and satisfaction are achieved when there is congruence between desired, actual and preferred
leader behaviors.
In an attempt to measure leadership behaviors, preference for specific behaviors, athletes‘
perceptions of their coaches‘ behaviors, and coaches‘ perceptions of their own behaviors,
Chelladurai and Saleh (1980) developed the Leadership Scale for Sport (LSS). The LSS has five
dimensions that include training and instruction (coaching behavior that is aimed at improving an
athlete‘s performance), democratic behavior (coaching behavior that allows greater athlete
participation in decision making), autocratic behavior (coaching behavior that stresses individual
authority and decision making), social support (motivational behavior that demonstrates concern
for the well-being of an athlete), and positive feedback (motivational behavior that reinforces an
athlete by recognizing and rewarding good performance) (Chelladurai & Saleh, 1980). The LSS
has been used to examine and identify athlete preferences and perceptions of coaching behaviors
in a variety of contexts (e.g. Chelladurai & Arnott, 1985; Dwyer & Fischer, 1988; and Terry &
Howe, 1984).
In one study using the LSS, researchers examined wrestlers‘ perceptions of their coaches‘
leadership as predictors of satisfaction with the leadership. Dwyer and Fischer (1990)
administered the LSS, as well as a satisfaction with leadership survey, to 152 wrestlers aged 15
and older. They found that wrestlers who perceived the behavior of their coaches to be high on
positive feedback, high on training and instruction, high on democratic behavior, and low in
autocratic behavior exhibited the highest satisfaction with coaches‘ leadership behavior.
However, one limitation of this study, as it pertains to the current study, is the sample included

33

both high school and college aged wrestlers and only examined the relationship between
perceived leadership and satisfaction.
Based on this research it is emphasized that coaches must understand their own
leadership style, recognize how the situation and followers influence the leadership context and
adjust their style to the needs of the situation. In consideration of the college coach of today, it is
critical to examine the coach as a leader beyond measures that only include his or her perceived
behavior as a coach leader. Coaching athletes, and the behaviors that are exhibited during this
interaction, is only one duty among many required of today‘s college coach.
Interestingly, the most recent work of Chelladurai (2007) has modified the
multidimensional model of leadership to include the effects of transformational leadership
examining its effects of situational and member characteristics. In pursing excellence he suggests
that leaders must create a vision, engage in inspirational communication, engage in intellectual
stimulation, provide individualized supportive behaviors, instill task and ego-oriented climates
and provide various types of training behaviors. Hence, the leadership research in sport is
expanding to a broader view of leadership effectiveness beyond optimizing coach and athlete
relations. This approach is consistent with the training program adopted by the NWCA.

34

CHAPTER 3
METHOD
The present study was conducted in three separate phases to correspond with each of the
purposes. The overarching purpose of Phase One was to clarify what specific criteria the NWCA
Coaching Leadership Academy should be evaluated on. The participants in Phase One, key
stakeholders in the NWCA Coaching Leadership Academy, were interviewed for their
perspectives on the program‘s goals and objectives. Phase two of the study then focused on
evaluating the effectiveness of two of the program‘s online modules. Finally, Phase Three
assessed the intervention group coaches‘ opinions regarding the ease and utility of the online
video components (i.e. coach interviews, PowerPoint with voiceover, etcetera).
Phase One: Qualitative Interviews of Key Stakeholders
Participants. To determine which specific criteria Modules 1 and 2 should be evaluated
on, qualitative interviews were conducted with key stakeholders. The sample in Phase One
included seven participants. Considering the objectives of Phase One (to identify objectives of
the first two online Modules) it was important to collect data from varying levels of stakeholders.
The sample in Phase One was comprised of two subgroups. The first subgroup included five
participants. Three participants were influential in the development of the program and the other
two participants were experienced coaches who were acting as mentors in the mentorship piece
of the Leadership Academy. The participants in the first subgroup included: the executive
director of the NWCA; the director of the ISYS who been the lead developer and director of the
content and its delivery; and the associate athletic director at a university in the Midwest who is
in charge of the mentorship portion of the leadership academy. Additionally, the first subgroup
included two mentors who went through the program as experienced coaches and could

35

objectively judge the content. The second subgroup included two participating coaches who have
already completed the first two modules. The two participating coaches were involved in the
inaugural class of the Leadership Academy, have completed the first two online modules and
offered a unique perspective that added to the richness of the data collected in Phase One. Each
group offers a unique perspective to the Leadership Academy and, therefore, can effectively
identify key components to evaluate in Modules 1 and 2.
Procedures. Permission to conduct this study was obtained from the Human Research
Protection Program at MSU. Based on Patton‘s (2008) recommendation to work closely with key
stakeholders to identify meaningful goals, Phase One consisted of qualitative interviews with key
stakeholders involved in the NWCA Leadership Academy.
In Phase One the researcher first contacted the director of the NWCA to identify
participants to include in this phase. The sample in Phase One was a purposeful sample that was
selected for a number of factors including convenience, criterion (they were all involved in the
NWCA Leadership Academy), and opportunistic (Patton, 1990). In addition to the lead program
architects (NWCA executive director, ISYS director, NWCA mentor program coordinator)
additional members of the sample were chosen based on which mentors and coaches would be
willing and available to participate and had completed the first two modules. Additionally, the
director of the NWCA was asked to recommend participants who would offer honest and open
feedback. The director of the NWCA then sent out an email to a number of potential participants
describing the purpose of the evaluation and requesting that interested participants respond to the
director of the NWCA. The researcher then emailed interested participants, informed them about
the evaluation study, attached the consent form, and requested formal consent. Additionally, a
link to complete the demographic survey (See Appendix A) was included in the email. The

36

researcher then scheduled appointments with the participants to conduct the interview. Prior to
the interview the investigator requested that the participants reviewed Modules 1 and 2.
Semi-structured interviews were conducted over the phone (six participants) or in-person
(one participant) and lasted between 20 to 60 minutes. All interviews were digitally recorded and
transcribed verbatim.
Measurements
Demographics. A demographic questionnaire was used to obtain background information
from Phase One participants. The demographic survey had items that included age, NCAA
division (if applicable), number of years as a coach (or coach educator), current position, and
role in the Leadership Academy (See Appendix A).
Interview Guide. A semi-structured interview guide was used to conduct the interviews
in Phase One (See Appendix B). The semi-structured nature of the interview guide ensured that
the participants were asked a set of core questions to cover the most critical topics. Additionally,
the interviewer asked probe questions to gather more detailed responses thus increasing the
likelihood of rich data. The interview guide focused on obtaining information regarding the
objectives and outcomes of the first two online Modules, as perceived by the stakeholders. Some
of the questions included: In general what do you perceive are the main objectives of Module 1;
what skills (knowledge) should the coach have following Module 2; if Module 1 is successful,
how will the coaches be different than they were before completing Module 1; which aspects of
the online Modules do you think will be the most successful in achieving the goals of Modules 1
and 2; which aspects of the online Modules do you think will be the least successful in achieving
the goals of Modules 1 and 2?
Phase Two: Evaluation of Module’s 1 and 2 Intervention

37

Modules 1 and 2. Module 1 of the NWCA Coaching Leadership Academy is titled: ―A
Call to Arms‖ and has five main purposes: (1) increase the coach‘s awareness of threats to the
sport of collegiate wrestling; (2) create a sense of urgency for the coach to take responsibility
for saving wrestling by making his or her own program all it is capable of being; (3) help the
coach formulate a plan for taking his or her program to the next level; (4) motivate the coach
to improve his or her leadership skills and act to improve his or her program; and (5) have a
coach develop a vision for his or her program. Module 2, entitled ―The Coach as a Leader,‖
has four main purposes: (1) help the coach understand the importance of effective leadership;
(2) increase the coach‘s understanding of components of effective and ineffective leadership;
(3) help the coach understand his or her leadership style; and (4) help the coach develop
effective leader communication skills.
Design. The original design of the study called for an evaluation of Modules 1 and 2 of
the program using a 2 X 2 (Group x Pre/Post) design. Subscales included the knowledge and
intended use items. However, as will be explained later in data analysis section, the inability to
have control participants complete surveys on the post-test (Time 2) called for an alteration in
the original design.
Participants. The 48 participants in Phase Two included male (n = 47) and female (n =
1) coaches who agreed to take part in the NWCA Coaching Leadership Academy. Of the 48
coaches 45 were head and three were assistant college wrestling coaches. Their age ranged from
26 to 61 years (M = 36.10, SD = 7.82). The participants reported coaching at the NCAA Division
I level (n = 13), Division II (n = 11), Division III (n = 10), NAIA (n = 12), at the community
college level (n = 1) and in a newly created women‘s division (n = 1). The coaches reported
having from 1 to 26 years of coaching experience (M = 8.63, SD = 5.30) and the majority of the

38

participants have earned a masters degree (54.2%), while 21 of them have earned a bachelors
degree (43.8%). Furthermore, of the 29 coaches (60.4%) who have had previous experience
taking online courses, 24 of them found online courses to be either moderately (n = 14) or very
effective (n = 10).
Procedures. The participants were identified, rather then randomly selected, because an
objective of the director of the NWCA is to launch this program with the coaches that he
believes have an urgent need to develop their leadership skills, are in highly visible critical
coaching positions that he believed are essential to the success of collegiate wrestling, or both.
The director of the NWCA outlined a number of warning signs in identifying a coach in urgent
need of the program; these warning signs include: (1) a part-time coach; (2) a program that is
partially funded; (3) a wrestling team that is not part of the multi-sport conference that the
institution participates in; (4) a wrestling team that has a consistent losing record; (5) a school
that has a football program and is egregiously out of compliance with the proportionality
requirement of Title IX; (6) a program with chronic off the field incidences; (7) a program that
lacks alumni involvement; (8) a team without sufficient spectator support; and (9) a program that
is part of a school that has recently released a press release announcing budget problems. Using
these criterion, the director of the NWCA identified a group of 31 coaches as participants in the
Leadership Academy. After identifying the coaches, the NWCA emailed the coaches inviting
them to participate in the Leadership Academy. In an effort to increase compliance for
participation in the NWCA Leadership Academy, the NWCA informed participants that they
would provide a full stipend to the participants who complete the Modules that will allow them
to attend the in-person workshop portion of the Leadership Academy in 2010 at no cost. In
addition to requesting their participation in the Leadership Academy, their voluntary

39

participation in the current study was also requested. After this initial group of 31 coaches was
contacted another four were later added to the intervention group. Additionally, the director of
the NWCA also contacted another 31 coaches, to create a control group, that were matched by
division and experience to the intervention group based on his knowledge of them and their
programs. Thirty-five coaches agreed to participate in the intervention group, with 23 completing
all of the necessary requirements. The control group began with 13 participants, with only four
coaches finishing all of the necessary requirements, which necessitated a change in the original
design.
Intervention group. After the NWCA director emailed the potential Leadership Academy
coaches, the researcher, to formally request their participation in the current study and to obtain
consent, then emailed the coaches. Included in the initial contact email, were instructions on
how to gain access to the demographic survey (See Appendix C for Survey 1) and Survey 2 (See
Appendix D for Survey 2). All surveys were completed online. Following completion of the
demographic survey and Survey 2, the intervention group participants were then provided access
to the online modules. In an effort to avoid an unequal dosage of the intervention (i.e. some
participants completing all 5 modules then completing Survey 3, while others only completing 2
modules and then completing Survey 3), the intervention group participants did not complete
Survey 3 (See Appendix E for Survey 3) until completing all five online modules. Additionally,
the intervention group completed a follow-up survey (See Appendix F for Survey 4), to obtain
more qualitative information used in Phase Three of the study. Finally, in an effort to test the use
and retention of key program content, the coaches in the intervention group completed a
retention survey one month after completing the modules (See Appendix G for Survey 5).
Following completion of the final survey, participants in the intervention group were then

40

emailed a thank you letter as well as instructions on how to obtain the stipend for the in-person
workshop.
Control Group. The control or comparison group coaches were first emailed by the
researcher and the director of the NWCA requesting their participation and formal consent as
control group coaches (Appendix H). After obtaining consent, the control group coaches were
then provided instructions on how to gain access to the demographic survey and Survey 2.
Following completion of the demographic survey and Survey 2, the control group coaches were
then provided an interim period of one month, to match the time the intervention coaches were
given to complete the Modules, before completing Survey 3. Following completion of Survey 3
the remaining control group coaches were provided the opportunity to complete the modules,
however, none of the coaches accepted the offer to do the modules on their own.
Measurements
Demographics. The demographic survey for participants in Phase Two has items that
include age, NCAA Division, number of years coaching the team, level of coaching (head or
assistant), total number of years coaching wrestling as a head coach, total number of years
coaching wrestling as an assistant coach, number of years playing experience (See Appendix C).
Surveys. In addition to the demographic survey (Survey 1), four surveys have been
developed to reflect the theoretical standpoint of Patton (2000; 2008) and the recommendations
put forth by Schweigert (2006), therefore the surveys are designed to determine the value and
effectiveness of the first two Modules of the Leadership Academy. Effectiveness was further
defined by the data collected in Phase One, thus the surveys were designed to examine
effectiveness by determining the knowledge, use, and intended use of key principles covered in
the first two Modules.

41

Survey‘s 2 and 3 (See Appendices D and E) contained items asking the coaches to rate
their degree of knowledge and intended use of the topics presented in Modules 1 and 2 of the
online portion of the Leadership Academy. The final survey, Survey 5 (See Appendix G)
measured the retention and actual use of the material presented in Modules 1 and 2 of the
Leadership Academy. (Note: The highlighted questions in the survey‘s represent new questions
and ‗**‘ next to the questions represents eliminated questions from one survey to the next).
Phase Three: Coaches’ Opinions Regarding Ease and Utility of the Online Video Format
Participants. The 28 participants in Phase Three included coaches who were involved in
Phase Two of the current research study. Their ages ranged from 26 to 50 years old (M = 34.32,
SD = 6.20). The participants reported coaching at the NCAA Division I level (n = 6), Division II
(n = 6), Division III (n = 7), NAIA (n = 8), and at the community college level (n = 1). Of the 20
coaches in Phase Three who have had experience taking online educational courses, 17 (85%)
found the method of online education to be moderately (n = 8) to very (n = 9) effective.
Procedures. The procedures for obtaining participants are identical to the procedures
outlined in Phase Two. Following completion of the first two Modules, the participants involved
in the intervention group were asked to complete a follow-up survey (See Appendix F).
Survey. The survey used in Phase Three (Appendix F) contained items asking the
participants to rate the most and least useful aspects of Modules 1 and 2. Additionally,
participants were asked to evaluate the usefulness of Modules 1 and 2. Figure 1 is a flow chart of
the participant experience for Phases Two and Three.

42

Figure 1: Flow Chart of Participant Experience Phases Two and Three

Identify Potential Participants
n = 40 – 60
Obtain Consent

All Participants Complete
Demographic Survey (Survey 1)
n = 40 – 60
Use Demographic Information to
Split into two Matched groups

Intervention Group
n = 20-30

Comparison Group
N=20-30

Both Groups Complete Survey 2

Intervention
Group

Complete
Modules 1 & 2

1 month to
complete

Comparison
Group

No intervention

Both Groups Complete Survey 3
*Intervention group completes
follow-up survey (Survey 4, Phase Three)

Intervention
Group

1 month

No Active
Involvement
Intervention Group

Complete Survey 5
(Retention and Use)

43

Data Analysis
Phase One
To analyze the interview data, audio recordings of the interviews were transcribed
verbatim. The primary investigator then read and re-read each transcript in order to become
familiar with the data. Using Miles and Huberman (1994) as a guide, the researcher identified
individual meaning units (codes) that emerged directly from the participants‘ words. After
coding all interviews the researcher then searched for similarities across the interviews, isolated
commonalities and differences, then gradually elaborated a small set of generalizations that
covered the consistencies from the data. Then the primary investigator identified goals and
objectives that the participants believed should be obtained through the first two Modules to help
formulate the quantitative measures that were developed based on the literature and will be used
in Phases 2 and 3.
Identifying the number of participants who cited particular themes can provide additional
meaning than just identifying themes. Therefore, the numbers in parentheses next to the themes
denote the quantity of participants citing the theme (first number) and the quantity of respondents
interviewed (second number).
Data was analyzed and organized by subgroups. These included: (1) program developers
and mentors; and (2) coaches who participated in the first iteration of the Leadership Academy
and who have previously completed the first two modules. Regarding the second subgroup
(young coaches), it was clear to the researcher and the member checker that one of the coaches
did not review the online modules prior to the interview, which was evidenced by the
interviewee‘s inability to answer questions specifically referring to the online modules. Rather,
the respondent provided general sweeping opinions about skills he believes all coaches should

44

have, most of which were not covered in the module that he was being asked about. When
probed for specifics about a module he would often answer that he did not know or could not
remember. Therefore, his interview has been removed from any further analysis. In an effort to
give the other coach a voice in this research and to examine common themes, his responses that
echo the themes from the first group (developers and mentors) will be discussed. Additionally,
unique responses and responses from questions that are different from the first groups‘ interview
guide will also be discussed.
Phase Two
Between-Group Analyses. The original study design, and the between-group
hypotheses, was intended to evaluate the effectiveness of the first two online modules by
employing a 2 X 2 (Group x Pre-post) design. However, the size of the control group at Time 2
(n = 4) made it impossible to warrant an analysis of this nature (see the number of participants in
each group at each time in Appendix I). This certainly weakened the present study and was
extremely disappointing to the investigator. However, it seemed unavoidable as many efforts
were repeatedly made to obtain and retain control group coaches in the study. To start, the
director of the NWCA made initial contact with potential control group coaches, informing them
of the study, and urging them to participate. Then, the researcher sent a follow-up email one
week after the first email was sent. Coaches who had not responded were then emailed up to five
more times over the course of three weeks. This process yielded 13 participating control group
coaches at Time 1, a fact that was very surprising to the director of the NWCA who during the
design phase of the study felt that there would not be problems getting control coaches to
participate. After completing the first survey, the control group coaches were then emailed
thanking them for their participation and reminded that in one month they would be contacted

45

again to complete the post-test (Time 2). The 13 coaches were emailed exactly one month after
they completed the pre-test. They were then emailed every week after that for 3 weeks. This
process yielded four control group coaches at Time 2; a second major disappointment for the
investigator.
While the original design of the study was impossible to implement without enough
control or comparison participants, this did not mean that some meaningful comparisons could
not be made. First, the intervention and control groups were compared at Time 1. An
independent samples t-test for each survey item was conducted. Conducting multiple t-test
analyses will greatly increase the chances of making a Type I error without an adjustment to the
p-value. However, if one were to protect to highly against a Type 1 error they would then
increase the chances of a Type II error. Therefore, the p-value was set to 0.01.
A MANOVA was conducted comparing the intervention and control groups with the
dependent variables formed as subscales of Knowledge and Intention at Time 1. The purpose of
a MANOVA is to examine if there are differences amongst the control and intervention group at
Time 1, so the researcher can control for those differences in further analyses. In this study, the
researcher wanted to examine differences beyond the item level and therefore conducted a
MANOVA with the subscales of Knowledge and Intention by group. To create the subscales, the
researcher first conducted a conceptual analysis of the items to identify which ones best
represented knowledge and then intention. Then, the researcher conducted a reliability analysis
of the 42 items (see Appendix J for the items that make up each subscale) that made up the
Knowledge subscale and the Chronbach Alpha at Time 1 was 0.852. For the Intention subscale at
Time 1, the Chrobach Alpha of the 14 items was 0.911. The Chronbach Alphas for Knowledge
and Intention at Time 2 were 0.905 and 0.887, respectively. For any analyses with Time 3 data,

46

the Knowledge subscale was reduced to 15 items and the Intention subscale to 13 items (certain
items that were asked at Time 2 were not asked again at Time 3). For ease, the smaller subscales
will be hereto referred to as KnowledgeSmall and IntentionSmall. The KnowledgeSmall subscale
had Chronbach Alphas at Time 1, 2, and 3, of 0.869, 0.902, and 0.905, respectively. The
IntentionSmall subscale had Chronbach Alphas at Time 1, 2, and 3, of 0.911, 0.886, and 0.930,
respectively.
Within-Group Analyses
Time 1 to Time 2. Despite the fact that there were not enough participants to compare
intervention and control coaches at Times 1 and 2, it was still possible to examine the
effectiveness of Modules 1 and 2 (Hypothesis 3). A paired samples t-test was conducted for each
item to explore differences within the intervention group from Time 1 to Time 2.
Time 2 to Time 3. To further explore the impact of Modules 1 and 2, the intervention
coaches were tested again one month after completing the Modules. A paired samples t-test was
conducted at the item level to examine retention from Time 2 to Time 3. Further, a paired
samples t-test was conducted with the subscales of KnowledgeSmall and IntentionSmall from
Time 2 to Time 3.
Phase Three
Phase three data were analyzed by examining the intervention groups‘ mean scores on
questions pertaining to their opinions of the online modules. Mean scores and open-ended
responses were analyzed conceptually.

47

CHAPTER 4
RESULTS
The present study had three primary purposes. The first was to clearly identify the
outcome and implementation objectives of the first two online modules as defined by
program stakeholders. The second was to examine the impact of the first two online modules
on the participating coaches with regard to the objectives identified by the stakeholders. In
particular, impact was defined by examining changes in coaches‘ self-perceived knowledge,
use, and intended use of the principles in the first two Modules. Finally, the third purpose
focused on an evaluation of user preferences and effectiveness regarding the delivery of the
first two online modules, the components of the program that were the focus of this
investigation. Results that follow are organized and presented around each purpose or phase
of the study.
Phase One
The purpose of Phase One was to identify the outcome and implementation objectives
of the first two online modules as defined by program stakeholders. This was accomplished
through interviews with key stakeholders, which focused on which information the coaches
participating in the Leadership Academy should obtain and be evaluated on as a result of
completing the first two online modules.
Mentors and Developers
General Goals, Changes, and Influences. When asked to identify the general goals of
the entire Leadership Academy, changes they would like to see within the participating coaches,
and how they would like the academy to influence the coach‘s behavior, four main themes
emerged from the analysis (see Table 1). First, the mentors and developers stated that they hoped

48

that the coaches would develop a broader perspective and awareness of their role as a coach
(4/5). Essentially, the mentors and developers hoped that after completing the module the
coaches would not only be concerned about themselves and their program, but other programs
and the sport of college wrestling as a whole. One participant stated, ―Coaches are taking a
broader perspective than just their program and how do they save college wrestling‖ (Developer
1).
Table 1
Objectives of Stakeholders in Phase One
General Dimension

Theme
Develop a broader awareness of role of as a head
coach

Goals as a result of entire
Leadership Academy

Approaching coaching role similar to that of a
CEO
Reflect on strengths and weaknesses
Increase personal responsibility at University and
amongst the sport as a whole

Purpose of Online Portion
of the Leadership Academy

General Goals of Module 1

Skills Resulting From
Module 1
General Goals of Module 2
Other

Convenience and Accessibility
Cost-effective
Increased awareness of threats to college wrestling
and solutions
Increased awareness of what it takes to run a
quality program
Ability to create a vision and engage in long-term
planning
Improve communication skills
Use the GROW model
Hire (or bring in people) to supplement
weaknesses

Two of these four themes emerged during this line of questioning as well as during
questions specific to Module 1. First, the mentors and developers indicated they would like the

49

coaches to be willing to do more than just coach and approach their job similar to a chief
executive officer (CEO) (4/5). One participant noted, ―I think the change is understanding that if
you want to be a CEO if you‘re going to be a CEO you can‘t do that by just being an X‘s and O‘s
type of guy‖ (Mentor 1). The other theme that emerged from this line of questioning as well as
during questions specific to Module 1, was the goal that the coaches will reflect on their
strengths and weaknesses (4/5). One mentor stated:
―…also that there is nothing wrong with identifying your strengths and your weaknesses
and especially your weaknesses and then working on those weaknesses. It doesn‘t make
you a weak person to have weaknesses, it makes you a strong person to identify your
weaknesses and then work on those‖ (Mentor 2)
Finally, three participants from this subgroup indicated that they would like to see an
increase in personal responsibility (3/5) among the coaches. One participant highlighted this
point by stating ―I hope that [the coaches] will consider a strategy that was discussed, and try to
realize how they can take personal responsibility in effectuating change‖ (Developer, 3).
Purpose of Online Portion of the Leadership Academy. Three of the five participants
in this group stated that the purpose of the online academy was for convenience and accessibility.
While two of the five participants stated that the purpose of having the online portion of the
academy was because it is cost-effective in delivering a large amount of information to a large
group of individuals spread across the country.
General Goals of Module 1. The mentors and developers were asked to identify what
they perceive are the general goals of Module 1. The two themes that emerged are the goal of
having the coaches increase their awareness of threats to college wrestling and solutions to the
threats (4/5) and recognize what it takes to run a quality program (2/5). Regarding threats and

50

solutions, one developer stated, ―Yeah so sort of these pros and cons, threats, and then we do a
little bit to try to get them to start thinking about solutions‖ (Developer 1).
Skills Resulting from Completing Module 1. One theme to emerge from this line of
questioning focused on having the coaches develop the ability to create a vision and engage in
long-term planning (2/5). A developer described this by stating:
―If you look at Module 1, you have all of these mentors and you have these other
administrators talking about how it's important to have long-term strategies for growth
and for success. So, I guess it's, you know it's identification of the need for the
reinforcement of it, but that's kind of an intro to know these are all the elements of
building that long-term strategy‖ (Developer 3).
General Goals of Module 2. When asked what are the general goals of Module 2, the
developers and mentors identified two main themes. The first was for the coaches to improve
their communication skills and use the GROW model (4/5). The GROW model is a
communication tool to help leaders shift away from always telling others what to do to asking
questions that engage subordinates to help derive and problem solve their own answers to
challenges and tasks confronting them. In the GROW model each letter of the acronym
represents a questioning style. The letter ‗G‘ represents questions pertaining to goals (i.e. what
are your goals for this season?). The letter ‗R‘ represents questions pertaining to reality (i.e. what
have you done so far to achieve those goals?). The letter ‗O‘ represents questions pertaining to
options (i.e. what are some ideas of how you could work on your footwork?). Finally, the letter
‗W‘ represents questions pertaining to will (i.e. what are you going to do these next two weeks at
practice to get closer to your goals?). One developer highlighted the importance of
communication skills by stating:

51

―Well I think a lot of times, you know coaches are always in their stance, their always
looking for the five point move, and when you're communicating with athletes, or you're
communicating with your administration, or you're communicating with parents, you
need to listen. And those communication skills are just so incredibly important‖
(Developer 2).
Skills and Behavioral Changes Resulting from Module 2. The mentors identified a
broad objective that as a result of Module 2, they would like the coaches to add tools to their
toolbox, but stopped short of identifying which tools specifically (2/5).
Other. One other important theme emerged regarding the goal that the coaches will hire
(or bring in people) to supplement their weaknesses (4/5). One developer aptly highlighted this
point with the following analogy:
―[Another objective would be] helping the coach to understand he's like the director of an
orchestra, he doesn't have to know how to play every instrument, he only needs to know
how many instruments he needs and he needs to find people to play them. And in this
case, here the coach needs to be able to surround himself with paid and unpaid volunteers
to help assist [him] with all the administrative duties that are far more than what any one
person could ever keep his arms around.‖ (Developer 2)
Additionally, 3 out of 5 believed the expert interview excerpts which appear in the modules
would be the most successful in achieving the goals of Modules 1 and 2. Four of the mentors
and developers (two each) believed that the exercises and the PowerPoint‘s would be the least
successful components in achieving the goals of Modules 1 and 2. Two of the five commented
that the greatest hindrance to the success of the online portion would be the length of the
modules, while another two mentors and developers cited poor time management skills on the

52

part of the coaches as the greatest hindrance to the success of the online portion. One developer
mentioned that while a goal was to keep the modules relatively short (to encourage completion),
it was important that they were not too short or superficial that the modules would not really lead
to behavioral change. This point was emphasized with regard to the NWCA director‘s charge to
save college wrestling. One developer emphasized that the duration would be the greatest
hindrance stating. ―Don't do hour long sessions. It's too classroom, and it's gonna turn the
coaches off‖ (Developer 3).
Parallel insights from the Coach
The coach from subgroup two shared many similar insights to the mentors and
developers regarding the goals and objectives of the first two modules. Specifically, the coach
believed that the goals from Modules 1 and 2 are for the coach to reflect on his or her strengths
and weaknesses; increase awareness of how his or her character impacts others as well as the
sport in general, thereby increasing their focusing beyond their own program; become a CEO
coach, and finally learning the GROW model and improving communication skills were
important.
One comment from the coach that highlights a common theme found across both groups
demonstrates the importance of a coach broadening his focus beyond his own program:
―I think the way they‘re going to act differently is to support wrestling, you know to uh,
to think about it as an entity rather than to just focus on their own program, I think that
everyone loves the sport, and um you know, kind of less selfish in a way, that you need to
support other programs, one of the biggest things that I got out of the, um, Module 1, is
that wrestlers really need to support wrestling, and to um get back and to help, and I
know there are a lot of programs out there that are real successful but it doesn‘t matter if

53

you don‘t have any opponents to go against…if some of the smaller schools aren‘t
prospering and the sport itself isn‘t getting the attention that it needs‖ (Coach 1).
Finally, the coach also echoed the opinion that the most successful components of the
online modules would be the expert interviews, and stated that the exercises would be the least
successful, unless they were actually used. Essentially, the coach believes that the exercises
would be really powerful because ―[Doing the exercises] really forces you [the coach] to think
about things‖ but he believes that most coaches would decide to skip the exercises. When asked
whether or not he completed the exercises the coach responded, ―Yup, I have it right here with
me…I went through it last night, I was up pretty late doing it but I got it done.‖
Other. The coach was also asked a series of questions regarding his likes and dislikes of
the online modules, as well as use and intended use of the information covered in the first two
modules. The coach liked the video format in that it was easy to follow along, as well as learning
from and seeing experienced, successful coaches. The coach did not like the length, but added ―it
was um late at night when I watched it, so um, I was tired but I don‘t know maybe if I would
have watched it in the afternoon it wouldn‘t have seemed as long.‖ The concern regarding length
and time management was also expressed by four of the five other Phase One participants.
Lastly, the coach was asked if he had used any of the information in the modules, however, he
had only completed the modules the night before the interview, and therefore, had not had
sufficient time to implement the information. When probed further, the coach expressed sincere
intention to utilize the information, specifically the GROW model, and also stated:
―One way that I think I am going to use [the GROW model] is to pass it on to my
assistant coaches so that we‘re all on the same page as far as the leadership is concerned
and uh, it‘s gonna help me get organized, you know as far as putting things together,

54

getting a little bit of vision on our program as far as okay, we‘ve got to have some
direction here and uh, you know everybody wants to uh, success but we got to kinda
focus our attention.‖
Summary
The developers, mentors, and the coach all felt that an important goal of the program is to
widen coaches perspective of their coaching duties (i.e. view themselves as a CEO) and their
perspective regarding their impact on the sport of wrestling beyond their college or university.
Furthermore, all of the participants in Phase One recognized that an important goal of the
Leadership Academy is to help a coach gain self-awareness regarding his or her strengths and
weaknesses, and then intentionally take action (i.e. hire assistant coaches who can fill those gaps)
to build up those weak areas.
The participants in Phase One hoped that a coach, after completing Module 1, would be
have a greater sense of awareness regarding the threats to college wrestling and be able to
strategize on how to overcome said threats. Further, the participants hoped that coaches would be
able to recognize the importance of having a long-term vision, and be able to create and fulfill
long-term missions.
The participants in Phase One felt that Module 2 focused on skill development and hoped
that coaches would improve in their ability to communicate effectively by adding tools to their
toolbox (i.e. the GROW model). Lastly, the participants hoped that the coaches would recognize
how important it is to bring in people (i.e. assistant coaches, student coaches, etcetera) to
supplement their shortcomings as a coach.

55

Lastly, many of the participants felt that the expert interviews would be the most useful
teaching method to achieving the goals of Modules 1 and 2. Additionally, the majority felt that
the modules are too long and that this length could hinder the effectiveness of the modules.
Phase Two
To recall, Phase Two was designed to examine Purpose 2 of the study; and as an
evaluation of the impact of the first two online modules on the participating coaches with
regard to the objectives identified by the stakeholders (especially changes in coaches‘ self perceived knowledge, use, and intended use of the principles in the first two modules). This
was accomplished by evaluating the effectiveness of Modules 1 and 2 in educating and affecting
change within the intervention group coaches as compared to a control group of matched
coaches.
Descriptives. Descriptive statistics pertaining to intervention and control group
demographics are presented in Table 2. Independent sample t-test results on selected descriptive
statistics are also presented in Table 2. Results revealed that the control group was statistically
older and had significantly more coaching experience than the intervention group.
Table 2
Intervention and Control Group Demographics (N = 35)
Variable

Intervention (N = 35)
Frequency

M

SD

Control (N = 13)
Frequency

Gender
Female

1

0

Male

34

13

Head

32

13

Assistant

3

0

Level

56

M

P-Value
SD

Table 2 (continued)
Variable

Intervention (N = 35)
Frequency

M

SD

Years Coaching at Current

3.73

Total Years Coaching
Level
Age

Control (N = 13)
Frequency

P-Value

M

SD

2.95

7.30

3.91

.013

7.40

4.56

11.92

5.92

.004

34.43

6.78

40.62

8.89

.007

2.38

.870

.375

NCAA Division
Division I

9

4

Division II

7

4

Division III

8

2

NAIA

9

3

Other

2

0

Yes
Courses

11

5

No

24

8

7

1

28

12

Experience with Online

Own Program Elimination
Yes
Risk
No
Time Available for Leadership
Development (in hours)

2.77

1.46

Between-Group Analyses. Independent samples t-tests were conducted for each item to
examine differences between the intervention and comparison group. Of the 64 items at Time 1,
13 yielded significant differences between the control and intervention groups at a   0.05 level
(see Table 3). All, but one, of the significant differences at Time 1 favor the control group,
meaning that the control group had greater self-reported knowledge and intention to improve
abilities.

57

Table 3

Independent Sample t-test on Survey 2
Variable (Question number, description)

Intervention

Control

M

SD

Q1: Degree the sport of wrestling is at risk for elimination

3.17

.891

3.23 1.09 .848

Q2: Degree you feel the need to take action to help
college wrestling

4.43

.650

4.38 .655 .837

Q3: To what degree do you have a plan to improve your
program
Q4: To what degree do you have a plan to improve your
leadership abilities
Q5: To what degree do you intend to formulate a plan to
improve your program

4.09

.818

4.46 .519 .130

3.71

.957

4.00 .707 .333

4.51

.658

4.31 .751 .357

Q6: To what degree do you intend to improve your
leadership abilities

4.40

.812

3.92 .954 .091

Q7: To what degree do you intend to formulate a vision
for your program

4.69

.530

4.54 .519 .394

Q8: Knowledge of threats

3.60

.775

4.46 .660 .001

Q9: Knowledge of strategies to overcome threats

2.94

.684

4.00 .707 .000

Q10: Knowledge of strategies to build bridges to
supporters

2.57

.778

3.62 .961 .000

Q11: Knowledge of AD’s perspectives on the ideal
college program
Q12: Knowledge of the purpose of having a vision or plan
for a program
Q13: Knowledge of how to develop a plan or a vision

2.86

.772

4.08 .862 .000

3.94

.838

4.31 .751 .175

3.14

.879

4.00 1.08 .007

Q14: Knowledge of the methods of how to keep a vision
alive
Q15: Do you understand that the college coach of today is
akin to a CEO

4.08

.862

2.94 .802 .000

4.51

.919

4.54 .519 .929

Q16: Do you believe that marketing is an essential skill of
a college coach

4.74

.611

4.69 .480 .790

58

M

Pvalue*

SD

Table 3 (continued)
Variable (Question number, description)

Intervention

Control

M

SD

Q17: Do you believe that fundraising is an essential skill
of a college coach

4.71

.572

4.46 .660 .199

Q18: Do you believe that budget management is an
essential skill of a college coach

4.91

.284

4.69 .480 .054

Q19: Do you believe that public relations is an essential
skill of a college coach

4.86

.355

4.85 .376 .926

Q20: Do you believe that academic support is an essential
skill of a college coach

4.86

.355

4.69 .480 .201

Q21: Do you believe that communication is an essential
skill of a college coach

4.97

.169

4.92 .277 .467

Q22: Do you believe that recruiting is an essential skill of
a college coach

4.97

.169

4.85 .376 .116

Q23: Do you believe that community service is an
essential skill of a college coach

4.51

.702

4.31 .855 .397

Q24: Do you believe that community relations is an
essential skill of a college coach

4.83

.382

4.46 .776 .033

Q25: Do you intend to further develop your marketing
skills

4.66

.725

4.46 .776 .419

Q26: Do you intend to further develop your fundraising
skills
Q27: Do you intend to further develop your budget
management skills

4.69

.718

4.54 .660 .523

4.23

.973

4.00 1.00 .476

Q28: Do you intend to further develop your public
relations skills
Q29: Do you intend to further develop your academic
support skills
Q30: Do you intend to further develop your
communication skills
Q31: Do you intend to further develop your recruiting
skills
Q32: Intention to further community service skills

4.54

.701

4.23 .927 .216

4.49

.702

4.00 1.08 .074

4.57

.655

4.23 .927 .161

4.83

.453

4.54 .660 .090

4.34

.838

3.62 .961 .014

Q33: Do you intend to further develop your community
relations skills

4.54

.701

4.15 .899 .121

59

M

Pvalue*

SD

Table 3 (continued)
Variable (Question number, description)

Intervention

Control

M

SD

Q34: Do you agree that leadership is about being the most
charismatic hero out in front

2.69

1.30

2.62 1.12 .864

Q35: Do you agree that leadership is about wielding
unilateral power

2.49

1.26

2.69 1.37 .626

Q36: Do you agree that leadership is about solving
everyone‘s problems

2.14

1.24

2.15 1.06 .978

Q37: Do you agree that leadership is about providing all
of the answers for others

2.03

1.27

2.15 1.14 .757

Q38: Do you agree that emphasizing collaboration as a
leader means the follower or members have to stifle their
competitive urges?

2.14

.944

2.62 1.26 .167

Q39: Do you agree that leadership is about defining an
appropriate and compelling vision for your program?
Q40: Do you agree that leadership is about getting things
done via other people?

4.54

.611

4.62 .506 .705

3.66

.873

3.38 1.04 .367

Q41: Do you agree that leadership is about letting others
decide how to make a vision happen?

2.74

1.03

2.62 1.26 .723

Q42: Do you agree that leadership is about inspiring
others to take responsibility?
Q43: Do you agree that leadership is about creating an
environment where people can develop the skills needed
to be successful?

4.49

.658

4.46 .660 .911

4.77

.490

4.69 .480 .620

Q44: Do you agree that leadership is about mobilizing
people to face new challenges that require new values,
priorities, and habits and ways of doing things?

4.34

.765

4.38 .768 .867

Q45: Do you agree that leadership is about helping others
learn how to solve problems and develop solutions for
themselves?

4.69

.530

4.31 .751 .057

1.51

1.09

1.53 1.19 .947

Q47: Do you intend to use the GROW model in the
future?

2.31

1.23

1.61 1.26 .089

Q48: Have you engaged in a formal evaluation of your
strengths and weaknesses?
Q49: Intention to do a formal evaluation

2.77

1.33

1.92 1.25 .052

3.69

.990

2.62 1.39 .005

Q46: Do you know what the GROW model represents?

60

M

Pvalue*

SD

Table 3 (continued)
Variable (Question number, description)

Intervention

Control
M

Pvalue*

M

SD

SD

Q50: Do you believe it‘s important to hire assistant
coaches (and or team aides) whose strengths are your
weaknesses?
Q51: Have you hired assistant coaches (and or team
aides) whose strengths are your weaknesses?

4.34

.802

4.15 1.21 .534

3.40

1.14

3.46 1.19 .871

Q52: Do you recognize the impact your behavior (and
your programs behavior) has on other programs within
the sport of college wrestling?
Q53: Knowledge of why or how a college coach is akin to
a CEO

4.54

.852

4.54 .660 .987

3.91

1.12

4.77 .439 .011

Q54: Knowledge of common characteristics of effective
leaders
Q55: Knowledge of what external factors impact of
influence leadership
Q56: Knowledge of how modern leadership has changed
from years past

3.66

.725

4.08 .759 .085

3.31

.870

4.07 .862 .003

3.06

.838

3.38 .869 .240

Q57: Knowledge of common myths of leadership

3.17

.785

2.92 .954 .363

Q58: Knowledge of the components set forth by Rainer
Martens that effective coach leaders do

1.51

.919

1.54 .877 .935

Q59: Knowledge of the components of a team culture

2.89

1.11

3.69 .947 .024

Q60: Knowledge of Jack Welch‘s 4 E‘s for Effective
Leadership

1.71

1.15

1.77 1.30 .888

Q61: Characteristics of great coach communicators

3.03

.985

3.85 .555 .007

Q62: Knowledge of methods on how to build trust

2.86

.974

3.77 1.01 .006

Q63: Knowledge of the components of the GROW model

1.51

.919

1.38 .767 .653

Q64: Knowledge of methods on how to use and
implement the GROW model

1.40

.811

1.38 .767 .953

* - Italics indicate significance at the p  0.05 level.

A MANOVA was conducted to compare the intervention and control groups by the
subscales of Knowledge and Intention. Significant multivariate effects were found, Wilks‘
Lambda = .66, F(2, 45) = 11.48, p = .000. Further, the Tests of Between Subjects indicated that
61

both the Knowledge (p = .027) and Intention (p = .028) subscales contributed nearly equally to
the significant differences. The control scored significantly higher on the Knowledge subscale,
while the intervention group scored significantly higher on the Intention subscale. The
differences could indicate a sample bias on behalf of the control group. Thirty, hand selected by
the NWCA director, coaches were invited to participate in the control group and only13
responded. This lack of response by the other 17 coaches may indicate that those who did not
respond were ones that were the least interested in helping wrestling, and potentially the most in
need of coaching leadership growth, within the group of 30 invited control group coaches. The
lone item level difference at Time 1 that favored the intervention group (intention to further
community service skills) could allude to the point that the intervention group is aware of their
involvement in the NWCA Leadership Academy and are therefore more willing to intentionally
enhance specific areas of their leadership than the control group.
Within-Group Analyses
Time 1 to Time 2. A paired samples t-test was conducted for each item to explore
differences within the intervention group from Time 1 to Time 2. As shown in Table 4, 34, of
the 64, items resulted in significant increases, at the   0.05 level from Time 1 to Time 2.
Table 4
Paired Sample t-test Intervention Group Time 1 to Time 2 (n = 27)
Variable (Question description)

Time 1
M

SD

Time 2
M

Pvalue

SD

Degree that the sport of college wrestling is at risk for
elimination

3.11 0.80 3.85 1.10

.001

Degree to which you plan to improve leadership abilities

3.74 0.94 4.41 0.64

.000

62

Table 4 (continued)
Variable (Question description)

Time 1
M

SD

Time 2
M

Pvalue

SD

Degree of intent to formulate a plan to improve program

4.48 0.70 4.85 0.36

.022

Degree of intent to formulate a plan to improve leadership
abilities

4.37 0.84 4.81 0.40

.001

Knowledge of threats

3.56 0.80 4.07 0.83

.002

Knowledge of strategies to overcome threats

2.85 0.71 3.89 0.85

.000

Knowledge of strategies to build bridges to supporters

2.48 0.80 3.70 0.91

.000

Knowledge of AD‘s perspectives on the ideal program

2.74 0.71 3.70 1.10

.000

Knowledge of the purpose of having a vision or plan

4.00 0.88 4.56 0.64

.003

Knowledge of how to develop a plan or a vision

3.07 0.87 4.26 0.59

.000

Knowledge of the methods of how to keep a vision alive

2.85 0.81 4.00 0.55

.000

Agree that leadership is about being the most charismatic
hero out front

2.81 1.38 2.07 1.20

.005

Agree that leadership is about wielding unilateral power

2.44 1.37 1.85 1.32

.018

Agree that leadership is about solving everyone‘s problems

2.26 1.26 1.59 1.08

.013

Agree that leadership is about getting things done via other
people

3.78 0.93 4.19 0.83

.009

Agree that leadership is about letting others decide how to
make a vision happen

2.74 1.10 3.59 1.37

.001

Agree that leadership is about inspiring others

4.56 0.64 4.81 0.40

.050

Agree that leadership is about mobilizing people to take on
new challenges
Knowledge of what the GROW model represents

4.44 0.75 4.81 0.48

.005

1.52 1.16 4.22 1.01

.000

Intention to use the GROW model

2.44 1.01 4.22 0.97

.000

Intention to do a formal evaluation of strengths and
weaknesses

3.89 0.80 4.37 1.01

.045

63

Table 4 (continued)
Variable (Question description)

Time 1
M

SD

Time 2
M

Pvalue

SD

Belief of importance to hire assistant coaches whose
strengths are your weakness

4.37 0.79 4.67 0.48

.043

Knowledge of why or how a college coach is akin to a
CEO

3.93 1.07 4.74 0.44

.001

Knowledge of common characteristics of effective leaders

3.67 0.73 4.59 0.50

.000

Knowledge of external factors that impact or influence
leadership

3.41 0.75 4.40 0.64

.000

Knowledge of how modern leadership has changed from
years past

3.04 0.81 4.30 0.78

.000

Knowledge of common myths of leadership

3.26 0.76 4.15 0.86

.000

Knowledge of Rainer Martens components of effective
coach leaders

1.59 0.97 3.96 0.76

.000

Knowledge of components that lead to a team culture

3.04 1.09 4.30 0.72

.000

Knowledge of Jack Welch‘s 4 E‘s for effective leadership

1.67 1.07 4.29 0.72

.000

Knowledge of characteristics of great coach
communicators
Knowledge of methods on how to build trust

2.93 1.04 4.44 0.64

.000

2.93 0.96 4.26 0.66

.000

Knowledge of the components of the GROW model

1.59 0.97 4.03 0.85

.000

Knowledge of methods on how to use and implement the
GROW model

1.44 0.85 4.04 0.85

.000

* Question numbers were omitted because the question number was different from Time 1 to
Time 2.
Further, using the subscales of Knowledge and Intention as the dependent variables, a
paired samples t-test yielded significant differences from Time 1 to Time 2 (Table 5). The
increase in Intention is important considering only seven of the 35 intervention group
participants believed that their program is at risk for elimination. Overall it appears that, from the

64

analysis of these results, Modules 1 and 2 (the intervention) positively impacted the intervention
group coaches. One point to consider, however, is that the mean scores for both knowledge and
intention started relatively high at Time 1 and while there was a statistically significant increase,
the meaningfulness of that increase may not be as great. That said, the scores did increase
demonstrating positive impact of the modules.
Table 5
Paired Sample t-test on the Subscales of Knowledge and Intention Time 1 to Time 2 (n = 31)
Subscale

Time 1

Time 2

P-value

M

SD

M

SD

Knowledge

3.45

0.31

3.98

0.53

.000

Intention

4.27

0.55

4.47

0.54

.019

Particularly Meaningful Significant Differences. While many of items in the paired
samples t-test (Table 4) from Time 1 to Time 2 were significant, it is important to highlight a few
that seem particularly important to the overall program. First, the intervention coaches‘ opinions
regarding the degree to which college wrestling as a whole is at risk for elimination significantly
increased from Time 1 to Time 2. Getting the coaches to truly recognize and understand the
severity of this risk was an important objective for the director of the NWCA. One goal that was
stated by the majority of the stakeholders interviewed in Phase One was that the coaches became
aware of the threats to college wrestling, and knowledgeable about some strategies to overcome
those threats. The difference from Time 1 to Time 2 was significant at the   0.01 level for each
of these items. Another explicitly expressed objective from Phase One, that resulted in a
significant difference from Time 1 to Time 2, was that the coaches would be able to recognize
that they have to do more than just coach on the mat, and that the job of a collegiate head coach
65

is akin to that of a chief executive officer (CEO). While the coaches‘ belief that a head coach is
akin to a CEO began fairly highly at Time 1 (M = 4.44) and did not significantly change from
Time 1 to Time 2, their understanding of how and why a coach is akin to CEO did significantly
change from Time 1 to Time 2 as evidenced by the result on the following question: knowledge
of why or how a college coach is akin to a CEO. The stakeholders in Phase One also mentioned
the importance of creating a vision and hoped the coaches would be able to create a vision and
engage in long-term planning. The items that addressed these goals (knowledge of the purpose of
having a vision; knowledge of how to develop a plan or vision; and knowledge of the methods of
how to keep a vision alive) all saw significant increases from Time 1 to Time 2.
Learning and implementing the GROW model was explicitly mentioned by nearly all
participants in Phase One as an outcome goal for the coaches who complete Module 2. The
questions pertaining to the GROW model saw the greatest growth of all of the questions from
Time 1 to Time 2 as judged by the results on the following survey items: knowledge of what the
GROW model represents; intention to use the GROW model; knowledge of the components of
the GROW model; and knowledge of methods on how to use and implement the GROW model.
These significant findings are strong indicators that the Modules achieved some of the objectives
stated by the stakeholders in Phase One.
Finally, it is important to examine more closely the items that saw a significant decrease
from Time 1 to Time 2. All three of the items pertained to commonly held beliefs or myths about
leadership (degree to which you agree that leadership is about being the most charismatic hero
out front; degree to which you agree that leadership is about wielding unilateral power; and
degree to which you agree that leadership is about solving everyone‘s problems). While the
mean scores were not very high to begin with (2.81, 2.44, 2.26, respectively) each did see a

66

statistically significant decrease from Time 1 to Time 2. In addition to seeing decreases on these
items pertaining to leadership myths, the results also revealed significant increases on other items
pertaining to current views about leadership (degree to which you agree that leadership is about
getting things done via other people, degree to which you agree that leadership is about letting
others decide how to make a vision happen, degree to which you agree that leadership is about
inspiring others, degree to which you agree that leadership is about mobilizing people to take on
new challenges). These findings may indicate that the coaches were honest about their beliefs
about leadership and that the Modules, effectively, changed or reshaped those beliefs.
Findings Failing to Reach Significance. While over half of the items from Time 1 to
Time 2 resulted in significant findings, some of the non-significant findings are worth
mentioning. First, one item (degree to which you agree that leadership is about providing
answers for others) approached a significant decrease (p = 0.051). One goal of the stakeholders
in Phase One was that the coaches would be driven to take action to not only help their program,
but also have the perspective that the success of their program impacts the sport of wrestling as a
whole. One item on the surveys addressed the coach‘s degree to which he or she feels the need to
take action to help college wrestling. This item saw a slight increase from Time 1 to Time 2, but
it is important to note that the mean score was relatively high (m = 4.41) at Time 1, indicating
that the coaches came in with a high deree of interest in helping college wrestling. Finally, the
remaining differences that did not reach statistical significance from Time 1 to Time 2 consisted
of questions pertaining to the essential (as perceived by the architects of the NWCA Leadership
Academy) tasks of a collegiate head coach (i.e. marketing, recruiting, public relations,
communication skills, fundraising, community service, etcetera). The recognition of the

67

importance of these tasks and the intention to further develop their ability to execute these tasks
started and remained very high from Time 1 to Time 2.
Time 2 to Time 3. To further explore the impact of Modules 1 and 2, the intervention
coaches were tested again one month after completing the Modules. A paired samples t-test was
conducted to examine retention from Time 2 to Time 3 (n = 23). Nine of the 28-paired items
resulted in significant decreases from Time 2 to Time 3 and are shown in Table 6.
Table 6
Paired Sample t-test Intervention Group Time 2 to Time 3 (n = 23, unless otherwise stated)

Variable (Question description)

Time 2
M

SD

Time 3
M

Pvalue

SD

Degree of intent to formulate a plan to improve program

4.83 0.39 4.65 0.49

.043

Knowledge of how to develop a plan or a vision

4.21 0.60 3.82 0.72

.004

Degree of intent to further fundraising skills

4.78 0.52 4.52 0.59

.011

Knowledge of Rainer Martens components of effective
coach
Knowledge of components that lead to a team culture

3.87 0.69 3.04 0.77

.000

4.17 0.72 3.39 0.78

.000

Knowledge of Jack Welch‘s 4 E‘s for effective leadership

4.26 0.69 3.39 0.94

.000

Knowledge of characteristics of great coach
communicators (n = 22)

4.36 0.66 3.73 0.63

.000

Knowledge of methods on how to build trust

4.21 0.67 3.57 0.84

.000

Knowledge of methods on how to use and implement the
GROW

3.96 0.88 3.44 0.90

.025

Further, a paired samples t-test was conducted with the subscales of KnowledgeSmall
and IntentionSmall from Time 2 to Time 3. To recall, both Knowledge and Intention increased

68

from Time 1 to Time 2, however, from Time 2 to Time 3, KnowledgeSmall significantly
decreased while IntentionSmall saw no statistically significant change (Table 7).
Table 7
Paired Sample t-test on the Subscales of Knowledge and Intention Time 2 to Time 3 (n = 23)

Subscale

P-value
M

SD

M

SD

Knowledge

4.03

0.29

3.66

0.53

.000

Intention

4.55

0.40

4.41

0.47

.062

While statistically significant mean decreases were seen from Time 2 to Time 3, many of
the means at Time 3 were still higher than at Time 1. Another paired samples t-test was
conducted to examine the differences in mean scores on items that saw a significant decrease
from Time 2 to Time 3. As seen in Table 8, many of the nine items that resulted in significant
decreases from Time 2 to Time 3 were found to still be statistically higher at Time 3 when
compared to Time 1. These results indicate that although there was a marked drop in mean
scores from Time 2 to Time 3, the impact of the modules still had lasting power as compared to
Time 1.
Table 8
Paired Sample t-test Intervention Group Time 1 to Time 3 (n = 23, unless otherwise stated)
Variable (Question description)

Time 1

Time 3

P-value

M
Knowledge of how to develop a
plan or a vision
Knowledge of Rainer Martens
components of effective coach
leaders

SD

M

SD

3.13

0.92

3.82

0.72

.002

1.56

0.99

3.04

0.77

.000

69

Table 8 (continued)
Variable (Question description)

Time 1

Time 3

P-value

Knowledge of Jack Welch‘s 4 E‘s
for effective leadership

1.65

1.11

3.39

0.94

.000

Knowledge of characteristics of
great coach communicators (n =
22)
Knowledge of methods on how to
build trust
Knowledge of methods on how to
use and implement the GROW
model

3.00

1.02

3.73

0.63

.006

2.91

0.84

3.57

0.84

.001

1.43

0.84

3.44

0.90

.000

Finally, means and standard deviations were calculated for items that were asked at Time
3 but were not on any of the previous surveys (Table 9). The results from these questions should
be taken with trepidation due the simple fact that the coaches completed the surveys during the
summer and, therefore, were not in season and may not have had as many opportunities to
implement some of the teachings found in Modules 1 and 2.
Table 9
Means and Standard Deviations for select Survey 3 items (n = 23)
Survey Item

Have you developed a plan to improve
your program
Have you developed a plan to improve
your vision
Have you written out a formal vision
for your program
Have you shared your vision for your
program
Have you used the GROW model with
your wrestlers
Have you used the GROW model with
your coaching staff

M
[1 (not at all) to 5 (a
lot) Likert scale]
3.91

0.85

3.61

0.84

3.39

1.27

3.22

1.31

2.09

1.08

2.22

1.00

70

SD

Table 9 (continued)
Survey Item

M
[1 (not at all) to 5 (a
lot) Likert scale]
2.26

1.05

2.87

0.97

Have you implemented the components
set forth by Rainer Martens

3.04

1.02

Have you implemented the components
that lead to developing a team culture

3.13

0.97

Have you implemented methods on
how to build trust

3.17

1.11

Have you used the GROW model with
other individuals besides your coaching
staff or wrestlers
Have you implemented Jack Welchs‘ 4
E‘s for effective leadership

SD

Phase Three
The third purpose of this study focused on an evaluation of user preferences and
effectiveness regarding the delivery of the first two online modules, the components of the
program that were the focus of this investigation. Hence, the coaches‘ opinions regarding the
teaching components of the online modules (i.e. coach interviews, PowerPoint with voiceover,
etc) and their general opinions regarding the efficacy of each module were surveyed. First,
coaches were asked how impactful were each of the teaching components (Table 10).
Table 10
Impact of Each Teaching Component in Modules 1 and 2 (N = 27)
Variable

PowerPoint‘s with voiceover

M
(1 – 10;
1 = Not at all impactful,
10 = Very Impactful)
8.11

Video of other coaches discussing topics

SD

1.31
1.12

Exercises in the workbook

9.11
21
8.26

Video of experts

9.00

1.07

71

1.53

Based on the mean scores, the coaches identified that the video of coaches and experts
were the most impactful. This result reflects the predictions made by the participants in Phase
One. Nine coaches identified other components that had an impact on the learning objectives.
Some of these examples included: the production, tactical implementation with the workbook,
and how the topics were well thought out. When asked if they would prefer an alternative
format, 25 of the 27 participants indicated no, they would not prefer an alternative format. The
two coaches who would have preferred an alternative format were asked to specify what type
format, one indicated an in-person class, and the other indicated ―a list of external resources that
are applicable to each section of the Modules and accessible directly from the modules‖
(participant # 20065). All but one of the participants indicated that they did indeed like the
format of having the workbook coincide with the video. The one who answered ―no‖ to that
question followed up by stating: ―I would have liked a copy in PowerPoint that I could print‖
(participant 20031). All of the coaches indicated that they would recommend this course to other
coaches.
Coaches were then asked if they had completed all of the exercises in the workbook. Six
participants indicated that they did not complete all of the exercises with four of the six coaches
reasoning that not all of the exercises were applicable to them at this stage in their coaching
career. The other two coaches stated that they had either already done some of the exercises on
their own or were in the process of obtaining more information to complete some of the
exercises.
Coaches were then asked to indicate how long it took them to complete each module. The
majority of the coaches needed between 30 and 60 minutes to complete each module. The fact
that the coaches self-reported the completion times between 30 and 60 minutes is important

72

considering that Module One is just over 27 minutes long and Module Two is just over 52
minutes long. The issue is while it may take only 27 minutes to watch the video in Module One,
that time does not include the amount of time it takes the coaches to complete the workbook
exercises. Therefore, while the coaches are reporting that they are completing the Modules, we
do not know the level of effort and quality with which they are completing the workbook
exercises, which, as discussed in Phase One could be the most important aspects of the Modules.
Lastly, coaches were asked to indicate how capable they perceive themselves to be to implement
the teachings of each module given their individual resources, time available, and personnel. All
of the coaches responded either moderately or very capable.

73

CHAPTER 5
DISCUSSION
In spite of the fact that the NCAA Division I Wrestling Championships has historically
generated revenue in the top 5 of all NCAA Division I championships (NCAA, 2008), college
wrestling programs are continuing to be eliminated at an alarming rate. While there are a variety
of reasons programs are being cut (i.e. academic concerns, rising costs, Title IX compliance,
etcetera), the National Wrestling Coaches Association (NWCA) has set out to develop a state of
the art Leadership Academy to enhance the leadership ability of college coaches in order to
better equip them to manage and overcome many of the threats to college wrestling. The purpose
of this study was, in short, to evaluate the efficacy of a portion of the NWCA Leadership
Academy. This chapter discusses the findings of the current study, identifies implications of
these results with regard to both the NWCA and evaluation research, and discusses strengths and
limitations of the study. Wider implications for coaching education are considered.
Phase One: Qualitative Interviews of Key Stakeholders
Patton (2008) noted that program staff or stakeholders could become quite burdened by
the terminology of goals or objectives. Patton believed that ―in the minds of many program
people, from board members to frontline staff and participants, goals are abstract statements of
ideals written to secure funding—meant to inspire but never achieved‖ (p. 240, 2008). Therefore,
Patton suggested that the evaluator guides stakeholders through a process of identifying
participant outcomes and what they will observe within program participants as a result of
completing the program. This process was accomplished in Phase One.
The stakeholders identified that the program outcomes, in general, were for the coach to
(1) broaden his or her perspective about facilitating the health and growth of collegiate wrestling

74

beyond just his or her program, (2) recognize that being a college coach is akin to being a CEO,
(3) take the time to reflect on his or her coaching leadership weaknesses, and (4) take greater
personal responsibility for the success of his or her program and the success of the sport of
wrestling as a whole. It is interesting to note that when probed for information about how the
stakeholders would like the coaches to behave differently as a result of completing the
Leadership Academy, participants often took a more broad approach to the outcomes they were
seeking. Rather than simply stating that they would like the coaches to develop and use specific
skills, such as marketing through the use of social networking tools, they really wanted the
coaches to essentially become more aware of what being a college coach truly involves and that
coaching on the mat is only a fraction of what the job entails. This outcome goal reflects the
sentiment put forth by previous researchers that the college coach of today must possess
leadership qualities of both an instructional nature directed toward their athletes but also
leadership qualities unrelated to coaching technique and strategy (Baber & Eckrich, 1998;
MacLean & Zakrajsek, 1996; Mallett & Côté, 2006). Further, once the coach becomes more
aware that his or her role is more broad than just enhancing wrestler performance on the mat,
then he or she can begin to view him or herself as a CEO who has goals that also include
academic performance of the athlete, fund raising, community relations and enhanced athlete
health and safety. The stated objectives of the stakeholders clearly then fit together. First, they
want the coaches to broaden their awareness of their role as leaders beyond traditional views of
coaching, then they want the coaches to become aware of all of the duties this broadened role
requires, and finally they want the coaches to reflect on their strengths and weaknesses of their
increased scope of duties.

75

Considering that the majority of stakeholders hoped that, as a result of completing the
Leadership Academy, the coaches would take time to reflect on their coaching leadership
strengths and weaknesses may indicate that the stakeholders are concerned about the bravado
and ego component that is common in college wrestling. As one mentor stated, ―It doesn‘t make
you a weak person to have weaknesses, it makes you a strong person to identify your weaknesses
and then work on those‖ (Mentor 2). This plea may be an unobtrusive indicator that many
college wrestling coaches are unwilling, for fear of being perceived as weak, to examine and
admit to their weaknesses or hold to traditional views that their job is to simply train and develop
their athletes technically and tactically. Moreover, some leadership experts perceive this ability
to recognize one‘s strengths and weaknesses and changing roles to be critical in leadership
development. Hackman and Wageman (2007) stated that ―error and failure provide far more
opportunities for learning than do successes and achievement, precisely because failures generate
data that can be mined for insight into how one‘s assumptions and [leadership] models of action
might be improved‖ (p. 46). Moreover, Hackman and Wageman further cautioned that reflecting
on ones weaknesses might not occur, or might occur with a defensive posture thus limiting the
effectiveness of the value of learning from one‘s mistakes and weaknesses. To extend this point,
researchers in the field of leadership development have indicated that self-reflection alone is not
enough, and that an outside entity should examine whether the self-reflection process is adaptive
or maladaptive (Avolio & Hannah, 2008). Simply, adaptive self-reflection is a constructive
process that is reflected by openness and eventually contributes to more effective choices and
actions over time (Trapnell & Campbell, 1999; Carver & Scheier, 1982). Maladaptive selfreflection is more destructive and often results in negative emotions, which can ultimately
diminish one‘s engagement in leadership development activities (Mor & Winquist, 2002). Based

76

on these points, one suggestion for the developers of the Leadership Academy is to teach the
mentors the components and indicators of adaptive and maladaptive responses to leadership
feedback, and then how to guide the coaches to engage in adaptive self-reflection more
frequently. For example, program organizers might accomplish this by requiring participating
coaches to complete all module exercises and activities and then in the in-person portion of the
Leadership Academy, work through key elements of the completed workbooks with their
mentors. This would not only hold them accountable for completing the workbooks but help
them reflect with an experienced mentor who could provide advice to ensure the reflective
process is adaptive versus maladaptive.
As a result of completing Module 1, the stakeholders indicated that they would like the
coaches to be more aware of the threats to college wrestling and the solutions to the threats, as
well as recognize what it takes to run a quality program. The stated objective of being more
aware of the threats to college wrestling could be a second order effect of broadening a coach‘s
awareness of his role. Meaning, a shortcoming of a coach could be that he or she solely exists
within his or her bubble of coaching his or her athletes, at his or her university, in his or her
conference. This narrow view of focus may limit the coach‘s ability to see the bigger picture and
be less aware of what are the larger scale threats to college wrestling as a whole because he or
she is only focusing on the threats he or she perceives are putting his or her, and only his or her,
program at risk. It seems that the stakeholders are keen on getting the coaches to see the bigger
picture in all aspects of coaching. The stakeholders want the coaches to see how they need to do
more than just coach, how they need to be more aware of the impact their program has on other
programs, and how they need to be aware of all of the threats out there, not only the ones directly
influencing their specific program. Interestingly, the coach interviewed in the first phase of this

77

study explicitly stated that as a result of Module 1 he became more aware of the importance of
broadening his focus beyond just his own program.
Specific to Module 2, the stakeholders indicated that they would like coaches to increase
their ability to communicate effectively by implementing communication tools such as the
GROW model. This objective may be especially important for coaches who have a traditional
view of coaching that their job is to simply train and develop their athletes technically and
tactically. With this traditional mindset, it is likely that the coach will have highly developed
communication methods for this narrow role. However, those specific communication tools may
not be the most effective when engaging in the leadership requirements unrelated to coaching
technique and strategy (i.e. communicating with alumni, fund raising, and community relations).
In addition, if the coach has not yet expanded his or her view of what the coaching role truly
entails (both traditional coaching duties and leadership duties unrelated to coaching technique
and strategy), then he or she will likely see very few, if any, weaknesses in the area of
communication skills. Furthermore, it becomes clear then that stakeholders‘ outcome objectives
involve both macro and micro points about coaching leadership, and that the outcome objectives
build off of one another. Thus, many of the foundational and macro goals (i.e. broader
perspective, self-reflection, view role as a CEO, etcetera) build up to and create the motivation
and internal desire for the coach to obtain some of the micro goals (i.e. awareness of threats and
solutions, ability to create and engage in a vision, and communication skills). Phase two
examined whether or not both the stated macro and micro goals were obtained as a result of the
two online modules.
Phase Two: Evaluation of Module’s 1 and 2 Intervention

78

Overall, the results of this phase and purpose of the study suggest that the two online
modules of the Leadership Academy were effective in facilitating change in the coaches‘
knowledge and intended use of the coaching leadership skills covered in the first two modules. It
should be noted that while the general outcome goals (i.e. greater awareness of all that a
coaching role entails, increase in personal responsibility for the success of college wrestling as a
whole, etcetera) identified by the stakeholders in Phase One were the outcomes they would hope
for as a result of completing the entire program (including all of the online modules, the inperson portion, and the mentorship piece). The results from this phase indicated that some of
those objectives have been achieved, in part, as a result of completing only Modules 1 and 2.
Unfortunately, because the control group did not complete all requirements and therefore
the original design to be implemented as originally planned, it cannot be concluded that there is a
causal relationship between completing the online modules and changes in the coaches‘
knowledge and intended use of the material. However, the fact that differences were seen from
Time 1 to Time 2 in the intervention group does provide some evidence that the intervention had
an impact.
The greatest changes in the study were seen from Time 1 to Time 2. Furthermore, in most
cases the 1-month retention evaluation (Time 3) remained statistically equivalent to the Time 2
levels. Moreover, most of the items that significantly decreased from Time 2 to Time 3 remained
significantly above Time 1 levels. This result is encouraging as they support the notion that the
intervention had a lasting effect. However, the fact that dissipation did occur and in a short
period (only one month) does give cause to consider that many of the skills taught in the modules
could be perishable skills. Even though there was perceived knowledge depreciation, the transfer
of knowledge may still be greater than in a traditional course. However, this knowledge

79

depreciation is likely comparable to that of a traditional course considering that a meta-analysis
comparing distance education to classroom found that there is strong evidence that distance
education is comparable to classroom education on achievement, retention, and attitude
(Bernard, et al., 2004). However, the material being covered could impact how a program
designer might decide between a traditional classroom approach versus a distance learning or
online model. Hansen (2008) demonstrated that the online educational environment might be
better suited for applying theoretical knowledge than a traditional face-to-face classroom.
However, the result that knowledge did depreciate over time is still of considerable importance
because it suggests efforts must be made to enhance material retention and potentially transfer.
The onset of the mentorship component of the Leadership Academy may serve this function but
there may be a time sensitive component, in that the mentorship piece should be implemented in
the months following the completion of the online modules. The role of the mentors, then, may
be significant in reinforcing the skills covered in the modules. It should also be noted that the inperson workshop is designed to come after all the modules are completed and will reinforce
much of the content and, in so doing, hopefully facilitate knowledge retention, especially when
the mentors are involved throughout both of the online and in-person portions of the academy.
Considering that one of the overarching goals of the academy is to create the need for
change within the coaches and then, actually, change behavior within the coaches (to engage in
more leadership behaviors unrelated to coaching technique and strategy), examining a theory of
change becomes important. One such theoretical perspective on change, is the transtheoretical
model (Prochaska & Norcross, 2001). Briefly, the model entails six stages of change that focus
on the decision making process of the individual. Each stage represents a process step in which
the individual grapples with a different set of issues and task that relate to changing behavior.

80

The stages are precontemplation (not yet acknowledging the problem and come to coaching at
others urging); contemplation (acknowledge the problem but are not ready or willing to make a
change); preparation (getting ready to change in the near future or start making small changes);
action (make observable changes in behavior and are enacting their plan); maintenance
(maintaining the behavioral change, but relapse risk is high); termination (former problem no
longer exists or presents any threat). Relevant to the current study, it is important to recognize
where the coach is when they are brought into the Leadership Academy. For example, a coach
who is unaware of the threats to college wrestling and his or her program may be in the
precomtemplation stage and therefore the first part of Module One where it is discussed that
college wrestling is under attack may be critical for the coach (and mentor) to help move the
coach to the next stage. Additionally, the mentor and the Leadership Academy will play a critical
role during the maintenance stage considering that relapse risk to old habitually behaviors is
high. Therefore, it is recommended that the mentors be made aware of these stages and briefed
on their role and strategies to aid in behavioral change during each of the stages.
As the results showed, many of the program and module objectives that were identified
by the stakeholders in Phase One appear to have been achieved as a result of completing the two
online modules. Most notably, the majority of the stakeholders wanted the coaches to become
aware of how encompassing their role as head coach truly is and how and why their role is akin
to that of a CEO. In many ways, achieving these objectives can result in a mindset shift from that
of a coach who focuses almost entirely on coaching and teaching skills directly related to
performance on the wrestling mat, as opposed to one who engages in those standard duties in
addition to coaching duties unrelated to technique and strategy. Once this mindset shift is
broached, the outcome objective of having a coach identify his or her strengths and weaknesses

81

becomes a much broader experience. Rather than just examining strengths and weaknesses
within the scope of only leadership duties directly related to technique and strategy (perhaps the
area where the most strengths exist), the coach now has to examine his or her strengths and
weaknesses amongst his or her ability to market the program, to interact with alumni, to fund
raise, to engage in community service, etcetera. This expanded scope of self-analysis can then
indirectly impact one‘s motivation to engage in leadership development and training. Motivation
becomes particularly important because the only other potential indicator of motivation was an
item asked at Time 1 about the coach‘s belief regarding whether or not his or her program is at
risk for elimination, in which 28 out of 35 coaches did not believe their program was at risk.
Therefore, it is unclear how motivated the coaches are to engage in leadership development, but
perhaps their motivation increases alongside the expansion of their view of their role as a head
coach.
Phase Three: Coaches’ Opinions Regarding Ease and Utility of the Online Format
Michael Scriven (1991) believed that one of the ways evaluators engage in the process of
evaluation is to determine the worth of a program by examining the extent to which the program
met the needs of their target audience and whether or not meeting those needs was accomplished
in a cost-effective manner. Some of the results from Phase Three of this current study indirectly
indicate that the Leadership Academy met the needs of college coaches in a cost-effective
manner. First, coaches were asked if they would prefer an alternative format other than the
online format; 92% of the coaches indicated that they would not prefer an alternate format.
Although, there were no probing questions following up on these responses, it can be culled that
for the material that was covered and the method in which it was delivered the online format met
the needs of the coaches. Secondly, coaches were asked whether they would recommend the

82

Leadership Academy to other coaches and every coach indicated that they would recommend the
Leadership Academy to their peers. It could be assumed that a coach would not recommend a
course to his peers if he did not himself believe that it met his needs. It should be noted that at
the time of completing this phase the participating coaches had not yet been involved in the inperson portion of the Leadership Academy and were, therefore, recommending the course to
other coaches solely based on their experience with the online modules. Moreover, the young
coach from Phase One, when asked if he would recommend this course to other coaches,
indicated that he would recommend the course adding, ―I think it helps [a coach] focus on things
[a coach] might not have thought about, you know there are just so many things here that it
covers…and I just think it‘s invaluable and I don‘t think there is a lot of training out there
available (on these topics) ‖ (Coach 1). This comment highlights that the NWCA Leadership
Academy satisfies a need that exists within collegiate coaching leadership development.
Lastly, determining whether or not the online portion of the Leadership Academy is costeffective could be accomplished by examining two areas. First, the participants in Phase One
indicated that they believed the purpose of the having the online portion of the Leadership
Academy was either for convenience and accessibility or that it was a cost-effective way to
deliver a significant amount of information to many coaches spread across the country.
Secondly, measures of cost-effectiveness can also include the amount of time spent completing a
task. Theoretically, the more time a task takes to complete effectively, the less cost-effective it
would be. The majority of coaches in Phase Three indicated that completing a single module of
the online portion of the Leadership Academy took between 30 and 60 minutes. This result is
another indicator that the Leadership Academy is cost-effective in both a fiscal nature and a
matter of time spent. However, the fact that the coaches indicated that it only took them between

83

30-60 minutes to complete the modules could indicate that they either did not complete the
workbook exercises (only 6 out of 28 indicated that they did not complete all of the workbook
exercises) or that they did them haphazardly. Furthermore, a tension appears to exist between
getting the coaches to complete the modules given their time demands and having the
appropriate amount of time to substantively deliver all of the information necessary to
accomplish the monumental goals of the NWCA (i.e. trying to change the coaching leadership
culture). Awareness of this tension could influence content development. For example, the
participating coaches may prefer expert interviews over PowerPoint slides, but expert interviews
are not always the most efficient way to deliver a significant amount of in-depth content.
Implications and Future Directions for the National Wrestling Coaches Association
The results of this study indicate that the online portion of the Leadership Academy
appears to be an effective method to enhance the leadership abilities (unrelated to performance
on the mat) of college coaches around the country. However, due to some of the limitations of
this study, including the lack of control group, the extent of the impact of the online modules is
unclear. To recall, Schweigert (2006) indicated that it is important to fully define the term
―effectiveness‖ when examining the outcomes or impact of a program. Schweigert posited that
effectiveness could be measured in different ways. First, a program could be considered effective
if the participants had an increase in understanding or knowledge of the material covered in the
program. While the measures of knowledge and understanding were self-reported, it appears that
the online program is headed in the right direction to satisfying this definition of effective.
Another method of measuring effectiveness according to Schweigert is causality. Causality, in
this instance, is the ability to attribute program outcomes to program actions (i.e. knowledge
changes associated with completing or not completing the online modules). Schweigert echoed

84

the idea that while experimental designs are the ―gold standard‖ of evaluation they are not
always possible or feasible, therefore, other research designs (i.e. quasi-experimental or natural)
can approximate a causal demonstration. The current evaluation was originally intended to
employ an experimental design (although it would have been even better if a placebo control
group who took unrelated modules were involved), but, due to some unanticipated factors
(outlined in the following section), this study resulted in a design that more closely reflects either
a quasi-experimental design or a single group case study design. Therefore, the extent to which
causality (and effectiveness) can be determined is limited. Thus, if the stakeholders within the
NWCA are looking to strengthen the determination of causality and effectiveness of the online
modules, it is advised that they implement an experimental design in their next evaluation.
Furthermore, the study only evaluated the first two of 10 online Modules. There are both
strengths and limitations of this design as it pertains to the NWCA. First, Patton (2008) argued
that component evaluation (examining only a part or parts of a program) could be a useful
endeavor for an evaluation. Essentially, evaluating a single entity of a program‘s actions could
help the program developers in determining potential improvements to that component, or it
could determine if further development of that component (i.e. completing the unfinished online
modules) merits continuation. Analyses of the results indicate that the online format is effective,
meets the needs of the coaches, and therefore merits continuation. Regarding determining
potential improvements of a particular component, this depth of this study allowed the researcher
to determine some potential improvements. For example, some of the stakeholders in Phase One
feared that the length of each Module would inhibit its effectiveness. Objectively, this did not
appear to be the case as the majority of participants only needed 30-60 minutes to complete each
module, but, as previously mentioned, the 30-60 minute completion time frame could indicate a

85

failure to effectively complete the workbook exercises which is a critical component of the
modules. Additionally, the coach from Phase One indicated that while he did not like the length
of the modules he completed them late at night and pontificated that had he completed them
during the day they might not have seemed so long. Although, there were no questions pertaining
to the ideal length of the modules, it would most likely be advantageous for the NWCA to keep
the online modules within the 30-60 minute time frame. To add to this point, as indicated in
Table 2, all of the coaches surveyed indicated that they have over 2.5 hours per week available to
work on leadership development.
Another potential improvement could be for the NWCA to consider incentives for
documented completion of the Modules. It was not the intention of the NWCA for the
participating coaches to passively complete the modules by watching 10 online videos. Rather,
the objective was for the participants to actively engage in the material by putting thought and
energy into completing the workbook exercises. Therefore, the NWCA could withhold the
promised stipend (for attending the in-person workshop) until the coaches have their completed
workbook checked by and discussed with an NWCA mentor. The NWCA, currently, has no way
of checking the quality of the effort put into completing the workbook exercises, and further the
participating coaches in this study may have avoided exercises that explored their weaknesses or
ones that they were uncomfortable with, which they could have benefited from the most.
Another finding of interest to the NWCA is that the coaches participating in the
Leadership Academy found that learning from other coaches was the most impactful teaching
component of the modules. While this result reflected the predictions made by the stakeholders
in Phase One of the study, it is still useful in that it could influence how the remaining modules
are formed. For instance, the developers of the NWCA Leadership Academy could consider

86

reducing the number of PowerPoint slides and increase the number of coach interviews, and
determine ways to get the same information across but through the use of personal stories and
accounts from experienced coaches. However, as mentioned previously, decreasing the
PowerPoint slides could compromise the richness and the amount of material covered in the
modules. To use a sport analogy, the wrestler may enjoy scrimmaging the most, but conditioning
drills may be more effective to his or her development.
One limitation of component evaluation (only evaluating the first two of 10 online
modules) is that it is unclear how all three components of the Leadership Academy interact to
affect change within the leadership abilities of the coaches involved in the program. It was
previously noted that the online modules started the process of achieving the overarching goals
of the entire Leadership Academy as determined by the stakeholders. This finding is promising
and could lead the NWCA developers to reconsider the direction and desired outcomes of the
mentorship and in-person workshop portions of the Leadership Academy. If the online portion is
accomplishing many of their larger scale objectives then gearing the other prongs of the
Leadership Academy towards the same objectives does not make much sense. However, surveys
from the in-person workshops demonstrate that the in-person workshops are very effective, and a
little overlap (between the in-person workshop and online modules) may be useful to aid in the
prevention of poor knowledge retention from the online modules. Therefore, it is strongly
recommended that the NWCA, perhaps with the help of an evaluator, create a logic model
detailing the connections between activities and outcomes. A logic model could help the NWCA
stakeholders recognize all of their available resources and inputs, and how those lead to specific
short-term and long-term program outputs and objectives. However, creating a logic model
might be a challenge for the NWCA.

87

The director of the NWCA is extremely passionate about his mission to not only save
college wrestling but also bring it back to its once thriving place amongst all college sports. This
passion has produced an untiring work ethic within the director but also a lack of patience
wanting to implement change right away and at times continually deriving new ideas for the
program. Essentially, ―[the director] is always a moving target, he always has new ideas every
time I see him…and to commit to a logic model for him might be a two or three year process, but
he might think he is committed to it in the first month, but he is reactive to what his is happening
in his current environment‖ (D. Gould, personal communication, October 11, 2010). This
approach to creating a world class leadership development program is certainly indicative of the
director‘s intense motivation to have this program be successful and to save college wrestling.
However, while the director is certainly well intentioned, he may not recognize that this lack of
commitment to a specific direction can negatively impact the effectiveness of the Leadership
Academy. Furthermore, the director of the NWCA is, much like a head coach, juggling a lot of
different responsibilities simultaneously and may lack the ability to step back and see the big
picture of how all of the components of the Leadership Academy fit together to lead to his
outcome objectives. Thus, although it will take some patience and commitment, the creation of a
logic model could be quite a useful exercise for the director to engage in.
Additionally, the long-term outcomes of the intervention were not examined in this
evaluation. Retention was measured after 1 month, which may not have been an adequate
amount of time to truly examine how well the coaches retained their increase in knowledge, and
the extent to which they are implementing the tools taught to them in the modules. Therefore, it
is recommended that the NWCA considers the timing of the interventions and examines longterm retention and use. Regarding the timing, it is recommended that the coaches complete the

88

online modules and the in-person workshop immediately prior to the start of college wrestling
season. College coaches of all sports are stretched extremely thin during the season and the
impact of the Leadership Academy could be greatly reduced due to the typical time demands of a
coach during the competitive season. Furthermore, as was discussed previously, it appears that
many of the skills taught could be perishable if not used shortly after learning them. This finding
gives further cause to implement the Leadership Academy immediately prior to the start of a
season so a coach can then begin using and refining the skills right away, during the season. It is
therefore recommended that the NWCA capture retention and usage more than six months after
the coaches complete the Leadership Academy. If the coaches complete the program prior to the
start of the season, then implement the lessons learned during the season, a 6-month retention
questionnaire would be completed nearly immediately after the end of the competitive season
thus providing the NWCA with valuable information regarding the full scope of whether or not
the coaches retained and utilized the information from the Leadership Academy. Considering
that the coach will likely be in the maintenance stage of the transtheoretical model, the mentors
role becomes critical at this point to aid in commitment to the new behaviors (Prochaska &
Norcross, 2001).
Finally, it is recommended that, in conjunction with the aforementioned
recommendations, the NWCA conduct an evaluation of the NWCA Leadership Academy that
examines the program in its entirety to truly get a complete picture of the impact that the
Leadership Academy is having on the coaches and college wrestling as a whole. This
recommendation can be completed, in part, by observing and evaluating (through 360-degree
surveys) the coaches who have already completed the NWCA Leadership Academy.
Implications and Future Directions for Evaluation

89

The current study demonstrated that while impact evaluations may be the most appealing
to researchers and stakeholders, an impact evaluation might not be the most appropriate
evaluation type. The director of the NWCA, the developers of the Leadership Academy, and the
researcher may have been swayed by the lure of an impact evaluation study that could nearly
definitively show that the hours and hours of manpower that were invested in creating the
Leadership Academy were validated by demonstrating a near definitive causal linkage between
the programs actions (online modules, in-person workshop, and mentoring) and the behavioral
changes within the coaches, and, ultimately, by saving college wrestling. The researcher of the
current study fully intended for this study to examine the impact of a component (two of the
online modules) of the Leadership Academy. While there were elements of an impact evaluation,
in many ways this study did not meet the criteria necessary for an impact evaluation. Instead this
study more closely reflected a component outcome evaluation. Furthermore, this study shed light
on the evaluability of the NWCA Leadership Academy.
To recall, an impact evaluation is typically larger scale and examines both program
outcomes and long-term impacts of the program. Moreover, impact evaluations are resource
intensive and are inherently comparative often utilizing a randomized experimental design.
Whereas, an outcome evaluation is usually more small scale, time-limited, and does not
necessarily involve a basis of comparison. Rossi, Lipsey, and Freeman (2004) outlined that a
program is ready (or evaluable) for an impact evaluation when there is a program theory, a
decent amount of process data has already been collected or is readily accessible, timelines and
expectations have been reviewed, and resources are available for the evaluation. While some of
these criteria have been met implicitly by the NWCA, they certainly were not met explicitly. At
the onset of this current study, there was no existence of a logic model; the specific process of

90

how the Leadership Academy is delivered was still unclear; and expectations may have been
intuitively understood but were not explicitly determined to a great degree. Furthermore, the
director of the NWCA was unable to fully commit to an experimental design. This reality
seemed to occur at the crossroad of what was appealing (an impact evaluation) and what they
were willing to sacrifice (control of group designation). An examination of the psychological
difference between hiring an evaluator and someone requesting to do an evaluation of your
program may provide some insight into this lack of commitment.
The current evaluation study was externally, rather than internally, driven. The director of
the NWCA was deep into creating the Leadership Academy (amongst his bevy of other duties)
and garnering support for the academy when the researcher offered to do a component impact
evaluation. Perhaps the director felt that a positive result on the evaluation could help him
generate some more support (both political and financial) and or would provide some insight
from his target audience (college wrestling coaches) on the program and potential future
directions. Another possibility is that he was merely trying to help the author who he knew was
trying to complete his degree and spent many hours developing modules for the NWCA.
Therefore, while the director was committed to an evaluation he may not have been as
committed had he himself reached out to an evaluator to examine the impact that the program
was having. The researcher made every attempt to ensure that the director was committed to the
study design of identifying 60 coaches and then randomizing them into two matched groups
(control and intervention). The director on numerous occasions, both by email and telephone,
agreed to this design but when it came time to identify 60 coaches, the director instead chose to
identify only 30 coaches and deem them the intervention group. While the researcher explained
the negative impact that this decision would have on the strength of the evaluation, the director

91

was significantly more concerned about ensuring that those 30 coaches received the Leadership
Academy training, which is understandable given that his primary task is to help save college
wrestling by training the coaches who most needed the program training. Thus, he made it clear
that reaching these coaches at this specific time was more critical than the quality of the
evaluation design. The implication for future evaluation research is to strongly consider the
evaluand‘s motivation for engaging in an evaluation. If an evaluator approached an individual
then he or she is less likely to be committed to the evaluation methodology, than an individual
who reaches out to an evaluator and is truly interested in determining the extent to which his or
her program actually effects change within the program participants. Because of this change in
commitment, the current study became more of an outcome evaluation, with a bit of an
evaluability assessment, than a true impact evaluation.
Evaluability assessments are often aimed at getting the program stakeholders to clearly
define and articulate the program‘s design and goals. Often evaluability evaluators will, from the
data collected, create logic models detailing the connections between activities and outcomes.
While this current study did not result in a completed logic model it did provide some insight
into those connections and did get the stakeholders to explicitly express their goals for the
Leadership Academy beyond the long-term goal of ―saving college wrestling.‖ Further, the
current study demonstrated that the director of the NWCA was singularly committed to initiating
the Leadership Academy and, perhaps, not ready, from both a psychological standpoint and a
resource standpoint, for an impact evaluation.
Implications for Coaching Education
As previously mentioned, many coaching education programs are designed for High
School coaches, are aimed at player safety and development, and few go beyond the instructional

92

duties of a coach. Furthermore, while programs do have an online presence (American Sport
Education Program, 2009; National Federation of State High School Associations, 2009) none,
other than the NWCA Coaching Leadership Academy, have been found that educate coaches on
coaching duties beyond player safety and development and combine online education with an inperson workshop and ongoing mentoring. The current study adds to research in coaching
education because while many programs rely on online education, the results from this study
indicate that a decrease in knowledge retention may be the result of just utilizing online
education, thus making a strong case for future programs to include an in-person portion to
supplement the online presence.
Another implication for coaching education is to consider not only the time it takes a
coach to complete the education, be it in-person or online, but also when the education is
delivered to the coach. While online education frequently provides coaches with the luxury to
complete it when they have time available, providing online education during the competitive
season can greatly impact the efficacy of the program if the coach is just trying to complete the
course because his or her other time demands are so great. Furthermore, time considerations
become even more critical if the educational model includes an in-person workshop that follows
the completion of the online education, as is the case in the NWCA Leadership Academy.
Therefore, it is recommended that future coaching education programs implement the education
during the off-season but near the start of the competitive season in order to provide the coach
opportunities to immediately implement the lessons.
Strengths and Limitations
Patton (2008) has strongly suggested that one role of an evaluator is to help stakeholders
identify meaningful goals and objectives regarding behavioral changes and differences they

93

would like to observe within program participants as a result of completing the intervention. The
current study aided the program director and developers in extending and further defining their
goals beyond ―saving college wrestling.‖ Rather than just assuming that the goals of the modules
are to develop the skills covered in the Modules the researcher successfully challenged the
stakeholders (program developers, mentors, and a coach from a previous Leadership Academy)
to identify more specific and meaningful goals.
Another strength of the study is that the participating coaches were not only tested on
their perceptions of learning, but also on their perceptions of their experience interacting with the
online Modules. Examining this component added to the richness of the study in that the study
not only examined changes, but also examined whether or not the method of creating those
changes met the needs of the participants.
Although there are strengths associated with this study, there are also some limitations
worth mentioning. One limitation was that there was no member check in Phase One. Because
the focus of the evaluation was on Phase Two the researcher did not send the transcribed
interviews back to the participants to verify the transcribed data. This lack of a member check
may have limited the accuracy of the data collected in Phase One, thus potentially impacting the
objectives that were collected in Phase Two.
Another limitation of the study was that the control group was more than half the size of
the intervention group to start and virtually non-existent (n = 4) for the comparison survey (Time
2, the post-intervention survey). The lack of a control group made it impossible to warrant a 2 X
2 (Group by Pre/Post) design and, therefore, limited the degree to which causality could be
measured. Further, causality determinations could have been improved with the existence of a

94

placebo group (a group of coaches who completed an unrelated series of modules), in addition to
standard control group.
Another limitation of the study was that the coaches completed the first post-intervention
survey (Time 2) after they had completed all five available modules, even though the focus of
the study was on the first two online modules. The researcher made this decision when he
realized that it was virtually impossible to ensure that the coaches complete the survey
immediately following completion of the first two modules. The coaches were provided access to
all of the five online modules and were required to complete all of them within a time frame
designated by the director of the NWCA (in order to provide them with their stipend with enough
time to schedule the in-person academy). The researcher chose to have them complete the first
post-intervention survey after completing all five modules in an effort to avoid unequal dosage
(i.e. some coaches would complete the survey after finishing the first two modules, while others
complete the survey after finishing four modules, etcetera). However, completing all five
modules could have impacted the knowledge, retention, and overall opinions regarding the first
two modules.
Another limitation is that purposeful sampling that occurred. Due to purposeful sampling
in Phase One the sample may not have been entirely representative of the wrestling coach
population due to subjectivity on behalf of the director of the NWCA. Meaning, the director may
have selected coaches, mentors, or architects that are largely proponents of the NWCA
Leadership Academy limited the possibility of disconfirming evidence for the program.
Lastly, the current study measured coach perceptions of learning and not actual learning
or retention. As summarized by Dunning, Heath, and Suls (2004) people‘s self-views have a
weak correlation to their actual behavior and performance. Essentially, people have a positive

95

illusion of themselves. Therefore, in this study the coaches may have rated their knowledge,
intended usage, and retention higher than it actually was. Furthermore, citing research in
education, Dunning, Heath, and Suls (2004) illustrate that students‘ assessment of their
performance moderately aligned with assessments by their teachers and mentors. These findings
further demonstrate the importance of future evaluation studies integrating 360-degree type
assessments for the coaches. 360-degree assessment tools allow a coach to not only self-assess
but also get feedback on the same items from peers, superiors, and subordinates (i.e. his or her
wrestlers).

96

APPENDICES

97

Appendix A
Phase One Participants Demographic Questionnaire
1. Name (Last, First): ______________________________________
2. Today‘s Date: __________
3. Age: __________
4. NCAA Division:
a. Division I
b. Division II
c. Division III
d. NAIA
e. Other
f. Not applicable
5. Number of years coaching (if applicable): _____________
6. Coaching level:
a. Head Coach
b. Assistant Coach
c. Not applicable
7. If applicable, total number of years as a Head Coach: _______________
8. If applicable, total number of years as an Assistant Coach: _____________
9. What is your current position?
a. Fill-in
10. What is your role in the NWCA Leadership Academy? (Select all that apply)
a. Mentor

98

b. Content Developer
c. Coach participant in inaugural academy
d. Other __________________________
11. Circle your highest degree earned:
a. High School Diploma
b. Junior College Degree (i.e. A.A.)
c. B.A. / B.S.
d. M.A. / M.S.
e. Ph.D/EDD
f. Other _________________
12. If applicable, what area of study did you earn in your undergraduate degree:
_____________________________
13. Have you obtained any coaching certifications? Yes

No

a. If yes, please specify: ______________________________________
14. Have you ever taken any formal leadership courses? Yes No
a. If yes, please specify: ______________________________________
15. Do you have any experience taking online educational courses? Yes No
16. If yes, how effective was/were the course(s) in delivering the content?
a. Very effective
b. Moderately effective
c. Neither effective or ineffective
d. Moderately ineffective
e. Very ineffective

99

17. If yes to question 14, why was the course effective or ineffective?
a. Fill-in answer

100

Appendix B
Interview Guide – Developers and Mentors
1. In general what do you perceive are the main objectives of Module 1?
2. What skills (knowledge) should the coach have following Module 1?
3. If Module 1 is successful, how will the coaches be different than they were before
completing Module 1?
4. In general what do you perceive are the main objectives of Module 2?
5. What skills (knowledge) should the coach have following Module 2?
6. If Module 2 is successful, how will the coaches be different than they were before
completing Module 2?
7. How will the coaches behave differently based on what they learned from Modules 1 and
2?
8. Which components of the online Modules do you think will be the most successful in
achieving the goals of Modules 1 and 2?
a. For example:
i. PowerPoint‘s with voiceover
ii. Video of other coaches discussing topics
iii. Exercises in the coaches workbook
iv. Video of Experts discussing topics
9. Which aspects of the online Modules do you think will be the least successful in
achieving the goals of Modules 1 and 2?
a. For example:
i. PowerPoint‘s with voiceover

101

ii. Video of other coaches discussing topics
iii. Exercises in the coaches workbook
iv. Video of Experts discussing topics
10. In your experience as a coach (or coach educator) what do you believe are the most
successful methods of teaching coaches?

Interview Guide – Coaches
(Questions in italics represent an additional question from the
Interview Guide for Stakeholders and Mentors)
1. In general what do you perceive are the main objectives of Module 1?
2. What skills (knowledge) should the coach have following Module 1?
3. If Module 1 is successful, how will coaches be different than they were before
completing Module 1?
4. What did you like about Module 1?
5. What did you dislike about Module 1?
6. Have you used any of the material in module 1?
a. If yes, how so?
b. If no, why not?
7. In general what do you perceive are the main objectives of Module 2?
8. What skills (knowledge) should the coach have following Module 2?
9. If Module 2 is successful, how will the coaches be different than they were before
completing Module 2?
10. What did you like about Module 2?

102

11. What did you dislike about Module 2?
12. Have you used any of the material in module 2?
a. If yes, how so?
b. If no, why not?
13. How will the coaches behave differently based on what they learned from Modules 1 and
2?
14. Which components of the online Modules do you think will be the most successful in
achieving the goals of Modules 1 and 2?
b. For example:
i. PowerPoint‘s with voiceover
ii. Video of other coaches discussing topics
iii. Exercises in the coaches workbook
iv. Video of Experts discussing topics
15. Which aspects of the online Modules do you think will be the least successful in
achieving the goals of Modules 1 and 2?
a. For example:
i. PowerPoint‘s with voiceover
ii. Video of other coaches discussing topics
iii. Exercises in the coaches workbook
iv. Video of Experts discussing topics
16. Would you recommend the course to your peers? If no, why? If yes, why?
17. In your experience as a coach what do you believe are the most successful methods of
teaching coaches?

103

Appendix C – Survey 1
Phase Two Participants Demographic Questionnaire
1. Name (Last, First): ______________________________________
2. Today‘s Date: __________
3. Age: __________
4. NCAA Division:
5. Number of years coaching: _____________
6. Coaching level
a. Head Coach
b. Assistant Coach
7. If applicable, total number of years as a Head Coach: _______________
8. If applicable, total number of years as an Assistant Coach: _____________
9. Do you hold any administrative duties at your university in addition to your coaching
responsibilities?
a. Yes - If yes, what are they? ________________________________________
b. No
10. Circle your highest degree earned:
a. High School Diploma
b. Junior College Degree (i.e. A.A.)
c. B.A. / B.S.
d. M.A. / M.S.
e. Ph.D/EDD
f. Other _________________

104

11. If applicable, what area of study did you earn in your undergraduate degree:
_____________________________
12. Have you obtained any coaching certifications? Yes

No

a. If yes, please specify: ______________________________________
13. Have you ever taken any formal leadership courses? Yes No
a. If yes, please specify: ______________________________________
14. Do you have any experience taking online educational courses? Yes No
15. If yes, how effective was/were the course(s) in delivering the content?
a. Very effective
b. Moderately effective
c. Neither effective or ineffective
d. Moderately ineffective
e. Very ineffective
16. If yes to question 14, why was the course effective or ineffective?
a. Fill-in answer
17. Do you feel that your wrestling program is at risk for being downsized or even eliminated
by your athletic department?
a. Yes
b. No
18. Given all of the duties and responsibilities you have as a coach, how much time do you
have available to work on your own leadership development?
a. Less than 1 hour per week
b. 1-2 hours per week

105

c. 2-3 hours per week
d. 3-4 hours per week
e. More than 4 hours per week

106

Appendix D – Survey 2

Effectiveness of NWCA Leadership Academy Modules 1 and 2
Directions: Please answer the following questions based on your experience as a coach. Circle the
appropriate number to the right of each statement.
Please note, that there are NO right or wrong answers and that your responses are confidential.
TO WHAT DEGREE…..
Not
at all

Module 1 Questions
Some

A Lot

1.

Is the sport of college wrestling at
risk for elimination?

1

2

3

4

5

2.

Do you as a coach feel the need to
take action to help college wrestling?

1

2

3

4

5

3.

Do you have a plan to improve your
program in general?

1

2

3

4

5

4.

Do you have a plan to improve your
leadership abilities?

1

2

3

4

5

5.

Do you intend to formulate a plan to
improve your program in general?

1

2

3

4

5

6.

Do you intend to formulate a plan to
improve your leadership abilities?

1

2

3

4

5

7.

Do you intend to formulate a vision
for your program?

1

2

3

4

5

Please rate your degree of “knowledge”
about each of the following
Very
Low

Very
High

8.

Factors that are threats to college
wrestling

1

2

3

4

5

9.

Strategies to overcome the threats to
college wrestling

1

2

3

4

5

10. Contemporary strategies for building
bridges to supporters

1

2

3

4

5

11. Athletic Directors perspectives of the
ideal college wrestling program

1

2

3

4

5

107

12. The purpose of having a vision or
plan for a program

1

2

3

4

5

13. The methods of how to plan and
develop a vision

1

2

3

4

5

14. The methods of how to keep a vision
alive

1

2

3

4

5

MODULE 2 QUESTIONS
TO WHAT DEGREE…..
Not
at all

Some

A Lot

15. Do you understand that the college
coach of today is akin to a CEO?

1

2

3

4

5

16. Do you believe that marketing is an
essential skill of a college coach?

1

2

3

4

5

17. Do you believe that fundraising is an
essential skill of a college coach?

1

2

3

4

5

18. Do you believe that budget
management is an essential skill of a
college coach?

1

2

3

4

5

19. Do you believe that public relations
is an essential skill of a college
coach?

1

2

3

4

5

1

2

3

4

5

1

2

3

4

5

22. Do you believe that recruiting is an
essential skill of a college coach?

1

2

3

4

5

23. Do you believe that community
service is an essential skill of a
college coach?

1

2

3

4

5

1

2

3

4

5

1

2

3

4

5

1

2

3

4

5

20. Do you believe that academic
support is an essential skill of a
college coach?
21. Do you believe that communication
is an essential skill of a college
coach?

24. Do you believe that community
relations is an essential skill of a
college coach?
25. Do you intend to further develop
your marketing skills?
26. Do you intend to further develop
your fundraising skills?

108

1

2

3

4

5

1

2

3

4

5

1

2

3

4

5

1

2

3

4

5

1

2

3

4

5

1

2

3

4

5

1

2

3

4

5

1

2

3

4

5

1

2

3

4

5

1

2

3

4

5

1

2

3

4

5

1

2

3

4

5

1

2

3

4

5

1

2

3

4

5

41. Do you agree that leadership is about
letting others decide how to make a
vision happen?

1

2

3

4

5

42. Do you agree that leadership is about
inspiring others to take
responsibility?

1

2

3

4

5

27. Do you intend to further develop
your budget management skills?
28. Do you intend to further develop
your public relations skills?
29. Do you intend to further develop
your academic support skills?
30. Do you intend to further develop
your communication skills?
31. Do you intend to further develop
your recruiting skills?
32. Do you intend to further develop
your community service skills?
33. Do you intend to further develop
your community relations skills?
34. Do you agree that leadership is about
being the most charismatic hero out
in front?
35. Do you agree that leadership is about
wielding unilateral power?
36. Do you agree that leadership is about
solving everyone‘s problems?
37. Do you agree that leadership is about
providing all of the answers for
others?
38. Do you agree that emphasizing
collaboration as a leader means the
follower or members have to stifle
their competitive urges?
39. Do you agree that leadership is about
defining an appropriate and
compelling vision for your program?
40. Do you agree that leadership is about
getting things done via other people?

109

43. Do you agree that leadership is about
creating an environment where
people can develop the skills needed
to be successful?
44. Do you agree that leadership is about
mobilizing people to face new
challenges that require new values,
priorities, and habits and ways of
doing things?
45. Do you agree that leadership is about
helping others learn how to solve
problems and develop solutions for
themselves?
46. Do you know what the GROW
model represents?
47. Do you intend to use the GROW
model in the future?
48. Have you engaged in a formal (i.e.
asking other individuals, using a 360degree tool) evaluation of your
strengths and weaknesses?
49. Do you intend to engage in a formal
(i.e. asking others individuals, using
a 360-degree tool) evaluation of your
strengths and weaknesses?
50. Do you believe it‘s important to hire
assistant coaches (and/or team aides)
whose strengths are your
weaknesses?
51. Have you hired assistant coaches
(and/or team aides) whose strengths
are your weaknesses?
52. Do you recognize the impact your
behavior (and your programs
behavior) has on other programs
within the sport of college wrestling?

1

2

3

4

5

1

2

3

4

5

1

2

3

4

5

1

2

3

4

5

1

2

3

4

5

1

2

3

4

5

1

2

3

4

5

1

2

3

4

5

1

2

3

4

5

1

2

3

4

5

Please rate your degree of “knowledge”
about each of the following
Very
Low

Very
High

53. Why or how a College Coach is akin
to a corporate CEO

1

2

3

4

5

54. Common characteristics of effective
leaders

1

2

3

4

5

55. What external factors impact or
influence leadership

1

2

3

4

5

110

56. How modern leadership has changed
from years past

1

2

3

4

5

57. Common myths of leadership

1

2

3

4

5

58. The components set forth by Rainer
Martens that effective coach leaders
do

1

2

3

4

5

59. The components that lead to the
development of a team culture

1

2

3

4

5

60. Jack Welch‘s 4 E‘s for Effective
Leadership

1

2

3

4

5

61. Characteristics of coaches that are
great communicators

1

2

3

4

5

62. Methods on how to build trust

1

2

3

4

5

63. The components of the GROW
model

1

2

3

4

5

64. Methods on how to use and
implement the GROW model

1

2

3

4

5

111

Appendix E – Survey 3

Effectiveness of NWCA Leadership Academy Modules 1 and 2
Directions: Please answer the following questions based on your experience as a coach. Circle the
appropriate number to the right of each statement.
TO WHAT DEGREE…..
Not
at all

Module 1 Questions
Some

A Lot

1.

Is the sport of college wrestling at
risk for elimination?

1

2

3

4

5

2.

Do you as a coach feel the need to
take action to help college wrestling?

1

2

3

4

5

3.

Do you have a plan to improve your
program in general?

1

2

3

4

5

4.

Do you have a plan to improve your
leadership abilities?

1

2

3

4

5

5.

Do you intend to formulate a plan to
improve your program in general?

1

2

3

4

5

6.

Do you intend to formulate a plan to
improve your leadership abilities?

1

2

3

4

5

7.

Do you intend to formulate a vision
for your program?

1

2

3

4

5

Please rate your degree of “knowledge”
about each of the following
Very
Low

Very
High

8.

Factors that are threats to college
wrestling

1

2

3

4

5

9.

Strategies to overcome the threats to
college wrestling

1

2

3

4

5

10. Contemporary strategies for building
bridges to supporters

1

2

3

4

5

11. Athletic Directors perspectives of the
ideal college wrestling program

1

2

3

4

5

112

12. The purpose of having a vision or
plan for a program

1

2

3

4

5

13. The methods of how to plan and
develop a vision

1

2

3

4

5

14. The methods of how to keep a vision
alive

1

2

3

4

5

MODULE 2 QUESTIONS
TO WHAT DEGREE…..
Not
at all

Some

A Lot

15. Do you understand that the college
coach of today is akin to a CEO?

1

2

3

4

5

16. Do you believe that marketing is an
essential skill of a college coach?

1

2

3

4

5

17. Do you believe that fundraising is an
essential skill of a college coach?

1

2

3

4

5

18. Do you believe that budget
management is an essential skill of a
college coach?

1

2

3

4

5

19. Do you believe that public relations
is an essential skill of a college
coach?

1

2

3

4

5

1

2

3

4

5

1

2

3

4

5

22. Do you believe that recruiting is an
essential skill of a college coach?

1

2

3

4

5

23. Do you believe that community
service is an essential skill of a
college coach?

1

2

3

4

5

1

2

3

4

5

1

2

3

4

5

1

2

3

4

5

20. Do you believe that academic
support is an essential skill of a
college coach?
21. Do you believe that communication
is an essential skill of a college
coach?

24. Do you believe that community
relations is an essential skill of a
college coach?
25. Do you intend to further develop
your marketing skills?
26. Do you intend to further develop
your fundraising skills?

113

1

2

3

4

5

1

2

3

4

5

1

2

3

4

5

1

2

3

4

5

1

2

3

4

5

1

2

3

4

5

1

2

3

4

5

1

2

3

4

5

1

2

3

4

5

1

2

3

4

5

1

2

3

4

5

1

2

3

4

5

1

2

3

4

5

1

2

3

4

5

41. Do you agree that leadership is about
letting others decide how to make a
vision happen?

1

2

3

4

5

42. Do you agree that leadership is about
inspiring others to take
responsibility?

1

2

3

4

5

27. Do you intend to further develop
your budget management skills?
28. Do you intend to further develop
your public relations skills?
29. Do you intend to further develop
your academic support skills?
30. Do you intend to further develop
your communication skills?
31. Do you intend to further develop
your recruiting skills?
32. Do you intend to further develop
your community service skills?
33. Do you intend to further develop
your community relations skills?
34. Do you agree that leadership is about
being the most charismatic hero out
in front?
35. Do you agree that leadership is about
wielding unilateral power?
36. Do you agree that leadership is about
solving everyone‘s problems?
37. Do you agree that leadership is about
providing all of the answers for
others?
38. Do you agree that emphasizing
collaboration as a leader means the
follower or members have to stifle
their competitive urges?
39. Do you agree that leadership is about
defining an appropriate and
compelling vision for your program?
40. Do you agree that leadership is about
getting things done via other people?

114

43. Do you agree that leadership is about
creating an environment where
people can develop the skills needed
to be successful?
44. Do you agree that leadership is about
mobilizing people to face new
challenges that require new values,
priorities, and habits and ways of
doing things?
45. Do you agree that leadership is about
helping others learn how to solve
problems and develop solutions for
themselves?

1

2

3

4

5

1

2

3

4

5

1

2

3

4

5

1

2

3

4

5

1

2

3

4

5

48. Do you intend to use the GROW
model with your wrestlers

1

2

3

4

5

49. Do you intend to use the GROW
model with your coaching staff

1

2

3

4

5

50. Do you intend to use the GROW
model with individuals other than
your coaching staff or wrestlers

1

2

3

4

5

1

2

3

4

5

1

2

3

4

5

1

2

3

4

5

1

2

3

4

5

1

2

3

4

5

1

2

3

4

5

1

2

3

4

5

46. Do you know what the GROW
model represents?
47. Do you intend to use the GROW
model in the future?

51. Do you intend to implement Jack
Welchs‘ 4 E‘s for effective
leadership
52. Do you intend to implement the
components set forth by Rainer
Marten‘s on being an effective coach
leader
53. Do you intend to implement the
components that lead to developing a
team culture
54. Do you intend to implement methods
on how to build trust
55. Have you engaged in a formal (i.e.
asking other individuals, using a 360degree tool) evaluation of your
strengths and weaknesses?
56. Do you intend to engage in a formal
(i.e. asking others individuals, using
a 360-degree tool) evaluation of your
strengths and weaknesses?
57. Do you believe it‘s important to hire
assistant coaches (and/or team aides)
whose strengths are your
weaknesses?

115

58. Have you hired assistant coaches
(and/or team aides) whose strengths
are your weaknesses?
59. Do you recognize the impact your
behavior (and your programs
behavior) has on other programs
within the sport of college wrestling?

1

2

3

4

5

1

2

3

4

5

Please rate your degree of “knowledge”
about each of the following
Very
Low

Very
High

60. Why or how a College Coach is akin
to a corporate CEO

1

2

3

4

5

61. Common characteristics of effective
leaders

1

2

3

4

5

62. What external factors impact or
influence leadership

1

2

3

4

5

63. How modern leadership has changed
from years past

1

2

3

4

5

64. Common myths of leadership

1

2

3

4

5

65. The components set forth by Rainer
Martens that effective coach leaders
do

1

2

3

4

5

66. The components that lead to the
development of a team culture

1

2

3

4

5

67. Jack Welch‘s 4 E‘s for Effective
Leadership

1

2

3

4

5

68. Characteristics of coaches that are
great communicators

1

2

3

4

5

69. Methods on how to build trust

1

2

3

4

5

70. The components of the GROW
model

1

2

3

4

5

71. Methods on how to use and
implement the GROW model

1

2

3

4

5

116

Appendix F – Survey 4

Follow-up Survey for Intervention Group
1. Which aspects of the online modules were most impactful? (Select all that apply)
a. PowerPoint‘s with voiceover
b. Video of other coaches discussing topics
c. Exercises in the coaches workbook
d. Video of Experts discussing topics
e. Other – Insert answer
f. All are equally impactful
2. Which aspects of the online modules were least impactful? (Select all that apply)
a. PowerPoint‘s with voiceover
b. Video of other coaches discussing topics
c. Exercises in the coaches workbook
d. Video of Experts discussing topcis
e. Other – Insert answer
f. All are equally impactful
3. Would you prefer an alternative format?
a. Yes
b. No
4. If yes, what type of format would you prefer
a. Fill-In answer
5. Did you like the format of having a coach‘s workbook to coincide with the video?
a. Yes
b. No
117

6. If no, why not?
a. Fill-In
7. Would you recommend this course to other coaches?
a. Yes
b. No
8. If no, why not?
a. Fill-in
9. A number of recommendations for personal and program improvement were included in
these two modules. To what degree do you believe that you are capable (e.g. have the
resources, personnel, and time) of implementing such recommendations within your
program?
a. Very capable
b. Moderately capable
c. Neither capable or incapable
d. Moderately incapable
e. Very incapable

118

Appendix G – Survey 5

Retention and Use of Information in NWCA Leadership Academy Modules 1 and 2
Directions: Please answer the following questions based on your experience as a coach. Circle the
appropriate number to the right of each statement.
TO WHAT DEGREE…..
Not
at all

Module 1 Questions
Some

A Lot

1.

**Is the sport of college wrestling at
risk for elimination?

1

2

3

4

5

2.

**Do you as a coach feel the need to
take action to help college wrestling?

1

2

3

4

5

3.

**Do you have a plan to improve
your program in general?

1

2

3

4

5

4.

**Do you have a plan to improve
your leadership abilities?

1

2

3

4

5

5.

Do you intend to formulate a plan to
improve your program in general?

1

2

3

4

5

6.

Do you intend to formulate a plan to
improve your leadership abilities?

1

2

3

4

5

7.

Do you intend to formulate a vision
for your program?

1

2

3

4

5

8.

Have you developed a plan to
improve your program

1

2

3

4

5

9.

Have you developed a plan to
improve your vision

1

2

3

4

5

10. Have you written out a formal vision
for your program

1

2

3

4

5

11. Have you shared your vision for your
program

1

2

3

4

5

Please rate your degree of “knowledge”
about each of the following
Very
Low

119

Very
High

12. Threats to college wrestling

1

2

3

4

5

13. Strategies to overcome the threats to
college wrestling

1

2

3

4

5

14. Contemporary strategies for building
bridges to supporters

1

2

3

4

5

15. Athletic Directors perspectives of the
ideal college wrestling program

1

2

3

4

5

16. The purpose of having a vision or
plan for a program

1

2

3

4

5

17. The methods of how to plan and
develop a vision

1

2

3

4

5

18. The methods of how to keep a vision
alive

1

2

3

4

5

MODULE 2 QUESTIONS
TO WHAT DEGREE…..
Not
at all

Some

A Lot

19. **Do you understand that the college
coach of today is akin to a CEO?

1

2

3

4

5

20. **Do you believe that marketing is
an essential skill of a college coach?

1

2

3

4

5

21. **Do you believe that fundraising is
an essential skill of a college coach?

1

2

3

4

5

22. **Do you believe that budget
management is an essential skill of a
college coach?

1

2

3

4

5

23. **Do you believe that public
relations is an essential skill of a
college coach?

1

2

3

4

5

1

2

3

4

5

1

2

3

4

5

1

2

3

4

5

24. **Do you believe that academic
support is an essential skill of a
college coach?
25. **Do you believe that
communication is an essential skill of
a college coach?
26. **Do you believe that recruiting is
an essential skill of a college coach?

120

27. **Do you believe that community
service is an essential skill of a
college coach?
28. **Do you believe that community
relations is an essential skill of a
college coach?
29. Do you intend to further develop
your marketing skills?
30. Do you intend to further develop
your fundraising skills?
31. Do you intend to further develop
your budget management skills?
32. Do you intend to further develop
your public relations skills?
33. Do you intend to further develop
your academic support skills?
34. Do you intend to further develop
your communication skills?
35. Do you intend to further develop
your recruiting skills?
36. Do you intend to further develop
your community service skills?
37. Do you intend to further develop
your community relations skills?
38. **Do you agree that leadership is
about being the most charismatic
hero out in front?
39. **Do you agree that leadership is
about wielding unilateral power?
40. **Do you agree that leadership is
about solving everyone‘s problems?
41. **Do you agree that leadership is
about providing all of the answers for
others?
42. **Do you agree that emphasizing
collaboration as a leader means the
follower or members have to stifle
their competitive urges?

1

2

3

4

5

1

2

3

4

5

1

2

3

4

5

1

2

3

4

5

1

2

3

4

5

1

2

3

4

5

1

2

3

4

5

1

2

3

4

5

1

2

3

4

5

1

2

3

4

5

1

2

3

4

5

1

2

3

4

5

1

2

3

4

5

1

2

3

4

5

1

2

3

4

5

1

2

3

4

5

121

43. **Do you agree that leadership is
about defining an appropriate and
compelling vision for your program?

1

2

3

4

5

44. **Do you agree that leadership is
about getting things done via other
people?

1

2

3

4

5

45. **Do you agree that leadership is
about letting others decide how to
make a vision happen?

1

2

3

4

5

46. **Do you agree that leadership is
about inspiring others to take
responsibility?
47. **Do you agree that leadership is
about creating an environment where
people can develop the skills needed
to be successful?
48. **Do you agree that leadership is
about mobilizing people to face new
challenges that require new values,
priorities, and habits and ways of
doing things?
49. **Do you agree that leadership is
about helping others learn how to
solve problems and develop solutions
for themselves?

1

2

3

4

5

1

2

3

4

5

1

2

3

4

5

1

2

3

4

5

1

2

3

4

5

1

2

3

4

5

52. Have you used the GROW model
with your wrestlers

1

2

3

4

5

53. Have you used the GROW model
with your coaching staff

1

2

3

4

5

54. Have you used the GROW model
with individuals other than your
coaching staff or wrestlers

1

2

3

4

5

55. Have you implemented Jack Welchs‘
4 E‘s for effective leadership

1

2

3

4

5

1

2

3

4

5

1

2

3

4

5

50. Do you know what the GROW
model represents?
51. Do you intend to use the GROW
model in the future?

56. Have you implemented the
components set forth by Rainer
Marten‘s on being an effective coach
leader
57. Have you implemented the
components that lead to developing a
team culture

122

58. Have you implemented methods on
how to build trust
59. **Have you engaged in a formal (i.e.
asking other individuals, using a 360degree tool) evaluation of your
strengths and weaknesses?
60. **Do you intend to engage in a
formal (i.e. asking others individuals,
using a 360-degree tool) evaluation
of your strengths and weaknesses?
61. **Do you believe it‘s important to
hire assistant coaches (and/or team
aides) whose strengths are your
weaknesses?
62. **Have you hired assistant coaches
(and/or team aides) whose strengths
are your weaknesses?
63. **Do you recognize the impact your
behavior (and your programs
behavior) has on other programs
within the sport of college wrestling?

1

2

3

4

5

1

2

3

4

5

1

2

3

4

5

1

2

3

4

5

1

2

3

4

5

1

2

3

4

5

Please rate your degree of “knowledge”
about each of the following
Very
Low

Very
High

64. **Why or how a College Coach is
akin to a corporate CEO

1

2

3

4

5

65. **Common characteristics of
effective leaders

1

2

3

4

5

66. **What external factors impact or
influence leadership

1

2

3

4

5

67. **How modern leadership has
changed from years past

1

2

3

4

5

68. **Common myths of leadership

1

2

3

4

5

69. The components set forth by Rainer
Martens that effective coach leaders
do

1

2

3

4

5

70. The components that lead to the
development of a team culture

1

2

3

4

5

123

71. Jack Welch‘s 4 E‘s for Effective
Leadership

1

2

3

4

5

72. Characteristics of coaches that are
great communicators

1

2

3

4

5

73. Methods on how to build trust

1

2

3

4

5

74. The components of the GROW
model

1

2

3

4

5

75. Methods on how to use and
implement the GROW model

1

2

3

4

5

124

Appendix H

Control Group Recruitment Email
Dear Coach,

Mike Moyer and the National Wrestling Coaches Association are requesting your help.

College wrestling is under attack. In these difficult economic times an increasing number
of programs are being dropped. In response to this state of affairs one thing the NWCA is doing
is developing a state of the art leadership academy for collegiate wrestling coaches to provide
coaches with critical information and the support needed to thrive. The NWCA has leaned on a
significant number of successful businesspersons, athletic directors, and coaches to aid in the
development of this program and it is now being initiated. So please help!!
The development of any program requires in-depth evaluation regarding its effectiveness
for its intended users. Therefore, the NWCA has embarked on evaluating a portion of the
Leadership Academy and are asking you to help in this evaluation process as a control group.
Our goal is to compare coaches like yourself who have not yet gone through the program with
coaches going through the program to see what information conveyed in the program helps
participants develop the greatest. This is critical because we know college wrestling coaches
already possess a number of coaching leadership skills and competencies and we want to make
sure what is presented in the program builds off these strengths and does not simply tell
coaches what they already know.
Your involvement would simply require you to complete two surveys that would only
take 15-20 minutes each to complete. The surveys will ask you about your views of coaching
and opinions of leadership and be completed about a month apart. A link to each survey will be
125

emailed to you and can be completed from any computer with Internet access. The information
collected from these surveys is completely confidential. Furthermore, the information gathered
from these surveys will be used to help educate those involved in the leadership academy to
better suit the leadership needs of intercollegiate coaches. Therefore, your participation is
really needed and greatly appreciated. Following completion of the surveys you will also be
offered the opportunity to complete the online portion of the Leadership Academy at no charge.
Please see the attached consent form for more detailed information about participation. If
you are interested in participating please click the link below and fill out the first survey. By
clicking below you are consenting to participate.
We realize how busy coaches like yourself are and that we are asking you to do one more
thing in a list of things you must do. However, getting information from coaches not yet in the
program is absolutely essential if we are to build a program that meets the needs of today‘s
college wrestling coach. SO PLEASE HELP!

Here is a link to the survey. CLICK HERE. Your survey ID is 20043

Thank you,
Eric Bean and Mike Moyer

126

Appendix I
Participants by Time
Group

Time 1

Time 2

Time 3

Control

35

27

23

Intervention

13

4

n/a

127

Appendix J
Subscales items at each time point
Knowledge Time 1
and Time 2

Knowledge Time 3
(KnowledgeSmall)

Intention Time 1 and
2

Intention Time 3
(IntentionSmall)

Q8: Knowledge of
threats

Q8: Knowledge of
threats

Q5: To what degree
do you intend to
formulate a plan to
improve your
program

Q9: Knowledge of
strategies to overcome
threats

Q9: Knowledge of
strategies to
overcome threats

Q6: To what degree
do you intend to
improve your
leadership abilities

Q6: To what
degree do you
intend to improve
your leadership
abilities

Q10: Knowledge of
strategies to build
bridges to supporters

Q10: Knowledge of
strategies to build
bridges to supporters

Q7: To what degree
do you intend to
formulate a vision for
your program

Q7: To what
degree do you
intend to formulate
a vision for your
program

Q11: Knowledge of
AD‘s perspectives on
the ideal college
program

Q11: Knowledge of
AD‘s perspectives
on the ideal college
program

Q25: Do you intend
to further develop
your marketing skills

Q25: Do you
intend to further
develop your
marketing skills

Q12: Knowledge of
the purpose of having
a vision or plan for a
program

Q12: Knowledge of
the purpose of
having a vision or
plan for a program

Q26: Do you intend
to further develop
your fundraising
skills

Q26: Do you
intend to further
develop your
fundraising skills

Q13: Knowledge of
how to develop a plan
or a vision

Q13: Knowledge of
how to develop a
plan or a vision

Q27: Do you intend
to further develop
your budget
management skills

Q27: Do you
intend to further
develop your
budget
management skills

Q14: Knowledge of
the methods of how to
keep a vision alive

Q14: Knowledge of
the methods of how
to keep a vision
alive

Q28: Do you intend
to further develop
your public relations
skills

Q28: Do you
intend to further
develop your
public relations
skills

128

Q5: To what
degree do you
intend to formulate
a plan to improve
your program

Knowledge Time 1
and Time 2

Knowledge Time 3
(KnowledgeSmall)

Intention Time 1 and
2

Intention Time 3
(IntentionSmall)

Q46: Do you know
what the GROW
model represents

Q29: Do you intend
to further develop
your academic
support skills

Q29: Do you
intend to further
develop your
academic support
skills

Q58: Knowledge of
the components set
forth by Rainer
Martens that
effective coach
leaders do

Q30: Do you intend
to further develop
your communication
skills

Q30: Do you
intend to further
develop your
communication
skills

Q59: Knowledge of
the components of a
team culture

Q31: Do you intend
to further develop
your recruiting skills

Q31: Do you
intend to further
develop your
recruiting skills

Q60: Knowledge of
Jack Welch‘s 4 E‘s
for Effective
Leadership

Q32: Intention to
further community
service skills

Q32: Intention to
further community
service skills

Q19: Do you believe
that public relations is
an essential skill of a
college coach

Q61: Characteristics
of great coach
communicators

Q33: Do you intend
to further develop
your community
relations skills

Q33: Do you
intend to further
develop your
community
relations skills

Q20: Do you believe
that academic support
is an essential skill of
a college coach

Q62: Knowledge of
methods on how to
build trust

Q47: Do you intend
to use the GROW
model in the future

Q47: Do you
intend to use the
GROW model in
the future

Q21: Do you believe
that communication is
an essential skill of a
college coach

Q63: Knowledge of
the components of
the GROW model

Q49: Intention to do
a formal evaluation

Q22: Do you believe
that recruiting is an
essential skill of a
college coach

Q64: Knowledge of
methods on how to
use and implement
the GROW model

Q15: Do you
understand that the
college coach of today
is akin to a CEO

Q16: Do you believe
that marketing is an
essential skill of a
college coach
Q17: Do you believe
that fundraising is an
essential skill of a
college coach
Q18: Do you believe
that budget
management is an
essential skill of a
college coach

129

Knowledge Time 1
and Time 2

Knowledge Time 3
(KnowledgeSmall)

Intention Time 1 and
2

Q23: Do you believe
that community
service is an essential
skill of a college
coach
Q24: Do you believe
that community
relations is an
essential skill of a
college coach
Q34: Do you agree
that leadership is
about being the most
charismatic hero out in
front
Q35: Do you agree
that leadership is
about wielding
unilateral power
Q36: Do you agree
that leadership is
about solving
everyone‘s problems
Q37: Do you agree
that leadership is
about providing all of
the answers for others
Q38: Do you agree
that emphasizing
collaboration as a
leader means the
follower or members
have to stifle their
competitive urges

130

Intention Time 3
(IntentionSmall)

Knowledge Time 1
and Time 2

Knowledge Time 3
(KnowledgeSmall)

Intention Time 1 and
2

Q39: Do you agree
that leadership is
about defining an
appropriate and
compelling vision for
your program
Q40: Do you agree
that leadership is
about getting things
done via other people
Q41: Do you agree
that leadership is
about letting others
decide how to make a
vision happen
Q42: Do you agree
that leadership is
about inspiring others
to take responsibility
Q43: Do you agree
that leadership is
about creating an
environment where
people can develop the
skills needed to be
successful
Q44: Do you agree
that leadership is
about mobilizing
people to face new
challenges that require
new values, priorities,
and habits and ways of
doing things

131

Intention Time 3
(IntentionSmall)

Knowledge Time 1
and Time 2

Knowledge Time 3
(KnowledgeSmall)

Intention Time 1 and
2

Q45: Do you agree
that leadership is
about helping others
learn how to solve
problems and develop
solutions for
themselves
Q46: Do you know
what the GROW
model represents
Q53: Knowledge of
why or how a college
coach is akin to a CEO
Q54: Knowledge of
common
characteristics of
effective leaders
Q55: Knowledge of
what external factors
impact of influence
leadership
Q56: Knowledge of
how modern
leadership has
changed from years
past
Q57: Knowledge of
common myths of
leadership
Q58: Knowledge of
the components set
forth by Rainer
Martens that effective
coach leaders do
Q59: Knowledge of
the components of a
team culture
132

Intention Time 3
(IntentionSmall)

Knowledge Time 1
and Time 2

Knowledge Time 3
(KnowledgeSmall)

Intention Time 1 and
2

Q60: Knowledge of
Jack Welch‘s 4 E‘s for
Effective Leadership
Q61: Characteristics
of great coach
communicators
Q62: Knowledge of
methods on how to
build trust
Q63: Knowledge of
the components of the
GROW model
Q64: Knowledge of
methods on how to
use and implement the
GROW model

133

Intention Time 3
(IntentionSmall)

REFERENCES

134

REFERENCES
Acker, J. V. (2008). Lawrence to suspend wrestling program. Lawrence University Sports News.
Retrieved December 9, 2009, from http://www.lawrence.edu/dept/communications
/media/sportsnews/0809/121208.html
American Sport Education Program. About (2009). Retrieved October 29, 2009,
from http://www.asep.com/about.cfm
Alkin, M. (2004). Comparing evaluation points of view. In M. Alkin (Ed.) Evaluation roots:
Tracing theorists’ views and influences (pp. 3 – 65). Thousand Oaks, CA: Sage Publications
Alkin, M. & Christie, C. (2004). Evaluation theory tree revisited. In M. Alkin (Ed.) Evaluation
roots: Tracing theorists’ views and influences (pp. 381 – 392). Thousand Oaks, CA: Sage
Publications
Avolio, B. J., & Hannah, S. T. (2008). Developmental readiness: Accelerating leader
development. Consulting Psychology Journal: Practice and Research, 60(4), 331-347.
Barber, H., & Eckrich, J. (1998). Methods and criteria employed in the evaluation of
intercollegiate coaches. Journal of Sport Management, 12, 301-322.
Barron, M. J. (2005). Evaluation of an injury prevention program (Doctoral dissertation,
Michigan State University, East Lansing, 2001).
Bennis, W. (2007). The challenges of leadership in the modern world: Introduction to the special
issue. American Psychologist, 62(1), 2-5.
Bernard, R. M., Abrami, P. C., Lou, Y., Borokhovski, E., Wade, A., Wozney, L., et al. (2004).
How does distance education compare with classroom instruction? A meta-analysis of the
empirical literature. Review of Educational Research, 74(3), 379-439.
Brown, A. (2009). Delaware State dropping wrestling. Retrieved December 9, 2009 from
http://wrestlingreport.com/current_news/viewtopic.php?f=16&t=14190
Carver, C. S., & Scheier, M. F. (1982). Control theory: A useful conceptual framework for
personality-social, clinical, and health psychology. Psychological Bulletin, 92, 111–135.
Case, B. (1987). Leadership behavior in sport: A field test of the situational leadership theory.
International Journal of Sport Psychology, 18(4), 256-268.
Chelladurai, P. (1990). Leadership in sports: A review. International Journal of Sport
Psychology, 21, 328-354.
Chelladurai, P. (2007). Leadership in sports. In G. Tenebaum, & R.C. Eklund (Eds.) Handbook
of sport psychology (3rd edition). Hoboken, NJ: John Wiley and Sons.
135

Chelladurai, P. & Arnott, M. (1985). Decision styles in coaching: Preferences of basketball
players. Research Quarterly, 56, 15-24.
Chelladurai, P. & Saleh, S. D. (1980). Preferred leadership in sports. Canadian Journal of
Applied Sport Sciences, 3, 85-92.
Cohn, J. M., Khruana, R., & Reeves, L. (2005). Growing talent as if your business depended on
it. (Cover story). Harvard Business Review, 83(10), 62-70.
Coyner, S. C., & McCann, P. L. (2004). Advantages and challenges of teaching in an
electronic environment: The accommodate model. International Journal of Instructional
Media, 31(3), 223-228.
Cunningham, G. B., & Dixon, M. A. (2003). New perspectives concerning performance
appraisals of intercollegiate coaches. Quest, 55, 177-192.
Dunning, D., Heath, C., & Suls, J. M. (2004). Flawed self-assessment: Implications for health
education, and the workplace. Psychological Science in the Public Interest,5, 69-106.
Dwyer, J. M., & Fisher, D. G. (1990). Wrestlers' perceptions of coaches' leadership as
predictors of satisfaction with leadership. Perceptual & Motor Skills, 71(2), 511-517.
Dwyer, J. M., & Fischer, D. G. (1988). Leadership style of wrestling coaches. Perceptual
and Motor Skills, 67, 706.
De Vries, M. K., Florent-Treacy, E., Ramo, L. G., & Korotov, K. (2008). A framework for
the assessment of Leadership Academys. INSEAD Working Papers Collection, (38), 415.
Finckenauer, J. O., Margaryan, S., & Sullivan, M. L. (2005). Evaluability assessment in
juvenile justice: A case example. Youth Violence and Juvenile Justice, 3(3), 265-275.
Gilbert, W., & Trudel, P. (1999). An evaluation strategy for coach education programs.
Journal of Sport Behavior, 22(2), 234-250.
Gould, D., Carson, S., Fifer, A., Lauer, L., & Benham, R. (2009). Stakeholders‘ perceptions
of social-emotional and life skill development issues characterizing contemporary high
school sports. Journal of Coaching Education,2(1).
Gould, D., Petlichkoff, L., Hodge, K., & Simons, J. (1990). Evaluating the effectiveness of a
psychological skills educational workshop. The Sport Psychologist, 4(3).
Griffin, N. S. (2003). Personalize your management development. Harvard Business Review,
81(3), 113-119.
Hackman, J., & Wageman, R. (2007). Asking the right questions about leadership: Discussion
136

and conclusions. American Psychologist, 62, 43-47.
Hansen, D. E. (2008). Knowledge transfer in online learning environments. Journal of Marketing
Education, 30(2), 93-105.
Kets de Vries, M. F. R., Vrignaud, P., & Florent-Treacy, E. (2004). The global leadership life
inventory: Development and psychometric properties of a 360-degree feedback instrument.
International Journal of Human Resource Management, 15(3), 475-492.
Larreamendy-Joerns, J., & Leinhardt, G. (2006). Going the distance with online education.
Review of Educational Research, 76(4), 567 - 605.
Li, C. S., & Irby, B. (2008). An overview of online education: Attractiveness, benefits,
challenges, concerns and recommendations. College Student Journal, 42(2), 449-458
MacLean, J., & Zakrajsek, D. (1996). Factors considered important for evaluating ca nadian
university athletic coaches. Journal of Sport Management, 10, 446-462.
Mallett, C., & Côté, J. (2006). Beyond winning and losing: Guidelines for evaluating high
performance coaches. The Sport Psychologist, 20, 213-221.
Miles, M., & Huberman, A. M. (1984). Qualitative data analysis. Beverly Hills, CA: Sage
Publications.
Miller, T. (2007). Oregon University adds men‘s baseball, drops wrestling. Retrieved
December 9, 2009 from http://standyourground.com/forums/index.php?topic=
13703.0;wap2
Mondello, M. J., & Janelle, C. M. (2001). A comparison of leadership styles of head coaches
and assistant coaches at a successful division I athletic program. International Sports
Journal, 5(2), 40-49.
Mor, N., & Winquist, J. (2002). Self-focused attention and negative affect: A meta-analysis.
Psychological Bulletin, 128, 638 – 662.
National Collegiate Athletic Association. Analysis of 2005-06 division I championships [Data
file]. Retrieved from http://www.ncaa.org/wps/wcm/connect/resources/file/ebca
b70ec47d4b8/Copy%20of%20Div1_2006_MotherSummarySS.pdf?MOD=AJPERES
National Federation of State High School Associations (2009). Retrieved October 29, 2009,
from http://www.nfhslearn.com/index.aspx
Northouse, P. G. (2004). Leadership: Theory and practice. Thousand Oaks, CA: Sage
Patton, M.Q. (1990). Qualitative evaluation and research methods (2nd ed.). Newbury Park,
CA: Sage Publications.
137

Patton, M. Q. (2000). Utilization-focused evaluation. In D. L. Stufflbeam, G. F. Madaus, &
T. Kellaghan (Eds.) Evaluation models (pp.425-438). Boston: Kluwer Publishers.
Patton, M. Q. (2008). Utilization-focused evaluation (4th ed.). Thousand Oaks, California:
Sage.
Petitpas, A. J., Cornelius, A. E., van Raalte, J. L., & Jones, T. (2005). A framework for planning
youth sport programs that foster psychosocial development. The Sport Psychologist,
19(1), 63-80.
Petitpas, A. J., van Raalte, J. L., Cornelius, A. E., & Jones, T. (2004). A life skills development
program for high school student-athletes. The Journal of Primary Prevention, 24(3), 325334.
Prince, S. (2009). Portland State eliminates its wrestling program. Retrieved December 4, 2009
from http://www.oregonlive.com/vikings/index.ssf/2009/03/portland_state_kills_
wrestling.html
Prochaska, J. O., Norcross, J. C. (2001). Stages of change. Psychotherapy: Theory, Research,
Practice, Training, 38(4), 443-448.
Riggio, R. E. (2008). Leadership development: The current state and future expectations.
Consulting Psychology Journal: Practice and Research, 60(4), 383-392.
Rosenbach, W. E., & Taylor, R. L. (2006). Contemporary issues in leadership. Boulder, CO:
Westview Press.
Rossi, P. H., Lipsey, M. W., & Freeman, H. E. (2004). Evaluation: A systematic approach (7,
illustrated ed.). Thousand Oaks, CA: Sage.
Schweigert, F. J. (2006). The meaning of effectiveness in assessing community initiatives.
American Journal of Evaluation, 27(4), 416.
Scriven, M. (1991). Evaluation thesaurus (4 ed.). Newbury Park, CA: Sage.
Singh, P., & Pan, W. (2004). Online education: Lessons for administrators and instructors.
College Student Journal, 38(2), 302-309.
Terry, P., & Howe, B. (1984). The coaching preferences of elite athletes competing at
Universaide ‗83. Canadian Journal of Applied Sport Science, 9, 201-208.
Trapnell, P. D., & Campbell, J. D. (1999). Private self-consciousness and the five-factor model
of personality: Distinguishing rumination from reflection. Journal of Personality and
Social Psychology, 76, 284 –304.
Vroom, V. H., & Jago, A. G. (2007). The role of the situation in leadership. American
138

Psychologist, 62(1), 17-24.
Weinberg, R. S., & Gould, D. (2007). Foundations of sport and exercise psychology.
Champaign, IL.: Human Kinetics.
Weiss, M., Price, M., Bhalla, J., & Bolter, N. (2006). Summary report: “More than a game:”
Longitudinal effects of the First Tee life skills programs on positive youth development.
Retrieved from http://www.cehd.umn.edu/tuckercenter/projects/TheFirstTee/summary_
report_06_final.pdf
Wholey, J. S., Hatry, H. P., & Newcomer, K. E. (2004). Handbook of practical program
evaluation (2, illustrated ed.). San Francisco : John Wiley and Sons.
Witkin, B. R., & Altschuld, J. W. (1995). Planning and Conducting Needs
Assessments: A Practical Guide. Thousand Oaks, CA: Sage.

139