A USE FOCUSED PARTICIPATORY EVALUATION OF THE IMPLEMENTATION, PERCEIVED ORGANIZATIONAL BENEFITS, AND DEVELOPMENTAL POTENTIAL OF A STUDENT ADVISORY COUNCIL By Jedediah Edward Blanton A DISSERTATION Submitted to Michigan State University in partial fulfillment of the requirements for the degree of Kinesiology – Doctor of Philosophy 2013 ABSTRACT A USE FOCUSED PARTICIPATORY EVALUATION OF THE IMPLEMENTATION, PERCEIVED ORGANIZATIONAL BENEFITS, AND DEVELOPMENTAL POTENTIAL OF A STUDENT ADVISORY COUNCIL By Jedediah Edward Blanton The primary motivation for this study was to formatively evaluate the Student-Advisory Council (SAC) for the Michigan High School Athletic Association (MHSAA). Specifically this study was designed to address four purposes: (1) to chronicle and evaluate the tasks and activities (actual and idealized) of the MHSAA SAC; (2) to evaluate the perceived organizational benefits of the SAC to the MHSAA and its executive staff and administration; (3) to investigate the perceived personal development gain or sense of empowerment experienced by the studentathletes; (4) to investigate the experience of being evaluated. These purposes were addressed through a two phase approach. Phase 1 was designed to address purposes 1 and 2. Four adult stakeholders were interviewed about the creation and purpose of the SAC as well as the perceived organizational (MHSAA) benefit of sponsoring the SAC program. Fourteen studentathlete members of the SAC were also interview in two focus groups and asked to explore the same phenomena. Phase 2 was designed to address purposes 3 and 4. Through the second phase, the evaluator interviewed two of the adult stakeholders and all 14 of the focus group participants individually to discuss the findings from Phase 1, the perceived sense of development experienced by the student-athlete members, and the perception of being evaluated. Phase 1 data revealed three goals of the SAC: (1) to gain a student’s perspective on high school sports and present issues facing the MHSAA; (2) to promote the values of the MHSAA, both ‘to’ and ‘through’ students; (3) to provide an environment for student-athlete leaders to become exceptional leaders. Additionally a logic model describing the purpose and functioning of the SAC was created and presented back to the stakeholder. Phase 2 revealed that the SAC experience changed the student-athlete participants through increasing their social skills and confidence in their leadership. Additionally, student-athlete members reported feeling empowered over aspects of the SAC program, while also feeling somewhat undervalued by the MHSAA. The experience of participating in the evaluation was overall a positive and reflective experience. This dissertation is dedicated to the SAC classes of 2012, 2013, and 2014, as well as Andy Frushour, Jack Roberts, and the student services team at the MHSAA. I am an extremely fortunate individual to have found a career that truly speaks to my passion for sport and its opportunities, and then to be able to work in such a way that seeks to illuminate and exemplify those opportunities. I am truly humbled to have found a group of individuals that feels the same way I do, and I can only hope to emulate their work ethic toward making sport a great place for kids to grow and develop into adults. The young people from the three classes of the SAC that I was so fortunate to have encountered in Michigan, represent the best of high school sports and the ‘pure sport’ spirit: passionate, intelligent, driven, curious, helpful, and humble. If I can offer a small token of my appreciation in the form of advice, I encourage you all to remain curious, always work passionately, and never doubt your individual strengths and abilities. Thank you. iv ACKNOWLEDGEMENTS I would like to first extend my gratitude to my committee for their guidance and support during this project. Dr. Lauer, you pushed me to believe in myself during a number of instances of self-doubt and to take risks that I might have otherwise ignored for a safer and more secure option. Those risks have provided great reward and self-growth, and I am deeply appreciative. Dr. Ewing, you have provided a calming and reassuring presence during my time at MSU. I sincerely appreciate your willingness to share in my inquiries, frustrations, and celebrations. Dr. Campbell, you exemplify the research-to-practice model all scholars should embrace. Using the privilege of the University, I will always attempt to serve the greater communities before my academic agenda because of what I have learned from you. I would like to especially thank Dr. Dan Gould, my dissertation chair and academic advisor. Your patience and support have shaped me in my development as a young scholar. Ten years ago, I saw a student studying your textbook at my undergraduate college, and I changed my major specifically to take the sport psychology course. It was a dream come true to be accepted to your graduate program and work alongside you at the Institute for the Study of Youth Sports. I have learned so much about the power of sport and about my own abilities to shape that world for others because of your work ethic, intelligence, and passion to make sport a safe and positive experience for young people. I would lastly like to thank my fiancé, Anna Kirk. Your patience, support, and friendship were incredibly appreciated during this entire graduate experience. Your remarkable intelligence and curiosity forced me to think much harder and more critically about my academic work and how to articulate myself as a scholar. Your passion and love for me has truly taught me about the v value and joy of family and I do not know the words to fully express my excitement about our future together. So simply, I’ll just say thank you and I love you. vi TABLE OF CONTENTS LIST OF TABLES...............................................................................................................x LIST OF FIGURES ........................................................................................................... xi CHAPTER 1 INTRODUCTION ...............................................................................................................1 Athletes’ Gain a Voice in Athletics .........................................................................2 Nature of the Problem..............................................................................................5 Significance of the Study .........................................................................................6 Purposes of the Study ..............................................................................................9 Phase 1 .......................................................................................................10 Phase 2 .......................................................................................................10 Organization of the Dissertation ............................................................................11 CHAPTER 2 REVIEW OF LITERATURE ............................................................................................12 Promoting Positive Youth Development through Sport Programs........................13 Life Skills Needs....................................................................................................16 Life Skills Education Framework ..........................................................................18 Extracurricular Activities in Adolescence .............................................................20 Evaluation as a Scientific Endeavor ......................................................................24 Responsive Evaluation...........................................................................................27 Participatory Evaluation ........................................................................................31 CHAPTER 3 METHOD ..........................................................................................................................35 Description of Investigator ....................................................................................36 Phase 1: Goals, Purpose, Activities, and Organizational Benefits of SAC ...........38 Participants.................................................................................................38 Procedures..................................................................................................39 Adult individual interviews ...........................................................40 SAC member focus groups ............................................................40 Interview guide ..............................................................................42 Stakeholder involvement ...............................................................43 Phase 2: Student Development Potential and Participation in Evaluation ............43 Participants.................................................................................................43 Procedure ...................................................................................................44 Interview guide ..............................................................................45 Data Analysis .........................................................................................................47 Grounded theory ........................................................................................48 Data collection circle .....................................................................48 Analytical procedures ....................................................................50 vii Triangulation of data......................................................................53 CHAPTER 4 RESULTS ..........................................................................................................................54 Phase 1 ...................................................................................................................54 Why does the MHSAA have a SAC? ........................................................55 The goals of the MHSAA SAC .....................................................56 The goal of the MHSAA SAC is to gain a student’s perspective on the high school sport experience in Michigan as well as present issues facing high school sports in Michigan to the ‘ultimate stakeholders’ ......................................................................56 To promote the values of the MHSAA: both ‘to’ and ‘through’ students ..............................................................................57 To offer an environment for great student-athlete leaders to become exceptional leaders ...............................................59 How does the MHSAA benefit from the SAC?.........................................60 Benefits to the general student-athlete...........................................61 The SAC benefits the MHSAA .....................................................63 The SAC benefits the MHSAA personnel .........................64 The SAC benefits the MHSAA brand ...............................65 The purpose and functioning of the MHSAA logic model........................67 Phase 2 ...................................................................................................................70 The SAC changed me ................................................................................70 Gained perspective: “Michigan View” ..........................................73 Stronger social skills ......................................................................74 Increased leadership skills .............................................................76 Gained confidence .............................................................78 Ability to share opinions....................................................79 It empowers me..........................................................................................80 My opinion matters ........................................................................81 Project power .................................................................................83 We benefit other student-athletes ..................................................84 We are the voice for the students.......................................84 Project participation ...........................................................85 ‘It empowers me’ thematic schema summary ...........................................86 Recommendations......................................................................................86 Recommendations: To the MHSAA..............................................87 More: Meetings..................................................................87 More: Executive Director ..................................................88 More: Input ........................................................................89 We are underestimated...................................................................90 Policies and rules ...............................................................91 Falls on deaf ears ...............................................................91 Recommendations: To future SAC members ................................92 Be open and involved ........................................................92 Voice your opinion ............................................................92 viii Have fun! ...........................................................................93 Evaluation Participation.............................................................................94 Positive and reflective....................................................................96 Changes the way you think ............................................................97 Logic model: ‘Hit the nail on the head’ .........................................98 Future use.......................................................................................99 CHAPTER 5 DISCUSSION..................................................................................................................101 Phase 1 .................................................................................................................102 Phase 2 .................................................................................................................106 Phase 2: Exploring empowerment ...........................................................109 Phase 2: Evaluation participation ............................................................113 Implications and Future Directions for the MHSAA SAC..................................114 Implications and Future Directions for Evaluation within Kinesiology .............119 Combining theoretical approaches...........................................................121 Strengths and Limitations ....................................................................................124 APPENDICES Appendix A. Evaluation Agreement....................................................................130 Appendix B. Consent Form – Adults and Administrators...................................133 Appendix C. Consent Form – Students and Parents ............................................137 REFERENCES ................................................................................................................142 ix LIST OF TABLES Table 1. Phase 1 Interview Guides ....................................................................................43 Table 2. Phase 2 Interview Guides ....................................................................................46 x LIST OF FIGURES Figure 1. Why does the MHSAA have a SAC?.................................................................55 Figure 2. How does the MHSAA benefit from the SAC? .................................................61 Figure 3. The Purpose and Functioning of the MHSAA SAC ..........................................68 Figure 4. Phase 2 Thematic Schemas ................................................................................72 Figure 5. Recommendations for the SAC ..........................................................................94 xi CHAPTER 1 INTRODUCTION Evaluation is becoming increasingly important in today’s world. Businesses are interested in determining ways to make themselves more efficient, governments are attempting to ensure wise investments of resources, and educational institutions are more concerned with student learning outcomes. The very premise of evaluative methods serves to inform decision makers and provide a carefully crafted guide toward understanding the broad scope of a given program as well as its more minor nuances that drive the daily programmatic activities. Using social scientific research methods, evaluators can inform program stakeholders on the overall functioning of their program as well as impart a detailed account of the program’s value or merit (Rossi, Lipsey, & Freeman, 2004). It is with evaluation that program directors and participants are most guided toward active reflection, critical thinking, and the co-creation of the shared meaning of their program. Evaluation methodology allows for the most accurate and scientifically valid form of investigating a multitude of programs. Within the field of kinesiology and the practical world of sport programming, few evaluations are ever conducted. This is not to say that a sport program that is left unevaluated is any less deserving of merit than those few who can afford the time and resources to engage in such a process. Anecdotally, many programs seem to function just fine and often engage in practices that closely relate to their goals, and may even have systems in place to monitor progress. However, the rigorous scientific measures provided by a strong evaluation are best suited for the most accurate and theoretically supported investigations into a sport program’s functions, especially when that sport program attempts to offer more than just a simple sport 1 activity (particularly those sport-related programs who claim to develop life skills, leadership, or other personal development assets). This dissertation will consist of a program evaluation that can serve as a model of how programs may be investigated. The central premise of this study is to understand how a sportrelated program functions and by doing so creates a shared meaning of the program amongst varying stakeholders. It will also allow the evaluand to craft more specific goals and better understand related activities of the program for future monitoring and a more summative evaluation. By engaging in theoretically supported evaluation methods, this qualitative investigation will explore and identify the methods by which the program of interest functions and best serves its supporting organization, as well as its youth participants. Athletes’ Gain a Voice in Athletics Over the years, there have been calls to provide athletes with greater voice in the programs they participate in. In 1978 the Amateur Sports Act was passed largely due to the strong lobbying power of the NCAA. This act not only allowed a re-organization of amateur sport, but also a re-conceptualization of the athlete as a person deserving greater voice or power toward their experience in sport (Moore, 1979). Furthermore, through a series of court cases against professional sport organizations, brought on by a rise in player unions, professional athletes began to be viewed as more than a means to an end. Instead they were viewed as employees guaranteed similar rights as those in the corporate or business worlds (Garvey, 1979). These changes are seen as important because they allowed participants to help determine the directions of the programs in which they are most involved. This dramatic re-conceptualization process also helped provide greater justification for providing young athletes a safer environment; to not only participate in sport for skill development and physical activity reasons 2 but to psychosocially thrive, using sport as an arena to develop as a person. These original efforts to provide athletes with better experiences in sport have continued as calls for change and instances of the evolution of the athletes’ voice have grown immensely in professional, amateur, and interscholastic sport. In 1989, for example, the National Collegiate Athletic Association (NCAA) via their governance process formed a Student-Athlete Advisory Committee (SAAC), with the intention of offering student-athlete input on NCAA activities that affected student-athlete welfare (NCAA, 2004). Today, the NCAA has national SAACs for each of its three divisions, as well as requiring each member institution to have a campus SAAC. SAACs are designed to promote communication between athletes and athletic directors, provide feedback and insight into departmental issues, and solicit student-athlete responses to proposed NCAA legislation (NCAA, 2004). Few studies, however, have examined or evaluated the components of such a group. Hendricks (2011) explored stakeholder involvement with regards to campus SAACs, and found that student-athlete members only reported being “very often” involved with community relations, as opposed to being “sometimes” involved with decisions regarding NCAA rules, and “rarely” involved with athletic department policies”. Hence, little is known about whether student-athlete advisory committees are achieving their stated functions, especially in regard to providing participants a true voice at the policy and decision-making levels. Some state level high school athletic associations have since mirrored this NCAA leadership opportunity for student-athletes and created their own variation. In Michigan, the location of this evaluation project, the Michigan High School Athletic Association (MHSAA) refers to their group as the Student Advisory Council (SAC). Organized in 2003, the initial purpose of the SAC was to serve as a sounding board and information exchange between high 3 school students and the leadership of the MHSAA, including the executive director. By connecting with the young people who are engaged in the very sports that the MHSAA sponsors, the overarching premise was to understand issues facing these student-athletes. Athletic directors were solicited to recommend and encourage academically strong, and leadership oriented sophomore students to apply for a position on the counsel, along with a letter of reference. Eight students were selected as the first class, and the next year, they would be joined by eight more, for a total of 16 members. This selection procedure allows for a two-year commitment of SAC student-athletes and, in so doing, provides for a rolling transition between junior and senior classes, and a total council of 16 student-athletes each academic year. The MHSAA maintains a diverse group of representative student SAC members, gathering four boys and four girls, each representing one of the four interstate competitive athletic divisions, determined by student enrollment size. Furthermore, the MHSAA also attempts to invite members from a variety of public and private high schools, as well as from all regions within the state including the Upper Peninsula. In addition to being a representative voice of the student-athlete participants from the MHSAA’s member schools, the SAC representatives are asked to attend award ceremonies at football and basketball state championships, facilitate a session at the bi-annual Sportsmanship Summits, compose small opinion essays and stories for the MHSAA website, and complete a yearly project, targeted at improving the high school sports experience, and supported by the MHSAA. The student members are also invited to an overnight camp to meet one another and become introduced to the overseeing MHSAA staff members. The students attend approximately 6 meetings, lasting about 3.5 hours each, throughout the academic year on Sunday afternoons to 4 accomplish these tasks. Organized and published in 2007, the SAC belief statement opens as follows: “As the voice of Michigan’s student-athletes, the Student Advisory Council’s role is to convey the message of how high school sports are supposed to be played. We are responsible for helping the MHSAA maintain a positive and healthy atmosphere in which interscholastic athletes can thrive.” Unfortunately, as was the case with the NCAA SAAC few efforts have been made to study the MHSAA SAC and its effectiveness or towards composing a statement of purposes to effectively evaluate the merit or perceived organizational benefits of the program. Nature of the Problem Many programs and structured opportunities exist that attempt to offer youth an empowering position within an adult controlled environment. Secondary schools often utilize a student government, colleges typically invite student representatives to serve on various committees and sports teams usually have team captains. Young people become involved in these leadership positions for a number of reasons. First, they can be motivated to improve their programs. Second, such experiences improve their resumes and future college application chances. Finally, these experiences are often designed to provide opportunities for “student voice” where student leaders can represent their peers on matters that will most assuredly affect their educational experience. However, all too often these positions, councils, and committees do not actually utilize their student representatives to any meaningful extent, nor is there much in place to create an educational opportunity, beyond mere observation of adults, to impart leadership or life skills development. A review of the youth leadership through sport and physical education research, 5 for example, showed that one of the largest barriers to facilitating youth leadership is for adults to give up control and allow young people to make meaningful decisions in their programs (Gould, Voelker, & Blanton, 2012). Furthermore, research is mixed on the value and outcomes of participating in athletics and other extracurricular activities toward adolescent development with some studies showing negative effects such as increased alcohol consumption or a higher likelihood to engage in risky behaviors (e.g., Eccles & Barber, 1999; Holland & Andre, 1987) while a number of other studies revealed positive outcomes of athletic participation such as high rates of initiative, emotional regulation, and teamwork (Hansen, Larson, & Moneta, 2006). Interestingly, some evidence shows that young people who are allowed greater control over their experiences in organized programming often respond well and learn how to navigate the skills they have, develop skills they need, and experience a greater sense of development than those involved in adult-controlled youth experiences (Larson et al., 2004). Larson (2000) has also discussed the necessity of providing an environment that invites initiative or the ability to develop creative projects and be supported in the endeavor to complete those projects. This supported initiative is a crucial ingredient, or mechanism of change, which may foster personal growth and lead to greater personal and leadership development (Larson, 2000). Few, if any studies, however, have been conducted to understand what benefits young athletes gain when they are given leadership opportunities in sport, especially at the decision and policy making levels. Significance of the Study Evaluation research, when carefully planned and executed, can shed the necessary light upon the program under investigation to highlight if and how the goals or intentions of the program’s activities are being perceived, used, executed, and consumed. In this particular 6 project, the global aim of the evaluative investigation is to gain information that will help the MHSAA and particularly the SAC programming to reach its potential. Adequately understanding how to optimize the educational and institutional opportunities that exist in such a program’s organization and activities can improve the student-athlete experience and the relationship between the MHSAA and their most important stakeholder group, the high school athletes from their member schools. Therefore, a formative evaluation approach has been designed to illuminate a more objective view of the MHSAA SAC. With such a strong “call of duty” from the MHSAA SAC belief statement, it is important to examine how those involved with the SAC perceive their role, if leadership and empowerment lessons are being learned, and to evaluate if the SAC is in fact conveying the message of “how sports are to be played” and, thusly, “helping the MHSAA maintain a positive and healthy atmosphere” for its high school student-athletes. With little other documented information regarding the activities, goals, and intentions of the SAC, a formative evaluation can provide the necessary guidance to help shape the program and particular activities to better abide by the belief statement, and more transparently and objectively serve the MHSAA and represent high school student-athletes. Documenting the SAC program through an evaluative lens can help the MHSAA design a stronger program with more intentional activities and requirements of its members and director. This documentation can also serve as a best practice model of state-level student-athlete advisory groups, and can be shared across the country. Helping encourage the formation of such groups would better inform the state high school athletic associations and offer high school students a more significant voice toward their athletic experiences and perhaps serve a meaningful role in shaping how interscholastic sports are conducted in their state. Lastly, the potential of state 7 associations adopting such programming would provide their student-athlete constituents a valuable leadership and educational opportunity aimed to further enhance their personal development. “Program Evaluation is the use of social research methods to systematically investigate the effectiveness of social intervention programs in ways that are adapted to their political and organizational environments and are designed to inform social action to improve social conditions.” (Rossi, et al., 2004, p. 16). Evaluation research primarily attempts to inform and fulfill the needs of a select group of program or organization stakeholders, while secondarily informing scholarly literature and practice. Alkin and Christie (2004) categorize theoretical evaluation standards into three groups or ‘branches’: methods, value, and use. The ‘use’ and ‘value’ branches represent the guiding methodological design in the evaluation research project presented in this dissertation. The ‘use’ branch represents an orientation toward stakeholder decision-making and the utilization of the findings of the evaluation. The ‘value’ branch attempts to grasp the value placed on the program by its stakeholders. Within the ‘use’ branch, and the most relevant and informative theory for this dissertation is participatory evaluation (Cousins & Whitmore, 1998). This involves working directly alongside the evaluand and program stakeholders to determine the direction, goals, and general usefulness of the evaluation. From the ‘values’ branch, responsive evaluation theory (Stake, 2000) best supports the evaluative endeavor. The premise of responsive evaluation is observing and taking careful consideration of those most involved with the program under review through careful and poignant practices. Through this consideration, the value placed on the program by its stakeholders is carefully drafted. Evaluation, then, has a primary purpose to produce knowledge that can help an educational program to make informed decisions about the program 8 under scrutiny and its advantages to adoption or modification (Stake, 2000). Finally, participatory evaluation represents a methodological guide toward the evaluator to facilitate a project where stakeholders work together to create a shared meaning of the program or organization (King, 1998). The evaluation as a source of inquiry regarding program functioning can be thought of as an organizational support structure that ideally leads to programmatic change through the knowledge formation or shared meaning, and then the use of the evaluation findings (Amo & Cousnins, 2007). Constructing a theory-driven evaluation design also adds to the literature regarding the use of particular evaluative theories and their practical use (e.g., King, 1998). Examining the ability of a theory’s reach and ease of use can further strengthen or debunk its validation as an adequate means of investigating programming. By combining the most appropriate elements from responsive evaluation theory and participatory evaluation theory, this dissertation study utilizes the strongest guiding framework to ascertain the shared meaning of the program as well as to articulately share the value of the program with its stakeholders to be of utmost aid in enhancing this educational program. Purposes of the Study There were two primary purposes guiding the design of this evaluation study, and two secondary, exploratory purposes. The first purpose was to chronicle and evaluate the tasks and activities (actual and idealized) of the MHSAA SAC. The second purpose was to evaluate the perceived organizational benefits of the SAC to the MHSAA and its executive staff and administration. Related to these two primary purposes this dissertation had four goals, coconstructed with the MHSAA. First, to clarify and objectify goals and purposes of the MHSAA SAC, toward identifying programmatic standards by which to evaluate further. Second, to offer 9 recommendations to the MHSAA regarding the administration of the SAC. Third, to identify and illuminate a description of best practices and perceived value for having such a group for the MHSAA and other state level high school athletic associations, or high school athletic departments. And lastly, contribute evidence to the growing scientific body of literature examining the benefits of participation in leadership activities for student athletes. The final two exploratory purposes of this dissertation revolved around investigating the perceived personal development gain or sense of empowerment experienced by the studentathletes (Purpose 3), as well as to investigate the experience of being evaluated (Purpose 4). Exploring their notions of empowerment or personal changes resulting from program participation could be beneficial for the MHSAA administration and also contribute to the literature regarding positive youth development. Discussing how the stakeholders perceived participating in the evaluation could provide beneficial information to the kinesiology literature and is expected when following participatory evaluation theory (Amo & Cousins, 2007). These purposes were achieved by conducting this evaluation in two phases. Phase 1. Phase 1 was designed to meet the two primary purposes of the evaluation study and entailed individual interviews with the adult SAC stakeholders. The MHSAA executive director, the MHSAA SAC facilitator, and members of the MHSAA representative council would serve as valuable informants as prompted by the executive director or facilitator were interviewed. Phase 1 included focus group interviews with the MHSAA SAC student-athlete members to gain their perspective on the implementation, goals, tasks, and potential benefits of the program. Phase 2. Phase 2 involved additional individual interviews with the executive director and MHSAA SAC facilitator for the purpose of following-up on the findings from Phase 1. 10 Second individual interviews were conducted with the MHSAA SAC student-athletes to gain further opinions about the program and their experiences as members. Phase 2 was designed to investigate the two secondary purposes of the evaluation study (Purposes 3 and 4). Specifically, the two purposes were to identify and explore the potential empowerment opportunities and skills gained by the student-athletes (Purpose 3) and to follow up on the experience of being evaluated to better inform the participatory evaluation framework (Purpose 4). Organization of the Dissertation This dissertation is organized into five chapters. Following this introductory chapter that explores the nature of the problem and overviews the studies purposes, Chapter 2 reviews the literature on evaluation, evaluation theories, and positive youth development; topics all germane to the purposes of the study. Given the formative evaluation approach, and the nuance of evaluating such a program as the SAC, particular attention is paid to conducting a need’s assessment, employing responsive evaluation, and using participatory evaluation tools. The particular methods of investigation and the evaluation design are provided in Chapter 3. Chapter 4 provides the outcomes and results from the evaluation. Particularly, the thematic concepts from Phase 1 are shared, along with a logic model for the MHSAA SAC. Thematic results from Phase 2 are also shared. Chapter 5 discusses the results of the project and the theories used to guide the evaluation. Guidelines for the MHSAA are provided here to inform future practice, share recommendations and encourage self-monitoring and other evaluative efforts. A description of the best practices that other state high school athletic associations might profit from are presented, followed by a discussion of how the study contributes to the growing body of evidence on ways of enhancing student leadership in sport. Future research suggestions are also explored. 11 CHAPTER 2 REVIEW OF LITERATURE The purpose of this chapter is to review relevant and related literature to this study. First, a review of literature exploring the relationship between positive youth development and sport participation is offered. Furthermore, helpful findings from research on extracurricular activities and outcomes for participants are presented. These findings are shared to best connect how the SAC could be presented as a sport-based extracurricular activity. Following this brief description, a global perspective on the field and purpose of evaluation is discussed. Because evaluation has been used so sparingly in the field of kinesiology and sport psychology, it is important to understand this paradigm in general; definitions, purposes, and theoretical orientations in evaluation are presented. Next, the most relevant guiding evaluation theories are covered, with particular attention paid to responsive evaluation and participatory evaluation. There are no shortages of practical applications of physical activity principles as well as a multitude of physical and psychosocial benefits of sport participation in the kinesiology literature. However, rarely are programs based on these principles subjected to the rigorous investigation offered by a structured, theoretically guided evaluation. Therefore, the psychosocial benefits of participation are generally retrieved either retrospectively through interviews after the completion of a sport experience, or are merely theoretically posited for some future practitioner and its application. This dissertation project offers a much-needed bridge between the field of evaluation and the practice of sport-based developmental programming. Utilizing the tools and methods from evaluation toward investigating a sport-centered experience offers a unique and over-due connection between two complementary fields of research. 12 Promoting Positive Youth Development through Sport Programs This portion of the paper will discuss how competitive sport has been viewed as a means for experiencing and learning transferable developmental skills. In describing a framework for sport psychology research, Thorpe (2009) suggested going back to the logic where a behavior is viewed as a function of the person and the environment B = f{P, E}, as originally set forth by Kurt Lewin in 1935. Using this notion, where in addition to one’s personal make-up, the environment is considered an important aspect of the person’s behavior, we can begin to understand how crucial it is for coaches and parents to craft the young sport participant’s experiences to have the most beneficial outcome on their behavior. Children participate in sport to primarily have fun as well as to stay in shape, long before they rate competitive outcomes as a means for joining a team (Ewing & Seefeldt, 1989). Using the sport context as an attractive means for participant recruitment, providers and practitioners can elucidate psychological changes for developmental growth with intentionally designed life skills lessons, while utilizing the physical activity component inherent in sports participation for promoting participation. Larson, Hansen and Moneta (2006) sampled 2,280 youth assessing their positive developmental and negative experiences and compared these findings across types of activity: faith-based, sport, academic clubs, service, arts, and community-oriented activities. Findings for sport showed that youth reported significantly high rates of initiative, emotional regulation, and teamwork experiences over all other types of activities. Fredricks and Eccles (2006) conducted a longitudinal analysis of youth involvement in sport and personal behaviors. The purpose of this study was to explore the impact of sport participation on educational experiences and personal behaviors to assess if sport participation th th could predict more prosocial behaviors. Participants completed surveys in 8 grade, 11 grade, 13 and one year after high school regarding their academic standing, extracurricular involvement and personal behavioral choices. Results revealed that those who participated in sport were th shown to have higher 11 grade GPAs, higher educational expectations, lower depression levels, less alcohol use, lower marijuana use in boys, and higher self-esteem levels over their peers who did not participate in athletics (Fredricks & Eccles, 2006). Moving toward investigating athlete’s experiences in sport, Fraser-Thomas and Côté (2009) interviewed 22 competitive adolescent swimmers to assess their positive and negative developmental experiences in sport. The positive developmental experiences reported by the young swimmers were being challenged, developing meaningful adult relationships, developing meaningful peer relationships, experiencing a sense of community, and positive life experiences. Some of the more specific skills learned through this sport experience were time management, developing good communication skills, opportunities to practice leadership, setting goals, and relationship building skills. Some of the negative developmental experiences reported by the athletes were poor coach relationships, being negatively influenced by peers, being burdened with parent pressure, and experiencing a psychologically challenging environment, such as stress. In this study, the young swimmers were part of a highly competitive athletic environment, yet they still experienced and learned some positive transferable skills beyond the sport realm. In an assessment of sport participation and successfully acquiring development skills and character, researchers have examined athletes after they have been involved with sports for some time (Busseri, Rose-Krasnor, Willoughby, & Chalmers, 2006; Greenwood & Kanters, 2009). In an examination of 230 collegiate football players, findings revealed that those with a higher ability level, also had higher character scores and stronger goal orientations (Greenwood & Kanters, 2009). Busseri, Rose-Krasnor, Willoughby, and Chalmers (2006) tested the breadth and 14 intensity on youth activity involvement over a 20-month period. At the first collection (Time 1), participants averaged 14.86 years old, and at the second collection (Time 2) participants were on average 16.87 years old. Participants were asked to respond to their involvement in 7 types of extracurricular activities. At Time 1 and 2, the researchers found that greater breadth and intensity of activity was associated with more positive well-being, stronger academic orientation, and successful development. Further, breadth of activity at Time 1 was related to less risk behavior, and more positive interpersonal functioning at Time 2. However, greater intensity at Time 1 was associated with increased risk behavior and decreased interpersonal functioning at Time 2. In another qualitative exploration of sport experiences and character development, Camire and Trudel (2010) interviewed 20 high school athletes, with an average age of 15 years, regarding sport and character development immediately following their competitive athletic season. Each interviewee was presented with two sheets of paper, each consisting of three words related to either values of social development or values of moral development and asked to discuss if and how their sport experience contributed to these values. These words were carefully chosen from the school’s athletic department mission statements. Most athletes agreed that sport provided ample ground for social development, as sport was the first place they were a part of a collective group. Interviewees suggested that both moral and social values were experienced through sport, though team sport participants discussed more development of social values and individual sport participants showed more development of moral values. Individual sport athletes may not have experienced the sense of community in the same way as team sport athletes, as they compete against one another, just as they compete against athletes from other schools. Athletes showed little understanding of “moral character”, and it was suggested that coaches 15 place more emphasis on teaching the meaning of character through sports. An interesting aspect of the study was that none of the athletes was even aware of their school’s mission statement regarding athletic participation. Perhaps these results would have been quite different if the coaches made a concrete effort at sculpting an environment that was more intentional in social and moral development, as the coaches in Gould, Collins, Lauer, and Chung’s study (2007). In an effort to understand how coaches might impact a player’s development, Gould, and colleagues (2007) interviewed 10 successful football coaches who had been recognized for their efforts to develop character in their players. These coaches were highly experienced with an average of 31 years of experience and successful with a 76% winning percentage. Interviews revealed that for these successful coaches, although winning was important, they emphasized personal development as a top priority, especially in regards to working with others, being accountable to self and others, and setting goals. These coaches were attempting to impart transferable experiences to their young athletes, beyond just tactical football skills. Life Skills Needs The World Health Organization (WHO; 1999) defines life skills as those cognitive and behavioral attributes that give one the ability for adaptive and positive behavior that enables individuals to effectively deal with the demands and challenges of day-to-day living. Common life skills include having the ability to communicate effectively with others, developing the capacity to work in groups, being able to cope with stress, and knowing how to develop and maintain relationships. Researchers have begun to examine life skills development associated with sport involvement. For example, Jones and Lavellee (2009) explored the life skills needs of youth through focus group interviews with adolescent athletes, sport coaches, and experts and graduate 16 students in sport psychology. The results of the interviews revealed a participant-centered definition of life skills, which stated that a range of transferable skills are needed for everyday life, by everyone, and are needed if people are to thrive in their environment. The key term here is “thrive”, which describes more than mere survival, but rather a dominant power over elements that would inhibit others who have not necessarily acquired these social skills. The focus groups revealed the social skills needed are respect, leadership, family interactions, communication, organization, self-reliance, discipline, and goal-setting. The skills of goal-setting, communication, and self-reliance were also described previously by those successful coaches, whose philosophy included an intention of teaching life skills through playing football (Gould et al., 2007). Larson (2000) described the need for initiative as the primary need and function of encouraging positive psychosocial development in young people. Larson (2000) further wrote that initiative is closely related to autonomy and capacity for agency. When one gains initiative, they are more likely to be internally motivated. This initiative, as described by Larson (2000) is a crucial step in learning more tangible skills such as leadership and community engagement, as well as personal skills such as creativity and altruism. Gould, Chung, Smith, and White (2006) found that leadership was one of the most important life skills needed according to high school sport coaches. Petitpas, Van Raalte, Cornelius, and Presbrey (2004) described the outcomes for youth participants of the National Football Foundation’s Play it Smart Program. Participants in the Play it Smart Program were 252 student-athletes (91% African-American), who lived in disadvantaged and “AmeriCorps” neighborhoods with high crime, gang, and school dropout rates, likewise, 89% lived in single-parent homes and attended 3 inner-city high schools. 17 Monthly evaluations and quarterly academic reports were used to assess the program over a twoyear pilot period. Participants raised GPA on average from 2.16 to 2.54, and average SAT scores were 829 compared to 801 for the general school populations. Ninety-eight percent of the seniors graduated, 83% of those went on to higher education, and participants engaged in over 1700 hours of community service. These findings coincided with those of Larson, Hansen, and Moneta (2006), who posited that sport settings can provide higher rates of initiative and emotional regulation over many other extracurricular settings. Often times, allowing disadvantaged youth a safe place to be physically active can alleviate the stressors they would otherwise face (Danish, 1996), though intentionally designed programming should capitalize on the arena that a sport and physical activity setting provides (Larson, 2000). Recent research, however, shows the degree that life skills are developed is dependent on the coach and the kind of climate he or she creates (Gould & Carson, 2008; Flett, Gould, & Lauer, 2012). Climates that focus on self-improvement and creating a caring environment are associated with more positive life skills (Magyar et al., 2007; Newton et al., 2007). Hence, programs like the MHSAA SAC are most likely influenced by the climate adult leaders create. Life Skills Education Framework In moving from sport programs to sport-based programs with a direct intentionality on teaching these life skills, Petitpas, Cornelius, Van Raalte, and Jones (2005) provided a framework for youth sport programs to encourage development. These authors provided four prongs on which to build a sport program that encourages psychosocial development regarding life skills acquisition. The first prong is context. The environment in which youth will thrive and have a better chance of acquiring the life skills intended, is crucial for program development. 18 Sports can provide this environment, as it encourages interaction, the opportunity to lead and can be a challenging activity. In addition to sport, other extracurricular activities could provide this context, such as music and art, although all seem to be at risk for being cut from school budgets (Larson, 2000). The second prong is external assets. These external assets are described as others support and a caring community system (Petitpas, Cornelius, Van Raalte, & Jones, 2005). The key characteristic of these external assets, is rather than focusing on fixing problem behaviors, focus on supportive and encouraging positive behaviors. Furthermore, these assets provide opportunities to for young athletes to gain confidence in their abilities to use these positive life skills beyond the sport context in which they might have been learned. External assets are a product of consistent contact with adults and peers that support these behaviors, as well as evolving from quality relationships, behaviors and the expectations placed on them by mentors and adults. The third prong is internal assets. Petitpas et al. (2005) described these skills as those which allow a participant to internalize behaviors related to social, planning, and problemsolving competencies. Allowing enough time to pass with ample opportunity to practice helps make these behaviors normal for the individual and then they will be seen beyond the program context. These internal assets are strongly related to the autonomy that Larson (2000) speaks to in his definition of initiative, as well as towards the notion of thriving in everyday life as interpreted by Jones and Lavallee (2009). The final prong is research and evaluation. Petitpas et al. (2005) asked that programmers with a mission toward teaching life skills through sport carefully document their lessons, outcomes, and efficacy. Further, these measurements of efficacy should be multidimensional so 19 that each internal asset is identified and evaluated based on the outcome goals of the program. In addition to measuring assets and outcomes, longitudinal data should be collected beyond the programs conclusion to illustrate if any change remained years later. This ideal notion of research, evaluation, and dissemination is crucial to the creation of programs that can induce the acquisition of valuable life skills beyond the sport context, which is the essence of life skills programming through sport. Extracurricular Activities in Adolescence An area clearly associated with student development is extracurricular involvement. While there are numerous types of organizations and activities considered to be included in the vague label of “extracurricular”, the diversity of youth experiences is too broad to assert that mere participation in any extracurricular activity would lead to similar developmental outcomes, if any developmental gains at all. However, because of the nature of the MHSAA SAC, it would be necessary to cover some of the few key findings from research related to student outcomes associated with extracurricular involvement, as well as most of the SAC participants will likely include a description of their involvement in SAC as an extracurricular club or activity in their college and job applications. Darling, Caldwell, and Smith (2005) described the value of school-based extracurricular activities (EA) in that these programs “provide highly structured leisure environments, in which adolescents can exert control and express their identity through choice of activity and actions within the setting, but which do not normally facilitate experimentation with roles and activities that are not sanctioned by adults. Because of these characteristics, participation in [EA] provides many of the positive development opportunities offered by other forms of leisure, but may 20 provide more protection against experimentation with problematic activities such as drug or alcohol use than unstructured social leisure settings.” (p. 52). A paper authored by Holland and Andre (1987) reviewed 30 studies on EA participation and outcomes, and concluded that those students involved in EA often have higher levels of academic achievement and commitment, as well as more positively developing relationships across diverse ethnic groups. Additionally, Holland and Andre posited that students involved in these types of programs experience lower levels of delinquency, arrest, or high school dropout. Two dissertation studies (Bulling, 1992; Smith, 1991) also supported the ability of those involved in EA to relate better with individuals from diverse ethnic backgrounds suggesting that as EA involvement increased so to did scores on personal development measures and the ability to more successfully navigate three areas: (1) more mature interpersonal relationships, meaning the ability to relate appropriately to peers and parents, (2) clarifying purpose, which involves initiating career and lifestyle plans, and (3) academic autonomy, which is a skill set directed toward achieving academic goals more independently. Another study found that college students were less likely to drop-out and more likely to enjoy their college experience when involved in EA (Fitch, 1991). Fitch’s findings also supported the dissertation work of Bulling (1992) and Smith (1991), with results indicating that those in EA were more likely to have increased maturity gains and enhanced career-decision making skills. When Fitch (1991) organized respondents by the level of involvement in EA, analyses showed those most ‘highly’ involved scored higher in Leadership, but warns that this score could be considered an orientation toward “power” or authoritarian leadership, compared to those who were less involved (categorized as ‘moderately’ involved). This group of moderately involved respondents scored highest in benevolence, which Fitch suggests 21 demonstrates a stronger desire to serve others, perhaps more relevant to a the type of leadership more closely discussed as necessary in life skill development. Finally, literature related to EA involvement supports that those participants have more positive educational trajectories (Eccles & Barber, 1999), and display more positive development, especially when involved in student leadership or volunteering type EA (Bundick, 2011). Dworkin, Larson, and Hansen (2003) defended the notion that EA stands out from most other activities in school for adolescents because these programs provide a more apt opportunity for identity work, along with developing initiative, new social skills and allowing youth to learn some emotional competencies. When viewing EA from an ecological macro-level, Dworkin et al. suggested that youth can gain social capital through the formation of new connections that might otherwise not have existed naturally, returning to the findings of Holland and Andre (1987) and the notion of more positive relationships across ethnic diversity. Additionally, EA provided one of the few opportunities for youth to regularly interact with unrelated adults, in an often meaningful manner, outside the classroom, thus again increasing the opportunity to gain social capital (Dworkin, Larson, & Hansen, 2003). Given the above, it is clear that participation in an extracurricular program like the MHSAA SAC might be associated with a number of possible developmental gains in life skills. This will most likely occur if the program is carried out in such a way that key conditions for development are fostered (e.g., Larson’s suggestions for youth development in adult-organized progamming, etc.). While the SAC is designed as a feedback and policy input mechanism for the MHSAA, it is the author’s experience that MHSAA feels that student athletes who take part in the program will have a strong developmental experience, especially in the student leadership area. This evaluation study will provide an opportunity to verify this assumption and to examine 22 what students perceive they gain from the participation, especially in the empowerment and leadership areas. Reviewing these studies showcases that a major focal point for life skills education through sport should have an intentional approach in teaching and allowing participants to practice these skills. Furthermore, relationships with caring adults and peers can be a valuable asset in determining the success of a program. With a presentation of outcomes from several life skills through sport programs, and a discussion of frameworks for developing these programs, practitioners should place an emphasis on the nature of encouraging youth participants to practice these skills in a sport setting while also discussing the value of these skills beyond their program. The inability to successfully measure for transference of life skills is a detriment to the field and researchers need to create means of assessing not only the immediate acquisition of life skills from a well-designed program, but also the long-term use and ability for those skills to be necessary in a diverse range of settings. Too often in the literature, programs designs are discussed but never evaluated beyond a one-time use of the curriculum. A long-standing program design is needed, and the frameworks presented here should serve as a place of origin for organizing curriculum. Though the SAC is not a sport program per se, all its participants must be studentathletes, and the discussion of sport is its focus. The SAC is not necessarily an intervention, but because of its natural design, could be a place to illuminate and foster developmental experiences. Reviewing Larson’s (2000) tenets for promoting developmental gains, the SAC utilizes a caring adult to facilitate its meeting and activities. From Petitpas et al. (2005), the SAC members are asked to actively plan their yearly project and are centered in problem-solving both in the formation of the project as well as working through issues that arise in the project’s 23 functioning. Larson, Hansen, and Moneta (2006) provided support for incorporating participantdriven initiative. From the very introduction to SAC, potential members must go through an application phase and once accepted are placed in a context of high-achieving peers, asked to create and participate in a number of activities, further promoting the potential of SAC to serve as a developmental setting. Avolio (2007) suggested a careful understanding of the context in which leadership may be developed. Evaluation guided investigations and research should serve as the primary mode for gaining an understanding of the context in which the participants engage with one another and the potential for leadership development. Gould, Voelker, and Blanton (2012) concluded that learning to lead is multi-faceted and young people must be empowered to make meaningful choices, engage in experiential learning, and offer formal education to develop leadership skills. The perspective that the paradigm of evaluation lends to such investigations in kinesiology is invaluable and should be utilized whenever possible to obtain the most accurate portrayal of program functioning and to ascertain exactly how stakeholders view, value, and experience a sport-based program designed to elevate youth agency and promote life skills development. Evaluation as a Scientific Endeavor “Program Evaluation is the use of social research methods to systematically investigate the effectiveness of social intervention programs in ways that are adapted to their political and organizational environments and are designed to inform social action to improve social conditions.” (Rossi, Lipsey, & Freeman, 2004, p. 16). It is important to understand that the process of conducting an evaluation is to impart varying methods to assess the ways in which programming effects the larger social context in which it exists. For instance, in this particular project, the immediate social context is the offices of the Michigan High School Athletic 24 Association (MHSAA) and its staff, as well as the 16 student-athlete participants of the Student Advisory Council (SAC). Further social contexts would incorporate high school sports in the state of Michigan, as well as any individuals who come in contact with the SAC members in a SAC sponsored event. Regarding the purpose of an evaluation, a conservative view on summative projects would suggest that evaluations are to determine the merit, worth, or significance of a program or product (Scriven, 1991). A more liberal perspective on the purpose of evaluation would be to help understand the underlying processes that affect a program or products outcome or ability to reach stated objectives. From either perspective, what differentiates evaluation from scientific research is that evaluative projects should serve to first meet the immediate needs of the programmatic participants, stakeholders, and decision makers (Rossi et al., 2004) whereas a scientific study first serves the development of knowledge. Upon meeting the needs of these individuals, then secondarily, through scholarly publication including empirical data, informative case studies, and relevant knowledge gained through the evaluation process, evaluation can then inform the scholarly scientific community and other practitioners. In this vein, a formative evaluation model would be used to provide feedback on midstream merit of the program under study, performed as a service to assist and identify methods for improving the program (Sciven, 1997). The current project utilizes this formative approach in evaluating the SAC of the MHSAA. Often times, after a program has been established or running for a long period of time, stakeholders’ daily routines can dictate program practice, and may be unable to articulate specific rationale for how these practices meet the program’s objectives. In this case, the role of the evaluator may be to help the program staff and personnel to formulate the connection and establish the rationale for particular activities (Rossi, et al., 2004). This is also referred to as a 25 process evaluation, broadly implying that as the evaluation is conducted, both the program stakeholders and evaluator work to better shape the program as issues, relative practices, and a deeper meaning of the program purpose is uncovered and discussed. The first step in planning the evaluation of a program is determining the evaluability of that program. Wholey (2004) suggests that the evaluator begin by identifying the goals of the program under assessment, then seeking performance indicators and sources for gathering data. These evaluability assessment techniques are intended to guide the initiation of the evaluation toward understanding the scope of the program and the intended users of the evaluation information so that the evaluation is conducted in such a way that not only details and identifies the intricacies of a program, but also helps to insure the outcome is of utmost usefulness to the stakeholder within the program. Theories are sets of propositions, which ideally, jointly provide explanation and integration toward the formulation of actual use in practice supported by empirical knowledge (Scriven, 1998). Alkin (2004) described the underlying purposes of theories designed in evaluation to inform readers and practitioners toward methods, value, and/or use. More practically and appropriately, Alkin offers a ‘prescriptive model’ toward evaluation, which is described as a set of rules and guiding methodology for what makes a good evaluation and how it should be conducted. Alkin and Christie (2004) have illustratively described an ‘evaluation tree’, shaped most strongly by three main ‘branches’ to describe these underlying informative purposes (methods, value, use). As a metaphorical tree, these three main off-shoots are thusly composed of smaller branches dedicated to the theorists guiding the scholarly literature, and the minor nuances that dictate a specific set of rules guiding particular approaches. The ‘methods’ branch most generally refers to identifying the most appropriate means of selecting the ways by 26 which information is collected in an evaluation. The ‘value’ branch is composed of theorists who most strongly suggest that the essential purpose of evaluation is placing value on data, that being a phenomenological investigation of the meaning toward the quantified variables, as they pertain to stakeholder values. The ‘use’ branch represents an orientation toward decision-making, how the evaluation can shift programmatic activities, and the experience of those involved in the evaluation. Because the nature of this evaluation is designed to help stakeholders make decisions about the program and uncover how participants feel about and experience the program, it is most necessary to draw theory from the ‘use’ and ‘values’ branch. Explicitly, this project used tools and methods of evaluation from responsive evaluation theory (Stake, 2000), and participatory evaluation theory (Cousins & Whitmore, 1998). Responsive Evaluation At the onset of forming responsive evaluation, Stake, the most driving theorist behind it, says that his original goal was to write a “methodological guide, rather than an epistemological mapping” (Abma & Stake, 2001, p. 8). Therefore, the premise of responsive evaluation is observing and taking careful consideration of those most involved with the program under review through careful and extent practices. Responsive evaluation, then, has a primary purpose to produce knowledge that can help an educational program to make informed decisions about the program under scrutiny and its advantages to adoption or modification (Stake, 2000). Alkin and Christie (2004) described the process of responsive evaluation as being critical to the value of the program in its given context. It is “as much a matter of refining early perceptions of quality as of building a body of evidence to determine level of quality” (Alkin & Christie, 2004, p. 38). Meaning, those participants actively involved at the program site or in program activities are best suited to determine the merit of the program for themselves as stakeholders. This 27 stakeholder defined value and the sources of its value lay the foundation for how future stakeholders may wish to monitor success of the program. Stake (2000) condoned that this approach is “evaluation based on what people do naturally to evaluate things: they observe and react” (p. 347). He suggested that an evaluator should pay careful attention to exactly what program participants are exposed to, the ways in which they react, and then attempt to understand why this relationship is such. Schwandt (2001) proposed the necessary skills of an evaluator should include an ability to fully describe the details of people’s experiences, their actions, and the surroundings in which all this occurs. A crucial element of responsive evaluation is the critical examination of the context in which a program and its participants engage with one another. The situations in which a program occurs can sway heavily the practices that occur there. “Resources, personal capacities, expectations, obstacles and constraints, and the like are unique to situations” (Schwandt, 2001, p. 76). Stake (2000) further suggested that the evaluator should not be concerned with some preconceived notions of success but focus their efforts to the people involved over objectives and data-collection instruments. This is not to say that objective measurement tools are to be ruled out, but the decision to their inclusion should be useful in the evaluation based on observing the program in its natural elements. It is of utmost necessity of the evaluator to fully understand the program as it operates and all of the uniqueness of its environment, prior to selecting the “success-indicators” and those tools against which to measure the program. Hence, in this study, a sustained effort was made to chronicle and understand the context in which the program is situated. Detailing and explicating the facts or truths of a program from those items that members of the program value is a difficult but necessary task. Abma (2006) described responsive 28 evaluation as an engagement of the stakeholder’s values on the meaning of their program. Responsive evaluation attempts to describe these values in such detail, whilst considering the context in which they occur, to become the truths of those members, or rather the facts of the program through the participants’ voices. House (2001) described these ‘fact’ and ‘value’ claims, as beliefs about the world. In order to ensure that an evaluation includes a description of unbiased claims regarding the program under scrutiny, the evaluator must use as many tools as possible to collect, process, and test these claims (House, 2001). House (2001) further suggested three principles to aid in this endeavor: (a) inclusion of all the relevant stakeholders’ values and perspectives; (b) dialogue between these stakeholders and the evaluator; and finally, (c) an extensive deliberation in order to achieve validity in the study of the program; all principles that were employed in the present study. The purpose of responsive evaluation is to focus on the stakeholder’s issues and engage these people in a dialogue about the quality of the actions taken towards a program’s ideal, in order to raise the understanding of all these stakeholders as a potential for improving the program (Abma, 2006). Responsive evaluation allows for these efforts to fully explore a program and determine its meaning to those directly and indirectly involved, in the program’s natural environment. The structure of an evaluation following the responsive evaluative procedure donates much to the observation of a program and the interactions of the environment and between participants and practitioners. Stake (2000) does not encourage evaluators to conduct the evaluation in phases as the observation and feedback should be considered priority from the first to last week. There are twelve crucial elements of a responsive evaluation that should be practiced. However, Stake (2000) objected to these elements occurring in any sort of 29 chronological order or the evaluator determining a priority of these events. The 12 recurring events that an evaluator should pay most attention to are (Stake, 2000): 1. Talk with clients, program staff, and audiences. 2. Identify program scope. 3. Overview program activities. 4. Discover purposes and concerns. 5. Conceptualize issues and problems. 6. Identify data needs, regarding issues. 7. Select observers, judges, and instruments, if any. 8. Observe designated antecedents, transactions, and outcomes. 9. Thematize, prepare portrayals and case studies. 10. Validate, confirm and attempt to disconfirm. 11. Winnow format for audience use. 12. Assemble formal reports, if any. Stake (2000) reminds the evaluator that at any point in an evaluation, the evaluator must be able to identify which of these 12 steps to use for the greatest purpose at hand. Given the extreme concern for a program’s context, not all of these may be necessary in a study, but rather should be made available if the need arises. To further describe the structure of a responsive evaluation, one of the central features of this approach is dialogue, as previously described (Abma, 2006). Exploring the meanings of a program through the relationships between all the stakeholders and between the stakeholders and evaluator are crucial to the study of a program in this approach. From this end, the evaluator’s role can be described as a Socratic guide, as well as, interpreter, facilitator, and educator (Abma, 30 2006). The most important aspect of a program’s evaluation is a rich description of the program, through the views and voices of the stakeholders to arrive at an detailed explanation of what is occurring to whom and why, for the end-result of improving the program’s functioning. This will be a primary goal of the present study. Participatory Evaluation A second theoretical orientation that informed the approach to this evaluation project, particularly from a “use” perspective, was that of participatory evaluation (PE). Most appropriately, Cousins and Whitmore (1998) defined PE when the researcher or facilitator of the evaluation collaborates in some way with those stakeholders in the program with a central function in fostering the use of the evaluation findings, supported by decision-making and problem-solving through the evaluative process. King (1998) reflects that PE requires those participating in the evaluation, the evaluator and stakeholders, work together to create a shared meaning of their experiences over time. The evaluation as a source of inquiry regarding program functioning can be thought of as an organizational support structure that ideally leads to programmatic change through the knowledge formation or shared meaning, and then the use of the evaluation findings (Amo & Cousins, 2007). Use of the evaluation can occur in three ways, which may be seen in the program in various combinations. Harnar and Preskill (2007) first described “instrumental use”, which occurs when the results of the PE experience are tangibly used to make programmatic improvements. Secondly, “conceptual use” is seen when stakeholders perhaps change their thinking or their understanding (the way they conceptualize the program) about their experience in the program on the basis of the findings revealed by the evaluation. Finally, “symbolic use” refers to the findings of the PE being used in a persuasive manner, such as using the experience 31 or the findings to lobby for more resources or to merely meet the obligation for evaluation as some funding agencies require of their recipients. Regarding this use perspective vested in PE, Patton (2004) described the purpose of evaluation aimed at changes in the way individuals involved in the project think or behave, as well as organizational changes in the program procedures and even in the program culture. Ideally, those who are involved in a PE project essentially learn more about their program as a result of experiencing the evaluation. Amo and Cousins (2007) described organizational changes as a result of PE as “charged use”, indicating that as a result of this learning and deeper reflection regarding their involvement in the program undergoing evaluation, participants may potentially disrupt or change their program during the evaluation. This is demonstrated by changes in actions or behaviors toward the program due to interaction between stakeholders and the evaluation process. Because PE fosters thinking and learning about the program, as well as the likely interaction between individuals of varying power levels within the organization or program, King (1998) suggested that beyond a shared meaning of the programmatic experiences, those who elect to be involved must be strong participants and even regarded as a ‘leader’ by their peers, capable of facilitating the evaluation process and recruiting members to the process. Hence, in this study the executive director of the sponsoring organization, and the program’s developer were involved, and asked to nominate other adults from the MHSAA’s representative council. The student-athlete members who are selected for the SAC tend to be active leaders in their schools and athletic teams, and are only selected for the SAC with a strong resume of previous leadership positions, along with a nomination from their athletic director. Again, PE is centered on the learning that occurs as a result of experiencing the evaluation and then using the 32 findings. This essentially enhances the capacity of the evaluand to monitor their programming and more adequately prepare them for using evaluation or eventually participating in a more summative evaluation (Amo & Cousins, 2007). Finally, regarding the approach to conducting a PE so that the stakeholders are more able to learn from the experience and use the findings, Harnar and Preskill (2007) suggested that the evaluation facilitator should be one “who uses a collaborative, participatory approach and is committed to an open, engaged process with ongoing communication” (p. 33). Therefore, the learning that may occur as a result of PE is thus intentionally designed in the evaluator’s methods, to include asking questions and engaging in dialogue. Furthermore, the evaluator should attempt to invoke reflective and evaluative thinking toward fostering a critical perspective about the experience in a program. Finally, the evaluator working under the vision of PE would explore the many assumptions about a program or organization, and give feedback to those practicing under such assumptions as informed by the PE process. By developing such a rigorous dialogue and encouraging the formulation of a deeper understanding of the program under evaluation, the closer this study aligned to the principles of increasing the utilization of research (Landry, Amara, & Lamari, 2001). King (2007) referred to one major theoretical foundation of PE in the construction of evaluation capacity building (ECB), or the ability to strengthen an organization’s ability to utilize evaluation and sustain efforts of evaluative merit. Most notably, PE offers to ECB the goals of: (1) increasing an organization’s capacity to design, implement, and manage effective evaluation projects; (2) accessing, building, and using evaluative knowledge and skills; (3) cultivating a spirit of continuous organizational learning, improvement, and 33 accountability; and (4) creating awareness and support for evaluation as a performance improvement strategy (p. 45-46). All of which dictate a primary relevance to the capacity of this evaluation study, as support in the use of evaluation as a most adequate means of investigating a program such as the SAC, with a strong intention to offer aid and utilization of findings toward programmatic improvements and the ability to self-monitor in future practice, much like the arguments of Patton (2008). Reviewing these two evaluation theories (responsive and participatory) sets the stage for the methods of evaluation used in this project. Together, these theoretical orientations toward ‘use’ of evaluation (Alkin & Christie, 2004) foster a caring guidance toward helping the evaluand, in this case the MHSAA SAC, to further their conceptual understanding of the varying roles of those stakeholders associated with the programmatic purposes and the actions of the participants, leaders, and administration. The application of the evaluation paradigm to aid the organization in the creation of a shared meaning, and then to explore a sport-based program for youth development serves as a strong model for not only utilizing evaluative methods in kinesiology but also for the evaluation of sport-based programs in general. 34 CHAPTER 3 METHOD The overarching purpose of this dissertation study was to uncover the goals of the SAC and develop a model of how the SAC functions as a program within the MHSAA. Additionally, based on the data analysis and the observations by the evaluator, a program theory could be proposed. Therefore, qualitative methods using the grounded theory approach (Creswell, 2007) are the most appropriate means to access data and information. The evaluation study will include two distinct phases, as informed by the literature discussed in Chapter 2. This is necessary in order to adequately meet the study purposes and follow the theoretical guidelines of ‘use’ based evaluation, specifically as it relates to participatory evaluation and responsive evaluation. Phase 1 is designed to ascertain the goals and perceived purposes of the SAC, as well as document the implementation and related activities of the group toward the perceived organizational benefits. The essential premise of Phase 1 serves to fulfill the first two purposes of the study: (1) chronicle and evaluate the tasks and activities (actual and idealized) of the MHSAA SAC and (2) evaluate the perceived organizational benefits of the SAC to the MHSAA. Through interviews with stakeholders, including the student-athlete SAC members, the adult facilitator, and the executive director of the MHSAA, the evaluator created a shared meaning of the SAC. Taking a participatory evaluation theory approach, facilitated by grounded theory qualitative methods, the outcome of Phase 1 was presented in a logic model detailing the organization and functioning of the SAC. Phase 2 is designed to explore the third and fourth purposes of the study: (3) investigate the perceived personal development gain or sense of empowerment experienced by the studentathletes, and (4) investigate the experience of being evaluated from the perspective of all 35 participants. This is done to understand how the findings from Phase 1 can inform future practice of the SAC and interaction amongst the various members of the SAC program and MHSAA staff. Description of Investigator The primary investigator/evaluator for this dissertation project spent almost two full years observing and interacting with the SAC, and four years working with the SAC facilitator through a number of MHSAA educational settings, including coordinating an educational program for high school team captains. This close relationship with the researched persons and setting should not indicate an overtly biased perception however. In fact, Patton (2002) proclaimed that “qualitative inquiry means going into the field – into the real world of programs, organizations, neighborhoods, street corners – and getting close enough to the people and circumstances there to capture what is happening” (p. 48). Additionally, because of the necessity of establishing a strong, close rapport with research participants, the qualitative investigator must reflect and report on any potential sources of bias (Patton, 2002). Furthermore, responsive evaluation theory encourages a close relationship with the evaluand to best engage with the stakeholders and assert the values on the meaning of their program (Abma, 2006; Stake, 2000). These four years with the facilitator and two years closely spent with the student-athletes of SAC allowed for a strong rapport and generous allotment of trust between the evaluator and the organization and its stakeholders. This trust, as discussed in Chapter 4, greatly facilitated an open and honest dialogue amongst stakeholders and with the evaluator, which Abma (2006) described as crucial to the responsive evaluation. And lastly, as covered in Chapter 2, the responsive evaluation procedure donates much time to the observation of the program. Because the SAC only meets 7 36 times a year on average, it was crucial to begin establishing rapport and observing the program long before the evaluation was fully designed and carried out. In the two years that the primary evaluator observed and interacted with the SAC members, the evaluator attended the summer camp 3 times (twice staying overnight with the group), attended almost every meeting, accompanied the SAC students during all four Sportsmanship Summits (explained in detail in Chapter 4), and traveled with the MHSAA to 4 “Battle of the Fans” finalists visits (explained in detail in Chapter 4). This amount of interaction with the SAC could consider the evaluator essentially embedded in the program, thusly allowing for the strongest ability to describe the program and the interactions of the environment and between participants and practitioners (Stake, 2000). Clarification of the researcher and their potential bias is also a necessary process of verification for interpretative quality in the qualitative nature of the study (Cresswell, 2007). This strong and long-running connection to the MHSAA and specifically the SAC might appear to overly bias the investigator. However, to most appropriately build trust amongst the high school students and fully observe every facet of the SAC program as required by responsive evaluation, the investigator was required to spend an exorbitant amount of time embedded with the group. In addition to observing the program’s activities and meetings, the evaluator also conducted several team-building activities, leadership workshops, and aided SAC members in the preparation for speaking at the Sportsmanship Summits. As such, advice was often shared between the evaluator and SAC facilitator in order to improve interactions amongst the studentathletes and the MHSAA when it was appropriate. Furthermore, in the year of the evaluation project, the primary evaluator was partially employed by a graduate assistantship sponsored by the MHSAA to enhance other student services projects generally unrelated to the SAC. 37 This being said the investigator had been trained in qualitative research and had conducted previous qualitative studies. He understood his biases and worked hard during SAC evaluation to look for evidence that both confirmed and disconfirmed his opinions. Most importantly, he strove to capture and accurately interpret the voice of those interviewed. Phase 1: Purpose, Activities, Goals and Organizational Benefit of SAC Participants. Because of their lack of experience undergoing evaluation, the MHSAA SAC does not currently have strong records or even a clearly labeled purpose statement (beyond the previously mentioned ‘belief statement’) with which to guide the evaluation. Therefore, in order to achieve the first purpose of the dissertation, to chronicle and evaluate the tasks and activities of the SAC, several adult stakeholders were interviewed regarding the creation of the SAC program. The creator and facilitator of the SAC, along with the executive director were interviewed to ascertain the initial driving prerogatives for designing and investing in such a group. This process can be viewed as a step towards conducting a needs assessment; essentially, determining what there is to be evaluated or discussed with the remaining stakeholders, the student-athlete members of SAC. Another step in this process involved interviews with two members of the MHSAA’s representative council, a supporting unit of the MHSAA activities, who are charged with maintaining the goals and vision of “educational athletics” as enforced through the behaviors of the MHSAA. These two members were recommended by both the executive director and SAC facilitator as being strong supporters during the creation of the SAC program, for a total of four adult interviewees (N=4). Additionally, these two members are also high school athletic directors who have nominated previously accepted SAC student-athlete members. The purpose of these interviews was to gain a better understanding of the support and feelings regarding the purposes of SAC. 38 Upon achieving a better understand of the ‘why’ for the implementation of the SAC, the next step involves understanding the ‘how’. This entails exploring the perceived value of the SAC program to the MHSAA, reaching the second purpose of the dissertation. This will be conducted through continuing conversations with the executive director, the facilitator of SAC, and the recommended representative council members. Exploring the purpose of the SAC with the student-athlete members in two focus group interviews also highlighted their perspective on the actual and idealized tasks of their group, and their perception of the value the SAC provides to the MHSAA. Procedure. At the creation of the evaluation study, first the evaluator and SAC facilitator (the primary evaluand) developed a contract (see Appendix A) outlining the expectations and rights of the evaluator and evaluand. Specifically, the MHSAA waived rights to anonymity regarding the organization in publications and presentations, however, specific individuals would remain as anonymous as possible beyond referencing specific job titles within the MHSAA. Next, permission to collect interview data was obtained from the Human Research Protection Program. Patton (2008) recommended that the evaluator work closely with stakeholders to identify meaningful goals, and King (1998) suggested that participatory evaluation is designed with an effort to create a shared meaning. Therefore, qualitative interviews were the primary means of data collection in this phase. The procedures within Phase 1 are intended to specifically meet the following events dictated by Stake (2000) as responsive evaluation theory: (1) Talk with program staff; (2) Identify program scope; (3) discover purposes and concerns; (4) overview program activities; and (5) conceptualize issues and problems. The interviews with program staff and student-athlete 39 members were the best means for accomplishing the above tasks. All interviews were recorded and transcribed verbatim for later analyses. Adult individual interviews. The executive director and SAC program facilitator were first approached to discuss the interview at a SAC meeting in February. At this time, a brief explanation of the study and an estimate of the time required to participate was discussed. Meetings were then scheduled at a convenient time and place for both parties. During the scheduled interview time, the purposes of the study and evaluation were again discussed, consent to participate was obtained, and the interview was conducted. Both the executive director and the SAC facilitator independently expressed that the same two individuals from the MHSAA representative council would be good to interview and participate in the initial phase of the evaluation. The SAC facilitator first sent these individuals an email with the evaluators contact information. Both individuals responded, expressing interest in participating. A consent form was sent as an email attachment and signatures were obtained via a reply email with an attached scanned image of the consent form. A time was scheduled to conduct the interview over the phone given that both individuals lived far from the MHSAA offices. SAC member focus groups. During one of the previously scheduled Sunday afternoon meetings in February, the study was explained to the student-athlete participants, along with giving an informed consent to participate. Student-athlete members under the age of 18 were asked to obtain parental consent to participate as well as their own assent. The student-athlete members were also emailed a consent form. The interview was scheduled to take place during their April meeting at the offices of the MHSAA. Fourteen of the 16 student-athlete SAC members agreed to participate and brought along their signed consent forms (N=14). Two student-athlete members declined participation without offering any rationale. 40 Focus group interviews are a well-established method of data collection in social science research, where they are often used and are considered an extremely valuable source of information (Finch & Lewis, 2003). Furthermore, focus group interviews offer a strong social context, optimizing the opportunity for ideas and language to emerge in a more naturalistic setting and to be shaped through conversations (Finch & Lewis, 2003). The focus group “reflects the social constructions – normative influences, collective as well as individual self-identity, shared meanings – that are an important part of the way in which we perceive, experience and understand the world around us” (Finch & Lewis, 2003). The focus group procedure provides the closest alignment with the participatory evaluation premise of creating a shared meaning of the SAC. Finch and Lewis (2003) suggested the optimal focus group size is between six and eight individuals, therefore the student-athlete members were split into two groups of 7, separated by class (1 senior class group; 1 junior class group). Regarding the collection and validity of data obtained in a focus group setting, Finch and Lewis further posited that: “Data are generated by interactions between group participants. Participants present their own views on experience, but they also hear from other people. They listen, reflect on what is said, and in the light of this consider their own standpoint further. Additional material is thus triggered in response to what they have heard and prompt others to reveal more. As the discussion progresses, individual response becomes sharpened and refined, and moves to a deeper and more considered level” (p. 171). The role of the researcher in the focus group setting is to encourage an open and interactive discussion process, but also to ensure the conversation is steered toward the purpose of the investigation (Finch & Lewis, 2003). In this setting, the facilitator followed the recommended 41 guidelines for progressing through a focus group interview as recommended by Finch and Lewis (2003): (1) set the scene and discuss the ground rules; (2) introduce individuals (if needed); (3) relay the opening topic or question; (4) facilitate an active discussion with further probing questions; and (5) end the discussion. Interview guide. Two semi-structured interview guides were developed: (1) the adult stakeholders interview guide, and (2) the student-athlete SAC members interview guide. This approach ensures that each member of the varying groups is exposed to the same set of questions to ascertain the multiple perspectives. However, because varying responses were expected, the interview will also pose probing questions aimed at uncovering assumptions and deeper meanings to stakeholder’s initial responses. The interview guide was focused on uncovering the purpose of establishing the SAC and the goals regarding the actions of the members of the SAC, both for facilitator and for the student-athletes. Secondarily, the interview asked participants to share their opinions on the value the SAC provided to the MHSAA organization, and to the student-athletes, both SAC members and the general student-athlete membership (see Table 1). Additionally, the student-athlete members were asked to discuss their perspectives to the goals and purposes of the SAC, and to offer any initial suggestions for how the SAC could change to better meet their perceived goals. 42 Table 1. Phase 1 Interview Guides. Adult Interview Guide Questions Student Interview Guide Questions Purpose (1): Chronicle and evaluate the tasks and activities (actual and idealized) of the MHSAA SAC. What is your opinion on why the SAC was initiated In your opinion, why does the at the MHSAA? MHSAA have a SAC? What is the design of the SAC and how was that What do you think are the goals of the design initially decided upon? SAC? To your best knowledge, what are the goals of the How do you know if these goals have SAC? been accomplished? How do you know/judge if SAC has reached their goals? Purpose (2): Evaluate the perceived organizational benefits of the SAC to the MHSAA. In your opinion, how does the MHSAA benefit from the SAC? If you were in charge of the MHSAA SAC, what would you do? If you could change anything about the SAC, what would it be? How do you benefit from the SAC? How does the MHSAA benefit from the SAC? How do the student-athlete members of SAC benefit from participation? How do the student-athlete nonmembers benefit from the SAC? Stakeholder involvement. As a participant in either the adult group, or student-athlete group, transcripts of the interviews were sent back to the participants to ensure a most accurate depiction of the interview or focus group. Additionally, an initial assessment of the data was sent back to the SAC program facilitator to gain further perspective and ensure that the researcher’s analysis was on par with the purposes, goals, and related activities of SAC before finalizing the results of Phase 1 (see Chapter 4). Phase 2: Student Development Potential and Participation in Evaluation Participants. Again, both the executive director of the MHSAA and facilitator of the MHSAA SAC were interviewed independently (N=2) after the data collection for Phase 1 was 43 completed and the findings were established with the facilitator (see Chapter 4). This interview was focused on the experience of participating in Phase 1 of the research study and to uncover the intended use of the Phase 1 findings. Secondarily, the executive director and program facilitator were asked to respond to the comments from the student-athlete members, primarily as they related to their developmental experiences in SAC. The value of this interview was to determine how the SAC can align the goals and activities of the student-athlete members to serve as a valuable developmental experience for the young student-athlete members. Finally, the MHSAA SAC members were also interviewed to gain their perspective on the goals identified in Phase 1 as well as how participating in Phase 1 of the research study may have shaped how they conceptualize the MHSAA SAC experience. All the student-athlete members from the focus group interviews in Phase 1 participated in these individual interviews (N=14). Additionally, those seven junior class student-athlete members who will return to the SAC in the following academic year were asked to identify how they intend to use the findings of the evaluation. Essentially, Phase 2 is designed to first meet the third and fourth purpose of the dissertation study: (3) to investigate the perceived personal development gain or sense of empowerment experienced by the student-athletes and to (4) investigate the experience of being evaluated. This reflection on the process of participating in evaluation attempted to gain an understanding of the utilization of the findings, as dictated by ‘use’ oriented evaluation and participatory evaluation principles. Procedure. The interviews with the executive director of the MHSAA and the facilitator of the SAC will use another semi-structured interview format. Both interviews will guide the participants through the same structured questions, only differing in probing questions to gain more information from the subject’s initial responses. The student-athletes were contacted via 44 email to establish a convenient time to conduct an individual interview over the phone. These interviews were recorded and transcribed verbatim for later analyses. The procedures within Phase 2 are intended to specifically meet the following events dictated by Stake (2000) as responsive evaluation theory: (1) talk with program staff; (2) discover concerns; (3) conceptualize issues; and (4) validate, confirm, and attempt to disconfirm. Furthermore, by actively engaging in a discussion regarding the participation and potential changes in perception of the SAC as a result of having participated in the evaluation, Phase 2 aligns closely with the principles of participatory evaluation theory. Interview guide. Each interview will open with an explanation of the goals and tasks related to the findings from Phase 1. This will be done to refresh the participant’s memory. Similarly to the interview guides from Phase 1, the semi-structured guide for Phase 2 ensures that all participants were asked the same questions, however, the course of the conversations included probing questions unique to the participants’ comments. Both the adults and the student-athlete SAC members were asked to reflect on how participating on the SAC may have changed the student-athletes. Additionally, the results from Phase 1 were shared in the format of a logic model (see Chapter 4) and participants were asked to react and discuss their perceptions on the model’s accuracy and the intended use of the evaluation findings. Lastly, participants were asked to share their perceptions about participating in an evaluation of a program they actively participate in (see Table 2). 45 Table 2. Phase 2 Interview Guide. Adult Interview Guide Questions Student Interview Guide Questions Purpose (3): Investigate the perceived personal development gain or sense of empowerment experienced by the student-athletes. How will the student-athlete members of the SAC benefit from participating in this council? Did being in the SAC change you as a person? In your opinion, what is the value of participating on this council as a student-athlete member? If so, how specifically have you changed from being involved in the SAC? What skills did you learn from SAC that you can apply elsewhere in life? How much power do the student-athlete members have in the governance of high school sport and the practices of the MHSAA, if any? What do you think caused you to change in these ways or how did you notice you had changed? In your opinion, do the student-athlete members change as a result of participating in the MHSAA SAC? If not, what could be implemented into the governance of the SAC to help people like you learn valuable life skills? What could the facilitator and/or executive director do to better meet your needs as a SAC member? What can future SAC members do to most optimize their experiences in the SAC? Do you feel like you had much power in the decisions made surrounding your involvement in the SAC? How much ‘weight’ or value does your opinion or voice matter to the MHSAA? If you had a serious issue with some aspect of the MHSAA, do you feel you possess the power to change it or at least receive an answer to why it occurred? How do other student-athletes in Michigan benefit from your presence on the SAC? 46 Table 2 (cont’d). Adult Interview Guide Questions Student Interview Guide Questions Purpose (4): Investigate the experience of being evaluated. Describe your thoughts on participating in the evaluation. Describe your perception of participating in the evaluation process. Did anything new occur to you regarding the MHSAA SAC as a result of the initial interview process? How did participating in the evaluation change your perception of the SAC, if at all? How do the goals and tasks identified in the evaluation align with your perception of the purpose of the SAC? What are your thoughts regarding the goals of the MHSAA SAC? How might you intend to use the findings of the evaluation in the future? (for returning members) How do these goals affect your next year in the SAC? How will the MHSAA benefit from having participated in the evaluation? What, if anything, did you learn as a result of participating in the evaluation process? Can the findings of the evaluation potentially influence how the MHSAA interacts with the student-athlete members of the SAC? Data Analysis All data collected for this dissertation evaluation study was qualitative in nature. The collection procedures outlined previously were in line with qualitative inquiry methods designed to understand and explain the meaning of social phenomena, in this case the MHSAA SAC (Baumgartner & Hensley, 2006). In qualitative inquiry, the researcher is the primary instrument for data collection and data analysis (Baumgartner & Hensley, 2006; Patton, 2002). Responsive evaluation theory also requires the evaluator to be deeply involved in the organization 47 undergoing evaluation (Stake, 2000). Therefore, the evaluator’s perspective on the context of the data collected also carries a significant, though necessary, influence over the analysis. Though difficult to assert when enough data has been collected in a qualitatively designed approach, the typical point of ending is referred to as data saturation. The data is considered sufficient when similar responses and descriptions have begun to repeat themselves (Beebe, 2001). In the particular study, the overlapping perspectives on the SAC program and experience within the SAC was atypically uncanny, making the analysis a very fluid and most accurate portrayal of the perspectives shared in the interviews. All analytical procedur es for data collected during Phase 1 and Phase 2 were conducted following the guidelines specified by Creswell (2007) for a grounded theory approach. Grounded theory. “The intent of a grounded theory study is to move beyond description and to generate or discover a theory, an abstract analytical schema of a process” (Cresswell, 2007, p. 62). Given that the underlying premise of the formative evaluation is to construct a model of the purpose and functioning for the MHSAA SAC, essentially a ‘program theory’, the grounded theory approach is the most logical and appropriate means for desconstructing the data and organizing an interpretation of the program. Furthermore, grounded theory requires that any participants in such a guided study would have all experienced the process and the program theory that is developed is rooted in the data shared by those participants (Cresswell, 2007). The resulting analysis from a grounded theory approach exposes a unique explanation for the specific processes, actions, and interactions on a topic or within a specific contextual setting (Strauss & Corbin, 1990). Data collection circle. In an attempt to organize how qualitative data should be captured in a qualitative study, there are seven aspects presented by Cresswell (2007) the investigator 48 should follow, described as a ‘data collection circle’. These seven aspects will be presented with a brief explanation of how this particular study aimed to successfully fulfill the requirements. First, the investigator must locate the site and/or individual(s) for data collection. The primary site for this evaluation occurred at the offices of the MHSAA, and the individuals interviewed were considered stakeholders in the program, those being the SAC student-athlete members and the facilitator, as well as the executive director of the MHSAA, and those recommended representative council members. The next aspect of the data collection circle involves gaining access and establishing a rapport with the individuals. This was accomplished through a personal invitation to meet with the SAC two years ago by the facilitator of the program, whom the primary investigator had met and worked with for two years prior. Rapport was established through observing and interacting with the student-athlete members for an entire year before presenting the notion of an evaluation as a study. The third step requires the investigator purposefully sample the individuals to take part in the data collection, or interviews. Due to the formative evaluation approach and lack of information concerning the functioning, activities and general design and purpose of the SAC, it was determined to bound the subject pool to the immediately available stakeholder group to gain the most current perception of the program and most relevant experiences and activities. In the grounded theory approach, Cresswell (2007) advised that the purposively chosen sample represent a homogenous group. In this study, those student-athletes currently involved in the SAC and the facilitator and executive director who most commonly interact with those student-athletes were considered the target sample. The fourth step initiated the actual collection of data. In this study, across both Phase 1 and Phase 2, data were collected through interviews, both individually and in a focus group setting (Phase 1). The fifth step involved the recording of information. All interviews in the 49 present study were audio-recorded (per the consent form) and transcribed verbatim. As a form of triangulation to further validate the data, all transcripts were sent back to the individuals to check for accuracy. The next step involved resolving field issues. In the present study, all participants were very willing to participate, with the only issue surrounding scheduling interviews, though this issue was hardly difficult to resolve. The last, seventh step, described data storage. All data were kept on the investigators primary computer during data analysis, then moved to a secure, encrypted hard-drive, per Human Research Subjects protection protocol. Analytical procedures. Regarding the specific aspects of analyzing data from a grounded theory approach, Strauss and Corbin (1990) dictated a three phase analytical process, marked the ‘open coding’, ‘axial coding’, and ‘selective coding’ phases. The ‘open coding phase’ involved the examination of the data (interview transcripts) for salient categories of information. These categories emerged from the purpose of the study and the questions posed to the participants, as well as their responses, particularly those responses that are repeated across participants. Coté (1993) proclaimed “although there is no one correct way of analyzing qualitative data, it is essential that qualitative researchers provide a detailed description of the procedures, decision criteria, and data manipulation that allow them to present the final results of a study.” (p. 128) Cresswell (2007) has arranged a six-step process to generalize how a grounded theory guided qualitative inquiry should progress from the culmination of data collection through presenting information. This process was built upon the foundational work of the grounded theory approach expertly developed by Strauss and Corbin (1990). First, the investigator must manage the data through the creation and organization of files. The second step involved reading through the data, in this case the interview transcripts, and creating an initial coding system based on the purpose of the investigation and the questions posed to the participants. Passages of 50 the text that were considered to be of most relevance to the purpose of the evaluation study or represented a notion of significance are often referred to as “meaning units”. Tesch (1990) defined a meaning unit as “a segment of the text that is comprehensible by itself and contains one idea, episode or piece of information” (p. 116). These portions of the text were selected as having a strong representation of the participant’s answers to the interview questions. Furthermore, Miles and Huberman (1994) posited that coding is analysis, and reflects how the investigator should differentiate and combine data. The codes are attached to the meaning units, which vary in size from words to whole passages of text. In the present study, the transcripts were read over two times. The first passage was to gain a basic understanding of the whole conversation. The second passage through the transcript was to highlight meaning units in the transcript that would later be placed into a stronger interpretative system, detailed below. Upon completing the review of the transcripts and having identified all meaning units, the third step involved describing these open coding categories. Initially, these meaning units, or codes, were categorized primarily by purposes of the study, then secondarily based on which question in the interview guide the respondents were replying (see Table 1 and Table 2). Miles and Huberman (1994) recommended that a conceptual framework, often shaped by the research aims, is the best defense against data overload in a qualitative study. All meaning units were copied from the original transcripts and placed into a computer spreadsheet to better organize and compare similar responses across participants. Initially, all data were kept organized by data source. The spreadsheet method allows for the analyst to view all meaning units a single document to gain a stronger, more singular understanding for the data collected from all participants. The fourth step is called classifying the data. In this step, each meaning unit from the open coding phase is reviewed independently to ensure best fit for the initial categories 51 designed from the interview guide. The axial coding phase followed this categorical review of meaning units. Here, the database was reviewed to better organize and preliminarily label the categories to best represent the central phenomena and the context that shapes those phenomena (Cresswell, 2007). In the present study, the meaning units were re-organized in the spreadsheet so that a better grasp of the categories could be described by individual responses. These coded meaning units were organized so that all related to one another in a coherent, study-important way, implicated by the governing structure of the interview guide (Miles & Huberman, 1994). Specifically, the cells within the spreadsheet were re-organized so that, for example, each phrase discussing the reasons why the MHSAA started the SAC were near one another, to ascertain the extent to which these phrases were similar or dissimilar, so the most accurate narrative could be presented. The fifth step required the interpretation of the data. This is when the investigator engages in selective coding, by slightly reducing the amount of codes for each category and organizing the story of the study, to best narrate the participants perspectives. Here, the data were organized into several figures to present a theoretical model of the processes under investigation. An initial matrix of primary themes, and secondary categories is created and formed. Finally, in the sixth step, the data are presented in a visual model (see Chapter 4). In the present study, these six steps were followed first for the data collected in Phase 1. The resulting model, per participatory evaluation theory, was presented to the research participants in the form of a program logic model. First, this logic model was proposed to the program facilitator to ensure it met the perspective of the MHSAA and included all the most relevant aspects of the program. Along with the other interview guide questions, participants were asked to respond to the logic model created from the data in Phase 1 to be sure it best captured their narratives and would be a 52 good model for the SAC program. The analysis for the data collected in Phase 2 also followed the exact steps previously discussed. The resulting display of information is the culminating mark of a grounded theory approach. The program’s theory is built upon the narratives and perceptions shared by the research participants (Cresswell, 2007; Strauss & Corbin, 1990). Triangulation of data. To help decrease the extent of researcher bias in the data, qualitative investigators should engage in various practices to ensure the accuracy of their data, and their analyses, known as triangulating the data (Miles & Huberman, 1994; Patton, 2002). In the present study, there were 3 methods of triangulation practiced to help decrease researcher bias. First, all interview transcripts were sent back to the interviewed participants to allow them to adjust or correct any errors in the transcripts. All participants were able to review and confirm that the transcripts were accurate, and on two occasions, participants revealed errors in the transcripts that were addressed before analysis. Secondly, the logic model captured from information collected in Phase 1 and based on the evaluators observations were sent back to the participants to ensure an accurate portrayal of the program. Lastly, the data analyses were reviewed with another, highly trained, qualitative investigator to ensure high scientific standards were maintained. Lastly, responsive evaluation theory requires the evaluator to spend a significant amount of time within the program (Abma, 2006; Stake, 2000), thus making the primary investigator a quasi-expert on the SAC program. 53 CHAPTER 4 RESULTS The present evaluation study had two distinct phases, representing the four purposes of the investigation. Phase 1 was designed to address the first and second primary purposes: (1) chronicle and evaluate the tasks and activities (actual and idealized) of the MHSAA SAC, and (2) to evaluate the perceived organizational benefits of the SAC to the MHSAA and its executive staff and administration. Phase 2 was designed to address the third and fourth secondary purposes: (3) investigating the perceived personal development gain or sense of empowerment experienced by the student-athletes, and (4) to investigate the experience of being evaluated. Results of the evaluation study are organized and presented according to each phase and discussed as the data resolves the related purposes. Phase 1 The first phase of the present study was aimed at describing the design and chronicling the tasks and activities of the MHSAA SAC. Additionally, Phase 1 sought to describe the perceived organizational benefit of the SAC to the MHSAA. These steps were accomplished through individual interviews with adult stakeholders (N=4) and two focus group interviews, separated by high school class (e.g., juniors, seniors), with the SAC student-athlete members (N=14). The adult stakeholder interviews lasted between 32 minutes and 70 minutes. The junior class focus group interview was 38 minutes long, and the senior class focus group interview was 30 minutes long. The analysis revealed two thematically organized schemas (see Figures1 & 2), which were then organized into a logic model describing the purpose and functioning of the SAC (see Figure 3). 54 Why does the MHSAA have a SAC? The first of the two thematic schemas represents an answer to the first purpose of the evaluation study (see Figure 1). The primary theme of “Why does the MHSAA have a SAC?” (“Why”) is detailed by accounts of why the SAC was created and perceptions of its central purpose. Secondarily, within the theme of “Why”, three goals (“Goals”) of the program were revealed. The first goal of the SAC is to gain a student’s perspective on high school sports and present issues facing the MHSAA. The second goal of the SAC is to promote the values of the MHSAA, both ‘to’ and ‘through’ students. Finally, the third goal of the SAC is to provide an environment for student-athlete leaders to become exceptional leaders; to develop and enhance already strong youth leaders and related leadership skills. Figure 1. Why does the MHSAA have a SAC? Initially, the executive director anecdotally relied on his own high school aged children, who were competitive athletes, and their friends to offer a first-hand account of the high school sport experience for student-athletes. This perspective allowed him to see how the athletes perceived rules and regulations, as well as gain an insight into issues the athletes themselves were having within the governance of the MHSAA. The executive director stated, “When my sons …graduated and moved on I no longer had that pipeline to what was being discussed in school, and the way that students were behaving and acting”. About that same time, the to-be facilitator of the SAC had approached the executive director about how such a program could be a great way to reach their young constituents. Agreeing to this prospect, the executive director 55 stated “At least a subtext of this was to have my own desire with a place to provide input to the MHSAA and to me for speaking, writing and planning. That had all left my life when my [children] graduated from high school. That coupled with the fact that we had some young staff here who at the same time were talking about the potential merits of this”. The argument made by the facilitator at the time of the creation of the SAC was “obviously everything we did was for the ultimate benefit of the student but we were not speaking directly to students or speaking with students. So…we intentionally set out to develop programs that would allow us to speak directly to students”. The students agreed, stating the purpose of the MHSAA SAC was “to have a bond between the association and the students”. The goals of the MHSAA SAC. Further prompts at what the goals of the SAC are or should be, presented the evaluation with three primary objectives. These three goals represent what has occurred as well as what participants think should occur regularly when the MHSAA engages with the student-athlete members of the SAC. The goal of the MHSAA SAC is to gain a student’s perspective on the high school sport experience in Michigan as well as present issues facing high school sports in Michigan to the ‘ultimate stakeholders’. Of most importance to both the adult stakeholders and the studentathlete members, was the opportunity and ability to share perspectives from the large governing body to actual participants in the sports. Often articulated was the notion that the adults get to make the rules, but it was the athletes who had to live by them. One adult representative council member stated, “we saw this as an opportunity to, um, bring together student athletes in Michigan and just discuss, um, what the MHSAA is doing and whether we’re meeting their needs”. The adults also wanted the student-athletes to “know that we’re there for them not just to 56 make rules and often tell kids no, uh, but there’s other things we can do and this was, this was where we wanted to start”. The students also presented a number of instances that the adults should listen to their perspective. On student argued that “the MHSAA makes rules but they don’t play high school sports, we do, so I mean we give students an opportunity to voice their concerns or their opinions otherwise they’re completely oblivious to how we feel and what we think”. Furthermore, the purpose of the SAC was presented from the student-athletes perspective that the MHSAA needed a student representation on issues “because they’re like, they’re adults and they’re running the MHSAA like for high school sports but none of them play high school sports, so we kind of represent the actual high school sport part of it”. The facilitator echoed the student-athletes sentiments by stating “it makes us as a staff more knowledgeable of truly what’s happening from the student athlete perspective in school sports”. Also, the executive disclosed that the “the goals from my standpoint are to serve us with input, be a sounding board, be an idea generator for our services here that relate directly to students”. The result of having such a goal, the executive director conceded, “there has been so much reshaping in what we have done based on what the student advisory council has taught us”. To promote the values of the MHSAA: both ‘to’ and ‘through’ students. The MHSAA values the unique educational opportunity high school sports can offer to its participants and spectators, yet has a strong orientation as to how sport participants should behave. The MHSAA wants to understand how best to reach student-athletes to share appropriate means of behaving as athletes and as spectators, and using the students on the SAC as a place to send their messages through to their peers has been a promising objective. Student SAC members agreed, stating that the MHSAA SAC experience has helped them to understand “just like sportsmanship, like 57 promoting that in like the most positive way” and how to “create like the best environment for [athletes]”. One student stated: “the MHSAA has like standards that they want, like sportsmanship standards that they want like athletes to live up to, and like through like the summit and stuff, we’re able to kind of like bring like the students actual point of view and like explain like how that’s possible”. Here, the student is referencing one of the programs in which the SAC is involved that they are able to help promote these values. The “Sportsmanship Summit” is a half-day workshop that the MHSAA hosts in four regions of the state, and invites up to 300 high school students to attend with coaches and athletic directors. The student-athlete members of the SAC present one of the four sessions at each workshop to their peers. In this session the SAC members open a conversation about what should be, and what shouldn’t be, acceptable behaviors for high school student-athletes and their audience to engage in. Sharing stories about instances of poor and positive sportspersonship creates a strong dialogue for the Summit attendees. Using the student-athletes to present the most appropriate behaviors to their peers allows for the strongest message to be relayed to over one thousand high school students across the state. The students stated this was important, because “when you hear it [sportspersonship rules] from kids that are in high school your age playing the sports, I think you’re more likely to listen and get involved. Like really like look at it a different way kinda, and you’re more likely going to listen to it instead of always hearing it from an adult or a coach”. The facilitator concurs that the student-athlete SAC members can “[deliver] messages that we give them and delivering them in a very positive way and using a voice that I can’t give”, referencing the peer-to-peer discussion as opposed to an adult-to-student message. 58 Another project that the MHSAA sponsors through the SAC is called the “Battle of the Fans”. In this contest, schools are able to submit short videos of their student cheering sections during the basketball season. The SAC student-athlete members select the top five schools, visit these locations and create a unique story and video, which is then posted online and a virtual vote takes place through social media websites. The student-athlete members of the SAC ultimately select the winner based on how well behaved, organized, lead, and loud the school’s cheering section is. The executive director state that this “Battle of the Fans” is: “probably the best way that we’ve talked about sportsmanship in the past decade or maybe 20 years. So here’s a time that the student advisory council had an idea and they had a lot to do with executing it. It has turned people on to talking about sportsmanship in a brand new way and more intensely than it has in the past. That’s a huge success, unique in the country so right now my evaluation of the student advisory council couldn’t be higher because they helped us reach one of our critical issues”. To offer an environment for great student-athlete leaders to become exceptional leaders. This third and final goal of the MHSAA SAC really speaks to the developmental opportunity that is presented to the members of the council. This thought was captured from a studentathlete: “like us individually too, like the people that are on it, its a huge like leadership opportunity too, like because, the things we do, like leading the sessions at the summit, like I would never have done that before”. First, to even be accepted to the council, student-athletes must be nominated by a coach or athletic director for demonstrating strong leadership qualities. The student-athlete must then complete an application detailing the leadership experiences and write an essay explaining their perspective on the purpose of high school sports. The SAC then provides a high-demand environment (as required by the first two goals) where student-athletes 59 are forced out of their “comfort zone”, as was often indicated by the SAC members. Another student-athlete reinforced this sentiment by saying “if you don’t know a lot about like leadership or like being a captain or anything like – you learn like how to be a better captain or better leader” through their experiences on this council. The two adults from the MHSAA representative council, who are also high school athletic directors, referenced the students they had nominated and that had been accepted on the SAC several years prior. One stated, “I think it’s one of the ways that I saw it, our student grow was um through his leadership. Through this [he] became a better public speaker, [it] was a step up and [he] worked in front of people”. Furthermore, this athletic director described during this athlete’s two years on the SAC, “I could see him become more self-confident and uh step up and… he just became a stronger leader by networking or uh hanging out with other strong leaders from across the state”. The second athletic director told a story about how their student made his college decision based on several experiences and conversation that he never would have been exposed to, if it hadn’t been for the SAC, “[he] made a college decision because he was able to ask questions and solve challenges that he wanted to make. I’m not sure [he] would have done that had he not [represented] the high school”. How does the MHSAA benefit from the SAC? The second thematic schema revealed from the interviews in Phase 1 specifically relates to the second purpose of the study. Particularly, this schema revealed how the members of the SAC think that the nearly 300,000 other student-athletes in Michigan benefit from having this group. Additionally, this schema describes how the organization, the MHSAA, benefits as a result of hosting an advisory council of student-athletes (see Figure 2). The primary theme of “Benefits” first considers the ‘General 60 Student-Athletes’ then describes the benefits to the ‘MHSAA’, divided into two sub-themes: (1) the ‘Personnel’ of the MHSAA, and (2) the ‘Brand’ of the MHSAA. Figure 2. How does the MHSAA benefit from the SAC? The notion of how the MHSAA benefits from the SAC was discussed to address the second purpose of the present evaluation study; to determine the perception of organizational benefit from having the advisory council. By questioning the organizational benefit, the evaluation is allowed greater access to determining why the MSHAA may continue to sponsor this program, as well as capture the “cost” of the program, not only in a financial sense, but also in staff time toward some grand outcome. Benefits to the general student-athlete. Though a stronger description of how the student-athlete members of SAC think that their non-member peers benefit is embedded in the results from Phase 2, some aspects were presented during the initial focus group interview. Particularly speaking to their presentations at the “Sportsmanship Summits”, the SAC studentathletes proclaimed “what we did there, like you could see in the [Battle of the Fans] video how much of an impact that [presentation] had because like there were schools saying that they’ve 61 never done anything like that before and now they have these like huge organized sections that were able to work together and we had a part in that”. The SAC student-athletes obviously felt that helped shift how sports were viewed in that particular school. Another student on the SAC attempted to create a better student section at their own school because of what they had learned on the SAC, stating “our students section like changed a lot this year, like it got better and got bigger and more positive cheers, and we had people, like other team’s coaches and ADs, like come up to like our coaches and our principals saying like that we had the best student section and like the best sportsmanship they’ve ever seen”. The benefits to the general student-athlete were essentially conveyed to imply that if students and schools opted to participate in a project directed by the SAC, then these studentathletes would receive more direct benefit, beyond just the representative voice. There were three projects discussed throughout the interviews with the student-athlete SAC members as being beneficial to the student-athletes. First, the Sportsmanship Summits were a series of workshops presented at four different regional locations. At each summit, schools from around that region would bring students in to meet with the MHSAA and work on defining positive sportspersonship in their schools alongside their athletic directors. Attendance at each summit ranged from 175 to 300 student-athletes. Four SAC members at each regional meeting would present a single workshop titled “Over the Line” four times each day, each time to a quarter of the participants. In this workshop, which was designed cooperatively between the SAC facilitator and by the SAC members, asked participants to literally step over a rope line if a certain scenario “crossed the line into poor sportspersonship”. The SAC students then debriefed complex scenarios with their groups with the goal to gain a better, student-driven notion of what should 62 and should not be considered appropriate sportspersonship. These series of workshops were also a great medium to advertise their second project: “The Battle of the Fans”. The “Battle of the Fans” project invited schools from around the state to submit videos of their student cheering sections during the winter basketball season to the MHSAA showcasing why they were the loudest, most well organized, and ultimately the best example of positive sportspersonship in the MHSAA membership. The SAC received 27 video applications, and the SAC student-athlete members decided on the “top-5” schools. These top candidates were then visited by the MHSAA SAC facilitator and a one to two SAC members. At this visit, the student cheering section leaders were interviewed by the MHSAA SAC facilitator and members, and then the MHSAA produced its own video to showcase for an online voting audience. Ultimately, the winner is selected exclusively by the SAC student-athletes. The third project the SAC worked on during the course of the evaluation was a revision of the “Captains 101” book published by the SAC class several years prior. This 20-page booklet highlights the most essential skills for high school team captains to possess, and steps to resolve common problems that team captain’s face. From the initial publication, the MHSAA has sold and distributed around 12,000 copies across the nation. Other, activities of the MHSAA SAC students include acting in a commercial in which all 16 members read a portion of the previously mentioned “Belief Statement” which is broadcast allover the state of Michigan, and they also hand out the championship trophies at the high school football and boys’ basketball state finals. The SAC benefits the MHSAA. One of the ways that the MHSAA generally benefits from the SAC is regarded as its reputation, both within Michigan and across the nation. The students reported, “I think like having students and friends and stuff like that, us being on the council, it gives a better view of the MHSAA, cause people, a lot of students could probably 63 think of it as this big governing body that we have nothing to do with you know? But having us really helps them connect more with it, feel they’re apart of it, instead of just being under it, you know?”, and “we’re out running around to our schools talking about the MHSAA… they gain recognition a little bit”. The facilitator stated it was difficult to ascertain the exact value of the SAC but did share that the positive feedback and reputation is “validating but I still can’t answer the question of ‘what real good has this done?’ other than I know that I wouldn’t have a captain’s book, I wouldn’t have a battle of the fans competition, and I wouldn’t have these summits run by these groups of kids”. The executive director praised the SAC, particularly for their work with shifting the culture of sportspersonship in high school sports and proclaimed: “Suddenly, without my help the student advisory council comes up with the greatest way to do this [discuss sportspersonship] maybe ever, at least in the past few years. So I couldn’t give them any higher marks right now because they seized upon an initiative that goes along with our core values, good sportsmanship in school sports”. The SAC benefits the MHSAA personnel. When the staff of the MHSAA get to interact directly with their student-athlete constituents, there were several benefits mentioned during the Phase 1 interviews. One of the representative council members reflected on her observations of the MHSAA executive director, “I have watched [the director] walk away from their meetings at different times with those kids, um, looking refreshed and knowing that some of the hard decisions that had been made, that had caused, uh, the media to go nuts with us, that those were the right decisions”. By interacting with the student-athletes the MHSAA personnel may be able to feel better about some hard decisions they have made, because the student-athletes, arguably the group most affected by these tough decisions, offered their validation and understanding. 64 The executive director himself commented on his own gain as a result of sponsoring this program: “Across the country they think I’m a cool director and I had almost nothing to do with it. So I get credit across the country for being a forward thinker, well this is part of that and I almost had nothing to do with it. So it reflects positively on the organization and my leadership in ways I didn’t envision when it first began”. The executive director also commented on the professional reputation of the SAC facilitator, particularly because of the amount of professional meetings he has been asked to attend and speak at about regarding his facilitation of the SAC program, “[he] is sort of a star right now across the country, part of it is that this [the SAC] has been a success. This benefits us as an organization, [the SAC facilitator] as a leader, and inclines us to make a better decision making process because we ourselves have gone through this and listened to our students”. Lastly, the SAC facilitator himself describes the personal benefits he receives from working with this group. In addition to the functioning and success of the group being a part of his professional evaluations with the MHSAA and executive director, the interactions with these young people “keeps [him] feeling young”. He explained, “I love being a coach…and what I love about coaching wasn’t the end game strategy, or the x’s and o’s, it was building a relationship with kids so they were able to see an adult as a positive role model. So I am still able to have that coaching role; while not on a ball field, it’s with these kids”. The SAC benefits the MHSAA brand. The image and connotations about an organization that is projected to its stakeholders and peers groups can be considered its “brand”. This element of the data analysis revealed how the student-athlete members and adult stakeholders felt the SAC added to the positive image and brand associations. Specifically referencing individuals in 65 Michigan, the facilitator described, “there’s certainly a perception piece where now that we’re speaking directly to students, we now have relationships with the student athletes for whom we serve…it’s a perception that we’re open to new things and new ideas that the younger folks have to say”. The facilitator continued describing this brand image as how the MHSAA is perceived beyond their own office walls: “if others see us being inclusive of students and listening and being there for students, that’s how we want to be known as an association. We do all different kinds of things to put that message out there with the advisory council being one of those pieces”. The student-athlete members were well aware of this brand image and recognition the MHSAA receives due to the SAC and the result of many of their projects. One student referenced a conversation with the facilitator, where she described, “[the facilitator] was saying how a bunch of other states like look at the MSHAA and say like ‘I wanna start a group like the student advisory council in our state’ so I think they benefit a lot from our group for the past couple of years to branching out on a national level instead of just a state level”. Another student commented on this invitation to speak about their group, indicating that, “[the facilitator] is going to talk to people from all over the country; I’d say that’s a pretty good measure of how well its doing”. Lastly, the executive director described their brand image as being ‘progressive’ in a field where often times groups like the MHSAA are too conservative and determined to just make rules and regulations. He stated that the SAC helped shape their ‘progressive’ image, claiming: “to be seen as progressive by your peers gives you confidence that they are not backward and also your critics can see that you are one of the more progressive in the country. Across the country most people think their state high school association is a good ol’ 66 boys network based on making rules that are out of date. That’s a characterization, so to be seen in a progressive light is good for an organization”. The purpose and functioning of the MHSAA logic model. Following the participatory evaluation theory approach. The results of the analysis from Phase 1 were presented to first the MHSAA SAC facilitator as a draft of a potential program model. The facilitator being the primary evaluand and contact for this evaluation study, remarked that the model components accurately captured the results of the Phase 1 interviews and his overall perception of the SAC program he had facilitated for eight years. His email message to the investigator stated: “I have no changes. You’ve done a great job encapsulating what we try to do with SAC, something I have not been able to do in the 8 years with this council”. The logic model represents the resources required to facilitate the SAC, along with the goals previously described. Adjacent to the goals is a reflection of activities of the SAC, followed by measurable outputs, larger thematic outcomes, and lastly the broad impact of the SAC for the MHSAA (see Figure 3). 67 Figure 3. The Purpose and Functioning of the MHSAA SAC – Logic Model 68 Figure 3 (cont’d). 69 Phase 2 As previously stated, once the logic model (see Figure 3) had been approved by the evaluand, the Phase 2 data collection procedure began. Phase 2 was designed to meet the third and fourth purposes of the present evaluation study. Specifically, Phase 2 was designed to achieve Purpose 3 - investigating the perceived personal development gain or sense of empowerment experienced by the student-athletes, and Purpose 4 - to investigate the experience of being evaluated. These purposes were fulfilled through individual interviews with the executive director and SAC facilitator (N=2) lasting 32 minutes and 42 minutes, respectively. Additionally, individual phone interviews were conducted with each of the participating SAC student-athletes (N=14) lasting from 20 minutes to 42 minutes in length with the mean being 28 minutes. The analysis revealed five thematically organized schemas, with a complex interrelationship amongst four of those schemas. It is worthy of note that the adult participants (executive director and SAC facilitator) interviews were analyzed along with the student-athlete SAC members to fully interpret a singular meaning of the program from diverse perspectives. The overwhelming similarity in their responses to related questions indicated that the program stakeholders are generally in tune with one another, however, the separate parties (adults and students) were unaware of their like-minded thinking. The results presented below should provide ample explanation and allow for a more streamlined conversation in the MHSAA and SAC determining the future of the program. The SAC changed me. The first thematic schema revealed from the participants during Phase 2 of the evaluation study offers and explanation to how the adults and students feel that the program changes or aids in the development of the young people selected as SAC members (see Figure 4). All 14 student-athletes reported that varying aspects of the SAC experience changed 70 them as young people. Particularly, this schema describes the changes experienced by the student-athletes broadly and includes a gain in perspective about high school experiences and lifestyles in Michigan, an increase in their comfort and skills in social settings, and finally, varying increases in their leadership ability. These leadership gains are further discussed through the increase in leadership confidence, particularly in learning to articulate and share their opinions without fear of judgment. One senior girl exclaimed that participating on the SAC was “the opportunity of a lifetime”. A more thorough description of why the SAC had changed a junior boy was shared through his description: “I think that being involved in something that’s such a, like its known and respected by a lot of people, and it’s also like, so much bigger than anything I’m involved in you know, where I am. Its, it empowers me, you know, it makes me feel good that I was chosen to be a part of something that has such an impact on so many more people than anything else I’ve had the opportunity to do”. In a more specific incident, one senior boy reflected on a technical foul he had received a year prior to his participation on the council. Participating on the SAC encouraged his reflection on that incident and the more appropriate sportspersonship behaviors an athlete should display, “sportsmanship is everything and I, you know, I look back on my high school career and I still th look at the one technical foul I got when I was in 10 grade and gosh, how much of an idiot I was, because that wasn’t sportsmanship”. 71 Figure 4. Phase 2 Thematic Schemas. 72 Gained perspective: “Michigan View”. The MHSAA SAC is deliberately composed of a diverse group of students representing schools of all sizes, regions, public and private institutions, and social classes. Because of this diverse selection of students, SAC members are exposed to a variety of perspectives regarding similar issues that most high school students must cope with in adolescence and maturation development. The facilitator of the SAC refers to their worldview being expanded, at least in their home state: “they get to talk in a safe environment with other kids who are going through the same experience – playing sports, being in high school, having relationships – but those other kids are doing what they’re doing in a completely different environment… it allows them to see a bigger kind of world view or at least Michigan view of what’s really happening in Michigan… They get out of their own local bubble, and so I think they are, just their understanding of their place in sports, and school sports in high schools, is expanded just by being around all these other kids” The executive director echoes this expanded view, “I just think that interaction with fellow students, finding or discovering how they are alike and how, uh ,their situations are different is really healthy”. Of course, the students had many descriptions of how their view had been expanded by interacting with one another in the SAC setting. One junior boy commented that meeting students who play other sports than he does, opened his view of the high school sports setting, “its allowed me to see like more outside than just what I’m used to, like, I’m used to football, basketball, and baseball guys, like hanging out with my friends, and all my friends play the same sports pretty much, so I don’t see as much of a view from outside like other sports and stuff so its interesting to see how other people have different viewpoints”. Another junior boy described 73 interacting with the SAC senior class before him and his desire to emulate their confidence and leadership style, “being surrounded by, I think the seniors probably had a little bit to do with that. Like, they were, I guess, role models for me, you know. Being on it for a year already and being experienced, I remember going into camp last year and you know, thinking they’re such, they’re so developed and, and comfortable and good at what they do already”. A few of the seniors also had some very powerful comments on how being on the SAC had changed their perspective. One senior boy described presenting at the Sportsmanship Summit and meeting peers from all over the state to learn how they perceive appropriate and inappropriate sportspersonship behaviors, “like I said before we have the 16 kids…they’re all from different backgrounds, that’s interesting. But then [at] the summit that’s magnified. Its hundreds of kids from different backgrounds and you could see what…they’re about and when we’re crossing the lines you actually got to see what does cross the line and then get to ask people, okay why did that cross the line and get their opinion”. This same senior boy also comes from a small town and talked openly about how its difficult to become comfortable with strangers when he had grown up with the same friends since he was very young. However, participating on the SAC and presenting at their summit had helped him become comfortable learning that people from different backgrounds are not all that intimidating. Stronger social skills. While meeting, working, and interacting with a variety of other high school students helped enrich their perspective, this experience also added to their ability to be comfortable and interact with others in a social setting. The students described their rich friendships, as one junior boy stated, “I feel that I made a lot of new friends with the new council and I became closer with my own council than I ever was last year, so it really helped me interact and develop relationships with people”. A junior girl reflected on her expanded social circle: 74 “overall, I’ve met a lot of different people. Um, like before, I mean…I’ve never been like a shy person, but um, I’d say its definitely like made me talk to people I wouldn’t have otherwise talked to and gotten to know people…having to be willing to meet new people and get along with others like right away. That was, kind of something new for me…I definitely noticed all the friendships and stuff that I didn’t have before”. Another senior boy related his recent college orientation visit to the bus ride up to the SAC overnight summer camp on the day he first met his SAC peers. This new experience for him facilitated empathetic feelings with others, on which they could form a friendship: “I’ve been in the situation before with at least one other person I know so I’ll be in the situation of not knowing anybody but somebody I know so then I can kind of go up to him and talk or her and then we can kinda talk and then we don’t have to really approach the rest of the group. But then going on the bus you don’t know anybody, you haven’t met anybody so you have to make the effort to talk to other people and then that translates to orientation where I didn’t know anybody but then I just felt comfortable going up to random kids and then another thing that helps I guess would go on the bus helped me realize that everybody is kinda in the same situation here. Like talking about the sophomores coming up the student advisory council, so they don’t know anybody either so you can begin to maybe kind of feel what they’re feeling so you don’t have to be scared I guess or timid to go out and make new friends”. This new confidence in social settings was greatly attributed to the SAC experience, especially the overnight camp and presenting at the summit to a number of peer strangers. One junior girl summarized, “I feel like I’m much more comfortable going in to situations where I got to speak 75 to people I don’t know”. Another junior boy also shared that, “another skill I’ve learned is getting along with people who are from completely different areas and backgrounds”. Increased leadership skills. Interacting with such strong leaders in a single group can offer a unique experience that can shape future situations where these young people may be relied on or expected to take charge. The executive director felt that the SAC experience offered a place to begin this long development, and commented, “are they going to solve all the world problems? Not today, but maybe some of them are going to be leaders who will use some of these skills in their community; whether it’s a school board, whether they’re on a city council, whether they’re in a corporation trying to decide whether to do something that is only profit motive or is driven also by some social conscience. All of this I think fits together. And it just doesn’t happen overnight”. The facilitator also described the leadership development opportunity presented to these young people through the SAC experience and interacting with several other peer leaders. As mentioned before, to be accepted on to the SAC, these young people had to describe previous leadership positions. Noting this past experience and the general type of student accepted to serve on the SAC, the facilitator commented: “I think that is a HUGE benefit to their learning and then how they adjust when they get to college or with other leader group…they’re now surrounded by other leaders, whereas some of these kids at their school are THE leader in everything they do, it’s not even an option, you know. You [they] are going to be a leader on the football team or in student council, or in the class, and now they come into a room with many people that have the same role at their schools, so they learn how to deal with other leaders…I just think it helps better prepare them for when they become decision-makers in the future. And that 76 may be in college, it may be after college, it may be in jobs - that they are starting at a young age with thinking about things critically, seeing how things operate, you know being asked questions about how we can change things to make things better”. The students also described a number of instances where the SAC had led to a noticeable increase in their leadership development. One senior boy mentioned being the captain of his teams at school, “I learned things from it so an example would be like leadership like when we talk about what makes good leaders and even just talking with the other kids, not like an actual meeting but talking with them and seeing how they lead their teammates, I mean that changes… how I lead my teammates”. A senior girl who also had team captainship duties with her school sports teams described the SAC experience as, “I think it helped me develop my leadership… within my sports teams that I had, I just noticed I had an easier time like taking charge… it just felt like more natural to like taking leadership and like um, I don’t know giving people advice and stuff like that.” One junior girl new to the captainship duties during her first year on the SAC expressed that she often reflected on meeting and conversations with other SAC members. She said, “when a situation does come up at practice or something and something isn’t like how it should be – like the first thing that pops into my mind is like the meetings, like one of the meetings that we talked about it and we sit there and like talk about how people would do it, and I sit and think about what would be the best way to do it”. Another junior girl discussed creating a student cheering section at her school to compete in the SAC’s “Battle of the Fans” contest, describing that being in SAC prompted her to take on even more at her own school, “as far as like, leading and stuff, I would just by the amount of stuff that I’ve been involved in this year. Um, I mean I’ve always been like, involved with stuff at school but I definitely took on, like a bigger role. 77 There were people like coming to me asking me questions and stuff that, I mean, before wasn’t ever like that”. Gained Confidence. More specifically mentioned within the student-athletes description of their enhanced leadership, was a notion of gaining confidence from being accepted and participating in SAC activities. One junior girl, who had trained to compete on a national level in her sport, opened up about how a previous injury that ended her career had also impacted her confidence. She described how applying for this position was one of the first times she had really put herself back in a major arena, “after my concussion, I, I doubted everything. I wasn’t sure of what skills I really had and um, then like kinda doing all the paperwork and then submitting [the application] was kind of a big step too because that was like just throwing myself out there uh to being rejected too and then I got accepted and I was confident”. A senior girl described how nervous she was to present at the Sportsmanship Summit, “I thought the summit helped a lot in like, uh, kind of like commanding the room and like, uh, I don’t know that was like a really big thing for like me because I never would have thought I would have been able to do that before. Especially with people my own age”. A senior boy also echoed the impact of presenting at this event, “I wouldn’t have gotten up in front of a lot a people, especially people that I thought might judge me, um, but after you know speaking at the sportsmanship summit, not even knowing anybody”, he felt more confident in himself. Lastly, a junior boy shared how he was shy when he first participated in a SAC meeting, “my first meeting, I was definitely a little bit more shy and maybe not willing to talk as much and by the end, I have the confidence in myself to just say what I thought and stuff and that’s still happening now. I’m still growing more, I still have that other year so um, I think its given me more confidence too”. 78 Ability to share opinions. The facilitator of the SAC first shared this opinion about the SAC student-athletes, “They have a chance to talk with adults in a manner where the adults are listening and want to understand things, so I mean…giving an opinion that makes sense is a skill they hone, some of them learn all together when they are here”. One junior boy shared that he had to learn to “just share your opinion, not worry about what’s right and what’s wrong but sharing your opinion with the group; you don’t have the same opinion as everybody else and that’s still okay and I think that’s really important and I think I learned that a lot throughout this council experience”. The SAC student-athletes are asked to share their perspective on the direction of their projects and are also occasionally presented with issues facing high school sports. Many of these young people shared that being on the SAC was a place they learned to defend their perspectives and opinions. One senior girl commented, “I think also we kind of learned to stick with our opinions. I think especially with high school, middle school, you kind of try and see what everyone else, what their opinions are, and I think that within this group, you kind of, I at least, no whatever your opinion is, you stick with that”. Another senior girl reflected on these experiences through SAC and stated, “I could just tell that like each time it got easier to present to the kids and since then it really has been a lot easier to like voice my opinions”. When the students discussed an increase in confidence to share their opinions, some contextual clarification is necessary. These students discussed how in high school, many students are tempted to either align with peers to form friendships, rather than truly state their opinion. Other times, students merely remain quiet for fear of hostile peer judgment regarding their perspectives. This ability to feel more confident to state their opinions in a group of peers 79 without fear of judgment and toward enhancing the MHSAA is an extremely valuable skill to hone at such a young age. It empowers me. The student-athlete SAC members were asked if they felt they had any power over the decisions that were made surrounding their experiences (see Table 2). Replying to this question, the students expressed a number of instances where they felt a true sense of agency over the SAC program and were often driving the direction of the projects and program itself. In the instances where the student-athlete members of SAC were representing the MHSAA, they spoke about feeling important and that others were aware of their sense of power, for example, when visiting schools in the “Battle of the Fans” visits, or presenting trophies at the state championships. A senior girl commented on the sense of responsibility and prestige she felt at the SAC meetings with the executive director, “I really enjoyed when [the executive director] would come in. Like I was saying before, you feel responsible and knowing that he was there, you were like ‘wow! This group is important’. It’s really important to the MHSAA”. A senior boy further echoes this sentiment of being important to the MHSAA and the staff, “they really count on us to help them out… I believe [the executive director] counts on us to give him a lot of insight, and [the facilitator] does too”. Also speaking about the meetings, a junior girl commented on the privilege she held by being able to bring issues from her hometown to the main offices of the MHSAA, “[MyTown] is a small town, no one really knows or no one really cares half the time like – but if I take what we say and what we think to [them] and get it to the MHSAA, like the little town of [MyTown] will be heard and everyone will know [about us].” A junior boy also described how his presence on the council helped facilitate a closer connection between his small town and the MHSAA, “I 80 guess like my friends and teammates and stuff, kind of see it, see what I am as like a link to that, that whole governing body and a link to the sports and that’s kind of been a cool feeling, that its, it seems so big and vast and controlling of all of the schools in the state and we’ve never really had a connection to them before until, until I got on the committee”. Lastly, a senior boy reflected on his two years with the SAC, summarizing, “Its something that I really enjoyed and its something that I’m going to be able to look back on and its going to be pretty cool to be able to say I was apart of the SAC that had 16 kids that represented the whole state of Michigan”. The facilitator of the SAC also described one way he attempts to facilitate a sense of empowerment by being a trusted adult that the young student-athlete members can express their opinions and recommendations too. He feels that the student-athletes are the best source of the actual experience of high school sports and they should be taken seriously. He commented, “they’ve told me some things they want to see change at the meetings. Done. I mean I want to change things so that, one I think its good for kids to see “hey we think this would be good if we changed this, and to have an adult say alright let’s do it, let’s see if it works.” But two, they are the ones that are experiencing these meetings, you know, they know what’s boring and what’s working and what’s not, so I better listen to them”. My opinion matters. One of the more specific sentiments the student-athletes shared regarding a sense of empowerment on a governing body for their sports, was that their opinion genuinely mattered to the staff of the MHSAA. They expressed a cognizant understanding that they often did not possess the power to change rules and policies, but felt invited and welcomed to share their opinions. It was these opinions and ideas that the executive director and facilitator of SAC felt helped shape their jobs and messages about high school sports. 81 A senior boy described this theme through his comment, “I feel like it matters immensely because um I actually an athlete and I feel that the MHSAA the student advisory council we are actually the athletes we’re the ones that all the decision are affecting. And I feel like our opinions matter a lot.” A junior girl expressed, “probably quite a bit, like cause that’s why its there, they use us to see how things are going or how things can be changed – I feel like we’re there to help them with that”. The executive director described how working with these young people and listening to their opinions help shape the messages, and the delivery of those messages, that the MHSAA presents to the rest of the state, “They influence to a great degree what the people who are working with them, think. Umm we’ve learned from them what kind of technology they use and they don’t use and so as that translates to the way that we should communicate with them as an organization”. The facilitator also shared: “their perspective on their experience is powerful, specifically to [the executive director] as he listens or as I talk to people, whether its these kids or at a captains clinic, to kind of gauge what the climate is with school sports and what kids are getting out of it…we lean on the SAC and their understanding of what is fun and what is sportsmanlike and – who they want to show as an example to everyone else is an appropriate way to be sportsmanlike” Lastly, a senior boy who had been on the council two years reflected on what it was like for adults to listen to these young high school students: “the great thing is, you know we’re only 16, 17, 18, but our opinions are heard and that as a young person is really cool. A lot of the time you go to school, and [they say] ‘well you’re just a kid, you don’t understand’. So its really cool to be a part of something 82 where you are taking part in it everyday, so you should have an opinion on it and your opinion’s probably correct or real close to correct, so it’s cool to be on something where that happens”. Project power. When describing specific instances of power, the student-athletes often retreated to their major projects. The student-athletes on the SAC for the MHSAA have engaged in 3 major projects over the period of the evaluation. The first project was the presentation during the Sportsmanship Summits. Here the student-athletes designed a session for high school students to discuss what behaviors “crossed the line” into poor sportspersonship, and what behaviors were acceptable. The 16 person SAC was split into groups of four, and each group gave this presentation four separate times at the different Sportsmanship Summits around Michigan. The second major project was the “Battle of the Fans” contest. The student-athlete SAC members continued this event from the year prior, updating the rules and regulations, and then reviewing application videos. The SAC then selected the top five applications. These five schools were then visited by the MHSAA and a few SAC student-athletes, to observe an entire game with the competing student section fans. The SAC then discussed and selected a winner based on the criteria established by the student-athlete members. Lastly, the third project the SAC worked on during this evaluation study involved editing, revising, and updating a book for team captains that the SAC had written several years prior. One senior girl said, “I know that like we definitely had power in the things that we came up with like, um, like battle of the fans that was all us”. Another junior girl commented, “We had the power to do, take that whatever we wanted whatever we felt like would best work. We had an outline for some but we didn’t necessarily have to follow that outline so I think we had a lot of the power”. A junior boy also summarized the power of the SAC over their projects, “I really 83 think we did have a lot of voice in that. Whether it was the summits, or the [Battle of the Fans], or even editing that captains book and stuff. That was mainly us, I would say. And not just one kid, it was all of us had equal power and responsibility in that I guess, so yea I think we had a lot of power in those things”. The facilitator of the SAC also commented on how the student-athletes have power in their projects, and the lasting impact of those projects through the MHSAA and over time, “they’ve done things, like the summit or the [Battle of the Fans] or writing the captains book, that over time that has made a powerful impact on other things we’ve done. On Sportsmanship, their role in helping with [Battle of the Fans], is making a powerful impact on sportsmanship, not on governance, but on the way people view sportsmanship [in Michigan high schools]”. We benefit other student-athletes. A third instance where the student-athlete SAC members described a sense of power in the MHSAA was in the ways that the almost 300,000 other student-athletes in Michigan high schools may benefit as a result of this program. Recognizing that sense of power and responsibility was well discussed and related to how these 16 young people feel about their participation in this program. Within this theme of ‘We benefit other student-athletes’, there were secondary themes describing the two unique ways in which their peers could benefit as a result of the SAC program: through (1) being represented as the voice of students at the MHSAA, and through (2) the participating in projects facilitated by the SAC. We are the voice for the students. The title of this secondary theme comes directly from one of the student-athlete SAC members during the Phase 2 interview. Many others also echoed this sentiment, describing along with the purpose of the program, was to be a representative voice for all the high school student-athletes in Michigan. One junior boy stated, “I feel like that 84 [other student-athletes] like idea or view on something would be able to be benefitted because, I would be able to help like share their …share what they think”. Another senior boy stated: “high school sports are becoming crazy nowadays, with clubs and everything else that goes on and sometimes that can get overlooked, especially what the student’s going through or what, you know, the student’s perspective or the athlete’s perspective is, so, yea, I feel like if something really needed to be changed that [the MHSAA] would get the student’s perspective and we would definitely benefit [other student-athletes]”. A senior girl also expressed the importance of having student representation at the governing body, because the adults at the MHSAA are not actually playing the sports or able to see the unique nuances from the athlete’s perspective. She shared: “I feel like, like the adults, they think they know like exactly like what’s going on and like what the big issues are but then like we’re actually there and there’s like a lot of stuff that even the coaches of teams don’t really know that goes on but where you’re on the team and you’re like basically behind the scenes and everything that’s when you really get like the full feel of everything that goes on in like high school sports”. Project participation. In as much detail as the SAC members expressed their sense of power over their projects, they also felt that other student-athletes who took part in their projects (i.e. competed in the “Battle of the Fans”, attended a Sportsmanship Summit, or read the book for team captains) received a substantial benefit. By helping these other student-athletes directly, the SAC members sense of agency and power over the high school sports experience was elevated. One junior girl felt that just sharing appropriate behaviors for student cheering sections at the Sportsmanship Summits had a better impact than when adults laid out rules and 85 regulations, “rules coming from the same age group, they are a lot better than rules coming from a principal or athletic director, or news coming from the same age group does the same thing”. A junior boy shared that he felt schools who participated in the Battle of the Fans contest were subtlety forced to change how they think and behave in student cheering sections. He expressed, “As far as the projects, as far as [Battle of the Fans], I think that definitely changed the outlook on a lot of schools’ students sections, and how they cheered for their teams, I would say that definitely improved a lot, for the schools that were doing the competition”. Another junior boy expressed that because of his membership on the SAC, his high school attended the Sportsmanship Summit, and once there, was motivated to create a cheering section that hadn’t previously existed, “if it wasn’t for my participation on the SAC, my high school would not have went to the [north] sportsmanship summit and if my high school didn’t go to the sportsmanship summit we wouldn’t have had an organized student section last year like we did”. ‘It empowers me’ thematic schema summary. Though Phase 2 revealed a significant narrative of the ways in which the student-athlete members felt a sense of power and agency over their experience in SAC, this sense was equally balanced with a sentiment of a lack of power, conceived as being underestimated in their role within the MHSAA. This is discussed in more detail along with the thematic structure revealing the recommendations the student-athletes made toward the MHSAA. Recommendations. During the Phase 2 interviews, the SAC student-athlete members were asked to express any recommendations or offer their opinions on anything they felt could be improved. Additionally, they were asked to offer any advice or recommendations to future members of the council to ensure their experience was as fulfilling as possible. Therefore, within the thematic schema of ‘Recommendations’, two primary themes emerged, a set of 86 recommendations for the MHSAA and a set of recommendations for future SAC members. Within the recommendations to the MHSAA, the notion of feeling underutilized will be explored as previously mentioned. Recommendations: To the MHSAA. The student-athletes overwhelmingly expressed their enjoyment with the MHSAA. As previously presented, the SAC student-members felt that they were well listened to and their opinions mattered. They also enjoyed creating and disseminating their projects which they felt benefitted the general student-athlete. As far as recommendations went, one junior girl summarized the general sentiment by saying, “I think the more they give us to do, the better”. Many of the recommendations echoed this sentiment, specifically in requesting that the MHSAA increase the number of times the SAC meets throughout the year from six meetings up to 9 (or one per academic month). The student-athletes also requested more access and interaction with the executive director. In the often sparse appearances and meetings where the executive director would appear, the students thoroughly enjoyed talking with him and participating in the activities he would present. Lastly, the studentathletes requested more input in the rules, regulations, policies, and procedures of the MHSAA. Each of these three requests for ‘more’ is explored in more detail below. More: Meetings. A primary interest in having more meetings was simply because of the enjoyment the student-athletes experienced. A junior girl said, “the only thing I can think of is have more meetings because I like going to them, it’s like one of my favorite things”. A junior boy felt that increasing the meetings would allow the opportunity to engage in more projects, “maybe more meetings like maybe, two a month or something like that just so we can have more time to get things accomplished, get things turned out”. Another junior boy very articulately described why he felt there should be more meetings: 87 “we have a group of 16 kids that are, you know pretty influential and uh, well respected at you know, their schools and their conferences and their communities, you know, throughout the state. So, I think we might as well get the most out of that as we can ‘cause its not too many times that 16 kids from 16 different schools that get together and make changes like we do already like the battle of the fans and stuff. So I think, to make the committee better, do more. Have more meetings, have more projects”. More: Executive Director. When it comes to having access to the executive director, the student-athletes enjoy their rare meetings with him. One junior girl commented, “what he did with us that kind of allowed us to think a lot deeper like into sportsmanship and stuff. So that was kind of cool. But, um, I mean, it seems like he does quite a bit, its just we really don’t see him all that often”. A senior boy also commented on how he wished the executive director would have made himself more present at the meetings, “the only thing that might be more beneficial would be if [the executive director] would show up, had been there more…he was the one that always asked us the most serious questions. The ones that always had us think more. Because he wanted to know himself like, what’s really happening in our lives and stuff like that and having him there more often might have helped him get more insight”. Two senior girls also commented on their desire to interact with the executive director more, particularly because they were aware of his power within the MHSAA. One said, “I know that [the executive director] is like really busy with a lot of things and stuff but I feel like we only saw him like once or twice throughout the whole year and he’s like the person when we think of, like the one that like makes the changes”. The other senior girl remarked, “when he came in, you were like ‘wow, he really wants us to do good things’ like the head of this bigger group that were apart of, like he wants us to do well, and wants us to help him out”. 88 Even the executive director himself commented that he should make a better attempt to interact with the student-athletes more. He enjoys the time he gets to spend with this group and expressed an awareness of how there is mutual benefit, for both the students and himself when he is able to present and talk with the SAC members. The executive director stated: “I regret that I’m more detached now… they will not have their full power, I will not feel the full benefit unless I spend more time with them. So that just has to, has to occur… it just reminds me how I interact personally, I gotta be there more often and make sure that I give them opportunities to speak when I’m in front of them; not just me talking to them, but them to speak up to me, that’d be good for them”. More: Input. Generally, the students felt they had opportunities to discuss their perspectives and hear issues facing the MHSAA. It is because of this access that they expressed wanting to be further involved on matters within their governing body. Of most importance to the student-athletes was to be able to express their opinions on the rules that affect their game play. One senior boy shared, “I’m sure there’s not a whole, like a lot of instances where we’d even care but just once in a while maybe there are some like big rule changes or something along the lines that really affect kids like that would be nice to know”. A junior boy validated his opinion stating that it is hard to represent the voice of the MHSAA student-athletes if they are unaware of the issues to have an opinion about: “I think if they kept us better informed about what’s currently going on in the MHSAA and maybe some of the rules that are up for discussion and things like that. I think that would help us because we are supposed to represent Michigan student athletes and voice their concerns and their opinions but we really don’t know um what their opinions are on any of the issues because we don’t know the current issue and maybe part of its on me 89 but I think just an hour or half an hour every meeting and they could fill us in on what’s going on, some of the things that are being discussed and get our opinion on it”. Lastly, a senior boy also shared his desire to have more input on rules, because he and his peers have the best understanding of how a change might play out, because they are actually on the field of play. He stated: “I would say one thing would be to talk more about like, what we feel is wrong in high school sports…or could be changed, whether its rules, whether its…just in general, going directly to high school sports and see if there’s anything that we as student-athletes see could be changed, because I feel a lot of the people that talk about that are all people that have been out of high school for 20-30 years, and they are all older, and they necessarily don’t have the same viewpoints as the actual athletes that are playing”. We are underestimated. When exploring perceptions of power and agency over their experiences in the SAC, the student-athlete members felt that the MHSAA did not fully utilize the program. One senior girl expressed, “I know there’s like a lot of times when we were, when we’re like going into meetings and there was other meetings going on with like the adults, with like the other adults and coaches and stuff and maybe if we were able to like sit in on those meetings like and be able to participate; I think that they like underestimate, like the amount of help that we can provide I guess”. One junior boy claimed that the MHSAA didn’t access the full opportunity of the SAC, “You have 16 different kids that have tons of different new ideas that are ready to provide what they can to make it happen, you know? So I think that’s the opportunity that we need to take advantage of”. A senior girl summarized, “I do think it would take a lot of effort on them to involve all of us in their decision making, but I definitely think 90 they could do it, because right now, the SAC isn’t involved in the decisions”. Lastly, a junior boy said, “for their overall governing body functions, we are not really valued too much”. Policies and rules. The students shared their desire to listen to more issues from the MHSAA. The executive director stated directly, “in terms of uh what’s written in our documents, what’s written in terms of official policy, they have none”. The students expressed their knowledge of this fact. One junior boy said, “the things that are made like by the higher up people, like the director and the-those people that affect us, I feel honestly like we don’t have much say at all in that kind of stuff”. A junior girl stated, “we never really get a chance to um have a just conversation solely on how things are going or like the rules and I thought we were gonna have that chance”. A senior boy said, “we haven’t been asked to like ‘we want your opinion on this specific thing, this rule’ or whatever its kinda like whenever we hear about it its ‘this is the ruling what do you guys think about it’. Its not like ‘we were thinking about making this rule and what do you guys think about it’”. Falls on deaf ears. In the instances where the student-athletes were asked their opinion, or attempted to share concerns about high school sports, the responses seemed to indicate that their words essentially ‘fell on deaf ears’. The student-athlete members felt that while their opinion mattered to the MHSAA, it generally never left the meeting room or they were never privy to the results of the issues and decisions they discussed. One senior girl shared, “[they would] talk to us about [issues] but then like that was the last we heard like we didn’t even hear like what they decided finally on”. A junior girl explained that if she had an issue to share with the MHSAA, she was not confident the issue would be resolved, “I feel like if I went back and I told [the faclitator] about it like in one of our just conversations that I brought it up that it would, nothing would like really be done so I don’t really have any power… they do gain the student’s 91 perspective but they don’t really do anything about it”. Lastly, a senior girl stated, “I obviously enjoyed talking about with the other students and with [the facilitator], you know, it kind of always ended in ‘well, you know, the MHSAA has either already decided this or they are going to decide it’, and that was kind of that, you know”. Recommendations: To future SAC members. When the student-athlete members were asked to share their advice for future SAC members, to ensure their experience was optimized on the council, they unanimously replied with at least one of three sentiments. First, theme of ‘be open and involved’ was shared to express how important it was that new members attempt to become comfortable with their SAC peers and immediately begin sharing their perspectives, regardless of other’s thoughts or fear of judgment. Within this sub-theme, they also advised future members to ‘voice your opinions’, as sharing and debating issues is a preeminent purpose of being on the SAC. Lastly, the SAC members suggested that new members remember to ‘have fun’, as this experience was highly regarded by all the student-athletes interviewed. Be open and involved. “I would tell them definitely like, go outside of your comfort zone, don’t be afraid to do something that you are thinking ‘wow, this is going to be interesting’” was the remark of one senior girl. A junior boy eloquently remarked about the SAC experience that future members will “you know you reap what you sow sort of like you get out of it what you put into it” and suggested that new SAC student-athletes attempt to become as involved as possible. Another junior boy claimed, “people are so much more fun and everybody gets along great when you open up and you’re not just the quiet kid over there”. Voice your opinion. Specifically within the advice to ‘be open’, the student-athletes shared their desire for future members to share their opinions, and they should not be afraid of judgment. One senior boy stated, “I’d tell them not to be shy or like not to feel like they can’t tell 92 their opinions ‘cause there was never really a time when someone said something and everyone like tore them down”. A junior boy shared, “they [new members] can’t be afraid to speak their mind because we are all chosen because we possess like, we were good enough people I believe to like respect their opinions and what they have to say”. A senior boy stated his thoughts on the importance of voicing opinions when invited to the SAC, “don’t hold anything back because it still could make a difference and you never know if it could change that outlook on one kid and that kid could take it back to their high school”. Lastly, a junior girl also offered some comforting advice: “definitely pay attention in the meetings and like contribute their opinion. There is like no wrong opinion, so if they have a question and you have something to say about it – say it. Don’t hold back otherwise you’re going to wish you would have said it – or if even if someone’s opinion isn’t the same as yours, doesn’t mean its wrong. Like state what you have to say, no one will judge you for it”. Have fun! A junior boy advised the new members should not approach the SAC as work or a chore. He said, “don’t treat it like it’s a chore because honestly once you have fun with it it’s not a chore at all, it’s like something I-I for me I always looked forward to going to the meeting”. One senior boy shared his one regret during his time on the council as a form of advice for new members. He missed the overnight summer camp both years due to other obligations in the summer and felt that new members should work hard to ensure their schedule allows them to attend camp so they can meet everyone else. He said, “everyone got their initial bonding there and I kind of had to play catch-up throughout the year at the meetings and its kind of hard to get that bonding done through a meeting”. The theme of ‘have fun!’ is best summarized by a senior girl exclaiming that new members should attempt to enjoy themselves as much as possible. She 93 shared, “this is the experience of a lifetime, and so take it all in. Don’t, when you’re done have any regrets that you didn’t say this or didn’t say that, because its something that not very many people get to do”. Evaluation Participation. This final thematic schema revealed from the data collected in Phase 2 is based on a line of interview questions that are required of the participatory evaluation theory (see Figure 5). This thematic schema also represents the results specifically aimed to address the fourth and final purpose of the present evaluation study. Though the central purpose of PE theory is to create a shared meaning, it is the job of the evaluator to capture any nuances of the evaluation process itself that may have shifted the way stakeholders viewed the program. Additionally, this final line of questioning asked stakeholders to offer any final perspectives on the logic model presented as the result of Phase 1 interviews and how they may feel the MHSAA and the SAC program could or should use the findings of the evaluation study. Overwhelmingly, the stakeholders expressed a ‘positive and reflective’ practice through experiencing the evaluation. This represents the first theme. The second theme within this final thematic schema, ‘changes the way you think’, represents how several stakeholders were opened up to a new way of considering the council and the more minor details they had failed to previously consider as thoughtfully. When viewing the logic model as an orientation of the results from Phase 1 of the evaluation, the student-athletes and adult stakeholders felt that the diagram most accurately captured the interviews and the overall functioning of the program, thus having actively participated the creation of a shared meaning. Regarding the ‘future use’ of the evaluation, several student-athletes remarked that the logic model and findings would add focus to the upcoming meetings and projects of the SAC. This focus would aid in ensuring they are 94 able to work toward all three goals throughout the school year to maintain the positive impact as a long-term result of the MHSAA sponsoring the SAC program. Figure 5. Recommendations for the SAC. Though almost all the participants had positive comments to share on their experiences, it is important to note that the facilitator, and primary evaluand in the present study, held some trepidations at the construction of the project. He disclosed feeling a bit apprehensive on the matter, mostly because this project was designed to scientifically explore something he created without any academic or empirical aid. Regarding the initial agreement to participate, the facilitator revealed: “One, it’s a little scary. This has been, well just to get your participation from the start, not just the evaluation, but inviting someone from the outside the organization to sit in, because I went into this, without having this report that you are going to create, I didn’t have, I couldn’t read it beforehand, so we’d been kind of in some ways, making things up as you go, and then you bring in someone who can do reports like this and is evaluating leadership, much more critically, or differently, in a different kind of critical way than I am, makes you nervous that their going to find something that goes against everything that’s been written in academia about student leadership…[and] while I was nervous to bring someone in, I looked at it as a huge opportunity to have someone come in and say 95 here’s how we can make what you do even better, and even then, and then, you know, could help give us a document at the end that was like here’s your goals, that I haven’t taken the time, because I do so many other things, that I haven’t taken the time to really write down goals and specific strategies”. So though the initial apprehension and anxiety, it was a calculated risk on the part of the facilitator to explore his program, with the ideation that an evaluation would reveal how he could improve his program. This context represents a large reason for the choice to evaluate this program from a ‘use’ based perspective. Positive and reflective. Throughout the interviews in Phase 2, all participants were asked to describe their perceptions of participating in an evaluation study. The initial responses to this question revealed that the participants enjoyed the process and engaging in a structured reflective practice about their experiences in the SAC. One junior boy shared, “its kind of nice to like, look back and reflect on, like really, like what happened and what we did”. A junior girl also shared the sentiment, “I feel like that’s always a good thing to be able to reflect on experiences”. This theme also demonstrates the benefit of the evaluator establishing a strong rapport and trusting relationship with stakeholders in the evaluation. The facilitator expressed, “I took this process as, let’s do this because he’s going to help us make this better, and he’s going to come up with some things that can change”. A junior boy shared, “I mean honestly I’ve, I even kind of like it cuz now it’s like open up to, this is my chance to get how I feel, through”. A senior boy disclosed, “I liked it because you know, I feel like you are going to write the truth”, while a senior girl echoed, “There’s a lot of ideas that we have and um like the opinions that we have its good to be able to get them out cause we know that you’re listening and that you like understand what we’re trying to say”. These positive reflections add to the responsibility of the evaluator to 96 accurately capture the stakeholders views on the program, and add to the validity of the evaluation, particularly from the value-centered approach of responsive evaluation theory. Changes the way you think. When the stakeholders expressed how participating in the evaluation shifted their thinking about the program, their responses implied a deeper understanding of the process, and a revelation about their potential power to facilitate the direction of the council. A junior girl shared: “you just reflect on it and truly understand what, what we’ve gained and helps us appreciate it more and maybe even build more of an opinion and uh help us, even some of us like, some of us aren’t as talkative or don’t speak our mind as much and I think that also helps them realize that you know it’s okay to do that, we’re not gonna judge if you speak up”. This same junior girl further disclosed that engaging in the focus group interview exposed to her, that she and her peers had not been completely forth-right in expressing some of the changes they desired, “reflecting with you made me realize that we didn’t do that and I wish we did”. A junior boy from the same focus group, reflected, “we’d never really would have talked about having [the facilitator] do more things and then the idea wouldn’t have stuck in our heads and we probably wouldn’t have done anything new this coming year, but I think we will now because of that group interview”. A senior girl also shared her thoughts about how participating in the focus group interview allowed all the students to listen to one another, react, and come to a general conclusion about the program and their recommendations. She expressed, “during meetings we don’t usually sit around and like talk about, um like what we each think of the council. We haven’t talked about like things we want the council to do that year like our big projects and stuff, so when we sit around with the group we might hear like what each other has to say about 97 things, like [get a] different perspective”. Finally, another junior boy spoke about particularly interacting with the primary evaluator in this study, “I would say it opened up my eyes a little more to the SAC as a whole I guess, and our mission and stuff. The things you provided, the questions you asked, made me look more into what it is, I guess. So I got a better view of it because of what we did I guess, with you being there”. Logic model: ‘Hit the nail on the head’. The adult and student-athlete stakeholders in SAC were also asked to respond to the logic model created in response to the data collected from the Phase 1 interviews. The title of this theme comes directly from what one senior boy said when reviewing the model, “I think you hit the nail right on the head”. The facilitator of SAC also stated, “I’m trying to think what else I would add as a main goal and I can’t…there is so many things that I want to do that would narrow it down a bit, these are the main pieces I think, that this is right”. Another senior boy stated, “every single one of those I feel like that’s kinda what I’ve kinda been talking about”. A senior girl also commented specifically on the three goals (see Figure 3), “I really like all three of those, I feel like those definitely kind of capture what we were doing the last two years”. A junior boy shared, “I was very impressed at how much I thought it represented everything we talked about”. Lastly, another junior boy stated that the model was an interesting document used to capture all the nuances of the SAC and represented an accurate overview of his perspectives on the program. He stated, “The document [logic model], it was interesting to see how much is put in and then how much we actually get out of it. For meeting 6-7 times a year, there is a lot put in and a lot that we get out of it, so it was cool to see the overall impact it has and it really helps me realize how awesome it is to have this opportunity”. 98 Future Use. The stakeholders were lastly asked to share what they thought the intended use of the evaluation findings would be in the future. The overwhelming response, particularly from the student-athlete perspective, was as a tool that add further focus to their meetings, as well as giving a better idea of what future members of the SAC should expect and work toward. A junior boy shared his thoughts on the use of the evaluation findings: “I think that it will because the current, or the kids that are going to be in it will know that what the goals that we have are so I feel that we would be more inclined to try to accomplish them, rather than before when we didn’t really know what are goals were… especially for new members, like giving those three things [goals] to them, you know, ‘hey these are our goals’, I think that would help them get in the right mindset. Because going in, I was ‘well, I don’t really know what we’re going to be doing, or how this is going to go’ so I think those three things, they’re just easy to understand”. Specifically regarding the goals revealed from Phase 1, a senior girl shared, “I think that it helps when you’re new to the council and you see those goals and you’re like this is what I need to do”. A senior boy also discussed the future use of the evaluation findings, remarking about the goals, “the goals will keep them in check so then maybe at one point they start to stray away from what the goals actually are and they say ‘no these are the goals we set out, like these are what we need to accomplish’ or at the beginning of every term it’s like ‘there are our goals, we’re gonna accomplish these goals’ and then it’ll become a set thing like trying to do throughout the year”. Lastly, a junior boy shared that, “those goals will probably focus us and do as much as I can to obtain those goals – it helps provide a finish line I guess, something that we want to get done”. 99 The facilitator expressed a new found confidence in his program because of the evaluation approach to reviewing the SAC. He said in addition to helping him document the program, he would be using the findings to share with his peers in other state high school athletic associations. The facilitator disclosed, “its mostly to help our own group get better, um, but then yes to share…we have a lot of other states ask how we do it and why we do it, it just makes me feel more credible that we’ve gone through this process and its kind of been vetted, so it will make me feel comfortable talking more about it and being even more confident in how we do things and why we do things”. The executive director of the MHSAA also shared that the evaluation findings will also serve as a model of how the organization should think critically about all their programs and attempt to best understand the process of the many activities the organization sponsors and engages in. The executive director shared, regarding the logic model, “well it is a good snapshot. Um it will be also if we take it off the shelf and read it occasionally and remind us or double check are we still on the path? You know five, ten years from now ‘are we still doing the things that we thought were important here and for the right reasons?’... it serves as a model of what should happen more and more”. 100 CHAPTER 5 DISCUSSION Eight years ago, the MHSAA created the SAC to fulfill two basic roles: (1) to replace the ‘pipeline’ of information about high school sports to the executive director after his own children graduated, and (2) to create a stronger bond between the organization and its ‘ultimate stakeholder’ as a source of creative exchange and positive marketing. The evolution of the program came about by way of collegial exchange between the facilitator of the SAC and other state high school athletic association directors attempting to manage similar programs. This organic ebb and flow of the programmatic scope also changed as student-athlete members came and left. The facilitator discussed how the varying classes would decide what they wanted to do as a SAC and then direct their efforts toward that resolution, with the first classes developing the previously mentioned Belief Statement. It seems that each class had its own perception of the SAC mission and purpose. The specific projects the SAC takes on also slightly shift the focus of the program and the unique experiences of the student-athlete members. While it is not atypical for programs to progress in this manner; without evaluation, stakeholders may never have the opportunity to reflect on previous events, nor examine the existential purpose for why they engaged in their projects, how the projects may have been beneficial for varying groups, or the ultimate impact the program serves its larger community. The present study examined the purpose of the MHSAA SAC, identified program goals, and clarified the development potential for its youth members, as well as documented the unique benefits the SAC program provided for its organization, student-athlete members, and outlying stakeholder groups. This chapter discusses the findings of the evaluation, identifies implications of these results regarding the MHSAA, implications for those interested in evaluation in 101 kinesiology, and speaks to the larger implications for positive youth development. The strengths and weaknesses of the study are also reported. Phase 1 The first phase of the present study was aimed at exploring the implementation of the SAC program, as well as identifying the actual and idealized tasks of the program. Furthermore, the first phase was designed to identify the organizational benefits of the SAC as perceived by the stakeholders. Two thematic schemas were revealed (see Figures 1 & 2). The first set of themes derived from interviews with the four adults and two focus groups with the student-athlete members essentially answered “why does the MHSAA have a SAC?” and offered the three primary goals of the program: (1) to gain a student’s perspective on the high school sport experience as well as present issues facing high school sports in Michigan to the ultimate stakeholder; (2) to promote the values of the MHSAA both to students and through students; and (3) to offer an environment for great student-athlete leaders to become exceptional leaders. These findings are in line with the purpose of formative evaluation, which is designed to verify what the program is, help shape the program to function better, and provide answers that are immediately useful to the program management and program stakeholders (Rossi et al., 2004). Patton (2008) noted evaluators often become heavily involved in identifying goals through the course of the evaluation, typically due to a lack of identifiable statements detailing the specific goals of a program. Prior to the evaluation investigation, the closest forms of goals available about the SAC were subtle and embedded in a “belief statement” (see pg. 4) engineered several years earlier. The goals designed as a result of the Phase 1 interviews add further 102 clarification to the essential purpose of the SAC and will provide a more clarified approach for the MHSAA staff interacting with the student-athlete members. Using a responsive evaluation theory approach dictates that the primary purpose for the evaluative investigation is to produce knowledge about the program that helps its stakeholder make better, more informed decisions and lay a foundation for how success within the program should be judged (Stake, 2000). Providing a clearer set of goals for the organization will aid in focusing the efforts of the SAC in the future. These goals, which classify the essential purpose of the program, also represent the efforts of a participatory evaluation approach as the goals were derived from the stakeholders themselves, and represent an aspect of the shared meaning of the program. Though seemingly small, these three simple statements represent the direction the actors within the SAC should take. These goal statements may be the most important result of the evaluation for the stakeholders themselves. Offering a clearer set of objectives, unmasked from the actual tasks previously enacted by the SAC, as well as representing their ideal vision of the program provides the most utilization from the evaluation findings for future interactions amongst program stakeholders and the organization and its constituents. The essence of a usefocused evaluation is that the results provide an estimable amount of intended use in the near future by the stakeholders (Patton, 2008). Immediately, the data captured from the Phase 1 interviews, and presented in Figure 1 and Figure 3 provide a snapshot of what should happen with the SAC, based on what has already happened and how stakeholders perceive their purpose within SAC. The adults respondents also indicated that the findings would be used in the future. The second thematic schema (see Figure 2) revealed in the data captured from the first phase of investigation describes the organizational benefits of the MHSAA from sponsoring such 103 a program. These results answered the second guiding purpose of the evaluation: to evaluate the perceived organizational benefits of SAC. Essentially, this theme describes the outcomes of investing time and resources into this program. These data capture one aspect describing what the stakeholder’s value about the program, an essential element of the responsive evaluation approach (Abma, 2006). Several aspects regarding the resources required to run the SAC program were revealed through the two-year interactions and observation by the primary investigator. Though there is no definitive, quantitative, economic cost-and-benefit analysis of the program (which is beyond the scope of a formative evaluation approach), a broad description of what the MHSAA ‘gets out of it’ was reported. When considering the largest human component of the MHSAA, its 300,000 studentathlete members are perceived to receive a representative voice, common in democratic practices. By recruiting a diverse group of students, designed to represent the many varying demographic backgrounds of its student-athletes, the student-athlete members of SAC provide a direct insight from the playing field view on the high school sport experience in Michigan. Secondly, the personnel within the MHSAA offices benefit as a result of the SAC program. Here, the personnel are allowed to preview their messages to students to ensure understanding and appropriate rationale behind some decisions. For the facilitator of the SAC, the ongoing success of the group is an aspect of his performance evaluations. He also described the youthful sense he personally gains by interacting with highly articulate and intelligent young people. Guiding a group like the student-athletes of SAC provides him a ‘coaching’ role he enjoys. Lastly, a unique aspect of the data captured in Phase 1 describes a benefit for the MHSAA in terms of ‘brand value’. To this end, student-athletes feel a somewhat direct connection to the reputation of a large organization. Within the state of Michigan, and primarily 104 related to the second goal of the SAC described above, the MHSAA is able to enlist young people to spread the values of the organization directly to their peers. Through the efforts of the Sportsmanship Summit and Battle of the Fans specifically, this group has created a paradigmatic shift in how some schools perceive appropriate behaviors during basketball games and other sporting events. By sponsoring a contest, the issue of sportspersonship finally becomes a desirable outcome as opposed to some sort of unreachable standard imposed by oppressive adults. The event is fun for the competing schools and students. On more than one occasion, coaches, athletic directors, and even superintendents made comments to the facilitator, evaluator, and student-athlete members of SAC, about how the contest has “changed the culture” in their schools, especially on days with a basketball home game. In this regard, because the motivation for change has come through governing body, both the facilitator of SAC and the executive director described how this initiative begins to tear down the perspective that the MHSAA is merely and only a rule-enforcing agency. The young people spoke with delight about the ever-growing positive image the MHSAA gleans across the nation when the SAC facilitator is asked to speak with other state associations about their program and projects as well. The executive director also spoke candidly about how he has become perceived as a “progressive” director in light of inviting student-athletes to have a large and active role within the organization. The MHSAA has also published and disseminated almost 12,000 copies of the handbook for team captains that the SAC group wrote previous to the evaluation and began revising during the course of the present study. Requests for this text provide a validation to the young people that they created a worthwhile product, and the MHSAA logo is presented across the country as a unit that understands the issues high school team captains face. 105 Phase 1 of the present study provided the most useful information to the organization and set the tone for the second aspect of the evaluation study. Providing a clear set of goals to guide the organization will aid in the future administration and overall functioning of the program both immediately and in years to come. Applicants and new student-athlete members will have a clearer set of objectives to work toward as they engage with the governing body. Furthermore, the MHSAA now has a set of guiding objectives to ‘check-off’ each time they engage with their student-athlete members of SAC. The creation of a logic model represents the culmination of a participatory evaluation approach (see Figure 3) and offers a snapshot of the program, as designed through their insight and perspectives. This one-page document not only simply captures their program, but also provided an easy document to share with the inquiries the facilitator receives regarding this program. Phase 2 The second phase was designed to address the third and fourth purpose of the evaluation study, specifically to explore the sense of developmental gain or empowerment experienced by the student-athlete members, and then to examine the experience of being evaluated, per participatory evaluation theory guidelines. The large, interconnected thematic schema (see Figure 4) revealed from the data collected during the Phase 2 interviews essentially discussed two broad ideas: (1) how participating on the SAC “changed” the student-athlete members, and (2) recommendations for the MHSAA regarding their administration and interaction with the SAC program participants. Intertwined within these two broad concepts was the notion of how aspects of the program were ‘empowering’ to the young student-athlete members, but also how some aspects of the way the MHSAA engaged with these members led to feelings of being undervalued. Rather than 106 attempting to create a debt and credit system of empowering and oppressing actions to assess the program as one or the other, it was important to dispel an explanation of both experiences to provide the most detailed explanation. The explanation of ‘empowering experiences’ was tied to the theme of increased leadership, and the feelings of being ‘underestimated’ were best connected as evidence for adjusting the program along with other recommendations. Gould, Voelker, and Blanton (2012) explained that a looming barrier to youth development was the inability of adults to cede control of the contextual environment to the youth participants themselves. In this developmental effort, the MHSAA stands on the forefront of youth development programming, even though it may not be the primary motivation for facilitating the SAC. Over and over again throughout the Phase 2 interviews, the young studentathletes felt they had almost total control over their projects (Sportsmanship Summits, Battle of the Fans, Captains Guidebook), from the conception and creation, to the determination and decisions involved throughout the course of each project. Furthermore, specific experiences shared by the student-athlete members describe having had the opportunity to practice and apply several life skills identified by Jones and Lavellee (2009); particularly, their communication, leadership, respect, and organization skills in order to most appropriately fulfill the duties of a SAC member. Interestingly, the leadership experiences of the SAC participants and its adult leaders reported here are also consistent with current motivation theories that emphasize the importance of facilitating self-determined behavior by meeting individual needs for competence, autonomy and relatedness (Deci & Ryan, 1985). In addition, the most effective ways strategies coaches can use to foster such goals is by providing an autonomy supportive coaching climate and the ways in which the coach relates to their athletes (Cumming, Smoll, Smith, & Grossbard, 2007). Hence, 107 the results of this study are consistent with a great deal of the current research being conducted by sport psychology researchers studying youth leadership and optimizing student-athlete motivation. Along with optimizing student motivation, the utmost contributing factor posited for the overwhelming response of ‘yes’ to the question ‘did the SAC change you as a person?’, comes from the opportunity to practice initiative as indicated by Larson (2000). In many of the meetings and interactions observed by the primary evaluator, the students were given very simple prompts, and then allowed to discuss and brainstorm answers and further develop these ideas into action. Larson (2000) wrote that the experience of initiative, essentially a freedom to create something and the support to see the design through is most related to a sense of autonomy and a capacity for agency over experiences. Being granted the opportunity to change the high school sport experience for themselves and their peers, through their own design is most likely the single most likely ‘mechanism’ for which these students self-report changing as people. Utilizing a sport-centered topic for passionately engaged athletes essentially capitalizes on the recommendation put forth by Larson, Hansen, and Moneta (2006). Though the SAC is not a sport program, it is sport-based and therefore allows passionate sport participants a strong sense of agency over their sport experience, and to shift the experiences of their peers for the overall improvement of the high school sport environment in Michigan. This carefully designed and actively engaging environment to allow for students to practice initiative resonates well with the SAC members and exemplifies the very premise of enhancing social development and increasing intrinsic enjoyment and motivation. Deci and Ryan (1985) long ago posited that in order to move toward the enhanced social development of the individual, three primary ingredients were necessary. Individuals must feel competent, 108 autonomous, and related to significant others or means (Deci & Ryan, 1985; Ryan & Deci, 2000). In the MHSAA SAC, these ingredients, though subtle and indirectly facilitated, are expertly facilitated. By allowing the SAC members to the power to suggest, create, and design programs and projects, and then by supporting their initiative with resources to bring their creations to actuality aids in the promotion of autonomy. By listening to the student-athlete members explain their perspectives on high school sports and by inviting these young people to have a voice on issues facing the MHSAA, they are able to feel competent. Finally, by placing 16 strong young leaders together, these individuals are able to relate to each other in their gifted leadership abilities and work to further enhance their innate desire to engage peers and teammates. Additionally, by bringing these 16 young leaders to their peers through the “Sportsmanship Summits” and “Battle of the Fans” visits, they promote their ability to relate to the peers they represent when in their meetings at the MHSAA offices. Phase 2: Exploring empowerment. Empowerment has been explored extensively in the Community Psychology literature. Rappaport (1987) suggests that empowerment is a drive to gain determination over one’s life and democratic participation in the life of one’s community. “Empowerment calls for a distinct language for understanding lay efforts to cope with challenges, adapt to change, and influence our communities” (Zimmerman, 2000, p. 44). Speer and Hughey (1995) described the notion of empowerment as the manifestation of social power, which occurs in several levels of analysis within a social setting, those being the individual, organizational and community. Maton (2008) discusses the importance of empowering a community setting to encourage the members of that community to seek and create change to better their quality of life and experiences within that community. When the MHSAA and high school sports in Michigan are designated as the ‘community’, through allowing the SAC to 109 identify issues and create means in which to resolve those issues, the SAC becomes an empowering context, immediately for the 16 student-athlete members, and perhaps to other high school student-athletes who engage with the SAC projects. First, evident through the notion that the SAC identified a lack in clearly articulated resources about what it means to be a team captain and how to behave once one has earned the position, they wrote a small guidebook. Furthermore, the MHSAA values sportspersonship, both in its athletes and its spectators. The student-athlete members of SAC recognized that dispelling and limiting the behaviors through 'rules’ dispersed by the governing body (MHSAA) were both boring and oppressive. Taking a perspective on what was enjoyable about participating in a student cheering section, remaining in the light of encouraging positive behaviors, the SAC created the Battle of the Fans, and was fully supported by the MHSAA. It was in these events that coaches, athletic directors, and superintendents applauded the MHSAA and its SAC for changing the culture of home games during basketball season, and promoting a more fun and appropriate way to practice positive sportspersonship. The culmination of empowerment happens when community organizations have garnered the elements of, and the capacity to exercise, social power (Speer & Hughey, 1995). Maton (2008) furthers this illustration of achieving empowerment as an active developmental process “through which marginalized or oppressed individuals and groups gain greater control over their lives and environment, acquire valued resources and basic rights, and achieve important life goals and reduced societal marginalization” (p. 5). Elevating the student-athletes, who are often at the far end of the control spectrum of interscholastic sports, into the MHSAA as a guiding representative group, fosters the ability for these young people to become empowered. Rappaport (1987) also suggests that learning about empowerment is to study a setting in which we would 110 expect empowerment not to exist because of some environmental constraints. Typically, the governing body of sports programs makes rules and enforces these rules to restrict the behavior of its constituents. Before the SAC, the MHSAA had little to no input from the primary recipients of the rules enforced by a small group of adults. Allowing these young people the opportunity and the support to create change is the strongest contributor to a sense of empowerment expressed through the interviews in Phase 2, and thusly the largest contributor to changes in their perceptions of their leadership ability (Gould, Voelker, & Blanton, 2012). Miller and Campbell (2006) recounted that empowering processes “are those that provide opportunities for people to work with others, learn decision-making skills, and manage resources” (p. 297). Bringing together a diverse group of driven student-athlete leaders was the premier primer for the SAC to become an empowering context. Kelly (1971) suggested that true community assessment can only be successful when the professional removes themselves from the office and perspective of ‘expert’ and fully immerses oneself into a community and adopts the role of facilitator and promotes the community members from subjects to ‘participants’ in the intervention and alleviation of a communitydefined issue. When the facilitator of the SAC designed the program and executive director approved these decisions, these professionals essentially followed Kelly’s (1971) advice to the letter in creating an empowering context. The community-defined issues become the perspectives shared by the student-athletes and their ideas, often radical and progressive comparatively, in how to resolve these issues. Albeit, a mostly empowering context, particularly when discussing the agency over the decisions and directions over their own projects, the notion that many of the student-athletes felt underestimated and undervalued by the MHSAA must also be critically examined. This 111 juxtaposition of empowerment and oppression was quite interesting to discuss with the young student-athlete members. The old sage of ‘words falling on deaf ears’ is the most appropriate way to capture their combined sentiments. When asked about the ability to resolve issues in the governance of high school sports and share their perspectives on issues within the MHSAA, the student-athletes felt a stronger connection to the MHSAA than the many thousands of other student-athletes, but this only made them privy to hearing the rationale for the MHSAA’s decisions, as opposed to making a meaningful impact on the direction of those outcomes, this offering the notion that the MHSAA merely allowed the SAC student-athlete members to discuss an issue but gave no further thought or power to their sentiments. Both the executive director and facilitator openly confessed that the SAC had no direct governing power over the MHSAA, essentially having no vote in matters and issues they face as a governing body. This finding is closely related to the dissertation findings of Hendricks (2011) and the lack of policy power granted to NCAA SAAC members. However, the MHSAA executive director and the SAC facilitator both openly discussed how the perspectives garnered from the SAC readily clarified, expanded, and shifted the ways they think about their role within high school sports and how they attempted to articulate varying aspects of their decisions to constituents, the representative council, and at national meetings. This lack of reinforcement and transparency in how the perspectives garnered from the student-athlete members are used toward the governance of high school sports is an underlying source toward the disempowering sentiments expressed by the student-athlete members. The findings suggest that if the MHSAA were more transparent in how the opinions of the student-athlete members were used, then perhaps the notion of being underestimated would distill, and a stronger sense of their role within the MHSAA would blossom. It was never stated by any student-athlete member that they wanted 112 a vote in the decisions, nor did they express distaste in not currently having a vote, merely they wanted the opportunity to express their views as a representation of the student-athletes, and requested some follow-up for how their opinions were used, and toward what outcome their voices mattered. Speer and Hughey (1995) suggested that the dimension of social power is constructed of three elements: (1) the acquisition of ‘superior bargaining resources’, (2) ‘an ability to construct or eliminate barriers of participation’, and (3) ‘a force that influences shared consciousness’ (pp 731-732). By allowing the SAC student-athlete members to merely voice their opinions on matters facing the MHSAA, prior to the decision being made, and including a more transparent description of how their opinions influence the work of the staff at the MHSAA, then perhaps the MHSAA can begin to eliminate this barrier of participation, coined as ‘deaf ears’ toward their student-athlete’s perspectives. Phase 2: Evaluation participation. Examining the experience of participating in an evaluation with the stakeholders and participants of the study provided a validating and unique insight often left unexplored in traditional qualitative studies. Harnar and Preskill (2007) described this source of investigation as exploring the “conceptual use” of the evaluation findings; essentially, gaining an understanding for both the evaluator and stakeholder how the experience of being evaluated may have shifted or changed how they conceptualized the program. The conceptual changes through this evaluation seemed to have occurred in one of three ways. First, the student-athlete members, especially the senior students who were leaving the program altogether, remarked that the experience of being evaluated was a positive end to their term on the SAC and allowed them to fully reflect on their time within the program. The junior students also shared that the added focus of having explicit goals would help them stay on task in the upcoming second year of their term on the SAC. Amo and Cousins (2007) described 113 this reflection and deeper understanding leading to a “charged use” of the evaluation findings. This can cause stakeholders to require a deeper and more intense voice within their program and its facilitation. Ideally, a more focused orientation within the SAC toward its stated objectives will result in an even stronger, empowering developmental experience for the youth members, and further, more rewarding experiences between the MHSAA and the student-athletes, leading to an enhanced reputation, built on an organization who is progressive and listens to its student members. Discussing the creation of the logic model with the adult stakeholders (executive director and SAC facilitator) and the student-athlete members of SAC, also added a means of triangulating data for qualitative validation, as well as presented an opportunity to share in the creation of the meaning of the program. The logic model presented is the most representative piece from the evaluation findings related to the evaluation capacity building (ECB) posited by King (1998). First, in agreeing to participate in an evaluation dissertation project, the SAC and the MHSAA accessed and built an understanding of evaluation skills, and through the logic model and recommendations will be able to use the findings appropriately. Secondly, through engaging in evaluative practice, the executive director also mentioned that the process should become the model for how the MHSAA and like organizations self-monitor and reflect on their programming, thusly cultivating a spirit of organizational learning and improvement. Implications and Future Directions for the MHSAA SAC The results of the present evaluation study indicated that the goals of the MHSAA are to use the SAC to: (1) gain a student’s perspective on the high school sport experience in Michigan as well as present issues facing high school sports in Michigan to the ultimate stakeholder; (2) promote the values of the MHSAA both to and through students; and (3) to offer an environment 114 for great student-athlete leaders to become exceptional leaders. Furthermore, the results posit that the SAC experience is an empowering and developmentally appropriate context for the young student-athlete members, while also contributing to stronger understanding of the sport experience to the staff at the MHSAA. Finally, the SAC contributes to a positive brand image of the MHSAA across the state of Michigan and to its organizational peers across the nation. However, this positive brand image can only be implied through the perspective of those interviewed, and was not actually examined or tested beyond the members of the present study, as that investigation was far beyond the aims of the present evaluation. The current study was designed as a formative evaluation aimed at gathering and organizing a shared meaning of the SAC program, and not designed to cast judgment or determine the merit of the program, especially in terms of an economic cost-benefit analysis. In determining the design of the evaluation, the evaluability of a program must be assessed. Wholey (2004) advised the evaluator to first determine the goals of the program and the performance indicators. When exploring the SAC program, little concrete evidence was found for having clearly articulate goals, and therefore the primary purpose of the evaluation was shaped, to chronicle and evaluate the implementation of the program and the actual and idealized tasks. Using a participatory evaluation theoretical orientation, the above referenced goals were created. A logic model (see Figure 3) was also constructed as a source to indicate related activities and quantifiable outputs as indicators of the SAC performing to its essential purposes. These activities should be closely monitored and outputs constantly updated in the instance the SAC chooses to undergo a more summative evaluation. In that case, evaluators will surely need such data. 115 Overall, the MHSAA should feel a sense of righteousness for their approach to the SAC experience they facilitate, especially for the young student-athlete members. As previously mentioned, the MHSAA essentially stands on the forefront of cultivating an empowering, developmentally appropriate environment for young people to learn new skills and practice essential life skills that could be transferred to other domains. Recently graduated student-athlete members of the SAC openly commented how certain aspects of their SAC experience had already transferred to an easier adjustment to college life in their recent orientation visits. Utilizing an engaging facilitator and an executive director that openly expresses appreciation for the work ethic and perspectives gained from the SAC members aids in the sense of empowerment expressed throughout the study. The data revealed through this study should strongly encourage the administration of the SAC to maintain the program and the means in which they rely on and support the members of the SAC. The data collected from the Phase 2 interviews also described the workshops the studentathletes prepared and delivered at the “Sporstmanship Summits” as one of the most influential experiences and had the strongest contribution toward their increase in confidence. The ability to prepare and deliver a message to their peers was at first a very daunting task. However, the agency to design their workshop and the act of actually going through the experience seemed to be one of the most unique and appreciated aspects of their SAC experience. This evaluation data should reveal to the MHSAA that allowing the student-athletes the opportunity to craft and present to their peers may be one of the most important developmental and empowering experiences available in their time on the SAC. Of course, the data also revealed a number of recommendations the MHSAA should heed in their future interactions with SAC and in self-monitoring for a future potential summative 116 evaluation. Of primary concern to the student-athlete members was their minimal interaction with the executive director, other staff, decisions, and policies within the MHSAA. An immediate response to these recommendations put forth by the student-athlete members would be to increase the number of meetings in an academic year. These student-athletes posit that through increasing the meetings, they would have a stronger opportunity to share and engage with important issues facing the MHSAA, as well as potentially work closer on their projects and even add projects to their workload, to more effectively serve the organization and their student-athlete peers. Considering the immense amount of positive reflections shared by the student-athlete members garnered during only six meetings a year, along with the overnight camp, the students were quite poised they could do much more and gain much more. The student-athletes also mentioned their disappointment with the rare occurrence of interacting with the executive director and almost null interactions with any other staff within the MHSAA. The executive director seemed to realize (i.e., “charged use”) that his absence could be curtailing the experiences of the student-athlete members without ever hearing their strong call for a more active presence. The student-athletes also seemed to feel left out on many issues facing the MHSAA. Here, they mentioned the number of other meetings that would occur during their time at the MHSAA offices that they were not privy to, or even aware of what was happening. The facilitator mentioned “looking forward” to the instance when the SAC demanded to speak to the representative council or appear at a meeting where rules and other policies were being discussed. However, the SAC student-athletes remarked never being aware of their ability to make the request, or their lack of knowledge on what was being discussed to form an opinion about, so that they could request such a presentation. 117 In moving forward with the SAC, and based on the evaluation findings, the data collected here reveals several important, specific recommendations the MHSAA should consider. First and foremost, the MHSAA should maintain its strong approach in aiding the social development of these young people by deliberately increasing means to enhance the SAC members autonomy, relatedness, and competence, as framed by the self-determination theory (Deci & Ryan, 1985; Ryan & Deci, 2000). Furthermore, the MHSAA should identify intentional and productive means to better market the SAC and place the student-athlete members in a position to interact with their peers in formalized settings such as the “Sportsmanship Summit” which only occurs every other fall semester. The MHSAA SAC facilitator should build in more time to deliberately debrief meetings and the overall experience of the SAC membership with the student-athletes. The student-athlete SAC members thoroughly enjoyed the evaluation interviews and the SAC facilitator could easily replicate that sense of caring and curiosity through small debriefing activities to best understand how the group is feeling and to glean more immediate suggestions for improvement. Lastly, increasing the interaction amongst the SAC student-athletes and the MHSAA executive director and other executive staff would increase the connection of the SAC members to the sponsoring MHSAA, and has a strong potential to impact other staff the way the executive director and facilitator have described. In continuing with self-monitoring for future evaluation, and the potential for a strong and favorable summative evaluation, the MHSAA SAC facilitator should keep strong records regarding a number of details for the SAC program. First, the SAC facilitator should keep track of how many student-athletes participate in SAC sponsored events. An increase in the explicit marketing this group and their projects to other Michigan high school student-athletes would enhance the knowledge of the program and could increase the potential for more schools to 118 actively engage in SAC sponsored activities. The SAC facilitator should also keep records of the number of invitations the MHSAA receives to explicitly discuss their student service projects and the SAC to gain a better understanding of the program’s national reach and reception. Lastly, this evaluation revealed several recommendations for the SAC student-athlete members to heed. First, the student-athletes should attempt to find and exert their maximal power. It is highly recommend that the student-athlete members request to speak to the MHSAA executive staff, and request time on the meeting agenda for the MHSAA representative council meetings. While the duty lies within the MHSAA to more fully integrate the SAC into their folds, the members of SAC should make such requests to explicitly state this concern. Secondly, the student-athlete members should engage in practices to better promote themselves and the SAC program to their peers, both at their own schools, as well as their student-athlete peers from opposing schools. Attempting to better market themselves would increase a sense of relatedness but could also potentially identify a number of issues the student-athletes across Michigan are experiencing to share with the MHSAA. Implications and Future Directions for Evaluation within Kinesiology As previously mentioned throughout the dissertation, evaluation is scarcely used in Kinesiology, especially guided with any sort of evaluative theoretical orientation. The key difference between what is typically referred to as evaluation in Kinesiology and that of evaluation within its own field is those who benefit as a result of the evaluation findings. In the typical, scientific approach, the production of knowledge is the primary purpose, and those others in the field of inquiry are served for future investigations and programming. The investigated participants are merely left with a series of recommendations, rather than participating in a deeper understanding of their program. The evaluative approach utilizes the 119 tools from scientific investigations to first and foremost serve those being evaluated, particularly from the formative approach, and then seeks to advance knowledge if the findings can contribute beyond the program and stakeholders. Though seemingly a small nuance, this shift in primary purpose for investigation can make a dramatic difference in the formation of the study design, as well as in the value of the outcomes for the participants. As previously mentioned, the resulting logic model (see Figure 3) from the Phase 1 data collection most accurately captured what the facilitator had been attempting to articulate for several years. This represents the epitome of participatory research and using the tools of the scientific investigator to first serve a very specific end-user group, rather than the field in which the investigator practices. Future studies within Kinesiology investigating the nuances of programs and their ability to achieve intended outcomes can glean much from the field of evaluation and their methodological theories. Particularly, the specific unit of investigators attempting to understand how to use sport as a means for positive youth development should look to employ formative evaluative methods within programs and with program staff. Exercising strengths as a social scientist, guided by use-focused and value-based evaluative methods can not only advance the field of Kinesiology and sport for development, but also add a more ecologically sound perspective to the literature. “Viewing a community ecologically means seeing how persons, roles, and organizations, as well as events, are interrelated” (Kelly, 1971, p. 898). Evaluation prepares the investigator to attend to the interrelated events of a program through the study design, rather than attempting to separate inputs and then describe the causal outputs as typically designed under the empirical model. 120 When considering programs centered in the positive youth development movement, this evaluation study presents several crucial, transportable elements that would enhance many sportbased programs. First, investing in already great student-athlete leaders from diverse experiences allows for individuals with a very unique set of skills to interact with like others toward their own improvement. As with the case of the MHSAA SAC, the program under investigation here works so well because key opinion leaders are accessed and the strongest potential for information exchange in present. When considering other leadership programs, perhaps with less elite or natural leaders, fostering a sense of empowerment and self-determination is absolutely essential (see Gould, Voelker, & Blanton, 2012). Promoting autonomy, relatedness, and competence gains aid increasing the commitment from young people (see Larson, 2000; Ryan & Deci, 2000), but more importantly increases the likelihood of positive growth and emotional maturational development in young athletes. Combining theoretical approaches. A unique contribution from the present study is the combination of two evaluation theories from divergent branches of the metaphorical evaluation tree (see Alkin & Christie, 2004). The employment of the participatory evaluation from the ‘use’ branch mostly guided the design of data collection in Phase 1, and aided in the culmination of the data being represented in a logic model. Specifically, asking the stakeholders their perspectives on what the goals of the SAC should be, and then gaining their perspectives on why the SAC was created to what benefit it presented to the organization. The employment of responsive evaluation theory from the ‘values’ branch fostered the approach to gaining a deeper understanding of how the stakeholders felt about their program and guiding much of the design of data collection in Phase 2. Through investigating how the SAC may have changed studentathlete participants, the study was able to reveal how the participants felt about their program and 121 the value they ascribed to their experiences in the SAC as it related to their development. Returning to the participatory evaluation theory guide, Phase 2 data collection materials also asked the participants to reflect upon the experience of being evaluated. This allowed an accurate portrayal of how the investigation may have shifted their perspectives on the program and revealed the positive aspects that the reflective practiced elucidated within the participants. Even though responsive evaluation rests in the ‘values’ branch and participatory evaluation rests in the ‘use’ branch of Alkin and Christie’s (2004) evaluation tree, the theoretical approaches are quite complementary. The decision to combine these theories speaks to a notion that perhaps recognizing a deeper understanding about the value of the program from a diverse array of stakeholders is the most useful finding in formative evaluation. In this specific study, it was uncanny and beyond coincidence that the adults and students often stated very similar claims about the purpose of the program’s activities, the specific benefits for the organization and the student-athlete members, as well as shortcomings that should be addressed. Due to the rapid pace of the MHSAA and the busy agenda of the SAC, it was unlikely that the time would have been donated to recognizing these similarities in perspectives or the organization of the shared meaning would have been vetted. Seeking a truer perspective on the value placed on the program by its participants toward its usefulness in future interactions amongst the stakeholders and program activities, the combination of evaluative methods was best serving for the essential premise of formative evaluation, which is to primarily serve the immediate needs of a program and its participants (Rossi et al., 2004). More specifically, the requirements of the responsive evaluation orientation for the evaluator to become entrenched in the program (Abma, 2006; Stake 2000) were quite advantageous in the present study. Of utmost advantage was the deep, trusting relationship the 122 evaluator was able to glean from the SAC student-athlete participants and the SAC facilitator. As mentioned in Chapter 4, the SAC facilitator was nervous to bring “an outsider” into the program for fear of overly critical judgment, inept suggestions, or discovery of poor social development behaviors. This close, trusting relationship significantly eased the formal data collection interviews. The students were extremely open and honest with their enjoyment of the SAC as well as with their criticisms, often making remarks about how they trusted the evaluator to detail their truth toward improving the program. The strong contextual understanding of the SAC also eased the design of the evaluation, from involving the SAC facilitator in the creation of the evaluation goals, to what would and would not be available to evaluate. Actively accompanying the SAC on many of their project visits, as well as attending the overnight camp with the SAC allowed for the strongest contextual understanding possible. This was evermore important when considering the few raw number of hours the SAC actually works together in a single place. An approximate sum of hours in the SAC program is only about 50 hours over the six yearly meetings, the overnight camp, and any project trips such as the “Sportsmanship Summits”. The evaluator not only gained a deep trust from the participants, and a strong understanding of the program context, but formed deep professional and personal relationships with the SAC facilitator, MHSAA executive director, and most importantly the young student-athlete members of the SAC which greatly increased the motivation to conduct and expertly organized and professional evaluation for improving this program. Lastly, under the primary purpose of evaluation and attempting to aid the program and its stakeholders, a grounded theory approach to data analysis was the most appropriate means for revealing the thematic schemas and overall in describing a theory of the SAC program. By selectively observing the program and the data collected unobstructed or biased from 123 developmental theory or predetermined hypothesis, the grounded theory approach allowed from the most unique, appropriate, and accurate representation of what the SAC actually is. Strengths and Limitations The foremost aim of the study was to formatively evaluate the SAC program within the MHSAA to uncover a stronger objective, identify goals, and explore the experience of participating in the SAC as a student-athlete member. The primary evaluator spent two years within the SAC infrastructure establishing rapport with the participants and observing the interactions amongst MHSAA staff and SAC members to most effectively determine the ways in which to conduct an evaluation and best serve the program through the evaluation process and results. This deep understanding and strong trust established by the evaluator allowed for an initially nervous facilitator to allow the evaluation in the first place, and helped ensure that the truth was shared by the participants through the data collection interviews. Additionally, member checks were used throughout the data collection and analysis process. At the conclusion of each phase, transcripts were sent back to participants to ensure accuracy of the data collected. Furthermore, the logic model, representing the findings from Phase 1 was also sent back to the participants to explore weaknesses and offer suggestions for change, though none were offered (see Figure 4). Another strength of the study lies in the strong theoretical orientation adopted in the design of the study to most appropriately aid the program and discover the stakeholders’ perspectives about participating in the SAC. Examining the program through stringent theoretical lenses aids in the validity of the findings and offers a stronger model for future researchers to replicate and extend. The resulting data offer a clearer description of what occurs within the SAC. Should the MHSAA and SAC administration choose to fully adopt the logic model, they 124 should be able to more easily self-monitor their program and prepare for a more summative evaluation which could be designed to consider the actual cost and value added of the SAC. Lastly, a strength of the study follows from the design of the investigation to be evaluative in nature, as opposed to empirical. The evaluative design requires that program staff and stakeholders reap the primary benefit of participating in the study, as opposed to the production of knowledge for the field as the primary beneficiary. In the present study, a truth was explored concerning the experience of participating in the SAC and how the SAC most accurately aids the organization, the staff and student-athlete members, and the greater number of student-athletes around the state of Michigan. The participatory evaluation theory design calls for the creation of a shared meaning of the program from the stakeholders and evaluator. In this project, the logic model (see Figure 3) as well as the thematic schemas revealed from the data most accurately depict the grand functioning of the SAC and portrays how those most recent participants view their experience in the program. Of course, this study is not without its limitations. A primary limitation in this study involves the purposeful sampling employed in determining participants. Only the current student-athlete members of the SAC were interviewed. Former SAC members perspectives were not sought to describe their previous opinions, and therefore the portrayal of how the SAC changed the participants and their general perspectives of the SAC experience is limited to the participants in a single academic year. Furthermore, student-athletes across the state of Michigan who are not members of SAC were not interviewed to garner their opinion of the benefit of the MHSAA SAC. Another limitation of the study was the lack of follow-up with the two external adult stakeholders interviewed in Phase 1. The researcher did not contact these individuals beyond the 125 member check of their transcript. In defense of this decision, they were nominated as excellent sources for understanding the creation of SAC, thus their inclusion in Phase 1 that was designed to explore this reason. These members have no direct interaction with the SAC (outside of having students from their school serving on the SAC) and therefore were not perceived as necessary in Phase 2, although they could have provided insight. Furthermore, no representative council members who may have opposed the SAC program were sought for counter-balancing the program’s positive characteristics from more critical perspectives. When interviewing the student-athlete members of the SAC, the interview guide did not explicitly ask for each individual program may have impacted them personally, or how they would recommend changes for that unique project. For example, the evaluator never asked “explain how presenting at the Sportsmanship Summit may have impacted you”, or “what was it like to organize and facilitate the Battle of the Fans context?” Gaining a more specific, and indepth understanding for each program may have been beneficial in offering more specific recommendations and uncovering a stronger understanding of the most specific and beneficial experiences from each unique SAC project. Lastly, the formative evaluation design of the study does not actually determine the worth of the program to the MHSAA and the overarching objectives of the sponsoring organization. It is unknown how the MHSAA feels about its financial investment in the SAC without a financial return. Furthermore, the findings here cannot be used to determine any direct economic gain toward sustaining the not-for-profit business model employed by the MHSAA. The guiding methodological theories employed in the data collection are also not without limitations, as positivistic ‘hard’ science scholars have critiqued interpretive social scientific methods for many decades. Though, as stated throughout this dissertation, evaluation seeks first to serve its 126 evaluand and their program, then to use the findings to inform future investigative approaches and add to the knowledge in corresponding fields. In conclusion, this evaluation study has and will benefit the SAC program and the MHSAA in the future, should they choose to follow the recommendations and continue to selfmonitor for optimal performance, student-athlete benefit, and organizational impact. This study has sought to add to the kinesiology literature, and particularly the positive youth development through sport movement. Using practices from evaluation to best serve a specific set of endusers, and greatly influenced by the work from the field of Community Psychology, it is with utmost aim that this project resolved to practice some ecological good beyond the earning of an advanced degree for the primary investigator. Kelly (1971) wrote, “ecological good means helping to develop the natural resources of a community…good is viewed as the development and creation of competencies within the community” (pg. 898). As a result of participating in this evaluation, the community of the MHSAA has engaged in deep reflective thinking and in cooperation with the evaluator, developed a shared meaning of their program. In this vein, ideally the primary investigator served more as a socratic guide to the MHSAA and helping this organization and the participants in SAC recognize their own truth about the program and organize their shared meaning, rather than a pesky researcher attempting to fulfill some personal scientific agenda. The findings in this study were not so much discovered by the primary investigator; more accurately, acting as an outside perspective, the findings were revealed and organized, as the participants were the holders of the truth and this process merely allowed them to fully explore their thoughts and perceptions about the SAC. The ancient greek philosopher Plato offered this creed: “one can learn more about a person in an hour of play, than a year of conversation”. Though this project involved dozens of conversations over two-years of 127 observations, it was through the invitation to engage and “play” with the participants in their meetings, activities, and summer camps where the ability to abstract such a truth about their program is truly rooted. 128 APPENDICES 129 Appendix A. Evaluation Agreement A Use Focused Participatory Evaluation of the Implementation, Perceived Organizational Benefits, and Developmental Potential of a Student Advisory Council Memorandum of Agreement This agreement is made and entered into by and between the Michigan High School Athletic Association (hereinafter referred to as “MHSAA”) and Jedediah E. Blanton (hereinafter referred to as “consultant”). Term of Contract: This agreement shall become effective on February 19, 2013 and shall continue in effect for a period of 6 months or until May 30, 2014, unless terminated in accordance with the provisions specified within this agreement. Status of Consultant: The consultant is conducting this evaluation primarily to assist and improve the SAC programming for the MHSAA. Additionally, the consultant is completing the evaluation as their dissertation project, toward earning a doctoral degree from Michigan State University. Purpose of the Evaluation: The primary purpose of this evaluation is to determine the midstream merit of the evaluand’s program. In so doing, the evaluator will attempt to craft a meaning of the evaluand’s program, which entails identifying programmatic goals, aligning practices to meet those goals, estimating the organizational benefits of the SAC, and determining what, if any, developmental gains come as a result of participating in the evaluand’s program. Services: 1. The consultant agrees to advise the MHSAA on the development of an evaluation of its program, the Student Advisory Council (hereinafter referred to as “SAC”). The consultant will assist Mr. Andy Frushour and the MHSAA staff in selecting appropriate evaluation questions, methods, analysis strategies, dissemination media, and uses of the data. 2. The consultant will attend regularly scheduled meetings with MHSAA staff and SAC members to plan the design of the evaluation and discuss issues related to the implementation of the evaluation of the MHSAA and SAC. As needed, the consultant will independently and individually interview relevant individuals to discuss the SAC and issues related the evaluation. 3. The consultant will provide leadership in developing an evaluation plan to assess the programmatic goals and activities of the MHSAA SAC, as well as determine studentathlete member satisfaction. 4. The consultant will contribute to meetings in which use of results is planned and those who have a right to know the results of the evaluation are determined. 130 5. The consultant will be discussing the terms, methods, analyses, findings, and uses of the data in accordance with procedures necessary to complete the dissertation project as indicated by guidelines from Michigan State University. Agreements: 1. MHSAA agrees to provide the consultant with full access to information, staff, services, and records as necessary to carry out the work outlined in this agreement 2. The consultant will determine the method, details and means of performing these services. MHSAA shall have no right to, and shall not, control the manner or determine the method by which the consultant performs the services. 3. The consultant may recruit research assistants such as the consultant deems necessary to perform services required of the consultant in this agreement. MHSAA may not control, direct, or supervise the consultant’s assistants in the performance of the services. The consultant assumes full and sole responsibility for the payment of all compensation and expenses of these assistants. 4. The consultant agrees to perform all services in a manner that is consistent with the guidelines for professional practice promulgated by the American Evaluation Association (attached). These guidelines hold that the consultant will adhere to principles concerning systematic inquiry, competence, integrity/honesty, respect for others, and responsibility for general and public welfare in carrying out all work performed under the terms of this agreement. The consultant is bound by these principles to prevent efforts to compromise, alter, suppress or otherwise misuse the findings of the evaluation. Confidentiality: 1. Confidential information means any MHSAA proprietary information, technical data, or know-how including, but not limited to research, software, processes, designs, finance, documents, or other information disclosed by MHSAA directly or indirectly, orally or in writing. 2. Consultant shall not, during or subsequent to the term of this agreement, use MHSAA’s confidential information for any purposes whatsoever other than performance of the contracted services on behalf of MHSAA. Consultant shall not disclose any confidential information to a third party unless authorized by MHSAA. Consultant further agrees to take reasonable precautions to prevent any unauthorized disclosure of confidential information. Confidential information does not include information that is publicly available or has become known or available through no wrongful act of the consultant. 3. MHSAA agrees to waive confidentiality and anonymity for any act, presentation, document or publication related to the procedures of the dissertation project, including, but not limited to the dissertation proposal meeting, dissertation defense meeting, or dissertation document. Termination and Renegotiation of Agreement: 1. This agreement shall terminate automatically on the occurrence of any of the following events: Death of consultant, consultant becomes unable to satisfactorily perform service because of physical or mental disability; bankruptcy of insolvency of MHSAA. 131 2. If the consultant is unable to perform the terms of this agreement due to illness or disability, he will assist the MHSAA in identifying a suitable replacement consultant. 3. Circumstances mar arise during the performance of this agreement that require it to be renegotiated, such as substantial modification of the evaluation design, or as dictated by dissertation protocol or advisement of the dissertation committee. General Provisions: Any mail notices should be addressed to the consultant at 308 W. Circle Dr., Michigan State University, East Lansing, MI 48824 and to the MHSAA addressed to Andy Frushour, 1661 Ramblewood Dr, East Lansing, MI 48823. This agreement shall be governed by and construed in accordance with the laws of the State of Michigan. If any provision is the agreement is held by a court of competent jurisdiction to be invalid, void, or unenforceable, the remaining provisions shall nevertheless continue in full force and effect without being impaired or invalidated in any way. The basis for mediating any disputes that may arise during the term of this agreement will be the Guiding Principles for Evaluators promulgated by the American Evaluation Association. Executed at East Lansing, Michigan: Consultant: Mr. Jedediah E. Blanton Michigan High School Athletic Association MHSAA Contact Person: Mr. Andy Frushour 132 Appendix B. Consent Form – Adults and Administrators A Use Focused Participatory Evaluation of the Implementation, Perceived Organizational Benefits, and Developmental Potential of a Student Advisory Council INFORMED CONSENT FORM – Adults & Administrators Dear Sir/Madam, We are inviting you to take part in an evaluative research study sponsored by Michigan State University, and toward the dissertation project of Jedediah Edward Blanton. We would like to involve the administration, executive staff, and representative council of the Michigan High School Athletic Association (MHSAA) to participate in the evaluation process of the Student Advisory Council (SAC) program, which would include at least one individual interview as well as participating in follow-up interviews. Introduction: Anecdotally, the SAC has been a wonderful program for its highly selective student-athlete members. However, the program has yet to be critically evaluated or prepared for objective assessment of its benevolence to the student-athlete members, administrative staff, or organization. This research study will carefully explore the perceptions of the many stakeholder groups to better understand the underlying mechanisms of the SAC and identify ways to improve the student experience and organizational gain. Purpose of the project: The purpose of this project is to craft a meaning of the SAC program, which entails identifying programmatic goals, aligning practices to meet those goals, estimating the organizational benefits of the SAC, and determining what, if any, developmental gains come as a result of participating in SAC. Procedures: In this study, we will ask that you first participate in an individual interview with a trained researcher at a time most convenient for you. We will ask you a few questions regarding your opinion of the SAC. We’d like to have permission to contact you with follow-up questions to ensure your opinions are valued toward the evaluation study. Please note that the interviews will be audio taped so that we better able to capture the things you said during the interviews and focus groups. Any emails exchanged will also be archived for evaluation purposes. You may request to participate in the interview without being audio-recorded. Time Commitment: Initial Interview will last approximately 30-45 minutes. Follow-up questions via email should take approximately 5-15 minutes. 133 Expected risks and benefits: Participating in this project involves minimal risk, although it is possible that we may ask a question or two that makes you feel uncomfortable. If that is the case, you will not have to answer it. Results will benefit other high school student-athletes and future SAC members by helping us and the MHSAA better understand the SAC program and more effectively monitor SAC-related activities. However, there will be no specific benefits to you as a result of answering the questions. Additionally, there is no compensation or incentive provided for your voluntary participation. Confidentiality and voluntary participation: All information collected during this study will be strictly confidential. We will not share any information about you with anyone outside the study. Interviews and focus group sessions will be audio recorded and transcribed. We will not include names of the subjects in the transcriptions, just what is said as part of the discussions. Emails will be archived, however identifying information will be removed. We will do everything possible to protect your privacy and will not include your name in any of the publications resulting from this study. Your confidentiality will be protected to the maximum extent allowable by law. All of the data will be stored in locked file cabinets or password-protected computer files at Michigan State University, and it will be kept for at least six years. Only the project investigators and staff will have access to your data. Only the project investigators, staff, and the Institutional Review Board and Michigan State University will be able to access and receive the results of the study. Your participation is voluntary, and you are free to withdraw from the project at any time without penalty. If you decide to withdraw, it will not hurt your relations with Michigan State University or Extension, MHSAA, or any other organization associated with this project. If you are injured as a result of participation in this research project, Michigan State University will assist you in obtaining emergency medical care, if necessary, for your research related injuries. If you have insurance for medical care, your insurance carrier will be billed in the ordinary manner. As with any medical insurance, any costs that are not covered or are in excess of what are paid by your insurance, including deductibles, will be your responsibility. The University’s policy is not to provide financial compensation for lost wages, disability, pain or discomfort, unless required by law to do so. This does not mean that you are giving up any legal rights you may have. You may contact Daniel Gould, Principal Investigator, at (517) 432-0175 with any questions or to report an injury. This form explains what is involved with participating in this evaluation project. Please read it carefully and ask any questions you may have. If you do not have questions now, you may ask later. If you have any questions about this study, please contact Daniel Gould at (517) 432-0175; 308 West Circle Dr., Michigan State University, East Lansing, MI 48824; drgould@msu.edu. If you have any questions or concerns about your role and rights as a research participant, or would like to register a complaint about this research study, you may contact, anonymously if you wish, MSU’s Human Research Protection Programs, at 571-355-2180, FAX 517-432, or email irb@msu.edu, or regular mail at: 408 W. Circle Dr., Room 207 Olds Hall, MSU, East Lansing, MI 48824. 134 PLEASE KEEP THE FIRST TWO PAGES FOR YOUR INFORMATION AND RECORDS. PLEASE FILL OUT THE ATTACHED PAGE AND RETURN IT. THANK YOU FOR YOUR CONSIDERATION 135 I voluntarily consent to participate in this research project. I have received a copy of the forms explaining the project. I have read the contents of this consent form and have been encouraged to ask any questions I may have. I have received answers to my questions. Participant’s name (please print) / Participant’s signature / Date Project coordinator Street/mailing address City, State, Zip Preferred Email address Daytime phone WE WILL NOT SHARE YOUR CONTACT INFORMATION with anyone outside this project. 136 Appendix C. Consent Form – Students and Parents A Use Focused Participatory Evaluation of the Implementation, Perceived Organizational Benefits, and Developmental Potential of a Student Advisory Council INFORMED CONSENT FORM – Parents & Students *The words ‘you’ or ‘your’ will infer either you (parent/guardian) or your child. Dear Students & Parent or Guardian, We are inviting you to take part in an evaluative research study sponsored by Michigan State University, and toward the dissertation project of Jedediah Edward Blanton. We would like to involve the high school student-athlete members of the Michigan High School Athletic Association’s (MHSAA) Student Advisory Council (SAC) to participate in the evaluation of this program, which would initially include one individual interview and participating in a focus group interview. Introduction: Anecdotally, the SAC has been a wonderful program for its highly selective student-athlete members. However, the program has yet to be critically evaluated or prepared for objective assessment of its benevolence to the student-athlete members, administrative staff, and the organization. This research study will carefully explore the perceptions of the many stakeholder groups to better understand the underlying mechanisms of the SAC and identify ways to improve the student experience and organizational gain. The research team is interested in your perception of the SAC as well as your ideas of improving the programming related to SAC. Purpose of the project: The purpose of this project is to craft a meaning of the SAC program, which entails identifying programmatic goals, aligning practices to meet those goals, estimating the organizational benefits of the SAC, and determining what, if any, developmental gains come as a result of participating in SAC. Procedures: In this study, we will ask that you to participate in a focus group interview during the last scheduled SAC meeting of the academic year (April 14, 2013). Your attendance at this meeting does not mean that you are required to participate in the focus group. During the focus group, we will ask you a few questions regarding your opinion of the SAC. We’d like to have permission to contact you with some follow-up questions via email to ensure your opinions are valued toward the evaluation study. Finally, we’d like you to participate in an individual interview over the phone with a trained researcher at a time most convenient for you, if you choose to do so. Please note that the sessions will be audio taped so that we better able to capture the things you say during the interviews and focus groups. Any emails exchanged will also be archived for evaluation purposes. You may request that the individual interview is not audio taped or recorded. 137 Time Commitment: The initial focus group interview is expected to last approximately 30-minutes. The individual interview is expected to last between 30-45 minutes. Any follow-up questions sent through email should be easily answered within 5-15 minutes. Expected risks and benefits: Participating in this project involves minimal risk, although it is possible that we may ask a question or two that makes you feel uncomfortable. If that is the case, you will not have to answer it. Results will benefit other high school student-athletes and future SAC members by helping us, and the MHSAA, better understand the SAC program and more effectively monitor SAC-related activities. However, there will be no specific benefits to you as a result of answering the questions. Additionally, there is no compensation or incentive provided for your voluntary participation. Confidentiality and voluntary participation: All information collected during this study will be strictly confidential. We will not share any information about you with anyone outside the study. Interviews and focus group sessions will be audio recorded and transcribed. We will not include names of the subjects in the transcriptions, just what is said as part of the discussions. We cannot guarantee however, that other participants in the study or focus group sessions will not share information with others outside of the study. Emails will be archived, however identifying information will be removed. We will do everything possible to protect your privacy and will not include your name in any of the publications resulting from this study. Your confidentiality will be protected to the maximum extent allowable by law. All of the data will be stored in locked file cabinets or passwordprotected computer files at Michigan State University, and it will be kept for at least six years. Only the project investigators and staff will have access to your data. Only the project investigators, staff, and the Institutional Review Board and Michigan State University will be able to access and receive the results of the study. Your participation is voluntary, and you are free to withdraw from the project at any time without penalty. If you decide to withdraw, it will not hurt your relations with Michigan State University or Extension, MHSAA, or any other organization associated with this project. If you are injured as a result of participation in this research project, Michigan State University will assist you in obtaining emergency medical care, if necessary, for your research related injuries. If you have insurance for medical care, your insurance carrier will be billed in the ordinary manner. As with any medical insurance, any costs that are not covered or are in excess of what are paid by your insurance, including deductibles, will be your responsibility. The University’s policy is not to provide financial compensation for lost wages, disability, pain or discomfort, unless required by law to do so. This does not mean that you are giving up any legal rights you may have. You may contact Daniel Gould, Principal Investigator, at (517) 432-0175 with any questions or to report an injury. This form explains what is involved with participating in this evaluation project. Please read it carefully and ask any questions you may have. If you do not have questions now, you may ask 138 later. If you have any questions about this study, please contact Daniel Gould at (517) 432-0175; 308 West Circle Dr., Michigan State University, East Lansing, MI 48824; drgould@msu.edu. If you have any questions or concerns about your role and rights as a research participant, or would like to register a complaint about this research study, you may contact, anonymously if you wish, MSU’s Human Research Protection Programs, at 571-355-2180, FAX 517-432, or email irb@msu.edu, or regular mail at: 408 W. Circle Dr., Room 207 Olds Hall, MSU, East Lansing, MI 48824. PLEASE KEEP THE FIRST TWO PAGES FOR YOUR INFORMATION AND RECORDS. PLEASE FILL OUT THE ATTACHED PAGE AND RETURN IT. THANK YOU FOR YOUR CONSIDERATION 139 I voluntarily consent for my child to participate in this research project. I have received a copy of the forms explaining the project. I have read the contents of this consent form and have been encouraged to ask any questions I may have. I have received answers to my questions. Child’s name (please print) / Child’s signature / Date Project coordinator Street/mailing address City, State, Zip Home phone number Daytime phone [ ] I am 18 years of age or older, and do not require parental permission. [ ] I am under the age of 18, and require parental permission. Parent’s name (please print) / Parent’s signature / Date Project coordinator WE WILL NOT SHARE YOUR CONTACT INFORMATION with anyone outside this project. 140 REFERENCES 141 REFERENCES Abma, T. A. (2006). The practice and politics of responsive evaluation. American Journal of Evaluation 27, p. 31-43 Abma, T. A. & Stake, R. E. (2001). Stake’s responsive evaluation: Core ideas and evolution. In J. C. Greene and T. A. Abma (Eds.) Responsive evaluation (pp. 7-21). San Francisco, CA: Jossey-Bass. Alkin, M. C. (2004). Evaluation Roots: Tracing Theorists’ Views and Influences. Thousand Oaks, CA: Sage. Alkin, M. C., & Christie, C. A. (2004). An evaluation theory tree. In M. C. Alkin (Ed.), Evaluation Roots: Tracing Theorists’ Views and Influences (pp. 12-65). Thousand Oaks, CA: Sage. Amo, C., & Cousins, J.B. (2007). Going through the process: An examination of the operationalization of process use in empirical research on evaluation. New Directions for Evaluation, 116, 5-26. Baumgartner, T.A., & Hensley, L.D. (2006). Conducting and reading research in health and human performance (4th ed.) New York, NY: McGraw-Hill Beebe, J. (2001). Rapid Assessment Process: An Introduction. Walnut Creek, CA: AltaMira Press. Bulling, T.A. (1992). The relationship of involvement in intercollegiate athletics and other extracurricular activities to personal development (Doctoral dissertation). Retrieved from ProQuest Dissertations and Theses. Order No. 9314393 Bundick, M., (2011). Extracurricular activities, positive youth development, and the role of meaningfulness of engagement. The Journal of Positive Psychology, 6(1), 57-74 Coté, J. (1993) Organizing and interpreting unstructured qualitative data. The Sport Psychologist 7(2) pp. 127-137 Cousins, J.B., & Whitmore, E. (1998). Framing participatory evaluation. New Directions for Evaluation, 80, 5-23. Creswell, J. W. (2007) Qualitative inquiry and research design. Thousand Oaks, CA: Sage Cumming, S.P., Smol, F.L., Smith, R.E., & Grossbard, J.R. (2007). Is winning everything? The relative contributions of motivational climate and won-lost percentage in youth sport. Journal of Applied Sport Psychology, 19(3), 322-336. 142 Darling, N., Caldwell, L., & Smith, R. (2005). Participation in school-based extracurricular activities and adolescent adjustment. Journal of Leisure Research, 37(1), 51-76. Deci, E.L., & Ryan, R.M. (1985). Intrinsic motivation and self-determination in human behavior. New York: Plenum Press. Dworkin, J.B., Larson, R., & Hansen, D. (2003). Adolescents accounts of growth experiences in youth activities. Journal of Youth and Adolescence, 32(1), 17-26. Eccles, J., & Barber, B., (1999). Student council, volunteering, basketball, or marching band: What kind of extracurricular involvement matters? Journal of Adolescent Research, 14(1), 10-43. Finch, H. & Lewis, J. (2003). Focus groups. In J. Ritchie & J. Lewis (Eds.) Qualitative research practice: A guide for social science students and researchers (pp. 170-198). Thousand Oaks, CA: Sage. Fitch, R. (1991). The interpersonal values of students at differing levels of extracurricular involvment. Journal of College Student Development, 32(1), 24-30. Flett, M. R., Gould, D., & Lauer, L. (2012). A study of an underserved youth sports program using the youth program quality assessment. Journal of Applied Sport Psychology, 24(3), 275-289. Gould, D., & Carson, S. (2008). Life skills development through sport: current status and future directions. International Review of Sport and Exercise Psychology, 1(1), 58-78. Gould, D., Voelker, D., & Blanton, J. (2012). Future Directions in Youth Leadership Research in Sport. In R. J. Schinke & S. J. Hanrahan (Eds.) Sport for development, peace, and social justice (pp. 163-180). Morgantown, WV: Fitness Information Technology. Harnar, M.A., & Preskill, H. (2007). Evaluators’ descriptions of process use: An exploratory study. New Directions for Evaluation, 116, 27-44. Hendricks, L. (2011). Perceptions of stakeholder salience for NCAA campus student-athlete advisory committees. (Doctoral dissertation). Retrieved from ProQuest Dissertations and Theses. (UMI Number: 3458910) Holland, A., & Andre, T. (1987). Participation in extracurricular activities in secondary school: What is know, what needs to be known? Review of Educational Research, 57(4), 437466. House, E. R. (2001). Unfinished business: Causes and values. American Journal of Evaluation 22, p. 309-315 143 Kelly, J.G. (1971). Qualities for the community psychologist. American Psychologist, 26, 897903.King, J. (1998). Making sense of participatory evaluation practice. New Directions for Evaluation, 80, 57-67 King, J. (2007). Developing evaluation capacity through process use. New Directions for Evaluation, 116, 45-59. Landry, R., Amara, N., & Lamari, M. (2001). Climbing the ladder of research utilization: Evidence from social science research. Science Communication, 22(4), 396-422. Larson, R., Jarrett, R., Hansen, D., Pearce, N., Sullivan, P., Walker, K., ...Wood, D. (2004). Organized youth activities as contexts for positive development. In P. A. Linley, & S. Joseph (Eds.), Positive Psychology in Practice (pp. 540-560). New York: Wiley. Magyar, T. M., Guivernau, M., Gano-Overway, L., Newton, M., Kim, M., Watson, D., & Fry, M. (2007). The influence of leader efficacy and emotional intelligence on personal caring in physical activity. Journal of Teaching in Physical Education, 26(3), 310-319. Maton, K.I. (2008). Empowering community settings: Agents of individual development, community betterment, and positive social change. American Journal of Community Psychology, 41, 4-21. Miles, M.B., & Huberman, A. M. (1994) Qualitative data analysis (2 CA: Sage. nd ed.). Thousand Oaks, Miller, R.L., & Campbell, R. (2006). Taking stock of empowerment evaluation: An empirical review. American Journal of Evaluation, 27, 296-319. Moore, K. (1979). The campaign for athletes’ rights. The ANNALS of the American Academy of Political and Social Science, 445(1), 59-65. National Collegiate Athletic Association (2004). Student-Athlete Advisory Committee. Indianapolis: IN: Author Newton, M., Watson, D., Gano-Overway, L., Fry, M., Kim, M., & Magyar, M. (2007). The role of a caring-based intervention in a physical activity setting. The Urban Review, 39(3), 281-299. rd Patton, M.Q., (2002). Qualitative research and evaluation methods (3 ed.). Thousand Oaks, CA: Sage Publications th Patton, M.Q., (2008). Utilization-focused evaluation (4 ed.). Thousand Oaks, CA: Sage Publications 144 Rappaport, J. (1987). Terms of empowerment/exemplars of prevention: Toward a theory for community psychology. American Journal of Community Psychology, 15, 121-148. Ryan, R.M., & Deci, E.L. (2000). Self-determination theory and the facilitation of intrinsic motivation, social development and well-being. American Psychologist, 55, 68-78. Rossi, P. H., Lipsey, M. W., & Freeman, H. E. (2004). Evaluation: A systematic approach. Thousand Oaks, CA: Sage Publications Scanlan, T., & Lewthwaite, R. (1984). Social psychological aspects of competition for male youth sport participants: I. Predictors of competitive stress. Journal of Sport Psychology, 6(2), 208-226. Schwandt, T. A. (2001). Responsiveness and everyday life. In J. C. Greene and T. A. Abma (Eds.) Responsive evaluation (pp. 73-89). San Francisco, CA: Jossey-Bass. th Scriven, M. (1991). Evaluation thesaurus (4 ed.) Newbury Park: Sage Publications Scriven, M. (1997). Truth and objectivity in evaluation. In E. Chelimsky & W.R. Shadish (Eds.) Evaluation for the 21st century (pp. 477-500). Thousand Oaks, CA: Sage. Scriven, M. (1998). Minimalist theory: The least theory that practice requires. American Journal of Evaluation, 19, 57-72. Smith, J.S. (1991). The relationship between involvement in extracurricular activities and the psychosocial development of Clemson University students. (Doctoral dissertation). Retrieved from ProQuest Dissertations and Theses. Order No. 9200845 Speer, P., & Hughey, J. (1995). Community organizing: An ecological route to empowerment and power. American Journal of Community Psychology, 23(5), 729- 748 Stake, R. E. (2000). Program evaluation, particularly responsive evaluation. In D. L. Stufflebeam, G. F. Madaus, & T. Kellaghan (Eds.) Evaulation models: Viewpoints on nd education and human services evaluation (2 ed.) (pp. 342-357). Boston, MA: Kluwer Academic Publishers. Strauss, A., & Corbin, J. (1990). Basics of qualitative research: Grounded theory procedures and techniques. Newbury Park, CA: Sage. Tesch, R. (1990). Qualitative research analysis types and software tools. New York: Falmer Press. 145 Wholey, J.S. (2004). Evaluability assessment. In J.S. Wholey, H.P. Hatry, & K.E. Newcomer nd (Eds.), Handbook of practical program evaluation (2 edition) (pp. 33-62). San Francisco: Jossey Bass. Zimmerman, M.A. (2000). Empowerment theory: Psychological, organizational, and community levels of analysis. In J. Rappaport & E. Seidman (Eds.) Handbook of community psychology (pp. 43-63). New York: Plenum. 146