Lang}: .. :a z \ 3:559: . u 3...: as” > y... 21.3. .2: \p 123.... 35:... 11$.i‘x . J)! n“ n .a. . fun. r13. u .2 :9": "L v .1.) ‘ IL 1.....4!:.v‘) ' 1...?) .2), , .. ..1s71&r cinvct LEI-n: Want a. ,_:Ih‘ . 7.) .v:;...,.. .1: £ was“... ..,... r. 3. 3 nl .11». ‘y.~. “1535 MICHIGM STATE UNIVERSITY UBRANES \ \\\\\\\\\\\\\\\\\\\\\\\i\\\\\\\\\\\\\\ W M 3 1293 01020 068 i This is to certify that the thesis entitled Evaluation of Program Processes: A Case-Study of an After-School Program presented by Melissa A. Freel has been accepted towards fulfillment of the requirements for Mus. degree in Wild Ecology Major professor Date 11115/94 0-7639 MS U is an Affirmative Action/Equal Opportunity Institution LIBRARY 5 Michigan State? ! Unlverslty I PLACE ll RETURN BOXtomnwohbchockamnmm. TOAVOID FINES Mum onetbdmddo duo. DATE DUE DATE DUE DATE DUE APR 10 2002 FE330$ZUM .; » « II I l L MSU I! An Affirmative Adlai/EM Opporhlnky lmtluion EVALUATION OF PROGRAM PROCESSES: A CASE-STUDY OF AN AFTER-SCHOOL PROGRAM by Melissa A. Freel A THESIS Submitted to Michigan State University in partial fulfillment of the requirements for the degree of MASTER OF SCIENCE Department of Family and Child Ecology 1994 ABSTRACT EVALUATION OF PROGRAM PROCESSES: A CASE-STUDY OF AN AFTER-SCHOOL PROGRAM by Melissa A. Freel This present study involved a qualitative evaluation of the processes comprising an after-school program for young adolescents in Grades 5-8. This program, implemented in the two middle schools of the Beecher Community School District during the 1993- 1994 school year, involved about 200 students. The outcome of this evaluation described the program's functioning and its assets and limitations (were determined by analyzing and comparing the results 10 characteristics of "successful" youth programs). Assessing the internal processes and the contextual processes that comprised the after- school program, the study used a semi-structured interview with the program participants and a focus group with the program personnel and the program's fimding proposal. The results indicated that the after-school program possessed both assets and limitations. The limitations of the program are discussed in terms of programming recommendations. This thesis is dedicated to the youth of the Beecher community, to the staff of the Beecher After-School Enrichment Program, and to the staff of the Beecher Teen Health Center. iii ACKNOWLEDGEMENTS My deepest appreciation is extended to the young adolescents of Beecher and to the staff members of the Beecher After-School Enrichment Program and the Beecher Teen Health Center. Without their cooperation and support, this study would not have been completed. In addition, this thesis does not reflect adequately what I have learned about myself as a result of my experience with each of the students who participated in the program as well as with each of the staff members. I am so grateful for the valuable lessons that I have learned from them, and I will carry those lessons with me for the rest of my life. Special thanks is extended to Ms. Teri Covington, Coordinator for the Beecher Teen Health Center, and to Ms. Yvette Flippen, Beecher Teen Health Center Health Educator/Club Director. I cannot thank them enough for giving me the opportunity to evaluate the after-school program as well as to serve as a staff member of the program. The completion of this thesis also would not have taken place if it were not for the support, guidance, love, and nurturance of my first teachers-Joann and Edward F reel. (Thank-you mom and dad). I am also very appreciative of the support and love of my twin sister, Jennifer. Without fail, she has always believed in me. I thank my sister, Andrea, and brother, Johnathan-- I could always count on both of them to make me smile. I am truly grateful to my fiancee, Jonathan Solomond. His love, support, iv and encouragement were fundamental to my completing this thesis. My gratitude is also extended to the members of my committee-Dr. Norma Bobbitt, Dr. Richard M. Lerner, Dr. Marvin H. McKinney, and Dr. Charles W. Ostrom. Each of you committed your time and energy to this thesis process and I am sincerely thankful. Warmest thanks is extended to Dr. Bobbitt for the wonderful learning experiences that she provided for me which greatly enriched my experience at Michigan State University. Appreciation is extended to Dr. Richard M. Lerner for his recognition of my potential at Penn State University, for his invitation to study at MSU, and for his commitment of time and editorial skills to this thesis process. I am grateful to Dr. Marvin H. McKinney for his opening my eyes to the powerful role of race in our lives. I have begun an understanding of this role and I hope I will continue to learn. Special thanks is extended to Dr. Ostrom for the opportunities to discuss the theories and types of program evaluation. These discussions certainly influenced the thesis process. I extend my appreciation to Ms. Theresa Ferrari, Ms. Jennifer London, Mr. Daniel Perkins, and Mr. Brian Turnbull for their support, feedback on the thesis, and friendship throughout my two years at MSU. Particular thanks is extended to Brian Turnbull for creating the figures that appear in the thesis. Also, I thank Ms. Linda Chapel for her teaching me the necessary editorial skills to make this thesis presentable. TABLE OF CONTENTS LIST OF TABLES ............................................. ix LIST OF FIGURES ............................................. x CHAPTER I INTRODUCTION: ADOLESCENTS AND ADOLESCENT NEEDS .......... 1 CHAPTER 11 REVIEW OF LITERATURE ...................................... 5 The Non-School Hours: "Successful" Programs .................... 5 The Relationship between Evaluation and Practice .................. 8 Program Processes within the Program's Context: Qualitative Process Evaluation ........................................ 9 CHAPTER III STATEMENT OF PROBLEM, DEFINITION OF TERMS, CONCEPTUAL FRAMEWORK .............................................. 13 Statement of the Problem .................................. 13 The Context of the Study .................................. 14 The Program of Focus in this Study ........................... 16 Race, Class, and Values of the Program Evaluator ................. 17 Definition of Terms ...................................... l8 Conceptual Framework: Relationships between the Individual and the Context ......................................... 20 CHAPTER IV EVALUATION FRAMEWORK .................................. 34 Objectives of Evaluation Study .............................. 34 Evaluation Questions ..................................... 36 Evaluation Approach ..................................... 36 Procedure ............................................. 37 Methodology ........................................... 39 CHAPTER V CODING AND ANALYSES .................................... 44 Descriptive Coding ...................................... 46 Pattern Coding ......................................... 49 vi CHAPTER VI RESULTS .................................................. 57 Questions Relating to the Context of the Program ................. 57 The Durational Contextual Forces ............................ 58 Operative Contextual Forces ................................ 67 Questions Relating to the Internal Processes ..................... 70 Descriptive Variables ..................................... 71 Maintenance Variables .................................... 79 Summary of Program Description ............................ 88 Questions Relating to Program Assets and Program Limitations ........ 93 Summary of Program Assets and Limitations .................... 101 CHAPTER VII DISCUSSION ............................................... 104 Theoretical Implications ................................... 104 Reliability and Validity .................................... 107 Limitations of the Study ................................... 109 Programming Recommendations ............................. 111 Conclusions: Perspective on the Research Process and Implications for Future Research .................................... 115 APPENDIX A]: PARENT CONSENT FORM ........................ 117 APPENDIX A.2.: ADOLESCENT CONSENT FORM ................... 119 APPENDIX A.3.: STAFF CONSENT FORM ......................... 120 APPENDIX B]: SEMI-STRUCTURED INTERVIEW FOR FIRST 11 INTERVIEWS .......................................... 121 APPENDIX B.2: SEMI-STRUCTURED INTERVIEW FOR SECOND 11 INTERVIEWS .......................................... 128 APPENDIX B.3. FOCUS GROUP QUESTIONS FOR STAFF ............. 135 APPENDIX B.4. PROGRAM FUNDING PROPOSAL ................... 138 APPENDIX C]: DOCUMENT SUMMARY FORM FOR PROGRAM PARTICIPANTS ........................................ 146 APPENDIX C.2.: DOCUMENT SUMMARY FORM FOR STAFF .......... 147 APPENDIX C.3.: DOCUMENT SUMMARY FORM FOR PROPOSAL ....... 148 vii APPENDIX D.1. CLUB RULES DEFINED BY STAFF .................. 149 APPENDIX D. 2: CLUB RULES DEFINED BY PROGRAM PARTICIPANTS . 150 REFERENCES .............................................. 1 5 1 viii LIST OF TABLES Table 1 Start List for Coding of Program Personnel and Program Participants' Perceptions of Program Processes ............................ 45 Table 2 Revised Start List for Coding of Program Personnel and Program Participants' Perceptions of Program Processes .................... 48 Table 3 Identified Constructs of After-School Program Processes and Components: Durational Contextual Forces ...................... 51 LIST OF FIGURES Figure 1. Developmental Contextual Model of Human Development ......... 25 Figure 2. Human Ecological Model ................................ 28 Figure 3. A Contextual Model of Internal and External Processes ........... 33 Figure 4a. Durational and Operative Forces of After-School Program ......... 91 Figure 4b. Internal Processes of After-School Program ................... 92 CHAPTER I INTRODUCTION: ADOLESCENTS AND ADOLESCENT NEEDS The United States is facing one of the most important challenges in its history. This challenge is to save its youth (Edleman, 1992). Adolescence is a time of cognitive, social, emotional, moral, and physical growth when young people develop the self-identity and the self-efficacy they will need to be healthy contributors to our society as adults (Dryfoos, 1990; Hamburg, 1993; Nightingale & Wolverton, 1993). However, many of the youth of this nation are at risk of not becoming healthy adults because their path to adulthood is blocked by poverty, unemployment, inadequate health care and nutrition, violent communities, drug dependency, teenage pregnancy and parenthood, school drop-out and racism (Benson, 1990; Dryfoos, 1990; 1993; Fordharn & Ogbu, 1986; Kozol, 1991; Lerner 1993a, 1995; Luster & Mittelstaedt, 1993; Prothrow-Stith, 1991; Schorr, 1988; Simons, Finlay, & Yang, 1991). For adolescents who are facing these critical issues, their capability to cope with these problems may be helped or hindered by the simultaneous developmental challenges that all adolescents face (Dryfoos, 1990; Prothrow-Stith, 1991). Adolescence is a time of dramatic changes not only within the young person, but also within the context of the adolescent, as family members and friends react differently to this growing person (Boxer & Petersen, 1986). The normative results of this rapid 2 transitional period for adolescents are not only a physically mature body, but also a sense of identity, an advanced sense of morality that includes feelings of accountability and responsibility to society, and an increased ability to have meaningfirl relationships with both peers and adults (Carnegie Council on Adolescent Development, 1989). Every adolescent experiences these changes; however, the meaning assigned to, and the experiences of, these changes are highly specific to the individual because these changes do not occur in isolation from the context or the environment of the young person (Lerner, 1987, 1989a, 1989b). The immediate context of the child and the adolescent is the family (Bronfenbrenner 1979, 1986). It is within the family that children begin to learn social skills and form their first significant attachments and meaningful relationships (Bronfenbrenner, 1979, 1986). Though the world of the adolescent is broader than the child's, in that peers begin to play a much more significant role in adolescents' lives, the family remains closely connected to the adolescent (Foster-Clark & Blyth, 1991). The American family has undergone dramatic changes over the last forty years (Hernandez, 1993). Hernandez (1993) noted that the percentage of children living in intact two-parent families decreased from 70% in 1940 to 50% in 1988. This dramatic decrease was primarily due to the increase in mother-only families (Hemandez, 1993). In addition to structural changes, American families have also faced dramatic economic changes. In 1960, 57% of America's children lived in "breadwinner-homemaker" families. In 1989, this rate decreased to 25%, and the percentage of children living in dual-earner families increased to 73% (Hernandez, 1993). Hernandez (1993) argued 3 that this increase in the percentage of both parents working was due, primarily, to the increasing economic needs of American families. Furthermore, Hernandez (1993) estimated that by the year 2000 more than 80% of the children in this nation will be living in dual-earner or single-parent families. With these changes and challenges facing the contemporary American family, adolescent developmental needs may not be adequately addressed within the context of the family (Carnegie Council on Adolescent Development, 1992; Lipsitz, 1986). Indeed, the work of Tirnmer, Eccles, and O'Brien (1985) indicated that 42% of young adolescents' time is discretionary, meaning that this time was not committed to school, homework, chores, eating, etc. Furthermore, much of this discretionary time was spent without the supervision of adults. Census data collected in 1984 indicated that approximately 24% of children and young adolescents aged 5-13 were cared for after school by someone other than a parent (Hernandez, 1993). Of these children and adolescents, 7% of them cared for themselves without any adult supervision (Hernandez, 1993). It is unlikely that the demographic changes of the American family are going to be reversed. Thus, it is probable that adolescents will continue to have large amounts of unsupervised discretionary time. Researchers state that harnessing this free time in a constructive manner is one means to provide for and/or enhance the positive development of youth (Carnegie Council on Adolescent Development, 1992). Furthermore, adolescents who live in impoverished communities, where violence is prevalent and quality education is absent, stand to benefit the most from adolescent 4 focused programs offered during the non-school hours (Carnegie Council on Adolescent Development, 1992; Lipsitz, 1986). Programs for adolescents in the non- school hours have been classified as based within the schools, as after-school programs; as based within religious institutions; or as based within the community--for example, Girl Scouts of America, 4-H, Camp Fire Boys and Girls, and the YMCA (Carnegie Council on Adolescent Development, 1992). However, regardless of the locale, researchers and policy makers believe that youth-focused programs possess tremendous potential to promote positive adolescent development (Carnegie Council on Adolescent Development, 1992). In order to capture the potential that lies within youth programs, researchers argue that evaluation should be undertaken (Dryfoos, 1990; Schorr, 1988). Dryfoos (1990) states that all-too-ofien evaluation of youth programs is not a priority among program funders and/or administrators; and, when evaluation is conducted, it is strictly focused on the accomplishment of the program goals. This focus on program goals or impacts, as Jacobs (1988) argues, neglects important information about the context and the functioning of the program that can explain how movement occurs towards the program goals and objectives by the identified users. This present study attempted to evaluate an after-school program by focusing on the context and the functioning of an after-school program. CHAPTER 11 REVIEW OF LITERATURE In order to understand why evaluation of youth programs is needed, this chapter describes what is known about "successful" youth programs. Furthermore, it explores the relationship between evaluation and practice that gave rise to the argument that program processes and program context need to be assessed through evaluation. Finally, this chapter describes the importance of selecting appropriate methodologies in program evaluation. The focus of youth program evaluations on the attainment of program goals is best evidenced in the 1992 report, W W, by the Carnegie Council on Adolescent Development. This report outlined several "goal-focused" evaluations of programs offered to youth in the non- school hours. The evaluations of these programs indicated that involvement in such programs has, indeed, had a positive impact on the lives of early adolescents. For instance, the evaluation of the Teen Outreach program, sponsored and implemented by the Association of the Junior League, revealed that the teens involved in the program were less likely than their peers who were not participants to drop-out of school, 6 become pregnant, or be suspended from school (Carnegie Council on Adolescent Development, 1992). Moreover, an evaluation of a drop-out prevention program implemented by WAVE Inc. demonstrated that participants improved their school attendance, scores on job readiness assessments, and mathematic, reading, and self- esteem scores (Carnegie Council on Adolescent Development, 1992). Finally, the evaluation of the Boys and Girls Clubs of America drug prevention program-«SMART Moves-demonstrated that communities wherein the program was implemented experienced lower rates of alcohol and drug abuse, drug trafficking, and drug related crime in comparison to communities where the program was not implemented (Carnegie Council on Adolescent Development, 1992). While the above programs demonstrate a positive influence on the lives of adolescents, many more programs geared towards adolescent needs are not subjected to the rigors of evaluation (Dryfoos, 1990; Lipsitz, 1986; Simons, Finlay, & Yang, 1991). For example, such programs as the "Midnight Basketball League" in Chicago, "The Door" in New York City, the "Youth Action Program" in East Harlem, or "E1 Puente" in Brooklyn, have not been evaluated either in terms of their eventual outcomes or the processes through which their impacts occur. Though evaluation data of many youth programs are lacking, researchers such as Dryfoos (1990), Schorr (1988), and Simons, et al. (1991), as well as young people interviewed by the Carnegie Council on Adolescent Development (1992), believe that "successful" youth programs have one or more of the following features: 7 * Intensive individualized attention from responsible adults who respect and listen to young peeple; * Engagement of peers in interventions; * A safe, protective environment where adolescents can be themselves; * Clear rules and high expectations for behavior; * Interesting activities that provide opportunities for young people to socialize, learn, and practice new skills, make new fiiends, have new experiences, and explore new options; * Creation of opportunities where young people can think about and plan for the future; * Attention paid to cultural and life-style diversity, as well as to individual diversity; "‘ Involvement of parents; * Effective organizational climate that is flexible and supportive of program's functioning; and * Arrangements for training of staff. Though these ten characteristics are viewed by both youth and researchers as being fundamental to the effectiveness of youth programs, they are by no means the "magic" ten. As Dryfoos (1990) and Schorr (1988) assert, there are limited amounts of evaluation data regarding youth programs. Thus, these characteristics must not be viewed as inclusive of all features that characterize "successful" youth programs (Dryfoos, 1990). Consequently, evaluation of programs must be conducted so that so that additional features of "successful" youth programs can be identified. IlBl'l'l El' lE' Evaluation of human service programs such as the ones noted above has remained inadequate because many practitioners are unwilling to subject their programs to formal evaluation (Jacobs, 1988). This unwillingness is generated from a relationship that began in the 19603 between evaluation and program practice (Jacobs, 1988; Schorr, 1988). In the 1960s, the Great Society programs were implemented, and with their initiation into communities, policy makers and social reformers advocated legislation that would document program accountability and effectiveness (Jacobs, 1988). The Head Start programs were the first programs of the Great Society to be subjected to the rigors of evaluation (Jacobs, 1988). The evaluation design for Head Start focused on assessing the long-term impact of the programs on the participating children. This long-term impact was believed to be a positive change in intelligence and academic achievement. Thus, the primary evaluation objective of this massive summative evaluation was to assess change in intelligence and academic achievement of the program participants (Jacobs, 1988). The first wave of data collected regarding the impact of Head Start was during the period of 1965-1968 (Jacobs, 1988). The results indicated that Head Start had immediate positive effects on the program participants and that there were, indeed, possible positive long-term benefits. However, in the second wave of data collected during the years of 1969-1974, any potential benefits of Head Start appeared to be 9 "washed out" by the time the participants reached elementary school age (Jacobs, 1988, p. 42). Jacobs (1988) asserts that these discouraging results were counter intuitive to her own experience and other child care workers' experiences with Head Start during its early days of implementation. Many practitioners believed that Head Start was making a difference in the young child's life, and that the "failure" of Head Start laid not within the programs, themselves, but within the narrowly focused evaluation design (Jacobs, 1988). This faulting of the evaluation design was also echoed in the criticisms of policy makers such as Senator Patrick Moynihan who stated that the evaluation results were "'fi'agrnented, contradictory, incomplete” (cited in Jacobs, 1988, p. 43). Because these results created a dissonance between what program practitioners and policy makers believed was occurring in Head Start and what science demonstrated, Jacobs (1988) states that the "...love affair with science had dissipated, leaving program directors warier, policymakers more suspicious, and the public increasingly disillusioned with both social programs and social science" (p.44). our“: ' I‘ ' list .I' ' out“: H ‘4 0.1.1. ' ' 0 ‘ ._ .9.” The "backlash" towards program evaluation that was generated from the initial Head Start evaluation not only created an intellectual distance between program practitioners and evaluators, but it also generated new thought in the field of evaluation research about the significance of the process of designing and delivering a program and of the context within which a program was implemented (Greene, 1994; Jacobs, 10 1988; Schorr, 1988). Communities are the contexts in which most direct human service programs are implemented, and communities are complex, multifaceted, and unique from one another (Kretzmann & McKnight, 1993; Lerner & Miller, 1993). The implication is that a program that is deemed effective in one community cannot necessarily be implemented in another community with the same success (Lerner and Miller, 1993). Thus, a program interacts with the context in which it is embedded, and addressing the complexity, multi-faceted nature, and uniqueness of a program's context is important to understanding the effectiveness and functioning of a particular program (Lerner & Miller, 1993). In addition to program context, evaluation researchers also came to argue that the relationship among program components must also be described if an understanding of a program's effectiveness and firnctioning is to be accomplished (Chen & Rossi, 1989; Schorr, 1988; Woods, 1991). The initial evaluation of Head Start was strictly summative in nature, and this means that only the program impacts were assessed (Greene, 1994; Jacobs, 1988; Smith, 1990). Another form of evaluation is process evaluation. Process evaluation is the description of the relationship over time among program components. Program components are defined to be characteristics of program personnel and participants, activities, and goals (Kosecoff & Fink, 1982), and the systematic relationship among these components across time is defined as a process (Ford & Lerner, 1992; Lerner, 1986). By assessing processes and context, Jacobs (1988) asserts that movement towards program goals and/or impacts by the identified users of the program can be understood. 11 Understanding a program's context and the processes within that context are key to evaluating a program's effectiveness and functioning. Moreover, a process evaluation when conducted during the implementation of the program contributes to enhancing program effectiveness because it can facilitate "mid-course" adjustments (Cronbach, 1982; Scriven, 1980). These mid-course adjustments would be generated from the results of the process evaluation. Indeed, both Cronbach (1982) and Scriven (1980) argue that a primary purpose of process evaluations is to provide feedback to the program implementors, so that improvements in the program's functioning, service delivery, accessibility to users, etc., can be made. Not only have evaluation researchers acknowledged the importance of assessing a program's processes within its context, but they have also recognized the importance of selecting appropriate and/or designing appropriate evaluation instruments (Jacobs, 1988; Patton, 1987). Similar to the issue of context, methods that adequately assess a particular construct in one program may not adequately assess that same construct in another program (Jacobs, 1988). To assess the processes of a program, the qualitative methodologies of interviewing, observation, mapping, and charting are thought to be most appropriately used during the initial phases of a process evaluation (Louis, 1981). Information collected by these methods can subsequently be used to generate hypotheses about a program's functioning and to design additional evaluation instruments of either a qualitative or quantitative type (Louis, 1981; Steckler, McLeroy, Goodman, Bird, & McCormick, 1992). As stated earlier, evaluations of youth programs often only focus on the 12 accomplishment of program goals. Thus, information about what occurs to the identified users of the program is yielded; however, information about how these impacts occur is often neglected. Consequently, researchers argue that process evaluations that examine program processes as well as program context yield this additional information about program functioning and effectiveness (Jacobs, 1988). CHAPTER III STATEMENT OF PROBLEM, DEFINITION OF TERMS, CONCEPTUAL FRAMEWORK Given that the examination of processes and context is often neglected in the evaluation designs of youth programs, this chapter describes the focus of this study: The processes that comprise an after-school program. These processes are defined to involve both internal and contextual components. Furthermore, the context of the study as well as the program of focus in this study are also described. Finally, the argument that the individuals involved in the after-school program are a primary source of information about the internal and contextual processes of the program, and the argument that the dynamic interactions of the context and the program can be identified, is legitimated by the conceptual framework presented in this chapter. This fi'amework was constructed by the shared and unique assumptions of Bronfenbrenner (1979, 1986) and Lerner (1987, 1989a, 1989b, 1990, 1993b). StatemenmehLErohlem Human service programs and the processes that comprise them interact with the contexts in which they are embedded (Lerner & Miller, 1993). These interactions influence the functioning and effectiveness of a program (Chen & Rossi, 1989; Schorr, 13 14 1988; Woods, 1991). As noted above, process is defined in this study as the systematic relationship across time among the program components and as the interactions of the components with the context within which they are embedded (Ford & Lerner, 1992; Lerner, 1986). Program components consist of activities, characteristics of the program personnel and participants, and goals (Kosecoff & Fink, 1982). The term "characteristics" is used broadly in this study to mean qualities of personality as well as behaviors and behavioral settings of the individuals (Barker, 1968). The purpose of the present study was to conduct a qualitative evaluation of the processes comprising an after-school program for young adolescents. The outcome of this evaluation was a description of the program's functioning, or of its assets and limitations. The program's assets and limitations were determined by analyzing and comparing the data to the ten characteristics of "successful" youth programs in Chapter 11. As noted earlier, non-school hour programs appear to be successful in promoting positive youth development in young adolescents; thus, the focus of this current study was on young adolescents. W The youth who participated in the after-school program of this study live in the Beecher community, located approximately eight miles north of the city of Flint, MI. The "community" of Beecher was established in the 18005. It was a primarily European-American, rural community. The residents of Beecher never established home rule as a city or town; however, its boundaries are defined by the Beecher 15 Community School District (Meister, 1979). Beecher remained primarily a European-American community until the 19608 when African-Americans were permitted to purchase homes in the area. Despite the European-American residents' protests, the African-American community in Beecher continued to grow in size when federally insured and subsidized housing was made available to them (Meister, 1979). Subsequently, the racism of the European-American residents of Beecher fueled the fires of racial tension in the Beecher community. This tension clirnaxed in 1972 when Paul Cabell, a young Afiican-American man, who was the assistant principal of the high school, committed suicide because he believed that the racial problems of the community could not be solved (Meister, 1979). The history written by Richard Meister in 1979 ended on the optimistic note that the European-American residents of Beecher and the Afiican-American residents of Beecher would some day live in racial harmony as neighbors. Today, the majority of the Beecher community is African-American because of the "white flight" of their neighbors; the beginnings of this flight was recognized by Meister when he stated that in 1971, there were 165 vacant homes in Beecher that were less than 41 years old. Furthermore, with the General Motors factory closings in the city of Flint during the 19803, the community of Beecher, like Flint, was devastated economically. Neither Beecher nor Flint has ever recovered from the plant closings. Thus, the Beecher community could be described as poor and, although segregation is illegal, the racial make-up of Beecher, like many urban communities, raises the question of whether desegregation ever really did occur in this nation (Kozol, 1991). 16 There are two middle schools in the Beecher community--Dolan and Summit Middle Schools. The schools are located within three miles of each other and are considered to be within walking distance for the students. Both schools house approximately the same number of students. The youth of the Beecher community are "at-risk." The Beecher Community School District has the highest percentage of free or reduced lunches in Genesse County, and the 1992 Michigan Kids Count data revealed that 43.6% of Beecher families with children under the age of 18 were below the poverty line. By the time Beecher teenagers reach 12th grade, over 20% of them have dropped out of school. As stated in Chapter 1, programs in the non-school hours possess the potential to promote positive youth development especially for disadvantaged youth. Thus, it appears that the youth of the Beecher community could benefit from a constructive adolescent focused program. I] E EE . l . S l The program on which this study is focused began in the 1992-1993 school year with funding from the Michigan Department of Public Health that was to be used for abstinence focused programs. The program is offered twice a week at each of the middle schools for two hour sessions each time. This program, through an initial review of the program proposal, was designed to delay young adolescents' initiation into sexual activity and to encourage sexually active teenagers to become abstinent. The maintenance or re—establishment of abstinence was to be accomplished through the 17 after-school program by assisting young teenagers to perform better in school, to decrease their truancy, to cope with negative peer pressures, to create positive peer pressures, and to improve their communication between themselves and their parents as well as to increase parents' involvement in their teenagers' lives. The creation of positive peer relations, the improved performance of teenagers in school, the maintenance of abstinence, etc., can be easily assessed through counts and frequencies; however, as researchers such as Jacobs (1988) assert, how these objectives are accomplished cannot be assessed alone through numbers. An evaluation that is focused on the process of a program can assess how movement towards program objectives by the identified users of the program occurs. Furthermore, this particular program has not been subjected to the rigors of evaluation. Thus, given that there are no evaluation data, and given that it is important to understand how program participants may be able to improve their school performance, decrease their truancy, remain abstinent, etc., it was appropriate that a qualitative evaluation be conducted. WWW To study a phenomena qualitatively, the researcher must enter the context of study and must try to understand the perspectives of the individuals in that context (Jorgensen, 1989). This process of entrance into a culture, community, family, etc. is one where the researcher must also become aware of her own value system because she acts as a filter for all of the information that the context and the informants of the context provide to her (F etterrnan, 1989). The evaluator of this program was European 18 -American and raised in a middle-class community. The informants and staff members were African-American and members of the working class community of Beecher. In order to gain entrance into the community of Beecher, as well as to gain acceptance of the program participants and the program staff, the evaluator became a staff member of the program. Prior to conducting the research of this study, the evaluator kept a personal journal of her own thoughts and experiences in the program and in the Beecher community in order to understand how her values, culture, and class were similar to and different from that of the Beecher community. This joumaling process assisted the evaluator in the data analyses of this study because the evaluator was able to ask herself whether or not the interpretations of the data were being driven from the context or from her own value system built from her own race and class experiences. Though it is impossible to separate the individual fi'om his or her value system, it is possible to recognize and to question how this value system may or may not influence the interpretation of the data. Thus, in the following chapters, the interpretation of the data is offered not as the "true" and "absolute" perspectives of the program participants and staff, but it is offered as a carefully examined and worded interpretation of what the staff and the participants had to say about the program. II E . . [I This study was an evaluation of program processes, and as noted earlier, a process evaluation differs from a summative or impact evaluation. A summative evaluation examines a program's achievement of its ultimate goals, whereas a process 19 evaluation focuses on the systematic relationship among program components across time so that feedback regarding the program can strengthen the functioning of the program (Smith, 1990). This study qualitatively assessed the internal processes and the contextual processes of this after-school program. The internal processes consisted of the systematic relationship across time of the program components, whereas the contextual processes consisted of interactions between the context and the program that enhance, maintain, or limit the functioning and existence of the program. By assessing the internal processes as well as the contextual processes of this program, the processes that, together, comprised this after-school program were assessed. The internal processes were defined in this study as the intended, perceived, and actual components of the program and the intended, perceived, and actual systematic relationship among these components. Scriven (1983) asserts that in program evaluation, one role of the evaluator is to build a picture of the reality of the program. Using multiple perspectives, or what Scriven (1983) calls "perspectivism," this reality can be constructed. Moreover, perceptions, per se, represent a legitimate and important focus of inquiry, especially in the initial and formative stages of program development and evaluation (Ostrom, Lerner, & F reel, 1994). In this study, the intended processes and components represented the perspective of the program coordinators when they planned and implemented the program, whereas the perceived components and processes represented the perspectives of the program delivers and the program participants. The actual processes and components represented the perspective of the evaluator. Finally, Scriven (1983) argues 20 that the goal of perspectivism is not simply an accumulation of a set of different pictures of reality; rather, it is one integrated picture. Thus, in this study, an integrated picture of the functioning of this program was created by identifying patterns within the data collected from both the program participants, the program personnel, and the project proposal that were consistent with one another. In short then, through this study, processes that comprised the after-school program for the young adolescents of the Beecher community were specifically addressed by examining the internal processes and the contextual processes of the program. As noted above, the result of the examination of the internal processes and contextual processes was a description of the program's functioning. It was with this description that the program's assets and limitations were outlined for the program designers. The program's assets and limitations are defined to be the comparisons and contrasts with the ten characteristics of "successful" youth programs. or ‘01-... «or .0 ' l‘ e.’.'0 Uh“. l' n ' ,7 9'. u' n- In this present evaluation, the fimctioning of a program was assessed by examining the internal and contextual processes that comprised the program. As noted above, neither internal processes nor contextual processes can be examined as if they are mutually exclusive of each other because the internal processes of a program interact with the context in which they are embedded (Lerner & Miller, 1993). Both Bronfenbrenner (1979, 1986) and Lerner (1987, 1989a, 1989b, 1990, 1993b) have studied extensively the nature of human development, and their theories of the 21 individual as a psychological, biological, and social agent interacting with the context offers a means to legitimate the view that internal and contextual processes influence the functioning and existence of a program. The conceptual framework in the present research was built upon the unique and the shared assumptions of these two theorists. Bronfenbrenner's (1979) theory of the "Ecology of Human Development" and Lerner's (1987, 1989a, 1989b, 1990, 1993b) model of "Developmental Contextualism" are very similar in their assumptions about the individual and his relation to his context. Primarily, both theorists argue that the relationship between the individual and the context is one where the individual is nested (Bronfenbrenner, 1979) or embedded (Lerner, 1987, 1989a, 1989b, 1990, 1993b) within the context, and the interaction between the individual and the context is dynamic (Lerner, 1987, 1989a, 1989b, 1990, 1993b), or reciprocal (Bronfenbrenner, 1979)--meaning that any changes in the individual influence and are influenced by changes in the context. The strength of these theories lies not only within the similarity of their basic assumptions about human development, but also within the depth that the theories offer in regard to certain aspects of the individual and the context. Lerner (1987, 1989a, 1989b, 1990, 1993b) contributes greatly to the study of human development by defining the processes by which human beings act as agents in their own development, while Bronfenbrenner (1979) defines the contexts of humans. . Lerner's (1987, 1989a, 1989b, 1990, 1993b) shared basic assumption with Bronfenbrenner (1979) is that humans are embedded in their contexts, and characteristics of the 22 individual (i.e., biological, psychological, social) and of the context (i.e., societal, cultural, and physical) interact dynamically (Ford & Lerner, 1992; Lerner, 1987). That is, changes in any one level (i.e., individual-behavioral) do not occur in isolation from changes in the levels of the context (i.e., societal); rather, changes in one level are influenced by and influence other levels (Ford & Lerner, 1992). These concepts of embeddedness and dynamic interactionism are demonstrated in Figure 1, which is adapted from Ford and Lerner (1992). The implications of this embeddedness and dynamic interactionism as defined by the "Developmental Contextual" model are threefold (Ford & Lerner, 1992; Lerner, 1987). First, because the levels of the individual and the context exist in a mutually reciprocal manner, the possibility for change is always present. This potential for change is defined by Lerner (1987, 1989a, 1990, 1993b; Ford & Lerner, 1992) as plasticity. Second, because of the plasticity of human development, there exists the potential for intervention by which the quality of human life can be enhanced (Lerner, 1987, 1990). Third, the individual is an active agent in her own development (Ford & Lerner, 1992; Lerner, 1987). This third implication offers three processes by which the individual is a change agent. First, people stimulate other people, and because of this stimulation, a circular feedback loop is created between the individual and the elements of her context (Ford & Lerner, 1992; Lerner, 1987). For example, children who are temperarnentally difficult place different demands on their caregivers (Thomas & Chess, 1977). A child that is biologically arrhythmic-«sleep and eating patterns are not predictable, is 23 negative and quick to withdraw from new situations, and is slow to adapt to changes in the environment-is more difficult to care for than is a child who is rhythmic, positive, and adapts quickly to new situations. Furthermore, children with this difficult temperament are regarded more negatively by their parents, peers, and teachers, and thus they are more likely to have personal, social, and behavioral problems than are temperarnentally easy children (Ford & Lerner, 1992; Thomas and Chess, 1977). Second, people process their world in unique ways (Ford & Lerner, 1992; Lerner, 1987). All individuals experience developmental change; however, they do not experience change in identical manners. For example, during adolescence, all individuals experience a period of rapid growth, however, the timing of this period is specific to the individual. Thus, adolescents in this period of rapid growth may respond differently to a program of physical education than young people who have not yet experienced this growth (Ford & Lerner, 1992). Third, people select and shape their contexts (Ford & Lerner, 1992; Lerner, 1987). This means that individuals choose certain contexts and certain aspects of that context within which to engage, and through their actions, they alter their environments. For example, an adolescent who is small in stature will most likely not select to be a team member of the high school basketball team; however, this same individual who possesses an incredible ear for music will most likely select to be a member of the high school band. Lerner's definition of the processes by which an individual is a change agent in 24 her own development creates a picture of the individual as a self-organizing and self- constructing organism (Ford & Lerner, 1992). The implication of this definition to program evaluation is that to assess the process of a program is to not examine what elements of a program are doing to the identified users, but to assess how elements of a program are perceived by the individual in relation to his or her own needs and development (Ford & Lerner, 1992; Lerner, 1990; Lerner & Miller, 1993). 25 Figure 1 26 W. While the exclusive contribution of Lerner's (e. g., 1987, 1990, 1993b) "Developmental Contextual" model to this current evaluation is that the individual is the source of information about program process, the important contribution that Bronfenbrenner's (1979) theory of the "Ecology of Human Development" makes to this present evaluation is his definition of the contexts of human beings. To Bronfenbrenner (1979), the context of human beings is an ecological environment. An ecological environment consists of nested structures each contained within the next where the individual is located within the innermost structure. These structures are the microsystem, the mesosystem, the exosystem, macrosystem, and chronosystem (Bronfenbrenner, 1979, 1986). This model is portrayed graphically in Figure 2, which is an adapted figure from Keith, Perkins, Ferrari, and Covey (1994). The microsystem is the immediate setting of the individual such as the family, the school, or the community. The mesosystem is the connection between two microsystems (Bubolz & Sontag, 1993; Bronfenbrenner, 1979; Perkins, Ferrari, Covey, & Keith, 1994). The exosystem refers to those settings where the individual is not an active participant, but the events that occur in the exosystem affect or are affected by the events in the microsystem of the individual. Examples of exosystems are the parent's workplace, school administration, parent's network of friends (Bronfenbrenner, 1979; Perkins, Ferrari, Covey, & Keith, 1994). The macrosystem represents the culture or subculture (e. g. government, economic conditions, media) that the other systems operate within (Bubolz & Sontag, 1993; Bronfenbrenner, 1979; Perkins, Ferrari, Covey, 27 & Keith, 1994). The chronosystem is the dimension that recognizes that development of the person and the environment occurs across time (Bronfenbrenner, 1986; Bubolz & Sontag, 1993; Perkins, Ferrari, Covey, & Keith, 1994). In the present study the immediate context, or microsystem, of the after-school program was the school buildings, while the exosystem was the community, the participants' homes, and the school administration. The macrosystem of the community of Beecher was the culture of the community. The macrosystem was described in the preceding section "The Context of Study." The mesosystem was defined as the interactions among the microsystem and the other systems that influenced the functioning of the program. This is the extended definition of the mesosystem that, researchers believe, is appropriate when conducting research within communities (Perkins, Ferrari, Covey, & Keith, 1994). 28 Bronfenbrenner's Ecological Model flociety (Mesosystem) (Chronosystem) ....................................... ,Life Span (Adapted by Perkins, Ferrari, Covey, and Keith, 1994) Figure 2 29 I,“ o 1' no U0" '. uni-.. U00‘ 0 ' m." ' .- . By viewing the individual as embedded in, and dynamically interacting with, the context, Lerner (e.g., 1987, 1993b) defines the individual as a change agent; thus, individuals involved in this program were viewed as the key sources of information about the internal and contextual processes of the program. Furthermore, by also viewing the individual as nested and reciprocally interacting with the environment, Bronfenbrenner (1979) defines the contexts of human beings; thus, the interactions of the context that dynamically influenced the functioning of the after-school program were identified. A synthesis of both Lerner's and Bronfenbrenner's models is provided in Figure 3. The center rectangle of Figure 3 represents the internal processes of the program. The internal processes are the intended, perceived, and actual components of the program and the systematic relationship among those components. As stated earlier, intended components and processes are those specified by the program designers. For example, a school district whose elementary school children have low math achievement scores design a program that they believe will increase the scores. The program that they design uses older children as mentors to the younger children after- school three days a week. The intended components in this program are the goal of raising the math scores, the personnel who display mentoring characteristics, the participants who are the elementary school children, and the content which is a specified curricula to be utilized by the mentors. The intended process in this example is a developed mentoring relationship between program personnel and participants. Perceived program components and processes are those defined by the program 30 personnel and program participants. In the example above, the perceived processes and components would be the thoughts of the mentoring program personnel and of the program participants: moreover, the feelings and attitudes about the goals and activities of the program and about each other also comprise perceived processes. Finally, the actual program components and processes are those independently identified by the program evaluator. In the example above, the evaluator would identify actual program components and processes by systematically observing the functioning of the program. In the present study the actual program components and processes were not examined because the evaluator served as a staff member of the program. This role was undertaken to develop rapport and trust between the evaluator and the program personnel and participants prior to the evaluation activities. As stated in the preceding section, "Race, Values, and Class of the Evaluator," by immersing herself within the context of the program, the evaluator attempted to understand how her race, values, and class affected her perceptions of the program. Thus, an objective, non-participant perspective could not be taken by the evaluator, and consequently, a perspective of the actual program components and processes was not sought. The outer circles of Figure 3 represent the context of the program. The school is the microsystem of the after-school program because this is where the program is conducted. The homes of the participants, the school administration, and the community represent the exosystem of this study. The macrosystem is the culture and designed and natural environments of the program. The mesosystem is represented by the bi-directional arrows between the circles and for the purposes of this study the 31 mesosystem is viewed as it is by Keith et. al (l994)-- those interactions between the micro-, exo-, and macrosystems. Furthermore, the mesosystem in this study is interpreted as the interactions between the micro- exo-, or macro-systems that maintain, or present assets and limitations to, the functioning and existence of the program. For example, if space for the program has been promised to the program by the school administration, but the administration fails to communicate this to other programs, then competition for space within the school represents a limitation to the program's functioning. This is a limitation that is initiated by the microsystem of the program. Finally, temporality--the chronosystem--is represented by the arrow labeled "Time." In addition to not examining the actual processes and components, another limitation of this study is that the design is cross-sectional. Although processes change over time, resources were not available to conduct a longitudinal evaluation of this after-school program. Thus, the chronosystem was not a part of this research; as such, each of the perspectives of the staff, program participants, and the program designers were viewed as representing single moments in time. In this study, the shared and unique assumptions of Bronfenbrenner's and Lerner's theories of human development provided support to two conceptual arguments in this study. First, the theories offered support to the argument that the individuals involved in this program were a legitimate source of information about the contextual and internal processes of the program. Second, the theories offered support to the 32 argument that the dynamic interactions of the program context could be identified. The operationalization of these arguments is presented in the next chapter. 33 EmkmkwomoSe w._.Zao< EEmEa wh2m20n=200 21.9.0011 szzowmma 500mm Figure 3 CHAPTER IV EVALUATION FRAMEWORK The previous chapter presented two arguments that were a part of the conceptual fi'amework of this study. One argument was that individuals involved in a program are a legitimate source of information about the program processes and context. The second argument was that the interactions between the program and its context can be identified. This chapter describes the operationalization of these arguments. That is, the chapter delineates the objectives of the study, the questions of the study, the approach of the study, the procedure, and the methodology. The overall objective of this current evaluation was to examine the processes that comprised an after-school program for the young adolescents of the Beecher community; this objective was pursued in order that the program's functioning could be described and the program's assets and limitations could be assessed. As noted in Chapter III, this program's assets and limitations were determined by analyzing and comparing the data derived from this study with the ten characteristics of "successful" programs outlined earlier (see Chapter II). In order to reach an understanding of the internal processes, as well as the contextual processes, that present limitations and 34 35 assets to the functioning and existence of the program, the following specific objectives were divided according to the intended components and processes; the perceived components and processes; and the contextual limitations, maintenance, and assets of the program. Finally, as noted earlier, the actual program processes and components were not examined because the evaluator also served as a staff member of the program. WW. Here the objective was to describe the goals, activities, expected characteristics of the staff, expected characteristics of the participants, and the relationship among the program components as defined by the program designers. WWW. Here the objectives were the following: to describe the program participants' and the program personnel's perspectives of the program goals; to describe the program participants' and the program personnel's perspectives of the program activities; to describe the program participants' and the program personnel's perspectives of the characteristics of each other; and, to describe the program participants' and program personnel's perspectives of the relationship among the program components. WWW. Here the objective was to describe the spectrum of interactions--assets, maintenance, and limitations-- of the program within the micro-, exo-, and macro-systems. 36 E l . U . By firlfilling the evaluation objectives listed above, the following evaluation questions were answered in this study: Questions Relating to the Internal Processes 1. What are the variables that describe the internal processes of the after-school program? 2. How are these internal variables maintained in the after-school program? Questions Relating to the Contextual Processes 1. What are the contextual variables that affect the program's existence? 2. What are the contextual variables that affect the program's functioning? Questions Relating to Program Assets and Program Limitations 1. How does the described functioning of the program compare and contrast with the ten characteristics of "successful" programs? ExalumimLAnpmach The evaluation approach for this current study was a cross-sectional case study of a dual-site after-school program. As indicated earlier, the focus of this case study was to assess the processes that comprised an after-school program in Beecher, MI. The after-school program--known as "Club" by the staff and program participan -- was implemented in both of the school district's middle school and serves approximately 200 students. Students attend the program twice a week. There are six adults and three 37 high school teenagers who staff the program. Data regarding the program participant's and personnel's perceptions of the goals, activities, and the characteristics of each other, as well as data regarding the contextual assets and limitations, were collected through the use of a semi-structured interview and a focus group. The semi-structured interview was conducted with a sample of program participants while the moderated focus group was conducted with all of the staff members. Data regarding the intended program processes and components was collected through program documentation, such as the program proposal. Procedure WW. Given that the methodology in this present evaluation yielded large amounts of data that were qualitative in nature, not all of the students involved in the program could have feasibly served as informants. Thus, the sample size was limited to 24 students in order to get equal numbers of respondents from each grade, gender, and school. The actual number of students that were interviewed was 22. Program participants were recruited through an announcement made during the after-school program. A total of 29 students turned in permission forms to the evaluator. The 22 students interviewed represented those students that were present at Club for the last three weeks of the program. All 29 program participants who turned in parent permission forms were given a small gift of colorful pens. A copy of the parental consent form is included in Appendix A. 1. Furthermore, a copy of the consent form used with each of the respondents an a copy of the consent 38 form used with the staff are included in Appendix A2. and A3. WW8. Of 22 students interviewed, 5 boys and 5 girls were from Dolan Middle School, and 8 girls and 4 boys were from Summit Middle School. Seven of the respondents were in fifth grade, 8 were in sixth grade, 6 were in seventh grade, and l was in eighth grade. Of the 9 who turned in permission slips but were not interviewed, 7 were from Summit Middle School and 2 were from Dolan Middle School. The difference between the schools in the willingness to participate in the program may have been due to an event that was occurring simultaneously with data collection at one of schools. Dolan Middle School, during the month of April, 1994, came to the attention of the media for the frequency of student behavior problems in the school. What sparked this interest was a letter, unauthorized by the school administration, sent home to the parents outlining the discipline problems of the students. A May 6, 1994 article in Iheflintjmmal described this letter and, as well interviewed some of the teachers of the school. It is believed that the students of Dolan may not have wanted to participate in the research because they may have viewed it as threatening. However, this difference in participation rates appeared to only affect the evidence supporting the presence of helping behavior. The existence of helping behavior among the program participants was supported primarily by students at Summit. In addition to the difference between the schools, more girls than boys volunteered to participate in this study. There does not seem to be a specific reason 39 related to the program for the gender difference, because almost all informants reported that there was an equal number of boys and girls participating in the program. However, Rosenthal and Rosnow (1975), who conducted a comprehensive study of volunteer subjects, found that, in general, volunteers in research studies are more likely to be female. Similar to the difference between the schools, this gender difference in participation rates was substantively significant (albeit, because of the N5 involved, not statistically) in the program participants' reporting of the existence of helping behavior among their peers. More girls than boys reported the existence of helping behavior among their peers in the program. Furthermore, even though there were more girls who participated in the study, boys outnumbered the girls in their comments about the "safeness" of their community as well as their own physical safety. Finally, more students in the fifth and sixth grades volunteered to participate in the program than did students in the seventh and eighth grades. The information that was collected from the older students did not appear to contradict the information fiom the younger students. However, the differing participation rates of the older students did seem to be a reflection of the difference in rates of participation in the after-school program. The implications and reasons for the gender, school, and age differences in participation are discussed in the "Results." Methcdclcgy WW. Two important issues to consider when designing and/or selecting an instrument for a study is the purpose of the study as well as the 4O developmental levels of the population (Baltes, Reese, & Nesselroade, 1977; Fisher & Lerner, 1994). In their qualitative studies of youth "at risk," Weber et a1. (1994) found that both unstructured or semi-structured interviews were feasible methods by which researchers could adequately address the purposes of studies that were focused on process issues. Furthermore, Weber et a1. (1994) found that researchers, when using these two methods in qualitative research, could remain sensitive to the developmental level of adolescents. For example, in their work with adolescents from "at-risk" populations, Weber et a1. (1994) state that, when reading interview questions aloud to respondents, a higher response rate was generated than was the case among those students who read the questions to themselves. Weber et a1. (1994) believed that the higher response rate was due to the greater sensitivity to the literacy levels provided by the "reading aloud" method. Weber et a1. (1994) also found that some pre- and young adolescents had difficulty answering fixed response format questions, such as Likert-type items, because they lacked the cognitive maturity to understand how to respond. Thus, Weber et a1. state that open-ended questions are a potential format that researchers could effectively use with such pre- and young adolescents. Given these methodological considerations, the measure utilized with the young adolescents of this present evaluation was a semi-structured interview that was administered individually to the informants. Completion of the interview took between 20-30 minutes for all program participants. Initial development of the instrument 41 involved using feedback from the adult staff members about the questionnaire. That is, after reading drafts of the questionnaire, staff members provided the evaluator with written and/or verbal feedback regarding both content and format of the questions; as a consequence, the instrument presented in Appendix B] was constructed. Furthermore, because the evaluator served as one of the adult staff leaders throughout the school- year, she had the opportunity to observe and to become cognizant of the various interrelationships of the program and the context. In addition, she was able to develop the necessary rapport with the program participants and program personnel that qualitative researchers deem important to obtaining valid results (Jorgensen, 1989; Marshall & Rossman, 1989). The questions were read aloud and the informants' responses were audiotaped. The semi-structured interview for the first 11 interviews consisted of 32 open-ended questions about the program participants' perceptions of the following: the program goals and activities; the characteristics of other program personnel and other program participants; the processes among the program components; and the contextual assets and limitations involved in the functioning of the program. As Miles and Huberrnan (1994) assert, the qualitative data analysis begins well before the final transcription of the last interview. Rather, qualitative data analysis begins fi'om the moment the researcher selects the topic to study (Fetterman, 1989). Thus, the qualitative research process is a continuous process of discovering, verifying, and re-discovering. This re- discovering occurred in this study when the evaluator found that after the first eleven interviews some of the questions needed to be re-worded and collapsed, and three 42 additional questions needed to be asked. Thus, the second half of the interviews consisted of thirty-four questions. A copy of the instrument with these revisions is provided in Appendix B.2. Questions 9 and 10 fiom the first instrument were collapsed into Question 9 in the second instrument and a prompt. This changes were made when it was realized that the participants usually participated in the activities that they liked the most. Question 13 was reworded into Question 12 on the revised instrument when it was realized that the answers to Question 13 from the first 11 interviews were ambiguous. Furthermore, questions were added that pertained to the program participants' associations with other extra-curricular activities, their perceptions of the safety of the program, and their perceptions of the differences and similarities between attending the school and attending the program. These questions were added because information in the first eleven interviews revealed that were some concepts that could be better discovered through direct questions. For example, the fifth interview conducted with a boy in fifth grade revealed that the "safeness" of Club should be assessed directly since this boy stated that he joined Club because "If we go home, there may not be nothing to do. At least here, you safe and there is something to do." Thus, a question about the safety of Club was added. W12. A focus group was conducted with those individuals who are the staff members of the program. There are five adult staff members, excluding the evaluator, and nine high school staff members. Although all nine of these students served as teen staff members throughout the school year, the program actually operated 43 with approximately three to four teen staff members daily because these teenagers had other extra-curricular and employment responsibilities. The potential impact of this inconsistency of the teen staff on the program is discussed below. The focus group questions examined the program personnel's perceptions of the following: the program's goals and activities; the characteristics of other program personnel and the program participants; the processes among the program components; and the contextual assets and limitations involved in the functioning of the program. The focus group was conducted by the evaluator, and as stated above, a rapport was developed between the evaluator and the individuals who participated in as well as staffed the program. The focus group took an hour and fifteen minutes to complete. A copy of the focus group guide is included in Appendix B.3. WW. While the semi-structured interview and the focus group with the staff were the methods used to obtain data about the perceived internal and contextual processes, the program's funding proposal was a third source of data pertinent to the intended internal and contextual processes. This fimding proposal was chosen to represent the intended program processes because it was an outline of the objectives, of the goals, of the target population of the program, and of the desired characteristics of the personnel written by the project designers. A copy of the program's funding proposal in Appendix B.4. CHAPTER V CODING AND ANALYSES The purpose of this study was to qualitatively examine the internal and contextual processes of an after-school program for young adolescents in order to identify the program's assets and limitation. Miles and Huberman (1994) argue that the coding of qualitative data is structured from the conceptual fiamework and the hypotheses of the researcher. As outlined earlier, the internal processes in this study were the intended and perceived components of a program and the systematic relationship among those components. Furthermore, contextual processes were conceptualized in this study as those conditions of the program's context that maintain and/or present assets and limitations to the functioning and the existence of the after- school program. Table 1 is a "start list" of codes that reflect this present study's conceptualization of processes that comprise this after-school program (Miles & Huberman, 1994). The codes are descriptive in nature because they were used to identify the intended/ perceived processes, and the contextual conditions that maintained and/or presented limitations and assets to the program. 44 45 Table 1 .-r ' 0 Hi." 'IO'IHI' .i-Buur-u'orui.“ ‘ou o "outzu'qo,' Intended Definition IN: Goals intended program goals IN: Process intended program process IN: Personnel IN: Activities IN: Program Participants intended characteristics -personnel intended program activities intended characteristics -participants Perceived/Program Personnel Definition PP: Goals PP: Process PP: Personnel PP: Activities PP: Program Participants Perceptions of goals by program personnel Perceptions of process by program personnel Perceptions of program personnel by program personnel Perceptions program activities by personnel Perceptions of program participants by personnel Perceived/Program Participants Definition PT: Goals PT: Process PT: Personnel PT: Activities PT: Program Participants Perceptions of goals by program participants Perceptions of process by program participants Perceptions of process by program participants Perceptions of activities by program participants Perceptions of program participants by program participants Contextual Assets/Limitations Definition PP: Assets PT: Assets PP: Limits PP: Limits Perceptions of contextual assets by program personnel Perceptions of contextual assets by program participants Perceptions of contextual limits by program personnel Perceptions of contextual limits by program participants 46 D .. :1. The semi-structured interviews and the focus group were transcribed by the evaluator. After transcription, each of the tapes were listened to again to verify the accuracy of the transformation from the spoken to the written word and to be certain that vocabulary and grammar were preserved and not spuriously manipulated. Following this process, the start list was utilized to code the data in this study. After completing descriptive coding of two of the interviews, it was recognized that there were three problems with the descriptive coding scheme. First, the concept of a "process" needed to be further operationalized. It was decided that a process would entail a description of the beginning, middle, or end of an event or condition perceived by the program participants or staff or as stated in the program proposal. For example, a girl in fifth grade responded to the question about to whom she goes when she has a problem with another student by stating "First, I tell this club (the staff) and then I tell my parents." This response demonstrates a beginning or initial step that the program participant takes to cope with problems she has with other program participants. The second problem was that the concept of a " goal" needed to be further operationalized. One author from the program planning and evaluation literature suggests that goals are terminal actions sought after through either approach or avoidance measures (Boyle, 1981). Thus, those segments of narrative text that best fit this definition of a goal were descriptively coded as such. For example, a boy in fifth grade responded to the question about the reasons for providing the after-school 47 program by stating that the program was provided to prevent kids from doing bad things like smoking cigarettes. This text fits this definition because it describes an action or a state that the program is striving to in the eyes of the program participant, that is, "kids not smoking." Third, it was found that a large quantity of data could not be logically classified by the initial codes. The common thread through these "unclassifiable" data was that an event, person, process, or activity in the program was presently filling a void or an absence for the program participants. For example, when answering the question regarding reasons for joining the after-school program, one boy in sixth grade responded, "If we go home, there may not be nothing to do. At least here, you safe and there is something to do." Another student, a girl in sixth grade, responding to the same question said "To have something to do after school and not go home. Here it's funner than going home and just playing or just listening and watching T.V. and cleaning up." Clearly these responses demonstrate instances where the program is providing participants with safety and entertainment. Thus, there were elements in the program that were serving a purpose for the program participants, and, consequently appropriate codes were added to the start list of coding. Table 2 reflects these additions. Table 2 i. ‘ II . Expresses d 0' I?" Intended Definition IN: Goals IN: Process IN: Personnel IN: Activities IN: Program Participants IN: Program Purposes intended program goals intended program process intended characteristics -personnel intended program activities intended characteristics -participants intended purposes of program Perceived/Program Personnel Definition PP: Goals PP: Process PP: Personnel PP: Activities PP: Program Participants PP: Program Purposes Perceptions of goals by program personnel Perceptions of process by program personnel Perceptions of program personnel by program personnel Perceptions program activities by personnel Perceptions of program participants by personnel Perceptions of program purposes by personnel Perceived/Program Participants Definition PT: Goals PT: Process PT: Personnel PT: Activities PT: Program Participants PT: Program Purposes Perceptions of goals by program participants Perceptions of process by program participants Perceptions of process by program participants Perceptions of activities by program participants Perceptions of program participants by program participants Perceptions of program processes by program participants Contextual Assets/Limitations Definition PP: Assets PT: Assets PP: Limits PP: Limits Perceptions of contextual assets by program personnel Perceptions of contextual assets by program participants Perceptions of contextual limits by program personnel Perceptions of contextual limits by program participants 49 Battemmding Following descriptive coding of qualitative data, Miles and Huberman (1994) state that descriptive codes can be used to construct pattern codes of the data. Pattern coding was defined in this study as the conceptual congruities within the perceived program processes/components and/or between the intended and perceived program processes/components. This pattern coding was accomplished through the development and use of "document summary sheets" that in this study served as a means to summarize the descriptive codes (Miles & Huberman, 1994). In each transcription, the descriptive codes were transferred from the margins of the transcription and written on the data display sheet according to the category that they described--i.e. program goals, program personnel, etc. Copies of the document summary sheets for the interviews, the focus group, and the program proposal are provided in Appendices C.1., C2, and C3, respectively. Though each of these sheets were utilized primarily to summarize the descriptive codes found in each of documents, one section labeled "Red Flag" was used for comments regarding data that did not fit into any of the descriptive codes. Use of this code avoided filtering of any data that could potentially lead to construction of new descriptive codes and/or identification of patterns that were not initially described by the descriptive codes. This "Red F lag" section was also used for comments about the program participants regarding the interview process with them. For example, in one interview with a fifth grade girl, the following note was made: "This student was 50 very difficult to interview. I only put things on this sheet that I felt confident about." Miles and Huberman (1994) note that document summary sheets provide a means to summarize a multitude of information; in this study, use of these sheets reduced 260 pages of text to approximately 30 pages. In addition to making the data manageable, use of these sheets allows for another layer of data to be added to the study because the document summary sheets can be used to construct pattern codes. In this study, the document summary sheets were, in fact, used in this manner. By photocopying each of the document summary sheets, and by then highlighting consistencies across all twenty-two interviews, the focus group, and the program proposal, as well as, by highlighting concepts unique to each of the three sources of data, pattern codes were created. These pattern codes or variables provided the information needed to generate the findings discussed in the "Results" section. Table 3 lists these variables along with the sources of the concepts and how they were noted with the descriptive codes. In sum, the variables presented in Table 3 represent data that were both unique to and shared among program participants, the staff, and found in the program's funding proposal. With these variables identified, the next step in data analyses was to provide answers to the evaluation questions presented in Chapter IV. The following chapter provides these answers. 51 manage a? -1585, Samoa v Samoa N we undo—Em . beam namugq $530389 Enact no , :5on c gang v 5&3ch . nfinfimta 8835 we _ :8..on m omen—3°5— fifinmumtag 08o: can? gong m dogma: afifioubm 3.58800 :3..an c £345 3:88an flfiqfigq 082. 8 DE bfim gag N Eur—moi N Ammaoamod flfiafimtam Ear—wen c Eovocom team flaufiomtaa 833:3. 8.2on w .650 323:qu can 6388532 .fiomm< $53298“— mommooocm 63:80 :5on 85:63. mam 809?. E380 “ 73.... v....&..fl-...-..<.... ..,. ..%3.I. .4: .5... .. ... m 03$. 52 “533:3 “in". 83:80 j :5on fl 3an cam 9:23 new Bose.— Bmoaoa ocaogm :3..on 186.2.— «confines— can 66:88:32 .383 fififiomtam 8308:— 88280 :5on moE>zo< beam 809:5 :80 E880 ||||I|I|fl|ll|l|lll fl Sauaomta noun Eur—wean _ flan—Sc. tam 3§&o_§a fifiafimtaa .8305 ton—96 :3on M: Samoa 3 aor— “scam 838385 23 6055532 .88.. 35925.— 83305 i880 Eflwoi 8:333. tam 869:5 Bow E0280 53 Em 388.5 32.89 338m 88»ch Eflwoa EEwPE 3.85593 flfifioga fifiafigq :5on e EEMEQ N Haw—moi m w£§3 flfiqmogq 358353 Samoa v :3..on E .5..— Emoaoa . 8&ch H agnmoga aquamoga moE>bo< * £08 2 8.2on m 920 80:83.5 62888582 .338 38863; 83895 38:80 paw—moi mo£>no< tam 809:5 Row 53.80 9809 g tam “.333 W Egon—um Begum 3% ~35 .8388: 805883 6888532 5034‘ awn—«£095 8332‘— 68280 :3..on 8:393 .mfim 8% :80 E380 54 .8805 figmen— 80323 .8235 fifinmogm 0328“" .«o :5on gas m canon—8m 3.389 Ewan—9a Samoa 82253“ 88»ch 35%.an *5. mo ham go:— a 5:835 tam .8oaoa Samoa anamomtaq £252 33— bfim Samoa 2 263cm .535; 33 Samoa _ tam .nozmm €8.35 :5on agqfifiaa bummm Efiwoa o bfim Bowman— _Soao.a Samoa mucus—om 35925a figqfifiaq 35%.»:an boom Efiwoa .N Efiwoa m :8..on m 2’58.“ mcozggq 60:38532 .8omm< aqua—Egg 8882A 38:80 ENEoE moE>zo< ham 089:5 .80 E830 55 580.5 58833.“ 35858; SEon :5on _ mega—Sm 3:8658 2:253” Eflwoa m mama—o: Aflowmaooc $§n§§n 88on 2 E25 afiaoia been; 82on 3 gm 823%: 6338532 .3omm< 35925.n— mommooci 88:50 Eur—moi moE>to< tam 089:5 390 33.80 56 Illllnlllllll = 3:36?“— Eflwoa m ooaoaoqaoo 23: #83830 Bacon—um .«o EofiBm 8:32 033 tam .Eofiaa‘ 828 fifiéomgq -3380 «o EEwEQ 2 8080285 .5333— Sm 83883.".— afiafiga mo cows :5on n 6658800 8233.54 605:3522 .38m< fifinmogm 8332A 3.8qu Eflwofi moE>zu< tam 089:5 .80 E850 n... "H— CHAPTER VI RESULTS Through the use of descriptive coding and the development of pattern coding, the internal and contextual processes of the after-school program in Dolan and Summit Middle Schools of the Beecher Community School District were evaluated. By interviewing the adolescents served by the program, by conducting a focus group with the adults and teenagers who staff the program, and by reviewing the program's funding proposal, information about the program was provided from different perspectives. As stated earlier, Scriven (1983) asserts that, in program evaluation, one role of the evaluator is to build an integrated picture of the program's reality from multiple perspectives. Thus, the findings in this section are not presented according to the "intended program processes" and "perceived program processes;" rather, they are presented as the answers to the evaluation questions in a conceptual manner that weaves the perspectives together. This weaving was accomplished by using the concepts generated in Table 3. Q . B l . l C E l E The first evaluation question that was answered by examining the concepts in Table 3 was: What are the contextual variables that affect the program's existence? 57 58 Certain variables listed in Table 3 appeared that affect the attendance of the middle school adolescents to the program. Without students to participate in the program, the existence of the program would be threatened. That is, the influence of these variables on the program participant's attendance could potentially mean the life of the program is supported or is threatened. The term "durational contextual force" was developed to describe an event or state in the micro-, exo-, or macro-system of the program that either positively or negatively affected the existence of the program. The terms positive and negative refer not to the inherent quality of the variable, but to the potential to contribute to the program's survival or the program's termination. Following the creation of this term, those variables in Table 3 that appeared to support this definition were classified as "durational contextual forces." These variables were the following: "Other Activities," "Boredom," "Perception of the Program," "Responsibility at Home," "Unsafe Community," "Unsupervised Home," "Knowledge of the Program," "Structure of the Program, " "Parent Support," and "Transportation." W mm. For the program participants, the variable of "Other Activities" provided a positive force to the existence of the program. This force was positive for three reasons. First, three program participants noted that outside of school organized athletics, there were very few extra-curricular activities offered for them in their community or in their school. The results of the staff focus group and the analysis of the program proposal also supported this notion. For example, the program proposal, 59 which represented the intended program processes, stated that "Currently, the middle schools lock their doors at 3:00 pm. each day, except for organized sports." Second, although eight children reported that they were involved in other activities, these activities either did not interfere with the after-school program or had been discontinued. For example, two participants, who were sisters, were involved in a Girl Scout troop that held meetings on opposite days of the after-school program; however, another participant who was involved in Girl Scouts quit the program "...'cause we ain't do nothing, it was just boring, the one we went to. We were suppose to go on a field trip on our last day, but we ain't go, and the Girl Scout lady kept on coming late..." Two boys were in an intramural basketball team program which was started by the middle school basketball coach but it was discontinued when the coach did not have the time to commit to the program. The variable of "Other Activities" provided a negative force to the existence of the program when it described the reasons non-participants had for not attending the program. As demonstrated above, there were very few activities available for the middle school students in this community; however, the activities that were available appeared to be offered primarily to the older students, i.e., those in seventh and eighth grade. Three program participants responded to the question about why other students do not attend the after-school program by citing "organized" athletics. This finding was, again, supported by information from both the program proposal and the staff focus group. For example, one staff member stated, "Some of the older kids, they are more involved in athletics, so they spend a lot of time doing 60 after-school athletics and that cuts into Club time." This concept that older kids have more opportunity to engage in after-school activities is also supported by the perceptions of the staff and of the program participants, in that the after-school program is seen as primarily being attended by younger students, those in fifth and sixth grade; and by the fact that of the eight children who reported that they were involved in other activities, four of them were in seventh grade and one was in eighth grade. Thus, the variable of "Other Activities" positively affected the existence of the program when it was applied to fifth and sixth graders who do not have as many after- school opportunities in which to engage as seventh and eighth graders; however, it negatively affected the existence of the program when it was applied to non- participants who tended to be seventh and eighth graders who have the opportunity to participate in organized athletics. Boredom. As demonstrated above, there appeared to be a lack of extra- curricular activities available for fifth and sixth graders in the Beecher community. Thus, it seems logical that the variable of "Boredom" would describe the Beecher young adolescents' attitudes about their time after-school. Indeed, the variable of "Boredom" also appeared to be a positive force towards the existence of the program. For example, one adolescent in fifth grade responded to the question about why she attends Club by stating, "Because, instead of staying at home being bored, you can have an after-school activity, and just stay off the streets and stuff. " Another girl in seventh grade stated that she comes to Club "so I could have something to do 61 after-school. So I ain't have to go straight home because it would be boring, especially in the winter time." Remnsibflmhgme. Similar to the variable of "Other Activities," the variable of "Responsibility at Home" positively affected the existence of the program as applied to those children participating in Club. Two program participants stated that they attended the after-school program as a means to escape responsibilities, such as chores or babysitting after-school. For example, one girl in seventh grade responded to the question about what she would miss if the program was not offered by stating, "Well, mostly the fun, 'cause sometimes we go home, we got to clean up the house..." However, when program participants were questioned about reasons why other adolescents have not joined the program, two participants stated that responsibilities at home most likely prevented them from attending. One boy in seventh grade responded to the question about why other students have not joined Club by saying, "...some parents just don't like them to come, 'cause they have other stuff to do. Like some parents have jobs and want the kids to babysit, stuff like tha ." 11W. For program participants, particularly boys, the variable of "Unsafe Community" seemed to be a strong positive force towards the existence of the program. When asked about whether or not they thought Club was a "safe" place to be, six of the nine boys interviewed responded in very similar manners. For example, one boy in fifth grade stated that Club was a safe place to be because it means being protected "...from getting beat up, wrestled with, stabbed, throwing rocks at you, people hitting you." Another boy in seventh grade responded to the same question by 62 stating, "...when you at Club you don't have to worry about nobody putting no gun to your head, it going off or when you at Club, you don't worry abut getting hurt, you go walking down and somebody putting a knife up to your neck and cutting it off." Thus, the evidence that the community of Beecher is unsafe, particularly for boys, created a positive force to the existence of the program. That is, boys attended the program to be protected from dangers in their neighborhood. W- The variable of "Unsupervised Home" is similar to that of "Unsafe Community" in that it provided a positive force towards the existence of the program and it was solely defrned by the male participants. When responding to the question about his reasons for attending Club the previous week, one boy in sixth grade responded by stating "'Cause, it's like I have something to do and I won't be at home by myself..." Another boy in seventh grade responded to the question about what there is to do when he comes to Club by saying "It's better than staying at home, 'cause we go home, I'm the only one that's there, and my mother's at work 'till like five..." Consequently, the program appeared to be providing supervision for some of the boys in the community. Wm. The variable of "Knowledge of the Program" was a negative force against the program when it was applied to those students who were not participants. When asked about why other students have not joined Club, five adolescents responded that other students in their school were not a part of Club because they lacked knowledge about the program's functioning and/or existence. For example, one boy in sixth grade stated that other students have not joined Club 63 "...because they don't know the good things that's happening." Consequently, this lack of knowledge about the program that other students may have potentially could be preventing their attendance, thus providing a negative force towards the existence of the program. WW. Another negative force towards the existence of the program which, again, was defined by non-participants, was a negative perception of being a member of the program. Both the staff members as well as program participants believed that other students have not joined Club because they held the perception that it was childish to be a part of the program. One teen staff member stated "I think some of the older kids think Club is childish or something;" while another teen staff member more clearly defined the root of the negative perceptions of Club by stating that other kids have not joined Club because of "peer pressure, 'cause somebody might say, 'Ahh, that's chatty,’ and might not go because they think like the older kids, that's probably why they don't go. They want to are a part of the program) but they friends say, 'Ohh, you childish to go to Club, you going to be at Beecher (the high school) next year." This belief that non-participants hold a negative perception of Club membership was also supported by program participants. For example, one student said that other kids do not come to Club because, "...some of them (non- participants) say 'That's kiddy--all kiddy games and I don't want to be in no Club.’ While non-participants appeared to hold the perception that Club was childish or, as in the words of a teen staff member, "chatty," this perception was not held by those involved in the program on a regular basis. One of the seventh grade boys who 64 consistently attended the program stated that the reason he came to Club was "....I came 'cause I always, I want to come to Club, it's fun,...., it's better than anything I rather want to do..." Consequently, the positive perception of the program held by the regular program participants was a positive force towards the program's existence while the negative perception of non-participants may be a negative force. W. The root of the negative perception held by non- participants may not lie alone in the peer pressure not to join; it may also lie in a variable that was coined by one of the adult staff members of this program. This is the variable of "structure." The variable "Structure of the Program" was defined in this study as the expectations for behaviors or the "rules" of the program. Though the variable of "Expectations for Behavior" played a very important role in the internal functioning of the program, the variable of "Structure of the Program" was used during the focus group by one of the staff members to refer to reasons why older students did not attend Club. This staff member believed that the older students in the middle schools did not attend Club because "...when you got rules and you have to follow rules, the younger ones (students) are more susceptible to 'O.K., this is great, this is structure here, let's follow the rules.’ and the older ones are like 'I'm not going in there, 'cause if I go in there I got to follow the rules.‘ So, structure versus unstructure, you know, they (older non-participants) try to avoid that I guess." Additionally, the existence of the variable of "Structure of the Program" as a negative force on the existence of the program was also supported by the program participants. For example, one girl in fifth grade responded to the question about why 65 other students have not joined Club by stating, "Because some of them think it ain't fun because they can't do what they want to do." Emma. Another variable that positively affected the existence of the program for program participants but may have negatively affected the existence of the program for non-participants was "Parent Support" of the program. In this study, three levels of parent support of the program as it pertained to the program participants were defined. One level was whether or not students had difficulty in getting parents to sign the permission form to be a part of the program. The second level was whether or not program participants who had parental consent were prevented, for various reasons, from coming to the program by their parents. The third level was whether or not parents have ever "helped" out with the program. On the first level, only one student reported that he had difficulty getting permission to join the program. This difficulty stemmed fi'om the fact that his mother worked nights and was ofien asleep when he was leaving for school, thus making it difficult for him to get the form signed before he lefi for school. Furthermore, in response to the question about whether or not anyone has prevented their attendance, only two program participants, who were sisters, stated that their parents had prevented them from coming once (because they were grounded for misbehavior). In terms of the parents assisting the program, only two students stated that their parents assisted the program (by going on two of the fieldtrips). In turn, six program participants believed that their parents would want to volunteer time at Club, but they were limited because of their work schedules. 66 The staff‘s view of parent support centered around this third level, i.e., parents volunteering time to the program. One staff member described the staff's efforts at getting parents involved in the following way: "In the past, we've made great efforts to get parents involved. We've signed parents up, we've sent out our permission forms, 'if you like to participate, turn this in, let us know.’ We've called the parents of kids who've already signed up. We've made massive efforts to get parents involved and to no avail..." Though parents of program participants did not volunteer to assist the program, they did support the program by allowing their children to attend. For non-participants, this lack of support for program attendance may be what was preventing their membership. Three of the program participants stated that other kids did not join the program because their parents prohibited them. Thus, "Parent Support," when defined solely as permission for attendance to the program, positively affected the existence of the program when applied to the participants; however, it may have negatively affected the program when applied to non-participants. Imspgrtatiqn. The last durational contextual force that played a role in affecting the program's existence is the variable of "Transportation." Both of the middle schools are located within the neighborhoods that they serve. However, one female student in seventh grade believed that lack of transportation prevented other students from attending Club by stating, in response to the question about reasons other students have not joined Club, "...maybe they don't get a ride home and they don't feel like walking if they stay far. " The lack of transportation was not supported by any other 67 program participants or by the staff or program proposal; but since this was a descriptive study of program processes, this potential negative force against the existence of the program should not be ignored. Qperatixefiontextualiom While the variables described above were defined as the forces that affected the existence of the program, there were also forces in this program that appeared to affect the program on a micro-level. This micro-level was the program's daily ftmctioning. These forces provided the answer to the second question about the program's contextual forces which was the following: what are the contextual variables that affect the program's functioning? These forces have been termed as "operative contextual forces," because they affected the daily functioning of the program, and they were generated from the program's micro-, exo-, and/or macro-systems. Following the creation of this term, those variables in Table 3 that appeared to support this definition were classified as "operative contextual forces." These variables were the following: "Physical Structure Provided," "Building Space Occupied," "Uncooperative School Personnel," and "Locked Doors." WW. When questioned about the challenges and benefits of having the after-school program housed in the middle schools, the staff of the program stated that the provision of a physical structure was a benefit to the program because "we have someplace to have Club." The promise by the school administration to house the after-school program was also mentioned in the program proposal: "The 68 sites for the Clubs will be each middle school gym and community room. Currently the middle schools lock their doors at 3:00 pm. each day, except for organized sports. The district has made the commitment to keeping the buildings open for the Clubs." Thus, a positive force that supported the daily functioning of the program was that the provision of a physical structure. WW. Despite the evidence that the program at least had a place to "be," the variable of "Building Space Occupied" demonstrated the challenge of negotiating space within that physical structure. Many times throughout the school year, the gym was promised as a space to hold Club activities; however, it was often the case that, when the staff arrived at the school, they would find the space occupied. During the focus group this situation was supported by one of the teen staff members when he responded to the question about the challenges and benefits of having the program within the school building, "...sometimes they have all the gym occupied and we can't do what we scheduled to do and that kind of brings a problem." The evidence that this was a negative force towards the daily functioning of the program was indirectly supported by a statement made by a regularly attending member of Club when she said "...I wish we could have a whole building to ourselves. We could have a recreation room, a game room, a place to make things, instead of just a school. We could have our own building to ourselves." WW. Not only did the occupation of building space present a challenge to program staff members, but there was also evidence that the interactions with the school personnel were negative. One teen staff member responded 69 to the question about the challenges and benefits of working with the school personnel by saying "Challenges, sometimes they act like they don't want to open the room where we have our equipment and it seem like they be saying, 'Why they (Club staff) here?--just get in the way.'..." This negative force was mediated by the evidence that the staff was able to obtain keys to some of the rooms. One teen staff supported this finding by saying, "I think we beat some of the challenges, 'cause we did get keys that work." Another teen staff member speculated about the reasons behind the uncooperative nature of the school personnel by saying "I think for them (the school personnel), I think they sometimes scared because they see a whole group of kids around and wonder, 'Oh, what if something happens, what if they all start getting into a group fight or what if they start vandalizing the school and everything like that.’ And you know they (school personnel) don't want to be bothered with all that, that's not what Club is all about and I think they should come in and see what Club is really about instead of being scared or scared of what they're (the students) going to do..." Thus, the uncooperative nature of the school personnel created challenges, or negative forces, towards the daily functioning of the program. This situation required that the staff to operate the program around these challenges. Lockedgates. In addition to the negative perception of the program held by the school personnel, the existence of one variable which was unique to only Dolan Middle School was the presence of expandable gates closing off certain corridors of the school. These gates were placed in the school to physically separate the fifth and 70 sixth grade wings from the seventh and eighth grade wings and to physically block access to the back part of the school that housed the gym, the home economics room, the art room, and the wood/metal shop room. Reasons given for the presence of these gates varied from preventing fights among the students to preventing arson. The afier- school program was housed in this back part of the building. One staff member described the gates as cutting down "...some of the chaos that might be going on in different parts of the building..." but not being beneficial to the program "...because sometimes, if they have the gate down (closed) and there's an emergency situation, you have to search for the person that has the key to open that gate..." Thus, the locked gates were primarily a negative force against the functioning of the program. [1 . B l . l I l E In addition to describing those contextual forces that affected the functioning and the existence of the program, the internal processes of the program were also examined. As stated earlier, the internal processes examined in this program were the intended and perceived components of the program and the systematic relationship among those components. Consequently, the findings in this section answered the following questions: what are the internal variables that describe the internal processes of the after-school program; and how are these internal variables maintained in the after-school program? Furthermore, in a manner corresponding to that used when answering the questions relating to the contextual processes, an effort was made to gain an integrated picture of the functioning of this program by weaving together the 71 perspectives of the participants, the staff, and the ideas of the program proposal. To answer the first question about the internal processes of the program, Table 3 was examined for the internal program variables that appeared to dgsgihe the internal functioning of the program. These descripjiye variables were the following: "Club Activities," "F un," "Learning," "Positive Peer Relations," "Physical Safety," "Psychological Safety," "Positive Role Models," "Prevention of Risk Behaviors," and "Promotion of Positive Behaviors." D . . II . l l Wes. As demonstrated above, it appeared that there was a lack of extra-curricular activities available, particularly, for the fifth and sixth grade students of the Beecher community. The after-school program appeared to fill this void by providing recreation, arts and crafts, games, and movies. Evidence that this void was filled was provided by answers to the question about what students would miss if Club were not offered. Eighteen of the participants stated that the activities of recreation, arts and crafts, games, and movies would be missed. Bun. Not only were program participants coming to Club to escape boredom after school, but they were also coming to Club because the program replaced this boredom with "Fun." Of the twenty-two program participants interviewed, eighteen of them stated that they participated in the program because they wanted to have fun. The existence of the variable of "Fun" was also supported by the information in the program proposal and by the data collected during the focus group. One adult staff 72 member described Club as a "Place where they can go and have fun." The variable of "Fun" seemed to be centered around the activities in which the students participated. Independent of whether the focus of an answer was on recreation, arts and crafts, games, or movies, the program participants stated that "Fun" was one reason why they liked Club activities. Learning. In addition to the variable of "F un," opportunities to learn were also provided by the program. Two program participants stated that they learned fi'om the Club activities, whereas five students stated that the purpose of Club was to have opportunities to learn. For example, a sixth grade boy stated that he learned how to make refereeing calls in basketball from one of the teen staff members while another student in eight grade stated that she participated in arts and crafts "Because it's a lot of things you can make out of different materials, and you can learn it and teach it to your sisters and brothers." Furthermore, it appeared that learning also occurred through the extended activities of the program, which involved fieldtrips. At the time of these interviews, three fieldtrips were taken by the program participants. One boy described a fieldtrip to a famous restaurant in Frankenmuth, MI as teaching program participants how to behave when going out to eat at a restaurant. The variable of "Learning" was not only related to the Club activities, but also to the variable of "Structure of the Program," defined above. Six of the adolescents interviewed stated that the "Structure of the Program" taught them such life skills as 73 manners, respect for property, and how to solve problems without fighting. For example, a girl in seventh grade responded to the question about whether or not Club "helps" students in some way by stating, "...most kids, they don't use their manners and, sometimes, well all the time here, you're required to use your manners. You have to say 'please' and you have to ask, 'cause most kids they go somewhere without asking or something...and sometimes kids that don't have discipline at home, they get more discipline here than what they do at home, 'cause they're required." This variable of "Learning" as it is applied to such life skills as "manners," was also supported by the evidence in the program proposal and by the responses in the focus group. The program proposal stated tha "...the program will provide an atmosphere of safety, fun, recreation, and learning." As defined by the program proposal, the variable to be learned was "life skills." These life skills were to be taught through a curriculum focused on "growth and development, sexuality education, development of positive peer relations, encouragement of abstinence, techniques to combat negative peer pressures, and life skills planning..." At the time of the interviews of the students, this curriculum was not implemented; but, as evidenced above, students were indeed learning "life skills." The presence of these opportunities to learn life skills was also supported by the program staff during the focus group, when one adult staff member stated that the students were learning such life skills as "...how to play with each other, behave, how to discipline, how to be responsible in other environments besides being at home, and how to respect other people's property, things like that. Skills they probably would use every day." 74 W. Besides learning life skills, the program proposal also stated that the program was "designed to encourage teens to compete without fighting and to develop positive relationships with their peers in an environment that promotes a sense of belonging." The evidence that Club was a place where positive peer relationships existed was found in the responses by three of the adolescents. They stated that the purpose of Club was to have the opportunity to be with friends or to meet new friends. For example, one girl in fifth grade responded to the question about why Club was offered by stating, "To really have fun and spend time with friends." This concept of friendship, or "Positive Peer Relations," was also supported when students were asked for their thoughts about whether or not Club should remain as an activity offered to both boys and girls. As an example, one boy in sixth grade stated that Club should not be limited to only boys or only girls, "...'Cause then we couldn't get to mingle. Talk to people." Other evidence that the relationships among the participants in the program were positive was found in the examination of the problems with other students. When queried about whether or not they had negative experiences (e. g. fights) with other participants in the program, the majority of the respondents said that they had not experienced any problems with other program participants. Thus, the absence of negative experiences may support the development of positive peer relationships. Mam. As evidenced by the durational contextual forces, the adolescents of Beecher, particularly the boys, appeared to be threatened by the "unsafeness" of their community. The six boys who stated that Club was safe did so 75 because it protected them from physical harm in the community. For example, one boy in seventh grade defined the "safeness" of Club for himself as "Being protected, not being in danger of anything or anybody." When the respondent was probed about what or whom he was being protected from the respondent said, "People in the streets, getting shot..." This respondent's comment about the "safeness" of Club provides two pieces of data. One piece is a description of the "unsafeness" of his community which lead me to identify the conceptual cluster of "Unsafe Community" discussed above. The second piece of data was the respondent's belief that Club was a place where protection from physical harm was provided. Of the six boys who spoke of "safety," all of their responses could be divided in this manner, that is, their responses described the "unsafeness" commtmity as well as the provision of "Physical Safety" by the program. The variable of "Physical Safety" being provided for the program participants was also evidenced in the program proposal and in the focus group. In fact, the program staff perceived the provision of a "safe haven" as the goal of the program. The staff believed that without the provision of safety, nothing else, such as the teaching and learning of life skills, could be accomplished. As one staff member put it, "If they (the program participants) have a safe place, I think all those other things (skills, positive peer relations, etc.) fall into place." WW. Not only was the provision of physical safety perceived as the goal of the program by the program staff, but psychological safety was also perceived as part of providing complete "safety" for the program participants. One adult staff member emphasized the importance of the variable of "Psychological 76 Safety" by stating that the participants of Club were "...not only safe just in 'physically safe,‘ but emotionally safe. All the teasing and all that, those things don't happen at Club." The presence of this variable of "Psychological Safety" is also supported by one of the participants when she defined the "safety" of Club to mean "...you can trust on the people (the staff), and you don't have to worry about anything, you can just be what you want to be." W. As outlined in the program proposal, a primary responsibility of the staff members was that they provide role modeling to the program participants. The presence of this function was also supported by the staff members in their responses during the focus group. The evidence offered by the program participants to support the presence of the variable of "Positive Role Models" was in the participants' reports of how they would act, and what they would do, when placed in the hypothetical situation of becoming a staff person for a day. Four of the program participants stated that they would behave just as the staff behaves. Six of the program participants described the manner in which they would behave, and when probed about whether that behavior was similar to that of the staff, they responded that it was indeed similar. As an example, one of these six participants stated, in response to this hypothetical situation, that he would "...act civilized..." and when he was probed as to what civilized meant he offered the following definition: "That means act a way, in a civilized manner, where you don't act the fool. You don't hit nobody, stuff like that. You don't cuss at nobody." When asked whether or not the staff act "civilized" most of 77 the time, some of the time, or none of the time, the respondent stated that the staff behaved this way most of the time. In addition, these descriptions of behavior offered supporting evidence that the staff of the program were viewed positively by the informants. The positive variables of the staff that were offered by program participants in this hypothetical situation were "helpful," "frm to be around," "willing to play," "watchful," "friendly," "calm," "protecting, nice," and "mature." Moreover, staff were viewed as being "not mean," and as not "yelling," "swearing," or "hitting." W The variables listed above as the answers to the question about internal variables of the program referred to events or states that were presently occurring in the program, and they were classified by the descriptive codes of purposes, staff, program participants, contextual assets, maintenances, limitations, and processes. The variables of "Prevention of Risk Behaviors and Promotion of Positive Behaviors" were solely described by the code of "goal." As stated above, a goal is defined to be terminal actions sought after through either approach or avoidance measures (Boyle, 1981). Using this definition as the operationalization of the concept of "goal," it was revealed that the program had goals that were either centered around preventing risk behaviors, such as engaging in early sexual activity, or around promotion of positive behavior, such as interacting positively with peers. The presence of these goals were evident in the program participants' responses to the question about how the program helped the middle school students. 78 An example of the presence of the variable of "Prevention of Risk Behaviors" was found in one of the program participant's statements, tha "...say for instance there was a fight and it was a real serious fight, and the person that was in Club Summit was out there (instead of being in Club) and she got into big trouble, but Club Summit prevent them from getting into trouble." This "Prevention of Risk Behaviors" was also evident in one seventh grade boy's response to how Club helps students when he stated, "...yeah, 'cause most, some people in Club are bad in school and they seem like they have something to do instead of going over a girl's house or going anywhere, just hanging out, smoking, doing drugs. It seem like it be better than anything for them to just be here." While nine program participants stated that Club helped students by preventing risk behaviors, three program participants stated that Club helped students by promoting positive behaviors. For example, one girl in sixth grade stated that Club helped students by "...trying to get them to work together and play together...," whereas one boy in fifth grade stated that Club helped students "To be an honors student, to get good grades." The presence of the variables of "Prevention of Risk Behaviors" and "Promotion of Positive Behaviors" were also supported in a statement of the program proposal that the after-school program was founded on the premise that to prevent risk behaviors, a program should "...focus on delaying early initiation of sexual activity, help teens do better in school, decrease truancy, help teens deal with negative peer pressure, create positive pressures, and improve parent and child communication and the involvement of parents in their children's activities." 79 l l . If . l l The variables of "Club Activities," "Fun," "Learning," "Positive Peer Relations," "Physical Safety," "Positive Role Models," "Prevention of Risk Behaviors," and "Promotion of Positive Behaviors" appeared to desgfim the internal processes of the after-school program. Thus, a sense of "wha " the program provided for the program participants was gained, but not a sense of "how" these variables were maintained in the after-school program. In order to answer this second question about the internal processes of the program, those variables in Table 3 that appeared to maintain the "descriptive" variables were identified. These variables were termed "maintenance" variables. As suggested in the conceptual framework of this study (see Chapter III), the individual and her context are in a constant state of change, and researchers, such as Lerner (1986), argue that to capture the development of the individual, longitudinal designs must be utilized in research. Longitudinal studies not only allow for changes across time to be discerned, but they also allow the researcher to discern causal relationships. Moreover, this is a descriptive, cross-sectional study; thus, it cannot generate data affording the causal ordering of variables. Accordingly, in order to remain within the constraints of this design, no statements of causality can made about the relationships among the descriptive and maintenance variables; however, the existence of these maintenance variables appeared to covary with the existence of the descriptive variables. These maintenance variables 80 are listed in Table 3 as: "Staff Visibility," "Helping Behavior," "Boundaries Maintained," "Communication of Expectations for Behavior," "Enforcement of Consequences," "Authoritative Nature of the Personnel,"and "Competence." W. As demonstrated above, the staff as well as the ideas within the program proposal supported the evidence that role modeling was an important function of the staff members. Indeed, the literature on adolescent development asserts that adolescence is a time of identity formation where young people "try on" various roles by observing and interacting with others (Lerner & Spanier, 1980; Steinberg, 1989). There are many reasons why young people chose particular people from whom to model themselves, but in this program, there was at least one identifiable variable that was associated with the variable of "Positive Role Models." This was the variable of "Staff Visibility." As demonstrated above, the staff of the program appeared to serve as positive role models to the program participants; however, there was reason to believe that when the program participants stated that they would behave just like the staff when given the opportunity to be a staff member for the day, they were referring to the behavior of the adult female staff members. This belief came from the response to the question about who are the staff of the program. Sixteen program participants either completely failed to identify any of the male adult staff members, or they identified only one of three male adult staff members. This was also true for seventeen of the program participants in terms of the identification of the teenage staff. This lack of identification came from the fact that there was a high turnover rate among the teenage 81 staff. When identification of teenage staff members took place, it was of two of the teenage staff who worked with the program for an extended time period. The other teenage staff members were not consistently a part of the program. Thus, the difference in visibility among the teenage staff was probably the reason why only two of them were consistently identified to be staff members by the program participants. In regard to identification of the adult male staff, the male staff occupied roles that also gave them less visibility to the program participants. Two of the women staff occupied the roles of assistant club directors. In these roles, the women were responsible for making announcements every day to the program participants about fieldtrips, activities, and/or the overall behavior of the program participants. The men did not address the program participants in these manners. Consequently, the variable of "Positive Role Modeling" did exist in this program; however, one variable that was related with this role modeling was the visibility of certain staff members. Indeed, with further investigation, the variable of "Staff Visibility" may maintain the existence of the variable of "Positive Role Modeling." Helpingjmhayigr. As demonstrated above, this program appeared to provide the opportunity for "Positive Peer Relations" to exist within the program. Having the opportunity to experience positive relations with peers is important to adolescent development because researchers find that the quality of peer relationships is predictive of later psychosocial adjustment (East, Hess, & Lerner, 1987). One variable that may be related to the existence of positive peer relations among program participants was the helping behavior demonstrated by program participants for their peers. The staff as 82 well as five program participants stated that fellow program participants assisted each other with certain tasks such as completing an arts and crafis project, putting games away, and/or completing homework. For example, one girl in fifth grade responded that "They (other program participants) help me learn new games and they help me with my school wor ." Another male respondent in sixth grade stated that fellow program participants helped him to talk about his feelings, whereas another female student in seventh grade believed that other program participants assisted her by providing team work in recreation. She states, "...like if we work as a team, like on a basketball team or if we playing dodgeball and get ready to hit with the ball or something...well, teamwor ." While this variable of "Helping Behavior" may be positively related to the positive peer relationships of the program participants, the presence of "Helping Behavior" appeared to be significantly supported at only one of the schools and only by girls. Five of the nine boys and eight of the ten informants at Dolan Middle School reported that there was an absence of helping behavior among their peers in the program. The gender difference may be due to socialization differences, whereas the difference between schools may be tied to negative perception of the students in one of the schools. As discussed in the "Procedure," fewer adolescents volunteered to be a part of the evaluation activities at Dolan Middle School. The reasons for this may lie in the fact that Dolan Middle School was being highlighted negatively by the media for students' behavioral problems. As stated earlier, because this study was cross-sectional study, knowledge about the perceptions the Dolan students of each other prior to the 83 attention of the media was not collected. However, the heightened awareness of behavioral problems at Dolan Middle School that was provided by the attention of media may have influenced the program participants' views of their peers. An example of this negative perception of Dolan students about their peers and of the media's potential influence on program participants' perceptions of their peers was supported by a boy in seventh grade who stated "we've been in the paper because our teachers ain't tolerating the students acting up..." when he was questioned as to whether or not it was a minority or majority of students causing problems at Dolan Middle School; his response was "Well, some of them, anyway you call it, it's some of them (students) that want to learn and most of them, I say, 40% want to learn and the other 60% don't. That ain't even equal." Thus, the presence of the variable of "Helping Behavior" potentially maintained the variable of "Positive Peer Relations" in this after- school program. WWW. As demonstrated above, the young adolescents of Beecher, particularly the boys, feel unsafe in their neighborhoods. The after-school program, on the other hand, was viewed as a safe haven. One variable that appeared to be related to the "safeness" of the program is the maintenance of the program boundaries by the staff members. During the focus group, one teen staff member believed that the staff was striving to provide a safe haven for the adolescents of Beecher by controlling who entered the program as well as at what time they entered the program. She says, "O.K., from what I've seen there's a certain time where you can... go to Club and then the doors close. I think that that also provides a safe haven, 84 that no other person can get through the doors past a certain time or that no other adults can get through the doors at this certain time..." The presence of the variable of "Boundaries Maintained" was also supported by the program participants when one girl in seventh grade stated "Yeah, it (Club) is safe because if a person, if we go outside and a person is not in Club, then they're not suppose to be around the Clubbers (program participant). Like if it is a relative or something, you guys (the staff) might say, 'Alright, go talk to your relative and see what they want and stuff.’ Usually if it's a relative, they let you go and stuff and talk to your relative, but if it's not a relative and it's just a friend, you have to wait to after Club." Thus, the variable of "Boundaries Maintained," appeared to maintain the safe environment of the program. ouuhr,.oro no; ,0. o of... .ro’ro 'u‘r o 0"! ‘r,’ These two variables were on a continuum which, when first coding the data, were called the "Discipline Process." It appeared that the variables of "Communication of Expectations for Behavior" and the "Enforcement of the Consequences" were associated with "Positive Peer Relations," with "Learning," and with "Physical" and "Psychological Safety." Those adolescents at risk for maladjustment are characterized by the literature to be aggressive and highly disruptive, and are rejected by their peers (East, Hess, & Lerner, 1987). This program appeared to control the occurrence of aggression and disruption through communicating very specific expectations for behavior and enforcing consequences for negative behavior. Moreover, as stated above, some of the program participants believed that they learned from the discipline process because as one girl 85 in seventh grade said, "It teaches us right from wrong." The variables of "Physical" and "Psychological Safety " were also related to these two variables because, as one staff member stated, "I think in the beginning (start of the program), before we could establish ourselves, we had a lot of play fighting. I do believe that since we have grounded that rule (no play fighting) and try to adhere to that,..., I haven't seen a lot of play fighting in a while." Play fighting is "mock" fighting, wrestling, laying hands on someone in a playful, yet aggressive manner, or threatening to lay hands on someone. The expectations for behavior were defined by both the staff and the program participants as the "Club Rules." Appendices DJ. and D2. list the rules as defined by both the staff and the program participants. When an adolescent would join the program, he or she was given a parent permission form and a copy of the "Club Rules" as defined by the program staff. The rule that was identified most frequently by program participants and supported by the staff focus group and the program proposal was the rule regarding the prohibition of fighting and play fighting. When five program participants were probed about who communicated the "Club Rules," all of them responded that it was the program staff who communicated the rules. Although the consequences for negative behavior were not distributed in written form to the program participants as were the "Club Rules," the evidence supporting the variable of "Enforcement of Consequences," demonstrated that there were uniform actions resulting from the "breaking" of one of the "Club Rules." For example, when answering the question about what a program participant does when someone else is "messing" with him or her, twenty-one of the program participants 86 stated that they went to a staff member. When these students were probed as to what the staff member does to control the situation, the most frequent answer was that the student causing the negative situation was put "on the wall." To be put "on the wall," meant that students were to discontinue their participation in an activity, and they were to stand against the wall of the room for a period of 15-20 minutes. During this time they do not talk to anyone. Other identified consequences for negative behavior were being suspended from Club for a certain time period, being given a warning from one of the Club staff members, and having parents called. W. In addition to being associated with the variables of "Learning," Positive Peer Relations," and "Physical and Psychological Safety" there was also evidence to support a relationship among the "Expectations for Behavior," "Enforcement of Consequences" and the discipline style of the staff. The style of discipline in this program seems to be best defined as "authoritative." Based on the work of Diana Baumrind (1978), authoritative parents firmly but warmly set expectations for their children's behaviors that allow children simultaneously to have self-direction and to take ultimate responsibility for their actions. The evidence to support the presence of the "Authoritative Nature of the Program Personnel" was found in the data from the focus group and semi-structured interviews. As an example of this authoritative discipline style, one adult staff member described the expectations for behavior of the program participants by stating "(We) expect them to get control of themselves, they know what the rules are, so, we place that responsibility upon them and they break a rule, it's their responsibility to adhere to 87 whatever the consequences are." The evidence that the program participants offered to support the presence of this variable was visible in the brief questionnaire given to respondents about the characteristics of the staff. By examining the two questions about the "strictness" and "leniency" of the staff as being on a continuum, almost all of the informants believed that the staff were neither too strict nor too lenient. The authoritative nature of the personnel may maintain the expectations of behaviors and the enforcement of consequences for negative behavior. One manner by which the authoritative nature of the personnel may have been fostered among the staff of the program was through the training received by the staff. Staff members attended two workshops, both of which focused on being effective team members. Furthermore, the staff of the program attended weekly staff meetings where adolescent, discipline, and staffing issues were discussed. As one teenage staff member stated, the training that she received taught her about "Being on time, responsibility, working with kids, accepting our mistakes and correcting them. Working with other people." Competence. Another variable that appeared to be related to the variables of "Fun," "Learning," and "Positive Peer Relations" was the variable of "Competence." When asked about the activities that they liked the most and why, the program participants responded with the following reasons: the activities provided opportunities to have fun; they provided opportrmities to be with friends or to meet new people; they provided opportunities to learn; and they provided opportunities to exhibit competence. In contrast, when program participants gave reasons for disliking activities, the most 88 frequent reason was incompetence at the activity. For example, one boy in sixth grade stated that he liked basketball "...'cause I can shoot good," whereas, a girl in sixth grade did not like football "...because I can't throw no football or catch a football." Thus, competence in activities may maintain the program participants' perceptions of the Club activities. WM As demonstrated above, by examining the internal and the contextual processes of this program as described by the perceptions of the program staff and the program participants and by the ideas in the program's fimding proposal, the functioning of the program was depicted. The contextual forces were divided into the durational forces that affected the existence of the program and the operative forces that affected the functioning of the program. The presence of the variables of "Other Activities," "Boredom," "Perception of the Program," "Responsibility at Home," "Unsafe Community," "Unsupervised Home," "Boredom," "Knowledge of the Program," "Structure of the Program, " "Parent Support," and "Transportation" were all believed to affect the program's existence in positive and negative manners. Whereas, the presence of the variables of "Physical Structure Provided," "Building Space Occupied," "Uncooperative School Personnel," and "Locked Doors" were believed to affect the functioning of the program in positive and negative manners. In addition to describing the contextual forces that affected the functioning and the existence of the program, the examination of the internal processes revealed that 89 certain variables described the internal processes of the program. These variables were the following: "Club Activities," "Fun," "Learning," "Positive Peer Relations," "Physical Safety," "Psychological Safety," "Positive Role Models," "Prevention of Risk Behaviors," and "Promotion of Positive Behaviors." This evaluation was cross-sectional in nature and, because of the limitations of cross-sectional designs, no causal statements about the relationship of variables can be made. However, there did appear to be certain variables that maintained the descriptive variables. These maintenance variables were: "Staff Visibility;" "Helping Behavior;" "Boundaries Maintained;" "Communication of Expectations for Behavior;" "Enforcement of Consequences;" "Authoritative Nature of the Personnel;" and "Competence." These variables were presented here as some of the potential variables that maintain the existence of those variables that dfi'me the internal program processes. Further investigation with a longitudinal design may provide additional information about these "maintenance" variables as well as information defining the presence of other key variables of the internal and the contextual processes of the program. Table 3 cross-tabulates the variables described above with the program components of goals, purposes, program personnel, program participants, and program processes (as well as contextual assets, maintenance, and limitations). In order to summarize the relationships among the variables, Figure 4a and Figure 4b graphically display these relationships. These figures conceptually relate to the figures presented in Chapter III, in that the variables are positioned according to the systems in which they 90 reside. (The micro-system is the school buildings, and the exo-system is the community, participants' homes, and the school administration). The arrows in both Figures 4a and 4b are meant to represent associations within the meso-system, and not, necessarily, causation. Furthermore, these figures only demonstrate those relationships evidenced in the data and not all possible relationships that could be identified with further investigation. Finally, poverty and segregation in the Beecher community are two concepts that were discussed in the "Context of the Study" in Chapter III. Although the data in this study do not specifically define the role of poverty and of the historic "white flight" in the functioning of this program, the very real impact of these two social ills on the community of Beecher cannot be ignored: because It is believed by the evaluator that to learn about and to understand the Beecher Community is learn about and to attempt to understand poverty and segregation. 91 szzowmmnu 400:0” EEWQOOOZD A: u r n 33.... 9::qu , All 3:0“. gamed Ecuafifmd E885 ”83%; 8:28:32 H0033.3) 8:38:32 ecomom Figure 4a 92 3328 9.3.2.. . . _ o 1 v I I I 00:28:80.1? 1:7 I I ’ ’ 35303.9”. 35:25;— I 3328 928: 532:2: . 332.8 v.3...— :o::o>2n. 0 32am , _8_uo_8>a%8_§a . I 20.5.3“. amaulv .69... 9530.".- 9:63.. o I i can. . 280: so: 2:38 . v A l 025530 .2529: 23.2 258553. g x u x . . . K + K «3:263:00 3 22022550323 I ‘ . .2 28.8on . v1 us 538.5888. ¢ \ \ \ I / £335 cam. _o::om._on_ lflaaaflll 8:29:35. Figure 4b 93 2.1“. E 2 IE I"' As Miles and Huberman (1994) state, qualitative research is a process of discovering, verifying, and re-discovering. If this study was longitudinal in nature, the variables listed above, which describe the functioning of this program, would have been taken back to the program participants, the staff, and the program designers for their verification and their insight into other variables that might have been missed. This process would have allowed all the program's assets and limitations to be described. Resources were limited in this study, thus this verification process with the program designers, staff, and participants was not engaged. However, as outlined in Chapter II, this program's functioning can be compared to the qualities that researchers, practitioners, and adolescents believe describe successful youth programs. These qualities are the following: * Intensive individualized attention from responsible adults who respect and listen to young p60ple; * Engagement of peers in interventions; * A safe, protective enviromnent where adolescents can be themselves; * Clear rules and high expectations for behavior; '1} Interesting activities that provide opportunities for young people to socialize, learn, and practice new skills, make new fi'iends, have new experiences, and explore new options; 94 * Creation of opportunities where young people can think about and plan for the future; * Attention paid to cultural and life-style diversity, as well as to individual diversity; * Involvement of parents; *- Effective organizational climate that is flexible and supportive of program's functioning; and A * Arrangements for training of staff. Thus, it is possible to discuss how the described functioning of this after-school program compares and/or contrasts with the ten characteristics of successful youth programs. One of the qualities that young people and researchers believe is vital in the effective youth programs is adult attention that is respectful, individualized, and intensive (Carnegie Corporation, 1992; Dryfoos, 1990; Schorr 1988). The after-school program in the Beecher community appears to have this quality, not only because the staff are viewed positively and as role models to the program participants, but also because the staff exhibit an authoritative manner with the program participants. The data do indicate, however, that the adult male staff and the teenage staff have a lower visibility to the program participants than the adult female staff. This difference in visibility could affect the relationship between the staff members and the program participants. Thus, one asset to this program is that the intensive, individualized, and respectful attention from staff does exist, but the "unbalanced" visibility among the staff may present a limitation to the program if certain program participants identify 95 more closely with adult males and/or teenagers. The evidence that teenagers have a low visibility to the adolescents participating in the program is not only limiting to the program in the sense that the development of an intensive, individualized, and respectful relationship between the staff and the participants may be hindered, but also it may be limiting in the sense that the engagement of peers in interventions is also viewed as an important quality of successful youth programs. Without teenagers being visible to the program participants as staff members, the teenage staff is not being utilized effectively in the program interventions. For example, one adult staff member stated that seventh and eighth grade teenagers did not attend the program because they rejected the "structure" of the program. The teenage staff stated in the focus group that they believed they were role models to the program participants because they were close in age and they, too, had to follow rules. However, because of their low visibility, the message that their slightly older and admired peers have to follow "rules" is probably not being communicated to the seventh and eighth grade adolescents. Thus, an asset to this program is that the older peers of the program participants are available as a resource for utilization in program interventions, but a limitation of the program is that their low visibility prevents them from being a maximally effective resource. A third characteristic of successful youth program is the provision of a safe and protective environment where adolescents feel comfortable expressing themselves (Carnegie Council on Adolescent Development, 1992; Dryfoos, 1990; Schorr, 1988). 96 The variables that relate to the existence of this quality in the after-school program are described as "Physical Safety," "Psychological Safety," and "Boundaries Maintained." The boys in the program felt threatened by dangers in their community; however, they believed that the after-school program protected them from these dangers. This protection from dangers and threats in the community seemed to be related to the data describing the staff‘s maintenance of the program's boundaries. In addition to the physical protection from dangers, the program also seemed to provide a space for young people to feel comfortable about expressing themselves. As one young girl stated, "...you can trust on the people (the staff), and you don't have to worry about anything, you can just be what you want to be." Thus, an asset to this program is that it provides physical safety through the maintenance of program boundaries and psychological safety for the program participants. Not only does the maintenance of boundaries potentially create a safe environment for the program participants but, as suggested above, the communication of rules and the high expectations for behavior may also create the safe environment of the program. As demonstrated above, this after-school program has a set of rules as well as consequences that appear to be consistently and clearly communicated to the program participants by the program staff. Having clear rules as well as expectations for behavior is considered to be an asset of the program because researchers and adolescents believe that having these rules and expectations are key to "successful" youth programs. Protection from an unsafe environment and being provided with "structure" may 97 be qualities about a program that attract young adolescents. However, the quality that is more likely the most important feature of a "successful" youth program in the eyes of a young adolescent is whether or not there is a provision of interesting activities that allow young people to socialize, learn, practice new skills, make new friends, have new experiences, and explore new options. This after-school program does seem to possess such activities because, as demonstrated above, the activities were a source of fun, learning, and positive peer relationships for the program participants. Furthermore, the data in this study indicated that there is a lack of youth programs for young adolescents in the fifth and sixth grade of the Beecher community. Consequently, another asset of this program is that it provides extra-curricular activities that are desired by the younger adolescents of Beecher; however, one limitation of this program is that the seventh and eighth grade students do not attend because of other activities that are provided for them and/or because of the negative perception among older adolescents that the activities are "childish" or "chatty." Another limitation of the program in terms of the activities was the potential lack of self-competence among the program participants. As stated above, the most frequent reason for disliking an activity was a lack of competence at the activity. Another key element to effective youth programs is the provision of opportunities for young people to think about and plan for the future. As discussed above, this program provided fieldtrips for the program participants. These fieldtrips were identified as a source of learning by one of the interviewed program participants. In general, the majority of the program participants interviewed did not identify 98 fieldtrips as a component of the program, probably because the program participants had not had a fieldtrip recently when they were interviewed. However, the evaluator participated in one of the fieldtrips that was designed to provide the young Beecher adolescents with the opportunity to think about the future. This fieldtrip was taken to Michigan State University during the middle of the school year. For an entire day, program participants toured various facilities on the campus, such as the planetarium, classroom buildings, and the auditorium/theater; in addition, they ate in one of the dining halls and participated in a discussion facilitated by two Michigan State University students who were Beecher alumni. The program designer stated that the MSU experience was a "fantastic opportunity" for the Beecher middle school students because so many of them do not see attending college as a reality or even a possibility (Robinson, personal communication, 1994). Thus, the provision of this opportunity could be considered an asset of the program; however, a limitation may be that this opportunity was not a frequently offered feature of the program. The fieldtrip to MSU was provided for the program participants of the Beecher community because the after-school program designers were sensitive to the fact that very few of Beecher's high school graduates receive post-secondary education. Another manner in which the program designers were sensitive to what was perceived to be a need of the program participants was in the selection of the staff for the program. Attention to individual, cultural, and life-style diversity is viewed as a component of "successful" youth programs (Carnegie Corporation on Adolescent Development, 1992; Schorr, 1988). According to the program designers, the staff of the program, excluding 99 myself, were selected because they were members of the young adolescents' community and because they also reflected the race of the students. As stated earlier, most of the members of the Beecher community are African-American. Attention to staffing issues is one manner by which cultural diversity was addressed this program; however, addressing "cultural diversity" through curricula and/or other programming means was not done in the program. The program staff stated that they believed that addressing cultural diversity was not a purpose of the program, and one staff person suggested that the provision of a safe environment and the provision of activities were more important than the provision of cultural diversity. For example, she stated, "I think in the cultural sense and I look at it in different ways as far as an ethnic perspective, we treat kids as kids. We don't come with afro- centn'sism or anything like that...from an environmental culture, I think that we're meeting their needs because we are back to square one, giving them all those things, a safe haven, things to do..." Thus, a potential limitation of this program is that it does not address the issue of cultural diversity through curricula or other programming means, but a potential asset of this program is that consideration to the fact that almost all of the students in the Beecher community are African-American was made when the program was staffed. The last three qualities of "successful" youth programs describe variables that are external to the program but support the program in some manner. As described above, the variable of parent support was defined to have three levels. The first level was the signing of the parent consent form for their child to attend the program. The 100 second level was allowing their child to attend the program on a regular basis. The third level was parent participation in the activities. As stated above, the first two levels were positive durational contextual forces on the existence of the program when applied to program participants, whereas they were negative durational contextual forces on the existence of the program when applied to non-participants. Moreover, the third level of parent support negatively affected the program because only two parents of the twenty-two informants participated in the program activities, and the staff described parent involvement as difficult to obtain. Thus, the involvement of parents in youth programs is viewed as a characteristic of "successful" youth programs; however, this program only has two levels of parent support. Parents allowing their children to attend this program is an asset to the program because without participants, the program would not exist. However, the lack of parent involvement on the third level could be considered a limitation of the program because parents' involvement in their adolescents' lives is believed to be one means of preventing risk behaviors in adolescents (Dryfoos, 1990). Not only is parent involvement considered to be an important external support to the program, but an effective organizational climate that is flexible and supportive of the program's functioning is also viewed as an important quality of "successful" youth programs. As demonstrated above in the variables of "Locked Doors," "Uncooperative School Personnel," and "Occupied Building Space," the organizational climate of the after-school program, which is defined here as the school and school personnel, appears to be hostile towards the program's daily functioning. This is certainly a 101 limitation to the program because it impedes the smooth operation of the program. The final component of "successful" youth programs as identified by Dryfoos (1990), Schorr, (1988), and the adolescents interviewed by the Carnegie Corporation on Adolescent Development is arrangements for training of program staff. As discussed above, the staff received training through two workshops and through weekly staff meetings that centered around discipline issues as well as around adolescent and organizational issues. The result of this training, as identified by the staff members, is "team work." Thus, arrangements for training of the staff were made and there seems to be a potential positive impact from this training. 5 EE 9 lI"' As stated earlier, resources were not available for returning to the community with the description of this after-school program's functioning so that community members could determine the program's assets and limitations as well as further identify key program components and processes. Consequently, ten characteristics of "successfiil" youth programs were chosen to be the "yardstick" for this after-school program's assets and limitations. As described above, these assets are: * The staff provided intensive individualized attention. * Older peers of the program participants were available as a resource for utilization in program interventions. * The program provided both physical and psychological safety. "' The program provided clear rules and high expectations for behavior. 102 * The program provided activities that are desired and well-liked by the younger adolescents of the Beecher community. * The program provided at least one opportunity for the young adolescents of the Beecher community to think about and plan for their futures. * The staff of the program reflected the diversity of the community and were residents of the community. * Parent support, as defined by consent to become a member of Club and permission of frequent attendance, did exist. * Training arrangements were made for the staff of the program. In turn, the limitations of the program are: * Lower visibility of adult male staff and teenage staff to the program participants may limit the power and/or the effects of the program's interventions. * Seventh and eighth grade students were a minority in the program; thus, the program has limited power to prevent and/or intervene in the behaviors and lives of those seventh and eighth grade students who are "at risk." * Students who experience a lack of competence in certain activities disliked those activities, and this could limit the program's power to maintain those students' involvement and/or limit the program's power to attract the membership of additional students. * The one opportunity provided for the young adolescents to think and plan for their future was limited to only one day; thus, the program's power to affect the course of the program participants' futures may be limited. 103 * Cultural diversity as a "curriculum" focus was not addressed; thus, the program's power to prepare the African-American youth of Beecher for survival in the mostly White world may either be non-existent or very limited. * Parents were not involved on a regular basis with the program; thus, limiting the program's power in affecting the parent-adolescent relationship. * The organizational climate of the programs, which was defined to be the school and the school personnel, was resistant to the smooth operation of the program. In sum, the description of the program's functioning was not taken back to the individuals involved with the program for their verification and their clarification. However, the comparison of the functioning of this after-school program to the ten characteristics of "successful" youth programs seems to have provided a manner by which to assess the program's assets and limitations. Furthermore, while the specific results of this study cannot be generalized to other after-school programs, the comparison of the program's functioning to the qualities of "successful" youth programs offers additional qualitative support to the perspectives of youth and researchers that these ten characteristics are, indeed, important qualities of "successful" youth programs. CHAPTER VII DISCUSSION The purpose of this study was to conduct a qualitative evaluation of the processes comprising an after-school program for young adolescents in Grades 5-8. This afier-school program was implemented in Dolan and Summit Middle Schools of the Beecher Community School District during the 1993-1994 school year. Approximately 200 students participated in this program. The outcome of this evaluation was a description of the program's functioning and of its assets and limitations. Based on these results, the following is a discussion of theoretical implications, reliability and validity issues, study limitations, programming recommendations, and perspectives on the research process and implications for fiiture study of the afier-school program in the Beecher Community School District. I] 'lIl" The assets and limitations of this program were constructed from the comparison of the program description to the ten characteristics of "successful" youth programs. This program description was outlined earlier (see Chapter VI). As noted in Chapter 11, programs and the contexts in which they are embedded are complex and multifaceted (Lerner & Miller, 1993). To understand the effectiveness and functioning 104 105 of a program, the complexity and multifaceted nature of programs and their contexts in which they are embedded must be addressed through the evaluation design. The evaluation design of this study was constructed from the human development theories of Lerner (1987, 1989a, 1989b, 1990, 1993b) and Bronfenbrenner (1979, 1986) because these theories give support to the arguments that the individual is nested (Bronfenbrenner, 1979) or embedded (Lerner, 1987, 1989a, 1989b, 1990, 1993b) within the context, and the interaction between the individual and the context is dynamic (Lerner, i.e., 1987, 1989a), or reciprocal (Bronfenbrenner, l979)--meaning that any changes in the individual influences and are influenced by the changes in the context. By viewing the individual as embedded in, and dynamically interacting with, the context Lerner defines the individual as a change agent; thus, the program participants and the staff members of this after-school program were viewed as a primary source of information about the internal and contextual processes of this program. By weaving together the perspectives of the staff, program designers, and program participants, a description of the program's complex functioning was defined in Chapter VI. By also viewing the individual as nested and reciprocally interacting with the environment, Bronfenbrenner defines the contexts of human beings into different systems, thus providing a method by which to identify the interactions of the between the program and its context. By examining the program's micro-, meso-, and exo- systems, information about the contextual forces of the program were defined in 106 Chapter VI. As demonstrated above, the durational contextual forces of this program represented the meso-system; these forces were generated from an event or state in the micro- and exo-systems, and positively or negatively affected the existence of the program. In turn, the operative contextual forces represented the meso-system; these forces were generated from an event or state in the micro- and exo-systems, and positively or negatively affected the functioning of the program. The macrosystem, although not specifically explored, was described in Chapter III in the section, "The Context of the Study," and is represented on Figure 4a. As noted earlier, two important forces in the macrosystem are poverty and the historic "white fligh " of the Beecher ‘ community. Recognition of the impact of these two social ills on the community of Beecher is important understanding why the youth of Beecher are "at-risk." Thus, the work of Lerner and Bronfenbrenner provided the theoretical framework in this study. This theoretical framework directed data collection as well as the data analyses of this study. An evaluation researcher, Chen (1990), asserts that evaluation literature has not been concerned with incorporating theory into program evaluation. The result of this atheoretical approach of evaluation, Chen (1990) argues, are evaluations that describe a program's inputs and outputs, but not the processes that occur between the inputs and outputs. In contrast, by using the theories of Lerner and Bronfenbrenner, the processes of this program have been described; consequently, the utility of Lerner's and Bronfenbrenner's work as applied to program evaluation is supported by this study. 107 B 1.1.]. Hill While the theoretical fiamework of this study directed the generation and explanation of the data, it is the case that, as with any study, the issues of reliability and validity play an important role in the accuracy and the generalizeability of the results. The issues of the reliability and validity of the results in qualitative research are highly debated (Kirk & Miller, 1986; Schatzmann & Strauss, 1973; Silverman, 1993). Researchers who view qualitative research and quantitative research as diametrically opposed argue that these two concepts belong only to research that is conducted from the positivisitic paradigm and have no place in qualitative research (Silverman, 1993). However, researchers like Silverman (1993) and Miles and Huberman (1994) argue that if qualitative research is to be considered scientific, these concepts must be addressed. To address the issue of reliability in qualitative research, Silverman (1993) and Miles and Huberman (1994) assert that more than one individual should code the data so that consistency among generated categories, patterns, and inferences or inter-rater reliability can be established. To address the issue of validity, Miles and Huberman (1994) argue that triangulation be completed by data source (people, times, places, etc.), by method (observation, interview, etc.), by researcher (investigator A, B, etc.), by theory, and by data type (qualitative text, quantitative). Both triangulation and the establishment of inter-rater reliability are methods that require more than one researcher and resources for the development and utilization 108 additional methods. As stated earlier, this study had only one evaluator; however, this does not mean that the issues of reliability and validity have been ignored. To Miles and Huberman (1994), the issues of reliability and validity in qualitative researcher rest upon the skills of the investigator. In qualitative research, the investigator not only utilizes instruments, but he/she is an instrument of the study (Jorgensen, 1989; Marshall & Rossman, 1989; Miles & Huberman, 1994). To be a valid and reliable instrument of the study, Miles and Huberman (1994) assert that a qualitative researcher must be familiar with the phenomena under investigation, must have strong conceptual interests, must have a multi-disciplinary approach rather than being grounded in a single discipline, and must have good investigative skills that allow him or her to draw informants out and to avoid premature closure. The evaluator attempted to become a reliable and valid instrument of the study in each of the these manners. First, she became an adult "staff" member of the program so that she was attending the program on a regular basis. She did not occupy any of the leadership roles of the program, nor did she take responsibility for any one of the activities. Thus, her role was that of a "floater" among the different activities. This flexible role allowed her to familiarize herself with the context, processes, staff, and participants of the after-school program before formal study was undertaken. Second, the strong conceptual interest in the program of the evaluator was not simply a function of the role of "evaluator;" it also was based on her experiences as a staff member of the program prior to the formal evaluation period. For example, the evaluator described in her personal journal that during one day 109 in November 1993, the Club staff arrived at one of the schools to find that all of the school space usually reserved for the after-school program was occupied. It was the recording of experiences such as this one that prompted her to consider assessing the contextual influences on the operation of the program. Third, the evaluator attempted to assess this program with a multidisciplinary perspective that utilized Lerner's and Bronfenbrenner's multivariate, multilevel theories of human development. Finally, the attention to improving her skills as an investigator was accomplished through her participation in the advanced qualitative methods course offered by the Family and Child Ecology Department; through other independent readings on interviewing styles; and through careful attention paid to the informant responses that led to the mid-course adjustment of the semi-structured interview form. I"' [151 Despite the evaluator's attempts to become a reliable and valid instrument of this study, the reliability and validity of the results of this study cannot be unequivocally established. There was only one researcher on this study; thus, inter-rater reliability was not established. Furthermore, this was a cross-sectional study. The theoretical framework of this study is based upon the arguments of Lerner (i.e., 1987) and Bronfenbrenner (1979, 1986). Both of these theorists assert that individuals and their context change over time. Consequently, longitudinal research designs allow the researcher to assess change over time, whereas cross-sectional designs do not allow the researcher to assess change. A longitudinal study conducted by more than one 110 researcher would allow for reliability and validity to be established through the methods suggested by Silverman (1993) and by Miles and Huberman (1994). A longitudinal study would also permit a researcher to explore causal relationships. As suggested in the "Results," several variables appeared to maintain the variables that described the internal processes. For example, the variable of "Competence" appeared to maintain the program participants' positive perceptions of Club activities because those activities that were well-liked were those through which the program participants expressed competence. Whereas, when program participants expressed dissatisfaction with an activity, the majority of them stated that their reason was lack of skill or competence in the activity. Thus, competence in an activity appears to be associated with the perception of that activity. This relationship between the students' competence and their perception of the program is associational, at best. That is, the cross-sectional design of this study does not afford causal explanation. Consequently, statements such as "competence in an activity causes an adolescent to like that activity" cannot be made. Thus, though the findings in this study must be interpreted with caution, they can provide valuable information to the designers of the program as well as point directions for the construction of hypotheses and future research in this after-school program. Finally as stated in Chapter III, in the section, "Race, Class, and Values of the Program Evaluator," all of the data in this study was filtered through the evaluator who was European-American and raised in the middle class. In order to understand how she filtered information through her own experience and background, the evaluator kept a 111 personal journal of her own thoughts and experiences in the program and in the Beecher community prior to the evaluation process. Though it is impossible to separate the individual fi'om his or her value system, it is possible to recognize and to question how this value system may or may not influence the interpretation of the data. Thus, the interpretation of the data in this study was offered not as the "true" and "absolute" perspectives of the program participants and staff, but it was offered as a carefully examined and worded interpretation of what the staff and the participants had to say about the program. E . B l . The feedback to the program designers and implementers was discussed above in the form of the program's assets and limitations. In the upcoming 1994-1995 school year, the program designers and the program staff are ultimately responsible for transforming this knowledge about the program's assets and limitations into actions. The following recommendations were constructed fi'om the program's limitations and offered fiom the evaluator's perspective as both a researcher and a program staff member: 1. The male adult staff and the teenage high school members were not recognized as staff members of the program by the program participants while the female adult personnel members were recognized as staff members. The data revealed that the reason for this lack of recognition was that the women occupied roles of leadership in the program while the men and the high school teenagers did not occupy 112 such roles. This "unbalanced" visibility of the staff means that the male staff and the teenage high school staff were not being used in the most effective way in terms of being positive role models since youths identify not only with women but also with men and other individuals close to their age. One method by which to balance the visibility of the staff may be to have the teenage staff organize and implement the program at least once a month and have the adult leadership roles within the program occupied by both men and women. 2. There were fewer older students than younger students who attended the program because of such reasons as other activities, negative perceptions of the program, responsibilities at home, and etc. In order to reach the older adolescents of the middle schools, the program designer and staff should explore seventh and eighth graders perspectives of "what" an afier-school program should contain for them and "when" this program should be offered to them. Methods by which to accomplish this exploration may be through focus groups with selected students and/or a brief survey administered to a large group of students. 3. The "competence" or how a program participant thought about his or her ability to perform an activity in the after-school program appeared to influence whether or not that student had fun, learned, and experienced positive peer relationships in the program. This competence or feelings about oneself relates to the self-concept of the individual. In adolescence, self-concept develops just as the body, mind, and emotions of the adolescent develop. In order to facilitate the developing self-concepts of the program participants, program designer and staff should explore the development of a 113 staff protocol for encouraging the development and/or maintenance of positive self- concept among the program participants. Methods by which to accomplish this may be through additional staff training sessions, centered around the topic of adolescent self- concept and through staff work sessions involving scenarios and/or role plays. These role plays could demonstrate practical methods for encouraging the development of positive self-concept among program participants. 4. As discussed in the "Context of the Study," in Chapter III, very few of the Beecher community teenagers go on for post-secondary education. The MSU trip was one experience that may have encouraged some of the young adolescents of Beecher to consider that college could become a reality for them. In order to facilitate Beecher's young people in their plans for their futures, the program designer and staff should explore the development of a program component focused on career development. Methods by which to accomplish this may be through the staff making arrangements for community professionals to visit the program and/or through program designers exploring avenues to involve such organizations as Junior Achievement in the Beecher community. 5. As stated in the "Context of the Study," the adolescents of Beecher live in a community, like many urban communities, that has experienced "white flight" as well as extreme economic hardship. In this program, cultural diversity was not a purpose of the program as defined by the staff. However, the staff of the program may want to examine what the values of cultural diversity are and why these values might be 114 important to the adolescents of Beecher. Methods to accomplish this understanding may be through staff training sessions focused on the definitions of cultural diversity; and through the use of cultural diversity in interventions as well as focus groups conducted with the adolescents of Beecher around the issues of what cultural diversity means to them and how could having an understanding of cultural diversity be helpful to them. 6. The residents of the Beecher community could be classified as members of the working class. Many of them work the second shift at one of the General Motors factories. Parents' involvement in their adolescents' lives is deemed as an important quality in the promotion of positive youth development, and, indeed, the staff of this program also thought it was important. However, the staff stated that they had great difficulty in getting parents to volunteer their time to the program. In this study, at least 6 of respondents stated that their parents would want to be involved in the program, but had to work during the time at which the program was offered. One manner by which to foster parent involvement other than through volunteering time is through the formation of a parent advisory group that met during a time convenient for the parents. This group could be informed of the daily functioning of the program as well as could make programming recommendations to improve the program and to increase other parents' knowledge about the existence, purpose, and functioning of the program. 7. The data indicated that the school personnel did not understand the program and had negative perceptions about the program. In order to reduce the hostility to the 115 program and ignorance of the school personnel about the functioning of the program, the program designer and the program staff should explore conducting in-service meetings where the school personnel could learn about and express their views about the program. on ..u ' " u; ' ' u .u‘ 1‘ 3., n' 0' .u no .__n O r ' i- an As stated above, there is utility in applying the human development theories of Lerner (i.e. 1987) and Bronfenbrenner (1979, 1986) to program evaluation. In this study, a method by which to conduct "process" evaluations was demonstrated. That is, by viewing the multiple perspectives of the individuals involved in this program as legitimate and valuable sources of information about the program and by viewing the context of the program as a powerful shaping force on the program's existence and functioning, information about "how" this program functioned was provided. In this study, there was also utility in being a participant of the evaluation process. That is, by becoming a staff member of the program, the evaluator was not only able to develop rapport and trust with the program participants and the program staff, but she was also able to gain a certain level of understanding of the program functioning that an "outside" evaluator would not have possessed. For example, the evaluator was familiar with the "language" of the program; and when the time came to evaluate the program, she did not need program participants to define the discipline term "on the wall;" instead, she could focus her energies on answering the evaluation questions. Not all process evaluations need to be completed in a participatory manner; 116 however, when the culture and organization is foreign to the evaluator, participation greatly aides the research process. Finally, this process evaluation answered the very important question of "how" this program functions. Future research should now focus on "wha " has been, and continues to be, accomplished in this after-school program. That is, the question that needs to be answered is: "What is the change in risk behaviors or adolescent development which occurs as a result of participation in this program?" As stated in Chapter I, programs in the non-school hours are believed to have a tremendous potential to impact the behaviors and well-being of adolescents. Given that this program possesses many of the characteristics of "successful" youth programs, it is believed that, as it develops, the program could positively impact upon the lives of the young adolescents of the Beecher community. Thus, measurement of the impact of this program must now take place. APPENDIX A.l: PARENT CONSENT FORM 117 Dear Parent, Your child has been participating in Club Dolan or Club Summit throughout the school-year. The purpose of Club Dolan and Club Summit is to provide young teens with a safe place to ' have fun and learn about positive ways of behaving. In order to know whether or not we, the staff, at B-103 and the Clubs, are providing your teen with this positive experience, we are evaluating the program. To evaluate the program, Ms. Melissa Freel, who is one of our adult staff members, will be interviewing twenty of the kids who participate in Club. She will ask them 26 questions about what they think of Club. For example, three of the questions are ”How often do you come to Club?"; What activity do you like the most about Club?"; "Why do you think that other kids have not joined Club?” Ms. Freel will interview each teen individually during Club. The interview is expected to take about 20 minutes. Also, in order to capture everything that your child has to say about Club, Ms. Freel will tape record their answers. All of the information that we collect will be compiled for the group as a whole. The identity of the teens will be kept confidential and, after Ms. Freel transcribes the tapes, she will erase them. No names will be connected with any of the information collected from your child. Your teenager will also sign a consent form before participating in the interview, but we can only conduct the interview if we have your consent too. Participation in these evaluation activities is completely voluntary. You can withdraw your child's participation or your child can withdraw from participation at any time during the evaluation. But we do need your consent for your teen to participate. For those teens who do participate, there will be a raffle held at each school for a gift certificate at McDonald's. If you have any questions, you can call Ms. Freel or Ms. Yvette Flippen at the B—103 Teen Health Center at 785-9869. If you would like to give consent for your teen to participate in this evaluation, please sign the following parent consent form. 118 Parent Permission Form I have read the above description of the Club Dolan and Club Summit activities, and I give my consent for my child to participate in the evaluation. I understand that these activities are voluntary and either myself or my child can withdraw participation at any time. (Print name of Child) (Their grade) (Parent signature) (Date) (Phone where you can be reached) APPENDIX A.2: ADOLESCENT CONSENT FORM 119 Consent form for Teens (TO BE READ BY EVALUATOR TO PARTICIPANT) I am going to ask you a few questions about what you think and how you feel about Club. What you tell me today will help all of the Club staff do a better job so that Club is a fun and a safe place for you to come after-school. Also, because I care about what you have to say about Club, I am going to tape record your answers. After we are done, I am going to listen to the tape and write your answers down on a piece of paper, but I am not going to use your name and no other staff person is going to know what you said about Club. Once I am done writing your answers, I am going to erase the tape. Do you understand what we are going to do here? Is it OK. that I am going to tape your answers? If you said yes to each of these, please write your name below on the line. Please keep in mind that you do not have to answer these questions and you can quit at anytime. (Program participant writes name) Gender Grade School Date APPENDIX A.3: STAFF CONSENT FORM 120 Staff Consent Form I understand that I will be participating in a focus group for the purpose of evaluating the after school program. I understand that this focus group will be audio-taped, and that my participation is voluntary. Furthermore, I may withdraw my participation at any time. (print name) (date) (Signature) APPENDIX B]: SEMI-STRUCTURED INTERVIEW FOR FIRST 11 INTERVIEWS APPENDIX B.1: SEMI-STRUCTURED INTERVIEW FOR FIRST 11 INTERVIEWS Semi-Structured Interview for Adolescents Date: Time: Gender: School: Grade: 121 122 Personnel 1. Can you tell me who are the staff of Club? Prompt: Tell me as many of their names as possible 2. When another kid in Club is "messing" with you, what do you do? Why? Prompt: Do you ever go to a staff person? Why? 3. What happens when you go to a staff person for help? 4. I am going to give you a sheet of questions about how you feel about the adult staff and the teen staff of Club. I would like you to read each question and circle one of the faces that matches how you feel. I am not going to look at your answers, and after you are done, I want you to fold up your sheet and put it in this envelope. 5. Are there any adults, other than your parents, that you like? Why do you like them? 6. Are there any adults, other than your parents, that you don't like? Why don't you like them? 7. Just for pretend, you get to be a Club staff person for a day, How would you act and what would you do? Activities 8. What is there to do when you come to Club? 9. What do you usually do when you come to Club? Why? 10. What activities do you like the most in Club? Why? 11. What activity do you like least in Club? Why? 12. What would you miss if Club did not exist? 123 Other Program Participants 13. Do you think that most of the other kids in Club are in the same grade as you? Are they mostly boys or mostly girls? 14. Does it make a difference if Club has both boys and girls? Would it be better if it were only boys or only girls? Why? 15. Does it make a difference that Club has younger and/or older kids? Would it be better if there were just older/younger kids? Why? 16. Can you think of any problems that you have had with other kids in Club? What are they? 17. If you have had problems with kids, why do you think they are "messing" with you? 18. Can you think of a way that other kids have helped you in Club this year? Contextual Assets/Limitations 19. How often do you come to Club? Prompt: Is it only once a week, once a month, all the time? 20. Why did you join Club? 21. Are there any special requirements that you have to do to stay in Club? Prompt: Are they easy or hard requirements? 22. What did you have to do to join Club? Prompt: What that easy or hard for you to do? 23. How did you find out about Club? 24. When you do not come to Club, what are you usually doing? Prompt: homework, watching T.V., hanging with friends, staying after school with a teacher? 25. Why did you come to Club today? 26. Why did you come or not come to Club last week? 124 27. Has anyone ever told you that you cannot come to Club? Who are they, and why did they tell you this? 28. Why do you think that other kids have not joined Club? Goals 29. Why do you think that we have Club? 30. Do you think that Club helps kids in some way? (If yes, how does Club help them). Parents 31. What do your parents/guardians think of Club? 32. Have your parents ever helped out with Club? Prompt: Do you think they would like to help out? 125 Question #4 Directions: For each of the questions below, circle only one of the faces. 4a) Adult Club staff listens to me: No“ Sort of 4b) Adult Club staff care about me: Not! Sort of 4c). Adult Club staff are strict with me: 0 Yes Notl 4d) Adult Club staff let me do what I want: Notl Sort of 126 4e) Adult club staff is respectful towards me: Nor! Sort of 40 Adult Club staff believes in me: 93 a. Not! Sort of Now, please tell us how you feel about the Teen Club staff by circling only one face for each of the following questions: 4g) Teen Club staff listens to me: Not! Sort of 4h) Teen Club staff care about me: Not! Sort of 127 4i) Teen Club staff are strict with me: Not! Sort of 4]) Teen Club staff let me do what I want: Notl Sort of 4k) Teen Club staff is respectful towards me: Not! Sort of 41) Teen Club staff believes in me: Not! Sort of APPENDIX B.2: SEMI-STRUCTURED INTERVIEW FOR SECOND 11 INTERVIEWS APPENDIX B.2: SEMI-STRUCTURED INTERVIEW FOR SECOND 11 INTERVIEWS Semi-Structured Interview for Adolescents Date: Time: Gender: School: Grade: 128 129 Personnel 1. Can you tell me who are the staff of Club? Prompt: Tell me as many of their names as possible 2. When another kid in Club is "messing" with you, what do you do? Why? Prompt: Do you ever go to a staff person? Why? 3. What happens when you go to a staff person for help? 4. I am going to give you a sheet of questions about how you feel about the adult staff and the teen staff in Club. I would like you to read each question and circle one of the faces that matches how you feel. I am not going to look at your answers, and afier you are done, I want you to fold up your sheet and put it in this envelope. [Attached] 5. Are there any adults, other than your parents, that you like? Why do you like them? 6. Are there any adults, other than your parents, that you don't like? Why don't you like them? 7. Just for pretend, you get to be a Club staff person for a day, How would you act and what would you do? Prompt: Do the staff people of Club act that way most of the time, some of the time, or not at all? Activities 8. What is there to do when you come to Club? 9. What do you usually do when you come to Club? Prompt: Is this the activity that you like the most?--Why 10. What activity do you like the least in Club? Why 11. What would you miss if Club did not exist? Other Program Participants 12. Do you think that there is an equal number of 5th, 6th, 7th, and 8th grade boys and girls in Club, or do you think that some grades have more kids in Club? Is Club made up of mostly boys or mostly girls or is it equal? 130 13. Does it make a difference if Club has both boys and girls? Would it be better if it were only boys or only girls? Why? 14. Does it make a difference that Club has younger and/or older kids? Would it be better if there were just older/younger kids? Why? 15. Can you think of any problems that you have had with the other kids in Club? What are they? 16. If you have had problems with kids, why do you think they are "messing" with you? 17. Can you think of a way that other kids have helped you in Club this year? Contextual Assets/limitations 18. Do you belong to any other after-school activities? What are they? Why do you belong? 19. How ofien do you come to Club? Prompt: Is it only once a week, once a month, all the time? 20. Why did you join Club? 21. Are there any special requirements that you have to do to stay in Club? Prompt: Are they easy or hard requirements? Who told you that these were the requirements? What do you think of them? 22. What did you have to do to join Club? Prompt: Was that easy to do or hard for you? 23. How did you find out about Club? 24. When you do not come to Club, what are you usually doing? Prompt: homework, watching T.V., hanging with friends, staying after school with a teacher? 25. Why did you come to Club today? 26. Why did you come or not come to Club last week? 27. Has anyone ever told you that you cannot come to Club? Who are they, and why did they tell you this? 131 28. Why do you think that other kids have not joined Club? Goals 29. Why do you think that we have Club? 30. Dottyou think that Club helps kids in some way? (If yes, how does club help em) 31. How is Club different or the same from school? 32. Some kids say that Club is a safe place, do you think it is safe? What does safe mean? Parents 33. What do your parents/guardian think of Club? 34. Have your parents ever helped out with Club? Prompt: Do you think they would like to help out? 132 Question #4 Directions: For each of the questions below, circle only one of the faces. 4a) Adult Club staff listens to me: 6 No" Sort of 4b) Adult Club staff care about me: Not! Sort of 4c). Adult Club staff are strict with me: 0' Yes Notl 4d) Adult Club staff let me do what I want: 133 4e) Adult club staff is respectful towards me: ”01' Sort of 4f) Adult Club staff believes in me: Not! Sort of Now, please tell us how you feel about the Teen Club staff by circling only one face for each of the following questions: 4g) Teen Club staff listens to me: Not! Sort of 4h)" Teen Club staff care about me: Not! Sort of 134 4i) Teen Club staff are strict with me: Not! Sort of 4j) Teen Club staff let me do what I want: Not! Sort of 4k) Teen Club staff is respectful towards me: Not! Sort of 41) Teen Club staff believes in me: Not! Sort of APPENDIX B.3: FOCUS GROUP QUESTIONS FOR STAFF APPENDIX B.3. FOCUS GROUP QUESTIONS FOR STAFF Focus Group Questions for Staff Introduction As you know, we have had many staff meetings throughout the school-year, and during these meetings we have often discussed our thoughts and feelings about Club. Today, I am going to conduct a focus group with you all in order to capture some of those thoughts and feelings on paper. I am going to audio-tape our discussion. I will transcribe the tape and use the information to construct an evaluation report. Following the transcription, I will erase the tape. I need to hear from every one of you throughout our discussion; however, each one of you do not have to answer all of the questions. When one person is speaking, I ask the rest of you to remain quiet, and when you are speaking, please speak loudly enough so that the tape recorder can capture what you have said. Finally, when I do the transcription, 1 will not identify you. Now, I am going to pass out a consent form and if you understand everything that it says and you have no questions regarding your participation in this focus group or in the evaluation, please sign it. Goals 1. Throughout the school year, we have often talked during our staff meetings about what we see this program is doing for the kids involved. Let's take a minute to reflect on these thoughts and list them on this newsprint: 2. Of all these concepts on this sheet, which one would you see as the primary goal of the program? 3. How do you think we, as a staff, are accomplishing this goal? Program Participants 4. Do we serve kids equally from all the grades, or do we have more younger ones, older ones? Do we serve more girls than boys, more boys than girls, or is it equal? 5. From our answers in question F our, why do you think it is that way? 135 136 6. In general, how do you think our kids act behaviorally towards one another? Prompt: What are some situations that cause negative interactions/positive interactions? 7. What are our expectations for behavior of the kids in Club Prompt: Are they too lenient, too strict? 8. In what ways do you think interaction with the high school staff affects the behavior of the Club kids with one another? Activities 9. What activities do you think the kids like the most? the least? Why? 10. What do you think the Club activities provide for the kids? Why? Personnel 11. On a piece of paper, I would like you to write down several characteristics about yourselves. Now, look over these characteristics, which of these do you think the kids in club would consider to be important in a Club staff person. Prompt: let's discuss some of these characteristics 12. On the same piece of paper, I would like you to write down as many characteristics as you can think of that would describe a "model" staff person? Prompt: let's discuss these characteristics, as a group do we demonstrate these characteristics? 13. In what ways do you think that we address or don't address the cultural and individual needs of the Club kids? Contextual Assets/Limitations 14.. Why do you think the kids have joined Club? 15. Why do you think some kids have not joined Club? Prompt: is there some activity or person preventing their participation? 16. Do we involve parents, if so, how? Prompt: How does parent involvement affect the program? 17. What are the benefits and the challenges of working with the school personnel? 137 18. Think about the school buildings, does the physical structure of these buildings present any challenges or benefits to the program? If so, what are they? 19. How has the training we received, prepared or not prepared us for our jobs in Club? APPENDIX B.4: PROGRAM FUNDING PROPOSAL 1. Problem Statement The consequences of teenage pregnancy are well documented and demonstrate the great need to prevent early childbearing among adolescents. Volumes of research has focused on programs that attempt to prevent pregnancies and mitigate the consequences to mother and child. Over the past several years, a number of leading adolescent specialists have demonstrated that effective prevention strategies must move from more traditional one- dimensional approaches to comprehensive models. Joy Dryfoos, in "Adolescemm 3151;; WWW" proposes that effective pregnancy prevention programs for high risk teens must provide intensive individual attention within a group setting, focus on early interventions, be based on promoting success in school, and be a community based, multi-agency collaborative approach. Effective interventions must especially focus on the antecedents or predictors of early childbearing among youth at highest risk. The research shows that one can predict those most at risk by identifying 6 antecendents. Children who become parents are more likely to: l. initiate sexual activity at younger ages than their peers. 2. do poorly in school and not be engaged in school activities. 3. have truancy and acting out problems. 4. be easily influenced by their peers. 5. come from low income neighborhoods and live in segregated neighborhoods. 6. have inattentive or nonsupportive parents. Early sexual activity becomes a powerfirl substitute for a sense of self othemise gained through school success, positive home life and good peer relationships. These antecedents are also strong predictors for substance abuse, delinquency and school dropout, in fact all four high risk behaviors are strongly interrelated Of particular importance in predicting teenage pregnancy is the prObability that teens, both males and females, will become sexually active at an early age and become teen parents if they are doing poorly in school. Several national surveys and studies have shown that contrary to conventional wisdom, more than one fourth of the teens do not drop out of school after but before the pregnancy. In the National "High School and Beyond Survey" sephomores with low academic abilities were twice as likely to become teen parents. The "National Longitudinal Survey" found that females in the bottom quintile on math and reading scores were five times more likely to become teen parents over a two year period than the tap quintile. The F urstenberg and Brooks-Gum study of teen mothers found that more than feeling despair about their futures, many young mothers felt "despair about the present, especially their lack of success in school.” This proposal requests fimding for an After School Intervention Program to decrease the number of teens that initiate sexual activity before age 15. The Beecher School District is located just north of the city of Flint, and has approximately 24,000 residents of which approximately 1200 are ages 9-14 and 1000 are 15-18. The area is over 80% African American, with high unemployment and a large percentage of single, female headed 138 139 households. 87% of the teens in Beecher schools are African American and 12% are white. In Genesee county, the teen pregnancy rate is over 16.7. It is estimated that Beecher currently has over 150 teen parents. In 1991 the Beecher Teen Health Center provided almost 400 pregnancy tests and confirmed 86 pregnancies, of which 48 resulted in a birth (of a service population of approximately 850 clients). Young teens attend one of two middle schools, Dolan or Summit. The biannual survey to assess needs for the Teen Health Center conducted in May 1991 found that these middle school students are indeed at high risk for early sexual initiation. In fact 36.1% of these teens in grades 5-8 reported that they are sexually active (have had intercourse), with 9.4% reporting that they have intercourse at least weekly. (Of these, 58.2% report they never use contraception and 24.3% always do. 8.4% have been or gotten someone pregnant.) The data shows that Beecher teens on average do poorly in school: by 12th grade, over 20% of all teens will have drOpped out. No Beecher students exceeded the 40th percentile in any one category on CAT tests in math and reading in 1991. The 1991 MEAP scores found only 4% of tenth graders at 75% or better. Truancy and related problems are endemic in the middle schools. In 1991, 80% of all Dolan students and 40% of all Summitt students were suspended at least once, including 1409 suspensions for fighting, 34 for weapons, 178 for physical abuse and 60 for vandalism. Beecher teens live in an impoverished area as shown by the fact that Beecher has the highest percentage of free or reduced lunches in Genesee County at over 80%. Parental unemployment estimates range fi'om 40-70%. Staff reports of parent participation in school activities is reported to be minimal. 2. Project Description This proposal for an Afierschool Enrichment Program is based on the premise that to have an impact in delaying early sexual activity, interventions must focus on the antecedents to such behavior. They should focus on dela 'rrgearly initiation of sexquens do better in schooWfiifiniieglethnegatmeer pressure and createpositive pressures, and improve parent and child communication mm involvement of parents in their children's activities. The program is modeled after the Pregnancy Prevention Project of the Children's Aid Society in New York and the Fifth Ward Enrichment Program in Houston. This program has already been piloted at each of the two middle schools during the summers of 1991 and 1992. Limited funding has prevented implementation during the school year. The A fterschool Enrichment Program consists of a school year program at each middle school, meeting two times a week/school for 2 hours. Known by students as Club Dolan or Club Summit, the program will provide an atmosphere of safety, firn, recreation and learning. Each two hour session will consist of 60 minutes of physical activities (open gym, competitive or non-competitive games), crafts, academic challenges, and special 140 projects such as talem showcases, and acting instruction; 15 minutes for refreshments and 45 minutes of life skills and human sexuality instruction and discussion. The first hour will be designed to encourage teens to compete without fighting and to develop positive relationships with their peers in an environment that promotes a sense of belonging. Interesting and fun activities will be planned so that teens develop a more positive concept of their own school environment. The life skills and human sexuality programs will be designed within the fi’amework of the "Special Beginnings and CrosSroads" curriculum for teen males and females This curriculum is open-ended and is adaptable to specific settings. It includes growth and development and sexuality education, development of positive relationships, encouragement of abstinence, techniques to combat negative peer pressures, and life skills planning (including development of preemployment skills, decision making skills, etc.) Tutoring and homework help will be available to any student needing or wanting it during the two hour period. During the first hour the play interactions among teens create opportunities to provide feedback and teach teens positive ways to relate to each otha'. Staff will continually exploit interactions among the participants as learning opportunities. The Life Skills and Human Sexuality component of the clubs is designed to formalize learning opportunities. The format will be one of outside presentations, group discussions, drama and group theater and role playing and written exercies. ' The program will include field trips as incentives for positive group behaviors. The field trips will be selected by the teens and become progressively "better" throughout the year. The group must earn points to buy their trips. Points will be earned by overall group grade point average (as it improves throughout the program), lack of suspensions or detentions in school by members, lack of fighting during the club by members, participation and completion of specific activities in the life skills and human sexuality program, attendance at the clubs, and group participation in community service activities. The focus will be on the entire group's behaviors, in an effort to promote a sense of something "good to be involved with" and to teach the benefits of positive peer pressures. It is expected that teens will experience group presarre to do "well" for the benefit of the entire group. Adult role modeling and interventions by staff are key to the program's success. 2 Teen Health Center Peer Educators, 4 college students, two tutors during the school year, the Teen Health Center Health Educator and Psychologist will staff each club and be on srte for each club meeting. Parent involvement will be encouraged in several ways. All teens and their parents will sign a contract to participate in the Club. The Crossroads and Special Beginnings curriculum have specific parent involvement components. Teens will earn additional points when their parents attend these sessions with the teens. Parents will also be encouraged to attend the field trips with the teens and to "chaperone" at other club activities. It is expected that through involvement in the curriculum and the field trips, parents will establish longer term committments to their children's school and 141 extracurricular activities as parent involvement is an essential ingredient to delaying early sexual experiences. The sites for the Clubs will be each middle school gym and community room Currently the middle schools lock their doors at 3:00 PM. each day, except for organized sports. The district has made the committment to keeping the buildings open for the Clubs. The Program's time line begins in December 1992, when arrangements are made to secure space, supplies, and staffing. The program will be ready for implementation immediately after the Christmas break. It will run each week, including during the summer, ending the week of August 24, except for Easter and spring vacations for a total of 30 weeks: eighteen during the school year and twelve weeks during the summer. Tutoring will not be provided during the 'surnrner, in as much as the school district offers summer tutoring to students. Pending continuation finding, the program will operate into the following school year. The evaluation design will be completed by early January, with an initial evaluation completed at the end of the first nine month period. Specific Outcome Objectives for the Program (and Evaluation Measurements) at the end of the first program year are consistent with Healthy Youth 2000 objectives and include: I 1. No more than 15% of the program participants (who had not engaged in sexual intercourse prior to the program) will be sexually active. (Written questionnaire) 2. At least 50% of the program participants will have abstained from sexual activity for , the final three months of the program (as compared to youth 2000 objectives of 40% ages 17 and under). (Written questionnaire) 3. Participants will report significantly improved abilities to resist negative peer pressures- and will report significant improvements in self concept. (Scored self concept instrument and questionnaire) 4. Student and Parent knowledge about human sexuality will be significantly improved. (pre and post tests during curricula implementation) 5. There will be a 0.50 average grade point improvement among participants. (school records) 6. There will be a 50% reduction in the rates of suspension and detentions among program participants. (school records) 7. There will be a minimum of 10 parents per program site actively involved in club activities. (Parent census) 8. All sexually active participants will be using dual methods of contraception and protection. (questionnaire) 142 9. There will be no teen pregnancies among program participants during the duration of the program. (questionnaire) 3. Coordinating with Related Community Resources The program will be coordinated through the Beecher Teen Health Center, which is located in Beecher High School. This Center was established in part because of the paucity of services available to teens in the Beecher School District. Located approximately 8 miles from downtown Flint and 3 miles from the county health department McCree clir'ric, the center has provided a number of services not otherwise available to teens. This program will be an unduplicated effort. There is currently no program efforts directed in Beecher towards middle school youth, with the exception of organized sports. Beecher schools participate in the Michigan Model for Comprehensive School Health, however human sexuality education is taught through the gym classes in only the seventh grade. . This program will be implemented in collaboration with a number of other services: the Beecher school nurse will be actively involved in recruiting club members and in providing services to them. Sexually active teens will be referred to the Teen Health Center where comprehensive family planning services are available, including referrals to Mott Children's Health Center for contraceptives. The Teen Center has two pediatricians with specialty training in adolescent medicine including gynecology. Prenatal care is also provided on site by a nurse midwife. Teen parents that participate in the program will be referred to the Center’s Teen Parent Support Program, which ofi‘ers case management support and weekly support group meetings for teens parents (male and female). Locally available resources will be utilized for the human sexuality and life skills component including human sexuality educators from the Genesee County Health Department, Planned Parenthood, HIV Wellness Network the Beecher Schools Literacy Center (currently being established), the Chapter I and II remediation support services, and the adult education and continuing education program for preemployment services. Local community agencies will be asked to participate as guest presenters at the Clubs. Beecher has an active group of neighborhood block clubs that will be contacted for community service projects. The Beecher Men's Club and local panhellenic societies will be asked to participate in specific club recreation activities. 4. Target Population The program will be open to any teen attending either Dolan or Summitt Middle Schools, in grade 5-8. During the pilot program over the past two summers, recruitment was 143 solicited via fliers and posters; enrollment remained steady at 50 students for Dolan and 35 for Summitt. Being a school year program it is anticipated that upwards of 100 and 60 students could enroll at each school respectively for a total of 160 participants. There will be no other eligibility requirements. The characteristics of the student body were described in some detail in the problem statement. 5. Evaluation Plan: The measures of success for the program will be whether or not the specific objectives for the program were accomplished. These objectives with specific measurements are described in Section 2. In instances where questionnaires will be utilized, students will be asked to complete a preenrollment and end of program survey. This survey will ascertain specific demographic characteristics of the participants, as well as health and sexual attitudes and behaviors. It will be similar to the survey that was completed by all middle school students, through the Teen Health Center, in May of 1991. ‘ Standardized instruments will be utilized to measure abilities to resist peer pressures and student's sense of self worth. Pre and post tests of knowledge gained will be collected at specific points throughout the curriculum's implementation. These will be developed by the Center's Health Educator. School records will be examined at the beginning, end and specific program intervals, to measure grade point improvements and suspension/detention data. There will be obvious problems with this evaluation methods, most particuliariy self reporting by teens of their sexual activities. However an analysis of the Teen Health Survey conducted in May of 1991, which included 170 questions, found a great deal of consistency within individual responses across a broad range of questions. In measuring if the program had an impact on reducing teen pregnancy, the evaluation will focus on specific results of the participants. A control sample will also be assessed for attitudes and behaviors related to sexuality, school grades and truancy issues in an attempt to make comparisons and interpretations. This control sample will be identified fi'om teens not participating in the program and attempts will be made to closely match both groups across as many variables as possible. This recognizes that participation in the program group implies a certain amount of self selection. These variables will be factored into the analyses of results. The results of the evaluation will be written in report form, published and disseminated. 6. Confidentiality All student generated questionnaires will be submitted without individual student ‘ identification. Referrals to Teen Health Center services will remain strictly confidential, in 144 accordance with Teen Health Center policies. Informed consent to participate in the programwill be obtained through development of a contract, to be signed by both teens and their parents. Informed consent for other services at the Teen Health Center will be in accordance with Center policy. Any teen utilizing Center services must have a signed parent permission form on file. 7. Personnel Qualifications The program will be coordinated by the Beecher Teen Health Center Coordinator, Teri Robinson. She has a Masters in Public Health Planning and Administration/Health Behavior and Education, as well as 12 years experience coordinating community based health programs including three as the Teen Center Coordinator. Micheal Young will be the Site Supervisor for the program. He will be responsible for on site management, staff supervision, scheduling for all activities and the program evaluation He will be on site during all hours of club activities. Dr. Young has a Ph.d. in clinical psychology and was a Bush Fellow specializing in adolescent issues. He spent 8 years in the Boston Public Schools and at the Wedeiko Institute, developing and supervising alternative intervention programs for high risk adolescents similar to this proposed intervention I The Health Educator for the program will be the Teen Health Center Health Educator, Yvette Flippen. “fith a bachelors in Health Education, Ms. Flippen has 6 years experience in community health education programs and is the coordinaotr of the Peer Educator Program Her responsibility will be to develop and serve as the primary instructor for the Crossroads and Special Beginnings Human Sexuality and Life Skills curriculum. The Beecher Teen Health Center‘s Dramatic Arts Instructor, Phil Wallace, is a professional actor who will work with the peer educators and the club members to develop talent shows and dramatic skits focused on role playing specific issues for the cuniculum. Two Peer Educators fiom the Beecher Teen Health Center will serve as staff to each Club, for a total of four peer educators. These are high school students, that have received 12 weeks at 30 hourfldch week of intensive health training during the summer in all aspects of adolescent health, but with a specific emphasis on sexuality, HIV, substance abuse, and violence/conflict resolution. They specialize in the presentation of information through theater. They will be responsible for assisting Dr. Young at each club site: leading the recreation activities, setting up the necessary sites, preparing refreshments, and serving as positive role models. They will also teach a number of lessons during the human sexuality curriculum. There will also be four adult program guides working with the Program. These guides will have similar responsibilities as the Peer Educators, but will be expected to provrde 145 more adult leadership, role modeling and individual interventions with students in the group. Two tutors will be assigned to the Clubs, to provide homework help and intensive tutoring to those students having particular difiiculties in school. Due to teacher contract issues, these tutors will probably be middle school teachers. 8. Qualifications of the Agency The program will be implemented by the Beecher Teen Health Center, which is a program of Mott Children's Health Center. The Teen Health Center has been in operation for three years and provides a wide array of comprehensive services in the district. The Center is very highly regarded in' the district and has been providing a number of Outreach services to the middle schools for the past three years. Mott Children's Health Center is a non- profit children's Health Center providing over 75,000 medical, dental, child development and behavioral services annually to low income children in Flint. The Center has over a 30 ' year history of providing comprehensive site based and community outreach services. 9. Budget Justification I Staffing: As described in the personnel qualifications on the budget pages, the staff are needed for their specified hours. All current Teen Health center staff services will be provided in kind through MCHC firnding. The adult guides and tutors will be paid for 100% fi‘om this proposal. Two of the Peer Educators will be JTPA eligible in the summer, thus 12 weeks of their salaries will be paid for by the school district. Each peer educator costs the district 17% in benefits. The Peer Educators and Adult guides will each work 10 hours per week, 2 hours per club meeting times four club meetings a week, and two hours of planning and preparation. Travel costs will be paid for through the WPH Teen Health Center funding agreement. Estimates for Club supplies are based upon the summer pilot programs. It is expected that refi'eshments in the form of a light snack and juice costs $20/meeting x 4mtgs/week x 30 weeks. It is anticipated that $2000.00 for make it-take it craft projects will be needed. General office supplies and curriculum materials will cost $500.00. Much of the curriculum materials is available through the Teen Health Center. Estimates for the field trips based on the summer pilots average out to $450/trip. This is based on all trips being of similiar distance, which will not be the case. The earlier trips will be closer and not as expensive. If students meet all objectives, then a field trip could be considerably more expensive. The figure was arrived at as $150.00 for the bus and $300.00 in admission fees per trip @ one trip a month x two clubs. APPENDIX C]: DOCUMENT SUMMARY FORM FOR PROGRAM PARTICIPANTS APPENDIX C.1: DOCUMENT SUMMARY FORM FOR PROGRAM PARTICIPANTS Document Summary Form Date: School: Gender: Grade: File Name: Program Participant's View of Goals: Program Participant's View of the Purposes of the Program: Program Participant's View of Personnel: Program Participant's View of Activities: Program Participant's View of other Program Participants: Program Participant's Description of Program Processes (beginning, middle, or end): Program Participant's View of Contextual Assets, Maintenance, Limitation RED FLAGS: 146 APPENDIX C.2: DOCUMENT SUMMARY FORM FOR STAFF APPENDIX C.2.: DOCUMENT SUMMARY FORM FOR STAFF Document Summary Form Date: Time: Staff's View of Goals: Staffs View of the Purposes of the Program: Staffs View of Personnel: Staff's View of Activities: Staff's View of Program Participants: Staff's Description of Program Processes (beginning, middle, or end): Staffs View of Contextual Assets, Maintenance, Limitation RED FLAGS: 147 APPENDIX C.3: DOCUMENT SUMMARY FORM FOR PROGRAM PROPOSAL APPENDIX C.3.: DOCUMENT SUMMARY FORM FOR PROPOSAL Document Summary Form Date: Proposal's View of Goals: Proposal's View of the Purposes of the Program: Proposal's View of Personnel: Proposal's View of Activities: Proposal's View of Program Participants: Proposal's Description of Program Processes (beginning, middle, or end): Proposal's View of Contextual Assets, Maintenance, Limitation RED FLAGS: 148 APPENDIX D.1: CLUB RULES DEFINED BY STAFF 149 CLUB RULES . Absolutely NO FIGHTING or play fighting! This kind of behavior will not be allowed! No food.drink, candy or gum in any club area. Food and drink are only allowed during designated times in designated areas. Proper dress is required. (Example: No sagging pants falling 011‘ of your but!) No hats (both males/females). You will be asked to leave if you wear clothing that promotes drugs, sex, etc. During competitive games. the males will not be allowed to remove their t-shirts. Pall-overs should be worn on top of t-shirt. Bring an extra t-shirt if necessary. . "On the Wall" Club members are not allowed to leave any club area without the consent of a club staff person. Consent should be attained if you want to go to the rest room, join in with another group, etc. Being caught in the hallway without consent will get you time on the wall. Respect all club areas! Clean up any area that you mess up before you leave. Respect other school areas. Stay off of freshly mopped floors. Stay out of areas that you are not supposed to be in. 8. Report all injuries to a club staff person as soon as it happens! If an injury 10. 11. 13. involves blood, please do not wipe the blood on your clothes. If you do, you will no longer be able to participate in any physical activities unless you change clothes. Everyone must participate in the health education presentations. No hanging from the rims by way of the bleachers. No sitting on the bleachers that are not pulled out. Club staff will not be responsible for any personal belongings that may be left at club or stolen from club. Lock up your personal belongings in your locker. Send them home with a family member. Please do not bring any money or anything valuable to club with you. Remember everything should be done in an orderly fashion. No running. pushing or verbal abuse will be allowed. APPENDIX D.2: CLUB RULES DEFINED BY PROGRAM PARTICIPANTS APPENDIX D. 2: CLUB RULES DEFINED BY PROGRAM PARTICIPANTS Rules Identified by Program Participants 1. No play fighting, (trans 5, 6,14, 15, 16, 21, 22). 2. No "real" fighting (trans 8, 14, 19, 21, 22). 3. No hitting (trans 10, 14, 18, 22). 4. No eating candy/gum (trans 6, 16, 18, 19). 5. Respect Staff (trans 1, 4, 17). 6. No wrestling (trans 16, 22). 7. Be Polite (trans 14, 16). 8. Do not get in-school suspension (trans 11, 17). 9. Do not swear (trans 4, 15). 10. Do not chase people (trans 6, 19). 11. Do not disobey (trans 2). 12. Have Fun (trans 19). 13. No teasing (trans 22). 14. Stay in school area (trans 14). 15. No pushing (trans 18). 16. Be in school (trans 17). 17. Control temper (trans 15). 18. No name calling (trans 8). 19. Have a good attitude (trans 1). Note: "trans" refers to the file name that represent each of the 22 respondents 150 REFERENCES REFERENCES Baltes, P. 8., Reese, H. W., and Nesselroade, J. R. (1977). Life-span developmental psychology. Wm,fl, 65-110. Barker, R- G- (1963). WWWM enmnmentgflhumanflhm. Stanford, CA: Stanford University Press. Baumrind, D. (1978). Parental disciplinary patterns and social competence in children. WJ, 239-276. Benson, P- L. (1990).Iherrauhled_innan_n£fitlL121hsradamth (Research Rep.). Minneapolis, MN: Search Institute. Bronfenbrenner, U. (1979). W Main. New York: Cambridge University Press. Bronfenbrenner, U. (1986). Ecology of the family as a context for human development: Research perspectives._D_eyelgpmmtaLEsycthm, 22, 723-742. Bubolz, M. M., & Sontag, M. S. (1993). Human ecology theory. In P. G. Boss, W. J. Doherty, R. LaRossa, W. R. Schumm, & S. K. Steinmetz (Eds), WM famihatheonaanamethads (pp. 419-448). New York: Plenum. Boyle, P. G. (1981). W. New York: McGrawHill. 151 152 Boxer, A. M., & Petersen, AC. (1986). Pubertal change in a family context. In G. K. Leigh & G. W. Peterson (Eds), Adolescentsjnjamflies (pp. 73-103). Cincinnati, OH: South-Westem Publishing. Chen, H. T. (1990). WW5. Newbury Park, CA: Sage. Chen, H. T., & Rossi, P. H. (1989). Issues in the theory-driven perspective. Evaluation anflmgtamflannins, 1.2, 299-306. Carnegie Council on Adolescent Development. (1992). W W (Research Rep.). Woodlawn, MD: Wolk Press. Cronbach, L- J- (1982). Desisningsxaluationanfleducatinnalaniaaeialmstams SanFrancisco: Jossey-Bass. Dryfoos, J. G. (1990). W. New York: Oxford University Press. Dryfoos, J. G. (1993). Schools as places for health, mental health, and social services. IeachersEnllese, 9.4., 540-567. East, P. L., Hess, L. E., & Lerner, R. M. (1987). Peer social support and adjustment of early adolescent peer groups. WWWJ, 153-163. Edleman, M- W- (1992). Ihemeanmfnunsncceamlettmnmuhfldrsnand yours. Boston: Beacon Press. Fetterrnan, D. M. (1989). WWW. Newbury Park, CA: Sage Publications. 153 Fisher, C. B., & Lerner, R. M. (Eds.). (1994). Applied developmental psychology. New York: McGraw-Hill. Ford, D. H-. & Lerner, R- M- (1992)- DexelnpmentaLsxstemsahenmAantegratme approach. Newbury Park, CA: Sage. Fordham, S., & Ogbu, J. U. (1986). Black students' school success: Coping with the burden of acting White. Wm, 18, 176-206. Foster-Clark, F. S., & Blyth, D. A. (1991). Peer relations and influences. In R. M. Lerner, A. C. Petersen, & J. Brooks-Gunn (Eds.),_Encyglgp_edia_Qf_Adgles_c_eng§, 2, 767-771, New York: Garland. Greene, J. C. (1994). Qualitative program evaluation: Practice and promise. In N. K. Denzin & Y. S. Lincoln (Eds), Handbookgfqualitafimmmp. 530-544). Thousand Oaks, CA: Sage. Hamburg, D. A. (1993). The opportunities of early adolescence. Me, 951, 466-471. Hernandez, D. J- (1993). WWW W. New York: Russell Sage Foundation. Jacobs, F. H. (1988). The five-tiered approach to evaluation: Context and implementation. In H. Weiss & F. H. Jacobs (Eds.),_Eyalnating_famithmgrams (pp. 37-68). Hawthrone, NY: Aldine de Gruyter. Jorgensen, D. L. (1989). W. Newbury Park, CA: Sage Publications. 154 Keith, J. G., Perkins, D. P., Ferrari, T., & Covey, M. (1994). Community applications of human ecological theory, Seventh International Meeting of the Society for Human Ecology. April 21-24, 1994. East Lansing: MI. Kirk. J., & Miller, M. Le (1986)..R:liahilihLand_xalidiU_m_qualitati¥s_r£ssamh. Newbury Park, CA: Sage. Kosecoff, J., & Fink, A. (1982). WWW. Newbury Park, CA: Sage. Kozol, J. (1991). W. New York: HarpersCollins. Kretzrnann, J. P. &, McKnight, J. L. (1993). WM Anathfontardfindmaanimnbrhzmgammnmntaassets Evanston, IL: Northwestern University. Lerner, R. M., & Spanier, G. B. (1980)._Adglesg_em_de1elgpmem;_A_life_-§pan My; New York: McGraw-Hill. Lerner, R. M. (1984). W. New York: Cambridge University Press. Lerner, R. M. (1986). W (2nd ed.). New York: Random House Lerner, RM. (1987). A life-span perspective for early adolescence. In R. M. Lerner & T- T- Foch (1548-). BinlngicahpsxchnlngicaLinteractionaineathadnleacsncs (pp. 9-34). Hillsdale, NJ: Erlbaum. 155 Lerner, R. M. (1989a). Developmental contextualism and the life-span of person- context interaction. In M. Bomstein & J. S. Bruner (Eds.), Intemfimjnhuman developing-m (pp. 217-239). Hillsdale, NJ: Erlbaum. Lerner, R. M. (1989b). Individual development and the family system: A life-span perspective. In K. Kreppner & R. M. Lerner (Eds.), WWW snanrlexelnpment (pp. 15-31). Hillsdale, NJ: Erlbaum. Lerner, R. M. (1990). Plasticity, person-context relations, and cognitive training in the aged years: A developmental contextual perspective. W W, 2.6, 911-915. Lerner, R. M. (1993a). Investment in youth: The role of home economics in enhancing the life chances of America's children. WW, 1, 5-34. Lerner, R. M. (1993b). A developmental contextual view of human development. In S. C. Hayes, L. J. Hayes, H. W. Reese, & T. R. Sarbin (Eds.),1meties_of scientifiuomcxnmism (pp. 301-316). Reno, NV: Context Press. Lerner, R. M., & Miller, J. R. (1993). Integrating human development research and intervention for America's children: The Michigan State University model. laumalanpnlredflexelnpmentalIthlongM 347-364. Lerner, R- M- (1995). Amencauhrldrenandadnlssscntsnnmsrafihallengeaand WWW. Thousand Oaks, CA: Sage. Lipsitz, J- (1986). Afletsshnolzlnungadnlessemsmfheinm Canboro, NC: Center for Early Adolescence, University of North Carolina at Chapel Hill. 156 Louis, K. S- (1981)me WW5. Paper presented at American Educational Research Association, April, Cambridge, MA. Luster, T., & Mittelstaedt, M. (1993). Adolescent mothers. In T. Luster & L. Okagaki (EdS-),RarrntingLAn_eannaicaLnersnecti1e (pp- 69-99). Hillsdale, N. 1.: Erlbaum. Marshall, C., & Rossman. G. B. (1989). W. Newbury Park, CA: Sage Publications. Meister, R- J- (1979). WWWMfis (unpublished manuscript). Ann Arbor: University of Michigan. Miles, M. B., & Huberman, A. M. (1994)“me W515. (2nd ed.). Thousand Oaks, CA: Sage. Nightingale, E. O., & Wolverton, L. (1993). Adolescent rolelessness in modern society W, 93, 472-486. Ostrom, C. W., Lerner, R. M., & Freel, M. A. (1994). Building the capacity of youth and families through university-community collaboration: The development-in-context (DICE) model. Available fi'om the Institute for Children, Youth, and Families, Michigan State University, Suite 27, Kellogg Center, East Lansing, MI 48824. Patton, M. Q. (1987). W. (2nd ed.). Newbury Park, CA: Sage. 157 Perkins, D. F ., Ferrari, T. M., Covey, M. A., & Keith, J. G. (1994). Getting dinosaurs to dance: Community collaborations as applications of ecological theory. Hgme EeonnmicaEnmm, 1, 39-46. Protth-Stith- 133- (1991). WWW W New York: HarpersCollins. Rosenthal, R. & Rosnow, M. (1975). W. New York: John Wiley & Sons. Schatzmann, L., & Strauss, A. L. (1973). Field research: Strategies for a natural sociology. Englewood Cliffs, NJ: Prentice-Hall. Schorr, L- B. (1988)-lMithin_ouLteachLBteaking_the_cxcle_oLdisad1antage New York: Doubleday. Scriven, M. S. (1980). W. Inverness, CA: Edgepress. Scriven, M. S. (1983). Evaluation ideologies. In G. F. Madaus, M. Scriven, & D. L. Stufflebeam (1348-), mmmnummmmmmmnm W (pp. 229-260). Boston: Kluwer-Nijoff. Silverman. D- (1993). mmmmmmammmnmkm andjmeraction. Thousand Oaks, CA: Sage. Simons, J. M., Finlay, B., & Yang, A. (1991). WWW hook. Washington, DC: Children’s Defense Fund. Smith, M. J- (1990) W New York: Springer publishing. 158 Steckler, A., McLeroy, K. R., Goodman, R. M., Bird, S. T., & McCormick, L. (1992). Toward integrating qualitative and quantitative methods: An introduction. HealtlLEducatianmrterlx, 12(1), 1-8. Steinberg, L. (1989). Adolescence. (2nd edition), New York: Alfred A. Knopf. Thomas, A., & Chess, S. (1977). W. New York: Brunner/Mazel. Timmer, S. G., Eccles, J. & O'Brien, 1. (1985). How children use time. In F. T. Juster & F. B. Stafford (Eds). MW. Ann Arbor: University of Michigan, Institute for Social Research. Weber, L. R., Miracle, A., & Skehan, T. (1994). Interviewing early adolescents: Some methodological considerations. Humanflrganizaflgn,_53_, 42-47. Woods. M. M. (1991). ammnammmmnmngmmm (Available from Program on Non-Profit Organizations, Yale University, P. O. Box 154 Yale Station, 88 Trumbull St., New Haven, CT 065250). "Illilllilllilllllllllllllf