‘ "n ,u‘ .J n . '1 awn 1451‘ '4 M h- “(.1 .. u . v5 r ~/ 7r o 3- o ,- ~39}. c A firm N: .~.. ”cm“ A w“:-“" a “.. \ . - w «<4 u, \ A“: .‘ .. V:¢.x,1,-...z~ '_-:..::- ‘r- “Ia: .3753"? as: A. » . 3.30.." r 5,9315% '—.. . I" It? . n. J'I'IJ. , ‘ n 23%. .r'ag‘wu' ‘l. ‘ h E5 95? ‘ ~."?‘. ' ’ ,1 ,r .fi':~!;§:§‘3§«’3 .r . 7 ‘ "kW "93% _ ~ ~ ; -- 7 ; 5.. «a,» . - a. ",3: 1r 2‘3: yaw-"tr g ' fi-«Z‘ti’fxw. yfiv'wm ‘- w . v. 5}“. ‘°' * —., firmg‘ ~' ' x 1'?” V J" " -3“ :2: . _ 7“": . : u u. ‘ gazgmw-t-sé w‘ .‘tfl . 1 3.1.534 w wz mugfibv’; ‘ “E92; 43‘ ,1" ~11 V “S A u {1‘ g; ,- ‘ fig» -. ;- , , u“ an I of u u; s .7 ., A 1‘ ur‘i\fl‘ .971? u ‘ v This is to certify that the dissertation entitled THE COMBINED EFFECTS OF SCHOOL ENVIRONMENT, PRESCHOOL EXPERIENCE, AND THE HEAD START/ EARLY CHILDHOOD TRANSITION PROGRAM UPON THE ACADEMIC ACHIEVEMENT OF YOUNG SCHOOL CHILDREN presented by Jeanette M. Gassaway has been accepted towards fulfillment of the requirements for PLD - degree in W Vh‘w Major professor ' Z/é/fa/ MSU is an Affirmative Action/Equal Opportunity Imtitution 0- 12771 llllllllllllllllllllllllllllIllIlllllllllllllllllllllll 3 1293 01027 6131 LIBRARY Michigan State University PLACE ll RETURN BOX to roman this checkout from your record. TO AVOID FINES mum on or baton date duo. DATE DUE DATE DUE DATE DUE W__—|l l I l ________l___l _fiT—T MSU Is An Afflnnatm WIN-l Opportunity Instituion m pas-9.1 THE COMBINED EFFECTS OF SCHOOL ENVIRONMENT, PRESCHOOL EXPERIENCE, AND THE HEAD START/ EARLY CHILDHOOD TRAN SITION PROGRAM UPON THE ACADEMIC ACHIEVEMENT OF YOUNG SCHOOL CHILDREN By Jeanette M. Gassaway A DISSERTATION Submitted to Michigan State University in partial fulfillment of the requirements for the degree of DOCTOR OF PHILOSOPHY Department of Psychology 1994 ABSTRACT THE COMBINED EFFECTS OF SCHOOL ENVIRONMENT, PRESCHOOL EXPERIENCE, AND THE HEAD START/ EARLY CHILDHOOD TRANSITION PROGRAM UPON THE ACADEMIC ACHIEVEMENT OF YOUNG SCHOOL CHILDREN By Jeanette M. Gassaway This research examined the combined effects of school environment, preschool experience, and the Head Start/Early Childhood Transition Program upon the academic achievement of young children at the end of kindergarten. A matched control study design was utilized where six elementary school in a small Michigan school district were matched on demographics forming two clusters of schools. These clusters of three schools were randomly assigned to either experimental or control group (Head Start/Early Childhood Transition Program participation or no participation). Two hundred and seventy one children who only had posttest scores on a standardized, school readiness test were included in the sample. A subsample of 204 children who had pre and posttest achievement scores were used for pre/posttest analyses only. Fifty six teachers and teaching support staff in these six school completed questionnaires which assessed teachers' perception of school environment. The data were analyzed through hierarchical linear model procedures. The results indicated that there were significant overall differences in children' academic scores between schools. Children were learning differently based on the school in which they attended. Surprising, children who attended transition program schools scored significantly lower on reading achievement tests than did children who did not attend program schools. Not surprisingly, children who had preschool experience had significantly higher math and reading scores than children with not preschool experience. Lastly, there was a near significant interaction between transition program and school environment for reading achievement. The findings of this study underscore the importance of looking at processes within schools which have an impact on the academic achievement of children, especially teacher variables which can be vital for children's achievement. It is clear that children learn differently depending upon the school in which they attend. In addition, it reveals the importance of preschool experiences for children. These experiences provide children with skills which enhance their academic achievement, at least during kindergarten. O praise the Lord, all ye nationals; praise Him, all ye people For His merciful kindness is great towards us.... Praise ye the Lord Psm. 117: 1-2 ACKNOWLEDGEMENTS It has often been hard to image that I would be writing this statement of acknowledgement. However, the day has come and I am more than grateful for all of the support, encouragement, and assistance that I have received from committee members, family, and friends. I must first acknowledge the support and assistance that I have received from my committee chairperson, Thomas Reischl, without whom this research would not have existed. I am grateful to William Davidson, Thomas Luster, and Stephen Raudenbush, the other members of my committee who provided needed guidance. To all who prayed for me, wished me well, encouraged me when I thought I would not succeed, I say "Thank You." But most of all, I am forever grateful to the One who answered the prayers, the Only Wise God, my Savior, Jesus Christ. I owe Him all... TABLE OF CONTENTS List of Tables Introduction Problems of Inadequate Education Brief Historical Overview of Head Start and The School Effectiveness Movement Literature Early Childhood Education Programs Preschool Programs Head Start Perry Preschool Project The Consortium for Longitudinal Studies Summary of Preschool Programs Other Programs Project Follow Through Summary of Early Childhood Education Literature School Effectiveness Overview Identifying Effective Schools & Effective School Correlates Case Studies Observational Studies Correlating School Effectiveness with Academic Achievement Summary Implementation of Effective School Research Summary of School Effectiveness Literature Summary: Integration of Head Start and School Effectiveness Present Study Hypotheses Program Description Head Start/Early Childhood Transition Program Health Social Services Education 11 ll 11 11 15 16 l7 17 18 20 21 22 25 25 26 29 34 35 39 4O 41 41 43 45 Head Start Education Methods Design Matching Schools Random Assignment Sample Children's Sample Teacher and Support Staff Sample Data Collection Procedures Children Children's Instrument Early Childhood Developing Skills Checklist Classroom Teachers, Other Support Staff and Principals Teachers' Instrument Connecticut School Effectiveness Questionnaire Measure of Program Implementation Classroom Observation Results Preliminary Analysis Hypothesis One: Program Main Effect Hypothesis Two: School Environment Effect Hypothesis Three: Preschool Main Effect Hypothesis Four: Transition Program and Preschool Interaction Hypothesis Five: Preschool and School Environment Interaction Hypothesis Six: Program and Environment Interaction Results Summary Discussion Research Limitations Conclusions Appendices Appendix A Appendix B Appendix C List of References iv 46 46 48 48 48 49 49 49 50 53 53 53 53 55 56 56 59 59 67 67 76 82 82 96 99 102 105 106 116 118 120 121 122 131 Table 10 11 12 LIST OF TABLES Sample Sizes Across Schools Subsample Demographics for Post Only and Pre and Posttest Groups Alphas for Connecticut School Effectiveness Questionnaire (N=56) Grand Mean and Standard Deviation and School Mean Scores and Standard Deviations for School Environment Scale ANOVA Model for Math Achievement (N=271) ANOVA Model for Reading Achievement (N=271) Grand Mean and Standard Deviation and School Mean Scores and Standard Deviations for Math and Reading Achievement ANCOVA Model Controlling for Pretest for Math Achievement (N=204) ANCOVA Model Controlling for Pretest for Reading Achievement School Pretest and Posttest Means and Standard Deviations for Math and Reading Achievement Program Main Effect for Math Achievement (N=204) Program and Comparison Means and Standard Deviations Page 51 52 60 63 68 69 71 73 74 75 77 78 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 Main Program Effect for Reading Achievement Scale Scores, Means, and Standard Deviations for the Assessment Profile for Early Childhood Education Programs School Environment Main Effect for Math Achievement (N=204) School Environment Main Effect for Reading Achievement (N =204) Preschool Main Effect for Math Achievement (N=271) Preschool Main Effect for Reading Achievement (N=271) Preschool Main Effect for Math Achievement (N=271) Preschool Main Effect for Reading Achievement (N=271) Preschool Mean Scores and Standard Deviations for Math and Reading Achievement School by Preschool Means and Standard Deviations for Reading Achievement (N =204) School by Preschool Means and Standard Deviations for Math Achievement (N=204) School by Preschool Means and Standard Deviations for Reading Achievement and Math Achievement (Posttest only Scores) (N=27l) Program and Preschool Interaction Effect for Math Achievement (N=27 1) Preschool and Program Interaction for Reading Achievement (N=27l) Preschool and School Environment Interaction for Math Achievement (N =27 1) School Environment and Program Interaction for Reading Achievement vi 79 81 83 84 86 87 88 9O 91 92 93 24 97 98 100 101 29 Program and School Environment Interaction for Math 104 Achievement (N=27l) vii INTRODUCTION Academic achievement of children is a complex issue, one which can be conceptualized by relationships among many variables within and across different levels of children’s educational experiences. For example, academic achievement may be influenced by home environment, socioeconomic status, race or ethnicity, parental involvement, student aptitude, student motivation, teacher expectations, preschool experiences, and school environment, just to name a few. It becomes clear as one begins for formulate hypotheses regarding the development and promotion of academic achievement that an ecological approach (Bronfrenbrenner, 197 9) should be applied to fully understand its complexity. This approach should represent a multi-dimensional, multi-level perspective and provides a critical analysis of the combined effects of the variables known to contribute to the development of academic achievement among children. This research attempts to examine the combined effects of an early childhood education program (Head/Start Early Childhood Transition Program), preschool experience, and school environment upon the academic achievement of primarily low-income and African American children. The need for comprehensive examination of academic achievement is reflected in the educational status of many of the children in the United States, especially poor, minority, inner city children who attend public schools. For these children, education . is often inadequate, inferior, and ineffective (Comer & Haynes, 1990; Joint Committee for Policy Studies, 1989; Levine & Havighurst, 1989; Texas Advisory Committee, 1990; Weinberg, 1990) and their academic achievement has lagged far behind that of their non-minority peers. Attempts to ameliorate the educational disparity have, for the most part, short-lived effects and have failed to change school environments to facilitate long-term achievement gains. The failure to produce lasting systemic changes in schools and children's achievement may be the result of the failure of scholars and educators to build and use multi-level system models. At one level, compensatory education programs and other programs such as Head Start, Follow Through, and other remedial programs have been designed to enhance the academic success of children who are at risk for school failure. Children who participate in such programs may, for a period of time, benefit but the effects wash out as they go on through their educational experience. Though there are some children for whom there are lasting educational benefits, there is still a large proportion of children who remain handicapped in their educational achievements. This points to the need for change at a different level, change in the educational systems (the public schools) children are entering upon completion of early childhood programs. When there is change at the school level, all children will benefit from improved instruction and enhanced environment conducive to learning. One attempt to address this problem is the Head Start/Early Childhood Transition Program which seeks to provide continued support to children beyond preschool as they enter the public school system. This is an attempt to address the short-lived achievement gains and to provide continuity between preschool and school experiences, seeking changes both in the children’s academic achievement and in the classroom practices and curriculum as well. Though these programs provide promise, it seems likely that schools will have to provide an academic environment which is conducive to promoting and enhancing academic achievement for all of its students. Therefore, despite the efforts of preschool, early childhood educational programs, compensatory educational programs and the like, children will continue to fail at large proportions when schools provide inadequate and ineffective education and do not provide environments conducive to enhancing academic achievement. The "Effective Schools Movement" model is another attempt to promote socio- cultural changes within schools and seeks changes that provide for better administrative leadership, higher teacher expectations, safe and orderly environment, and more (Brookover, et aL, 1983; Edrnonds, 1979; Goodlad, 1984; Lezotte, 1980, Rutter et al., 197 9). The research surrounding effective schools suggests that the school social-cultural and academic environment can have a impact on the academic achievement of children. These issues raise several questions. Are early childhood education programs enough to support academic achievement in children? Will school environment support the gains made by children who have attended early childhood education programs or will they negate those gains? How does the school environment mediate programmatic effects upon academic achievement? The research described here seeks to examine the combined effects of early childhood education programs and school environment (based on the school effectiveness concept) upon the academic achievement of children participating in the Head Start/Early Childhood Education Program. Presented in the following section is a discussion of the problems related to inadequate education for poor, minority children, focusing on African American children, a literature review of Head Start and similar early childhood education programs in addition to a review of the school effectiveness literature. Problems of Inadequate Education Historically, education has been ascribed great power by the citizens of the United States who have exhibited a relentless faith that education will move social and cultural mountains and dissolve political and economic barriers to the "American Dream" (Jeffrey, 197 8). This faith dictates that education will lead to upward mobility and economic success, eradicate racial and social inequality and injustices (Jeffrey, 1978; Joint Center for Political Studies, 1989) and be the key to the elimination of poverty (Goodlad, 1984; Jeffrey, 197 8). For many, especially poor, minority, inner- city children, however, education does not provide the opportunities it promises. Public schools have systematically failed to adequately educate these children (Jeffrey, 1978: Joint Committee for Policy Studies, 1989; Levine, 1991; Levine & Havinghurst, 1989; Texas Advisory Committee, 1990) for whom it has provided second class schooling (Joint Committee for Policy Studies, 1989). Unfortunately, the consequences of ineffective education, on both an individual and societal level, are dire (Levine & Havinghurst, 1989; Slavin, 1989). Without intervention, the result may be a permanent underclass "doomed to poverty" (Bain & Herman, 1990) and a polarized, dual society, a condition which may create political conflict and social upheaval (Baptiste, 1992; Levine, 1990; Slavin, 1989). Despite the accomplishments and gains made over the more recent decades (Slavin, Karweit, & Madden, 1989), poor and minority children are yet the prime burden bearers of an inadequate system School failure for these students have reached epidemic proportions (Joint Center for Political Studies, 1989). Poverty and racial-ethnic background are closely associated with academic failure. Poverty is one of the strongest predictors of academic failure and a significant portion of minority citizens are poor (Graham, 1987; Ramey, et al., 1987). African American children are three times more likely to be born into a poor family than white children (Graham, 1987). National surveys reveal that the differences in academic achievement among African American and white children appear early in the elementary school years and continue throughout the elementary and secondary school years (Jacob, 1989; Stevenson, Chen, and Uttal, 1990). African American children’s plight within the educational system is dismal as well. African American students are much more likely than white children to be placed in classes for the educable mentally retarded and only one half as likely to be in classes for the gifted and talented (Joint Center for Policy Studies, 1989; Levine & Havinghurst, 1989; Chunn, 1989). In high schools, African American students are suspended about three times more often than white and are greatly overrepresented in vocational tracks and underrepresented in academic programs (Joint Center for Political Studies, 1989). The dropout rate for poor and minority students is high. Poor students are 3 to 4 times more likely to drop out of high school than those from more affluent families (Joint Center for Policy Studies, 1989). African American students graduate from high school at rates well below those of white students (Jacob, 1989; Slavin, et al., 1989). Dropout rates in some cities are close to or above 50 percent (Graham. 1987;Slavin, et al., 1989). Students who fail in school are likely to become adults who live in poverty (Schweinhart & Weikart, 1988), and whose unemployment rates will be high. According to the US. Bureau of Census, the poverty rate for high school dropouts in 1986 was 19%, while it was only 10% for high school graduates and 4% for those who attended college (Schweinhart & Weikart, 1988). Enrollment in higher education is not only extremely low for African American students, but declining as well (Graham, 1987; Jacob, 1989). The rate of enrollment for African Americans in 4 and 2 year degree schools and graduate school are considerably lower for African Americans than for whites (Jacobs, 1989; Summary Report, 1992). Students attending inner-city schools represent the most imperiled group of the increasing numbers of students at risk of failure. Life of students in inner-city has deteriorated in recent years and is characterized by social isolation (Levine & Havinghurst, 1989). This isolation, resulting from the flight of more affluent families to the subtu'bs or other areas, has lead to school districts becoming overwhelmingly populated with working or lower class minority students (W axman, 1992). This isolation prevents interaction with individuals and institutions that represent mainstream society, and therefore, creates fruther structural constraints, limits opportunities for social mobility and provides conditions for underachievement, student and teacher alienation and high dropout rates (Baptiste, 1992; Waxman, 1992). According to Slavin, et al (1989), the crisis of inadequate education in the United States will lead to serious consequences. These consequences include: a) the emergence of a dual society with a large and poorly educated underclass, b) massive disruption in higher education, c) reduced economic competitiveness of the nation as well as states and industries most heavily impacted by these populations, and (1) higher costs for public services that are response to poverty (Levine, 1990; Slavin, 1989). As schools continue to flood this society with students who are ill-prepared, the numbers of the "disadvantaged," which include a disproportionate number of minority persons, will continue to escalate (Levine & Havinghurst, 1989). In a high technological society, where more sophisticated skills are required, the "disadvantaged" will be faced with high unemployment rates, low earnings, and menial occupations (Levine, 1990). From this seed will grow a dual society, the more afl'luent, largely non-minority citizens, and the lower and working class who are substantially minority (Levine & Havinghurst, 1989). Some predict that the political power of persons who are from poor and minority backgrounds will increase as its numbers grow and be a basis for political conflict and possibly social upheaval (Slavin, 1989). Awareness of the consequences of inadequate education has been the motivation for the creation of early childhood educational programs and the school reform initiatives. The following sections will provide a brief historical overview and review the literature for two types of interventions, early childhood educational programs and the effective schools movement. Brief Historical Overview of Head Start and The Effective School’s Movement Both the school effectiveness movement and Head Start grew out of President Johnson’s "War on Poverty,” the initiative designed to "cure" poverty in the United States (Washington & Oyemade, 1987; Zigler & Valentine, 1979). To eliminate poverty, sweeping social change was necessary, and education became a central focus of the change efforts. Education was viewed an accessible, non-threatening avenue for change (Jeffrey, 1978; Madaus ct al., 1980; Zigler & Valentine, 1979). The immediate objectives were to document the existing inequities in education, inequities that Congress believed to be a primary cause of disparities existing between groups in society, and plan and implement programs which would ameliorate the problems associated with educational equities (i.e. poverty) (Madaus, et al., 1980). The Commissioner of Education was asked to conduct a nationwide survey. The Effective Schools Movement began as a reaction to the final report entitled "Equalities of Educational Opportunities Survey," commonly known as the Coleman Report. The report contradicted the assumptions made by the Johnson Administration that inequities existed in schools resources, and gave support to the belief that schools had little or no impact on students’ academic achievement. The authors concluded that family background (socioeconomic status, race/ethnicity) were key to academic achievement. From the time the report was made public until today, social scientists, educators and others have conducted research to refute and disprove the findings of the Coleman Report. Their research and subsequent implementation of the findings was known as the "Effective Schools Movement." Before the survey was completed, however, the Johnson Administration formulated and implemented a demonstration project for young children, Project Head Start (Madaus, et al., 1980; Zigler & Valentine, 1979; Zigler & Styfco, 1993). Project Head Start was designed to be a comprehensive preschool program in which children of low-income families would be prepared for future academic success by strengthening the child’s cognitive and social capabilities, and additionally, to ensure children were receiving adequate medical and dental care (Madaus, et al., 1980; Zigler & Valentine, 1979; Zigler & Muenchow, 1993). This program would provide health, education, and social services to low-income families (Washington & Oyemade, 1987). In addition, parents were viewed as important partners and participants in the education and development of their children. Head Start provided services for parents including parent education and employment (Zigler & Valentine, 1979; Washington & Oyemde, 1989). Head Start began during the summer of 1965 as a 6 to 8 week summer demonstration project Today Head Start represents a stable institution in American society (Zigler & Valentine, 197 9). It has served over 11 million children and provided extensive involvement opportunities (including employment opportunities) for parents and families. Over the years, it has changed to meet the needs of its 10 recipients. Today, in addition to summer programs, it includes full-year programs. Head Start is currently seeking to augment its scope by providing continued support for children who attend Head Start by offering these same services to children and their families until children reach the third grade (Zigler & Meunchow, 1993). In January, 1991 the Oflice of Adminisuation, Youth and Families announced a new demonstration project, The Head Start/Public Schools Early Childhood Transition Project. It is hypothesized that with continued education, health and social services, children will continue to be successful at higher levels of education. LITERATURE REVIEW Early Childhood Education Programs There is considerable evidence to suggest that early childhood education programs such as Head Start, Perry Preschool Project, and others have positive impact on the academic achievement (cognitive development) of children. Several studies suggest these program have immediate effects as children enter into the school setting and a few studies suggest there are long-term benefits as well (Condry, 1983; Datta, 1979; Lee, Brooks-Gunn, Schnur & Liaw, 1990; Washington & Oyemade, 1987). The following will describe a few of the research studies of early childhood education programs. Preschool Prog:ar_ns Head Start Volumes of studies exist representing the effects of Head Start (Lee, Brooks- Gunn, Schnur & Liaw, 1990). Originally, national evaluations were conducted by Head Start as well as many other independent studies (Gordon, 197 9). Some of these studies included those conducted by the Westinghouse Learning Corporation, 1969, Kirschner Associates, 1970, the Office of Economic Opportunities, 1969, the General Accounting Office, 1979, Bronfenbrenner, 1975, and Ryan, 1974 (Washington & Oyemade, 1987). The original attempts at national evaluations were fraught with problems including too many uncontrolled variables, inadequate measurement tools, 11 12 and a lack of true experimental design, just to name a few (Gordon, 1979; Condry, 1983; Cole & Washington, 1986). Head Start’s commitment to program flexibility in order to meet the needs of local communities and its failure to impose or prioritized goals produced programs throughout the country of varying quality, design, and focus (Cole & Washington, 1986; Lee, et al., 1990). Despite these difficulties, early findings suggested that Head Start had immediate effects and possible long-term benefits for children who are poor (Datta, 1979). They showed that Head Start children, upon entering school were close to or at national norms on measure of school readiness, and these gains were maintained during the first year of school. Unfortunately, however, the gains "faded" in grades two and three (Cole & Washington, 1986: Condry, 1983). The "Westinghouse Study" supported the findings of these previous studies. Westinghouse Learning Corporation and Ohio University evaluated Head Start to determine the long-term effects of Head Start by comparing Head Start children with non-Head Start children on standardized tests in grades 1, 2 and 3 (Condry, 1983: Datta, 1979; Washington & Oyemade, 1987). Using a posttest-only experimental and control design, they evaluated 104 Head Start centers. (Seventy percent of the programs were summer-only programs and the remainder were full-year.) Results indicated that the summer program had no lasting impact and the full-year program was only marginally effective. Head Start stimulated cognitive and language gains which lasted through the 1st grade but comparison children caught up by 2nd or 3rd grade (Condry, 1983; Datta, 1979). From these results the Westinghouse study l3 concluded that Head Start was not successful (Washington & Oyemade, 1987). The Westinghouse study, which jeopardized the very existence of Head Start, was severely criticized for lacking a representative sample of the Head Start population, using a post-only design, inequivalent comparison groups. In addition, it was criticized for its failure to examine the effect of different curricula (Cole & Washington, 1986; Zigler & Styfco, 1993). Though the Westinghouse study was severely criticized for its blatant condemnation of Head Start, other subsequent studies had similar findings. Some of these studies are reviewed below. Reanalysis by Smith and Bissell, 1970 (Datta, 1979) of the Westinghouse data confirmed that Head Start effects, at least for children tested in the first grade, were greater for children who apparently needed the program most: urban children, children from Southeastern states, and minority children (Datta, 197 9). The Head Start Synthesis Project was a meta-analysis of all available Head Start studies conducted by CSR, Inc. (Haskins, 1989; Schweinhart, Barnes & Weikart, 1993; Washington & Oyemade, 1987 ). The results showed strong evidence that Head Start produced immediate effects on intellectual performance (Haskins, 197 9; Schweinhart, et al., 1993; Washington & Oyemade, 1988). However, as in the Westinghouse study, the academic or cognitive gains "faded out" after the first grade. In addition to cognitive gains, there were gains in socio-emotional variables including self-esteem, achievement motivation and social behavior as well as in child health, motor development, nutrition and dental care. There was some evidence to suggest that there were long-term school achievement outcomes. In a few studies, children were less l4 likely to be retained in grade or assigned to special education classes. There were several criticisms of the CRS study. First, it was criticized for overgeneralizing its finding (Washington & Oyemade, 1987). Its sample was not representative of the variety of Head Start programs. Secondly, the report did not distinguish between high and low quality design studies. They were also criticized for not including other early childhood educational programs in their sample (Schweinhart, et al., 1993). Lee, Brooks-Gum, Schnur, & Liaw (1990) examined previously collected data in the ETS Head Start Longitudinal Study. In that study children in Head Start centers in two cities in 1969-1970 were compared to two comparison groups, children not in preschool that year and children in non-Head Start preschools for children of lower socioeconomic backgrounds. The inquiry was resuicted to the African Americans in the sample. The results indicated that Head Start children scored higher on measures of cognitive and social competence dimensions immediately after the Head Start experience. These gains were maintained through the first grade but dissipated thereafter. The authors presented several hypotheses regarding the results. The decrease of academic gains over time for Head Start children might be due to the differential treatment that poor and minority children receive in school. These children might often have poor school experiences. They recommended widespread expansion of stimulating educational experiences targeted at young children in the elementary years. 15 Pg! Preschool Project The Perry Preschool Project was designed to determine whether a cognitively oriented preschool program could help children of low-income status to be more successful in school (Schweinhart 8r. Weikart, 1988). Cross-sectional data was collected as well as longitudinal data at various periods during the children’s lifetime including until the age of 27 (Schweinhart, et al., 1993). One hundred twenty three African American children, most of whom attended the program for 2 years at ages 3 and 4 participated in this study. Three and 4 year olds from families of low socioeconomic status were randomly assigned either to the Perry Preschool Program or to a control group of children who did not attend. Pairs of children were matched on initial IQ and then randomly split between the groups. Results indicated that participants scored significantly higher on various test of intellectual ability after one year of the Perry Preschool Program and maintained a significant advantage through the end of the first grade. The IQ difference between experimental and control groups diminished over time, however, and by second grade, was no longer statistically significant. Early education led to increased academic achievement as measured by standardized tests throughout the elementary and middle- school grades. In high school, the preschool group had better grades and fewer failing grades. They were less likely to be classified as mentally retarded and spent fewer years in special education during their years in school. At age 19, those who had attended preschool had higher measures of literacy and competency in skills of everyday life. 16 They also expressed more favorable attitudes towards school. By age 27, the program group had completed a significantly higher level of schooling than the no-program group. In addition, the program group had a significantly higher rate of regular high school graduation and a nearly significantly higher regular high school graduation rate or the equivalent (Schweinhart, 1993). The Consortium for longitudinal Studies Several of the best early education programs were organized into a consortium by Irving Lazar during the mid 1970’s to obtain longitudinal data on each projects’ original sample in order to assess the long-term effects of pre-school interventions (Haskins, 1989; Schweinhart, et al., 1993; Washington & Omeyade, 1987). Eleven early education projects, originated between 1962 and 1972, were part of this Consortium. Of the 3,593 children who had originally participated in the 11 projects, 2,008 (56%) were located and submitted to at least part of the 1976 follow-up data collection. The participants were predominantly poor and predominately African American children (Palmer & Andersen, 1979). Results revealed that children who participated in the early education programs had higher achievement and IQ scores, however, these effects would fade after some years. In addition, children who participated in the program were less likely to be placed in special education, and less likely to be retained in grade (Zigler & Styfco, 1993). The more long-term studies revealed that the program participants were more likely to graduate from high school as well. l7 Sm: Preschool Programs The studies reviewed above suggested that Head Start had immediate impact upon children’s academic achievement or cognitive development. These immediate gains, however, would decline shortly after children enter school, usually about the first grade. There was also modest support for long-term benefits of Head Start. Children were less likely to be retained in grade and less likely to be placed in special education. Other programs such as the Perry Preschool program provided additional evidence that preschools programs enhance children’s academic achievement, however, as with Head Start, these gains dissipate over time. In addition, Perry program provided stronger evidence that preschools can have long-term effects as well including better grades in school, higher literacy rates when in adulthood, higher graduation rates, and fewer placement in special education. W Schweinhart, et al. (1993) reviewed 11 long-term studies which provided evidence that early childhood educational programs could have beneficial, long-term effects on children attending. There were four "intensive" long-term studies which included evaluations of the Abecedarian Project and Project Care conducted by Ramey, Bryant, Campbell, Sparling, & Wasik, 1988, the Early Training Project by Gray, Ramsey, & Klaus, 1982, the Harlem study by Palmer, 1983, the Milwaukee Project by Garber, 1988, Mother-Child Home Program by Levenstein, O’Hara, & Madden, 1983, the Rome (Georgia) Head Start Program by Monroe & McDonald, 1981 and the Syracuse University Family Development Research program. The four 18 "extensive" studies included the Chicago Child-Parent Center Program conducted by Fuerst & Fuerst, 1993, Project HOPE, Home-Oriented Preschool Education by Gotts, 1989, the Houston Parent-Child Development Center by Johnson, 1988, and the New York State Prekindergarten Program by Irvine, 1982. All of the programs served young children living in poverty who were at special risk of school failure. Children entered the program between birth and 5 years of age and remained for, at least, one year. Most remained until they entered school. Some followed children only through early childhood, others followed children through adulthood. Sample sizes ranged from 90 to over 2,000. Each study collected information on intellectual performance. The results indicated that for each study, program participants had significantly higher intellectual performance scores than children in the no-program group. As with Head Start, the children attending the programs were significantly less likely to be placed in special education, less likely to be retained in grade and had a higher percentage of children who graduated from high school. Of these projects, there were a few which, according Zigler & Styfco (1993), offered convincing evidence that supported the premise of the Head Start/Public Schools Early Childhood Transition Project. The Abecedarian Project extended services and provided a school-age program for children through second grade. These children continued to do better than those in comparison groups. Their peers who did not receive continuing services yet participated in the early intervention no longer had a statistically significant advantage over comparison groups, and those who received 19 only the school-age program did not benefit substantially. The Chicago Child-Parent Centers (Zigler & Styfco, 1993) provided comprehensive services and required parent involvement from preschool until the third grade. Though children who attended the preschool and kindergarten did not differ from controls by the time they were in fifth grade, fifth graders who had received 4 to 5 years of the intervention had higher achievement scores and less grade retention. In addition, children who attended 4 to 6 years or more had better high school graduation rates. Project Follow Through Particular note should be made of Project Follow Through since its goals mirror those of the Head Start/Public Schools Early Childhood Transition Project. It was implemented because the evaluation of Head Start indicated that gains from early childhood programs were not maintained after kindergarten and first grade (Levine & Havinghurst, 1989). It was designed to examine whether a continuing program would produce more sustained effects (Lee, et al., 1990) in addition to making school more effective for low-income children (Hodges, Branden, Feldman, Follins, Love, Sheehan, Lumbley, Osborn, Rentfrow, Houston, & Lee, 1980). The intent was to maintain those gains by an extension of educational intervention and comprehensive services through the third grade (Hodges, et al., 1980). It was organized on a national "planned variation" basis (Levine & Havinghurst, 1989). Participants could choose from 22 educational approaches (Hodges & Buzzelli, 1984). All approaches were supplemented by comprehensive services (medical, dental, psychological, and social) 20 as well as with parent involvement (Hodges & Buzzelli, 1984). The program’s span was from kindergarten until children reached the third grade. Many of the projects were well-designed field studies using pre and posttest, experimental and control group designs with random assignment of subjects. Children from regular and experimental Head Start classes were compared to children who did not enter Head Start or Follow Through (Datta, 1979). The data were collected over a six year period. The results of the national Follow Through Planned Variation evaluation revealed that only a small number of approaches showed sufficiently large numbers of positive effects (Hodges & Buzzelli, 1984). Head Start children who continued in Follow Through had steadier and higher performance than those who did not continue. Problems with me Follow Through Planned Variations project included lack of randomization, confounding of "treatment" or "no treatment" with school building (principals, teachers, community support) (Datta, 1979). Some major problems for the evaluation included comparing students with different socioeconomic status (severely versus moderately disadvantaged), high attrition of students, teachers, and sites, complicated technical issues involving comparison of gain scores at differing levels of performance, and uncertainty whether a given model at a particular site was implemented well enough to be considered a good test of its potential effects (Levine & Havinghmst, 1989). Summary of Early Childhood Education Literature The research presented above clearly provided evidence that Head Start and other model programs could have a significant impact upon the intellectual performance. 21 However, these cognitive, academic gains declined within a few years. On other variables of school performance such as special education placement and grade retention, there was strong evidence of positive effects for model programs and modest evidence of effects for Head Start programs. In addition, there was some evidence to support the benefit of continued academic support for children beyond preschool years. After examining the effects of Head Start, several researchers suggested that the decline in academic scores among Head Start children might be attributed to the quality of education they received upon entering elementary school (Lee, et al., 1990). The school effectiveness research could be instrumental in providing insight into the nature and quality of schools in which these children enter. It examined schools to determine if schools were effective in enhancing children’s academic achievement, compared effective schools to ineffective schools to determine what characteristics differentiated the two, and then provided a framework from which schools could develop an improvement plan. Once characteristics were identified, many school district implemented school improvement plans. The following review of the School Efl'ectiveness Literature provided some insight into what changes were needed within the schools so they could be more supportive of children’s academic achievement. School Effectiveness The school effectiveness movement began as an attempt to prove that schools did impact student achievement, especially for students from mostly lower socioeconomic and minority backgrounds (Crisci, 1986; Schmidtt, 1989). The effective school literature, therefore, suggested that structural and programmatic changes in the school 22 could produce an environment that lead to academic improvement for students from lower-income backgrounds (Purkey & Smith, 1985). Unfortunately, there was no consensus on the definition of an efiective school (Frederick, 1987), and the research was characterized by a variety of designs, methods, and measmes of effectiveness making comparisons difficult. It was criticized for design flaws including measures used, sample sizes, and unit of analysis. In addition, it was criticized for overreliance on surveys and ignoring qualitative data collection and analysis. Despite its shortcomings, the effective schools research suggested that schools could make a difference in how much and how well children learn and contributed to the understanding of what school practices promoted high academic achievement (Brookover and lezotte, 1977; Crisci, et al., 1986; Edmonds, 1979; Good and Brophy, 1986). Overview In 1964 Congress commissioned a study as a part of the "War on Poverty" campaign to explore the disparities between schools, especially between schools which were racially different (Madaus, et al., 1980). It was hypothesized that there would be disparities in school resources which would account for the disparities in academic achievement. The Coleman Report was the end-product of a nationwide survey conducted between 1964 and 1965 of 645,000 pupils who were categorized into six racial and cultural groups, and of 60,000 teachers from 4,000 schools (Madaus, et al., 1980). Students in grades 1, 3, 6, 9, and 12 were assessed on their ability and achievement by standardized achievement tests while teachers were assessed regarding 23 their educational background, experiences, and attitudes. In addition, information was gathered on many school characteristics from average teacher salaries to age of text books. The premise of their research was that teacher and school characteristics would be a decisive influence on the academic achievement of children. Their results revealed quite the opposite. According to the "Equality of Educational Opportunity" survey (Madaus, et al., 1980), though schools were, for the most part, racially segregated. and there were sizeable differences between white students and other racial or ethnic minorities, there were little differences between students’ level of achievement in relationship to school variables (Maduas, 1980; Squires, 1980). The study concluded that home and family background characteristics were more important to academic achievement than schools. Schools had virtually no impact on students’ academic achievement as the following statement reported: Schools bring little influence to bear on a child’s achievement that is independent of his background and general social context...this very lack of an independent effect means that the inequalities imposed on children by their home, neighborhood and peer environment are carried along to become the inequalities with which they confront adult life at the end of school. For equality of p educational opportunity must imply a strong effect on schools that is independent of the child’s immediate social environment, and that strong dependent effect is not present in American schools (Madaus, p. 30, quote fi'om Coleman report). The Coleman Report (Maduas, et al., 1980) was criticized for several reasons. It was criticized for using inappropriate independent variables. They studied static variables such as teacher-student ratio, number of library books, circulation of library books, number of microscopes per biology lab, etc. (Schmitt, 1989), instead of more ’ social and socio-cultural variables. In addition, they were criticized for using 24 inappropriate analyses, multicollinearity among variables, the lack of a true experimental design, and the response rate. A flurry of research began to appear which would conuadict the Coleman report. Crisci (1986) suggeswd this rapid response was not an effort "so much to disprove the general connection between economic class and student achievement as to prevent educators from absolving themselves of all responsibility for student achievement on the grounds that socioeconomic status infallibly predestined levels of achievement" (p.19). The effective schools literature documented what schools did which made a difference in student achievement; and instead of examining static resource variables, it focused on the social and social psychological factors of the school environment (Brookover & Schnieder, 1975). The research reviewed below appeared to move through several stages during its historical development. First, the research was characterized by the search for effective schools, especially schools within poor, urban communities with a large minority population. Once schools were identified, observations were made within the schools to determined the factors which promoted effectiveness. Effectiveness was usually defined by academic achievement, and for the most part, scores on standardized academic achievement tests were compared to the district average or the average achievement score for schools with similar student populations and demographics. The next stage included research which examined the relationship between the characteristics of school effectiveness and academic achievement in addition to identifying effective schools and their correlates. Lastly, the research from 25 previous stages were used to implement school improvement programs. This literature review reflected these stages of the school effectiveness research. Id m ' g Effective Schools & Effective School Correlates Studies One of the first school effectiveness studies was conducted by Weber in 1971 (Crisci, et al., 1986; Edmonds, 1979; Good and Brophy, 1986). Weber observed 17 schools in 7 large cities and identified 4 schools where children performed well on reading achievement tests. The results of this case study revealed there were nine characteristics of these four schools which made them effective. These characteristics included strong leadership, high expectations, a good atmosphere, emphasis on reading, additional reading personnel, use of phonics, individualization, and careful evaluation of pupil progress. Other case studies were conducted by Gordon (1984), the New York State Office of Education Performance Review (1974), and Venezky & Winfield (197 9). Through interviews, observations and surveys, these researchers gathered data to determine characteristics of effective schools. The findings indicated that strong leadership was central to effective schools. Effective schools were characterized by instructional leadership, emphasis on reading achievement, and high expectations by administrative leadership and teachers as well. In addition, the schools had a plan for dealing with reading problems and implemented the plan throughout the school. In the New York State study, the findings revealed that in less effective schools many of the professional personnel attributed children’s reading problems to outside factors. These 26 school personnel were pessimistic about their own ability to have an impact. This pessimism, in true, created an environment in which children failed because they were not expected to succeed. Observational Studies Klitgaard and Hall conducted a study in 197 4 (Good and Brophy, 1986) to determine if school effectiveness could be assessed by regression residuals. They operationally defined effectiveness as student performance on standardized reading and mathematics achievement tests by examining residuals from a regression of achievement scores on background factors. They studied a series of residual distributions. If schools were one standard deviation above the mean more often than chance would predict, they would be considered efiective. They examined data from Michigan, New York City, New York state and Project Talent schools from 1970 through 1971, 1967 through 1970, and 1960 respectively. Students were in the 2nd through 5th grades and in high school as well. They also examined data for unusually effective school districts (Good & Brophy, 1984). The study revealed that there were unusually effective schools in the sample as well as unusually effective school districts. The study did not assess the characteristics of effective schools. Madden, Lawson, Sweet (197 6) conducted a study of school effectiveness in California. Madden, et a1. (1976) studied 21 pairs of California elementary schools, matched on pupil characteristics and differing only on standardized achievement measures. The purpose of the study was to identify those school characteristics that seemed most responsible for the achievement differences between the low and high- 27 achieving schools. The results revealed that when compared to the low-achieving schools the high-achieving schools were characterized by greater principal support, greater teacher effort, a more orderly and task oriented classroom atmosphere, more time spent on various subjects like social studies, limited and non-instructional use of teacher aides, greater access to materials, limited faculty participation on overall instructional policy, more support services from district administration, and greater teacher job satisfaction (Edmonds, 1979, p. 29). Brookover and Lezotte (1977) conducted a study in which they examined schools to determined if there were changes in students academic achievement over time and to determine whether achievement was improving or declining. Having classified schools as "improving" or "declining,” they examined school characteristics to determine if social structure, climate, programmatic or personnel changes were occurring in relationship to changing suldent achievement. They outlined 10 differences between improving and declining school. The characteristics of improving schools included the following: a) acceptance and emphasis on the importance of basic reading and mathematics objectives; b) teachers and principals had higher expectations about their students’ abilities to master basic objectives; c) the staff had higher expectations for students’ future educational accomplishments; (1) teachers and principals were much more likely to assume responsibility for basic skills and were committed to do so; e) teachers devoted a much greater amount of time toward achieving reading and math objectives; f) principals were more likely to be instructional leaders, more likely to be assertive in that leadership role, and were more 28 likely to be disciplinarians; g) appeared to have a greater degree of acceptance of accountability for student achievement; h) teachers were less satisfied with teachers in declining schools; i) appeared to have a higher level of parent-initiated involvement; and j) improving schools tended not to rely on paraprofessional staff or involve teachers heavily in selecting students for compensatory education programs. The following meta-analysis conducted by Clark, Lotto, and McCarthy (1980) provided a summary of studies which examined school effectiveness. They analyzed an aggregation of case studies and reviewed more than 1200 screened on the basis of a) focus on elementary schools, b) evidence of empirical data gathering, c) inclusion of at least one of the independent and dependent variables of concern in this study, and (1) minimal quality standards in execution. The results indicated that there were 6 major areas into which effective school characteristics could be categorized including leadership, teaching personnel, finance, resource and facilities, curriculum and instruction, and community resources. They found that in successful schools the behavior, attitude, and expectation of school leaders were crucial. Successful schools were characterized by clearly stated curricular goals and objectives and specific and focused training and staff development. This training and staff development was important in helping schools reach these goals. Smaller adult/child ratios were important to success. Successful schools were often supported with special project funds from federal, state, and local sources. For urban schools, structured learning environments were particularly successful in urban classrooms and individualized instruction were part of successful schools. The success 29 in urban schools was unrelated to any particular cunicular organization (e.g., open classrooms, homogeneous grouping) or any particular instructional strategy (e. g, programmed instruction, language experience approach). Successful urban schools and programs were characterized by high levels of parental contact with the school and parental involvement with school activities. Correlating School Effectiveness with Academic Achievement The following studies examined the relationship between academic achievement and school characteristics. Studies by Blust (1986), Brookover & Schneider (1975), Brookover, Beady, Flood, Schweitzer, and Wisenbaker (197 9), and West (1985) provided substantial evidence that school social composition and social-cultural characteristics, which often determined children’s school experiences, were correlated with academic achievement These researchers conducted studies, which, in combination, represented elementary schools across the country. Their results indicated that though a large and significant proportion of the academic achievement score variance was attributed to socioeconomic and racial composition, other school climate factors contributed significantly to the variance as well. These factors included teacher initiated environment, teacher involvement in planning functions, teachers’ perception of freedom from disruption, discipline, need to motivate individual students, and their future evaluations and expectations. Student characteristics associated with academic achievement included student’s sense of futility, student motivation and sense of competence and mastery. In the Brookover, et al. (1975, 1979) studies, lower achieving schools were compared to high achieving 30 schools. In Brookover, et al. (197 9) study, higher achieving school students felt they had control or mastery of their academic work, teachers and principals had high expectations of students and communicated these expectations to the students. Teachers and principals also were committed to ensuring that the students learn. , Teachers consistently and indiscriminately rewarded achievement. In the 1975 study, higher achieving schools were more likely to be characterized by students who had less of a sense of futility, more positive perceptions of future evaluation and expectations, and had motivation. Teachers had more positive expectations and had higher evaluations and expectations of students. These studies provided evidence that academic achievement was related to school climate variables. A longitudinal study of school effects was conducted by Rutter, Maughan, Mortimore, & Ouston, (197 9). They studied ten-year olds as they progressed through three years of schooling in the twelve London city schools these children attended (Brophy & Good, 1986; Crisci, 1986). They examined school processes and related these processes to student outcomes. The results indicated that children tended to make better progress both in their behavior and in their academic achievement in schools that focus on academic matters. Outcomes were better in schools where teachers expected the children to achieve well, provided pleasant working conditions for students, provided immediate, direct praise and approval, and where teachers acted as role models for punctuality, were concerned for the physical well-being of the school, concerned for the emotional well-being of the pupils, and used restraint in the use of physical punishment (D’Amico, 1982). This study showed that experiences in 31 school made a difference in students’ behavior and attainments and these differences could be attributed to the particular set of values, attitudes, and behaviors characteristic of school as a whole (Porter, lemon & Landry, 1989; Good & Brophy, 1986). The following two studies examined academic achievement based on the school effectiveness correlates, those concepts which were described by researchers as being distinguishing factors between effective and ineffective schools. The correlates were summarized into usually 5 to 7 dimensions. These correlates were varied but included such concepts as instructional leadership, high expectations, opportunity to learn, etc. It was useful to include these studies in this review in order to examine the validity of the school effectiveness correlates. Bedford (1988) examined the relationship of seven characteristics of effective schools (instructional leadership, environment, expectations objectives, classroom practices, monitoring of student progress, and home school relations), and six demographic variables including size of school, socioeconomic status of students, race of students, location of school, tenure of principal, and sex of principal along with student achievement. Data were collected from 131 Georgia public middle schools containing grades 4-8, 5-8, and 68. They used the Connecticut School Effectiveness Questionnaire to assess school effectiveness. The results showed significant correlations between academic achievement and home-school relations, orderly environment, instructional leadership, expectation for student achievement and student achievement. Insu'uctional leadership and student achievement were negatively correlated with academic achievement. Socioeconomic status, race, and school size were also significantly correlated with student 32 achievement. Controlling for demographics, home-school relations and instructional leadership accounted for more than 30 percent of the variance in achievement scores for all three groups, reading, math and combined scores. When environment was included in the equation, the three variables accounted for 38% of the variance in math scores. The researchers concluded that there was evidence to suggest that schools could improve student achievement. Witte & Walsh (1990) tested the effective schools model by examining the relationship between school achievement and measures of school environment, particularly "effective schools" characteristics. The data were collected from 38 high schools, 32 middle schools, and 134 elementary schools, across 22 districts in the Milwaukee metropolitan area. The achievement measures included standardized test data in math and reading, as well as dropout rates. In addition to a wide range of school characteristics, key measures of school environment were collected from 5,500 teachers in the districts. In addition to an effective schools index, researchers analyzed the effects of parental involvement and variation in teacher control of key decisions in schools. The findings provided evidence that school environment had an effect on achievement but the evidence was very modest for elementary schools. The only result that approached a reasonable confidence level was the effective schools scale on reading scores. In addition, effect sizes were all much smaller than for the middle or high school models (and they were not that substantial at those levels). They also showed that the environmental variables which were significantly related to 33 achievement were highly intercorrelated and very much affected by the location of the school and the student population in the schools. Teddlie and Stringfield (1993) conducted a long-term study for the Louisiana state schools which moved along the continuum of identifying effective schools, correlates, and relationship to academic achievement and then providing a more micro level examination of effective and ineffective schools by conducting longitudinal case studies of effective schools. The results revealed that there were differences among schools. They found that schools and teachers, not just socioeconomic status of students, do strongly influence achievement. In addition, they found that factors involving teachers’ and principals’ expectations for students’ long-term achievement were highly correlated with socioeconomic status. There were several factors independent of socioeconomic status associated with achievement including a) student perception of positive academic climate, b) principals’ sense of school efficacy, c) family commitment to education, (1) student sense of long-term educational achievement, and e) absence of a negative school climate. They found that students’ sense of cmrent and future academic accomplishments and student sense of academic futility added the greatest non-SE8 variance to the prediction equation. In there longitudinal case studies, they found that there were effective schools that remained stable and ineffective schools which were stable as well. In addition, there were ineffective schools which improved over time. During the first phase of the case study research, (3 years) there were stable school effects. For the longer duration, half the schools retained their effectiveness, while the other half changed. Factors which 34 contributed to the change in effective status of schools included change in principals which resulted in dramatic changes in teacher behavior and student achievement, and change in faculty as well. Only 52% of the teachers were still at their original schools atthetimeofthelastphaseofthis study. Sam In the preceding studies, researchers provided evidence that there were efi’ective and ineffective schools, though effectiveness was, for the most part, narrowly defined by scores on standardized academic achievement tests. In addition, there were observable differences between effective and ineffective schools. These differences were not solely student related characteristics such as race and socioeconomic status but were school social-cultural characteristics which could influence childrens’ educational experiences. The results of these studies had important implications. These findings exposed the fact that the burden of responsibility for academic achievement fell squarely upon the broader shoulders of the schools rather than on the children’s. In addition, it provided support to the belief that all children could learn regardless of socio- economic status and race, and that schools could be a major force in providing an environment conducive for positive academic achievement. It suggested that schools could be held accountable for children’s academic achievement. This was a sharp contrast to the findings in the Coleman Report which held that children’s backgrounds determine their academic achievement. On the contrary, these findings suggested that minority and poor children were not disadvantaged because of their status or race but 35 disadvantaged because of die school in which they attended. These findings should begin to illuminate the problems of the demise of cognitive gains for children who attended or participated in early childhood education programs such as Head Start or the Early Childhood Transition Program. The quality of the school environment could either enhance those gains or nullify the gains. When the quality of the educational environment was deficient, it was not illogical to assume that children’s academic achievement would suffer despite compensatory programs. This research also provided a foundation for the argument that compensatory program were not sufficient alone to change the status of children at risk for school failure. It had become an issue of schools failing hundreds, even thousands of children. The question, therefore, became, could school change to become more effective and efficient at enhancing children’s academic achievement. The following research suggested that they could. Ir_nplementation of Effective School Research New York City’s School Improvement Project (Good & Brophy, 1986; Edmonds, 1982) was based on five factors derived from the school efi’ectiveness literature which focused on causes of school failure. These five factors were administrative style, instructional emphasis on basic skills, school climate, ongoing assessment of pupil progress, and teacher expectations. In the New York City public school systems 19 schools were chosen through an application process to participate in this project from 1979-81. Most of the students in 8 of the 10 schools in the first cohort were minority, and over half were designated as low income. The results indicated that for 2 of the 3 years data were analyzed, SIP schools showed greater increases than other city schools 36 in percentage of students reading or above grade level. Gauthier, Pecheone, Shoemaker (1985) conducted the study of The Connecticut State Department of Education’s School Effectiveness Project, which was an attempt to improve effectiveness in schools. They developed and implemented school change strategies which was based on their findings of an assessment conducted using The Connecticut School Effectiveness Interview and Questionnaire. In addition, they collected achievement data and other archival data such as mastery skills checklists, report cards, and student handbooks. Ten schools were included in the evaluation. The results showed that nine of ten schools showed significant positive growth in the effective school characteristics and overall significant growth was found in 26 of the possible cells. Schools that were rated as having high implementation showed the most change in the characteristics compared to schools with low implementation ratings; these showed the least change. In addition, the schools rated higher in implementation had the most significant improvement in achievement. As implementation ratings declined, the academic achievement was less as well. The combination of principal leadership and implementation of a school’s action plan produced a change in effective school characteristics and in turn improvement in student achievement. Marzano, Guzzetti, & Hutchins (1984 ) examined eight "independent variables" in order to implement school level changes. These variables included the following: a) building leadership and support to teachers, b) staff collegiality and collaborative (planning, c) staff expectations for students, (1) teacher use of time, c) teacher use of 37 efficient managerial techniques, f) teacher use of efficient managerial techniques, g) teacher use of motivational techniques, h) student time on task, and i) student success. Over a period of one year, 10 teachers received 20 days of training in the theory and implementation of the eight school effectiveness variables. Teachers were given decision-making power in the planning and formatting of the training sessions and given systematic feedback regarding implementation of instructional and organizational techniques. The results established that some positive changes occurred in school climate and managerial activities including more time explaining the goals for lessons and activities. Teachers reported increase in collegiality and collaboration. Teachers also used more managerial and motivational techniques. Observations and classwork for the teachers’ 141 students showed that, following teacher training, students had less allocated time for reading and math but that they were more engaged and more successful in classwork. A decrease in math and reading achievement was indicated on standardized tests. Jefferson County, Kentucky conducted school improvement project in their schools (Miller, Cohen, and Sayre, 1985) in which dming 1982-1983 in which the findings and practices of the effective schools research were implemented. During the spring of 1982, the faculties from 10 elementary schools volunteered to participate in the program. These schools were primarily in the inner city with predominantly low socioeconomic status students whose previous achievement was below average for the district. 38 The results indicated that the ten project schools, which had substantially lower reading and math scores than the district for 1981-82, had, in one year, caught up with and gone slightly ahead of the other elementary schools in the district for math; for reading, project schools were still below the other schools, but the difference was slight. Although the entire district improved slightly in total reading and total math achievement on the CTBS-U from 1981-82 to 1982-83, gains for project schools were substantially higher than for the total district. Although some control schools gained more than some project schools, overall scores for project schools improved substantially more than did those control schools. Similarly, the increase for project schools was above the district increase for each grade in both reading and math and substantially so for all grades except fifth. Milwaukee’s Project RISE (Rising to Individual Scholastic Excellence) (McCormack-Larkin & Kritek, 1982) was a school improvement project implemented by the Milwaukee school district which included 18 elementary and two middle schools that scored lowest on the annual achievement tests. These schools were those which served primarily low-income and minority students. Using the school effective literature as a guide, they developed a plan which included the following: a) cultivating the belief that all student could learn and that the school was primarily responsible for their learning; b) developing a strong sense of academic mission; c) cultivating a high level of professional collegiality among staff members; (I) establishing a strong sense of student identification and affiliation with the school; e) grade level expectations and standards in the areas of reading, math, and language; 1') 39 using an accelerated learning program for students performing well below grade level; g) increasing the amount of time allocated, and actually used, for active student learning; and h) establishing of a structured learning environment. Through regular meetings with principals, and teacher in-service, staff were introduced to effective school concepts and later, how to implement the principles. The results of this 3 year effort revealed that Project RISE school made improvements in mathematics and some improvement in reading compared to the city-wide averages. Summary: School Effectiveness Literature The school effectiveness literature provided evidence that contradicted the findings of the Coleman Report which suggested that schools were not contributing to the academic performance of their students. On the contrary, schools were significantly contributing to the academic achievement of schools in ways which were under the conu'ol of school personnel. This literature reported that schools were different in effectiveness, and effectiveness was characterized by socio-cultural variables or "correlates." These correlates were varied and included such domains as leadership, expectations, learning climate, academic orientation, and many others. They were related to academic achievement and found to contribute significantly to its variance. Based on these findings, school improvement programs were implemented. The findings suggested that schools could make organizational and structural changes which would enhance the academic achievement of their students. 40 Summary: Integration of Head Start and School Effectiveness Research on Head Start and other early childhood programs provided evidence that these programs had beneficial outcomes for children, enhancing children’s academic achievement. Though long-term academic and social effects were reported, the immediate academic effects in academic achievement tended to fade over time. The effective school literature provided evidence that school environment effects academic achievement as well. In light of these findings, it would seem that there would be both school and program effects on academic achievement for children participating in educational programs operating within school environments. Therefore, the following research is proposed that would examine both program and school effects for children who participate in the Head Start/Public Schools Early Childhood Transition Project. PRESENT STUDY This present study seeks to answer several research questions regarding the combined effects of the school environment, the Head Start/Early Childhood Transition Program, and preschool experience upon the academic achievement of children. Of particular interest are the main effects of school environment, preschool programs, and the Head Start/Early Childhood Transition Program on children’s academic achievement. In addition, this study will examine interaction effects produced by these independent variables as well. Hypotheses There are many hypotheses which refer to the main efiects and the interaction of effects of school environment, program participation, and preschool experience. The hypotheses include the following: H1: There will be a significant difference in mean reading and math scores between children who participate in the Early Childhood Transition program and children who do not. The children who have participated in the program will have higher mean scores than the children who do not. 112: There will be a significant differences in mean academic achievement scores (reading and math) between children who attend schools with higher school effectiveness scores than children who attend schools with lower school effectiveness scores. Children who attended more effective schools will have higher mean scores 41 42 in both reading and math. H3: There will be a significant difference between the children who have attend Head Start preschool program or some other preschool program and the children who have had no preschool experience. The children having preschool experience will have higher achievement scores in reading and math. H4: There will be a significant interaction effect associated with exposure to the transition program and to a preschool program. The difference in the means scores between no-preschool children and the preschool or Head Start children will be greater for children who participated in the transition program than for children who did not participate in the transition program in both reading and math achievement. H5: There will be a significant interaction associated with exposure to a preschool program and exposure to different school environments. The difference in the mean reading and math scores between children have no preschool experience and preschool experience will be higher for children who attend more effective schools than those who attend less effective schools. H6: There will be a significant interaction effect associated with exposure to the transition program and exposure to different school environments. The difference in mean reading and math scores between children who have participated in the Transition program and those who do not participate will be greater for those who attend schools with higher effectiveness scores than children who attend less effective schools. 43 Program Description Following are descriptions of the Head Start/Early Childhood Transition project as well as the Head Start in which some of the children in the study have participated. H l ' iti n Pro : Pr am Descri 'on The Head Start/Early Childhood Transition Program, sponsored by the Administration for Children, Youth and Families, provided Head-Start like services to all children kindergarten and their families who attend the program schools. These services included educational. social, and health services, in addition to family support services and parent activities. The Transition Program to was designed to be a collaborative effort between Head Start, the public school system and other community agencies to provide families with comprehensive services which would enhance and encourage the successful growth and development (physical, emotional, intellectual) of the children, and which would promote family involvement in the educational process of the children. During 1992-1993, the program began to serve kindergarten children and their families in the schools designated to receive the program. The program had four components - health, social services, parent involvement and education. Each component will be briefly described below. M The goals of the health component were to promote the importance of good nutrition, to evaluate the nutritional status of child, to make families aware of any nutrition related programs available in the community, and most importantly, to assist families in obtaining and maintaining high quality health care. During the year, the health coordinator developed linkages with clinics and medical centers to provide routine health care for Transition families, made home visits to conduct health screening and compile a medical history of the family. In addition, the Health Coordinator conducted educational and presentations for both parents and children, made internal referrals to appropriate agencies, wrote a health segment for each edition of the program’s semi-monthly newsletter. Social Services The goals of the social service component were to coordinate the reception of social services, to communicate to community agencies the needs of the families, to facilitate delivery of services to families and provide appropriate follow-up to assure delivery of needed services or assistance. During the year, family service coordinators conducted home visits and family assessments in order to develop a family service and individual support plan for each family. Assessments were made of family needs, parenting skills, child’s home environment, child’s developmental abilities, psychological condition, and health history. (Home visits were to occurred monthly). 45 The family service specialists were able to acquire services for the Transition families including emergency resources for families needing one-time assistance for specific needs. Referrals were made to appropriate social service agencies. Types of referrals included financial, counseling, transportation, clothing, education, legal and medical. A "Clothes Closet" was established for families in need of clothing. They also sponsored adult group activities and provided training to parents in child development, parenting and life skills. Topics for parenting workshops and training included games to play with children, sex education, substance abuse, positive parenting, budgeting, protecting children against child abuse, talking to Children about Drugs and celebrating Black Families. Activities for children conducted by the Transition program included field trips to parks, zoo, museums, etc., and holiday celebrations (Thanksgiving, Christmas, Easter, etc.). We The goal of educational component was to provide developmentally appropriate educational curriculum to the children. The program contracted with High/Scope to develop a plan and implement the High/Scope Curriculum within the program schools. High/Scope was an early childhood program formulated by the staff of High/Scope Educational Research Foundation in the 1960’s and the 1970’s (Roopnarine and Johnson, 1987) in conjunction with the Perry Preschool Project. The High/Scope curriculum was based on Jean Piaget’s theories of child development who believed that children were active learners, who learned best from activities that they planned and executed. 45 During the 1992- 1993 school year, a representative from High/Scope provided instruction and training to the teachers, and supervised the implementation of the High/Scope cmriculum within the schools. Training sessions were held for kindergarten, 1st and 2nd grade teachers who worked in the 3 transition schools. Teachers attended training workshops, seminars, and visited to High/Scope sites throughout the school year. The High/Scope representative assessed the progress of the cmriculuin implementation through class visitations and observations. From these observations, the representative provided teachers with feedback. Head Start Because this study will also examine the difference between Head Start and children with other preschool experiences and those with no-preschool experiences, the following overview of the area Head Start programs is presented. The goal of Head Start was to provide children with a safe and nurturing environment that promoted intellectual, emotional, social and physical growth and development while responding to the needs of the families. The Head Start agency which provided services for children in the sample represented 16 centers in a tri-county area. Each center had one teacher and a teacher aide and 16 to 20 children in each class. Education Head Start centers used several educational strategies or models to implement curriculum such as High/Scope or a thematic model. Each site could choose a strategy or model. However, curriculum guidelines, developed by the Education Advisory 47 Committee and which included parents and teaching staff, ensured that the sites incorporated the following elements in their curriculum. The curriculum must: 1) be relevant and reflective of the population and community served 2) provide cultural awareness, including food experiences 3) include an integration of nutrition, health, safety, social skills, parent involvement, and special needs 4) be developmentally appropriate, and 5) encompass the whole child. All of the above aspects of the curriculum should be included in the daily routine. The daily schedule included time for small and large groups and individual activities, child-initiated activities (free play should be one hour long), large and small motor skills, washing hands, eating, and brushing teeth, and incorporating math concepts into the daily routine. Activities were to have a balance between staff-direct and child initiated activities. METHODS Design This study’s data were collected fi'om an ongoing evaluation project examining the impact of Head Start Transition Project implemented in a small Michigan school district. There were six elementary school within this district. Schools were matched on demographics, grouped into 2 clusters of 3 schools each. These clusters were randomly assigned to control and experimental groups. Three schools served as transition program schools and the three other schools served as a comparison group. School effectiveness levels were determined by aggregating teacher and other teacher support staff scores on the Connecticut School Effectiveness Questionnaire in each of the six schools. Participation in a preschool program was determined by each family’s self selection. Matching schools There were 6 elementary schools in the school district. These school were matched on demographics (see Appendix A), including student population, ethnicity, single parent families, and income. As a result, two clusters of schools containing three school each were identified. Cluster one represented 750 students in which 84% were African American, 2% were Hispanic/Latino, and 14% were Non-Hispanic whites and others. Fifty eight percent of these children were from single parent homes, 61% were from families which earned less than $10,000 per year, and 27% of the parents 48 49 had not received high school diplomas. Cluster two represented 695 students for whom 88% were African American, 2% where of spanish descent, 10 percent were white and other. Sixty one percent of these children were from single family homes, 62% were families earning less than $10,000 per year and 21% of the parents were without a high school diploma. Random Assimt School clusters were randomly assigned to either the control or experimental group. As a result, cluster 2 was assigned to the experimental group and cluster 1 was assigned to the control group. Sample Data were gathered for two groups of participants representing two levels of assessment, children and teachers. Children’s Sample Three hundred and thirty four children attended kindergarten in the district during the 1992 school year. Two subsamples of this total were used for this study, a posttest only sample, those children from whom there are only posttest scores, and a pre and posttest sample, those children who have both pre and post scores on the Developing Skills Checklist (DSC). (The children for whom there were no test scores available or for whom there were no scores for fall or spring were children who, most likely, were absent during testing periods, moved into the school district after initial testing or moved out of the district before the final testing period or were children with special needs.) There were 271 children in the post-test only subsample, and 204 50 children in the pretest and posttest sample. Table 1 illustrates the sample sizes of children according to preschool participation, program participation across schools. The schools were divided into two groups, experimental and control, and three groups of preschool experience. One hundred and three children were in the program or experimental group and 101 children were in the comparison or connol group. The majority of children had some type of preschool experience whether Head Start or some other program However, within this group, the majority attended some other program besides Head Start. Table 2 outlines demographics for both subsamples. The sample did not appear biased based on the subsample selections The ethnicity and gender for both subsamples were proportionately the same of the total sample (n=334). For example, of the 334 children, 1% (n=3) were native American, 86% (n=288) were African American, 12% (n=39) were white, 1% (n=4) were Hispanic. In addition, 53% (n=176) were females and 47% (n=158) were males. Teacher and suppprt staff Sample Elementary principals, teachers, and support staff in the schools completed school effectiveness surveys. During the 1992-1993 there were fifty eight classroom teachers, 12 support staff, 8 compensatory education staff, 4 pre-school teachers and 3 principals. Of these 85 stafi‘ members, 58 persons responded to the survey including 49 classroom teachers, four compensatory education teachers, three principals, one Table 1 Smle Sizes across Sphools 51 Schools Transition Comparisons l 2 3 4 5 6 Totals Pre & Posttests Head Start 4 9 3 7 6 12 41 Other Pre-school 2O 22 16 12 0 9 79 No Preschool 4 9 16 13 27 15 84 Totals (N) 28 40 35 32 33 36 204 Posttest Only Head Start 7 l3 5 8 7 13 53 Other Preschool 23 22 19 13 O 11 88 No Preschool 21 16 20 18 33 22 130 Totals (N) 51 51 44 39 40 46 271 Table 2 52 Sumpk demomphics for Post only and Pre and mst test ggopps. Posttest only Mom Sex N % N % Female 143 (53%) 104 (51%) Male 128 (47%) 100 (49%) Race/ethnicity Afiican American 237 (87%) 173 (85%) White/Caucasian 28 (11%) 25 (12%) Hispanic 4 (2%) 4 (2%) Native American 2 (>1%) 2 (>1%) 53 instructional assistant and one preschool teacher. Since each of the three principals were in charge of two schools each, they completed 2 surveys, one for each school. However, the principal data were deleted because of problems related to independence of sampling. One additional survey was deleted from analyses because of missing data. The total number of surveys included in the analyses was 56. Fifty three persons in this sample were female and three were male. These teachers were divided between six schools. There were 8 teachers in school 1, 10 teachers in school 2, 10 teachers in school 3, 9 teachers in school 4, 6 teachers in school 5 and 12 teachers in school 6. Data Collection Procedures 9211311211. Children’s academic achievement scores will be obtained from the school archival records at the end of kindergarten. Children’s Instrument Early Chilth Developing Skills Checklist. Children were assessed using the Early Childhood Developing Skills Checklist. This test was given to all kindergarten children twice per year, in the fall and again in the spring. It was a standardized. criterion-referenced, readiness test which was designed specifically for pre- kindergarten and kindergarten children. It measured the child’s proficiency in skills that were characteristic of kindergarten and were prerequisite to formal instruction in reading and mathematics. As a readiness test, it identified particular curriculum- related knowledge and skills that the child had acquired prior to entering kindergarten 54 or formal academic training. These skills and behaviors included mathematical concepts and operations, language, memory, visual, auditory, fine and gross motor skills and print and writing concepts. The DSC was normed in the fall of 1988, winter of 1989 and spring of 1989. Tested in pre-kindergarten and kindergarten children from public school districts, private schools, and Catholic dioceses. There were 633 pre-kindergartners and 1033 kindergartners in the sample. The reliability for these samples was assessed using both the Standard Error of Measurement and the Kruder-Richardson 20 statistics. The reliabilities for the scales ranged from .64 to .94 (see Appendix B). The scales fiom the DSC used in this research were as follows: mathematical concepts and operations, language, memory, visual, auditory, and print concepts. The mathematical concepts and operations scale asked children to name shapes, copy and extend patterns, count, identify numerals and match sets, join and separate sets, and demonstrate knowledge of ordinal positions. The logical operations included the conservation of number, seriation of length, classification, and estimation. The language scale assessed the ability to communicate ideas, feelings, and knowledge. The memory scale assessed children’s ability to recall information that was presented visually or orally. The motor scale assessed children’s fine and gross motor abilities. For the visual scale children were asked to find likenesses and differences in forms. letters, and words. In the auditory scale children identify likeness and differences among sounds, and for print concepts, children were asked to demonstrate their knowledge about the pinposes and functions of print. 55 A total math score was computed from the mathematical concepts and operations scale and a total reading score was computed from the combined scores of the other scales. Though math and reading achievement as measured by the normed DSC data were significantly correlated (r=.74) and could be viewed as one concept, it was decided to separate the two for the purposes of this research. Oftentimes policy decisions are made based on the separate reading and math scores. Educators and others want to know about these scores. In addition, there may be differences the way and rate in which children learn both skills. Furthermore, there may be factors that effect math scores differently than reading scores. Te er rt taff Princi als Classroom teachers, other support staff, and principals were given a questionnaire, the Connecticut School Effectiveness Questionnaire, during the spring of 1993. Principals were consulted for permission to survey teachers within the school. Once permission was obtained, surveys, a letter explaining the purpose of the study and asking for their participation, and an envelop in which they could place their completed surveys, were placed in all teachers’ and staff mail boxes in the school’s main office. Teachers and staff were asked to complete the questionnaires and place it in the envelop provided. They were instructed not to write their names anywhere on the questionnaire or envelop. In addition, they were instructed to place the sealed envelops in a box which was located in the teachers’ lounge. Anonymity, confidentiality, and voluntary participation was assured and the conditions of their participation were explained in the letter. First, participation in the 56 study was voluntary. Teachers and other support staff were informed that by completing the survey they had provided their permission to participate. They could refuse to participate without penalty. Teachers were also infomled that the information reported on the sru'veys anonymous. They were not to place their names anywhere on the survey or on the envelop. In addition, they were asked to place their envelops in a box located in the teachers’ lounge rather than it being collected by the researcher. Confidentiality was guaranteed as well. The information in the survey would not be shared with anyone other than the research team members and, in addition, the information would not be used to evaluate teacher performance or would the information be given to the school district or principals. Teachers’ Instrument anectigpt fighml Effectivpnppp Survey. This instrument was developed by Villanova, Gauthier, Proctor, and Shoemaker in 1981 and later revised by Freedman, Lappert, and Waterman in 1989 in conjunction with the Connecticut State Department of Education (see Appendix C). It was designed to gather data on the extent to which a school displayed seven school effectiveness characteristics (V illanova, 1984). It was a 97 item, 5 point liken-type questionnaire in which respondents were asked to indicate agreement with the statements on a scale from strongly disagree to strongly agree. There were seven scales which reflected seven school effectiveness characteristics including safe and orderly environment, clear school mission, instructional leadership, high expectations, opportunity to learn and time on task, frequent monitoring of student progress, and home-school relations. 57 For content validity, this instrument was reviewed by a panel of experts composed of Connecticut School District officials and school district personnel familiar with the school and classroom effectiveness research who sorted items into the defined categories. Items associated with the appropriate criteria at least eight pacent of the time were included. The questionnaire was then field tested with six classroom teachers. A multi-trait-multi-method analysis supported construct validity (V illanova, 1984). Reliability was assessed through alpha internal consistency for each scale as well as a test-retest reliability. The alpha scores for each scale on the original measure ranged from .66 to .93 (n=423). Test-retest scores ranged from .67 to .80 (n=60). The reported reliabilities were .80 and above. Reliability analysis of the data collected from the teachers and other teaching support staff for this study (N=56) revealed the that reliabilities using Cronbach’s alpha ranged ficm .73 to .95. The item-total correlations for the scales ranged from .59 to .77 for "Safe and Orderly climate; from .38 to .78 for Clear School Mission; from .33 to .86 for Instructional leadership; from -.09 to .60 for High Expectations; from -.05 to .66 for Opportunity to learn; from .33 to .55 for Frequent Monitoring; and from .25 to .71 for Home School Relations. Seven items were deleted from these scales for being conceptually inconsistent with the scale or had very low item-total correlations. These items were as follows: 1) Item 45: Students are absent from school only for good reasons; 2) Item 47: Ninety to one hundred percent of your students’ parents attend scheduled parent-teacher conferences; 58 3) Item 49: Dming classroom instruction students do not work independently on seatwork for the majority of the allocated time; 4) Item 56: In this school, remedial programs are the last resort; 5) Item 57: The number of low-income children promoted is proportionately equivalent to all other children promoted; 6) Item 58: Parents of students in your class have regular opportunities to observe the instructional program; and 7) Item 85: Parents of students in your class have a significant rather than a superficial role in the educational program. The resulting changes in scale alphas are reporwd in Table 3. In addition to the alpha analyses, a hierarchial linear models analysis was performed in order to estimate the reliability of the school effectiveness scores as an indicator of the true school mean. (The hierarchial methods will be explained in more detail later on in this discourse). In this analysis, an estimate for each school mean was calculated by dividing the between school variance by the sum of the between school variance plus the within school variance divided by the n of that school. The average of these scores produced the reliability estimate indicating whether the means were indicators of true school means. The results of .06 indicated that these means were not reliable as indicators of true school means. For this analysis, there was more variation within school than there was between schools on effectiveness scores. This measure was not a reliable indicator of school effectiveness. (The consequence of the unreliability of this measure will be discussed further in the discussion section). 59 A factor analysis (principal components) was conducted on the scale scores for the seven scales of the school effectiveness questionnaire. The result revealed that one factor accounted for 72% of the variance. The eigenvalue for that one factor was 5.03 and the only value above 1. The values for the six other factors were between .16 and .63. The variance accounted for by each of the remaining factors ranged fiom 2% to 9%. Since a single factor accounted for most of the variance, a single score which includes all 89 items was be used in the regression analysis. Measure of Progrprp Implementation W. Classroom observation were conducted during the spring of the 1992-1993 school year in order to assess the extent to which developmentally appropriate practices occurred within the classrooms. The observation tool was entitled the "Assessment Profile for Early Childhood Program." It contained a total of 87 items. These items were divided into 5 scales which included learning environment, scheduling, curriculum, interacting, and individualizing. All the items except 10 were scored yes or no. The other 10 items asked for the number of different types of learning materials present in the classroom ranging fi'om 3 to 5. Analysis Hierarchical linear models procedures were used to analyze the program, preschool, and school effects for this study. This procedure was chosen because it not only replicated the information and had the statistical advantages of both the mixed model ANOVA and the standard regression, but offered additional statistical s0phistication as well. As with standard linear regression, the hierarchical linear Table 3 Al has for Co 'cut School Effectiveness estionnaire = 6 Original Revised Ms M flakes ILanS. Alphas. School Environment 96 .97 89 97 Safe and Orderly Climate 9 .90 9 .90 Clear School Mission 16 .93 16 .93 Instructional leadership 24 .95 24 .95 High Expectations 14 .75 12 .80 Opportunity to learn and Time on Task 10 .73 8 .71 Frequent Monitoring 8 .76 8 .76 Home School Relations 14 .87 12 .88 61 model handled unbalanced data and predictors for both discrete or continuous variables quite well and, in addition, allowed for designs having fixed and random effects. This procedure reduced the problems of misestimated standard errors which occurred with multilevel data that failed to take into account the dependence among individual responses within the same organization (Bryk & Raudenbush, 1992). It incorporated a unique random effect for each organizational unit and considered this variability in estimating standard errors. "Hierarchical linear models enable the investigator to estimate a separate set of regression coefficients for each organizational unit, and thento model variation among the organizations in their set of coefficients as multivariate outcomes to be explained by organizational factors " (Bryk & Raudenbush, 1993, p. 84). The outcome variables used in these analysis were reading and math achievement scores as measured by the Developing Skills Checklist. These scores were reported as standardized normal curve equivalent scores or rankings. The normal curve equivalent scale was an equal interval scale which allowed for comparisons between scales. Math and reading scores were used as separate outcome variables and, therefore, a separate set of analyses (though the same analyses) were performed for both. The independent variables were preschool participation, transition program participation, and a school environment index. There were two levels of preschool, children who had participated in a preschool program and children who had no preschool involvement. The transition program had two levels, participation and no 62 participation. School environment was an index of teachers’ and support staffs’ perception of school environment as measured by the Connecticut School Efi'ectiveness Questionnaire. Scores were summed across all variables for each teacher within a school and the means for each school were assigned to the students within each school. (Table 4 illustrated the means and standard deviations for each school.) This present research represented a two-level hierarchical model in which students were nested within schools. The level-1 represented the relationship among the student-level variables, pretest scores on math and reading achievement and preschool participation. The level-2 represented the influences of school-level factors which included transition program participation, school environment rating, and a factor representing the interaction between transition program and school environment rating. There were i =l,...,n,- level-1 units nested within j=1,..., J level-2 units. In this instance there were 271 students nested within 6 schools. The model is illustrated in the following equation: itu = 13C,j + aux“, + 13,,xnj.s + Bojxqq, + n, where Y“ is the outcome variable - math or reading achievement; B,” (q = 0, l,...,Q) are level-1 coefficients; X,” is level-l predictor q (preschool, pretest on reading or math achievement) for student i in school j; rll is the level-1 random effect; and 63 Table 4 Grand Mean and Standard Deviation and School Mean Scores and Standard Deviations for School Environment Scale Grand Mean 56 M 276.34 SQ 55.24 Schools Experimental Control 1 2 3 4 5 6 N 8 10 11 9 6 12 X 278.50 306.50 257.00 279.56 265.83 270.33 8D. 72.63 28.30 49.84 81.81 53.85 38.63 64 s2 is the variance of r”, the level-1 variance. It is assumed that the random term is normally distributed with a mean of zero and variance (Bryk, Raudenbush, & Congdon, Jr., 1994). Each of the level-l coefficients, B”, defined in the level-l model becomes an outcome variables in the level-2 model: B,” = qu + 6,],le + ogwq + + 64,,qu + uq, where G, (q=0,l,...,Sq) are level-2 coefficients; W,‘ is a level-2 predictors (program participation, school environment); and u... is a level-2 random effect (Bryk, et al., 1994). The hypothesis tests commonly used for these analysis depends on the number of parameters being estimated. For a single parameter, the hypothesis test for the fixed effect of the independent or predictor variable is a t ratio. It tests the null that the fixed effects are equal. The test for the random level-1 coefficient is a t ratio and the variance component is a univariate chi-square or 2 ratio. For multiparameters, the test for fixed effects is a general linear hypothesis test for the fixed effect, and a likelihood-ratio test (chi-square) for the variance component (Bryk & Raudenbush, 1992; Bryk, Raudenbush, Congdon, Jr., 1994). Two types of hierarchical regression equations analyses were performed. First, each of the three independent variables were analyzed in separate zero order equations, equations representing a single independent or predictor variable for each dependent variable, math and reading achievement. The next analyses performed were two way 65 interactions equations where two independent variables were entered into the equations. The combination of these variables included program and preschool, program and school environment, and preschool and school environment. This procedure was chosen above entering all the independent variables at the same time because the of the small school sample size. Using an equation with several independent variables and a small sample would use up too many degrees of fieedom. This would produce an analysis which would lack considerable power. According to Cohen and Cohen (1975), the greater the number of hypotheses tests and the greater the number of independent variables, the lower the power of the test and the greater the probability of the occrurence of spurious significance. With only 6 units, an analysis with 4 independent variables would use up all but n-k-l degrees of freedom or 1 degree of fieedom. Cohen and Cohen (1975) also noted that as the independent variables increases, the Rs (correlations) among the independent variables would increase, which, in true, would increase the standard errors of practical coefficients and would reduce the t-ratio’s and, therefore, the power. They concluded that "having more variables when fewer are possible increases the risks of both finding things that are not so and failing to find things that are" (Cohen & Cohen, p. 160). Therefore, the analyses in which there would be more degrees of freedom and fewer independent variables would produce more reliable results than a two or three way interaction analyses which would use several degrees of freedom. The results for each analysis were reported in tables. The estimated parameters and statistics reported in these tables included coefficient for the intercepts and slopes 66 for each level. These coefficients represented the main and interaction effects for each independent variable. The coefficients represented the strength of association between the independent variable and the outcome variable, achievement. Reported along with the coefficients were its standard error as well as the t ratio, the ratio of the estimated coeflicient to its standard error. The significance level for this study was set at below .05. Included in the table, also, were random effects parameters and statistics. The random effects were the variances for each level, between school and within school, as well as the variances in slopes between schools when slopes were allowed to vary in the equation. The chi-square statistics tested the hypothesis that the variances between and within schools were the same. For example, if there was a significant chi-square for between school variance, then one could suggest, with a high degree of certainty, that there was variability among school means. The proportion of total variance accounted for by each level of variance could be calculated from the variance parameters by dividing the between school variance by the sum of between and within school variance. RESULTS Preliminary Analyses A one-way ANOVA model of the hierarchial linear models was the first analysis performed to examine the differences in means scores between schools in math and reading achievement at posttest. The model formulated a regression equation in which there were no predictor variable or independent variable. The one-way AN OVA model represented a baseline analyses from which the results from subsequent analyses could be compared. By comparing subsequent results to this model, it could be determined how much variance was accounted for by adding predictor variables. If the variance was smaller in subsequent analyses after adding predictor variables, then it would be probable that the independent variables were contributing to these differences. However, from this preliminary ANOVA analysis, one could only conclude that there were significant differences among school means. The results reported in Table 5 revealed that there were significant differences among school means (p (.0001) in math achievement. Twenty percent of the variance was between schools. In addition, a one-way AN OVA model was performed to examine the differences in mean scores between schools in reading achievement at posttest. The results reported in Table 6 revealed that there was significant differences between the means scores on posttest among schools (p=.001). Nineteen percent of the total of the 67 68 Table 5 VA M lfor th Achievement =271 Fixed Effect Coefficient Se g-test p Main Achievement Effect 38.88 5.7 6.81 .001 Random Effect Variance SD d.f. x2 p Between School 174.86 13.22 5 48.51 <.0001 Within Subject 688.05 26.23 69 Table 6 ANOVA Model for Reading Achievement (N=271) Fixed Effect Coefficient Se t-test p Overall Reading achievement Intercept 37.90 5.42 6.99 .001 Random Effect Variance SD d.f. x2 p Between Schools 157.34 12.54 5 47.83 .0001 Within Subjects 651.86 25.53 70 variance was between schools. A one-way analysis of variance and post hoc procedure, the Student-Newman- Keuls, supported this finding, and identified the how schools differed. There were differences among schools for both math and reading (F (1,5)=14.18 p < .0001 and F (1,5)=11.29, p < .0001). Table 7 indicated that the school math and reading achievement means and standard deviations for each school. Children attending school 2, on the average, scored significantly lower on math achievement than children attending schools 1, 4, 5, and 6. Children attending school 3, on the average, scored significantly lower in math achievement than children in school 1 and 5. For reading achievement, children attending school 2, on the average scored significantly lower on than children attending schools 1, 4, 5 and 6. Children attending school 3 scored significantly lower on reading achievement than from schools 4, 5, and 6, and children attending school I scored significantly higher than children attending schools 4 and 6. In order to provide a more statistically powerful analysis, an AN COVA model was planned for subsequent analyses in which posttest means between schools were examined for math and reading achievement controlling for protests. The model represenwd an regression equation in which pretest was entered as a predictor variable at level one either alone or with other level two variables for which pretest could be used as a covariate. However, this model was not useful for this data. Children’s achievement raw score was transformed into a normal curve equivalent score in which children were ranked with a score of 1 to 99. (The raw score data were not available to the researcher). As a results of this transformation, the standard deviations on 71 Table 7 M Mean and Standard Deviation and School Mean Score and Standard Deviations for Math and M' g Achievement Scores Grand Mean Grand Mean Math Achievement Reading Achievement N 271 271 X 37.66 35.50 SD. 29.77 27.18 Math Achievement Experimental Control 1 2 3 4 5 6 N 51 51 44 39 40 r 46 X 59.08 17.82 28.09 39.26 46.12 36.35 8D. 29.12 19.87 23.70 27.62 30.27 28.99 Reading Achievement Experimental Control 1 2 3 4 5 6 N 51 51 44 39 40 46 X 32.18 18.33 28.32 52.67 42.28 44.63 8.13. 25.65 25.41 19.50 29.11 23.12 25.77 72 achievement at pretest were very low for low scoring groups of children. This suggested a serious floor efiect and that variances were artificially restricted at the lower end of the scale. These problems seriously limited the utility of the pretest data. The ANCOVA model analysis was performed, however, and the results did not reveal a different pattern of results than the posttest only analysis as a comparison of Tables 5 and 6 with 8 and 9 would reveal. Consequently, the ANCOVA model analyses would not be reported for subsequent analyses. A repeated measures AN OVA was performed to examine the differences between pre and post scores on math and reading achievement. The results indicated that there was a significant difference between pre and posttest scores in math achievement (F (l,198)=339.l3, p < .0001) and a significant time and school interaction (F (1,5)=22.35, p < .0001). The differences between pre and posttests depended upon the school in which children attended (See Table 10). For reading achievement, there was a significant difference between pre and post scores (F(1,198)=226.76, p < .0001) in addition to a significant time and school interaction (F(l,5)=18.53, p < .0001). This result suggested that differences in pre and posttest scores depend upon the school children attended. School were having a differential impact on children’s reading achievement. The question to answer now is why are these schools different? The subsequent analyses seek to answer the question of why are these school different. Do preschool participation, the Transition Program or School Environment account for the differences in means scores? 73 Table 8 AN VA M l ntrollin for Pretest for Math Achievement =204 Fixed Effect Coefiicient Se .t_-test p Main Achievement Effect 30.16 8.44 3.57 .01 Pretest Achievement Slope, 1.14 .20 5.81 .002 Random Effect Variance SD d.f. x2 p Between School 396.84 19.32 5 122.35 .0001 Within subjects 583.76 24.16 Table 9 74 ANCOVA Controlling for Pretest for Reading Achievement (N=204) Fixed Effect Coefi'icient Se t_—test p Overall Reading Achievement Intercept 28.68 6.55 4.38 .005 Pretest Main Effect .70 .10 6.71 .001 Random Effect Variance SD d.f. x2 p Between Schools 231.01 15.20 5 82.17 .0001 With Subjects 530.20 23.03 75 Table 10 um arm-ac...- 'O‘ll."l u'il .u .tuuu D.-r".-u o turns . Achimmsm Schools 1 2 3 4 5 6 Math N 28 40 35 32 33 36 Pretest M 2.21 15.72 13.77 3.69 3.45 7.22 SD 1.20 14.68 9.58 5.03 3.79 7.60 Posttest M 58.39 18.97 27.74 41.47 47.09 40.42 SD 28.93 20.81 21.36 26.94 28.87 30.27 Reading N 28 40 35 32 33 36 Pretest M 1.75 19.82 15.51 8.91 14.73 18.19 SD .93 15.99 15.36 8.56 17.21 22.89 Posttest M 37.71 20.60 24.94 55.72 42.76 40.06 SD 29.75 27.75 15.83 27.23 24.86 26.1 1 76 H esis ° Pr Main Effects A hierarchical linear regression analysis was performed to examined the program effects in which the transition program independent variable was entered into the equation as a level-2 variable. The hypothesis stated that there would be differences between the means scores of children who attended transition programs schools and children who did not attend transition program schools. The children who attended program schools would have higher means scores. The results reported in Table 11 indicated that there was not a significant relationship between the transition program and math achievement after controlling for pretest (p.=.23). Children who attended transition program schools did not score significantly different on math achievement tests than children who did not attend transition program schools. Table 12 reported the means and standard deviations of children who attended transition program schools and comparison schools. The results for math achievement did not support the hypothesis. Using the same analysis for reading achievement, the results reported in Table 13 indicated that there was a trend in the data indicating a negative relationship between program and reading achievement. Though the results did not reach the .05 level, the analysissuggested that children who attended transition program schools scored, on the average, approximately 19 points lower on reading achievement tests than children in the other schools. A comparison of the ANOVA model between school variance and the program main effect between school variance revealed that the program effect did explain some of the variation among schools. Six percent of the 77 Table 11 Pr Main Eff f r Math Achiev t =204 Fixed Effect Coefficient Se t-test p Overall Achievement Intercept 37.44 12.11 3.09 .02 Program-Achievement Main Effect -l4.78 17.12 .0.86 .23 Premath-Achievement Slope 1.16 .20 5.86 .002 Random Effect Variance SD d.f. x2 p Between School 420.39 20.50 4 96.26 <.0001 Within School 583.72 24.16 78 Table 12 Pro and arison Means and Standard Deviations Math Reading Program Comparison Program Comparison N 146 125 146 125 M 35.33 40.38 26.18 46.38 SD 30.29 29.05 25.45 26.23 79 Table 13 Fixed Effect Coefficient Se t-test p Overall Reading Achievement Intercept 38.52 7.51 5.13 .004 Program Main Effect -l9.49 10.43 -1.87 .07 Pretest Main Effect .69 .10 6.62 .002 Random Effect Variance SD d.f. x’ p Between Schools 147.31 12.14 4 39.99 <.0001 Within Subjects 530.39 23.03 80 variation among schools was explained by the transition program main effect after controlling for pretest. The results for program effects on reading achievement did not support the hypothesis; on the contrary, children did not perform better on achievement test because they were in the program, but performed worse. The results from an observation conducted in conjunction with the national evaluation of the Head Start/Early Childhood Transition Program provided a way of examining the extent to which the program was implemented within the 3 program schools. Table 14 reported the scale scores and total scores for this observation. An ANOVA analysis of the total scores revealed that there were no significant differences between program and non-program schools (F (1,10)=.08, p=.78). In addition, these program school did not significantly differ from non-program schools on scale scores. There were no significant differences for scheduling (F(1,10)=.15, p=.71), individualism (F(1,10)=.15, p=.71), interacting (F(1,10)=.56, p=.47), and learning (F(1,10)=.32, p=.32). There was, however, a difference between program and non— program groups which approached significance for the curriculum scale. The non- program schools had higher scores than the program schools. However, on the average, program schools were not significantly different from non-programs in providing a developmentally appropriate education. At the end of the school year, the program schools should have implemented a curriculum which looked significantly different from the comparison schools. 81 Table 14 Scale Scores, Means and Standard Deviations for the Assessment Profile for Early Childhood Eucation Prom Schools 1 2 3 4 5 6 Total N 9 10 10 9 6 12 M 43.5 48.0 53.5 53.5 51.5 34.5 SD .36 18.38 2.12 .71 19.09 .70 Scheduling M 4.5 4.5 9.5 8.5 4.5 3.5 SD 2.12 .70 2.12 3.53 2.12 .7 1 M 12.0 8 6.0 12.0 8.5 6.5 SD 1.41 .0 1.4 2.83 .71 2.12 Interacting M 10.0 10.0 10.5 11.5 10.5 11.5 SD 2.82 1.41 3.53 3.53 3.53 2.12 Learning Environment M 8.5 17.5 17.0 10.0 13.5 3.5 SD 2.12 19.09 8.48 .0 10.61 2.12 Curriculum M 8.3 9.0 10.5 11.5 14.5 9.5 SD 1.21 1.41 .70 2.12 2.12 2.12 82 Hymthesis Two: School Environment Effect A hierarchical linear regression analysis ANCOVA design was performed to test the hypothesis that school environment would impact math and reading academic achievement of children. The hypothesis stated that children who attended schools with higher school environment scores would have higher achievement scores than children who attended schools with lower school environment scores. The results reported in Table 15 indicated that there was not significant relationship between school environment and math achievement. Children who attended schools with higher environment scores did no better or worse than children who attended scores with lower environment scores (p=.09). The hypothesis was not supported for math achievement. For reading, the same analysis was performed. The results reported in Table 16 indicated that there was not a significant relationship between reading achievement and school environment score (p =. 13). Children in schools with higher school environment scores did not significantly differ on achievement scores than children who were in school with lower school environment scores. The hypothesis was not supported for reading achievement. Hymthesis Three; Preschool Main Effect A hierarchical linear regression analysis was performed using a single preschool variable to test the hypothesis that there would be a significant main effect for preschool, that is, children who attended preschool would have higher mean achievement scores than children who did not attend preschool. The results reported l./— Table 15 83 ghool Environmeng Main Effect for Math Achievement (N=204) Fixed Effect Coefiicient Se g-test p Overall Math Achievement Intercept 284.45 153.01 1.86 .07 Environment Main Effect -0.91 .55 -1.66 .09 Covariate - Premath Slope 1.15 .20 5.86 .002 Random Effect Variance SD d.f. x2 p Between schools 288.59 16.99 4 66.18 <.002 Within Schools 583.73 24.16 Table 16 School Environment Effect for Reading Achievement (N=204) Fixed Effect Coefiicient Se g-test p Overall Reading Achievement Intercept 200.86 126.37 1.59 .10 School Environment Main Efi‘ect -.61 .45 -1.36 .13 Pretest Main Effect .70 .10 6.72 .002 Random Effect Variance SD d.f. x2 p Between Schools 194.95 13.95 4 52.43 .0001 Within Schools 530.22 20.03 85 in Table 17 indicated that there was a significant relationship between math achievement and preschool (p=.05). Children who attended preschool scored significantly higher than children who did not attend preschool on math achievement scores. The main efiect for preschool accounted for 4% of the between school variance for math achievement. For reading, the results reported in Table 18 indicated that was a relationship between preschool and reading achievement that approached significance (p=.06). Children who attended preschool had higher reading achievement scores than children who did not attend preschool. The main effect for preschool accounted for 30% of the between school variance for reading achievement. Preschool participation was divided into two variables, one representing children who attended Head Start and the other representing children who attended a preschool other than Head Start in order to examine the difi‘erences between these two preschool experiences and no preschool experience. Each preschool variable was entered into the regression equation as a level-1 variable. The results reported in Table 19 indicated no relationship between Head Start participation and nurthachievement (p=.25). Children who attended Head Start and children with no preschool experience did not significantly differ on math achievement scores. However, there was a significant relationship between other preschool participation and math achievement (p=.02). Children who attended a preschool other than Head Start had higher math achievement scores than children who attended Head Start and children who had no preschool experience. Children who attended other preschools 86 Table 17 whom Main Effect for Math Achievement (N=271) Fixed Effect Coefficient Se g-test p Overall Math Achievement 33.68 5.78 5.82 .002 Preschool Main Effect 8.33 3.80 2.20 .05 Random Effect Variance SD d.f. x2 p Between Schools 167.40 12.94 5 29.34 .0001 Preschool slope 17.97 4.24 5 5.66 .34 Within subjects 694.64 26.36 87 Table 18 Egghml Main Effg for Reading Achievement (N=271) Fixed Effect Coefficient Se _t_-test p Overall Math Achievement 33.00 5.03 5.23 .001 Preschool Main Effect 6.90 3.47 1.99 .06 Random Effect Variance SD d.f. x2 p Between Schools 122.91 11.08 5 24.23 .0001 Preschool slope 12.08 3.48 5 5.20 .39 Within subjects 692.09 26.31 88 Table 19 Preschool Main Effect for Math Achievement (N=271) Fixed Effect Coefi'icient Se L-test p Overall Math Achievement 33.41 6.39 5.23 .003 Head Start Main Efi‘ect 3.45 4.25 .79 .25 Other Preschool Main Efi’ect 11.71 3.85 3.04 .02 Random Effect Variance SD d.f. x2 p Between Schools 212.24 14.57 5 79.32 <.0001 Within subjects 692.09 26.31 89 had an 8 point advantage (11.71 - 3.45) over the Head Start children and close to an 12 point advantage over thechildren with no preschool experience. As the literature suggested, preschool participation did have a positive impact on children’s math achievement score. This effect continued through the end of kindergarten. For reading achievement, the results were the same (see Table 20). There was not a significant relationship between Head Start participation and reading achievement (p=.33). Children who attended Head Start and children who had no preschool experience did not differ significantly on reading achievement scores. However, there was a significant relationship between other preschool participation and reading achievement. Children who attended a preschool other than Head Start were at a distinct advantage. These children scored higher on reading achievement than children in Head Start and children who did not attend Head Start. The children who attended other preschools had a 10 point advantage in reading achievement over children who did not attended a preschool and a 9 point advantage over children who attended Head Start. At this point, it would be appropriate to take a closer look at preschool effects. Table 21 illustrates the sample size, means and standard deviations of the achievement scores of children who attended preschools (Head Start, Other preschools) and those who did not. As stated above, children who attended other preschools performed better on both math and reading achievement tests than children who did not. An examination of the pretest scores on math achievement by a one-way AN OVA and the Suldent-Newman-Keuls post hoc test revealed similar relationships at Table 20 Preschppl Main Effects fpr Reading Achievement (N=271) Fixed Effect Coefficient Se y-test p Overall Reading Achievement 32.78 5.74 5.72 .003 Head Start Main Effect 1.57 4.07 .39 .33 Other Preschool Main Effect 10.28 3.59 2.86 .02 Random Effect Variance SD d.f. x2 p Between schools 168.85 12.99 5 68.50 .0001 Within Subjects 603.71 24.57 91 Table 21 Preschool Mean Scores and Standard Deviations for Math and Reading Achievement Math Reading No No Preschool Preschool Preschool Preschool N 141 130 141 130 M 39.27 35.27 36.84 34.05 SD 29.94 29.52 28.82 25.31 Math Reading Head Other No Head Other No Start Preschool Preschool Start Preschool Preschool N 53 88 130 53 88 130 M 34.89 42.86 35.27 33.85 38.65 34.04 SD. 27.34 31.17 27.34 25.76 30.52 25.31 92 Table 22 School by Preschool Means and Standard Deviations for Reading Achievement (N=204) Schools Preschool 1 2 3 4 5 6 N 24 31 19 19 6 21 Pretest M 1.79 20.23 15.79 8.84 15.00 25.24 SD .98 16.22 14.91 7.40 17.94 26.56 Posttest M 38.67 20.06 24.89 57.58 51.17 56.24 SD 30.95 27.39 17.01 26.47 24.26 22.96 No Preschool N 4 9 16 13 27 15 Reading M 1.50 18.44 15.19 9.00 14.67 8.33 SD .58 6.04 16.38 10.34 17.40 1 1.09 Reading M 32.0 22.44 25.00 53.00 40.89 31.80 SD 23.87 30.60 14.87 29.17 25.05 24.04 93 Table 23 h l l M S ° tions f r Math Achievement Schools Preschool 1 2 3 4 5 6 N 24 31 19 19 6 21 Pretest M 2.0 12.89 14.31 4.38 3.48 3.87 SD .82 10.90 12.00 7.29 4.07 3.56 Posttest M 38.00 16.33 25.81 43.31 45.07 27.07 SD. 35.38 12.98 22.98 28.37 30.31 26.08 No Preschool N 4 9 16 13 27 15 Pretest M 2.25 16.55 13.32 3.21 3.33 9.62 SD 1.26 15.67 7.27 2.74 2.42 8.82 Posttest M 61.79 19.74 29.37 40.21 56.17 49.95 SD 27.10 22.70 20.39 26.64 20.89 22.96 Table 24 School by Preschppl Means and Standard Deviations For Reading Achievement and 94 Math Achievement (Posttest only Scores ) (N=271) Schools Reading Preschool 1 2 3 4 5 6 N 30 35 24 21 7 24 M 36.8 18.97 28.42 52.52 50.57 53.62 SD 28.5 26.12 21.46 29.79 22.20 22.96 No Preschool N 21 16 20 18 33 22 M 25.57 16.94 28.20 52.83 40.52 34.82 SD 19.7 24.53 17.4 29.16 23.25 25.5 Math Preschool N 30 35 24 21 7 24 M 64.63 18.57 32.58 37.29 51.71 46.04 SD 25.86 21.71 24.85 27.08 22.42 46.04 1 No Preschool N 21 16 20 18 33 22 M 51.14 16.19 22.70 41.56 44.94 25.77 SD 32.21 15.51 21.62 28.86 31.85 24.35 95 pretest (see Tables 22-24). Children who attended preschools other than Head Start had significantly higher math achievement scores upon entering schools than children who attended Head Start or children who had no preschool experience. However, in reading achievement, children enter school virtually the same (p=.66) regardless of preschool experience. To further explore the differences between children who attended Head Start and children who attended a preschool other than Head Start, the results of cross tabulations by preschool and ethnicity revealed that of the 28 Caucasian students in the sample, 39% attended other preschools, 11% attended Head Start and 50% of these students did not attended preschool There were a total of 237 African American children in the same. Thirty one percent of the African American children attended a preschool other than Head Start, 21% attended Head Start, and 48% attended no preschool. (There were only 2 Native American children and 4 Hispanic children in the sample and they were evenly divided between preschool and no preschool. None of these children attended Head Start). This result indicated that the proportion of Caucasian children in any preschool group did not significantly differ from the proportion of Afiican American children. A chi-square analysis revealed that there were no significant difi‘erences between these groups (152 (6, N=271) =1.88, p=.39). In addition, a preschool by gender cross tabulation revealed that of the 143 females in the sample, 20% attended Head Start, 29% attended a preschool other than Head Start, and 50% of the females attended no preschool. Of the 128 males in the sample, 19% attended Head Start, 36% attended some other preschool, and 45% did not attend 96 preschool. A chi-square analysis revealed that there were no significant differences between these two groups (x2 (2, N = 271) = 1.33, p=.51). Hyppflis Fpur: Transition Prom and Preschool Interaction Effect An hierarchical linear models regression analyses was performed in which preschool was entered as a level-1 predictor and program was entered as a level-2 predictor in order to test the hypothesis that there would be a program x preschool interaction. This hypothesis stated that the difference in the means scores between no preschool children and the preschool or Head Start children would be greater for children who participated in the transition program than for children who did not participate in the transition program in both reading and math achievement. The results reported in Table 25 indicated that there was not a significant main effect for program, that is, there was not a significant relationship between math achievement and program participation (p.=.29). This confirmed the result of the zero order equation of program effect. In addition, there was not a significant main effect for preschool participation. Children who attended the program schools did not significantly differ, on the average, in math achievement from children who did not attend program schools. There was not a significant relationship between math achievement scores and the interaction between program and preschool participation. For reading achievement the same analysis was performed. The results reported in Table 26 revealed that there was a significant negative program main effect (p=.02). Children who participated in the program had significantly lower achievement scores than children who did not. Children in the program scored, on Table 25 Prom and Preschool Interaction Effect f0; Math Achievement (N=271) Fixed Effect Coefficient Se L—test p Overall math Achievement Interc. 37.15 9.60 3.87 .01 Program Main Effect -7.39 13.66 -.54 .29 Preschool Main Effect 8.29 5.08 1.63 .09 Program-Preschool Interaction .37 6.80 .05 .35 Random Effect Variance SD d.f. x2 p Between Schools 246.79 15.71 4 73.84 .0001 within Subjects 700.40 26.47 98 Table 26 whoa] and Prom Interaction for Reading Achievement (N=27” Fixed Effect Coefficient Se t-test p Overall Reading Achievement Intercept 42.06 3.84 10.96 .0001 Program Main Effect -18.59 6.15 -3.02 .02 Preschool Main Effect 10.42 4.68 2.23 .04 Program-Preschool Interaction -6.08 6.32 -.96 .19 Random Effect Variance SD d.f. x2 p Between School 18.74 4.32 4 9.44 .05 Preschool Slope .83 .91 4 4.50 .34 Within Subjects 611.48 24.72 99 average, 18 points less on reading achievement tests than children who were not in the program. This program main effect controlling for preschool effects was significant in the interaction equation but only approached significance in the zero order equation. The difierence in results could be that the interaction equation had fewer degrees of freedom. With fewer degrees of freedom and more independent variables in the equation, there was greater the probability that spurious significance could occur (Cohen & Cohen, 197 5). This equation did report a significant main preschool effect controlling for program effects. Children who attended preschool, whether Head Start or some other preschool, had higher reading achievement scores than children who did not attend preschool. There was a significant other preschool effect in the zero order equation. It could be that the other preschool makes up most of the variance in this effect. There was not a program and preschool interaction for reading achievement. Hyppthesis Five: Preschool and School environment interaction An hierarchical linear regression equation was performed in which preschool was entered as a level-one variable and school environment was entered as a level-two variable in order to examine the preschool x school environment interaction. The hypothesis stated that there would be a significant interaction effect associated with exposure to the Transition Program and exposure to different school environments. The difference in mean reading and math scores between children who had participated in the Transition program and those who did not participate would be greater for those who attended schools with higher effectiveness scores than children who attended less effective schools. The results reported in Table 27 indicated there 100 Table 27 Preschool and Schgl Enym’ nment Interactipn for Math Achievement (N=271) Fixed Effect Coefi'icient Se g-test p Overall Math Achievement Intercept 95.34 110.40 .86 .21 School Environment Main Effect -.22 .40 -.56 .28 Preschool Main Effect 69.24 64.33 1.08 .17 Preschool-School Environment Interaction -.22 .23 -.95 .20 Random Effect Variance SD d.f. x2 p Between School 191.69 13.85 4 27.47 .0001 Preschool slope 11.65 3.41 4 4.62 .33 Within Subjects 695.98 26.38 101 Table 28 School Environment Pro Interaction for Readin Achievement =27 l Fixed Effect Coefficient Se g—test p Overall Reading Achievement Intercept -26.58 72.40 -.37 .32 School Environment Main Effect .28 .27 1.03 .18 Transition Program Main Effect -22.62 4.75 -4.76 .005 School Environment x Program Interaction Effect -1.00 .54 -l.84 .07 Random Effect Variance SD d.f. x2 p Between Schools 10.93 3.31 2 3.59 .16 Within subjects 620.37 29.91 102 were no significant effects. There were no significant differences between overall math achievement intercepts (p=.21), no relationship between math achievement and school environment (school environment main effect) controlling for preschool (p=.28), and no relationship between preschool and math achievement controlling for school environment (p=. 17). In addition, there was no preschool-school environment interaction (p=.20). The zero order equation indicated that there was a significant relationship between preschool and math achievement but not school environment main effect. For reading the results were reported as Table 28 illustrated. No significant effects were revealed. There was not a relationship between reading achievement and school environment controlling for preschool and the preschool and school environment interaction (p.20). There was no relationship between preschool participation controlling for school environment and the preschool and school environment interaction (p.29). In addition, the results revealed that there was not a significant interaction between preschool and school environment (p=.3l). This results contradicted the earlier findings of preschool main effects for reading achievement. The zero order equation indicated a relationship between preschool and reading achievement which approached significance. Hymthesis Six; Program gig Environment Interaction controlling for Pretest An hierarchical linear regression analysis in which program and school environment were entered into the equation as level-2 variables and the product of program and school environment was entered into the equation as a third level-2 variable in order the examine the interaction between program and school 103 environment. (The program and school environment interaction variable was a the product of the deviation score for school environment and the mean score for program). The hypothesis stated that there would be a significant interaction associated with exposure to a transition program and exposure to different school environments. The difference in mean reading and math scores between children who have participated in the Transition program and those who do not participate would be greater for those who attended schools with higher effectiveness scores than children who attended schools with lower scores. For math achievement, the results reported in Table 29 indicated that there was not a significant relationship between math achievement and school environment controlling for u-ansition program and the school environment (p=.33). There was not a significant relationship between math achievement and transition program conuolling for school environment and the school environment and transition program interaction (p=.35). In addition, there was not a significant interaction between program and school enviromnent (p=.35). For reading the results reported in Table 29 indicated that there was not a significant relationship between school environment and reading achievement controlling for transition program and the school environment and program interaction (p=. 18). However, the analysis did reveal a significant relationship between transition program and reading achievement (p=.005). Children in the transition program schools scored significantly lower than children in the non-program schools. The zero order equation for program participation approached significance (p=.07). In addition, Table 29 Pr and School Environment In 104 'on for Math Achievemen =271 Fixed Effect Coefficient Se y-test p Overall Math Achievement Intercept 38.08 1 1.88 3.20 .02 School Environment Main Effect -.33 1.10 -.30 .33 Transition Program Main Efi‘ect -2.69 19.64 -.14 .35 School Environment-Transition Program Interaction Effect .11 2.20 .05 .35 Random Effect Variance SD d.f. x2 p Between Schools 420.68 20.51 2 60.09 .0001 With Subjects 712.58 26.69 105 there was a school environment and program interaction which approached significance (p=.07). Ninety four percent of the variance between school was accounted for by this equation in which there was a significant program effect and near significant school environment and program interaction. Results 8mg The analysis of math and reading achievement revealed that mean scores did significantly differ between schools. For math achievement, the differences were accounted for by preschool effects only. There was a significant positive relationship between math achievement and preschool participation. Children who attended preschool had higher achievement scores than children who did not attend preschools. More specifically, children who attended a preschool other than Head Start had higher achievement scores than either those children who attended Head Start or those children who did not attend a preschool. There were no significant interactions between program and preschool, preschool and school environment, or program and school environment. The results were difi’erent for reading achievement. There were significant differences between school mean achievement scores as with math achievement, however, transition program main effect, preschool main effect, and a school environment and program interaction accounted for some of the differences between mean SCOTCS. DISCUSSION The results from this research confirm that schools do matter. For each analyses there was a highly significant amount of between school variance. This establishes that the academic achievement scores of children are different depending upon the school in which they attend. This point is only strengthen when one considers that the salnple is relatively homogenous based on age, race, and SES. Schools do differ in how well they educate children. How schools impact children differently, and even why schools impact children differently is still unclear, but maybe this research has provided some insight. Hopefully, it will have answered some questions such as what happens to children’s academic achievement after leaving preschool and entering school? Do school programs enhance or diminish academic achievement? What type of environment are the schools providing for children and what are the effects of this environment on children’s academic achievement? Will school environment support the gains made by children who have attended early childhood education programs or will they negate those gains? How does the school environment mediate preschool effects on academic achievement? The first question answered in this study is does the Head Start/Early Childhood Transition Program effect children’s academic achievement? The findings suggest that the program has a negative impact on children’s reading achievement. This finding is unfortunate and contradicts the hypothesis stated earlier that children will benefit as 106 107 transition program participants. The studies in the research literature suggest that children often benefit from continued and comprehensive services beyond preschool. Zigler & Styfco (1993) noted in their review of such programs the Abecedarian Project and the Chicago Child-Parent Centers that with continued and comprehensive services, beyond preschool into the third grade children showed gains in academic achievement above that of the comparison groups which lasts for a substantially longer period of time. This does not happen in this study. There may be several reasons for these negative effects, some of which are discussed below. First, many environmental and situational influences should be considered when interpreting these results. This research reflects data collected dming the implementation year of a 3 year grant. Several significant events occurred dming the year which might have hampered program implementation and, therefore, hampered positive program effects. To begin with, the district had recently hired a new superintendent who tried to make abrupt and sweeping changes. The political fall-out from this superintendent’s action was great. The school district stayed embroiled in controversy and public and private debates. This set a pessimistic tone throughout the district. In addition, the school district operated under severe budget constraints. Many teachers were laid ofi or were under constant threat of being laid off. More than half way into the school year, 3 principals were laid off in the district’s six elementary schools. The three remaining principals were assigned an additional school besides the one they had. Not only did some teachers have to adjust to the new curriculum and teacher style but had to adjust to a new principal, the second in less 108 than a year for many of thorn. It was a difi’icult, chaotic year for the district. At the same time, transition program teachers were introduced to the new program in which they were asked to implement a developmentally appropriate curriculum in their classrooms, and were expected to comply with the new standards within a relatively short period of time. Having to make the transition from their traditional teaching style to that of one which was developmentally appropriate would take time. Teachers had to not only change their cm'liculum but had to adjust to the changes as well as become comfortable and feel competent about making those changes. It was probably not until the end of the year that the changes were substantive and teachers could feel comfortable using the new curriculum As teachers were adjusting, children might have been adjusting as well. The negative relationship between program participation and academic achievement might well be related to this transition where teachers were changing directions, more than likely, in the middle of a school year. It would seem that children would be adversely impacted until teachers became comfortable with the new cuniculum and able to provide instruction and direction with relative ease and assurance. Other classroom specific factors should be considered when interpreting these results as well. For instance, school 2, a transition program school, had lower academic achievement scores than any other school in the sample. This result may not only represent the transition from traditional classroom practices to a more developmentally appropriate curriculum but may be a reflection of personnel changes in this school. At the beginning of the year, the kindergarten teacher was often ill and 109 frequently absent. Consequently, she took a leave of absence and was replaced by a middle school spanish teacher, but not before several substitute teachers were placed in the classroom. This former spanish teacher faced many challenges including learning a new curriculum, teaching young children, and all of the concomitant issues associated with this change. Children’s academic achievement performance could certainly be negatively affected by these changes. A closer look at school 1 may provide insight as well. School 1 had significantly higher math and reading scores on average than schools 2 and 3, the other transition program schools. School 1 had only one kindergarten teacher for 2 kindergarten classes. As opposed to school 2, many classroom factors remained stable throughout the school year. These classes had the one teacher the entire the year, a teacher who appeared experienced and competent. She was reported to be somewhat resistant to the transition program initially. Eventually, however, she did incorporate the curriculum into her daily activities. Her initial resistance may be a factor which would help explain the higher scores for the children she taught. It may be that by maintaining a traditional classroom for a longer period of time gave her the ability to teach those skills necessary for children to obtain higher achievement scores. She might have been teaching children in a manner proved successful before. Regardless of the crn'riculum being offered, it secured that teachers were always concerned about children’s performance on academic achievement tests. They would often try hard to prepare children for this test, especially as the end of the year grew closer. This may have been true for the teacher in school 1 as well. 110 School 4 had the highest reading achievement score on the average than other non-transition schools. This classroom was different than any other kindergarten classroom in the district because it was the only full day kindergarten. Children had more time to learn reading skills. In addition, the classroom remained relatively stable, having only one teacher who seemed both competent and experienced. Her classroom techniques were quite similar to the developmentally appropriate techniques being implemented in the transition schools. She was committed to allowing to the learn experientially rather than in a more traditional way. An additional explanation for this negative effect might be the nature of a developmentally appropriate curriculum itself. The style and techniques used in teaching were often quite different from a traditional cuniculum The developmentally appropriate curriculum stressed child-initiated experiences that met children’s needs and stimulated learning in all developmental areas including physical, social, emotional, and intellectual. Teachers were to prepare the environment for children to learn through active exploration and interaction with adults, other children, and materials (National Association for the Education of Young Children, 1986). Such a curriculum did not view standardized tests as an appropriate tool for assessment of children’s growth and development and therefore, would not "teach to the test." On the other hand, teachers in more traditional roles may narrowly focus on the child’s intellectual development having a highly structured, teacher-directed orientation. They would often prepare children to perform well an standardized test. Children who attended developmentally appropriate classrooms may not perform as well on lll standardized tests as would children in more traditional classrooms because they had not been directly instructed as other children may be. Whatever the reason, children in the transition program during this implementation year did not performed as well as those children who were not in the transition program. One would hope that this effect was temporary. As the program becomes more stable, as teachers become more efficient and competent, and as children become comfortable in their settings, the program should begin to have a positive impact on children’s academic achievement. What the above discussion did was to expose the need for examination of classroom and teacher variables in school environment research. Even though the transition program has a negative impact on children’s academic achievement, it is encouraging to know that children who have preschool experience have higher reading and rmth achievement scores than children who do not. It was even more encouraging to know that these gains persisted at least throughout kindergarten. Previous studies provided evidence for both of these findings and supported the hypothesis for this study. Preschools had immediate impact upon the academic achievement of children although these gains usually faded out soon after the children enter school (Datta, 1979; Haskins, 1989; Schweinhart, et al., 1993; Washington & Omeyade, 1987; Zigler & Styfco , 1993). Further examination of preschool effects revealed that children who attended other preschools perform better on academic achievement tests than children who attended Head Start. There may be several reasons for this difference. First, there 112 may be differences between children who attended Head Start and those who attended other preschools. Though there were not significant differences in children’s ethnicity and gender, there may be differences in socio—economic status. Although the majority of the children in this school district were low-income, children in Head Start should represent children who were from the lowest socio-economic background. Head Start had a requirement in which they must accept the children with the greatest need first. Many children were not enrolled in Head Start because they did not meet this eligibility requirement. Thmefore, children from more afi‘luent families may be represented in other preschools. The other preschools might provide a difierent preschool experience for children. This difference could be defined, in one way, by teacher experience and education. There were preschool teachers in these area preschools who had undergraduate and post graduate degrees in early childhood education or other relevant fields. In addition, many of them had many years of experience. Head Start had no degree requirement and only required that their lead teachers have 12 hours of early childhood education course work completed. They also required a child development associate credential which represented some coursework and classroom experience. In addition, the differences in experiences might be defined by the goals of each program. A primary goal for Head Start was to build children’s social competence, to prepare children for successful social interactions in a school setting, and thereby, enhancing children’s ability to relate to others. Academic achievement may sometimes be a secondary goal. Other preschools might have these same goals, 113 however, emphasized these goals differently. There might be a greater emphasis on preparing children academically for kindergarten in other preschools than in Head Start. Therefore, it seemed likely that children who attended preschools other than Head Start whose primary focus was to prepare children academically for school would do better in academic achievement than children who attended Head Start. The school environment findings or the lack thereof must be examined in the light of the extremely low reliability estimate given in early in this discourse. This suggested that the measure used was not an adequate indicator of school environment. There was much more variance within schools than between schools. This indicated that teachers within a school were not in agreement about the school’s environment. Therefore, one must consider if the school effectiveness instrument really measured school environment or was it something else. Why would some teachers within the same school rate the school high on effectiveness characteristics while other teachers rate the school very low? Did teachers have different experiences within the same school? Did teachers perceive the same experiences differently? Whatever the school situation, it seemed as if this measure did not reflect school environment well. The school environment should be consistent and teachers should be rating each school similarly. However, during such a chaotic year, it would not be hard to see how teachers feelings could have fluctuated and changed based on their experiences. During this year, 3 of the 6 schools had new principals. These three new principals were soon laid off shortly after the new school year began. School environment may change with the arrival or departure of a new principals. After these principals were 114 laid off, the existing principals of the 3 other schools were given responsibility of one additional school. Therefore, 3 principals were responsible for 2 schools each. At the timethatthismeasurewascollected, theprincipalshadonlybeeninchargeoftheir new, additional school for three months. It may be probable to believe that teachers might be responding to these changes, unsure of what to make of the school environment given so many changes. Whatever the case, the results from analyses of school environment must be interpreted with great care and caution. The school environment (school effectiveness) literature suggests that there is a relationship between it and academic achievement. Research indicates that factors such as instructional leadership, high expectations, opportunity to learn, classroom practices, and others, are related to academic achievement (Bedford, 1988; Brookover, et al., 197 9; Walsh & Walsh, 1990). However, this study does not support those findings and or the hypothesis that there will be a school environment effect. The results in this study indicate that school environment has no direct relationship to math and reading achievement. The school environment measure had seven subscales. Based on the results of a factor analysis, the decision was made to combine these scales into one school effectiveness score. It is possible that are relationships between academic achievement and individual scales. However, the analysis did not permit this discovery. Subsequent research might examine these scales. There is an interesting finding of a interaction between school environment and the transition program for reading achievement when controlling for the school 115 environment main efiect and the transition program main effect. (This results must be interpreted with extreme caution because the lack of power from the analysis with only 2 degrees of freedom). What this indicates is the transition program effect is dependent upon the school environment. This may be a clue to suggest that the school environment may well mediate the effects of school programs. Unfortunately, the effect is negative. One can only hope that a better school environment will mediate theeffects ofapoorlyirrrplemcntedprograrmthatis, theprogramwillnotbcas detrimental if it operates within a school which has good instructional leadership, high expectations of students, a safe and orderly environnrent, good home-school relations, as well as children are given an opportunity to learn. This seems to be the relationship with these schools. A positive school environment reduces die negative impact of the transition program. It suggests that what happens in schools do impact children for the better or worse. At least for kindergarten children, the school environment seem not to be significantly relevant to what is occurring in the classrooms. In many of the studies regarding school effectiveness, the relationship between academic achievement and school environment is based on the mean achievement for the entire school. This study uses the mean achievement for kindergarten children only yet collects data from all the teachers in each school. School effectiveness or school environment may well be related to academic achievement when measured across all grade levels. However, for this study, school environment has little to do with the academic achievement for kindergarten children. This may suggest that kindergarten children may be isolated or 116 even insulated from the effects of the school environment. As mentioned above, teacher variables might be more important factors to consider when solely discussing kindergarten children. Research Limitations Sample size was a major factor which could have seriously effected the results of this research. There were only six schools from which to examine between school variance. Consequently, there was a lack of statistical power, and, therefore, the results could be tenuous and, at times, spurious. In addition, the generalizability of the findings were questionable because of this lack of representative sample of schools. As does so many other studies regarding school effects, this study narrowly defined academic achievement as results on standardized test scores. This was a limitation to this study. Other measures of academic achievement could have provided a more comprehensive examination. Such factors included social-emotional competence of children, teacher ratings, parent assessment of children’s academic achievement as well as children’s self-competence. Another limitation of this study was that classroom level variables or teacher characteristics were not measured. What teachers did in the classroom was ultimately important to program implementation and to children’s academic achievement. Future research should incorporate variables that assessed the classroom processes and events which could impact academic achievement. Teachers characteristics should be assessed as well. Their background characteristics such as ethnicity, socio-economic status, and gender could be related to both school environment effects and classroom 117 effects. This study was not longitudinal. A longitudinal study could assessed change and growth over time. It would be important to know how the nature of the program and its impact on children change. Would the results be consistently negative over time or will children improve to a point in which they achieve at higher levels than the non- transition group? Without a longitudinal design, these questions could not be answered. CONCLUSIONS This research represents a multi-level analysis of the combined effects of the three independent variables upon the academic achievement of children in kindergarten. This study points to the need continued research which incorporates a multi-level, ecological framework to understanding academic achievement. Children are entering school with various experiences and skills. Once they enter school their experiences differ as well depending upon the school they attend and the classroom to which they are assigned. How to capture these experience and relate them to academic achievement can probably be best understood in a framework which allows the research to view these experiences as a whole. Moreover, this research underscores the importance for immediate and thorough evaluations of new programs which are intended to support children and have a positive impact upon their academic achievement. There is probably a plethora of programs implemented within school systems on a consistent basis which have not been well researched or thoroughly evaluated and many of which may be doing more harm than good. Such programs should not be tolerated. There should be enough research, however, to support the need for continued services for children and their families. How the changes are implemented in the schools should be a primary concern. It may be that there are better ways of helping schools and teachers make this transition. More planning and a development time may be needed before the start of a new program. Even pilot 118 119 programs within a school might help. Piece-meal, quickly and haphazardly implemented programs should not ever be substituted for well researched, well designed, well implemented, and well managed programs. In addition, it underscores the necessity of preschool experience especially for minority, low-income children and the need for continued services beyond preschool. Children can enter school with an academic advantage. However, these advantages will not last if schools do nothing to enhance and promote these gains. Continued support is necessary for children in order to maintain these goals, but how will children be supported? The perpetual cycle of inadequate support and then school failure must stop. Systemic, lasting changes must be address at the classroom and school levels where all children can benefit from quality education throughout their academic careers. This research supports the conclusion of the school effectiveness literature which suggests that schools make a difference. Children’s academic achievement can be depends upon the nature of school they attend regardless of their previous or current programmatic educational experiences. However, this research may have provided evidence that subsequent research look more closely at teacher and classroom variables. Teachers may have the ultimate power to impact academic achievement. Teacher may be effected by the school environment, which may somehow influence their behavior and therefore, their teacher. However, teachers may more directly impact children’s academic achievement. Appendix A 120 «aaafi $5 $8 $8 efi *8 sum e: *8 620m .3: m :55; Beam m .30.» .59 M 83; is 32 $2.. $2. *6 5n 8% sun 88 can madam BEBE . 35 gm H emu es 3% eon *8 $2 «an .3? 628 «3:6 as? e: an $8 $3 $8 *3 ea .5 0325:. 82 s8 s8 .3 a: .3 so so .3 808a... 232 as em a... as .5 so so as ofinam 6.5% e3 $8 $2 e8 3: $8 e8 e8 5:25. 5.5. bug 8338 an. an an an as emu emu «a 3% Sam 38. e 338 n 693 v Snow 38. m Saw a 328 a 320m 33> assuage 383m 8 Bass: cause 32% Appendix B ScalL Mathematical Concepts Language Memory Visual Auditory Print Concepts Pre-reading Total 121 D l . 51']! Cl 11' Scale Reliabilities New 37 29 29 16 21 21 116 .89 .89 .91 .64 .86 .82 .94 Appendix C 122 THE CONNECTICUT SCHOOL EFFECTIVENESS QUESTIONNAIRE CONNECTICUT STATE DEPARTMENT OF EDUCATION PILOT EDITION ALL RIGHTS RESERVED NOT TO BE REPRODUCED WITHOUT PERMISSION Principal Author: Robert M. Villanova Co-Authors: William J. Gauthier, Jr. C. Patrick Proctor Joan Shoemaker Revision Authors: Hilary E. Freedman Richard E. Lappert Holden T. Waterman Copyright 1981 Revised October 1989 123 The Connecticut School Effectiveness Questionnaire The purpose of the questionnaire is to survey your perceptions of this school based on your experiences. There are no right or wrong answers. We are interested in how you feel. ’ There are 97 items in this questionnaire in which you are asked to indicate the degree to which you agree or disagree with the statements by circling a number from 1 to 5. I indimtos strong disagreement with the statement and 5 indicates strong agreement with the statement. Please circle only one number for each answer. Answer each question as best you can according to your own experience. Please answer this questionnaire on your own without consulting with others. It is important that we know your perceptions. It should take you approximately 30 minutes to complete this survey. To ensure confidentiality, please do not write you name anywhere on the questionnaire. ‘ 10. 11. 13. 14. 15. 124 Most students in this school are eager and enthusiastic about learning. In reading, written, sequential objectives exist in all grades The principal is highly visible throughout the school. In this school low-achieving students are as well-behaved as other students. Pull-out program (e.g. Chapter 1, special ed., instrumental music, etc.) do not disrupt or interfere with basic skills instruction. Achievement test scores are analyzed separately for subgroups of students (e.g., gender, race, ethnicity social class, etc.) to assure that all students are achieving. Most parents understand and promote the school’s instructional program. Beyond parent conferences and report mrds, teachers in this school use other ways of communicating student progress to parents (e.g., home visits, phone calls, newsletters, regular notes). Many students received honor and recognition for academic performance. Factors outside the classroom rarely interfere with instruction in this school. In reading, instruction is often presented to a heterogeneous ability group of students. ' Individual teschers and the principal meet regularly to discuss what the principal will observe during a classroom observation. Mathematies objectives are coordinated and monitored in all grades. Teachers, administrators, and parents work cooperatively to support the discipline policy in this school. Student behavior is generally positive in this school. Strongly Disagree Strongly Agree 16. 17. 18. 19. 20. 21. 22. 23. 24. 25. 26. 27. 28. 29. 30. 31. 125 Reading objectives are coordinated and monitored in all grades The principal is accessible to discuss matters dealing with instruction Teachers in this school believe they are respomible for all students mastering basic skills at each grade level Special instructional program for individual students are integrated with classroom instruction and the school curriculum Teachers and the principal thoroughly review and analyze test results to plan instructional program Parents support the school in matters of student discipline Parent-teacher conferences result in specific plans for home-school cooperation aimed at improving student classroom achievement Criterion-referenced tests are used to assess instruction throughout the school Teachers implement the homework policy in this school In this school, there are clear guidelines for grouping students for instruction The principal makes formal classroom observations In mathematics, written, sequential objectives exist in all grades A positive feeling permeates the school Students in this school abide by school rules The principal is an important instructional resource person in this school Teachers believe that all students in this school can master basic skills as a direct result of the instructional program 32. 33. 37. 38. 39. 41. 42. 43. 45. 47. 48. 126 There are few interruptions due to discipline problems during class time Students have many opportunities to demonstrate talents in art, music, drama, dance, and athletics Parents support the homework policy in this school During parent-teacher conferences there is a focus on factors directly related to student achievement Multiple indicators are used regularly to assess student progress (e.g., grades, tests, attendance, discipline referrals, extracurricular, etc.) Two hours or more are allocated for reading/language arts each day throughout this school Supervision is directed at instruction In language arts, curriculum objectives are the focus of instruction in this school This school is a safe and secure place to work This school has a written statement of purpose that is the driving force behind most important decisions Teachers in this school turn to the principal with instructional concerns or problem The principal regularly brings instructional issues to the faculty for discussion This school has successful preventive strategies for helping students at risk of school failure Students are absent from school only for good reasons There is an active parent/school group in this school Ninety to one hundred percent of your students’ parents attend scheduled parent-teacher conferences Many parents are involved in school activities Strongly Disagree Strengly Agree 49. 50. 51. 52. 53. 54. 55. 56. 57. 58. 59. 61. 62. 63. 127 During classroom instruction students do not work independently on seatwork for the majority of the allocated time Student have many opportunities to demonstrate leadership skills The principal regularly gives feedback to teachers concerning lemon plans In language arts, there is an identified set of objectives that all students must master in all grades The school building is neat, bright, clean and comfortable All materials and supplies necessary for instruction are available The principal makes informal contacts with students and teachers around the school In this school, remedial program are a last resort The number of low-income children promoted is proportionately equivalent to all other children promoted Parents of students in your class have regular opportunities to observe the instructional program There rs cooperation with regard to homework between parents and teachers in mi school The school’s daily schedule supports the goals of the instructional program Teachers believe that a student’s home background is M the primary factor that determines individual student achievement in this school The principal requires and regularly reviews lesson plans The principal reviews and interprets test results with the faculty Language arts objectives are coordinated and monitored in all grades The discipline policy is consistently enforced by all staff in this school Smarty Disagree Strongly Agree 67. 69. 70. 71. 72. 73. 74. 75. 76. 78. 79. 128 In language arts, written, sequential objectives exist in all grades The principal leads frequent formal discussions concerning instruction and student achievement In language arts, instruction is often presented to a heterogeneous ability group of students ' Many parents initiate contacts with school each month In this school, all teachers apply consistent criteria to assigning course grades Fifty minutes or more are allocated for mathematics each day throughout this school Ninety-five to one hundred percent of the students in this school can be expected to complete high school At the principal’s initiative, teachers work together to coordinate the instructional program within and between grades In reading, curriculum objectives are the focus of imtruction in this school The principal frequently communicates to teachers their responsibility in relation to student achievement Almost all the students in this school try hard to get good grades Discussions with the principal often result in some aspect of improved instructional practice During follow-up to formal observations, a plan for improvement frequently results In mathematics, instruction is often presented to a heterogeneous ability group of students The testing program is an accurate and valid measures of the curriculum in this school 81. 82. 89. 91. 93. 95. 129 Almost all students complete assigned homework before coming to school Class atmosphere in this school is generally very conducive to learning for all students Formal observations by the principal are regularly followed by a post-observation conference All teachers in this school hold consistently high expectations for all students Parents of students in your class have a significant, rather than a superficial, role in the educational program Most problem facing this school can be solved by the principal and faculty without a great deal of outside help In this school, the primary emphasis is on teaching and learning In mathematics, there is an identified set of objectives that all students must master in all grades The principal emphasizes the meaning and the use of test results Within the classroom, students are assigned to groups for extra help on a temporary basis only In mathematics, curriculum objectives are the focus of instruction in this school The principal ’3 very active in securing resources, arranging opportunities and promoting staff development activities for the faculty During follow-up to formal observations, the principal’s main emphasis is on instructional issues Most parents would rate this school as excellent There is clear, strong, centralized instructional leadership from the principal in this school Swoosh Disagree Shoagly Agree 130 3W! Smash Dingree Agree In reading, there is an identified set of objectives that all students must master in all grades 1 2 3 4 5 Student have many opportunities to demonstrate leadership skills 1 2 3 4 5 Tinnkyouforoompletingthissnrvey. LIST OF REFERENCES LIST OF REFERENCES Bain J G & Herman. J L (EdS) (1990) WWW: W New York: Greenwood Press. Baptiste, Jr., H. P. (1992). Philosophical and conceptual issues related to students at risk. In H. C. Waxman, J. Walker de Felix, J. B. Anderson, & l-I. Prentice Baptiste Jr W learning (p. 11- 16). Newbury Park, CA: Corwin Press, Inc. Bitting, P. F., Cordeiro, P. A., Baptiste, Jr, H. P. (1992). Philosophical and conceptual issues related to students at risk. In H. C. Waxman, J. Walker dc Felix, J. B. Anderson. & H Prentice Baptiste Jr WW2: WW 0! 17- 32) Newbury Park. CA: Corwin Press, Inc. - .. - , . . r ? Paper present at the Annual Meeting of the LAmerican Educational Research Association, San Francisco, CA. (ERIC Document Reproduction Service No. ED 269 474). Brookover, W., Beady, C, Flood, P, Schweitzer, J., & Wisenbaker, J. (1979). School it «.‘t‘tu a'Wr‘ "'ur rm a-ru 'i.' r “Brooklyn NY: Pracger Publishers. Brookover, W. B. & Schneider, J. M. (1975). Academic environments and elementary school achievement. . -. . . - , . .. 91. Bryk. A S & Raudenbush S W (1992) HierarchicdmneaLmadelLAnplications Wildi- Newbury, Ca.: Sage Publications, Inc. Cohen, J. & Cohen, P. (1975). ‘ ' , ° - ' - - Wm. Hillsdale, New Jersey: Lawrence Erlbaum Associates. Chunn, E. W. (1989). Sorting black students for success and failure: The inequity of ability grouping and tracking. In W. D. Smith & E W. Chunn (Eds.,) Black Wrestling: (pp 93- 106) New Brunswick. NJ: Transaction Publishers. 131 132 Clark, D. L., Lotto, L.S., & McCarthy, M. M. (1980). Factors associated with success in urban elementary schools. W 61, 467-470. Cohen, M. (1981). Effective schools: What the research says. Won, 10, 3841. Cole, 0. J. & Washington, V. (1986). A critical analysis of the assessment of the effects of Head Start on minority children. Wmfion, 15,, 91— 106. Coleman. 1 (1990). WWW Boulder: Westview Press. Crisci, P. (April, 1986). An aomo' nment formula that apoho' s the oonolatos of II II 'I I ‘ ,IIlI 5II I' A IIIII II III I _I ' I ’ (’IAIJI I ”(I W A pager Presented at the Annual Meeting of the American Educational Research Association, San Francisco, Ca. (ERIC Document Reproduction Service ED 270 455). Datta, L., McHale, C. & Mitchell, S. (1966-1969). W . O.- I C 0 . Q A I {.‘I.A.II .III I II ”.III 5‘1")! I I I .4 II" I. 5 IIII'I WWW. US. Department of Health, Education, and Welfare. (DI-IEW Publication No. (OHD) 76-30088). Edmonds, R. (1979). Some schools work and more can. W, 2, 28-31. Edmonds, R. (1979). Effective schools for the urban poor. W, 31, 15-18. Edmonds R. & Frederiksen. J R (1979) W a tee .Itr.rt:r,: ‘1 t \ firtt r}. .' r‘ue not. 'e‘,l ' I out ohlltlten. ERIC Document Reproduction Service (ED 170 396). Elkind, D. (1989). Developmentally Appropriate practice: Philosophical and practical implications. W, 11, 113-117. Epstein, A. S., Schweinhart, L., & Bond J. T. (1978). Wool n u I rumpus ’aO‘. ' 'emrH ' rt ”'10! {“1 W Ypsilanti Michigan Gauthier, W. J., Pecheone, R. L., & Shoemaker, J. (1985). Schools can become more effective. WWW. SA. 388-408. Goodlad, J. I. (1984). W- New York: ' McGraw-Hill Book Co. 133 Gordon. C. H. (1984). WWW Paper presented for the 1984 Annual Meeting of the American Education Research Association, New Orleans. (ERIC Document Reproduction Service ED 310 524). Graham, P. A. (1987). Black teachers: A drastically scarce resource. Em_D_elta Wtfifis 598'605. Hallinger, P. & Murphy, J. F. (1986). The social context of effective schools. Americauonmalntfldncation. 9.4.. 328-355. Haskins, R. (1989). Beyond metaphor: The efficacy of early childhood education. Amsrimfimhaloaist 45, 274-282. Haskins, R. (1989). Beyond metaphor: The efficacy of early childhood education. Americanmmaist. 45.. 274—282. Head Start Evaluation, Synthesis and Utilization Project. (1985). Annotated .ContractNo. 105-81-c- 026. (DI-[HS Publication No. (OHDS) 85-31194). Hodges, W. L. & Buzzelli, C. A. (1984). School change and evaluation: Follow Through revisited. WW .4. 163- 194 Jacob, J. E. (1987). Taking the initiative in education: The National Urban League Agenda In W D Smith & E W Chnnn (Eds ). W W (pp. 123-.17) New Brunswick, New Jersey: Transaction Publisher. Jeffrey. J R (1978) WWW III‘II IVIII I I "III_-| ilI .‘IlI_-| "I ._II Columbus, OH: Ohio State University Press. Joint Center for Political Studies (1989) W W. Landham, MD: University Press of America. Kasscbaum, NL. (1994). Head Start: Only the Best for America's Children. WM. 52. 123- 126 Kreft,I.(1990). ‘1': u I" -r. . I. .' .rm 1‘ I. ‘r‘. . ‘1 t 0 W. (Report No. CSE-TR-303). Paper presented at the Annual Meeting of the American Educational Research Association, San Francisco, CA. (ERIC Document Reproduction Service No. Ed 340 721). 134 Lee, V. E., Brooks-Gunn, J., Schnur, E., & Liaw, Fong-Ruey (1990). Are Head Start effects sustained? A longitudinal follow-up comparison of disadvantaged children attending Head Start, no preschool, and other preschool programs. Child Dexelonmem. 121. 495-507. Levine, D. U. & Havighurst, R. J. (1989). W (7th Edition). Boston: Allyn and Bacon. Lezotte. L. W. (1980). W? Paper presented at the Annual Summer Conference of the American Association of School Administrators. (ERIC Document Reproduction Service No. ED 193 762). Mandeville. G K & Heidari K (1988) MmrianchoolJffectixenesusina WWW. Paper presented at the Annual Meeting of the National Council on Measurement in Education, New Orleans, LA. (ERIC Document Reproduction Service No. Ed 303 477). March J & Crisci P E (1991) WWW . - z z ., - - Paper presented at the Conference-Within- A-Convention of the National Council of Professors of Educational Administration at the Annual Meeting of the American Association of School Administrators, New Orleans, LA. (ERIC Document Reproduction Service No. ED 333 545). Marzano, R.J., Guzzetti, B. J., & Hutchins, C. L. (1984). W We: (Report No 143) (ERIC Document Reproduction Service N 0. ED 253 328). Meyer, L. A. (1984). Long-term academic effects of the Direct Instruction Project Follow Through» WM 8.4.. 380-393. Meyers H W & Carlson R (April 1992) W W. Paper presented at the Annual Meeting of the American Educational Research Association, San Francisco, CA. (ERIC Document Reproduction Service No. ED 346 602). Miller, S. K., Cohen, S. R., Sayre, K. A. (1985). Significant achievement gains using the effective schools model. W, 42,, 38-43. Mortimore, P., Sammons, P., Stoll, L., Lewis, D. & Ecob, R. (1988). W. Berkeley, CA: University of California Press. Mutual”, R. 1 (1975). We childm. Cambridge, MA: Ballinger Publishing Co. 135 National Research Council (1993) W W. Washington, D. C.: National Academy Press. Neuield 13.. Et al (1984) WWW—KW cralnationnrmt Hartford, CI‘: (ERIC Document Reproduction Service No. ED 281 290). Neufeld, B. (1985). Wye—MW , : . - - ' . Hartford. CT: (ERIC Document Reproduction Service N 0 ED 275 715). ’ Piotrkowski, C.S., Collins, R., Knitzer, J., & Robinson R. (1994). Strengthening Mental Health Services in Head Start: A Challenge for the 1990s. Amotloah We. 52. 133-139. Porter, AC. & Raudenbush S..W (1987). Analysis of Covariance: Its Model and Use in Psychology Research Watchdogs 3.4.. 383- 392 Purkey, S. C. & Smith, M. S. (1982). To soon to cheer? Synthesis of research on effective schools. Win, 40, 65-69. Ramey, C. T., Bryant, D. M., Campbell, F. A., Sparling, J. J. & Wasik, B. H. Early intervention for high risk children: The Carolina Early Intervention Program. In R. H. Price, E. L. Cowen, R. P. Lorion, & J. Ramos-McKay (Eds), Eonfloen W (p 32-43) American Psychological Association: Washington, D. C. Raudenbush, S ..,W Rowan, B., & Kang, SJ. (1991). A multilevel, multivariate model for studying school climate with estimation via the EM algorithm and application to U S high-school data» W22 1.6. 295- 330 Schmitt. D. R. (1989). WWW Paper presented at the Annual Meeting of the Eastern Educational Research Association, Savannah, GA. (ERIC Document Reproduction Service No. ED 308 230). Schweinhart, L. J. & Weikart, D. P. (1986). What do we know so far? A review of the Head Start Synthesis project. W. 11, 49-55. Schweinhart, L. J. & Weikart, D. P. (1988). The High/Scope Perry Preschool Program. In R. H. Price, E. L. Cowen, R. P. Lorion, & J. Ramos-McKay (Eds), - = - (p.53 - 66). Washington, D. C: American Psychological Association. Shimoni, R. (1990). A histono' al memo w of tho doyolopmont of early ohlldhood' ‘ genius. (ERIC Document Reproduction Service No. ED 334 000). 136 Slavin, R. E, Karwcit, N. L., & Madden, N. A. (Eds) (1989). W ‘llII- ~eIIil-‘1II'VI ‘1 trn,“ I I W Boston: Allyn and Bacon. Smith. W. D. & Chunn. E. W. (Eds) (1989). WWW 21114221121122- New Brunswick: Transaction Publishers. Sprigle, J. E. & Schaefer, L. (1985). Longitudinal evaluation of the effects of two compensatory preschool programs on fourth through sixth-grade students. Watchdogs. 21. 702-708. Squires. D- A. (1980). WWW processes. (ERIC Document Reproduction Service ED 197 486) Stipek, D., Daniels, D., Galluzzo, D. & Milburn, S. (1990). W W Los Angeles. CA: UCLA, Graduate School of Education. (ERIC Document Reproduction Service No. ED 329 373). Stringfield, S. & Teddlie, C. (1988). A time to summarize: The Louisiana school effectiveness study. Win, 16, 43-49. Takanishi, R. & DeLeon PH. (1994). A Head Start for the 2lst Century. Amefioan Waist. 42. 120-122. Teddlie, C., Kirby, P.C., Stringfield, S. (1989). Effective versus ineffective schools: Obscrvable differences in the classroom. W, 21, 221- 236. Teddlie. C & Stringficld S (1993) Wed Whats New York: Teachers College Press Texas Advisory Committee to the United States Commission on Civil Rights (1990). ElCl'llllEl" 'I’Il'°fi"l'l. U. S. Department of Health, Education, and Welfare. (May 1970). Aflatlonaljumey Warm Washinan DC: Office of Child Development. Villanova. R (1984) W I' III! It! I I I It‘.‘tt|t,o,|tt Paper presented at the Annual meeting of the American Educational Research Association, New Orleans, LA). (ERIC Document Reproduction Service No. Ed 247 287). 137 Washington V & Oyemade. U I (1987) W W New York Garland Publishing Waxman, H. C. (1992). Philosophical and conceptual issues related to students at risk. In H. C. Waxman, J. Walker de Felix, J. E. Anderson, & H. Prentice Baptiste, Jr. W (p 1-9) Newbury Park, CA: Corwin Press, Inc. West, C. A. (1985). Effects of school climate and school social structure on student academic achievement in selected urban elementary schools. mm Education. 511.. 451-461. Witte, J. F. & Walsh, D. I. (1990). A systematic test of the effective schools model. Wis. 12. 188- 212. Zigler. E & MuenchOW. S (1992) W W New York: Basic Books Zigler, E. & Styfco, SJ. (1994). Head Start: Criticisms in a Constructive Context. Amenmflmholosist. .42. 127- 132 Zigler, E. & Styfco, SJ. (1993). Using research and theory to justify and inform Head Start exvansion W 1. l- 21 Zigler. 13. & Valentine. 1. (Eds) (1979). W W. New York: Free Press. "ITiliiiiililifliliiiii