momm mo swam or AN msmuus‘m son m QUALflAflVE summon or CLASSROOM mums. ma THE momma mm Dissertation for the Degree of Ph. D. MICHIGAN STATE UNIVERSITY BERT LEE DONALDSON ' T 1974 31s 534‘ It; 1%. H AA. 5715‘, . .3. ~. ';.'V"~‘:'!'"-“' .. “4-,“- )x I» '2 ~ 7,5“ g. F? W3 mitigan SM ff: 3% {Jim'ersity - I 1‘ ' 1r ”r‘ra'r'VafirflLmfim345 "P. ,1. .k 3mg. " .3 t This is to certify that the \ ' thesis entitled ' DEVELOPMENT AND EVALUATION OF AN INSTRUMENT FOR THE QUALI Tfi TIVE EVALUATION OF CLASSROOM PROGRAMS FOR THE EMO TIONALLY IMPAIRED presented by Bert Lee Donaldson has been accepted towards fulfillment of the requirements for Ph.D. degree in Education 4‘ / "QEMMA E“. AM gm / Major professor ‘. / November 4, 1974 a: . Date_____..___ '1‘- 0-7639 llllilllWillillillIllIllllHIWlililllllllllilill R 3 1293 10020 1965 ii Mlg gtL 1:33?“ e ABSTRACT DEVELOPMENT AND EVALUATION OF AN INSTRUMENT FOR THE QUALITATIVE EVALUATION OF CLASSROOM PROGRAMS FOR THE EMOTIONALLY IMPAIRED By Bert Lee Donaldson This study is an evaluation of one attempt to secure qualitative information about programs for emotionally impaired students in the State of Michigan. It is a follow-up to an earlier study by Larry S. Schaftenaar in l973. The general goal of the present study is to test out a short-form of the questionnaire developed by Schaftenaar. The present study is intended to prepare an instrument that might be used in the study of programs for the emotionally impaired, and that might serve as a model for instruments designed for other areas of special education. This study had two major objectives. 1. Determine if the short form questionnaire is as effective as the long form questionnaire in predicting the re- sponses to the perceived adequacy questions based on responses to the specific condition questions. 2. Determine if interviews with the respondents would assist in understanding the responses to the perceived adequacy questions. The questionnaire used in this study was developed from a much longer form which had been used in 1972 with all public school teachers of emotionally impaired. - Bert Lee Donaldson The subjects for this study were all teachers of the emotionally impaired who were employed by eight school districts. The school districts were selected to participate in this study based upon the Mean PA score computed on the responses to Schaftenaar's study for the teachers in each district. The following conclusions were arrived at: l. The short form appears to be as effective as the long form when the purpose is to predict the PA response from the responses to the specific condition questions in five of the seven areas of programming. a. Responses to specific condition questions are the same for both the long and short form questionnaires in three areas of programming. 1) Student Composition--"workability” of Group 2) Attitudinal Climate 3) Supportive Provisions and Personnel Responses to specific condition questions are similar on both the long and short form questionnaires for two areas of programming except for one question in each of the following areas: 1) Availability of Instructional Materials 2) Administrative Direction and Leadership Responses to specific condition questions are con- siderably different in two areas of programming. 1) Educational Planning and/or Screening Provisions 2) Inservice and Professional Improvement These particular areas may have provided differ- ent results because of the implementation of the Mandatory Special Education Act for handicapped students. Bert Lee Donaldson 2. Analyses of the interviews suggested areas of information that could profitably be tapped in a further revision of this instrument. DEVELOPMENT AND EVALUATION OF AN INSTRUMENT FOR THE QUALITATIVE EVALUATION OF CLASSROOM PROGRAMS FOR THE EMOTIONALLY IMPAIRED By Bert Lee Donaldson A DISSERTATION Submitted to Michigan State University in partial fulfillment of the requirements for the degree of DOCTOR OF PHILOSOPHY Department of Elementary and Special Education 1974 ACKNOWLEDGMENTS The author of this research wishes to express deep appreciation to the many friends and relatives who have provided assistance and support in my pursuit of the doctoral degree. To Dr. James E. Keller for continual guidance throughout the program of study and especially his support and assistance in com- pleting this study. To my guidance committee members, Dr. Frank B. Bruno, Dr. Ronald Wolthuis, Dr. Donald Melcer, and Dr. Glen 0. Cooper, for their guidance and cooperation. To Dr. Larry Schaftenaar, for his support and guidance throughout the development and writing of this study. To the Staff of the Michigan Department of Education, Special Education Services, for their constant and unrelenting encouragement to complete the study. To the special education directors and teachers of the emo- tionally impaired in the eight school districts for having given so freely of their time to assist in the collection of data. To Marcia, Diane and John for their assistance in developing final copy. To my Mother and Father, who have provided encouragement and moral support throughout my life. ii To my loving wife, Norma, for assistance and encouragement throughout our marriage and who has tolerated my varying moods during the development and writing of this study. Finally to my two wonderful children, Bert Ross and Patricia Lynn, for their assistance and understanding throughout the many years of academic endeavor in which I have been involved. TABLE OF CONTENTS LIST OF TABLES ....................... INTRODUCTION ....... '. . . . ............ Purpose of the Study . . .............. Background and Need for Study . . .......... Related Research ....... . . . ...... . . Statement of the Problem .............. Definitions . . . . ..... . . .......... Overview ...... ‘ .............. METHODOLOGY ....................... Development of the Short Form Questionnaire . . . The Interviews ................. Selection of Teachers as Respondents to the Questionnaire ................. Selection of Teachers for Interview ....... . Uncontrolled Variables . . ............ Research Questions and Related Procedures . . . . Procedures for Dissemination of Questionnaire and Follow-up . . ..... . . . . . . ...... FINDINGS . . . . . . . . . . . . . . . . . ..... Questionnaire Returns ..... Results ..... . . . . . .......... . . . Additional Observations and Interpretations ..... Summary ......... . . . . . . ...... SUMMARY AND IMPLICATIONS . Summary . . . . . . . . . Overview ......... . . . . . . . . . . . . . The Survey Questionnaire ............ . . Respondent Population . . . ............ . Findings . . . . . ................. Conclusions .................... Implications . . . . . . . . . ...... iv Page BIBLIOGRAPHY .......................... 78 APPENDICES ' A SURVEY INSTRUMENT .................... 79 B MINIMAL LEVEL OF CRITICAL CONDITIONS AND MINIMAL NUMBER OF FAVORABLE CRITICAL CONDITIONS ........ 85 C SELECTION OF QUESTIONS FOR INTERVIEWS .......... 89 D ENDORSEMENT LETTER FROM DEPARTMENT OF EDUCATION ..... 91 E LETTER TO DIRECTORS OF SPECIAL EDUCATION REQUESTING COOPERATION ................. 92 F PROCEDURAL DISSEMINATION OF INFORMATION FORM ....... 93 G NOTICE TO TEACHER ABOUT THE SURVEY ............ 94 H COVER LETTER FOR SURVEY INSTRUMENT ............ 95 I FOLLOW-UP LETTERS FOR NON-RESPONDENTS TO SPECIAL EDUCATION DIRECTORS AND TEACHERS ........ 97 J LETTERS TO DIRECTORS TO ESTABLISH THE INTERVIEW APPOINTMENTS ................. 99 Table LIST OF TABLES Validation procedures passing criteria for classroom teachers ..................... PA scores for a school district . ...... Selection of teachers for interviewing district A . Attitudinal climate .............. Comparison of responses to specific conditions . . . Phi coefficients and ratios for the cross validation procedure . . .................. Number of criteria passed according to area of programming .............. Distribution of areas according to number of criteria passed ................. The phi coefficient for each specific condition in relation to the PA response for each area of programming ..... . . . . . ..... . ..... vi Page 13 23 24 27 33 47 51 54 55 CHAPTER I INTRODUCTION Purpose of the Study This study is a further evaluation of Schaftenaar's (l973) attempt to secure qualitative information about programs for emotionally impaired students in the State of Michigan. The purpose of the present study is to develop and test a short form of the questionnaire devel- oped by Schaftenaar for his study. Because of the length of the original questionnaire, it seems unlikely that it would be used widely, regardless of its merits as a measuring instrument. Thus, the present study is intended to prepare an efficient instrument that might be used in the study of programs for the emotionally impaired and that might serve as a model for instruments designed to gather qualitative infor- mation in other areas of special education. Background and Need for Study The Michigan Department of Education collects a vast amount of quantitative data on special education programs: cost of programs, number of personnel and students in each category of special education, etc. However, there is also a need for some systematic method of securing data relating to the quality of educational delivery systems. During the late l960's and early 1970's the Michigan Department of Education (hereafter referred to as department) became concerned about educational accountability. This concern was followed by the State Board of Education approving an accountability model which has come to be called the Common Goals of Education in Michigan. Following the devel- opment of this "model,” a statewide assessment procedure was designed to be administered to all fourth and seventh grade students. This assess- ment procedure and the common goals were both directed at the students' achievement. The "Common Goals of Education" was written for all students, but the state-wide assessment did not include handicapped students. The Michigan Department of Education, Special Education Services, became concerned in 1970 with the sparsity of information available concerning the evaluation of programs by special education teachers. While individual assessment of student achievement and the previously mentioned quantitative data were felt to be important, the department also felt the teachers' views would provide insight into some qualitative aspects of the programs. The qualitative aspect of programs became a concern for three specific reasons: 1. The number of staff serving emotionally impaired students was rapidly increasing every year. In the l960-62 school year there were l6 teachers employed by public schools to teach emotionally impaired students. The number of teachers increased to 90 in 1965-66, approximately 400 in l970-7l and to 865 in the 1973-74 school year. There were five consecutive years (l966 to l97l) in which the number of teachers of the emotionally impaired employed by districts increased between twenty-nine and thirty-three per cent. During a ten year period (l960-1970) little or nothing had been done to evaluate the quality of the service being provided through these programs. The turn-over rate of teachers of emotionally impaired appeared to be quite high. This information came about because of two specific pieces of research. Bruno (l968), while doing some research, found difficulty in identifying the sample of teachers of the emotionally impaired which was originally written into his project. Originally he intended to use teachers with one, two, and three years of experience. He was unable to find enough teachers with three years' experience to provide significant information to the study in spite of the fact that schools had been operating programs since l960. From this, one could assume that there must have been a high attrition rate within this particular group of teachers. Kotting and Brozovich (l968) conducted a study to follow up emotionally impaired students previously served in special classes in Oakland County, Michigan. The purpose of the research was to evaluate the success of students who had been returned to general education. The study, as designed, required the researchers to contact the teachers of emotionally impaired who had worked with each student. An unintended finding was that many of these teachers of emotionally impaired could not be found. While it was not reported in the research, Kotting has stated: that there was an attrition rate of approximately 50% of the teachers over a one-year period and that there was a 75% turnover every two years. His feeling at that time was that there must be some severe problems in the system to cause this turnover. The causes of the attrition rate, however, were not investigated at that time. The department felt that the high attrition rate of the teachers of emotionally impaired was partially caused by working conditions which were not conducive to good educa- tional programming. However, there was no information available to substantiate this. What the department needed was data regarding how teachers of emotionally impaired viewed their programs. Then department consultants could provide better leadership and consultation to administrators and teachers as programs were developed. In addition, university teacher trainers would have this information available to better prepare teachers of emotionally impaired for future job responsibilities. Without a common core of knowledge about teachers' specific problems, attitudes, and opinions concerning quality programming, training institutions and the department would have difficulty in assisting school districts and their special education personnel to strive for quality. The State rules and guidelines for programs for emotionally impaired were not generally followed by school districts. Coleman (l968) investigated the operation of programs for emotionally impaired regarding the degree to which school districts in Michigan were complying with State laws and suggested quidelines as they applied to programs for emo- tionally impaired students. Coleman's data were collected through a mailed ques- tionnaire to 187 teachers of which 106 (56.6%) responded. Included in his findings were: a. Nearly lO% of the teachers listed their program as a holding source for emotionally impaired students while waiting residential placement. In other words, these programs were not available to students until the problems were severe enough to require institutionalization. About l/3 responded that educational planning and placement committees (EPPC) were not held prior to placing an emotionally impaired student in the program. Twenty—five per cent of the teachers were not involved with the E.P.P.C. and had no voice in the decisions. While school social workers were, by law, required to be a part of the programs for emotionally impaired, social workers' attendance at the E.P.P.C. were least frequent of the six suggested participants in the E.P.P.C. Only 13.7% of the teachers reported that an E.P.P.C. was held on 9l% or more of their emotionally impaired students. Also, 12.6% of the teachers reported that less than lO% of the students were given an annual review. No teacher reported that a psychiatrist, psychologist, and social worker were involved in the initial screen- ing of all_emotionally impaired students. The most frequently involved were the social worker (58.1%) and the psychiatrist (56.8%). 9. Less than 48% of the teachers received initial reports on emotionally impaired students placed in their programs. h. The teachers reported that educational consulta- tion was sufficiently available to them only 75.9% of the time; social worker consultation, 66.4% of the time; and psychological consultation 52.9% of the time. Coleman summarized his study with the following state- ment: It appears from the information obtained that many programs are not providing the resources and services for their children and teachers that are required to constitute "reasonable" opportunity for both. The implications for teacher discouragement and turnover as well as the continued reinforcement of the problems of mentation and emotionality in their pupils is obvious. Related Research In 1970, staff of Michigan State University and the Michigan Department of Education worked cooperatively on the development of a questionnaire which would provide information regarding teachers' attitudes about how their programs for emotionally impaired students functioned. The questionnaire had approximately 250 items to which the teachers were asked to respond. The 250 items fell into seven broad categories which are later referred to as "areas of programming." The categories were: 1. Variation between student academic abilities and problems in the behavioral area. 2. The procedures for and the persons involved in the screening and placement of students. 3. The various types of support personnel available, such as social workers and psychologists. 4. How well rooms were equipped with materials and supplies. 5. The types of inservice needed. 6. The types of support and leadership provided by administrators. 7. How other teachers and students within buildings felt about the programs for emotionally impaired. In the Spring of 1971 the instrument was mailed to every teacher and teacher consultant of the emotionally impaired who was reimbursed by the Department of Education. After the data were sum- marized and studied, additional concerns developed relative to the need for collecting qualitative information on programs. It appeared that the survey responses could provide valid qualitative information if the survey instrument was refined. In the 1971-72 school year, Larry Schaftenaar signed a contract with the department to refine the existing questionnaire. The desire of the Department of Education and of Schaftenaar was to develop an instrument which would have some practical use in helping administrators become change agents to improve the quality of emotionally impaired educational programs by becoming knowledgeable about teacher attitudes and concerns. The focus of Schaftenaar's research was on emotionally impaired programs and student-group characteristics rather than on individual student or teacher characteristics. The purpose of the Schaftenaar study, in general, was to find out in what types of situations the teachers of emotionally impaired were working, as determined by their opinions about various aspects of the program. In order to refine the questionnaire originally developed by the department and by Michigan State University, five steps were taken. The first step included: a. removing items highly intercorrelated and those showing similar correlation to teacher attitudes, b. removing nondiscriminatory items, c. reconstructing items that were misleading or unclear, d. reconstructing the format and layout for better continuity and e. reconstructing the items which measured perceived adequacy in order that the instrument be consistent. The second step was to completely revise two ”Areas of Programming" because they did not adequately explain the differences found in the 1971 survey. The two sections pertained to: a. inservice training opportunities, and b. administrative support. The third step involved the use of persons knowledgeable about programs for the emotionally impaired. These persons reviewed the questionnaire to recommend additions and deletions as they felt the questions related to programs for emotionally impaired students. Schaftenaar referred to these persons as the "judgmental standards group." The fourth step included revising the questionnaire in line with the recommendations and administering a pilot study on 40 persons. The persons involved in the pilot study were also asked to help clarify items and to make recommendations. During the final step, the instrument was revised based upon the pilot study and then administered to a selected population. Schaftenaar's questionnaire was developed in order to gain insight into the seven general categories of educational programming mentioned earlier. He identified these categories as Areas of Pro- gramming and defined them as follows: 1. Student Composition-—"Workability” of Group. The specific conditions (questions) were about the variability of reading and arithmetic achievement, severity of emotional problems, and the amount of time the students are integrated into regular classrooms. Attitudinal Climate. This category included specific condi- tions about other teachers' attitudes about the program, amount of contact with regular teachers, and proximity to a regular classroom. Educational Planning and/or Screening Provisions. This category included the length of time for evaluation and the identification of who attended planning meetings. Supportive Provisions and Personnel. This category included the various types of consultative support and what is available when students are in need of individualized intervention. Availability of Instructional Materials. This category included amount of money allocated for materials and supplies, convenience of getting materials, and availability of equip- ment. Inservice and Professional Improvement Opportunities. This category included opportunities for inservice and with whom the inservice takes place. Administrative Direction and Leadership. This category included identification of who the teacher of emotionally impaired is responsible to, and the type of direction and support the teacher receives from the supervisory standpoint. Each "area of programming" in the instrument had a group of questions related to that particular subject. For example, under "Administrative Direction and Leadership” there was a question regarding 10 the number of times the administrator either consulted with or visited the classrooms. These questions under each area of programming were called "specific conditions.” The judgmental standards group was also used to develop levels of quality for each specific condition question which would be favorable or unfavorable in the operation of a program. One of the questions and possible responses was: a. How often does this person (supervisor) consult with you or visit your class per month? Zero times 1 to 4 times 5 to 9 times 10 to 14 times 15 to 19 times 20 times or more mm-hoom—I The minimum favorable level for this question was response number 3 (5 to 9 times). Therefore, a check mark by a teacher in box 3, 4, 5, or 6 would have met the favorable standard as determined by the judg- mental standards group. A check mark in box 1 or 2 would be considered unfavorable. There was another section to the questionnaire following the seven areas of programming called "Personal Perception of the Program for Emotionally Disturbed Children.“ This section included a summary type question which asked for the teacher's opinion about the quality of an area of programming. For example, one question in this section was: How would you describe the Administrative Direction and Leadership you have received in the operation of your program for emotionally disturbed children? 11 There were five possible responses, (1) excellent, (2) very good, (3) good, (4) fair, and (5) poor. The responses to the questions in this section were referred to as "Perceived Adequacy." Schaftenaar's sample included all teachers of emotionally impaired students in all public schools in Michigan. The list did not differentiate between persons classified as classroom teachers or as teacher consultants. The sample included 474 teachers and the survey was returned by 391 (82.3%). Those of Schaftenaar's research questions which are directly related to the present study were specifically stated as: I. What are the frequencies and percentages of the Condition Levels as reported by Michigan's public school teachers of emotionally disturbed children? III. In terms of Perceived Adequacy, how do Michigan public school teachers of emotionally disturbed children view their programs? IV. How well can teachers' perceptions of the adequacy of program areas be predicted from specific conditions? V. Can a limited number of specific program conditions be located, and, in turn, can criteria be established to rate these conditions as either favorable or unfavorable, whereby a numerical count of favorable conditions will previde a reasonable and useful means of predicting Positive PA for certain homogeneous groups in all seven Areas of Programming? (Schaftenaar, pp. 15—18.) Research question IV required an extensive analysis, referred to as Multiple Regression Stepwise Deletion Analysis (MRSD), for six of the areas of programming. The other areas of programming could not be analyzed by the MRSD method and therefore the Multiple Classifi- cation Analysis (MCA) was used. The purpose of these two analyses was to take each area of programming and identify the specific condition 12 questions which provide the greatest degree of predictability of response to the perceived adequacy question. The two statistical pro— cedures, MRSD and MCA, were described on pages 40-58 of the original study. The next step was to run three validation processes to see if the items identified through MRSD and MCA were predictive of the perceived adequacy response by the sample population. The teachers were split into two groups. Group 1 was used to establish a scoring system. Group 1 included 60 per cent of the population after some persons were excluded. Some persons were excluded because their situation was atypical such as working in a juvenile home or preschool program. Group 2 was the independent group upon which the scoring system was tested. The validation procedures were described on pages 61-85. Since each of the areas of programming had three validation procedures run, there was a total of twenty-one criteria. The results of the procedure are presented in Table 1. According to the results of the study, it appears that there is predictability regarding the teachers' perceived adequacy of each of the program areas based on responses to that area's specific condition questions. The validation procedures were used to establish a ”favorable level of critical conditions" (FLCC). The FLCC basically identified the level at which the teachers found each specific condition acceptable. For example: Inservice meetings should be held once per month or more often. 13 Table 1. Validation procedures passing criteria for classroom teachers. Classroom Validation Area of Programming Procedures 1 2 3 Supportive Provisions and Personnel + - + Administrative Direction and Leadership + + + Student Composition-- "Workability of Group" + + + Education Planning and Screening Provisions + - + Inservice and Professional Improvement Opportunities + - + Availability of Instructional Materials + + + Attitudinal Climate - + + Total Pluses for Items 6 4 7 Total Pluses = 17 Total Minus = 4 Total Applications of Criteria = 21 Passed criteria Did not pass criteria Note: + 14 The FLCC was used in developing the ”minimal number of favorable critical conditions” (MNFCC). The MNFCC was the number of FLCC's met which would determine the prediction of the perceived adequacy response. For example, in the area of programming entitled Classroom Attitudinal Climate, there were four specific conditions questions of which three responses must meet the FLCC to be able to predict a positive perceived adequacy response. Therefore, if less than three FLCC responses were made, the prediction would be a neutral or negative response to perceived adequacy. Following are the FLCC's for Attitudinal Climate as presented by Schaftenaar: III. Attitudinal Climate* MNFCC = 3 1. At least some students are spending part of the day in a regular classroom. 2. Most of the regular classroom teachers in the building attempt to understand the unique needs of ED children. 3. The teacher has very much contact with regular class teachers in the school. 4. There is a regular classroom teacher in the adjacent room. *Please note these conditions do not apply to teachers in a ”building entirely for special education programs. (Schaftenaar, p. 71 Positive perceived adequacy is defined as a response of excellent or very good. Neutral or negative perceived adequacy is a response of good, fair, or poor on the perceived adequacy questions. The findings of the study were: 1. That experts functioning in various capacities in the area of programs for emotionally impaired showed a high consistency in judgments regarding the minimal level of acceptable condi- tions that should be established so that adequate services can be provided. Statement 15 That adequacy of program could be predicted from certain specific conditions. That the favorable level of critical conditions show promise as some specific things which may be done to improve pro- grams. However, a caution was presented: school districts should not drop many things they are doing and address these variables only. Much of what is happening in districts is probably positive. The FLCC should be looked at as some- thing to evaluate and work on improving in addition to present practices. (Schaftenaar, 1973) of the Problem The problem of this study is two-fold: To determine whether a short form questionnaire based on the original study will yield comparable results to the long form. That is, can predictions be made about the response to the perceived adequacy questions based on responses to the specific conditions questions. To shed light on the interpretation of responses to the short form questionnaire by interviewing some of the teacher respondents. Definitions Several terms have been defined up to this point. The following list includes terms which have been used plus some additional terminology. 1. ”Area of Programming” is a general reference to a group of particular questions. e.g., Workability of Group is an ”area of programming" and questions include the range of reading and arithmetic ability, integration of students into regular classes, and severity of behavior problems. 10. ll. 12. 16 "Specific Conditions" are the questions asked under each area of programming. "Perceived Adequacy" (PA) is the teachers' opinion about each area of programming based on responses to each of the last seven questions (26-32) in the survey instrument (Short Form). "Long Form" is the questionnaire used for the Schaftenaar research. "Short Form“ is the questionnaire developed for this study and includes only some of Schaftenaar's questions. "High Scorers" are persons whose responses met the minimum criteria for prediction of a high Perceived Adequacy score. (MNFCC) “Low Scorers” are persons whose responses did not meet the criteria in six above. "Original Study" is the study done by Schaftenaar. "Favorable Level of Critical Conditions" (FLCC) is the level at which each specific condition is considered as positive. e.g., When it takes less than two weeks to get a student reevaluated. "Minimal Number of Favorable Critical Conditions (MNFCC)'I is the number of the FLCC's which discriminate between the prediction of positive PA and negative PA. "High or positive PA” is a response to questions 26-32 of "excellent," or "very good." "Low or neutral/negative PA“ is a response to questions 26-32 of "good," ”fair,“ or ”poor." Overview In Chapter II the procedures for gathering data, writing the short form instrument, and data processing are presented. The results and discussion of the data analysis are presented in Chapter III. Implications of this study are presented in Chapter IV. CHAPTER II METHODOLOGY Development of the Short Form Questionnaire The questionnaire used for this study is given in Appendix A. The questionnaire development was based on the minimal level of critical conditions (MLCC) identified in Schaftenaar's study (Appendix B). Each of the critical conditions he identified is related to a specific condition question in his instrument. For example, one of the minimal level of critical conditions was, ”It takes less than two weeks to get a student reevaluated.” This level of the specific condition was identified through Schaftenaar's analysis described in Chapter I of this paper and was arrived at from the specific condition question: (9) If you had a student in your classroom who you felt did not belong there (e.g., was not appropriate for your classroom, was impossible to work with, etc.), how long would it take to get him reevaluated? Less than 2 weeks 2 weeks to 1 month 1 month to 2 months more than 2 months would not be able to get him reevaluated U‘l-DWNA l9 Schaftenaar identified from four to seven MLCC's for each area of programming (Appendix B). For this study, each specific condition question from which the MLCC was taken in Schaftenaar's instrument was identified. These questions then made up the content of the question- naire (short form) for this study. The questions in the short form have been maintained under the same area of programming as they were in the long form. Some of the MLCC's apply to more than one area of programming. For example, one specific condition is under both Supportive Provisions and Personnel, and also under Classroom Planning and Screening Provisions. This specific condition under both MLCC areas of programming pertains to the length of time it takes to get a student reevaluated. In other words, this MLCC evolves out of the same question which is in the Educational Planning and Screening Provisions section of the question- naire. There are some questions included in the questionnaire which are not analyzed. These questions were included because they were a lead-in to a question of importance. For example, question 1 and 2 were incorporated as a lead-in to question la and 2a respectively. The questions could have been rewritten; however this might have changed the context of the questions and thereby not have been com- parable to the long form questions. Question 24 was also asked to let the respondents' know as to whom questions 25-25e referred. In addition, it should be noted that questions 1, 2, and 25 are not mentioned in the MLCC's. 20 The Interviews Each teacher of emotionally impaired was asked questions regarding the way they had responded to two of the PA questions. The interview was open—ended, allowing the teacher to respond freely to the questions. The four teachers of emotionally impaired interviewed in each district were asked to respond to the same PA question. That is, for example, in District X teachers were asked to respond to questions 29 and 31, in District Y to questions 26 and 28, and in District Z to questions 27 and 31. The specific questions asked in each district were used because of the obtained differences in the responses to the question by the teachers in that district. The concern was to see how teachers of emotionally impaired functioning within the same administra- tive structure could have responded in such different ways. A chart like Table 3 was developed for the four teachers interviewed in each district (Appendix C). Then a process of elimination was used to identify and eliminate the PA questions for which most of the teachers responded in a similar manner. For questions 27 and 28 in Table 3, the responses were identi- cal; however, an arbitrary decision was made to choose only one of those PA questions in favor of question 30, to which two persons had responded, ”excellent“ and two had responded ”fair.” The basic interview question for each teacher of emotionally impaired was handled in the same manner. That is, first the teacher's attention was directed to the PA question and response on their question- naire. Then the interviewer would say, ”This question is concerned 21 with your opinion on (area of programming). You have rated it (good, fair, etc.). I would like to know what types of things may have caused you to rate it (______). Another way of putting it is: "Why couldn't you have rated it higher; what kinds of improvements do you feel should take place?" After the initial statements, the questions would vary somewhat among the teachers for purposes of clarification or expansion of ideas. Selection of Teachers as Respondents to the Questionnaire This study was conducted with classroom teachers of emo- tionally impaired students. The teachers were employed by eight local school districts in the southern one-half of the lower peninsula of Michigan. The classroom teachers of the emotionally impaired were included in this study because they were employed by the eight specific school districts chosen for the study. Eight school districts were selected for inclusion, based upon two requirements. One requirement was that the district have five or more teachers of emotionally impaired who had responded to the 1972 survey (Chapter I). The teachers selected for this study had to have approval in special education and had to be reimbursable by the State Department of Education as a teacher of the emotionally impaired. All teachers employed by the eight school districts received a copy of the questionnaire and other items mentioned later in this chapter. 22 The second requirement was that the districts could be equally divided into two groups: one group having a low PA average score, and one group having a high PA average score. The two groups each had four districts assigned to them based upon the results of the PA scores of the districts obtained from the original study (1972). The PA score for each district was arrived at by assigning values to each of the seven PA questions and computing the arithmetic average. The responses ranged from, "excellent” with a value of l, to ”poor” with a value of 5. The responses to all PA questions for all teachers in the district were summed and the sum was divided by the number of teachers in the district. This dividend was then divided by five to arrive at the mean PA on a scale of one to five. The result is an average PA score for the district. Table 2 shows the scores for one school district. According to data from the original study there were thirteen school districts which had five or more teachers who had responded to the 1972 survey. The districts with extreme mean PA scores (high and low) were selected because the extreme responses to the survey would provide greater ease in evaluating the predictability of responses to the PA questions based on responses to specific condition questions. Selection of Teachers for the Interviews There were four teachers of emotionally impaired interviewed in each of the eight districts. The teachers interviewed were selected on the basis of their own PA score on the short form. The PA score was determined for each teacher within each district by summing the assigned 23 Table 2. PA scores for a school district. Teachers Scores PA Question I 2 3 4 5 6 7 8 9 l 5 3 3 5 3 l 3 2 4 2 4 4 3 5 4 l 3 2 5 3 5 l 3 5 4 l 3 l 5 4 3 l 4 4 4 l 2 2 5 5 2 2 4 4 3 2 1 l 5 6 l 1 l l 3 2 3 l 5 7 3 2 l 2 3 l 3 l 3 Note: Mean PA = 3.889. values of their responses to the PA questions. Then the two teachers having the highest and the two having the lowest PA scores were identi— fied as the ones to be interviewed. By looking at the scores for the nine teachers in Table 2, one can see that teachers 6 and 8 had the lowest scores, and teachers 4 and 9 had the highest PA scores. There— fore, teachers 6, 8, 4, and 9 were interviewed in that district. Uncontrolled Variables This study, under perfect conditions, would have used a population carefully matched to that used by Schaftenaar. This was not possible and therefore some attention should be paid to uncon- trolled variables that may have created differences in responses to the two forms. One of the variables which may have affected the results is the method used to identify classroom teachers in the two studies. The teachers included in this study were identified as classroom 24 Table 3. Selection of teachers for interviewing district A. High PA Low PA Question Teacher A Teacher B Teacher C Teacher D 26 3 2 i 4 27 4 5 1 2 28 4 5 4 2 29 3 4 1 2 3O 4 4 l 1 31 3 3 3 ——2-- 32 4 5 2 2-- teachers based upon the way they were classified on program application forms submitted by each school district to the Michigan Department of Education. In Schaftenaar's study teachers classified themselves. Therefore, there may be some differences based on who was classifying persons as classroom teachers. Other variables which were not controlled were the changeover of personnel, the years of teaching experience for each teacher, and the background of training of the teacher (at which university the teachers were trained). In addition, some changes have taken place in the laws governing the operation of special education programs due to passage of the Mandatory Special Education Act, P.A. 198 of 1971 (effective September, 1973). The Special Education Code, which specifies rules for enforcing this Act, has forced many districts to make adjust- ments in programs which may have influenced the results of the study to some degree. The Law and the Code will be considered again in the discussion of the findings. 25 This study, then, will be looking at various sets of measures (responses) from the long form and short form and will evaluate the relationship between these sets of responses on the two forms. If the short form is functioning in a manner similar to the long form, responses to the specific conditions questions will predict responses to the perceived adequacy questions as they did in the long form. The data used in the original study were collected in May of 1972 and the data for this study were collected in May of 1974. If the results of this study show that some specific conditions, and/or some perceived adequacy responses, have changed, these changes may be related to changes that have taken place within classroom settings since 1972. Validation of the short form assumes that the two forms may be considered comparable if in the short form responses to the "specific conditions” will allow prediction of the "perceived adequacy" response as in the earlier study. Research Questions and Related Procedures Question I. Does a short form utilizing Schaftenaar's identified critical conditions yield similar results? A. Are the responses to each specific condition question similar on the long form and on the short form? A Chi square analysis on responses to each question was performed. The acceptable level of significance was set at .05. For example: 26 6. Do most of the regular classroom teachers in your building attempt to understand the unique needs of emotionally disturbed children? Short Form Long Form X2 % N % N 1 Yes 64.8 35 68.1 124 2591 2 No 35.2 19 31.5 57 ' Chi square is not significant at the .05 level of confidence, indicating that there is no difference in the responses to the long or short form for this specific condition. 8. Are predictions of PA of the same order on the short form and long form using Schaftenaar's procedure? Schaftenaar has established a minimal number of favorable critical conditions (MNFCC) for each area of programming which were used to predict the response to each PA question (Appendix B). The creation of a short form changes the setting of each response, and it may be assumed, without evidence to the contrary, that the setting could influence the response. Furthermore, in this instance, one might expect that the original form, being very long, might have created boredom or might have been viewed negatively because of the size of the job when responding to each question. The long form was eleven pages, while the short form is six pages long. As a result, the short form may elicit a more careful response; hence, the strength of the relationship of critical conditions to perceived adequacy may be changed. This question was answered through the use of what Schaftenaar has identified as validation procedure 2 (Schaftenaar, pp. 80-82; see 27 page 13 of this study). Validation procedure 2 includes three steps, all of which must produce acceptable results of the particular area of programming to meet the test. The first step involves placing the data in a 2 x 2 table. One dimension is for high PA and neutral/low PA. A person is cate- gorized as High PA when he has answered the PA question with "excellent" or ”very good”; neutral/low PA persons are those responding ”good,” ”fair, or “poor " The other dimension of the 2 x 2 table is for the minimal number of critical conditions (MNFCC) which were met. A person is a high scorer if his responses met the MNFCC identified in each area of programming in Appendix B. This dimension is labeled high scorers/ low scorers. Phi coefficients were calculated to portray the strength of relationships in these fourfold tables. Table 4 is an example of the data base for evaluating this second part of the first research question. Following Schaftenaar, the significance level for the phi coefficients was set at .01. Table 4. Attitudinal Climate. Short Form Long Form Low PA High PA Low PA High PA High Scorers 16 15 20 19 Low Scorers l9 5 19 4 n = 55 n = 62 ¢ = .284 4 = .313* Percentage of High PA with High Scores 75.0%** 82.6%** . _ 48.4% _ _ 48.7% _ _ * Ratio - 20.8% — 2.3 1** 17.4% - 2.7.1 * *Significant at .01 level for o. **Meets formula criterion. 28 The second and third steps of this validation procedure involved using one of the following formulas: a. l. 2. b. l 2. Table 4. Percentage of high Percentage PA teachers being high scorers 2 50% Rat1° = Percentage Percentage of high of high scorers with high PA > 2 of low scorers with high PA ’ 1 PA teachers being high scorers 2 40% Ratio : Percentage of high scorers with high PA > 2.5 Percentage of low scorers with high PA ’ l The tables have two criteria for whether or not the minimal number of favorable conditions were valid: I. A phi coefficient significant at the .01 level. II. And either: a.l. At least half of the high PA people were located in the high scorers group and (2) the ratio of high PA to low PA was twice as great in the high scorers group as in the low scorers group. b.1. At least 40 per cent of the high PA people were located in the high scorers group and (2) the ratio times of high PA to low PA was two and one-half as great in the high scorers group as in the low scorers group. Examples of the data for this procedure are also included in By reviewing Table 4 it can be seen that Attitudinal Climate passes the criteria for all three steps. Therefore, the specific conditions in the area of programming, Attitudinal Climate, appear to have some value in predicting the perceived adequacy as teachers would evaluate the program. 29 Question II. Are the relationships between each specific condition and the perceived adequacy of an area of programming con- firmed by information obtained from interviews? A. Is the phi coefficient significant at the .05 level when computed for each 2 x 2 table for each specific condition vs. perceived adequacy? For example: Specific condition question 6 00 most of the regular classroom teachers in your building attempt to understand the unique needs of emotionally disturbed children? The responses to this question set up in table would appear as follows: Perceived Adequacy Low High High 20 15 Specific n = 56 Condition ¢ = .19245 Low 16 5 B. Do the interview data assist in understanding each cluster of tables for each area of programming? The results presented in the tables will be compared with information gathered in interviews. This comparison will provide some insight into why teachers of emotionally impaired within the same school district answer the same PA questions in a different manner. That is, some of the teachers of emotionally impaired within the same school district may have responded, “excellent,” and others, “fair,” or ”poor,” to the same PA question. 30 Procedures for Dissemination of Questionnaire and Follow-up First, the eight school districts to be used in the study were identified as described earlier in this chapter. The Department of Education then provided a list of all teachers of emotionally impaired students employed by the eight districts and the names and addresses of the special education directors of each district. The study was discussed with the State Director of Special Education for the purpose of getting the department's endorsement. The Director of Special Education then wrote a letter in support of the study (Appendix D). The next step was to contact each director of special education of the selected districts by telephone to request his assistance and support in having teachers respond to the questionnaire and in setting up interviews at a later date. A short description of the‘study was provided during this discussion Following the telephone conversation, a letter which described the study, was sent to the directors (Appendix E). Enclosed with the letter was a self-addressed memorandum to be checked off by the director and returned (Appendix F). This memo was to provide instructions as to the method of mailing the questionnaires, to whom and the method of setting up interviews. Also enclosed was a copy of the letter of endorsement of the study by the State Director of Special Education. All directors requested that the mailings to teachers be done through their offices. In addition, all directors answered ”yes" to setting up interviews at a later date. 31 The next step was to notify the teachers in each district regarding the forthcoming questionnaire. A copy of the letter is presented in Appendix G. As requested by the directors, these letters were sent through their office, but each letter was personally addressed to each teacher. Each teacher then was sent a copy of the questionnaire (Appendix A) with a cover letter (Appendix H) explaining the ques- tionnaire and explaining the study in more detail. A stamped, self- addressed envelope was enclosed for ease of return. Each questionnaire included the teacher's name, address, and an identification number. The teachers were allowed twenty days to respond to the questionnaire. After twenty days, the non-respondents were sent a follow-up letter encouraging their participation (Appendix I). Ten more days were allowed for additional responses. After that time the directors of special education were contacted by tele- phone and asked if they would encourage the non-responding teachers to respond. After five more days, preliminary evaluation of the PA ques— tions was initiated. The responses to questions 26-32 were summed: these are the perceived adequacy questions which are on a nominal scale. After summing these scores, the two teachers with the highest scores and the two teachers with the lowest scores were identified. The directors of special education were then notified by telephone and by letter regarding their four teachers to be interviewed (Appendix J). The interviews were scheduled and took place within a two and one-half week period. CHAPTER III FINDINGS Questionnaire Returns There were 95 teachers of emotionally impaired in the eight districts. Questionnaires were completed and returned by 75 of the teachers. After reviewing the 75 questionnaires, two had to be dis- carded leaving a total of 73. One was discarded because the teacher found most of the questions inappropriate to her situation and did not respond to them. The other teacher did not respond to one full page of questions which included three of the PA questions. There- fore, 76.8 per cent of the sample population responded in a manner which allowed the data to be used. Results Question I. Does a short form utilizing Schaftenaar's identified critical conditions yield similar results? A. Are the responses to each specific condition question similar on the long form and on the short form? For Question IA responses of the two groups were compared using a chi square analysis. The frequency of responses from the original study were ascertained from Appendix B of Schaftenaar's study. The data are presented in Table 5. 32 33 pomm._ Ao.mmv omp Ao.muv pm oz .N Ao.mmv on Ao.mmv up mo> ._ wooppepp mp mpcooopm Loo» 4o momma Focomgoo oco opeooooo .pocopposo ogp poms op zppppoo coox pogp pomsomocos co ocppopompo co mspp nose om ocoom op o>oc so» on .o oaom.o Ao.emv amp Ao.omv mm oz .N A_.m3v mop Ao.omv mm ma> ._ Naoocmmopo coo» op oopoco; on op oco>om oop oeo meopooeo pocopposo .mpcooopm coo» mo meow poop poop 3oz on .m Am.pev aop Ao.omv mm oz .m mppm._ A_._ev om A_.mev om paszoeom .N A3._pv mm AN.o v 3 ma> ._ wpcooopm mopeeomeoo pmozop oco poooopm mopseoe -coo pmosmp; cooz ooozpoo pooEo>opgoo opmeSppeo op xppppoopco> ocp >o ooppepp mcpgooop Loox mp .om A3.va Op AN.oNV Np oz .m po_3._ Ao.o3v mop AN.mev mm paszosom .N Ao.mpv p3 Ao.3mv op ma> ._ Npcooopm mopscomcmo pmozop oco pcooopm mopEcoe -coo pmocmp; coo» cooZpoo pcoeo>opcoo mcpoooc op xppppoopco> ocp >o ooppepp mcpcooop Loo» mp .o_ azoco co zzppppomxeo3=--eopppmoaeoo pcoozpm x p z a z acopppocoo opppooam N Econ mcoo Eco; pcozm .mcopppocoo opopooom op momcoomoc 4o compcoosou .m mpnoh 34 owoo.o Ae.¢mv 4o AN.mmv mp oz .N Ao.mmv omp Aw.omv mm mo> ._ meoox op pcooonoo Eooc mop op Logooop Eooemmopo Lopomoe o oemgp mH .w Am.m V pp Am.“ v e oppppo >La> .m wmmm._ Am.mmv mo Ap.mmv op anm .N AN.mmV pop Am.mmv mm goo: xeo> .— Npoocom eoox cp meozooop Eoocmmopo Lopomoe ogp cppz o>og so» oo poopcoo cogs 3o: .R pmmm.o Am._mv Rm Am.mmv mp oz .N Ap.oov emp Aw.3ov mm ma> ._ wooeoppgo oooeopmpo appocoppoEw mo momma ozopc: ogp ocopmeoooo op poEoppo mcpoppoo Loo» op meocooop EooEmmopo copomoc mop 4o pmoE on .o opoeppu pocpoopppp< pmpo.m Ap.wmv 3o Am.mmv ow oz .N Am._pv 3op Ap._ov 33 ma> .p mEoomeopo copomoe o op zoo osp mo peoo o mcpocoom mpcmoopm cooz 4o zoo oc< .m x a z a z m:o_ppocoo opppoaam N Econ mooo Eco; pcocm .owchpcoo--.m opaap 35 Am.N_V o3 A3.__V 3 L232 .m “3.3 V pp Am.3 V m Eoupam .3 *RmNm.NN Aw.“ V w— Am.NNV op moeppoEom .m Ao.o_V Rm AR.mNV Np copmo .N A_.3mV mmp Ap.mmV mm asazp< ._ NEoocmmopo Loo» opcp muzo pocp oopppeaoo mcpcooeom Lo ompppeeoo mcpccopo pocoppooooo ocp 4o mmcppooe ocoppo Loxcoz popoom mopocom ocp mooo :oppo 3o: .oop A_.m_V 33 Ak.mmV mN Lo>oz .m Ao.N V o A3.__V o soopam .3 388.3 GNV o 23.: _ 8553.38 .m AN.m V Np Am.NpV m copapo .N Ao.oNV mop Ao.omV pm asazp3 ._ Neooemmopo Loox opcp mozo poop oopppEEoo mcpcooeom Lo oopppEEoo mcpcoopo pocoppooooo mop mo mmcppoos oooppo :o» oo :oppo 3o: .oop Au.m V mp AN.N V m oopoopgowL 5p; pom op opoo on po: opooz .m Ao.N_V o3 Aw.w_V mp wepcos N gasp 3205 .3 Nmmw._ A3.mNV mm Ao.mNV ON mspcos N op cpcoE _ .m Am.mmV o“ Ap.oNV op specs _ op 33333 N .N Am.mpV m3 AN.m—V mp mxmoz N coop mmoo ._ Noopoopo>ooc Ep; pom op oxop pp opooz mcop 3o; .A.opo .gppz xcoz op opopmmooep mo: .Eoocmmopo Loo» cop opopcooeooo poc mo: ..m.oV oeocp moopoo poo opo ppow so» oz: Eooemmopo Loo» op pcooopm o oo; oo» 4H .m mcopmp>oeo mcpoooeom Lo\oco mcpccopo poooppoooom x x z N z mcopppocou opppooom N Econ mcoo Eco; pcocm .333cppcoo--.m wpamp 36 .mooo.o_ zo.opV 33 zo.me om oz .N AN.ooV map z_._oV 33 moz ._ Nsooemmopo Loo» soap mpoooopm 4o o<>ozmm ogp op oopo> opoooooo co om; o>oz so» poop 3oz on .m— oomo3.Np zo.oNV mo zo.3mV O3 oz .N Ao.NpV mop AN.m3V mm moz ._ Nsoocmmopo coo» op mpcooopm po pzm2mu opoooooo co om; o>oz :oz pomp oo» oo .N_ .ommo.m Am._mV mpp “3.33V 33 oz .N zp.w3V oop Am.mmV mm moz ._ chp>com moo so» coco—35o ocp zz< po moooc ozp mmoompo op zzzoz .3 Ac.“ V op A3.__V w Eoopom .3 aNppw.w A_.m V _N Am.NpV m moEppoEom .m AN.__V RN A3._NV mp ooppo .N AN.NmV omp zo.N3V om mzozp3 ._ NEoomeopo Loo» opcp Nozo pozp oopppEEoo mcpcoopom Lo oopppEEoo mcpocopo pocoppoooom mcp 4o mmcppoos ocoppo pmpmopocozmo ocp mooo :opmo go: .oop z 3 z 3 z moopppoooo opppooom N Econ moo; Econ peogm .Du::_o:DJII.o v.35. 37 zo.o3V oop Am.o3V mm oz .m Noom.o A_.o_V p3 z_._NV mp mocppocom .N zo.mmV op A3.NNV MN moz .— Npcoozpm Mpcp mcpco coo so» copcz op coppooop co Eooc ozcoppom o o>oc so» on .N_ A3.NNV mo A3.NmV MN oz .m omNo.p AN.NNV mo AN.mmV 3N mocppocom .N AN.3NV 3N zo.mmV 3N mop ._ Npcooopm ocp cppz zcoz so» oppcz Eoocmmopo coo» mcpxop ocooEom coo: ocoooo 50> cou .mp zo.m3V mop A3.NNV MN oz .m mmww.m A3.RNV No AN.mmV mN mmEppoEom .N zo.NNV po A3.NmV MN 33> ._ Nmmopo coox cppz cposoc coo so» pocp om .pcooopm mpcp cppz zcoz 0p ocooeom coo: ppoo Apcopomoc 3oz coo .m_ mNp3.N zp.N_V NN zo.m V 3 oz .N Aw.pwV NON A3.3oV No moz ._ Nmpcooopm coo» po moooc pocoppoEo oco pocomcoo ocp mcppooe cp so» opo op opcoppo>o zpcopzmoc oco ocz mpcoppomcoo zco o>oc so» on .3_ pmccomcoo oco mcopmp>occ m>ppsooozm z N z 3. z mcopppocoo optooom N Eco; mcoz Eco; pcocm 38 Noom.N zo.3_V 3m Am.NNV op oz .N AN.mNV mop Am.NNV mm moc ._ Noe» op opcoppo>o pcoEopooo oco moppooom Noomp>lopooo opoooooo m>oc so» on .NN No.0NV m3 AN.opV mp mzpcoc 3 oozp cocooz .m AN.N_V NN No._mV pN mzpooc 3-m .3 .3oNo.mp N3.mNV pm NN.o_V mp mzpooc N .m AN.mNV mm AN.N_V Np zpooc _ .N zo.NpV om zo.opV N zpooc _ oozp mmoo ._ NEmcp oopmooooc pmcpp o>oc :ox copmo 243.20me pwm on 8.3. NAP—95: .t. mmou acop 30: ON Am._ V m N3._ V _ oooz .m zo.o_V 3N Am.3_V op ocom .3 mmoo.o z_.N V o_ zo.o_V N Npoz .m Am.mmV pr No.NoV 33 pmoz .N No.NNV mo A3.__V N ppz .p No>poooc appocpoo so» on pmwzcwc 30% mpmzmpwE mEp “Vb cowpcoaoca was: .2. zN.mNV o3 AN.3_V op ppcpp oopcpoooo oz .3 AN.NNV N3 N_.NNV mp ococ co-_NN3 .m AN.3_V pm AN.3_V op oNNp-_Npp .3 oNoN.m AN.3_V om Am.o_V N ON_3-_Np3 .m NN.m_V Nm No.ONV 3p ON_3-_N3 .N zo.o_V pN zo.NpV Np oNp oozo mooo ._ wEoocmmm—u coo» com pomooc mpopcopoE xpcooz ocp mp pocz .Np mpmzmppmz chowpuchmg “+0 Napwznm—EBN x N z N z mooppcoooo opcpooom N Econ. pcocfi Eco; mcoz 39 ANN V o 8N. V N pcomoco cocoppco--mcooc poocom copomoc mcpcco .m AN.mmV pop Am.NmV mN mommopo cmcpo mcpocoppo co ommmpEmpo coco—pco--mc:oc poocom copomoc mcpcoo .3 moom.N NN._ V N zo.m V N mooozooz .m AN.N V m Am.3 V m mmcpco>m .N AN.NmV mo Am.pmV 3m poocom copco appoocpc .N N.>_co mzo coocoV Ncoooo mEocmoco oop>cmmcp coox No pmoE oo cocz .DMN A3.mNV Na AN.m—V mp .mcocooop Eoocmmo—o copomom .— oomN3.o NN.3mV NN NN.NNV mp .mopppppoompo cozpo cppz cocoppco cop mcocooop coppcozoo pcpooom .N N3.mmV mm No.NmV mm .cocoppco cocczpmpo AppocoppoEo No mcocoooN ._ z.zpooo poop ppo coozoV .opoz Nppoooo moocmccoz oco mmcppooE oop>comcp coox occ Eocz cppz .omN N3.N_V _3 Am._ V _ co>oz .o N3.N3V mop Am.N3V mm mcpcoE N Nco>o cocp copco mmoz .m Nm.N V op Am.o_V N mzpooc N Nco>m .3 ammoN.ON Am.m_V om N3.mNV 0N cpcoE o moco .m Am.o V 3_ No.3 V o zpooc o oopzp .N No.3 V pp Am.p V N zoo; o ooco ._ Npopcpmpo coo» cp opoc appoom: moocmzcoz co mmcppooE oop>comcp oco coppo so: .NN mopppcopcoooo pcwEo>ocoEH pocopmmomocc oco oop>comcp x N z N z moopppoooo o_cpooom N Ecod mcoz Econ pcocm 40 NN.ppV NN AN.ppV N cooo .N AN.NpV N3 pN.NpV 3p cpoc .3 NN3N.o NN.NNV 3N No.NNV Np oooN .N N3.NNV NN p3.NNV Np oooo coop .N NN.NpV p3 NN.NpV 3p pooppooxo .p NEmLUFEU bwnczpmzu NAP—.wcowporcw “No muwwc 95.2.5 93 we mem—zocx m.:0mcwa mpcp 9:15me 30% U303 30: .omm NNN3.o AN.NpV p3 No.pNV Np oz .N AN.pNV NNp Np.NNV NN mop .p wmmcoqmwc .5533; $0 vwmam 93. cuts Uwgmwupmm cox oco comcoo Npcp No oocopmpmmo pmooooc so» Np .omN Np.opV NN AN.3 V N ococ co Nocpp NN .N No.3 V pp AN.3 V N Nocpp Np op Np .N AN.N V Np pN.ppV N Nocpp 3p op op .3 33pp.3 AN.NpV NN AN.ppV N Nocpp N op N .N po.N3V ppp po.N3V 3N Nocpp 3 op p .N AN.NpV NN Np.pNV Np Nocpp ocoN .p NcpcoE coo moo—o coox ppmp> Lo 30% 53.3 p—smcoo cemcwa mt: mwov £038 30: .me pN.NNV N3p zo.NNV NN ocppNoz .N AN.NpV NN N3.NNV NN Nocooooo oopopoc-ooN poccoo zppz .3 NNpN3.3p AN.ppV NN AN.NNV Np Nocooooo oopopoc-ooN coNoc zppz Npoo .N NN.m V mp AN.N V o NopocomcoEo oEocpxo cp Npco .N p3.3 V op NN.N V N co>oz .p N.Npoo Nzo zoozoV Ncomcoo Npcp coo: ppoo op omoo pN poop 3oz oo cocz .mN szmchMmJ Ucm compuwcmo w>wumcumwcw=€< Nx N z N z Ncopppocou opopooom Econ ace; Econ pcocm 1X .oozzppzoN--.n o_oo_ 41 .: pcwpmpmcou N oup>opa no: Nwov cups; vwxumsu ma cpzoo Npmzmcm mpapp—ze wmzmumn coppmmzc mpg» pom wouzasou we: mm: x pmzpo prnsms ppmpw we Ncpmucou pocompwm pcmsapzcm can Nmppaazm ooppooooom pruume pampa pwopmxga prppms Nopwmo poppcwo Ncoppmpmp ppcpm Ncoppmpop xpp::EEou o '_.,—,._,_,—_,_,—,__,—,_.,__'_ N Npmppoe popcmpco pcmso>opqep wwmpm . pcmeo>opasp —Ncoppu=ppmcp . coppmuzcm wop>pmmcp . pop>Ngwn pcmuzpm .p p.Npooo poop ppo xowcuV mwspp pm:\mp; ucoqm op prpwpm Compaq Npgp pmmm so» on apgmpmcwwp 30 NNNLN sows; :p .mmm po.NpV oN pp.opV 3p po>oz .3 pNNN.N pp.33V pop pp.NNV pm moappoEoN .N Am.pmV mp Ap.pNV mp :wpwo .N Ao.ppV mm Ac.“ V m coppo Apm> .p Npmppms paupcsump Lo pmcopmmmmopq N :o :opcpqo pacoNme psox xmm pm>m :ompma szp Nwoo .vmm x p z N z Noopppoooo opppooom N Egon weep Ego; apesm 42 AN.pNV N3 pN.NNV pN Loom .N po.NNV NN pN.NpV 3p NpNN .3 NNNN.N po.NNV ON pN.pNV Np oooN .N pN.pNV N3 pN.NNV Np oooN Ngo> .N pN.o V pN pN.N V N pooppooxm .p Ncogwppzu vwngzpmpu NNNNcoppoEw pop ENLmoLa Lzox mo cosmeao 95 E. Uw>pwumg m>mz 30>. QHImmmopp .N pN.N V N pp.3 V N pooppwoxN .p NooN op opoNppo>o NNpppZNpNoooo pZNzN>oNozp pNNsz ogp oopgomoo oo» upooz 3o: .NN pN.N V Np pN.N V N Nooo .N pN.NpV N3 pN.NpV Np LpoN .3 NNpN.o pN.oNV pp pN.pNV NN oooN .N pN.NNV 3N pp.oNV NN oooN Ngop .N pN.ppV p3 pp.NpV op pooppooxN .p NENNNoNo Loo» co; op NN< ogp oopgomoo ooN opooz so: .NN oogoppgu ooogopmpo NAP—0:05.85 Lopv EGLWOLQ 9.3 “+0 cowumwoepwn— PacemLma x N z N z Ncowpwucou upwpomam N Epou mcoN Egon “Logm .ooooppooN--.N Npomp 43 pN.3 V op pN.N V 3 Loom .N pN.3NV NN pN.NNV Np NpoN .3 NpNN.N pp.p3V NN pN.3NV NN NooN .N p3.NNV 3N po.NNV Np oooo Ngop .N pN.N V 3p po.ppV N pomppooxN .p ANmup>oga 30> Nmup>gmm esp Lop mpmpgqopaqm appzupppvu No mmgmwu vcm qup .xpmppnppwasou .NprNNNNLN> prcp Np pcwpxm pas: op ..w.pV Nm>LwN so» cmgvasu No azogm esp No =>prpm<¥Noz= wsp mnpgummu so» upzoz 20: .pm Ao.mpV om po.ppV w Loom .m p3.NNV 3N Ap.oNV NN NpoN .3 mqmp.N AN.—mV NN Am.NmV NN vooo .m AN.NNV pN pN.NpV 3p oooN Ngo> .N pq.opV NN AN.N V N pcwppmuxN .— NNpcmusz Lao» No Ncmo: pmcopposw ucw pmcomgma asp mappmme cp 30» 0p anmppm>m puzzommmg oz< mzopmp>ONN m>ppmoaazm wcp mnpgommc so» upzoz 30: .om pp.N V 3p AN.N V N Loom .N pN.NpV N3 p3.NpV Np NpoN .3 mwmm.m po.mmV NN AN.mmV mN uoow .m pN.oNV pN pN.NNV 3N oooN NLN> .N Am.ppV NN A—.N V m paw—Fmoxm .p ANNLoz so» 50;; sppz pprN wucmcwpcpme ucm Ngmcummp .NpcmNNg asp No qusppppm mgp NLN mpnmppsm so: ..m.pV NENLmopa Lao» mcpcgmmmg Nhop No. po pcmopppoNpNN p3.p V Np pN.NNV Np Loom .N po.oNV NN AN.NNV ON NpNN .3 *N3NN.Np AN.pNV NN pN.NNV pN nooN .N pN.pNV N3 pN.NpV Np NooN Ngo> .N pm.m V NN Aw.N V N pcmppmoxm .— NNNooo ooppooop so» Nzopr>oNN szZNNNNN NoNNoN szzz