_'IIIIIIIIIIIIIIIIIIIIII I I'y J '7 Fae-£1!) { l_ r . i. .1.” , “iv—H L. __ “I rm 5“ " a U.‘.33::::7::;!*J This is to certify that the dissertation entitled State of the Art of Alpha and Beta Needs Assessment Models in 1983 presented by Gary H. wanamaker has been accepted towards fulfillment of the requirements for Ph .D . degree in Educational Systems Development b.\ Castelle G. Gen ry Major professor ' Date May a, 1986 MS U is an Waive Action/Equal Opportunity Institution 0.12771 llfllllllllllilllllflllmlmlmlll L 3 1293 10748 0604 MSU RETURNING MATERIALS: Elite in book drop to remove this checkout from w your record: [Emil] be charged 1f book 15 returned after the date stamped below. J ' f ‘ '1 emflWsflé 200 0327 , fl Hazdsgkrdfii M,..~~~ W‘s: _. mm; m 3 .“ S - $3“. '13,: . ”wt-s. legx»-a “New news: - ym 0X 5 '5 6 2 W [33; SW27» ‘5» ». v‘ ‘R “3‘ State of the Art of Alpha and Beta Needs Assessment Models in 1983 BY Gary H. Wanamaker A DISSERTATION Submitted to Michigan State University in partial fulfillment of the requirement for the degree of DOCTOR OF PHILOSOPHY College of Education 1986 ©1987 GARY H. NANAMAKER All Rights. Reserved analyze ABSTRACT STATE OF THE ART OF ALPHA AND BETA NEEDS ASSESSMENT MODELS IN 1983 BY Gary B. Wanamaker The purpose of this study was to collect, present, and data concerning the current state of needs assessment models in the “Alpha“ and 'Beta" categories so that: 1) 2) 3) 4) educators and non-educators alike will have better understanding of the characteristics of the two types of models. a greater illumination of just what is available in the way of Alpha and Beta models will result. both educators and non-educators will be enlightened as to some of the benefits and possibilities of needs assessment in general and Alpha and Beta types in particular. those desirous of conducting a needs assessment can better choose among the available models for their own particular appliction. To accomplish this, it was necessary to: 1) 2) search the professional literature to compile a list of a broadly representative sample of currently available models of the Alpha and Beta types. decribe and evaluate each model according to data collected a) from educational and other related literature, b) developers of the models, c) those who have been served by or who have conducted needs assessments, and d) experts in related areas. 3) develOp a questionnaire and distribute it to model developers and those reported in the literature as having conducted a needs assessment. The results of the mailing are included in a matrix (Table 3) that allows for a reasonably easy comparison of the twenty-one models represented in this study. The major conclusions of the study are that 1) there is a discernable difference between extant models that might enable the models to be classified as either Alpha or Beta. But, 2) although application of such models and their presence in the professional literature is widespread and 3) a ”Unified Model“ incorporating the best characteristics of both types can be developed, there is 4) little research evidence to suggest that the processes connected with such models are useful and valid. ACKNOWLEDGEMENTS My appreciation to the members of my committee: Richard Farace, Lary Lezotte, and Steve Yelon who have provided not only valuable input but inspiration over the years. My special thanks to Cass Gentry, my advisor and committee chair, for his continued guidance, input, understanding, friendship, and, particularly, for the (gentle) pressure he so aptly applied when it was sorely needed. Further appreciation goes to those who responded so thoroughly and promptly to. my requests for the information necessary for this studyls completion. Finally, a thanks to my family and friends who have provided me with the love, patience, and understanding that supported me through this endeavor. LIST OF LIST OF LIST OF Chapter I. II. III. TABLE OF CONTENTS TABLES O O O O O O O O O O O O O O O O O O F I GURES O O O O O . O C C O C O O O O O O . APPEND ICES O O O O O O O O O O O O C O O 0 Introducing the Study. . . . . . . . . . Overview . . . . . . . . . . . . . . . The Problem. . . . . . . . . . . . . . Purpose. . . . . . . . . . . . A Taxonomy of Needs Assessment Models. Need for the Study . . . . . . . . . . Value of the Study to Education. . . . Value of the Study to Educational Systems Development. . . . . . . . . . Definition of Terms. . . . . . . . . . . Summary. . . . . . . . . . . . . . . . . Review of the Literature . . . . . . . . Overview . . . . Alpha and Beta Types Needs Assessment Models and the Systems Approach. . Systems. . . . . . . . . . . . . . The Systems Approach . . . . . . . . Evaluation and Needs Assessment. . . Needs Assessment and Instructional Development. . . . . . . . . . . Needs Assessment and Education . . . Models, the Systems Approach, and Alpha and Beta Needs Assessments . . . . The Roots of Alpha and Beta Type Needs Assessments in the Systems Approach and Scientific Methodology . . . . . Some Early Steps in the Development of Alpha and Beta Needs Assessment Models . . . . . . . . . . . . . . Related Studies. . . . . . . . . . . . . Summary. . . . . . . . . . . . . . . . . Reseach Design . . . . . . . . . . . . . Overview . . . . . . . . . . . . Research Questions . . . . . . . Scope and Limitations of this Study. Data Collection Procedures . . . . . Page iv vi 47 51 55 57 59 59 59 60 61 IV. TABLE OF CONTENTS (continued) Data Analysis Procedures . . . . . . . Procedures for Reporting Data. . . . The Collection, Analysis, and Reporting of Data as Applied to the Individual Research Questions, and the Projected Value of the Derived Answers . . . . Summary. . . . . . . . . . . . . . . . Findings 0 O O O O O O O O O O O O O O overView O O I O O O O O O I O O O O The Matrix and Questionnaire Questions The Matrix Questions . . . . . . The Models . . . . . . . . The Matrix . . . . . . . . Research Question Results. Question #1. . Question #2. Question #3. : Question #4. . Question #5. Question #6. :. . . Z The Unified Needs Assessment .2. e Construction of the Model. Preassessment Consideration Monitoring the System. . . 00003000000000 1 Setting the Parameters . . . . Public Relations . . . . . . Determining System Capabilities. . Decision Makers and Pertinent Data Survey Similar Projects. . . . . . Present Plan to Decision Makers. . Choosing and Modifying the Model . Training/Orientation . . . . . . . Determination of Goals . . . . . Collection of Felt Needs . . . . Reach Consensus. . . . . . . . . Validate Needs . . . . . . . . Reach Consensus on Validated N 3 Two Levels of 'Needs' . . . . . . Futuristic Input . . . . . . . . Input from Previous Goals. . . . Determine Minimum Output Levels. . Current Outputs. . . . . . . . . . Determine and Prioritize Gaps. . . Continue Monitoring. . . . The Interrelationship of Alpha and Beta Needs Assessments and the Other Processes of the Instruc- tional Deve10pment Procedures . . . . . . eed ii Page 62 64 179 179 180 180 182 185 186 187 189 TABLE OF CONTENTS (continued) oneStj-on ’7. O O O O O O O O O O O O oueStion ’8. O O O O O O O O O O O O sumary. O O O O O O O O O C O O O O 0 Summary, Findings, Conclusions, and Implications for Research. . . . . . sumarYo O O O I O O O O O O O O O O I Findings, Conclusions, a d Suggestions Research . . . . . . . . . . . . . . Findings . . . . . . . . . Conclusions. . . . . . . . Implications for Research. Summary. . . . . . . . . . APPENDICES O O O O O O O I O O O O O O O O O 000000-30 A. A Narrative Description of the Alpha Models Included in this Study. . B. The Questionnaire Used to Gather Data Necessary to Answer the Research Questions. . . . . . . . . . . . BIBLIOGWHY O O O O O O O O O O O O O O O I iii cocoon. Page 192 195 196 198 198 202 202 214 222 227 228‘ 228 287 294 LIST OF TABLES Possible Needs Assessment Models and Their Suggested Relationship to a Systems Approach Model . . . . . . . . . . . . . . . . . An Abstract of the Matrix Presenting the Twenty- one Models and the Twenty-three Questionnaire oueStiOflso O O O O O O O O O O O O O O O O O I O A Comparison of Twenty-one Needs Assessment Models as They Relate to Twenty-three concerns 0 O O O O O O O O O O O O I O I O O O 0 Individuals, Events, and Applications Significant in the DeveIOpment of Alpha and Beta Type Needs Assessment Models. . . . . . . . . . . . . Comparison of the Twenty-one Models to 'Idealized' Characteristics. . . . . . . . . . . iv Page 16 84 86 137 194 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. LIST OF FIGURES Needs Assessment Categories and the Systems Approach to Planned Change . . . . . . . . . . The Relationships Between Evaluation Activities and the Stages of a Developing Program . . . . The Major Stages of Evaluation . . . . . . . . A Comparison of the Alpha and Beta Processes . The Unified Needs Assessment Model . . . . . . Dallas Planning Model. . . . . . . . . . . . . The Illinois Problem Index . . . . . . . . . . A Needs Assessment Procedure . . . . . . . . . Phi Delta Kappa Planning Model . . . . . . . . WOrldwide Planning Model . . . . . . . . . . . Coffing's Needs Assessment Model . . . . . . . Lee's Needs Assessment Model . . . . . . . . . Fresno Model for Needs Assessment and Goal Determination . . . . . . . . . . . . . . . . Model of Competency-based Vocational Education Kaufman's Needs Assessment Model . . . . . . . Harless's Front-end Analysis Model . . . . . . Page 17 33 34 149 158 237 243 245 256 263 268 270 277 280 281 284 LIST OF APPENDICES Appendix Page A. Individual Model Descriptions. . . . . . 228 B. Cover Letter and Questionnaire for Survey . . . . . . . . . . . . . . . 287 vi CHAPTER I Introducing the Study Overview The main purpose of this first chapter is to lay the foundation for a study of needs assessment models. The basic problem addressed here is the proposed lack of the application of systematic methods of determining the pur- poses, goals, and objectives that guide the development of educational programs. The proper application of valid needs assessments is offered as one step in this problem's solution. This chapter further documents the current con- fusion concerning needs assessment practices and tools and how this confusion indicates a need for this research. Some of its possible value to educators, generally, and educational systems developers in particular, is suggested. To provide a common context, a list of pertinent terms and how they are defined for the purpose of this work is included. The Problem It is an assumption of this study that much of what is amiss in the condition of public and private education in the United States today can be traced to a lack of a well documented, systematic approach to the definition, develop- ment and evaluation of the purposes, goals, and objectives that form the programs of our educational institutions (Burton and Merrill, 1977; Kaufman and English, 1979; Hannum and Briggs, 1982). Further, it is assumed that, all too often legislators, administrators, educators, and other involved community members, base their educational planning (particularly in the area of curriculum development) on unvalidated information rather than upon well documented data gathered from, and reflecting the educational require- ments of the society that the educational system was estab- lished to serve (Saba, 1980: Hannum and Briggs, 1982; Kaufman and English, 1979). No doubt, this likely has been the result of many influences, not the least of which is a general lack of awareness on educators' part about the value of, and the means for collecting such data (Trimby, 1979; Kaufman, 1977; Sarthory, 1977). One means of identi- fying educational purposes, goals, and objectives of our institutions has been through the use of a set of tech- niques called ”needs assessment“. Currently, information on these techniques and their respective values is not clear. This research is designed to provide findings for facilitating a user's understanding, selection, and imple- mentation of such techniques. That public education has been subject to a loss of credibility, prestige, and public confidence over the last decade is a widely publicized and comon observation. The May 9, 1983 cover story of Newsweek opened with: The writing on the blackboard in Washing- ton last week was bleak. In an "open let- ter to the America people,“ The National Commission on Excellence in Education stated bluntly that "a tide of mediocrity" has devastated public education. It likened the shambles to ”an act of war." ”We have in effect," warned the report, “been committing an act of unthinking, unilateral, educational disarmament.” The article went on to say: The sum of this report is that one of the fondest assumptions of America life-- progress from one generation to the next-- has been nearly shattered. “Each genera- tion of Americans has outstripped its par- ents in education, in literacy and in eco- nomic attainment: For the first time in the history of our country, the education skills of one generation will not surpass, will not equal, will not even approach, those of their parents.“ Some specific and “frightening“ findings of the Commission conclude that: 1. the quality of teaching was woefully lacking. 2. academic and vocational studies are being aban- doned for ”general track” electives. 3. one fifth of all four year state colleges must admit anyone with a high school diploma. 4. 29% of our colleges became less selective in admissions over the decade. 5. colleges are more interested in maintaining enrollment than high academic standards. 6. on most levels, U.S. students are seriously defi- cient when compared to those of other industrial- ized nations. 7. there is a vicious cycle at work in that many teachers are themselves the product of that same system that produced the generation‘s educational decline. This current ”dreary state of American education" including “abysmal teacher education" has left a “genera- tion of young people ill prepared for the new era of tech- nology and global competition.” At risk is ”our very future as a Nation and a people." (Terrance Bell, 1983). "The evidence of a consistent and significant decline seems incontrovertable, and is rarely’ disputed” (Weller, 1982). For the past fourteen years, the Gallup Poll of the Public's Attitudes Toward the Public Schools has measured the public's opinions. In 1981, the 'bad" grades outweighed the ”good“ grades by eighteen percentage points (54% to 36%) (Weiler, 1982). SAT scores have dropped from 466 out of a possible 800 in 1968 to 424 in 1981 (Freidrich, 1982). Unfounded or not, the following com- ments are not uncommon: Of incoming freshmen: “They haven't received an education, they've just had babysitt- ing" (Professor Norman Land, Art. History, University’ of Missouri, Time, Sept. 27, 1982). "We might begin to define the educated person as one who can overcome the deficien- cies in an educational system“ (Wisconsin's President Robert O'Neil, Time, Sept. 27, 1982) "...1arge groups of parents and taxpayers were becoming increasingly disillu- sioned with their schools (and) are beginning to look at educators with mounting distaste (Brodinsky, 1982). Proponents of the New Right charge the schools with “total failure”. They claim the ”schools have bungled their job", pointing to lack of discipline and study, unprepared teachers, subjective grading systems, charging “in general, too much pedagogical faddism“ (Marshner, 1978). The educational system is under attack because it has become suspect (Stein, 1982). “Quality is almost certainly going to turn out to be the foremost national education concern of the 1980's“ (Chester E. Finn, Professor of Edu- cation and Public Policy at Vanderbilt, 11113, Sept. 27, 1982). Decline of public confidence is almost certain to effect public behavior (Weiler, 1982). If the opinion that 'On the basis of their record alone, educators should, for the most part, be embarrassed to ask for more money“ (Dixon, 1982) is widely shared, then eroded public support for the public school system (Weiler, 1982) may be in dir- ect proportion to their willingness to continue funding. Since the list of criticisms are not totally from the uninformed but from those who have obviously done their homework (Weiler, 1982) and educators as well, they may not be totally unfounded. Certainly the educational system may lack the necessary political, community, and financial sup- port to make the system function optimally (Brodinsky, 1982). But educators who use these as excuses to answer all of the proposed failures of the educational system (McGraw, 1982; Stein, 1982) ignoring the above criticisms, may be setting the stage for a great deal more unwanted, but possible warranted, turbulence from an already wary public. These substantial criticisms exist. It is not the purpose of this work to gloss over the current condition of education in America, the plight of the recent generation of learners, or the potential consequences to our society. But a closer scrutiny of some of the criticisms might indi- cate that fingers are not being pointed entirely in the most appropriate or constructive direction. If it is true that learners are being insufficiently prepared to assume a responsible and productive place in society as the result of ill-prepared educators, teaching useless subject matter, and that poorly, a paradox seems to exist. Strong contrary evidence exists that we, as educators, have made signifi- cant strides (Hannum and Briggs, 1982) toward the develop- ment of a science of instruction (Glaser, 1976). Evidence is mounting as to the ability of systematically developed instruction to produce positive results (Anderson et al., 1977; Glasser, 1978; Klahr, 1976). Implications have been drawn from research-produced data (Gage and Berlinger, 1979; Gagne, 1977b) strongly suggesting our current ability to produce the necessary, effective, and efficient instruc- tion. This may likely come about if a specific statement of the objectives if instruction is made (Mager, 1962), based upon needs assessments conducted in the community served by the schools (Burton and Merrill, 1977; Kaufman and English, 1979), and appropriate conditions of learning (Gagne, 1977a). Proper attention should also be paid to learner entry behavior (DeCecco and Crawford, 1974; Ausubel et al., 1978; Gagne, 1977), and the nature of the particular instructional strategies (Bretz, 1977: Briggs and Wager, 1981) and media (Peterman, 1982; Jamison, Suppes, and Wells, 1974; Schramm, 1977), supported by proper evaluation and concurrent modification (Dick, 1977). Kaufman (1979) offers the suggestion that part of the problem lies not so much in the ability of the present system to produce the necessary results as in the misappli- cation of technology and the lack of proper criteria to judge their merit. He suggests that, all too often, ”cor- rection" consists of adopting solutions before the problem is systematically defined. “When you do not know where you are going and select a method for getting there, you are open to failure at best, and failure plus severe criticism at worst. If we keep on going the way we have been with the application of technology both 'hard" technology (the “things“ or our field, such as TV, computers, and the like) and “soft“ technology (the use of a process for identifying and then resolving problems), we will fail for the wrong reason, and that is a reality to worry about." It is no wonder that faith has been shaken in a much heralded "edu- cational technology" (Kaufman, 1979). Kaufman adds that it may seem logical that if resources or means are not available or if ends are not defined that no one could really be held accountable. But the fact is, it is the educator that is put in a position to be blamed. 'They plead for necessary resources, are turned down for lack of money and are then blamed for poor results. Educators put themselves on the defensive, and, being anxious to demonstrate improvement to the critical sources of their finances, emphasize newsworthy but super- ficial efficiency, cost reduction programs, or center stage budget cutting. Ends are ignored and means are put in the spotlight. But if the emphasis is switched from the means (process) to the ends (product), educators may find them- selves in a better position to 1) help the learners help themselves to measure their success, 2) plan on the basis of ends rather than on the latest means that are available, 3) gain more results from their efforts and a better feel- ing about their jobs, and 4) improve the image of themselves and their profession to the world (Kaufman, 1979). Kaufman offers needs assessment as partial solution to some of the problems currently plaguing education: Needs Assessment. A great idea! In an era of accountability, a public management concept whose time has come. And why not: A management tool which can identify gaps between organiza- tional ideal and real as well as suggesting staffing patterns, program emphasis and resource allocations designed to close the gaps is a valuable one in a milieu of shrinking resources and distrust of public instruction. Most education administrators like the concept. They know the promise it holds. They are aware of the rational basis it affords for program, personnel, and resource decisions. They appre- ciate its capability to effect institutional accountability in the finest sense of that term. They sense its ability to improve organizational direction-finding by helping administrators differentiate between means and ends; to identify the difference between desired and current organizational achievement; to assess the validity of means being used to attain organizational purposes; to set forth indicators to redesign educational means; and to sense appropriate timing for changing organization goals (Kaufman, 1979). Needs assessment can provide a "clear cut approach to determination of needs“ including data useful for both formative and summative evaluation (Helge and Marrs, 1978). ”Needs Assessment represents a formal set of tools and techniques and a way of viewing the world, of intervention and positive, productive change for putting means and ends into proper perspective“ (Kaufman, 1979). It is a ”process by which the unfulfilled educational requirements of a pop- ulation of students are identified (and)...for determining the educational objectives most appropriate for a particu- lar situation" (Lee, 1973). Through a properly conducted needs assessment, data can be gathered to articulate and coordinate curricular programs from kindergarten through the twelfth grade (Buhl, 1978). Further, needs assessments can be used to positively involve those other than the professional educator. The process may "actively pursue the development of the concept of "educational partners”. The partners have been defined 10 as the community, the professional staff, and the learners. Kaufman has advocated that the partners should have equal weight in the ranking of education goals“ (English, 1977). The public is asking for a greater voice in educa- tional decisions (McGraw, 1982) and for control to be turned from government back to locally elected school boards (Dixon, 1982). There are such extremist groups as the New Right who would abolish the present system alto- gether rather than further supporting it (Brodinsky, 1982), including the discontinuation of even the Department of Education (Dixon, 1982; President Reagan, Newsweek, May 9, 1983, p. 53). Educators themselves are attacked when it is suggested that they should ”clarify their values and sub- ject themselves to rigorous self-examination of their goals, motives, values, and methods” (McGraw, 1982). The Federal Government, many of the states, industry, and concerned citizens have expressed an interest in, sup- port of, and involvement for upgrading the quality of pub- lic education. North Carolina, Kentucky, Tennessee, Florida, and other states have prOposed or begun programs and the necessary funding and laws that may have far reach- ing, and hopefully, positive consequences for their educa- tion systems (Newsweek, May 9, 1983). When asked to con- tribute privately, the citizens of a suburban area north- east of San Francisco, population 10,000, raised $420,000 in support of local education (Newsweek, May 9, 1983). Honeywell, Motorola, and other corporations put up 10 mil- 11 lion dollars for research grants for a new engineering cen- ter at Arizona State provided the State match with an additional $20 million, which it did promptly. More than 100 businesses in Boston have agreed to give hiring priority and additional training to city school graduates if standards are raised, including adequate verbal and math skills (Newsweek, May 9, 1983). A good case can be constructed demonstrating the will- ingness of industry, government, and private citizens to become involved in solving our educational problems. Needs assessment offers one positive way of directly involving these groups in the educational process (English, 1977). Proper application of needs assessment offers a greater likelihood that efforts and resources will be used in a more positive and productive manner by setting the founda- tion for a systematic application of extant educational personnel, knowledge, and resources (Heldge and Marrs, 1978: Kaufman, 1979; Lee, 1973, Buhl, 1978; English, 1977). The direction provided by the well defined and valid goals and objectives that a properly applied needs assessment can provide, gives some assurance that the vast resource that can be tapped from private and public sectors will be used productively to aid the next and future generations of learners, and serve society. 12 Purpose A 'need" may be defined as “a measurable outcome discrepance between 'what is' and 'what should be':" if there is no discrepancy, there is no need (AECT Glossary, p. 207). With that definition, "needs assessment", stated simply, is a process for determining whether or not such discrepancies exist in a system. The broad purpose of this study is to examine the use of needs assessment models in education, for purposes of bringing together what is known about such use, for the benefit of educators. To be even of minimal benefit, educators themselves must, of course, be knowledgeable of and able to choose and apply the proper methods, tools, and techniques, associated with needs assessment. The main purpose of this work is to collect, present, and analyze the current state of needs assessment models of the 'Alpha' and 'Beta' categories (Kaufman, 1977) so that: l. educators and non-educators alike will have better understanding of the characteristics of the two types of models. 2. there will result a greater illumination of just what is available in the way of Alpha and Beta models. 3. educators and non-educators will be enlightened as to some of the benefits and possibilities of needs assessment in general and Alpha and Beta types in particular. 4. 13 those desirous of conducting a needs assessment can better choose among the available models for their own particular application. A study undertaken for Project Next Step: Mutuality of Planning (Kaplan, 1974), in the handbook Needs Assessment in Education, listed the following as the likely benefits and results of a carefully conducted needs assessment: 1. 2. 3. 5. 6. provide necessary community pressure calling for citizens to (express their views, leading to a systematic tapping of those views, and providing for community involvement. Such information can prove helpful for not only gathering data to set educational objectives and goals, but also to gain information that may be beneficial if a school is suffering from such ailments as school budget defeats. help pin-point problems precisely. show how real problems were not being attacked, if, for instance, a new program isn't working. provide data which can lead to well-defined, verifiable educational goals. clear a smoother path toward goal achievement. provide a data base for future educational decisions. establish a resource bank for information of a district. generally, provide a means for a. reducing internal and external problems, b. developing an organizational structure and process for continued self-evaluation, and c. providing a basis for future planning and problem solving. 14 Given the proliferation of needs assessment models and tools (Witkin, 1977), the lack of clarification as to what is and is not a viable needs assessment process (Sarthory, 1977; Trimby, 1979: Kaufman, 1977), and limited knowledge of users of needs assessments (Sarthory, 1977; Witkin, 1977), this study is dedicated to completing the following specific tasks: 1. to enhance and broaden the readers understanding of needs assessment in general and Alpha and Beta types in particular by presenting a brief overview of their historical development and application. 2. to furnish descriptions and analysis of the established Alpha and Beta models currently available in Education. 3. to further enlighten the reader as to possible benefits of undertaking Alpha and Beta type needs assessments. 4. to» compare and contrast the characteristics of extant Alpha and Beta type models. 5. to describe an 'ideal' or “unified” Alpha and Beta model. 6. to describe how extant models compare to a unified model. 7. to ‘present references to any research evidence available demonstrating the ability' of the available models to acquire the information for which they were designed. 8. to present information that could aid in the user's choice of a model for a particular application. 15 A Taxonomy of Needs Assessment Models According to Kaufman (1977, 1979), in order to minimally achieve the benefits listed by Kaplan (1974) of a well conducted needs assessment, it is necessary that the educational system begin its planning efforts and base all future decisions and planning upon what he classifies as an Alpha type needs assessment. Before explaining his reasoning, it could prove useful at this point to take a look at needs assessment in general and, since this work will be dealing with Alpha and Beta type models, look also at how these specific types fit into that overall picture. Kaufman (1977) offers a taxonomy of needs assessments through which he attempts to categorize those models, methods, surveys, and other approaches that fall under the blanket label of ”needs assessment”. The six step taxonomy (Alpha, Beta, Gamma, Delta, Epsilon, Zeta) in Figure l, is relate to the models and other means to needs assessment used as the first step in accomplishing a systems approach to any planned change (Figure 2). To best understand how the models and other means to needs assessment are classified, it might be helpful to begin by describing one of the major divisions of Kaufman's taxonomy. The major dividing categories used by Kaufman to separate models into types are the designations ”External“ and "Internal". First, it should be understood that there are considerable similarities between "External“ and 'Internal' 16 Reeds Assesmnt Systm Approach Functions Type OIIIOCIOHSHCS 1.0 NINTH-Y m External criteria. partnership $683009 w ALPHA based, utility reierent for survival. Single aphasia upon “need“ as outcore gap. 2.0 WEE 501mm Partnership-based, analysis of perfor- REQJIWS MD BETA nance gaps within the system analysis llfllllFY scwnm of process and solution gaps. Focus ALWIVES on outcom and process gaps. 3.0 $8.83 SOJJTICN Ranking oi solutions by partners. SIRATEGlES BOA ova Cost-efficiency nudels, cost-effectiveness M06 ALTBNVTWES nodels, etc. 4.0 m Determination of gaps in pre-speciiied [ETA pertomnces. Minagmnt—by-obiectives. magmntoby-exception, scheduling, etc. 5.0 WE htermine discrepancies between m B’Sllfll results and objectives for the H’FECTWNSS end-oi-ter'mlprojectlprogran tor decisioning. 6.0 EVISE AS Err-routs evaluation oi both processes REQJIRED ZEIA and progress towards outcms. Table 1 Possible Needs Assessment Models and Their Suggested Relationships to a System Approach Model (Kaufman, 1977). 17 0.0 “I!” m I A) A. .‘o eeooeeeaewoonoeeoaoe’eaeeoaaao%oaoao’ O O o O . e ' ° 0 1 . 3 WM ’ '-) WINS 3.0 tum mm (W) «arm (“I“) Figure 1 Needs Assessment Categories and the Systems Approach to Planned Change needs assessments and some overlap in the methods used in conducting the two. Although, by definition, a properly conducted external needs assessment must gather verifiable data outside of the target education system, the same type of processes are used to gather information from within the system. By the same token, internal needs assessments may gather information from external sources as well as from within. The chief distinction between the two approaches are l) the beginning assumptions made concerning the purposes and goals of the organization involved, and 2) the point in the educational develOpment process at which the assessment begins. Both internal and external approaches have a central thrust: the determination of needs. Nowhere in the literature has it been suggested that a particular mode of needs assessment is ”right“ or 'wrong', but only appropriate or inappropriate for the specific task in question. 18 Some of Kaufman's and English's (1979) philosophic views on education might be useful in understanding the distinction between the concepts "External“ and 'Internal'. They believe that if what is taught in schools is not related to what the learners are doing or are expected to do in the future world outside school, why bother with it? School is a prelude to life, a life of change, of coping, of success in, and contribution to society. If education does not provide the tools for accomplishing these, it is a waste of both the learner's and educator's time and of the resources provided by the public which the educational system is meant to serve. Society is viewed as a suprasystem, with one of its subsystems being the school. The school's purpose is to prepare the learner for ”survival and contribution...within a flexible society governed by laws that could be changed" (Kaufman, 1977). In their monograph, Needs Assessment (1979), Kaufman and English offer the following explanation for external and internal approaches to needs assessment. An external needs assessment, or "The Way It Should Be", enables educators to determine whether what is being taught in the school is valid and useful in the suprasystem, the society, both for current and future needs. This process does not begin with the assumption that the educational institution's present purpose, goals, objectives, and practices are valid or useful. The main purpose of such an assessment is that level of policy 19 formulation concerned with defining the purpose and goals of the organization in question, the needs of the learners in the system as they relate to the society which the school is meant to serve, and setting goals and objectives accordingly. External needs assessment is concerned with the 'what“, the product, or content of education, not the "how“, process, or means for accomplishing those goals and objectives. This process necessarily involves "partners in education“ (educators, learners, and community) in order to gather verifiable data to deve10p these purposes, goals, and objectives. Such an assessment serves a dual purpose, not only of defining and describing the social purpose of the educational system and the ends toward which it should proceed, but that of identifying and defining criteria by which relevant persons may judge the outcome of educational processes. The term 'need", then, is viewed as an outcome 922- Alpha type needs assessments are of the external variety. The processes associated with the Beta type needs assessment take up from where the distinctive processes of the external, or Alpha type, leave off and is classified as "internal“. Kaufman and English refer to internal needs assessments as "The Way It its" because they represent the current prevalent way of determining needs and planning change. The main emphasis and purpose of these approaches is to Optimize the desirable features of what is already going on. It is assumed that the organization is the 20 proper starting place for planning, changing, and doing, and that its current purposes, goals, and objectives are valid and useful. Needs, viewed as either product ('what') or process (”how to“) gaps, are determined in accordance to the aforementioned assumptions. The degree of revision and renewal is, or course, limited to the confines of those assumptions. To be minimally beneficial to an institution, these assessments should be based upon data gathered from external, or Alpha, assessments. The Beta type, as mentioned above, is the most basic type of internal assessment (Figure 2), and like the Alpha, is concerned with the 'what" or product of education. Unlike the Alpha, the Beta assumes the validity and utility of previously established needs, goals, and objectives, and instead: 1) focuses on the analysis and development of these into specific subordinate goals and objectives, 2) may further assign these a priority, 3) identify solution requirements, and 4) suggest possible alternate means to fulfill those requirements. It should be noted that Kaufman and English do not assume that these types of models exist in a "pure" form, but that models and means to needs assessment have been developed that contain the features or whose main emphasis and priorities will allow them to be classified into one of the six categories of the taxonomy. A model containing the necessary features to qualify it as Alpha, may contain, in addition, all the features of the other types. But its 21 ”external“ features would qualify it as an Alpha type. A Beta, likewise, may contain those characteristics associated with the Beta type and, in addition, all the features of subsequent types in the taxonomy, but will be classified ”Beta“. It will not, though, contain the characteristics that classify a model "external". And so on up the taxonomical progression. This work is limited to Alpha and Beta needs assessment categories of Kaufman's taxonomy. Need for the Study Boards of education and educational administrators and managers are facing the dilemma of being asked for instant solutions to educational problems (Kaufman, 1979). ZIt has been suggested that properly conducted needs assessments can provide the necessary questions and answers to remove that pressure (Kaufman, 1979). In the last decade, there has been a proliferation of needs assessment models, methods, and tools of needs assessment (Witkin, 1977), and, as with most techniques in the development state, there exists considerable confusion as to what is and what is not needs assessment (Kaufman, 1977). Further, the literature reveals differing means and uses of central terms (Sarthory, 1977: Trimby, 1979). A first step in a well conducted needs assessment is the identification of tools and their availability (Melton, 22 1977: Sweigert, 1977). Administrators desire help in choosing from among available needs assessment models (Witkin, 1977), but unfortunately, efforts to present necessary information and to tie together this undeveloped area have been piecemeal (Witkin, 1975). With the exception of Kaufman's suggested taxonomy (1977), there is no systematic framework presented in Educational literature to bond the area of needs assessment into a comprehensive whole, no consistent criteria or guidelines (Heldge and Marrs, 1978: Kaufman, 1979). This work will seek to present selected needs assessment models of the Alpha and Beta types to further clarify, describe, and evaluate the present state of the first two categories of the taxonomy. Value of the Study to Education A presentation, description, and evaluation of extant Alpha and Beta type needs assessment models: 1. could be a step forward in developing the much needed framework, definitions, and terms central to needs assessment. 2. could lead to an operational understanding of Alpha and Beta type needs assessment models. 3. could create a greater awareness of the types of information that these models could produce. 4. could help create a greater awareness of the need for the type of information that assessments of the Alpha and Beta types can produce. 23 5. could create an awareness of what is available for practitioners. 6. cold create a framework for judging the usefulness of such models for application in particular settings. Value of the Study to Educational Systems Development Because an educational systems developer may be called upon at any stage in the development, process (Dick and Carey, 1978), it may be necessary, if the developer has a concern for the validity and utility of a program, to know the origins of the goals and objectives that set the guidelines for the entire development process. This study, besides being useful for educators in general, could prove beneficial to educational systems developers in particular by: 1. providing description, evaluation, and reference to those needs assessment models which might prove useful to specific educational systems development applications. 2 . helping the developer better understand the origins of a system's perceived needs, goals, and objectives if called upon to enter the development process after they have been established. 3. enabling the developer to better identify possible strengths and limitations that may be (or could have been) inherent in the application of a given needs assessment process in a particular situation. 24 Definition of Terms Alpha Needs Assessment: An Alpha type needs assessment is a formal procedure that involves relevant persons and resources to identify and select the data needs to: 1) define and justify the purposes, goals, and objectives of an educational system and 2) identify, justify, and prioritize the needs, both met and unmet, of its learners, judged upon exiting the educational system (Kaufman, 1979). Beta Needs Assessment: A Beta type needs assessment is a formal procedure, involving relevant persons and resources, designed to identify, justify, and prioritize the needs, both met and unmet, of its learners, as judged according to pre-established purposes, goals, and objectives. Evaluation: Evaluation is ”the process of (delineating, obtaining, and providing useful information for judging decision alternatives“ (Stufflebeam, 1971). gage}: A model is a conceptualization in the form of a narrative or graphic analog representing a real life situation. Need: "A need is a necessary or desirable condition, state or situation--whether it be an end result in actuality (met need) or discrepancy that should be closed between a current or projected actuality and necessary or highly desirable end result (unmet need)--as judged by a relevant person or group using multiple objective criteria that have been agreed upon“ (Lenning, 1978). 25 Needs Assessment: A needs assessment, often used interchangeably with “needs analysis” in the literature (Provus, 1972; Kaufman, 1972), is a formal procedure which identifies, justifies, selects, and prioritizes the quantitative and qualitative extent of the needs in both 1) the ability of the recipient to survive and contribute upon exiting an educational setting and 2) the ability of an educational system to provide the necessary experiences to meet the needs of the recipient of instruction (Melton, 1977: Kaufman and English, 1979). Summary This chapter has addressed both the purpose and rationale of this study. The purpose of the study, generally, is to provide useful information to both educators, and others involved in needs assessment, for understanding and choosing appropriate models for part- icular application. To help the reader better conceptualize needs assessment in general an the particular focus of this work (Alpha and Beta type needs assessments), a taxonomy of needs assessments was offered as well as necessary definitions of pertinent terms. Chapter II will present a review of the related literature, Chapter III the research design, Chapter IV the description and analysis of the individual models, and Chapter V, a final summary and the conclusions, and 26 implications for continued research immanating from the study. CHAPTER I I Review of the Literature Overview One of the purposes of the previous chapter was to make clear how Alpha and Beta needs assessments are interpreted for this study. The purpose of this chapter is four fold. First, to provide additional clarification of the meaning of .Alpha and Beta needs assessment models, attention is directed to literature reporting how some knowledgeable individuals in other areas related these varieties of needs assessments and their possible uses to Education, as a ‘wholee Second, since needs assessment “models" are being considered, a look is taken at some general considerations of model building and how this affects the analysis of Alpha and Beta needs assessment models in particular. Third, to gain insight into the nature of these types of needs assessments and their uses, an historical look tracing the development and application of these models is offered. Fourth, and finally, since one of the major purposes of this study is to present information to the potential user, helpful in both understanding and choosing from among extant models, a review of similar studies regarding these types of models is presented. 27 28 Alpha and Beta Needs Assessments Models and the fistems Approach The concept “system" was introduced in Chapter I in connection with the Alpha and Beta types of needs assessment with which this study is concerned (Kaufman, 1977). Although there are needs assessments that may not take a "systems approach” to the discovery of “needs” (discrepancies between 'what is' and 'what should be“), the Alpha and Beta type needs assessments as described by Kaufman (1977), necessarily involve this approach to problem solution. Before attempting to trace the evolution of these types of needs assessments, and, before describing the systematic interrelationships of needs assessment, other forms of evaluation, instructional development, and education as a whole, it will prove useful to take a look at the systems approach itself. First will be offered a definition of ”system", then the pertinent characteristics and activities of the ”systems approach“ as they relate to Alpha and Beta type needs assessments. Systems For the purpose of this discussion the concept ”system“ will be defined as "the sum total of separate parts working interdependently and in interaction to achieve previously specified objectives“ (Kaufman, 1972). This contains the considerations found in most other 29 definitions of “system”, that is composed of: 1) separate parts that 2) interact with one another to 3) achieve an objective. This concept is necessary to both the understanding and application of Alpha and Beta needs assessments. The entire purpose and functions of these assessments depend upon their being applied in a systems way. First, by definition (Kaufman, 1977), the Alpha assessment must go outside the target system to draw information from the larger societal, or 'suprasystem", that will eventually be the basis for the purposes, goals, and objectives of the "suprasystem' (Silvern, 1965). Secondly, the products of both Alpha and Beta needs assessments should serve as the basis for all subsequent decisions. Finally, the needs assessments themselves, as they are conducted, are made up of processes and events that act interdependently to achieve a specified objective. Attention should be drawn to one aspect of the above definition that: the components act “interdependently“ and in "interaction”. Kaufman's addition of these parts functioning individually and in relationship to the other components, could prove useful when analysis of an educational system is undertaken for the purpose of singling out those parts of processes that may be the source of a particular concern. This, and the other ideas above,. will become more evident as the "systems approach' itself is discussed and a description of the interrelationship of needs assessment, evaluation, 30 instructional development and Education is offered. The Systems Approach The "systems approach” has been described as a way of "thinking about the total system and its components" (Churchman, 1968). Churchman suggests that though there are many ways of defining the "systems approach“, the inclusion of the following five areas are at least minimal and informative. We should consider: 1) the total system and, more specifically, the performance measures of the whole system. 2) the system's environment: the fixed constraints. 3) the resources of the system. 4) the components of the system, their activities, goals and measures of performance. 5) the management of the system. Even though the description of the system in question may not be completely definitive, if enough of the necessary characteristics of the above five areas are properly arrived at in an analysis, consideration of all these possible complex interactions should allow us to solve problems effectively and efficiently (Kaufman, 1972). Churchman suggests that if we do not at the outset of our planning consider at least these five areas and their interrelationships, or postpone our thinking about any of 31 them for too long, we may not be able to think about or plan for their influence adequately at all. With appropriate management, a negative situation that could be avoided by previous planning may never be encountered. The main processes of the systems approach (i.e. analysis and synthesis) have been coined 'Anasynthesis' by Silvern (1965). He defines analysis as the "breaking down of a whole into its parts, showing the relationship of the parts to each other and to the whole itself." Synthesis is defined as I'the process of combining non-related elements into a meaningful relationship such that the new product is a whole system." Anasynthesis, or the “systems approach”, is analysis and synthesis applied together on an interactive basis. It's two main operations are: l) the construction of models and 2) simulation of a real-life situation. More will be said later about the functions of models in this relation. The above is compatible with the “synthetic aspects of simulation“ advanced by Norlen (1972): l. The analysis implies a breaking down of a complex problem concerning a "whole” into 'parts" on one level, where earlier pronouncements and reasonable new hypotheses concerning the 'parts" and their relations can be actualized and investigated. 2. The construction consists of the formulation of a model which implies a joining together of the 'parts" and their relations which are obtained from the analysis. 32 3. The synthesis consists of simulation of ”the behavior of the parts" within the frame of the model. The formulated Alpha and Beta needs assessment models are constructs used to picture the processes of needs assessments. The components of the system are pictured, demonstrating their interrelationships in such a way as to suggest the functions of the individual aspects acting in conjunction to arrive at the behavior of the whole, or the results of application. Evaluation and Needs Assessment Evaluation is one component or subsystem of an educatibnal system (Gagne and Briggs, 1979) (Figure 2) and is of prime importance, when coupled with appropriate revision, to the instructional systems approach (Hannmm and Briggs, 1982: Dick, 1977). It should be an ongoing process systematically planned and based upon data derived from the educational systems environment, the society, from which information is drawn to define the learners' performance in terms of expressed objectives (Burton and Merrill, 1977; Kaufman and English, 1979: Hannum and Briggs, 1982; Mager, 1962). Alkin and Fitz-Gibbon (1975) break the evaluation process down into three major stages: Pre-formative Evaluation, Formative Evaluation, and Summative Evaluation (Figure 3). iTKIIUNW STAGES initial determination of Problem Conceptual izat ion oi Program: Early cycles of prograns Progranlundergoing continual revision Programiin later cycles-- stabilized. 33 Iterative. Continual interchange. continual interchange of inionmation. Figure 2 ETHLJIJUWNTIVE ENAUUNTKJQ RUHWNTTVE EMAUUKTNJQ SINNHJIVE ENAUUKTNJQ The Relationships Between Evaluation Activities and the Stages of a Developing Program. 34 ENMUJJNTKJN STAGES W Needs Pr ogran assessnent Planning evaluation tneasures of current status, goal selection, pilot studies J. EKJUWATlVE.ENUUJJNIl Implementation Progress evaluation evaluation measures of program fidelity, student progress in individual units l SLNNPJTVE.EVAUUNTNJQ Documentation Outcam evaluation evaluation unobstrusive, non-intefiering, no feedback to program Figure 3 The Major Stages of Evaluation. 35 Pre-formative Evaluation, consisting mainly of needs assessment and program planning evaluation, is the first stage of evaluation that should be conducted prior to the implementation of a program (Alkin, 1973). It's chief functions are to gather information on the perceived importance of relevant goal areas, their current status, and relative priorities, and to provide evaluation information about alternative educational programs that might be used for achieving the desired goals (Alkin and Fitz-Gibbon, 1975). Formative Evaluation (Cronbach, 1963; Scriven, 1967) is used by those responsible for developing and running a program for program improvement. “After several cycles of formative evaluation, a program should stabilize and run on its own with no more monitoring or information feedback than it will receive if it is adopted as an on-going program“ (Alkin and Fitz-Gibbon, 1975). The third and final stage, summative evaluation (Scriven, 1967), measures a program's success at reaching its goals. Unlike formative evaluation, the results of summative evaluation are not used for program improvement but as the basis of such decisions as program adeption or continuance/discontinuance. As briefly mentioned above, needs assessment is one of the stages making up the evaluation component in an educational system (Figure 3). One major distinction between needs assessment and other forms of evaluation may 36 be the time element (Witkin, 1975; Trimby, 1979). While "needs assessment looks at 'what is' and compares it to 'what should be', looking from present to future, evaluation looks from the present to the past as it asks 'what has been the impact' of a given program or product on student learning or 'what was done'.” (Trimby, 1979). Needs assessment has been suggested to be a system whose functions span the total gamut of evaluation (Kaufman, 1977). The major concern here are those classified as .Alpha and ‘Beta type needs assessment, or those of the pre-formative varieties. One important outcome of a needs assessment is the development of the information that is formed into the criteria upon ‘which subsequent forms of evaluation take place (Burton and Merrill, 1977; Kaufman and English, 1979: Hannum and Briggs, 1982: Dick and Carey, 1977). An assessment, according to Sullivan (Popham, 1979), spawns needs stated in measurable performance terms with minimally acceptable performance standards for each. Concurrent with their assessment of the learner's current status and likely instructional activities is the derivation of the information necessary to formative evaluation. If needs are prioritized, their interrelationship established and minimal performance standards set, not only appropriate treatment, but the necessary evaluation measures can be accordingly designed, applied and subsequent revision made . Summative 37 evaluation, accordingly, draws upon and systematically gives feedback to the needs assessment (Dick and Carey, 1977). The needs assessment provides the necessary information that will be formulated into the final standard or outcomes by which the completed program will be measured. Information derived from the summative evaluation is in turn applied to the needs assessment phase of the instructional develOpment process for further study of the system's needs. Because of its close interrelationship to evaluation, and because it may likely be the starting point of any evaluation, a needs assessment could reasonably be seen as a system proceeding along the evaluation system's spectrum. On the other hand, when looking at only the Alpha and Beta type needs assessments, or pre-formative evaluation, these two forms are more readily discernable as the most basic form of assessments, providing, at once, the information that forms the foundation to all subsequent evaluation procedures, and then, inclusively, other types of needs assessment. This is the foundation or referent point for the entire instructional development process. As such, needs assessment operates interdependently, and systematically, with all the other components of the educational system. 38 Needs Assessment and Instructional Deve10pment The systems approach to planning and problem solving, based upon systems theory, has been variously termed systematic instruction (Dick and Carey, 1978; Popham and Baker, 1970), instructional systems development (Branson et al., 1975), instructional design (Briggs, 1977: Briggs and Wager, 1981; Gagne and Briggs, 1979), instructional development (Silber, 1978), and educational or instructional technology (Armsey and Dahl, 1973; Wittich and Schuler, 1979). A similarity among these is that ”the instructional system is viewed as composed of various interrelated components functioning together to achieve a purpose. The goals of the instructional system are derived from the environment of the instructional system. Thus, the instructional system is a subsystem of the larger system, the environment of the instructional system“ (Hannum and Briggs, 1982). The gathering of data, from the world external to the educational system, (i.e. the community) to form the purposes, goals, and objectives of the educational system is one probable function of a pre-formative needs assessment. (Mager, 1962: Kaufman, 1972; Kaufman and English, 1978; Hannum and Briggs, 1982). It is through the application of an Alpha type needs assessment that the link is established between the external society and the target system. From this process, 39 data is gathered that provides the information necessary to establish the educational goals, objectives and the subsequent means necessary to prepare the target learners to ”survive and contribute” to that society upon exiting the educational system (Kaufman, 1972). After needs have been selected, stated in measurable performance terms, and prioritized, the Beta type assessment identifies what is needed to close those gaps and descr ibes the advantages/disadvantages of possible alternative solutions. "In needs assessment, the evaluator provides information on the perceived importance of relevant goal areas, their current status, and the relative priorities of each...(and) information about competing educational programs that might be utilized for achieving the desired goals.” (Alkin and Fitz-Gibbon, 1975). If the instructional process has its roots in such assessments, the processes and inputs selected are more likely to result in socially valued ends (Kaufman and Stakenas, 1981). The closing of these established needs or 'gaps' should be the accomplishment toward which the efforts of the educational system's efforts proceed. They provide the direction for systems planning and development allowing the system's planner to know where he is going and when he has arrived (Mager, 1962: Kaufman, 1972). Instructional development activities, facilitated by a well conducted needs assessment include such information for the writing of measurable performance objectives (Mager, 1962: Popham, 40 1969) and subsequent evaluation (Alkin and Fitz-Gibbon, 1975) as: 1) type and level of performance expected when instruction is completed, 2) type of environment in which the criterion referenced assessment should take place, 3) interrelationship of instructional goals, 4) expected level of performance for formative evaluation and a guide to decisions for the revisions process (Dick and Carey, 1977), 5) descriptions of not only the current status of the target learners, but, 6) of the educational system by which the needs are to be solved (Alkin and Fitz-Gibbon, 1975). A thoroughly conducted needs assessment, then, provides information intricately related to and forming the foundation for all other instructional development decisions and subsequent activities. Needs Assessment and Education Many educators have wr i tten of educational institutions as systems that are themselves subsystems of a greater system (i.e. suprasystem) called the society (Kaufman, 1972: Hannum and Briggs, 1982: Burton and Merrill, 1977; Kaufman and English, 1979). These same authors have further broken the educational system down into its various subsystems. iDepending upon the level approached, some of those subsystems include components such as curriculum, students, staffing, management, and facilities. 41 It is a major contention of these educators that an educational system cannot function properly, or be understood and dealt with efficiently and effectively, if it is not seen in this relational perspective (Saba, 1980: Kaufman and English, 1979: Hannum and Briggs, 1982). An educational system, whether a large one on the national level or a single classroom, does not function in a vacuum and should not be dealt with or operated as such. It has been argued that because the school is not a self-serving entity but an institution designed to serve its community, that all purposes, goals, and objectives should be derived from the needs of that community (Burton and Merrill, 1977; Kaufman and English, 1979). In his monograph, Saba suggests that the ”policies and procedures of the educational system should be developed with the cooperation of all the government and private agencies that are directly involved in planning and operations of national development efforts." He supports this argument by pointing to the idea that rather than dealing with economic policy as the basic element of national development, "that educational technology should be brought into the mainstream of national planning.” This would provide a “more realistic planning" because “economic objectives without a clearly defined program for training the kinds and levels of skilled manpower required to carry out the various development tasks, have proven insufficient in the past.“ He further says that economic plans, to be 42 effective, "should be integrated with a system for the education and training of the multiple publics who are to be the implementors as well as the benefactors of the plan." The systems approach, viewing the system as a whole, and the individual and interactive functions of the component parts and their effect on the whole, must be taken into consideration if any planning, development, or Operational efforts are to be successful (Gagne, 1977a; 1977b: Jamison, Suppes and Wells, 1974: Kaufman and Stakenas, 1981). For such an educational plan, it would be the function of Alpha and Beta type needs assessments to gather the necessary information, internally from the educational system and externally from the society, that could be used as the basis for making the decisions concerning educational purposes, goals, and objectives and to suggest alternative programs which might be adopted to accomplish these (Alkin and Fitz-Gibbon, 1975). Models, the Systems Approach and Alpha and Beta Needs Assessments Attention will now be turned to a tool and component of the systems approach, the model. Models will be defined here as a conceptualization in the form of a narrative or graphic analog representing a real life situation. First, explanation is offered as to how the modeling of a system is a necessary activity when approaching the solution of a 43 problem through the systems approach. Next, the general uses and functions of models will be discussed. Throughout, mention will be made tying the information presented to Alpha and Beta type needs assessment models. As previously mentioned, the construction of a model, implying a joining together of the component parts of a system and their interrelationships, is a necessary step in what has been termed the “systems approach" (Silvern, 1968: Norlen, 1972). Silvern has suggested that a 'model' conforms to the definition of a system in that it is ”the structure or organization of an orderly whole, clearly showing the interrelationships of the parts to each other and to the whole itself.“ He further suggests that 'Anasynthesis', which consists of the iterative application of analysis of a system, and synthesis (or formulation of a new system) has two characteristics: models are first constructed and then used to "simulate“ the real life situation. According to systems theory (Banathy, 1968), each part in a system depends upon every other part and parts can not be studied in isolation from the whole. Models provide a conceptual framework through which these interactions can be considered and studied. Deutsch (1948-49) suggests that 'men have tended to order their thoughts in terms of pictorial models since the beginning of organized thought.“ Models are a most useful way of ordering experiences so that more intelligent decisions can be made on probable 44 solutions to problems. As was noted, the first step in the systems approach is analysis, or the "breaking down of a whole into its parts, showing the relationship of the parts to each other and to the whole itself" (Silvern, 1968). Synthesis follows analysis by ”combining non-related elements into a meaningful relationship such that the new product is a whole system.” (Silvern, 1968). The resulting system may be an arbitrarily ordered set of concepts that has been constructed in such a way as to, for example, explain or predict some phenomenon or process. The concepts chosen and the particular structure, or relationships between components presented, is from among an almost unlimited set of possibilities. ”The human mind can not manipulate the whole conceptual system in all its complexity. Science proceeds by isolating individual relations into greater wholes. These isolated objects of study are called, here conceptual models, defined as relationships between two or more concepts" (Vickery, 1973). Models are constructs used as tools to picture these components and describe these interrelations aiding the human mind in the perception and conceptualization of the process or phenomenon in question, providing a more ”powerful way of thinking" (Churchman, 1968). 45 Hull, Mapes and Wheeler (1971), have offered these basic uses for models: 1. Description or the rapid understanding of salient features of the system under scrutiny. 2. Prediction of the future behavior of that system. 3. Anal sis by manipulating the model for the best method of achieving particular ends. Since Alpha and Beta type needs' assessment models basically picture the processes through which a potential user might proceed to discover needs, attention will be focused upon those related models of the "analytic" type. Analysis will, necessarily, include the descriptive and predictive functions with the addition of a greater understanding of interrelationships (Hull, Mapes and Wheeler, 1971). Alexander and Yelon (1969) look at a model as a 'Conunon Experiential Referent (CER)." They contend that since people come from different backgrounds, they tend to perceive problems differently. The CER flowchart model provides a stimulus and referent from which instructional developers, in the case, can move toward consensus on the best strategies and procedures to follow in problem solution. They theorize that greater consensus leads to greater commitment to a process. By definition, the application of an Alpha type needs assessment will involve members chosen from among educators, learners, and 46 community. Some of the procedures included in a Beta type assessment may also involve these “partners” (Kaufman, 1979). Understanding and consensus are paramount if the purpose of the needs assessment is to be accomplished. The model chosen for a particular application should be presented in such a format that can be understood with reasonable ease by all participants. It should further provide for the reaching of consensus among participants, where necessary, throughout the process. Gerlach and Ely (1971) see models as guidelines used as a checklist for planning, showing major components of a system, although not in fine detail. Accordingly, Alpha and Beta models should present all pertinent components to help the user organize more carefully in such a way that all components, their place in the process, and complex problems, to .assure proper interactions between all the variables, or components related to that problem is the basic premise of systematic planning (Kaufman, 1968). Silverman recommends the use of models rather than theory, in this case in instructional development, because ”models offer greater flexibility in dealing with" what he refers to as 'fieLd or dynamic forces operating within the environment“ (p.5). He explains that the 'key property of a field is the dynamic one; every part depends upon every other part and parts cannot be studied in isolation from the whole". Although needs assessment models should be specific enough in the components pictured and in 47 accounting for their interactions that they can be used as guidelines to the assessment process, they must be flexible enough to be adopted and applied across a broad spectrum of possible users and situations. Thus, in summary, models in general, and needs assessment models in particular, should have the potential of: l) describing, predicting, and/or analyzing a system, 2) acting as a common referent that aids in understanding and consensus among all participants in a process, 3) being used as a checklist for planning, 4) ordering all variables to assure their proper consideration in a process, and 5) being flexible enough to be generalized to a variety of situations. To accomplish these purposes the needs assessment ‘models must contain all necessary' major components, their place in the assessment process, and the interactions among the components. These must be presented in enough detail to be easily comprehended but not to the extent that they prescribe too rigid a structure rendering them inappropriate to broad applications. The Roots of Alpha and Beta Type Needs Assessments in the Systems Approach and Scientific Methodology This section discusses the possible roots of current needs assessment practices, models and tools. Some of the likely .historical steps in the development of models designed to aid in this process are presented, as well as some significant applications. 48 When appropriately applied to education, needs assessment appears to be an integral part of the systems approach to problem solution. In his monograph, Instructional Systems, Banathy (1968) states that one of the most conspicuous aspects of the systems approach is: ”An insistence upon a clear definition of the system, and upon the formulation of performance expectations stated specifically enough to enable the construction of criterion measures that will reveal evidence of the degree to which expected performance has been attained.” He goes on to explain that without adequately specified objectives, it is difficult to assess not only input capabilities relevant to objectives, but textbook selection and output measures. Students are often unclear as to what is expected of them, are tested on irrelevant material, or material they have not learned, and are involved in learning tasks that they have either already mastered or that they will never be given an opportunity to be assessed on. Banathy (1968) further suggests that to be operated in a systems manner, Education must be viewed as a subsystem of the suprasystem (i.e. society) from which it derives its purpose, and must continually check the needs of that referent system to test the adequacy of its output. This is the main function of the Alpha and Beta type needs assessments (Kaufman, 1972: 1977; Burton and Merrill, 1977: Kaufman and English, 1979). 49 Banathy goes on to explain that the systems approach (and consequently its various functions, such as needs assessment) is ”not a new invention nor is it a miraculous discovery. Rooted in logic, philosophy, communication theory, psychology, and other disciplines, it has received widespread application since World War II in ever increasing sophistication.“ It is not suggested here, either, that this approach is new to education but has been used for years by educators, even though not labeled as such (Tyler, 1934). But the exploration of the area might provide a framework for the synthesis of effective methods helping develop them into a comprehensive methodology of planning and development. Although developed to a certain level of sophistication in other disciplines, such application may not be transferred directly, but must be “transformed“ for educational use (Banathy, 1968). Rooted in the scientific method, educational evaluators seek to rely heavily on scientific approaches and principles (Struening and Guttentage, 1975). This to some degree involves validity and reliability of methods and data used for decision making. And, although scientists would be the "first to admit that reliable and valid information about many important aspects of education can not now be obtained”, it is necessary to provide the most valid and reliable information possible (Duncan, 1980; Kaufman and English, 1979) if we are to engender maximum 50 learning (Banathy, 1968) and proceed toward a science of instruction (Glaser, 1976). The systems approach (Banathy, 1968; Churchman, 1968) dictates that the first step toward obtaining adequate results from a system is to properly define its purpose. And that purpose is derived from the suprasystem which the system was designed to serve. From. that purpose, the processes necessary to carry it out, and the component parts (subsystems) of the system necessary to perform those processes, can be derived (Banathy, 1968). The freer from vagueness and ambiguity and the closer that purpose reflects the needs of the referent system, the more adequate the efforts to write objectives, to design a program to meet them, and to evaluate those efforts (Mager, 1962; Gagne 1965; Banathy, 1968). Alpha and Beta needs assessments are those integral parts of the systems approach that gather the information necessary' from. the referent system, externally and internally, to be used by decision makers (public, learners, educators, admini- strators, legislators) in making educational decisions based on the most valid and reliable data possible, according to the systems approach (Banathy, 1968: Kaufman, 1972; Burton and Merrill, 1977; Kaufman and English, 1979). 51 Some Early Steps in the Development of Alpha and Beta Needs Assessment Models Needs assessment became of interest to educators and hundreds of models, kits and tools became available. This was shortly after the notion of accountability became prevalent in the 1960's. It has been suggested that most current models of needs assessment, to some extent, are based upon the "classical" method of assessing educational needs (Kaufman, 1972: Southard, 1974) and the pioneering work of Kaufman who placed the discrepancy model in the context of systematic educational planning (Witkin, 1977). Although ordered differently or having additional ones, the four steps included in most needs assessment models are: 1) the generation of goals and their ranking for importance (i.e. exiting condition). 2) the determination of the present status of the goals (i.e. existing condition). 3) the identification and analysis of discrepancies between goals and existing status. 4) the. assignment of priorities to the discrep- anc1es. Most extant models are, to an extent, patterned after the early model developed by Kaufman, Corrigan, and Johnson (1969) (Trimby, 1979). Their's was a utility model for determining the overall goals of education. To them, the overall goal of education was that the learner would be able to survive in and contribute to society upon exiting 52 from the educational system. The economic indicator for measuring success was that consumption should equal production. They had a concern, too for the ”humanistic" requirements necessary to that survival and contribution. These were derived from Maslow (1968), Frankl (1962, 1967, 1969), Rogers (1964), and Ruckers (1969). The model was based, partly, upon the Hanna formulation (1966) that states the three equally important foci of curriculum: 1) the nature of knowledge, 2) the nature of the learner, and 3) the nature of the society. Hanna suggested that the logical entry point, if there is to be one, would be through the dimension "nature of society". Another part of needs assessment's early development, according to Harsh (1968), is the CIPP (Content, Input, Process, and Products) model of Stufflebeam (1968). Although not concerned directly, his view that planning and evaluation are inextricably related, is certainly relevant to needs assessment. The CIPP model emphasizes the requirement to determine context and inputs to a program before beginning. Also stressed is the necessity of including those elements in any evaluation of the utility of the programs. Needs assessments can and have been designed to provide the information necessary to fulfill these purposes (Kaufman, 1972) Two other early models worthy of note are those developed by Sweigert (1969) and Rucker (1969). The Sweigert model presents a process for determining a 53 possible starting point for educational design including the need for the inclusion of “partners“ (community, educators, learners) in determining needs. The model provides a powerful tool for identifying areas of concern, the possible array of goals and the importance attached to each. One helpful extension would have been the collection of data after the "concerns analysis" to determine if the concerns are 'real", (i.e. a discrepancy between current and required performance) (Kaufman, 1972). A definite concern for the individual potential and uniqueness that may have to be developed for a learner to proceed beyond the minimal requirements of contribution to and survival in society, was addressed by Rucker. His model, with contributions from Maslow (1968), Axtelle (1966), Rogers (1964), and Laswell (1948), offered a ”value oriented framework” for education and the behavioral science. Eight value categories were offered, representing concerns from societal to those considered the deepest personal. Their purpose was to assist in naming and defining areas of concern and of required accomplishment for an educational needs assessment. Of additional utility, to the development of current needs assessment procedures and models, are the ”determination of current self-concept and desired self-concept and/or purpose in life (Crumbaugh and Maholick, 1969) and possible assessment procedures based, in part, upon Frankl's concepts (Frankl, 1962, 1965, 1967, 1969: Kaufman, 1972). 54 Two early applications of needs assessments that serve as examples of the inductive and deductive methods that might be used to assess the needs of a system, are those pioneered by the Newport-Mesa Unified School District (1968) and by the innovative Temple City Unified School District (1968), both in California. The inductive approach derives its name from the fact that the goals, objectives, and outcomes of the educational process is first obtained from the members of the subcommunities in the district. The program is based upon these data. The first step is to see how the learners in the system are presently behaving. The district credited with pioneering this approach, the Newport-Mesa Unified School District in California (Shuck, 1968), used Flanagan's critical incident technique (1954) to indicate whether the schools were doing a satisfactory or unsatisfactory job. Next, the critical incidents were compiled into program areas and behavior expectancies. The next steps were to compare these subcommunity expectancies to broad, district-wide goals of education, to reconcile discrepancies, and then to set detailed behaviors for bringing about desired behaviors. The program would be developed from and evaluated against these behaviors. The deductive approach proceeds to deduce an educational program from existing goals and outcome statements. The starting point is the identification of existing educational goals. In the innovative Temple City 55 Unified School District, (Kaufman, Rand, English, Conte, Hawkins, 1968), educators selected the goal formulations done by the Pennsylvania State Department of Instruction and Educational Testing Service (1965). From these ten goals, criterion measures (indicators) representative of behaviors were first developed. The goals would be considered ”indicated” when the behavior was observed. The next step was to obtain ”change requirements” from the partners in the system (i.e. learners, educators, and community members) then to collect data concerning whether the indicators were presently being realized. Based upon discrepancies, detailed objectives were set, and an appropriate program developed. Related Studies With more limited scope and different purpose or emphasis, there are four extant studies presented in Educational literature that deal with some of the assessment models that are the concern of this study. Following is a brief description of the four works. In her article "Needs Assessment Models: a Comparison" (1979), Madeline Trimby offered narrative descriptions and graphic representation of four models that would likely be classified Alpha or Beta. Her purpose was to compare and contrast the models with one another by: 1) pointing out similarities and differences, 2) suggesting the conditions 56 under which, or the tasks to which, each might be applied, and 3) describing their relationship to evaluation. In a project undertaken for Project Next Step: Mutuality of Planning (1974), conducted for the purpose of providing information useful to the user in the 'how-to's" of conducting needs assessments and for providing further guidelines in the selection of instruments, Kaplan summarized four models. These models were selected because of their 1) widespread participation, 2) comprehensiveness, 3) field testing, 4) replicability, and 5) reasonable cost. In Educational Needs Assessment: Theme and Variation, a work sponsored by the National Institute of Education (DHEW) in 1976, abstracts describing twenty-five needs assessment projects appropriate for a variety of educational levels are presented. These abstracts provide an overview of each project as well as information on the developer, source, cost, copyright, procedures for use, implementation needs, and special features. The Rhode Island State Department of Education, Bureau of Technical Assistance offered a Needs Assessment Compendium of Abstracts (1976). These abstracts provide a means for selecting or developing a needs assessment instrument. Included are descriptive sumaries of twenty- three approaches to needs assessment including three kits, nine models, and eleven questionnaires. Although providing some useful information, the above studies, because of purpose, emphasis or scope, do not 57 fulfill the purpose of this present study. First, none of the above studies goes into an indepth description of the characteristics of both the Alpha and Beta needs assessment models. Although the studies offer information on many of the extant models, they do not alone or in combination offer as extensive an inclusion as in this work. Because of their briefness, it is thought that the present study will do more than any of the above studies to enlighten users as to the benefits of such assessments. It is further planned that this work will build upon those previously mentioned in offering guidelines by which the potential user might compare extant models, judge their individual merit, and choose from among them for the application in question. Finally, a synthesis is made of the elements found in major models, this to provide a more appropriate and generalizable needs assessment model. Summary The purpose of this chapter has been four fold. The first was to provide further understanding of Alpha and Beta type needs assessments by looking at how they have been related, in the literature, to the systems approach, evaluation, instructional development, and Education as a whole. Second, a look was taken at "models” in general, and particular attention was focused upon the development of Alpha and Beta type models. Third, an historical 58 tracing of the development of these, models was offered, and, fourth, a review is presented of similar or related studies. The actual. presentation, description, and evaluation of extant needs assessment models of the Alpha and Beta varieties are dealt with in Chapter IV of this work. CHAPTER I I I Research Design Overview This chapter discusses: l) the questions to which this study is addressed, 2) the scope and limitations of the study, 3) the general methods and procedures used in gathering, analyzing, and reporting the data necessary to the completion of its purposes, 4) the collection, analysis, and reporting of data for the individual research questions, and S) the proposed value of the answers derived. Research Questions The following eight questions guided this study: 1. Who were the significant individuals and what were the significant events and/or landmark applications that figure into the evolution of Alpha and Beta type needs assessment models? 2. What are the names and descriptions of established Alpha and Beta type needs assessment models? 3. What types of information could ideally proceed from the application of Alpha and Beta type needs assessments? 4. Are there major variations among Alpha type needs assessment models? 59 60 5. Are there major variations among Beta type needs assessment models? 6. What characteristics could be included in idealized .Alpha and Beta type needs assessment models? 7. How do the characteristics of extant models compare to characteristics considered ideal for such models? 8. What research evidence exists as to the ability of these models to produce the information they were designed to acquire? Scope and Limitations of this Study This study will be primarily concerned with the collection, analysis, and synthesis of data dealing with 1) published needs assessment models that are of the Alpha and Beta varieties (Kaufman, 1977), and 2) have been specifically developed for educational settings. Because of the extensive proliferation, over the last decade, of various models and tools for needs assessment (Witkin, 1977), it was necessary to limit the scope of this work to those models referred to as "Alpha" and "Beta” needs assessment models (Kaufman, 1977). It is assumed that this effort could help tie together a largely undeveloped area where, currently, such efforts are piecemeal (Witkin, 1975). Mention of other types of models and tools will be made only as they enhance understanding of the development, application, and definition of Alpha and Beta type models. Among the untrained who are often re3ponsib1e for 61 carrying out the needs assessment process in educational settings (English, 1977: Sarthory, 1977), there exists confusion as to the availability of appropriate models and tools (Melton, 1977) and as to the criteria for choosing among these, once they have been located (Kaufman, 1977). This study is designed to: 1. explain the benefits and information that could result from the application of the processes presented in the Alpha and Beta type models. 2. describe extant needs assessment models of the Alpha and Beta varieties. 3. provide helpful guidelines for potential users' choice from among the extant models for a particular application. Data Collection Procedures Because this work deals with published, educational models of the Alpha and Beta type, the first likely place to look for information is in the professional literature of education. But, since a need has been expressed as to the development of the area of needs assessment into a cohesive whole (Witkin, 1977), and confusion exists in such basic areas as central terms and definitions (Sarthory, 1977: Trimby, 1979), and even as to what is and is not a needs assessment (Kaufman, 1977), it was found that all necessary data was not locatable within the Educational literature. In order to gain the necessary insights and information to answer the research questions guiding the 62 study, it was necessary to make direct contact with: l) a number of the developers of the models, 2) those who have been served by or who have conducted needs assessment, 3) experts in related areas, and 4) literature in related areas. The first step in the location of pertinent data was an extensive search of related literature. It was assumed, and correctly, that if all desired information was not found there, that reference to other possible sources would be. Direct contact with sources outside the literature was accomplished through telephone interviews and mailed questionnaires (Appendix B). Since the amount and quality of information varied among the models, it was necessary to design instruments pertinent to each individual model. These were, for the» most part based upon the questions presented in the matrix (Table 3) in Chapter IV. Data Analysis Procedures Criteria for the analysis of the data collected from the literature and other sources included three types: 1. criteria for selection of the models to be included in the study. 2. criteria for selecting and developing the characteristics to be included in 'ideal" Alpha and Beta type models against which extant models were compared. 63 3. criteria for selecting those concerns (e.g., cost, ease» of 'management, time involved) about which potential users may want to be aware when choosing a model apprOpriate for a particular application. The above was dealt with as follows: 1. Models included in the study were a. published models b. of the Alpha and Beta types c. developed for the purpose of conducting needs assessments 2. Criterion types presented in two and three above were developed as a result of the discussions in the literature and through direct contact, with model developer/users and related experts. Procedures for Reporting Data The collection and analysis of data revolved around these three basic considerations: 1. the characteristics of "ideal” Alpha and Beta type needs assessment models, 2. the description of extant models of these varieties, and 3. a comparison of extant models to those characteristics deemed “ideal” in this study. To be comprehensive and as convenient to the reader as possible, the following methods are used to report that data. In Chapter IV, in the section entitled ”The Modeling of the 'Ideal' Alpha and Beta Needs Assessment Processes“, 64 explanation is made first as to the concerns with modeling itself that were taken into account when designing those "ideal' processes. Also offered are both a narrative and graphic description of the same. That same chapter contains a matrix (Table 3) summarizing information gathered to describe extant needs assessment models. The questions used to ferret out information useful to the models' understanding are also discussed. The matrix further presents information about some of the concerns that potential users may have. The former may not lend to the evaluation of the models as "needs assessment models" but does add information on a practical plane, answering such questions as 'Are materials available for actual application?” Such a matrix makes possible not only the easy location of information on a particular concern, but allows the reader to compare across models regarding that same concern. Some of the information about the models was best presented in narrative and graphic form. For this purpose, Appendix A was added containing a narrative description of each model with graphic portrayals, where available. This narrative offers information not readily presented in matrix form plus evaluation of each model, listing some possible assets and deficiencies. Such a presentation makes an overview of the models more readily available than was possible in matrix form. 65 The criteria by which the models are judged is presented in the “modeling" section. These 'ideal' characteristics were taken into account when the model descriptions were written. In Chapter V, when conclusions are drawn and summary or findings are made, and research implications presented, an overview of the current state in these models' development is included that makes specific reference to extant models, where helpful. The CollectionL Analysis, and Reporting of Data as Applied to the Individual Research Questions, and the Projected Value of the:Derived Answers Following are the eight research questions previously posed and the proposed value of the answer for each: 1. Who were the significant individuals and what were the significant events and/or landmark applications that figure into the evolution of Alpha and Beta type needs assessment models? Answers to this question could aid in a fuller understanding of the elements in Alpha and Beta needs assessment models, by giving a brief overview of their early development, reasons for inception and those who were involved. Because information on the topic was scant in the literature, it was necessary to contact some of those individuals who have been closely associated with the area 66 over a period of years to gain some historical perspective. The information gathered is presented in Chapter II where these processes are related to the Systems Approach and scientific ‘methodology and presentation is made of some early steps in the development of the models. In addition, a matrix in Chapter IV will provide a temporal sequence of events, applications, and relevant individuals most responsible for the development of Alpha and Beta needs assessment models and processes. 2. What are the names and descriptions of established Alpha and Beta type needs assessment models? The answer to this question provides the major content of the study and helps fulfill its main goals and purposes. Because the reports in extant literature do not fully present all of the information needed to describe the established models of the Alpha and Beta types as related to "ideal“ characteristics, it was necessary to make direct contact with those reported in the literature as developers of the models and, in some cases, individuals involved in projects where the various needs assessment models reportedly were used. The models are individually described both graphically and in narrative form in Appendix A, and many of the concerns are presented in matrix form in Chapter IV (Table 3). The content for the descriptions was derived from such 67 sources as journal articles and user's manuals. New insight and information were added resulting from the data collected on the current state of Alpha and Beta models in general and through direct contact with pertinent individuals and organizations. 3. What types of information could ideally proceed from the application of Alpha and Beta type needs assessments? The descriptions and evaluations of the models should help prospective users more aptly choose one for their own purposes. Coupling this information with the answers to research question number three would help the user plan for better results because a) the possibilities of application could be kept in mind along with b) the capabilities of the model(s) in question. This should aid in choosing and/or adapting a model for use in a specific situation. I Information to answer this question was collected from the literature, model developers/users, and related experts. From each of the sources, data was found and options and concerns were expressed as to what information should proceed from an Alpha or Beta type needs assessment and the most desirable and successful means to achieving that end. Possible outcomes were also deduced from the design and descriptions of the models themselves, with the assumption made that they were systematically developed 68 with a particular purpose in mind. The answers to this question are presented throughout the opening chapter under such headings as 'A Possible Taxonomy of Needs Assessment Models" and "Definition of Terms“, and. in Chapter II in the discussion about the models' historical development. Deductions can be made also from the data presented in the matrix in Chapter IV. The Appendix A description of each model further answers this question. 4. Are there major variations among Alpha type needs assessment models? 5. Are there major variations among Beta type needs assessment models? The answers to both of these questions, through written comparisons and contracts, should result in insights that demonstrate not only the potential of the individual models, but present a fuller understanding of the current state of both Alpha and Beta models in general. The comparisons among models are made in several ways. First, a criterion was set for "ideal" models of these two types. Next, the models were individually described with these 'ideal" characteristics in mind. The models can easily be compared one to another by making a horizontal scan of the matrix in Chapter IV (Table 3). A general discussion of the models as they compare and contrast to an “idealized" model is also dealt with in Chapter V. 69 6. What characteristics could be included in “idealized“ Alpha and Beta needs assessment models? Such information should serve the purpose of shedding light on the current state of development of the individual models, and of both categories of models in general, by providing a framework against which the models can be compared. The characteristics of ”ideal“ Alpha and Beta models were developed from an overall look at extant models, discussions and questions concerning needs assessment by developers of models, and others who have been involved in the process. Opinions and insights offered by related educational experts were also considered. This information was gathered through the literature and first hand contact with other sources. An "ideal" model, of either type, is a generic one containing as many as possible of all those characteristics considered necessary and appropriate to the categories (Alpha and Beta). Characteristics not found in some form in extant models are not included. The components of these models are compiled, refined and synthesized into a “Unified Model” and presented in both schematic (Figure 6) and narrative form in Chapter IV; The inclusion of components currently found in Alpha and Beta type needs assessment models was decided upon by listing them and ordering and prioritizing them according to the opinions 70 and findings of developers, users, and other related experts. 7. How do the characteristics of extant models compare to characteristics considered 'ideal' for such models? After ferreting out those characteristics considered appropriate, or "ideal“, to both Alpha and Beta type models, the next step was to compare extant models to these characteristics. This process was performed not necessarily with the hope of finding fault with existing models, but to present a more comprehensive picture of any model's individual characteristics and likely applications. A model may contain the necessary characteristics in order to be qualified as an Alpha or Beta, but may emphasize only certain of those characteristics and only mention others. This emphasis could produce different effects in the scope and type of information provided in a needs assessment. The conclusions reached from these. comparisons ‘were presented in written form in the models' individual descriptions and evaluations in Appendix A and in matrix form in Chapter IV with characteristics recorded on one axis and the models' names on the other, and with each model individually related to the characteristics. 71 8. What research evidence exists as to the ability of these models to produce the information for which they were designed? The answer to this question could help take the potential of the models out of the theoretical realm and give them a greater degree of validity. It could also demonstrate how models, produced under specific circumstances, help users generalize to their own. In. addition, further gaps in research on needs assessment would be identified for future research. This information was presented in the written descriptions of individual models and dealt with again in a matrix in Chapter IV, and in the summary and conclusions in Chapter V. Summary In order to answer the questions listed in this chapter, it was necessary to undertake an extensive search of educational literature and make first hand contact with pertinent individuals to gain information not found in the former. The study is limited to published, educational needs assessment models of the Alpha and Beta varieties with reference made to others, where helpful. Information was analyzed according to criteria developed from data collected from the literature, educational and other experts, and needs assessment model developers/users. 72 Reporting of the research consists mainly of written and graphic description and analysis of selected models according to developed criteria. Chapter IV We Overview In Chapter III eight research questions were posed. It is the purpose of this chapter to analyze the information that has been gathered in an attempt to answer those research questions. This analysis is presented in the form of narrative explanations, a matrix and through graphic representation, where appropriate. The bulk of the chapter is concerned with an explanation as to how the information derived from a questionnaire and related literature can be used to answer the research questions. A matrix is used to summarize the results of the questionnaire. The information from that matrix proved very useful in answering several of the eight questions. Others were answered through telephone interviews with experts on needs assessment, and through the study of manuals and other materials that accompany many of the models in this study. 73 74 The research questions are answered in order one through eight. Several of the research questions are answered in the next immediate sections where the information gathered from the questionnaire is presented in the form of a matrix (Table 3). Research questions one and three ( 1) Who were the significant individuals and what were the significant events and/or landmark applications that figure into the evaluation of Alpha and Beta type needs assessment models? 3) What types of information could ideally proceed from the application of the Alpha and Beta needs assessment models?) have been dealt with extensively in the first three chapters of this study. They too are answered in the form of reviews and summaries of material previously presented. The Matrix and Questionnaireguestions The twenty-three questions listed below (hereafter, referred to as "matrix questions” to distinguish them from the eight research questions) were used to guide the gathering of the necessary information to more fully answer research questions 2, 4, 5, 7, and 8. In most cases, the twenty-three questions posed in the matrix were answerable through such literature as user's manuals which accompany many of the models. Individual questionnaires were tailored for each of the model 7S developer/marketers. After reviewing the manuals and other accompanying or explanatory materials through use of the twenty-three questions, questionnaires were designed that asked only those questions not dealt with in the literature. In the instances where answers to the matrix questions were not found in the literature, appropriate developer/marketers were contacted in a telephone interview or individualized questionnaire (Appendix B). In some instances, developers indicated no information in relation to a specific matrix questions. However, the user should be aware that the issue could still arise during actual application of a particular model. Most of the questions and their intent are pretty much self-explanatory, but explanation is offered where it was thought helpful. The intent of these questions was to gather the information necessary to answer the following research questions: 2) What are the names and descr iptions of established Alpha and Beta type needs assessment models? 4) Are there major variations among Alpha type needs assessment models? 5) Are there major variations among Beta type needs assessment models? 7) How do the characteristics of extant models compare to characteristics considered ideal for such models? 76 8) What research evidence exists as to the ability of these models to produce the information they were designed to acquire? After the presentation of the matrix, analysis of its content will be offered as it relates to these five research questions. The Matrix Questions The twenty-three questions listed below went through several revisions as a result of feedback from professors at Michigan State University. The suggested value of some questions are discussed after specific sets of them in this section. 1) Are the services and/or materials connected with the model still available? If yes, to whom? 2) What is the major purpose of the model? 3) At what educational level is the assessment aimed? 4) What pre-assessment planning stages, or considerations are addressed? 5) What are the final outputs of the assessment? 6) Who are the sources of information used to determine needs? 7) How are the information sources chosen? 8) In what aspects of the assessment do the various categories of participants (e.g. learner, parents, community) engage? 77 The answer to matrix questions 6, 7, and 8 can help the users of specific models to evaluate the utility and validity of the information produced. Utility and validity may depend upon the source of data, and whether the sources were representative of the population that the educational system serves. They may further depend upon the extent of involvement of the various participants. Is the greatest potential of non- educators taken into consideration? If the community, for example, is used merely as a source of data for need determination, but are not included in such aspects of the needs assessment as prioritizing needs or deciding which of the needs warrant inclusion in a program, again the utility and validity of decisions may come into question. 9) What methods are suggested for deciding which "perceived needs“ are in fact needs? 10) What staff or committees are required to carry out the assessment? 11) Are the staff/committee responsibilities outlined in materials available to users of the needs assessment? 12) What costs are involved in the assessment? 13) What formal training is required to conduct the assessment? 14) Are outside consultants necessary to the assessment? If yes, are they available? 15) What are the instruments of the assessment? 16) Do you provide the necessary instruments? 78 17) How long does it take to complete a typical needs assessment with your model? 18) Are needs prioritized? 19) Are needs validated through empirical means? If yes, which ones? The validity and utility of the outcome of the Alpha and Beta assessments are dependent upon the extent to which the resultant goals are documented by objective data. If needs are based merely. upon ;participant's perceptions and feelings there is little reason to believe that the resultant program changes will be more useful to learners than the previous one (Kaufman and English, 1979). 20) What interrelationships between the needs assessment and other stages of the educational process are specified? Since a needs assessment of the Alpha or Beta type is designed to assist in the design and development of the entire educational undertaking, the interrelationship between the assessment's resultant data and the ensuing program are important when choosing a model that relates to users' purposes. A model's design may have been developed with specific emphases in mind, for instance budgeting or problem detection, that renders the model in question more capable for those purposes than another. 79 21) To what extent has the model been field tested? 22) Has research been conducted concerning the model? 23) Approximately what percentage of the model's use has taken place in industry? in education? Some models may have been found useful in industrial settings. Others may have been designed chiefly for such purposes, such as those of Coffing and Harless. This inclusion in the matrix indicates which of those models included have been applied in both settings. The Models Below is a list of the models presented in the matrix. The models are grouped as they appear on the matrix allowing the reader easier access to those of interest. A narrative presentation of each model, including a contact source for further information is presented in Appendix A. Taxonomical Model Classification Page The Atlanta Assessment Alpha 86 Project The Battelle Needs Beta 86 Assessment Survey Dallas's Model for Beta 86 Shared Decision Making 80 Mott Foundation Community College Model Illinois Problem Index Ohio Department of Education Needs Assessment Guidelines Quality Education Program A Comprehensive Needs Assessment Module Institutional Goals Inventory Phi Delta Kappan Model Sensing Educational Needs in the Far west Region WOrldwide Planning Model Coffing's Client Needs Assessment Lee's Needs Assessment Model Consolidated Application's Needs Assessment Guide Fresno Planning Model Vocational Needs Assessment Project Kaufman's Needs Assessment Model Harless's Front-end Analysis Model Skyline West Educational Plan Beta Beta Beta Beta Beta Beta Beta Other Other Other Beta Beta Alpha Alpha Alpha Beta Other 94 94 94 102 102 102 110 110 110 118 118 118 126 126 126 Not listed Not listed 81 The following models appear in the educational literature but are either currently unavailable or not locatable. This information is offered to save the reader a search for them. No further information is given on these unless characteristics are found that are unique or enhance understanding of Alpha and/or Beta models in some way. South Carolina Needs Assessment Model Westinghouse Model New Mexico Needs Assessment Model Allyn and Bacon Lewin and Associates The Matrix The twenty-three matrix questions are listed on the vertical axis of the matrix in Table 3 and the needs assessment models are listed on the horizontal axis. Information derived from the literature, from contact with model developers, and from such sources as user's manuals was used to answer the twenty-three matrix questions. The matrix is set up so that a clearer picture is given not only of a model as it relates to a particular question, but how models compare one to another. A number of models are presented on each page so that a horizontal scan across the column will demonstrate how the models compare as they relate to the concern of the 82 question asked. Some of the questions pertain to the availability of tools and other resources that may be of help to potential users. The presence or absence of entries in a particular box may or may not reflect upon the quality of the model. While a well constructed model may have been develOped as a guide for the assessment process, the developers may not necessarily have been interested in producing or marketing the tools or other resources of an assessment. The information produced by answering the twenty-three questions may also help potential users choose from among the models, for their particular applications. If a box remains blank, it may be because it is inapplicable to a particular model for a number of reasons. A question may apply to Alpha type models and the model may be a Beta or "other”. As previously mentioned, some questions pertain to materials needed to conduct an assessment and some models are not accompanied by those materials. In a few cases, information was not available concerning a particular question. Information was specified neither in a user's manual nor answered by the developer when the missing information was requested through the survey. Sensing Educational Needs in the Far West Region is not included since it ‘was only used to validate one technique related to the needs assessment process. 83 Harless's model is not included because its very general nature also rendered its categorization in most of the matrix's entries, extremely difficult. The table (Table 2) on the next page is an abstract of the very lengthy matrix which immediately follows the table. The matrix contains so much information that the researcher thought it would be helpful if it was preceded by an abstract that summarizes its content. 84 extent oi parti- cipant involve- sent? us) oi iniorrration. Kaufran suggests partners participate in all aspects. Qatatron Alpha lets I) Are services] All are available to the 0in Let's and the as,“ "as are nodels avarla- general public escept W currently unavailable. All others are? vhich is ior inhouse use only. All others are lot the general (u) prblic except m ahich la ior inhouse use. 2) Ihat is the All the categories listed Mast are lot deterrnr'ninatlon oi rrodel's naior were represented across the gap! RN00 NORM outputs and purpose? rrodels. desired goals. Coiling's is ter ((2) decision NIH“ and as for ‘goal determination'. 3) At shat educa- Iron elmntary through con. Cenerally elorentary, middle and tional level is tinuing education. ‘Ihe Atlanta secondary. 1hr Mm node), Battelle the «serum Abdel specliied only secondary Survey and If.) a'an at higher ainerf.’ Nautran's nodal covered all education. The Collin and l-hrless ('3) areas. nodels at industry. a) that preassess- Except tor the Kauinan and VDIP 0in the Collin and I" nodels nent stages are nodels, little consideration is heavily orphasiae this ”ea. addressed! given this area. i“) 5) Ihat are the All produced at least a list at Famed iron a list oi prioritised iinal outputs? prioritised needs. needs to all the concerns teprescn. (0:) covered all the areas. ‘ ted in the Wendel. d) the is involved? All numbers oi the comity All 'partners' in each step except (dd) represented in each step. Coiiing's rrodcl shere this is determined by the user. 7) the are sh: Each according to comity Randonly or comin dosagraphic participants derographic characteristics characteristics in all except chosen? except the fresno nodel whids Coiling's and "I IO ahere the ('7) suggests a connrnity invitation. user decides andQ‘OG share there is peer selection list speciiied in m 8) that is the All use participants as sources The I”. m as, Wand let's nodels call ior partners to «cage in a variety oi activities. Table 2 An abstract oi the matrix presenting the twenty-one rrodcls and the twenty-three questiomaire questions. 85 (Nestton Alpha beta 9) Are percteved needs validated? (0’) Fresno uses adninistration's arbritrary choice. The others all use consensus. Quinn's and'oridvide call ior objective data. orphasis on objective validation in m FIX. and ONO. i0) Qrestions con- cerning process's inple- tion. id's i0. ii. i2. i3. i0. i3. id. Each require iron one coordinator to a conra'ttee. The costs range lronneterials reproduction to paid consultants. All oiier orientation/training progrote but W. lone require outside costs range iron copying noteriais to paid consultants. Corpletlon oi proass ranges iron six mths Mott lbdei) to one year and three nonths (DIEM. Tools are noinly surveys except Fix oiiers a i7) consultants. All provide surveys prioritisation gore. Wand but Fresno (ehich uses 'spealt 0N6 have oeteriais in several ups'l and button. Corpletion languages. the ranges iron less than six nonths to over too years. il) Are needs All prioritize needs. All prioritise needs except prioritized? Collin} (dependent upon user's (dill choice . i2) Are needs or» All validate needs through m. Mud WC rely mt piricaiy valir’iated? otpirical neans except Fresno. heavily won objective validatitlt. (ll! i3) Are inter re- Ali recotnend extensive Fron setting oi goals and obiec- lationships interrelationships. lives in $5 to all considerations between needs in rover. hbt applicable in m assesorent and m. Leis up to decision and instruc- ulcer in Coiling's nodei. tional developtent speciiied? (020) to) Field All have been widely iieid Banged iron one iieid testin ior tested? tested. sum to throughout the 0.3. ior “ill to: i3) Researdi? oily Atlanta and W report Lee. lbtt. ION and Ex indicate ' ('22) cornected research. related research. id) Percentage oi use in industry] its driei application being in Education. industry. ((23) iii! have been used in industry settins. Emotional application is the ooin use oi all except Coiling's oodel. Table 2 (cont) 058:8 00:24:53... 0. 0.0.3. .03... 00 0.0.002 30500000... 0.0002 50.5.... .0 :833800 0. 86 d 030... .0050 I .0050 I .0050 I «£50... 1.30:3. I u£50... 3.0.0000. I m:.:.0.. 3303:. I so... I :0..- I :0... I 3.0.0.0.... \- h. .05... N.- h. .23... I 000000 0.30000 0.30000 \1 0.0...E\.u 0.025% 0.02:. I .00....0 305000000 0... 0. b.5560. \fl C5:0Eu.0\.r E55203 I .25. 300.3300 .2... .< .n v.0...0 I 0.050 I 0.0....0 I. 05202. I 9.2.0.5..Nq 05:00.... I «:30... :0.0.00.. I 3.30:. 5.0.00.0 I 330:. :0.0.00..\q. 5.00300 6030.: I 5.80.0.0 :.030...\.i :0..00.0.. E030...\Ju 5.5.5600 0:- 0.0.5000 0.0500. 53:06:80 0:0 I 0.0.5000 0.0500. I 5.:0EE00 6:0 I. 0.0.5200 50:000. I 2000:. 0... .0 0009:: 0.5.0.. 0>.0>:.\1 500...... 00.00:. I 030...... 00.0.5. I 00.06 0... 0. .055 .N ".033 I .0050 I v.0...0 I 0.3:.— .0.0:0u\_.. 0.3:: .80:0u\.. 0.3:: 300:8. \1 2:0 00: 00:03:. I 3:0 00: 00:03:. I 2:0 00: 00:03:. I «.50.... 0. 00» = 332.000 ....0 .000... 0... 0: I 0: I 0: I 5.3 00.00::00 0.0.00.2: 00>\... 00* Va 00>\.. 330:0 0003.00 0... 00¢ .— uoio: hot-om «ouaoouoo< 000.0...— .:u.=00000< ".5902 8.0.909 YES—m 0—0002 3.0on 0.: 3:090. 2:. 00. .03: .0519 87 Taco... . m 01.0... 000000.: 0. .200: 0.:0EE00 x300: 0:0 00...00.0: .0 .0.. .0050 I .0050 I “003.0 I 00:00.0 00. 00.00. 00:00.0 00. 00.00.00 00:00.0 00. 00.00.00 3:03 0000: .0 .0..\... .0:—.5 0000: .0 .0._ N. .09.... 0000: .0 .0.. N4 0.00.. 000....0.0: .0 .0.. I 0.00.. 000...00.0: .0 .0.. I 0.00.. 000...00.0: .0 .0.. I 2005000000 0... .0 0000: 000.3000: .0 .0.. I 0000: 000....00.0: .0 .0..\.. 0000: 000...00.0: .0 .0.. \v 05:50 .0:... 0... 000 .003 .m 00...00:0 .0: .0050 .......§.\« 50000.0 0.00 0.000. I 030..0>0 0000:0000 b..:00. I 00000: 00000000. 3.300. I 0.8.0:. 00.0.000 80.000030 0.000. I 0.000:. .0 300.00 I 00.0..0 0:0.»00: I .00..E.0 .0 30.000 I 0.20:0 .0:0..00:00 0:0 0.3:: 0. 000000: .0 0000:0003. I 00...00:0 .0: 003.0 \1 .30.... 00000.0 0.00 0.000. I 030..0>0 0000:0000 33:00. I 00000: 0000:0000 3.300. I 00040:. 00.0.000 0.0.0.8030 0.000. I 0.000... .0 30.50 I 0.00:0 0:00.00: I .00...:.0 .0 300.00 I 0.0.0.50 .0:0..00:00 000 0.3:: 0. 000000: .0 0000:0003. I 00....00:0 .0: .0050 N. 033.000 3000.0 0.00 0.000. I 030..0>0 0000:0000 3.300. I 00000: 0000:0000 3.300. I 008.0... 00.0.000 0.0.2.009... 0.000. I 0.000:. .0 30300 I 0.00:0 0:0.»00: I .00..:..0 .0 30.000 I 0.0.9.0 .0::..00:00 0:0 0.3:: 0. 000000: .0 00230055 I .000000000 000 0:0..0000.0:00 00 0000.0 9.300.: 30600000000: .0:...» .0 0.2.: =0.0.00A_ 00.00.—m 00. .000: .0:—on a0o0om .8500000< .082 2.8.... 2.... 000.00.. 000500000< 0.0004. 00... "Qua—OE 88 .2300. .m 0.000 . "00...: I .00...0 I 000.0000 .0: .00...0\l.:.. 03.302. I 00 : u:..0..0:.. .00:00.0 3.. 0 .. 00:00.0 I .0330 .....:::.300 258- On I 8.— AanMv @900: «90—8 . a {OEGU— 0: .00.00 I 00. .000». 0000: .00. 00.. 00....0.00\.. 0300 .0. .8500» .00 008.. 000.00.... I 0.. .... .0. I 0.30.03.00.00 008.. 00.3.00” 008.. .o 5000...: I 0.50.. .c 820...»: .0 802.28 26...» 0000: .0 :0..0".0.0: 0000: :00: u:.0.000 “”003 I 0... 00 303000000 d :85 Man—Vague 6m Vmfl I . . . . 0.: .0 0.00600 «2’ C— :00: u:.0.000 :. 0.0 . I :0..0:.00.:. .0 0000:00N... :0..0300.:. .0 0000:0\\...1 :0..0:.00.:. .0 000300 00000320: 0.3.0.0.... E... .. as...» ”060—0 .55.. UN. swammhvanvflhflp—U 80m” wage—v dunnfihv 020050300 .. 0:30.00 0... 3.000000 I ..:000..0 000500 0. . 0 0 0:. 000 I .. . I 0300 0. 0... 000 0.6: . 3.53300 0. 00.000000m 3.0:330 . 330......0.:00\.. 33003.0 00.. . 0.30.2000 . I .0003 I 22.8 I 0.2.... 020.. 00.0.. 00 :. 0.00un I 30:..000 :. 0.000..“ I I 30:..0 . 000:0 £00.00: 0. 0.00..“ 3.33.0.8 100:0qu 3.000.300.5303“... 33:330.. .000: N: . 000:000.\.. :00 00 I .0000: 05300.00 000:00.“\.u 0.:0000\.u 00.0.“ 00:00 W... 0. 000: :0..0:.00.:. . 0300 N. 000.0300» 0. 00.0.:.:.00\.. .0 000.500 0... 000 2.3 0 000.00:00\.u :0..00.0.:.300\... : .. . . :0..E.0.:.:.00\.. . 303000004. June.— gm 30.000000< 090.004.0q.< 2.... 620.00.. 00025 00. .0002 .002 89 7.2.00. . m wanna. 20.33 30330000 300.. 0... 3 0.0.... 0. 032.02. 03.333 3. .3 I 3\.. 0.. I 33.33 002......2390. 00>\4 00» I 00>\.. 33.338550 0... 0.< .2 3.3.33.3..- 3- 80:08. 3 003. .35 ".050 3.3.338 0... 3 0.3303 033300030. .0 3.3.38 003338 a I 3.3338 0.3 .3... 033 I 3.33.8 0.3 I. 2003...? 2 2050 3.3338 0... 3 9.3303 02.830030. 3 3.3.38 003338 a I 3.3338 0.3 3.... 033 I 3.3338 03\. 20:3 I 3.3.338 0.: 3 9.3303 3.3300030. 3 3.6.38 002.338 0 I 3.3338 0.3 ca... 033 I. 3.3338 0.3\fl .3333»... 0... =3 3.5 0. v0.38. 0... 0002.338 3 .33 355 .2 ".050 .I 003338 I 03> 3.333 I 3.3.3.333 3. 00.2.0 39:33 I 833.023.. 303.. n..3038\.u 0.3.0.3.... 3.. 80:3 00...338\I.. 03> 3.3.33 I 3.3.3.333. 3. 00.2.0 32.3.... I 033.023.. 9330 33038 I 3030320.. 303.... 30:... \q 003.338 I 03> 3.3.33 I 3.3.3.333 3. 00.2.0 32.33 I «3.3.0.3... 303.. 33038\.. 3.000.. .0... c. 0.... :33: 3020000.... 3...... 33.03 3. 30.005...» 0.0 30503 .33 .a 0.51: 838.. 02...... 8. .03: 3...... 339m 30330.0( 0600... 0=0u3m 2:. 000?...— 3030003< 55.3 3... "awn—O: 90 ...coo. .n manna O..— I 2 II 8 II... 00» I 00>\.u 00>\4 «.02....02. >0... 0.0 .00» = 2005000000 2. 2. I 0.. 0... 01.000000: 00“.... 00>\ 008m 0.00.3030 020.3 0.< .0. ".0:... I 350000 .2. ".0:... ”.0:... I 3.3.00.3 .0.0..0u\.. 3.3.00.3 .20.. I 02.0.5.3 3.0000 V.- 2008000000 3.3.9.533 .0 9.3.0.. I 33.05033 3 9.3.0.. I .0.3.0..:.33 3 9.3.0.. \fi 0... .0238 o. $0.390. .03..»133 I 3.35:3? I fi...c..»x33\fi 0. 3.0.0.. .053. .055 .2 0000098 0:... 2.38“» ".0:... I 020230.. ".0:... 2.0.3230 ".0...o\.. A ".50. 88» ".60 A H.5... 0.3.0.0... 3 .3 I 0.3.0.0... 3 :qu 0.3.0.0... 3 .E I 0:... .0500. 0:... .0500. 0:... .0500. 3 3.2.03.6... .o 3 3.3.0.5....3 .3 3 3.2.0.5800 .o 5.3... .0: >00 03 .03 I. 5.3... .0.— »00 0.3 .03 I 5.3... .0: >00 03 .03 I 0...: .0500. 0:... .0500. 0:... .0500. 3 3.2.0.0300 .o 3 3.2.0.5500 .3 3 3.2.0363 .3 5.3... .0.— »09 0.3 .02.: I 5.3... .2. >00 03 3...... I. 5.3... .0: h... 0.3 .0..::\. 0.2.000. 0......330Nq 0.2.000. 03.22.90 I 0.2.000. 0.9.22.3 I 9.50:. I 9.50:. I u:.__0..:.\| 0.3.0.0... 0.0.3.0... 03.0... 2008000000 0... a. uc_>aou\u:_.c.3 I u...€8§:..£.n I u:§3§£.ctn\fl $0302.. 0.0 0.000 .055 .m— uaidz 3th 2.0300025 «00.0..— .c0E00000< «um—GOS— 030309 $03.5 0.2002 0:020: 0E. 0.:0..< 0...... .3 .0002 .00—.0: 91 7.2.03 . m «Hana. mm. a». h». “85.8.... 88.. «2 .2 2:62.. 025 v...- 3: 0:0 p.959\4 2059 I: 3050 I. 20.8.: .3» a :2: «no. I. .3» a as... any. I. anus a :2: 39. I... .50» its .5538»: 228:. 5a :5: So. I. 2.28:. An :5: nay—Na 22.8.: 5n :5: 80. \fi 38: .835 a 829:8 so: a :3: 30. .l :33 a :2: 30. .l :33 u an... «no. I 8 8.2 g 98—. use. to: .2 £05.. .I 33.8 .I £050 .I. o: .l. o: I. o: I. .55838... 5.3330: 3s\« EN EN Ezovigaoo .3 a§.§:ou>£ 28. 9.95 :25 v:- EoEo>oEua 93:95.2 8.2.8339. 22.8 \q 2059 I. $8.8 8050 \q 3328. 225 I 2.25.8. 223 l 26.28. 22.5 I «33:85 I atortuafi I. atomic...“ .| 9:25:30: .l 9:33.303 I. ogaccfifioacNfl 355808- 2: .0 5963\a hot—SN >033\4 8:02:23. 2: 9.- ; .m— Iflid! gm «603-0034. «9&9..- §Eooooo< Jun—O: I303 Veal—m 8.002 0:03.: 2:. be. .016: .IIEIO 55€< 9:. 92 ..0coo. .m manna 00....0000 .00 "0050 0.0002000 .00 "0050\4 0.0.0.000 00:00.86 00- 00.0030 =a00>0 “0050\0 2000.803 8000—80000 8000500000 000.0. 00. 0.03 I 000.0. 00. 0.00.. I 000.0. 00. 0.00.. I 000500. .0 00:00.00 I 000500. .0 00:00.00 I 000500. .0 00:00.00 I 00.000000 00.0.5000 00.05000 03.0000... .0 00:00.00 I 000-000... .0 00:00.00 I 0000000... .0 00.80.00 I 00:00.30 I . 00:00.30 I 00....0...>0\... .005000: 000 0000000 0002000..- 05.35.50 I 000.300.: 3:00.950 I 0000000.? 5.200.280 I 100000000 0... .0 00:20. u:..0u00.. I 0000000.. I 0020200.. I 0050 000 00000000000 00380.30 .0 002.00 I 000200.30 .0 000.00 I 003.0030 .0 000.00 I 00000 0... 00050.. 0.3300050» I 0.000 .0 00.30» I . 0.00.. .0 000.00. I 0020000200020. .003 .8 "00.00 I 0.003.000 .00 ”00.20 V. "000.0 I 3000000 50000000 3000000 000600.080 .50.. 3.0. 20050000 00.0.00 00022.00» I 300000 00.0.00 000000 .00.\4 030000.00 00.00..00\.. 0.00. 000.0000030N4 00I 000\ 20.0.3350 .30.. 0.0. 20060000 00.0.00 0002000030 0.00000 00.0.00 000000. 7.0. 030000.00 00.03.00 800. 000.00.000.30 00I n0..I 200—02020 .30.. 0.0. 20080000 00.n.00\.. 30.030030 20000.0 00.0.0 I 00000.. .00. 000080.00 00.08.00 \1 0.00. 000.0000030N. CCI 00>\.. .0000 00...) .00. .. .0000... 30.0.00; 00000... 00.00:? 00000 02 .m— 0.0.0 .00021002300 3. .03: .510 5000-5 0000.00000< 00002 0:030: 05. 000.00.. 0:0—00000.0( 0.00..< 2.... "ANA—G: 93 A.»coo. .m manna $9589 .95 5 3a... :83 as. on: Page... Ill 5326.: ..l..... C323: Ill 3.33:. 2: .9 8.3:»qu 5:828 lg 8:335 ll: 5:825 IE .2.) »_o.aE_uo§< .3 .2853 2. .o a... 3 .03.: o: \1 2. Va 2. I... 2: 3.53:3 tog—Eco no I. no» I. 85‘ :02. 5380.. 82 .NN «.986 c. 2:8»: .928 me a. 25.5% .028 .3; .689... Sec :02. .32: «5-0 a. 9.3» 9.5 .8589 .2. 89m— 8 523:2- .Szcm 0.: on: 2.3.8 .23 a... .3 uni-2 gm «6030-025 £0.25 «£050.02; “awn—O: 8—39.1— 3 .0002 0:03.: 2:. 5.3—; 2:. 59- —ovoz .0:—ID 94 .3500. . m 0.30... ".050 I ".050 I. ".0:.0 .I. 00.30.. 3:02.... .I 3.0.0... 33000:. I. 3.0.0... .0...0....:. I. .300 .l .300 I. :3... I. .0:»... I. .05... I. 00.......\.an C3038 \ b000008\.u b35000 I. 0......E\.u 0.0.0.5 N. 0.00.5 I. 2.0.0... .008000000 0... 0. a...«.:0:.0.0\q >.0.:0:.0.0\.4 ».0.:0:.0_0 I .0:—100.302.03.33 .0 805.0 I 00......0..0 .00 ”.050 .l ".050 .l «5.02:... In 3200...... In 00:000.... I. 0:30.: :0.0.00n\.u 0:...0E 5.0.00: I. 3.30:. :0.0.000\fl :0..00.0.. 8032\4 5:00.00 E0...0..0\.. 00:00.00 E0.00..0\.. 5.5.5800 :: 0.0.3300 0.0500. 3.5.5600 0:0 20.00000 0.0500. 3.55580 0:0 0.0.3000 0.0500. 20.00:. 0... .0 0000...... 0.5.0: 0228. In 5:03.. 0>.0>:.\4 5:02.: 0200:. I. 3.0.: 0... 0. :23 .N ”.0:.0 .I. ”.050 .l ".050 II 0.5.... .0.0:0u\.. 0.3:: .0.0:0..\. 0.3:: .0.0:0..\4 3:0 00.. 0000.3. .l 0: .l 00$ 2:0 00.. 0000.2. I. 0.. 00‘... 2:0 00.. 030.2. I. 5.60..) 0. 00> .— .0333. =2. .08... a... 5.3 00.00::00 0.3.0.0.: .3000 000.....00 0... 00¢ .— 000..0_....° .00300000< 0:002 00.3093 _0 .00—50001— 0:5 00::— 82020 32...: 0.. .l 00..\.. .0002 0u0=00 5.5.5500 :0..0:.50: :02 "Am—GO: 95 ..0000. .m 0.009 "000.0 I 00000.0 00. 00.00.00 .000... 00000 .0 .0.. I 0.00.. 000....300 .0 .0..\.. 0.000 0000.00.00 .0 .0.. I :0..00 00. .502. 0358.... 02.2.: .e 0... c...§\q 00000.0 00. 00.00.00 .000». 00000 .0 .0.. I 0.00.. 000.000.... .0 .0.. I .08.. 00.0.3... .0 .0. I "00...: I 00000.0 00. 00.00.00 .000... 00000 .0 .0.. N. 0.00.. 000...00.00 .0 .0..\.fi 00000 0002.00.00 .0 .0.. I ..00.000000.. 0... .0 0.00.00 .00.. 0... 000 .003 .m 00.000 0.00.. .000..a00..0 .00...0.E00 00.30.00 .0050 00...: 00000.0 0.0.. 0.000. I 002...: 000000000 0.000... I 000000 000000000 0.000... I 00830. 00.0.000 0.0.0000000 0.000. I 0.0.000. .0 30.50 I 0:0..0 000.300 .00..E.0 .0 30.30 I 0.0.000 .000..000.00 00...0.0.00 00.00000 00.00.... .00.00 "000.0 030...... 0.0000... 0.00 0.000. I. 0.50....>0 000000000 0.000... I 000000 000000000 2.2.0... I 008.00. 00.0.000 0.0.0000000 0.000. I 0.000... .0 30.30 I 00.0..0 000.500 .00..0..0 .0 30300 I 0.0.000 .000..000.00 0.9.0.0000 0.00. 000.303.0000 .020... "000.0 I 00.......0 0.2.00... 0.0.. 0.000. I 0.00....»0 000000000 $.00... I $0000: 000000000 3.0.0... 0.8.00. 00.0.00 0.0.0000000 0.000. I 0.0.000. .0 30.>00 I 0.00..0 000.0000 .00..0..0 .0 30.30 I. 0.0.0.... .000..000..0 0600000200 000 000300020000 000 0.300 0. 000 0.200 0. 000 0.300 0. 00 0000.0 «0.0030 0000000 .0 000000000. I 0000000 .0 00..00..00.0.\.. 00000.0 .0 00..00..00.0.\.. 208000000000 .003 .0 000.7206 .00800000< 0000— .0002 «5.902 0.0002 00.00003 .0 0.0.0000 0.00.... 080:8 3.000.050 .00—5.0008 0.06 03.0.0000... :02 96 A.ucoo. .m wanna 9.050 «I I. .0002 .50 ...0...0\.. «5.000.... I ".050 l. 05.0003 .l 0500.0 600.50 3.5.5500 95020.... 00:00.0 I. .0 .0005 0.000.. .00—00 .I . .0...0..00..u..0500. 2:8.» 8. .85 .82. .028 l . 88.. 8....2... I 0.... 2:33.... .5. Ana—DU. 0‘00: «90—00 ”.000: ouzmbotn II 0‘00: .6 Sing-Evy.- .. ave-Olga $33.2; 0.000: 00.3.5. \1 0000.. .0 00:02.2. 0.100.. .. . 0.... 0.. 50500000... 2.8.. .o S.......§\v 2.8.. I :8: 5.2.8.. 5 c... N o... .c 28...... .2... c. ... 0000.. I. :00: 5.200.. c. 2.. .l 00:055.... .0 000.00.. \fl 0...... 5.200.. 5 20\...I 00:053.... .0 000.00.. . . 5:55.... .o 8u..§\. . I: ".050 II II £050 mosaics-20:9 hum.” 8.5.0.022.“ 5:05.052. .88.... 80......» 8.3.0 u. .200...» u... .83 .l .a 3.. . ... 02005050.. 5505500 0... 9......0000 I. 5.055500 0. 2...".0003N 00:05.2... 2.. 0 _ 5.5.5500 0. 9.5.0000 Va . >.50.52\4 . ~5.50.59. Va £050 .I .l ".050 .l 020... ".050 0.0.0.. :0...t00 0. 3.09.0 II 0.0.0.. 0.. 0. 2.09.0 l. . . 5500 .0.0:0u\.l 50......00 c. 3.09.0 .l .muuwfiEg .0..0:0..\1 5...: 0.050% ~0.000.. 05.5.0.0.v 53.35500 .ouocoux . guano-N. 3:0.— 0. «.00: 33.2.5.5 . 0.00.00 MM 050.3 \4 0.0.3000N.‘ 000.500 0... 0.0 0:3 .c 050.00 0.0.33.0 N: 00:0. .0....5..0\Ifi .0 £28.... \..u 8:23....EEN. . June: =0..0.3_:_5v0\fl 7—00: :00..— 0 5.005500 003.0200 000500000< 50300.. 0.00:: 0&0: U 0e00,. 00:00:3— .0 «CUE-60G 0.89 97 3:8. .0 0.000 .5005 505000000 0000.. 0... .0 0.00.. 0. 03020.6 0.0.5.05 0. 0.. I. 0: ..I 0.. I. 00.....00 00.....a.0..0000.. 00..\.. 00»\..... 00>.» 00...5500\..0.0 0... 05 .= 50:00.... 5.000 .0050 I ".050 .l ..0.0 0m0=00 v.0...0N ~3.55500 5.5.5500 3.5.5500 0... .0 0.00505 0... .0 0.0..505 0... .0 0.00505 02.050550. .0 5.5500050. .0 035500050. .0 05.0.0000 002.5500 0 5.5.0.50 0055500 0 I 9.30.0000 003.5500 0 .I. 5.5.050 5.5.0500 5.5.0500 .....05000000 0... 0.5 00... 055 .I 0.5 00... 055 .I 0.5 :0... 055 .I 50 .5.00 0. 00.500. 0.0 5.5.0.000 0.5 I. 5.5.0500 05‘... 5.00.0500 0.5 I 0002.558 5 ..0.0 .53 .0. 00.00.?» 3.00.5050 ...0...0\.. .50....0 ...0...0V. ".050 0055500 .I 003.5500 I. 003.5500 I 0.0.. 3.5.05 .I 05> 3.5.05 I 05> 3.5.05 .l 5.5.0.5500 00.50.05.500 5.5.0.5500 .0000: .00. 0. 0.0 3 00.2.0 50.3.0 I. an 00.0..0 000......0 .I 3 00.0..0 50......0 I. :0000: 00300000.. 5...? 0500.09.00 0500.02.00 0500.09.00 5.0.000 5. 00.00500 9550 0:0..0050\1 5050 00000050 I. 5050 00000000§ 0.0 000505 .53 .0 05:00.30 000500000< 0.00..— .0002 Jun—OS. 00002 00.00003 .0 5030...— 0.0:.:— 0:0—.00 00.5.5500 :00:—000000 02¢ gacG—vC-dnvh— :0: 98 70300. . m 05.0... O... I c... I o: I 000 I 00» I 000\... 5.00.500 00... 0.0 .000 .— 3005000000 0..\.. 00 \0 00 I 0.30. 0.0000000 00.. I 00 I 000\.. 050300000 00.0.00 0... .2 0.030.505. 20.30 I £0.30 I 0.0.. 30.30 5.5.00.5 5.000» I 5.5.00.5 5.0000 53050.5 5.00 I 3005000000 .0 30.5.0500 .0 5.0.0.. I 55.5.0500 .0 00.0.0. \1 35.00.0500 .0 5.0.0.. I 0... .000000 0. 00.500. 5.09.8.5... I 5.09.0353 I 3.000035... I 0. «0.0.0.. .055. .003 .2 0.00.000 ..0.30 I 00.0 00000 00.30\ 30.30 I . ".50. A ".50. A ".50. 0.03055 .0 .3. I 0.0.5.05 .0 .3. I 0.0.5.05 .0 .3. I 05.. .0050. 0.0.. .5000. 05.. .5000. 5 5530.0.500 .0 5 5.0.5.0.500 .0 5 5.0305500 .0 .3005 .00 000 000 .000 I .3005 .00 000 000 .000 I .3005 .00 000 000 .000 I 05.. .00000. 05.. .00000. 05.. .00000. 5 5.2.0.5500 .0 .0 5.030....500 .0 .0 5.030....500 .0 .3005 .00 000 000 .0000 \... .3005 .00 000 000 .0000 I .3005 .00 000 000 .0000 I 050000. 0.0.3050 I 200000. 05.30.50 I 0.0..000. 0.0.3500 I 003.0% 5.3.0... I E0355 I 0.0.5.0 0.0.5.0... 0.0.5.05 3005000000 0... 0. 5.0000xw0..0..0\.. 00350503030 I u...»08x0030..0\fl 000.000. 0.0 0.000 .003 .N— 00050309 ¢¢0aco0oo< Nov:— —0362 “Am—GO: .0002 .3080...— .0 0.0.00... 0.00.... 000000 0.500.500 3.0.5.0009 035 00.50000.— «.02 99 ..ucoov . m 0.3mm. 2. I 2. I. o: I. ao>\.‘l easy: 3qu $03.22... .63: 2< d— 2220 .1. £050 1... £2.29 .I move... .30» a :2: nous use» a :2: one. I. use» a :2. on». I... use» 5.3 .5838: 2:3... in :2. an». al 23:2: a.» :2: non—\JI 2:58 «3 ca... nan-Na. 302. .335 a 80388 :02. a :5: ano— .| :00) a :2: one. I. 3003 a :2: no». I 8 8.5 g 32. ago. to: .3 82:9 1.. 2050 a... 8050 I... 9...... au>\ o: 3?! MN. ~8:o£=.:a£ b.8390: 2: 02.6.8 .6» on .2 2050 I. £05: I. 8050 .l 2.3.28. 223 l 25.5.8. 225 I 2.25.8. 2%: I 23033.... I. 2633.5 .I £E8c.\fi 2.263339 l. 9:35.326 .l o.._a:¢o_.no:c\fl 2:09.308. 2: 2. 33% ago\ sot-SN 3:08:33. 2: a... .25» .m— ool—oSIG calo§< unva— .0—.o= “AflfiO—g uvooz 66:38.5 s0 5039-.— 32-5— emu—.60 b15358 ales-Cg 2&0 Gaza-v.59.— 80: 100 ..0000. .m 0.009 ”0050 .I 200003080 003:. 00. 0.23 .I 000500. .0 00:00—00 I "0050 cl 20098080 002:. 00. 0.000 .I 0000.00. .0 00:00.0» .I "0050 .I. 2009000000 002:. 00. £23 I 000500. .0 00:00.00 I. 200900.. 2000300 200030.. 03.0000:- .0 00:00.00 030.000.... .0 00:00.00 I. 02.000000 .0 00:00.00 I. 00:020>\0 00..00.n>0\—. 00_.0:_0>0\I «.00E009. 000 0000000 000230.? 55005.2 I. 000330.... 55005.50 l. 0002002... 35005.07. I. 200300000 00. .0 000.0.» u::0w00$ «0:00.005 I. 0.030202. I 0050 000 20.030000 33.00.30 .0 «0.:le 002.0030 .0 u:.:0o\n 002.0030 .0 0.0.0.00Nfi 00000 05 0003.00 0.00» .0 00:. \.|.. 0.00» .0 u0_:0m\..’l 0.00.. .0 «02.00\A. 320.038.2080. .003 .8 "0050 I. "0050 I. "0050 .I $000000 $000000 “0000000 00005010000500 *0. .00—0.0.00.0 .50.. $0. 000050.900 .30.. 0.0. 80005000 00.0.00\.u 0.0005000 003.00 I 20005000 00.0.00 I: 0003800030 00020070030 .I n00_.a>0030\4 3009.0 00.0.0 I. 3009.0 00.0.00 .I 3009.0 00.0.00 .I 000000 .n0\. 00000» .00. 000000 .00. 030000.00 00.000.00N0 030000.00 00.00300 I. 030000.00 00.02.00 I. 800. 000_000000.0\. 0000. 0030000030 I. 0.00. 003000003... I. .0000 00...? d0» .— 0.000000 00 I. 00\... 00 II .003an 5.000... 00‘... 00» .1. n0»\ 00.00:; 00000 0.2 .2 000.7036 00030000: 0000— .0002 "Am—GO: 00902 60:00:0u .0 50300.— 0.0:.:— 0:0—.00 5.0200500 000.50% 0.8 CA: «G‘s-gh— :5: 101 Ahucouv . m wanna. €53:ch 5 3a... :8... as. 8: >226... II... C523: All bust... Ill £0.85 05 .o ‘35qu 5.8:? :84 8.5....» Am: 8:82.» {I .2.) 223.333 . .5983 a: .9 sea .8 .03.... 3\ EVA 2. l. o... 3.53:8 08328 3 I no» I. uqu :93 sou-80.. an: .«N 3.5:. .o .255 «3.3 $28. 23. :8; .03... but: a c. 98: «3.5a... 8 bias—Sou .695 cum 2: a2. 22x... .23 oh. .3 005.0396 «60303.0( unva— —ovo= "AHA—OE ovooz Eva-02$ so 8039....— ..OHE— 09:60 3.658800 «avg 015 cots—:59.— to: 102 7.0000. .m canoe C050 I 8050 I "0050 I. 00.0.0... 3.0.2.00. I 00.0.00. 3.0.0000. I 00.0.0... 3.0.0000. I :00- I :00. I :00. I .23.\« .20... I .202 I a..- I 80000000 I. fa00000o\.. 0.00.8 I 0.00.8 I 0.00.8 \ .00—0.0 808000000 0... a. 50080.0 I b50080.0\.. 850080.0Nfi .96. 300000000 .23 E . 00.000... "0050 :0..00003m 000.08 00.0.000 00:00.00 80300“... ~3.008800 00.. 30.00000 «000.50. 5000!. 0>.0>0. I 30.00.00 000 00000 .0 00:00.00 "0008‘ 0030002. I 00.0.08 00.3000 I 00:00.00 8030.... I 5.008800 00.. 80.00000 $000.30. 50000.. 0330. I 000.0000 0.00» 8050\4 0000000.. I 00.0.08 00.0.000 I 00:00.00 8030.... I 5.008800 000 20.00000 0000000. 500000 0>.0>0. I 20008 0... .0 000000.. 00.08 0... a. .003 . 8050 I 0.32. .20000\..«. :00 00: 0009.0. I £050 I 0.300 .2000u\..4 :00 00: 02.2.0. I "0050 I 0.300 .00000u\.. £00 000 000000. I .80..) 0. .00.. .— 0325. :0» .08... 2.. 00 I 00 I 00 I 5.3 00.000000 0.0.0808 00$. 00>\I 003 03000 0000600 0... 00¢ .— 8030»:— 0.:00= >095 80000.!— “Am—GO: 0.009 100.5500. 060800000< 00002 00.00000“.— 3:050 03002—000800 < 103 ..0000. .m 0.000 .0050 I 00000.0 00. 00.00.00 .0000. .0000 .0 .0..\.. 3.00.. 08000.00 .0 .0.\.. .68.. 080.0000 .0 .0. I 00.0.0000 .00 "0038\4 00000.0 00. 00.00.00 .0003 00000 .0 .0: 0.000 0003.00.00 .0 .0.s\.au .09.... 030000.. .0 ..__\I.. .0050 I 00000.0 00.. 00.00.00 .0005 00000 .0 .0..\.| 0.00.. 0002.00.00 .0 .0..\u 0000.. 0003.00.00 .0 .0..\... 3008000000 0... .0 0.00.00 .00.. 0... 000 .003 .0. 00000000 0.. 0. 00005300 0.05... 02:0 030..0> 00000.0 0.00 0.000. I 030....00 000000000 3.800. I 000000 00000000. .9800. I 000308 00.0.000 0.0.0000000 0.000. I 0.0008 .0 30.00. I 0.00:0 000.0000 00.8.0 .0 30300 I 80.000 .000000000 000.0000 .00 .0050 I 030..0> 00000.0 0.00 0.000. I 030:000 000000000 £800. I 000000 000000000 00.800. I 0000.08 00.0.000 0.0.0000000 0.000. I 0.0008 .0 30.000 I 0.00:0 000.0000 .00..8.0 .0 300000 I 80.000 .000000000 000.0000 .00 "003.0 030:0» 00000.0 0.00 0.000. I 03.2.03 000000000 3.800. I 000000 00000000.. @3000. I 0000.08 00.0.000 0.0.0000000 0.000. I 0.0008 .0 30.00.. I 0.00:0 000.0000 00.8... .0 30.000 I 80.000 .000000000 0000000000 000 000000020000 000 0.300 0. 000 0.300 0. 000 0.300 0. 00 0000.0 00800.0 0000000 .0 00.0000050‘ 0000000 .0 000000008. I 0000000 .0 000000008. I 808000000000 .00? .0 000.002.— 00.005 00.5% 80000....— "Am—GO: 0.006 300.005.000— 80800000< 00002 00.00000” 3:000 020000000800 < 104 70.000. . m 01.0... 000.0000 .00 "000.0 "000.0 I .0008 I 05.00.... I 0000000.. I 05.000... I 00000.0 00000.0 00000.0 .0. .000». 00000 80.00 I .0. .0000. 00000 80.00 I .0. .000... 00000 .00.00 .8080 3.308800 00000 00300.00 I 00000 00....0.00\.. 00000 000.00. \I... 0.00.00 0.00.00. 00000 .0 00.80:? I 00000 .0 00.80.30 0000: .0 00:00:00 I 0.0. 0800.0..000 00000 0002\- 00000 .0 00.000080 000.00.. 0000 8.0.000 0. 0.0 I 00.88.08. .0 0000000 I 0000 8.0.000 0. 0.0 I 00.88008. .0 0000000\\... 0000 u0.0.000 0. 0.0V... 00.88.08. .0 000.000\...... 0.... 00 .008000000 0... .0 0.00000 .00... 0. .c 00.300 0000 3 000.000 "000.0 000.0000 .00 “.008N 800.80 .00...0\.. 000000.000... 030.00.000.30 000008000000 0300000800 0300000800 0300000800 5.008800 0. 00.0.0000 I 5.008800 0. «0.0.0000 I 3.008800 0. 00.0.0000 I .000000 000.000 0.80000. I 0.800000 I 0.80000. I 00:08.08. 0... 0.0 30... .0 000000300 .00. 00 "000.8 "000.8 I "000.0 I a...» 8...... 2...... 800.000 0. 00.0000 I 800.000 0. 0.00000 I 800.000 0. 0.00000 I 00008800300000 I 3.008800 .000000 I 0.308800 .0.000..\n.l 0.00.00 0.00.00. 0.00.00. 080.00.“.II 080.00% 080.0%: 0.00000 00.800.00 0.0.0000”\_.. 0.080000N... 0.080000\u 0. 0000 00.88.08. 00..0..0.0.80 \....I 00..0..0.0.800\nfl 00..0..0.0.800\I.u .0 000.000 00. 0.0 00.3 .0 0.08000. 0.0002 000.0 8000000 «.0002 0.00.. 80023.80. .00800000< 00002 00.8000.— 5:000 020000000800 < 10$ ..ucoov .n 0.... 2000:. 205000000 0.80: 0... ..0 0.00.. 0. 020.33 0.0.8.0.: :. 0: I. 0.. II 0: V. 00:...00 00.....a.0:0..00. 00>N 00AM... 00. I. 00.258890... 0... 0.< ..— 00.0:0.....0:8 0.5.:00 .0 00...:..:8 ..0...0 ”.050 ..I .08....0 ..0...0\.. 5.5.... 5.5.58 5.5.58 0... .0 0.0080... 0... .0 0.0..E0E 0... .0 80:50:. 03.0.8028. .0 02.0.8830. .0 0>..0.:000...0. .0 3.338 08.3.58 0 .l 3.33:8 00......E00 0 ..I 3.33:8 00....:.:00 0 I. 8.0:..8000 8.0:..808 8.0:..808 285000000 0... 0:0 :0... 0.0.: I. 0:0 :0... 0.0.: .l 0:0 :0... 0.0:. ..I ...0 800 0. 00.300. 0.0 3.0:..808 0:0 .I .0.0:....000 0:0\. .0.0:....08 0:0 .I. 000......68 .0 ..0.0 .055 .0— ..0...00..0 .0: ..0...0\.. ".050 .I 0.000 acts. ".0...0\.. 00......E00 II 00...:..:00 ..I 00...:..:00 .l 0.0.. 330.0... ..I 0.0.. 530.0... .I 0.0.. 3.30.0... nu :0..0..0.:.:...0 :0..0..0.:.:...0 :0..0..0.:..:..0 3:00: .00. :. 0.0 3 00.2.0 b0......0 .I 3 00.0..0 80......0 ..I 3 00.0..0 5.03.0.0 I. :0..00: $0200.00: 5...... 2:00.033: 0.50.0.5... 2:00.033: «$0.00.. .0. 00.00550 9.060 08:88 ..I ":080 08:00:00\.. u:0:.0 08:02.8 I 0.0 0..0...0:. .055 d 5.0.00.5— 0.2.0: in E0080...— Jun—OS. 0.009 160.2530:— 0880000< 0.802 8300:3— 5:090 0>.0..0..0.......00 < 106 ..ucoo. .m manna SI 00 I 00 I no» I 8‘. no» I .0302?- »o... 9.- .aoa .— 6330: .00. 3090.332. 00 I 00 I. 00 \n o... 0. E3330 no» I. 3.“... no» I 202.3000 028:0 8< .2 0059000 .00 800%... 0050000 .00 Bv...0\q 005000» .00 ".850 \ 00055.8 .800 002500.00 1.80% I 00.030300 .8000» I $020882. firs-0.0.000 .0 900.9: I Any—0.00.0300 .0 00.0.8. I Aareafigooo .0 00.0.8. I 2.. 600000 0. 000.009.. $3003.03 I 30008.00} I €32.38) I a. 9:000. .250. .23 .9 20003» 83 uctaoa 8. 8w» €050\.. ..0. 90880.5 “.050 00580» .00 22.8 cue—033v "=00. 8mm“ 3.00 A 30.0 «.3082: 00 .0. 03.820 .0 g I 3.33.20 00 .3 I 00.: 02.03 00.: .8... . 20: .9003. 8 00.83.0000- .0 500.0 .90 >00 000 .25 I 00.: 02.03. 3 00.33.0000. .0 5000. .2. >00 000 .0000 I .00-.000. 3323:. I ..0 00.0..—3.0.5.3 .0 £000. .00 >00 000 .20 I 20: .0003. ..0 00.83.0000. .0 5000. .80 s... 000 .500: I £2.30. 23:33 I .0 8.2.0.0000- .0 500.0 .00 use 000 .25 I 20.. 02.03. 3 8.8.30.0...- .0 5000. .2. a... 000 .000: I 0.00000. 0.05300 I 95.20 I M0520 I 9.2.0.0 I «3.800. 03.800. 20.0200. 2000.338 0.: 0. 230830.03 I. u:.>000§0:0t0 I 3.303.323... I 0020.6. 0.3 utou .23 N. b02500— ofiiez h.vaum HIE JflGAV—g 0.000 160.252.0— aaoiooooo< 09002 £3.02...” h3.000 00.002.92.300 < 107 ..0000. . m 0.30... . 00 I 00 I 00 I 00>\.. 00>Vq 00>\.. .000....0t0 00000 0.< .0. 0050000 .00 ".050\.. ".008 I 0050000 .00 02.0 Va 20000. .00» 0 00... 000. I .00» 0 00... 000. I .00.. 0 00... 000. I .00» 5.? .020000000 00.000. 0.0 00... 000. I 00.000. 0.0 00... 000. 00.000. 0.0 00... 000. I 0000.. .0203 0 0.0—00.00 ..003 a 00... 000. I :00? 0 00... 000 I 3003 0 00... 000. I 0. 0x... ._ 000.. ..00. 30: .0. ”.050 I ".050 I ”.0:.0 I 00 I. 00 I 00 I ..0.00E0..00_ 0.0000000 00>\.. 00>\... 00.. N4 2.. 02.6.0 00» 0c .0— 0.0.0. «0...... 0.0005000. 0.000 2030. . .00. 0.020250 ..00 0.0008 000.0 ".2..0 I 3.00.2.2. v.2..0 000.0000. 20. I 000.0000. 20.00 I 000.0000. .00. I 303.0.0. I 32.0.0.0. I 303.20. I . 00000030000 I 000003.002. I 00000020000 I 2005000000 0... .0 02:00 I 0030‘ 000.0%. 0.0200000. 2.. 0.0 .003 .m— b0.00>0. 0.0.002 0005 0.0000...— "Am—GO: 0.00m. 100202.00. .00-000000< 0.0002 02.0003— 3:000 0300000000000 4. 108 .2000... .m 0.000 03000 00. .0 00.0000 2008\1 ..0 .0. 0.0.00: 000.0 200.0 I 8005000000 0.000.00 2008000000 0.0.0. .0. 0.000 I 0.0.0. .0. 0.000 I 0.0.0. .0. 0.000 I 0000.00. .0 00:00.00 I 0000.00. .0 00:00.00 I 0000.00. .0 00:00.00 I 00.0...0.0 060.050 0.00.00... 0.500.000 .0 00:00.00 I 0.0.00.0..0 .0 00:00.00 I 03:00.2... .0 00:00.00 5. 00:00.0>0\.. 00:00.30 I 00:00.56 I .00...0000 0.0 00000.0 000:000..0 3.000.220 I 000..,000..0 5.000.920 I 000..000..0 ......00.\..0.0 I .000..00000 00. .0 00.0.... 00..00000 I 00..0..000 I 00..0..000 I .00.0 00.. .006000000 00.0.0030 .0 00...00\.. 00.0.0030 .0 00.000 I 00.0.0030 .0 20.200 \I... 00000 00. 0003.00 0.30 .0 00.200 N 0.00» .0 05:00 I 0.000 .0 00...00.\.. 00.0000..0.0..0.0. .003 .8 000.0000 .00 ".00.: ..00.0 I 000.0000 .00 200.0 30.000. 30.000. 00.000 .000...0.00.0 .300 0.0. 3053.050 .300 0.0. .006..0.0E0 .300 0.0. 0.0005000 00.0.0. I 0.0080000 00.0.0. 0.000.0000 00.0.0. I 000..0>.0000 I 000..0>. \ 000..0>..0000 I 8.0000 00.0.0. I 8.00.8 00.0.0 I 2.00.8 00.0.0. I 00.000 .00. 00.000 .00. 00.000 .00. 00000.0.0. 00..0...0 I 00000.0.0. 00..0...0\.u 00000.0.0. 00..0...0 I 0.00. 00..0.0000.0 I 0.00. 000.0.0000.0\., 0.00. 000.0.0000.0 I .0000 00.03 .000 .. 00000.0 00 I 00 I 00 I .00...00.0 0000.0. 00» I 003 000 I 0200...... 00000 0.< .0. 0.0.00.3. 0.0003. 000% 0.0.00.0 3.0.062 0.000 .000..0...00— .00—000000< 0.0002 02.0000”.— .0:—00G 03000080800 0. 109 ..ucoov .m manna .8589 85 6958... 85 60588 35 5 8.... :9... .2. 8.. Diane. Ill >525... III Eta—x: ..I... £0.86 2: .0 $3582. 8:325 III 8:825 ..I... £58.60 .II. .2... u.8uE.xok—n< .nu ~889- GvEuR—o 85 605999 85 a: .o >5 3 .03.: o: N. o: I. 2. Va 05 3.53:8 cog—E8 «0 I: .6®\fl Ri.l. S!£=2528~2$_du .688. 20c 50.. .32: 33:8 8.. .26 c. via—.26 .9. «9:6: 0332.»- xx. 826: 0.: as. 2.35 .23 o... .—N @3625.- 01162 in East?!— «EDGE .3 1:93:23:— .Ivlooooo< 3.002 8:353 5:25 03-92—95-600 < 110 7.2.00. .n manna £05.. I 30:8 I 3050 I «532. 3.3.5:. I 9.2.0... .0326... I 93:3... .0326... I :2! I ..av0\.I :9... I .20... l 00..0...\.. .20... I sun—05.0% Concave“. Paves! \J. 0.2.. I 0...... .\...I 0:02... I .606.- .c0Eac0aoa 0...... C5=0Eo.0\u b5:0£0.0\|.. P8=0£0.0\fl .0:—1.8.33.5 .0..) .< .n 333.9890. 0.3» o— c. 80.58.. 20:! I 0......— 000000... £0...o\fl .5050 0.59.80 C0...o\ 9:032. I 9:032. I 95025.. I 9.3!.— coEuoe I 9.30:. 5.200.. I 9.50.: co.m.00..\... 5.60.0.» E039... I. 5.680.. E0395 I 3.580.. 6038.. 3.55.58 v:- 0830300 505-0. 3.5553 9.0 95.3.60 30500. 3.58250 w: 288.60 .0.—0500. £0.85 0... .o 008...... 5:003 0225‘ 5:003 0202.. I 5:03.. 0295.. I 3.2.. 0... a. .055 .N 22.8 I 20...: I C050 I 0.32. .8060» 0.32. .0.—gum I 0.22. .uuocuuNfl 2.8 on: 032.... I 2.8 on: oaacgcma 2.8 00: 025...... I «.62.? o. .00.. .— 5.0.92.2... E.» .092: 0... c: I c: I o: I 5.? $20053 0.3.0.2: I I I a 000.30» 0 .0.. . 00? 00?. no»\. :50: . .. < . .032 I553..— Gfiuol «003 0....— Iuaav— 5.09 ..r— Jun—OE 0.33—0.33 0.: c. 0.502 1:033:3— 350:8 111 ..0000. 0000000. 000 0.00030 .0 00000000 ‘60 0509: ”50—20 N. 00000.0 00. 00.00.00 .0003 00000 .0 .0.. I 0.00.. 0002.00.00 .0 .0... I 0000: 0003.00.00 .0 .0.. I 20000000000 0... .0 0.00.00 .00.. 0... 000 .003 .m 000000.003 ”0050 0.00:3 00000.0 0.00 0.000. 032.0000 000000000 33000. 00000: 0000:0000 2:000. 0000.00. 00.0.000 0.0.0000000 0.000. 0.000.: .0 30300 00000.0 0003000 00..0..0 .0 30300 0.0.000 .0:0..00000 0... 0.32. 2 0000000 .0 0000000000. . m 01.0.... "000.0 I 000.0 I 00000.0 00. 00.00.00 00000.0 00. 00.00.00 .000... 00000 .0 .0..\.. .000... 0000: .0 .0.. i 0.00.. 000200.00 .0 .0.. - 0.000. 000.300....— .0 .0..\.- 0000: 0000.00.00 .0 .0.. I 0000: 0000.00.00 .0 .0..\.. 00>..0.00000000 000.0000 .00 .00...0\.. 00.0.0.0 0.0.0.00. 00050 0.00:0: 030...; 00000.0 0.00 0.000. I 3000.0 0.00 0.000. I 030...": 020.0: 000000000 ...2000. I 000000000 @3000. I 00.08.. 0008.. 000000000 b.0000. 0000.00: 00.0.000 0.0.0000000 0.000. 0.0000. .0 30.0000 00000.0 00000000 00:00.. .0 30.0000 0.0.000 .0:0..00000 000 0.500 0. 0000000 .0 00.0—0000.0. 0000:0000 ©2000. I 0000.0:— 00.0.000 0.0.0000000 0.000. I 0.000.: .0 30300 I 00..0...0 0:00.000 00:00.0 ..0 30300 I E00000 3:023:00 05 0.22. 2 0000000 ..0 0020000000. I 002020.... 000 000200020000 00 000.80 03000.0 2000000000000 .055 .0 .000: 00.05.: 000.0003 00.00: 0.03 00.. 0.0 c. .082 .0.—0.00025 0.0.0.—0m 0003— 00.0G ..r— «um—DOE 112 ..0000. . m 0.1.0... .00...0 I ”000.0 I 000.0 I 00:000.... I 20:000.... I 0.0200000 I 0300.0 00:00.0 0500.0 .0. .0000. 0000: .00-00 I 00. .005. 0000: 80.00 I 00. .000... 0000: .00.00 I .0055 3.00.0080 0000: 000.00.... I 0000: 000.00.... I 0000: 80.00.00 I 0.00000 0.00000. 00000.0 00..00..0> I 00000.0 00:00:00. I 00000.0 00:00:00. I 0.0. 0.0-0.00.00 0000: 0000: 0000: .0 00.08.33 000.00.. 000.. 00.0.90 0. 0.0 I :0..0:.00.:. .0 0000:00\ 000.. 050.000 0. 0.0 I :0..0:.00.:. .0 0000.50“. 000.. «0.0.000 0. 0... Va. :0..0:.00.:. .0 0000000.\... 0... 00 .00E000000 0... .0 0.00000 .0..) 0. .c .0:0..00 .00...0\.. 0.003.000 .0: 000.0N 000...: I 0020.00.00.20 0020.00.00.20 00..0..0.0000..0 00.00.800.00 00.00.50.000 00.00.000.00 5.000003 0. 00.0.0000 I 3.000.030 0. 00.000000 I 5.00.0080 0. u:.0.0000\.. .000000 000.000 0.0.0000. I 0.0.0000. I. 0.0.000... I :0..0E.0.0. 0... 000 30.. .0 "000.0 I .00...0 I ..00.0 I 00.0.. 00.0.. 00.0... 300.00 :. 0.00000 I .00:...00 0. 0.00000 I .00:.t00 0. 0.00000 I 5.000.030 .3000qu 2.0000000 .00000» I 5.000.080 .000000\__. 0000000. 0.050% 0.00000.\u 0.00 \.n 0.00. \1 0.:0000\A .0000: 00.0..—0.00 0.0.3000 0.200000 \fl 000.3:00\.- 0. 000.. :0..0:.00.:. 00..0..0.0.0.0 \fl 00..0..0.0.0.00\.. :0..00.0..0.0.00\.| .0 000.000 0... 0.0 003 .0 .000! 00.000... 00.03— .0005 .0.— 00000. 0:09 ..r. “AM—GO! 00030.00? 0... 0. 00002 102.08.... 00.2.00 113 ..ucoo. .m ounne ¢¢l| n9..\| hm Cell EN 293... 3958989 399.. 9... .o 9.99: 2 933.56 £9.33... c. 03:30 3.3.3.9339. 992.5689...» 9... 9.< ..— 9E.. 39.953 9.5.9999... 5.3 3.....3. 2.... 99%.: 5.3 93953 ..9..3\q =39 £93999. 29:3 0. 39.9.. 93 ”3.....N 3.5.8.39 3.3.5.3 5.36.39 9... .3 £2.59... 9... .3 9.9559... 9... .o €2.59... 93.5.3339. .o 93.539939. .3 3.3399939. .o 3.3.239 99....539 - l. 3.3.239 99......239 a ..I. 3.3.939 99.36.39 9 ..I 3.3....38 3.5....38 3...:...399 256398.. 9... 93 an... 93:. I... 93 ca... 93:. I. 93 ca... 93... ..I .3 >939 9. v9.33. 9... 3.3.9309 9.3 1... 3.3.2999 9.3 .1 3.3.6339 9.3\ 999......539 3 =9.» .23 .2 03.. 93.99.30 ..9z3\« "99:... In 29:3 .l 99......39 I 99......39 .l 99...:339 nu 93> 5.3.3:. I: 9.9» 5.3.2.. I. 93> 5.3.9: I 3.3.3....63 3.3.3.563 3.3.3.563 .399: .99.. c. 9.... 3 99.2.9 5.2.53 .l 3 99.2.9 >333; «I .3 99.2.9 >323... |.. .239: 3399.2... p.923 3:33.95. 830.933.. 353.933.. 3.299.. 3. .3395...» 93.... 2.2.9239 I. 93.3 23:99.3X. 53...... 32.9939Nq 93 9.359... .23 .m .932 uni-lar— Ifiuol 393 3.— 233— ...99 ..r— Jun—OS. 9.9.3333 2.. a. 9—9992 13.82... 3.89.. 114 7.0.000. . n 0.30... 0.. I. O: I: o: I. 00.. ..I 00.. ..I 00» I. .030..0>0 ..0... 000 00. .— .0...0..0>0 .00. 2008000000 00 00 Va 00 \1 0... 0. b00030: 00 I. 00. II 00 II 0.00.-00000 00.0.00 00¢ .0. “000.0 I "000.0 I: 000.0000 0?. "000.0.Vq 0020.00.00 10000.. I 0030.00.00 .00000: I 0020.00.00 10000.. ..I 2000.00000- .0..0.00.0008 .0 00.0.0... I 0.00.-0.0008 .0 3.0.00. Va 0.00.-0.00000 .0 ..0.0.00. ..I. 0... .3008 0. 000.0000 .0...000..00§\.. 3.02.0100.» II 3.02.0300? N. 0. «0.0.00. .00—00. .003 .m— .§_:o¢8 in... 8. .88 SEN. .2. .3. 25 nosey. as... I .000 as... . as... 80.0 as... 0.0.8.00. 00 .0. 20.00.00. 00 .... II 0.0.00.0... 00 ..x\.. 00... 02.000 00... 02.000. 00... 02.000. 00 00.2.0.0..000 .0 00 00.050.50.00 .0 00 00.020.50.00 .0 0.000. 000 .0.. 000 00.5 I. 0.000. 00.. >00 000 00.6 I 0.000. 00.. >00 000 00.6 I.. 0.0.. 000000. 00... 000000. 0.0.. .00000. 00 00.00.00.050 .0 00 00.00.00.060 .0 00 00.00.00.050 .0 5000. 000 .0.. 000 0000.. .l 5000. 00.. .0.. 000 0000.. I... 5000. 00.. .0.. 000 0000: I. 002.000. 0.0...0000\.u 0.000000. 0......0000 I: 002.000. 0......0000 I. 05...... I 00...... I. 2...... I. 0.0.00.0... 0.0.00.0... 0.0.00.0... 8006000000 0... 0. u:.>08i0..0.00 I... mega—00.55.50: 2.3003030... .l ..0>.0>0. 000 0.000 .0....» N— .ovo! will-r— I2U0l .003 00.— 05 0:09 ..r— w—flGOZ 8.30.83 2.. a. .082 102.89.... 05.80 115 7.0.000. .n 0.000 00 I 00 I 00 I 000\.. 00..\.. 00>\.. .000...00.00 00000 00¢ .m— 0...000. a 0. 0 00.000. 000... 000 0000. 0). .00...0\.. "000.0 I 000 000» 000 "0003\1 ..0000. 0000 0 00... 000. I 000.. 0 00... 000. I 000» 0 00... 000. I 000. 0...... 205000000 00.000. 0.0 00... 000. I 00.000. 0.0 00... 000:: 00.000. 0.0 00... 000. I. 00000 .003... 0 0.0.9000 ..003 0 00... 000. I ..003 0 00... 000. I ..00) 0 00... 000. I 0. 0.0. .. 0000 000. .00: .2 "000.0 I “000.0 I "000.0 I 00 I 00 I 00 I .0.:0600.00. 000000000 00‘ 00» I EN.- 0... 000.00.. 00» 0O .0. 030000.000 00000000 0.00m ..0.0...00.0.. 00:...0000 "000.0 "000.0 I 00. 00.0.. 0 "0008\ 30.28. Eczema. 2.25.8. 20.00 I 26.5.8. 20.00 I 030.t0.0. I 030.>00.0.\.. 0300080. I. 00.0000..00ac\1 00.0—50.30.... I 000.000.3000 I. 2000300000 0... .0 00E:0\fl 0:3 I 00?...0N. 8000.00.00. 0... 000 .003 .m. .03: 3.5-... 8.00.. .83 .00 2.0.... 5.00 .00 “.050... 00.30.00? 0... 0. 00002 102.80.... 05.80 116 ..0000. . m wand... I 000.. . 000.0 I n000.0 0.00Ehu0.u\q 2008000000 nap-OEGQOOGI I 0.03:. 50. was I 8.5:. Ba 013‘ 00...... 00. 0.000 0 .0.: 0 00..0...00 I 0000.00. .0 00.80.00\..I 0.00.—.0:. .0 6350.8 “.0.. .. 000.0? 0.00.59... 00.0....0 I 00 00 02.000000 .0 00..00.00\a. 0>..0000..0 .0 dawn.” I 0>..0000..0 ..H.00..“.0.:.0.>0W 000.000.. 000 asap“ . 00 I 00.. 0 0... .0 0 0000000..- ..flWHyaafiNM 000.300.... 20.00.90... I 0000000.? ..mmflnwu“ I .00....“ .0 W00 .008000000 . . . I 90.00000 I . 00 0 .000500 00.0.03 . I ....§.€.=.=_..3\.. .8 g . 3:00.... .o 0:...0.\« 82.8% .M ”um” I . 0.80 .e 0.....8\4 002.5385... .23 .0 0.00M .0 ..0...00\.. 0. . . I "000.0 I . I 000.0 . an...” 028.... 02%.. 000 0... 00.0. 0.. .0. . 0.0 .00 0. 3 - ~CUE%£QEU ~59” & 0v «CUfihhflfiu—fiafl—Uw II. map-0530C? ‘Uofl—Uh\ 0.0005000 00.0.00\.. 0003030000 I 002.0000000 I 282.3%... . a... 3...... I fix: 8...... I 0.00000 00.0.00 .0 000000 .00. 00 0000 .00. 000000 .00. I 00 .00 I .00 030000.00 00. .. . 000020.00 00.00..00\.. 0H00HHWH.WH........H.0.0 I 0.00. 000.0.0000.0\.. .0000 00...... .00. .— 0.00. 000.000000.0\.. .0 . 0.00000. 00 I 1.0.0.900 0:000... I O: 00 . W. .0 . 0.. 00000 < m. 0.0.”\.. 00 I 00.\.. 00 0. «EGO—z 08'00— .003 00..— éau— 0:09 .0.. . .000! 3.000: 0... 0. 00002 . 00.000-00.5 103.00.00.— ..0.000m 117 ..0000. .0 0.000 0. 000... 000.0. 00.. 00.. b.0200. .dvi. 50.00.00. 3.00.00. Iql 0.00000 00. .0 0.....000000 00. .00....0 Isl 00:00:00 Ian. 00.0.0000 Imul .003 30.05.0009... .3 3.00000 0.. .0 >00 00 .0000. 00 00 00 I 00. 00.000003 00.00.0000 00“ 00>\|..I 00>\.. 0000 00000000 00.. .NN 00.0.m .0000: 00. 00.0.m 02.03 00. .0300. 0.0.. 0000 .0000. 0.0000000. 203000000 0000 0.0000000. 303000.00 00. 000 .0300 .003 0 .—. ..N .000: 00.000..— 00.U0¢ .003 00.— 0000.— 0..0O .00. Juno: 00.0.0-0.003 0‘. 0. 0.0002 100.000....” 00.00% 118 ..ucoo. . m 03:... :05: I 2050 I 20...: I u:.:.a.= 30.26:. I u:.:.n.: 3526:. I u:.:.2. 3.0.3.3. I :25 I :20: I :25 I. :05... I. 00am... I 00:»... I but:¢00o\4 Ea::30n\.. aha—0:300 I 0.20.... I 0.3.... I 0.3.5 I :00:...“ 20.53.03: 05:. C!:0E0.0 I 35550.0 I 35:09:30 I .0>0. 3:02:02; 22.3 .4. . 00.32.0000... .0 00:00 2050\q. £05: I :05: I u:..0u_z5\fi u:..0u.0:a I «5.050.... I 3.an 50.0.00qu u:...a:. 5.0.00? «:32. 5.0.00qu 8.0030: £030....Nq 50:00.0: E038 I 50:00.0: E039... I 53:98:80 .0:." 22:03.00 «00:30. 5:03: 039:. I 5.5.8600 .05 9.23.60 .30an— d.:00.x. 023:. I 3.55.600 .05 9.28300 «00:30. 2:03: 023:. I 20.02.: 0... .0 0892. 8.2.: 05 a. .23 . v.05: I 0.3:: .20:0& 2:: 0n: 00:05:.qu o: I 00$ :05: I 0.3:: 120:0» I 3:: 0»: 039.5 I a...“ @0500...» .c: 2050\4 0.3:: 220:0“. I 2:: 0a: 039?. I 0: a0»\.. .82.? 2 d0» = 5033.96 ....a .090:— 0... 5.3 3.00::00 03:03:. 00:05 «00.30.. 0... 0..< . 03.5 0:08.002: c.0002 o.¢°3!0.—§G< toga-0:00:00 .032 «:0Eoo0oo< 0.0002 900‘— 0:0Eco0ac< 0.0002 05.6 ..ucEco "Am—GO: 119 ..0000. .0 0.000 0000000 0..:08000000 00.0:000000.0 0. 00.08000 00. 0000000 “0008\1 “00:8 I 000.0000 "0008\4 00000.0 00. 00.00.00 00:00.0 8. 00.00.00 00:00.0 8. 00.00.00 .000... 0000: .0 .0..\. .000... 0000: .0 :0." .000... 0000: .0 .0.. I 0.00.. 0000.00.00 .0 .0.. Na 0.00.. 0000.00.00 .0 .0.. V0 0.00.. 0000.00.00 .0 .0.. I 2008000000 0... .0 0000: 0000.00.00 .0 .0..\.. 0000: 000...00.00 .0 .0..\.. 0000: 0000.00.00 .0 .0.. I 0.00.00 .00.. 0... 000 .003 .m 003.0030 "0050 I 080800.000 "02.8N. 0008.00 @0000. 00:8\: 0.0.0..0>0 0.0.0.0....0 0.00:0»0 00000.0 0.00 0.000. \ 00000.0 0.00 0.000. I 00000.0 0.00 0.000. I 0.00..0>0 0.02.0000 00.0..-$0 0000:0000 x...:00.\.. 0000:0000 0...:00.\.. 0000:0000 b..:00.\q 00000: 00000: 00000: 00080000 0.0000. N... 000000000 b..:00.\4 0000:0000 0.000. I 0000.08 00.0.000 000x08 8.0.000 0000.08 8.0.000 0.0.0000000 0.000. \1 0.0.0000000 0.0003 0.0.0000000 0.000. \0 0.0008 .0 30300\.. 0.0008 .0 30.000 I 0.0008 .0 30.80 I 0.8..0 000.800 0.00..0 0.8.0.000 0.00..0 000.0800 .00..8.0 .0 30$00\.n .00..8.0 .0 30.0000 I .00..8.0 .0 30.80 I 0000000000 80.000 .000000000 80.000 .0:0..00=00 80.000 .000..00:00 000 0:0..0000.0:00 000 0.300 0. 0:0 0.300 0. 0:0 0.300 0. 00 0000.0 u:.::0.0 0000000 .0 00.800008:- 0000000 .0 000000008: 0000000 .0 000000008. I 808000000000 .0....» .v 0.0.36 080800000< 00002 .000: 000800000< 05500000< 00002 "Am—GO: 98.33.20. 03.2.8000 .082 98‘. 08:0 90:88 . 120 ..0000. .0 0.000 0000000 0.000.. "000.0 I "000.0 I 00 000.800.00 "0008\4 08.00000 \ 08.00000 I 08.00000 I 00000.0 00000.0 00000.0 00. .0000. 0000.. .00.00 0000: 00.....0. N... 00. .000... 0000: .00.00 00000 00...00.00\I..| 00. .000... 0000: .00.00 I 0000: 00800.00 I 0.00.0200 3.808800 0.0800 0000000. 0000: .0 00..00..0>\.u 0000: .0 08.00.05 I 0000: .0 00..00..0> I 00.0. 0800.0..000 00000 00000 00000 .0 00.000200 000.00.. 0000 080.000 8 0.0\.. 000: 080.000 8 0.0\.. :00: 080.000 8 0.0 I 0... 00 808000000 00.88008. .0 0000000\... 08.08008. .0 0000000\.. 08.08088 .0 0000000 I 0... .0 0.00000 .00.... 0. .0 0:00:00 0000.000 800. 0000000 0.0000 00:00.00 0000 "000.0\ ”000.0 I ..0 008800.00 "000.0 Va 00..0.00600000 00..0.00800000 00..0.00.0000...0 0200000800 8000000800 0.000000800 3.008800 0. 08000000 I 5.008800 0. 08000000Na 5.008800 0. 08000000 I .000000 0000000 0.800000 I 0.800000 I 0.800000 I 00.88008. 0... 000 30—. .0 0:00:00 800.30 800. 0000000 0.0000 0000800 0.600 "000.0 \. "000.0 I 00 008800.00 "000.0 V. 00.00 00.0.. 00.0.. 808.000 0. 0.00000 I 808.000 :. 0.00000 808.000 0. 0.00000 I 5.00888 .8000» I 5.008800 .000: \. 5.008800 .0.—0:00 I 0000000.\.. 0000000. Va. 0000000. I 0.00000\.. 0.00000 \q 0.00000 I .0000: 08800.00 000.3000\.. 000.00000Nn 000.3000 I 0. 000: 08.08008. 00.80.08.800 V0. :0..00.0.:.800\.. 00.80.08.800 I .0 0000000 0... 000 003 .0 00.00 .00800000< 00002 .0002 .00800000< .00800000< 00002 “Am—GO: 0.00..00..00< 00.00.—00000 00002 0.000— .00..0 0.08.000 121 70000. . m manna. 600.0000 .00. 2000.0 .00—0000000 0.0000 05 .0 E00: 0. 030.006 0300.00. 0. 00 I 00 I 00 I. 000.200 00.0—5.0000000 00". 00$ 00» I 005.00.009.50 0... 00< .2 .0000. 0000 .0 "0003\4 200.0 I 000.0000 .00 “0008\4 5.000.500 5.00.0080 3.000.500 0... .0 0000.020 0... .0 0000.000. 0... .0 00000.00. 003500000000 .0 00380000000. .0 003500000000 .0 00.50000 00.000000 0 I 05.0.0000 00.0.0—000 0\.. 0020.008 00.000000 0 I 00.00.—.0000 00.008300 00.00.—00000 :00:—000000 0... 000 00... 0000. I 000 000. 0020\1 000 00... 000.0 I .00 E00 0. 000.000.. 000 00.00.0008 000 \. 00.00.—00000 000 I 00.00.0800 000 I. _ 000.0588 00 ..0.0 .003 .0— 00.00.10 03.0030 2008\- C0...0 I 0050000 .00 "000.0\ 00.3.5000 I 00.0.00—00 I 00.0—0.000 I 0.00 5.00.30 I 0.00 3.00.00. I 0.00 5.00.00. I. 03.00.0005... 02.050.000.00 03.00.30.060 .0000: .00. 0. 000 3 00.000 0000.300 I 3 00.000 buzz..- I 3 00.000 b00000- I :0..000 00300000.. 00...) 0.03.8.0! 0.000.020!— 0.000.0.t00 00.2000 ..0. 00.000000 000.00 30000000 I 0000.0 0:0000000\.. 0000.- 000000000 I 000 0000.00. .003 .m 030$ 000300000< 0.0002 7-00! .00-000000< 2.0300023 0.0002 «AHA—OE 0.0.0.-0:03 00.02.3000 .0002 0.03 .0000 00.00.00 122 ..0000. . m 0.30... o: I o: I O... I. 00‘ 000 I 000\fl 0.0.02.0; .000. 000 .00.. .— _ 2005000000 00 I 00 \q 00 I. 00. 0. E80000: 000\ 000 I 00>\.. 0.00.—00000 00.0.00 00< .2 . "000.0 I 000.0 I 00.20000 .00 .000.0\. 0030.00.00 .000000 I 0020.00.00 .00000u I 0020.00.00 30000.. I 2008000000 3.00.0500000 .0 «5500: 00.00.05.003 .0 ..0.500.\a. A0V00.00.00000 .0 050.00. I. 00. .000000 0. 000.0000 00.00.—00003 I 00.00000003\fl 3.0000000? I 0. 050.00. .00.00. .003 .m— 0550... 000 00... "000.0 000.0 I 000.0000 .00 000.0 A 0.00 A 0.0.0. A ".0... . £500.00. 00.... I 00... 000000. 50.00.00. 00 .0. I 0.0.. 000000. 0.0.00.00. 00 .0. I 00... 000000. 00 00.00.550.00 .0 00 00.000.50.00 .0 00 00.00.55.000 .0 0.00.0 000 000 000 00.6 I 0.000. .00 000 000 0000\ 0.000. 000 000 000 00.6 ....| 00... 000000. 00... 000000. 00... 000000. .0 00.020.50.00 .0 00 00.00.550.00 .0 00 00.020.50.00 .0 0.000. .00 000 000 00000 I 0.00.0 000 .000 000 00000 I 0.000. .00 000 000 00000 I 0000000. 0.0...0000 I 0000000. 0.0...0000 I 0000000. 0.0...0000 I 03...... I 0.....H..\.. 05...... I. 0.0.00.00. 0.0.00. 0.0.00.00. 00020000000 00. 5 00.0000\u0..0.00\4 u0.>000\u0..500\0 050000.05520 I 00.0005 000 0.000 .003 .N— 00.06 .00300000< 00002 .000: .00—000000< .00.000000< 00002 "Am—DO: 9:008:80 08......880 .082 98.. .00.... 9000.00 123 ..0000. . m 0.30... ., 00 I 00‘. 00 I 000\q 600.0000 .00. 00 I 000 “I .02....200 0.8.. 00¢ .0— 00... 0 .0000 0...; 000.0000 .00 0008‘ 0. 00002 00 "000.0 Na "000.0 I 00000. 000» 0 000. 000. 0000 0 000. 000. I 0000 0 000. 000. 1| 000» 0...! .00—0000000 00.090 0.0 000. 000 I 00.00... 0.0 000. 000. \0 00.000. 0.0 000. 000. ..I 00000 .00.00. 0 0.0.0.000 ..00) 0 000. 000. I ..003 0 000. 000. I 0.000, 0 000. 000. ..I 0. 000. .. 0000 000. 30.. .2 "000.0 I 200.0 I 000.0000 .00 .0008\.. 00 I 00 I 00 .I 00.000.00.00. 000000000 000‘ 00S 00» I 00. 000.000 00.. 0A. .0— . "000.0 I "000.0 I 000.0000 .00 "000.0 0 000.0000. .00.00 I 000.0000. .00.00. I 000.0000. .00. I 0B0.>00.0.\.‘ 0303085 I 0300.030. I 00.000020000\.. 00.000030000\.4. 050005.0000 I 2005000000 00. .0 00.0.0.. I >0>000N4 000000 I 3000.00.05 00. 000 .003 .m— 0306 000500000< 00002 .000: 000800000< 0:00.000..0< 00002 “Am—DO: 0.3.03.9; 00.02.0008 00002 0.00.— 0035 0.00.200 124 ..0000. . m 0.30... . .0.—0... 00.0.00: ".050 I ”.050 I. 0. a: ..0. ".0:.0Na 0.00800000- 20050000: 0.0063002. I 0.0.... .0. 0.03 I. 0.0.... .0. 0.03 ..I 0.0.... .0. 0.0!. ..I 000.20... .0 03.00.00 .I 00050:. .0 00.80.00 00050... .0 00:00—00 .I 00.0.80... 050.80 050.20.... 01.3.2.0... 00:00.00 - - 03.058... .0 :0..00_00\fi 03.00.02: .0 00..0...0. ... :0..0:.0>0\.. :0..0:_0>0.\..... 00:02.50 I. 80:30.... 0.0 00000.0 002.000... 05.00:..20 I. 003.000.? 5.200.980 \a. 003.000.? 3:30.950 ..I 22.3.3000 0... .0 0020.0 «5.03.2. .l 5.2022... V4. 20:05.2. ..I .050 0:0 508000000 003.0030 .0 95.00 \4 002.0030 .0 «0.300 Va. 003.0030 .0 $5.00 I. 0000: 0... 0003.00 0.30 .0 u.....00\.. 0.0% .0 9....00V4 0.00.. .0 5.2.0.. ...I 00.500.58.02. .23 .8 ".050 I ”.050 I. 00....0000 .0.. ..0...0\.. $0.000. $0.000. $0.000. 2.0.0.0350 {Sm do. 2.05.3350 .50» 0.0. 2.06.3380 .50.. 0.0. . 0.0005000 ..0.0.0.\.1 0.00530: 00.0.0. I. 0.0080000 00.0.0. I. 0:0..030000N... 0:0..20000 I. 2.0.3.6030 .l 3.09.0 v0.0.0.\.. 2.09.0 90.0.0. .I 2.09.0 90.0.0. I 00.000 .00. 00.000 .00. 00.000 .00. _ 00000.0.0. ..0..0...0\. 3000.0.0. 00..0...0 .I 00000.0.0. 00..0...0 I. 0.00. ..00....a....0.0\.. 0.00. 08.0.0000.» .I 0.00. 03.9.0250 .I .0000 £023 .00.. .— 0.. .I 00>\q 0: .I 00: 0.. .I 00. I. .0000... 28.....60 5.00.... 00.00.-0., 0.000.. 0.< d. 039° 20800000< 0.0002 0.60..Iv._.3< $0.0..._0000Q 7.32 .00—000000< 0‘002 0.00; 0:0-5.00004. 0.0002 2.0.5 0.9.500 "dag—OE 125 ..0000. .m manna .0300103 0. 000.0 00.... 0!. 00: 0.3000. II... 0:00.00. III b.0200. .I...... 0.0.30. 0... .0 09.03.00 00:00:00 .II. 00000000 I..l 00.0.0000 I....... .0...0 u.0.00..00.00< ..N .300000 0.. .0 >00 .0 .0000. 00 \a 00 I. 00 \q 0... 00.0.0008 00.00008 00 ..I 00‘. 00. ..I 000.. 00.0000. 00... .NN 0.3. 0 2 0. .088. 0.0.. :8. .82.. 00:00. 0.0.. 03000.00 0305.0: 0... 00.. .00.00 .0..) 0... ..N 03.5 0600-0000:. 0.000... .02: 000300003 «603000004. 0.302 JflGOE 0.00.0-0.3.3 00.0-500000 0.0002 0.003 «00.5 0.80500 126 70000. . m 0.30... 0.600 0... .0 ..0 ..00.0\ 0.00.0 I 0.00.0 I 00.0.0.. .0...0000. I 00.0.0.. .0...0000. I 3.0.0.. .0...0000. I ..000 I ..000\... ..000 I .00.00 I .00...0.\... .008... I E00030: I 0.0000000\.. be...§.00\. 0.00.0. I 0.00.0..Vfl 0.00.0. I 0.000.... .005000000 00. 0. 0.0000620 I 50.00.00.0V4 a.0.000.0.0\a .000. .000..00000 .003 .< . 0000.000 00.0050 ..003\.. 00.0.0.0 ..00.0V.. ..00.0 I 00000000 I 00..0..000N4 00000000 I M0300. 00.0.000 I 00.000. 00.0.000Vq 00.000. 00.0.000 I 00:00.00 0.0.00.0 00000.00 0.0. .0 00:00.00 .00 00.0 \4 ..o \q . \4 0000.00.00 000 0.200000 0.00.00. 0000.00.00 000 0.0.8000 0.00.00. 2.00.0600 00.. 0.200000 0.00.00. .0000... 0... .0 0000.00 0.00.00 0>_0>0.\.. 0.00.00 0.0000.\. 0.00.00 009.0. I .0000 00. 0. .003 . ".008 I 200.0 I 000.0 I 00000 .0.000..\., 00000 .2000» I 00000 .2000”. ..00 000 000000. I 0.00 000 000000‘0 0.00 000 000000. I .0503 0. .000 .. ..0.00..0>0 .50 .0000. 00. 00 I 00 I 00 I 0...: 00.000000 0.0.0.00. 00..\.. 00: 00>\.4 .00.00 0003.00 00. 0.< . .00.000000< 00002 000.0..— .00500000< .0002 80.000... "Am—DOE 9.38.30. 00002 .000..000> 0000..— 127 ..0000. . m 0.7.0... 0.00.0 I 0.00.00 .0. 00.00.00 .0000. 00000 .0 .0..\... 0.0.... 000........0 .0 .0.. I 00000 000....0..0 .0 .0.. I 00.000.000.00 00.. 000.0 00.0.0... ..0 0.00.0 0.08.0 .0. 00.00. .0000. 00000 .0 .0.. I 0.00.. 000....0..0 .0 .0.. I 00000 000....0..0 .0 .0.. I 0.00.0 I 0.00.0.0 .0. 00.00.00 .0000. 00000 .0 .0.. I 0.0.... 000.05.... .0 .0.. I _ 00000 000.03.... .0 .0..\.. 20050000.... 00. .0 0.00.00 .00.. 00. 0... .003 .0. 300.0 I 0.00..0>0 .000..0 0.00 0.000.\a. 0.0.0.0.... 000.0000. ....000. 000000\0 000.0000. ....000.\.. 0.0000. 00.0.000 0.3.00.000 0.000. I 0.000... .0 300.0. 0..0..0 000.>0.\.I 00..0..0 .0 30.>0.\.. 0.0.0.0 .000000000 800.0 I 0.00.030 .000..0 0.00 0.000. 0.0.0.00 000.0000. ....000. I 000000 000.0000. ....000. I 0.8.0.0 00.0.000 000.00.000 0.000. I 0.0000. .0 30.00 I 0..0..0 00030.0 ..0..0..0 .0 300.0. I 0.0.0.0 100000000 0.00.0 I 0.0.0.030 .00000 0.00 0.000. I 0.0.0.0.... 000.0000. ....000. I 00000.. . 000.0000. ....000. I 0.0..00. 00.0.000 0.0..00.000 0.000. I 0.0000. .0 300.0. I 008.0 0000.0... 0.0.8.0 .0 300.0. I 0.0.0.0 .000..00000 000000.000 0.0 003.200.0000 000 00000 0. 000 00000 0. 000 00000 0. .0 .0000... 00.00.00 00000.0 .0 00..0000..0.\.. 00000.0 .0 000000030‘ 00000.0 .0 000000000. I 500.009.00.20 .003 .0 .00300000< 00002 000.0... .00—00000: .0002 00.000: "AMA—OZ 0.003.03— 00002 .000..000> 0000.... , 128 ..0000. .m 0.000 00000 0... .0 =0 ..0...0 ..00.0 I ".00... I . 00..0..00 I 00..0..000 I «000000.. I 0500.0 0.00.0.0 0.00.0.0 .0. .000... 00000 .00.00 I .0. .000... 00000 .00.00 I .0. .000... 00000 .00.00 I. $00.50 .3.0050.00 00000 00.0.2.0 I 00000 00....0..0.\.. 00000 00....0..0\.. 0.00.00 0.00.00. 00000 .0 00:00:00 I 00000 .0 00..00..0>\.. 00000 .0 00:00:: I 0.0. 0500.00.00 00000 00000 00000 .0 00..0...0... 000...... 0000 850.000 0. 0.0 I 0000 0.0.0.000 0. 0.0 I 0000 050.000 0. 0.0.\... 0... 00 .005000000 00:05.30. .0 000.000 I 00:05.30. .0 000.000 V4 Ccuacsbchga .6 c00u..0¢\| 0... .0 0.00000 .0..) 0. ... 3.005500 0.0..3 ".0:... I ”.0....0 I 0. 00..0...5. ..0...0\4 02.000.00.000 00.000.00.000 00..0..0.00.0..0 020980500 00.00.3500 00.00.3500 3.005500 0. u0.0.0000\ 3.005500 0. 00.0.0000\.. 3.005500 0. 5.0.0000 I ..0000..0 000.000 350000.\. 350000. I 350000. I 00.0.5.0.0. 0... 0.0 30.. ... 0.600 0... .0 =0 ..0...0\q 0.00.0 I 0.00.0 I 00.0.. 00.0.. 00.0.. .000...00 0. 8.0000 I .000...00 0. 0.00000 I .000...00 0. 2.00.8 I 3.005500 .2000» I 3.005500 .0.0000\ 3.005500 .0.000..\« 0.00.00. I 200.0% 0.00.00. I 0.00.00 I 0.00. \q 0.00.00 I .0000: 055.300 0.300000 I 0.0.00000\.| 0.200000 I 0.0000 00005.80. 00..0..0.0.500 I 02.5.0.0.500\.. 00..0..0.0.500 I .0 000.000 0... 0.0 2.3 .0 000500000< 00002 000.0...— .c0500000< .0002 :00:—0..— “ANA—OE 0.000—.00.— 00002 .000..000> 0000.... 129 ..wcoov . m wanna. 2002: 205000000 0.000: 0.: .0 0:00: 3 032.03 0.3.8.2: 5 SN 0: .l 0: ..I 90:53 0022335500: 00 I. 00>\.. 00a\.. 00:_E.:cu\=50 05 0.2 .: 520.50.55.00 . 50v :5 30500. $050000 .0: 2050\4 .25.: 0.50.300 2050\0 ..o 00.5. «.00. 3050 Na 3.5.858 £55.68 £55.58 0.: .0 80:50:. 0.: .0 0:09:06 0.: .0 30:80.: 03.550050.- .0 03.550030.— .0 03.550030: .0 actfincs 003.5500 0 I. u::0_0:00 005883 “\a. u:=0_0:00 0055800 0 I... 3.36.33 3.05—0.60 3.050.800 2:05.300: 0:. 0:0 :05 0:2: .| 3.05208 0:0 ..I 0:0 :05 0.5.: ..I 3.05.380 0:0 1| 0:0 :05 0.5.: ..I 3.05.358 0:0.\—. :5 3:00 2 $03.50: 0.8 00055800 :0 =20 :23 d— 23 03.0030 2050\q 2050 .I £0.20 I. 002553 :1 00.55.50 I. 0025500 I 08> 3.3.0:. I. 20> 310:2: I. 06> 31022: I. 5.3:»??? 532.3563 5:83.580.— ..0000: .00. :_ 0.3 3 00.2.0 ESE..- l. a: 030:0 b02300 ..I .3 00.9.0 30.5300 ..0000: 00300.09. :02? 3:33.03 8:090:30 8:093.» 9:200: .5. 00.00590 9860 n:a:00:3\q ace—:0 03:00:00Nq m:0Ea 30:00:00 .I 0.5 0:052: .23 .m «60300000< 0.0002 30.5 000800000< 7-002 Uta—Ear— "Am—DOE 0.58.55— 0—0002 1:30:00.» 0:00....— 130 70.000. .m wanna 8 .II a all 2 ..I. 000 In EN. 000 I. .0:—0:0: 00... 000 00s .— 6058? .0:. ..:08000000 0: I. 0: 0: 0:. 0. 8000000: 00» I 000” 000V]... 0.:0._00:00 00.0.00 00¢ .0. 0050000 .0: "00:.0Vq "000.0 .I 0:0: "0008\0. 003080.00 .8050 ..I 03.55.00 .2050 I 0020.00.00 1030M In 3008000000 0.8.8.0383355 l 0.8.3.0880??? I. 0.802.088.0055... l 2.. 0:008 3 08.3.2 3000.300) I. 300:0..003 .l 3.00:0..00) t... 0. 08800. .0800. .0:? .m— 00...00:0 .0: 005.00.. .00 00.00300 .0050\.. 1.0.00... "00:.0N .0050 I A “.80. A “.80. 0.0.3.» ".80. 0.0.00.0... 00 .3 ..I 08.. 00:000. 20.00.08 00 .0. I. 08.. 00:000. 0.0.00.0... 00 .0. 08.. 02.000 00 00.00.08.800 .0 00 00.00.08.800 .0 00 00.00.08.800 .0 5:08 00.. 000 0:0 00.6 .| 5:08 00.. 000 0:0 0000 .| 5:08 00.. >00 0:0 0000 .I 08.. 00:000. 08.. 00:000. 08.. 00:000. A 00 00.00.08.800 .0 00 00.00.08.800 .0 00 00.00.08.800 .0 5:08 00.. 000 0:0 000:: ..I 5:08 00: 000 0:0 000:0 I. 5:08 00.. 000 0:0 .000: ..I 80:000. 0.0.2300 .l 0000000. 0.0.0300 I. 002.000. 0.0.3300 ..I 05...... l 00:02. .I 0:50.. I. 0.0.00.2: 0300.08 0300.08 ..808000000 0... :. 0500003582: .l acraooxmczctn ll 05300.0:320: ..I 00308. 000 0.000 .0:? .N. .90'00000< 00002 000.060— .000.300: .0002 00.5.0: «am—DO! 0.000-.00.— 00002 100300000 00000.— 131 ..0000. .m 0.000 0: I 0: I 0: I 000\ 000\ 00»\..« 0000.38.00 0000: 00¢ .c. 05:08 0... 000.0000 .0: .0050 .0050 I. 0:0 000» 0:0 .0050N ..0008 0000 0 :05 000 I 0000 0 :05 000. I 000» 0 :05 000. I . 0000 5.00 .:08000000 05:08 0.0 :05 000. I . 05:08 0.0 :05 000. I 05:08 0.0 :05 000. I 0000: .00.00. 0 0.0.0800 :00... 0 :05 000. I 300.: 0 :05 000.\.. :00? 0 :05 000. I 0. 0:0. .. 0000 0:0. )0: .2 .0050 I .0050 I .0050 I 0: \q 0: V0 0: I .0.:0800.0:. b00030: 000I 000I 00>\a 2.00800000000— .0. 00.00300 000:808 5.:08800 000:.0 0:0 000:0 "0050\4 .0050 I 00. 0000—0000 "0050\0 2.25.8. 20.00 I 2.35.8. 20.90 I 2.0.28. 20.00 .I 030.0008. I 0000.00.08. 030.0008. I 2.080.800: I 2.0::0..000_mflfl 2.082.000: I 558000000 0:. .0 000000 I 000000\.. 000000 I 0.:080..0:. 0:. 000 .0:? .m— 000I00000< 00002 .00.00.— .fl0500000< .000! I550..— «EGO! 0.000-.009— 00002 .0.-0.0000.» 00000.— 132 ..ucoo. . m 0.30... 000:0 0:. .0 ..0 .0050 .0059. I .0050 I 0.:0800000 0.:08000000 0.:08000000 0.0.0.00. 0.00: I 000.0. 00. 0.00:\0 0.0.0. 00. 0.00: V0 000508 .0 :0..00.00 I 000508 .0 :0..00.00\.4 000:.08 .0 :0..00.00\.. 08000000 08000000 0800.000 05802.0 .0 8500.00 I 005800.... .0 :0..00.00\.n 03:800.... .0 8500.00 I :0..00.000 I :0..0:.000 :0..00.0>0\fi 0.000.0000 000 0000000 0:0..000..0 3.0.80.0..50 I 0:0..000..0 0.“.00.\u..0.0w 80:82.0 3:803:08 Va. 88.500000 0:. .0 0000.0 :..0 00: I :..0 00: 0:: 00: \fl 0050 0:0 808000000 003.0030 .0 08.80 I 00.50030 .0 0:...00mflq 00.50030 .0 0:...00 Va. 0000: 0:. 50.0.0: 0.000 .0 8500 I 0.000 .0 8500\14 0.000 .0 8500\0 00.:0:0..0.00008. 8:3 .8 26:0 0:. .0 ..0 .0050 \a .0050 I .0050 I 000000. .0000000 .000000. _ 80800880 .50» 0.0. 8088.080 .30.. 0.0. 8088.080 .300. 0.0. 08080000 00.0.0. I 08080000 00.0.00\.. 08080000 00.0.0.. I 0:0..08008 I 80.8.8008 N. 80503030 I 0.00000 00.0.0. I 0.00000 00.0.0 \0 00.0000 00.0.0. I 000000 .00. 000000 .00. 000000 .00. _ 008000.00 :0.00..00 I 0.00. 008000850 I 008000.00 8.08% 0.00. 000.0808 \0 008000.00 :0.00..00 I 0.00. 000.0000:0.0 I .008. :0.:.0 .00.. .— .8008 0: I 0: I 0: \4 .00.0.080 5.000... 00? 00g 00» I 00.00.—0.. 0000: 0..< .0. .60800000< 00002 «90.0.0.— .00800000< .0002 .0:—8.0... “Am—DOE 0.008.05— 00002 1:0..000> 0:000.— _ 133 73000. .m manna 3.0.0.800 .0: 000.6... 0. 000.0 000.0. 00.. 000 00.0000. I 8.0000. 10.01 8.000.... I 0..0008 05 .0 000800000 03.8....» I 508....» .91 808....» 18.. .2... 0.0.3.0280 .00 0.0.0.0000 0.. .0 80 00 .0008 00 00 I 00 05 8.0000000 00.000000 00\.I 00..\.. 00%.... 000.. ..0000000 00.. .«N 00.0“ 0085 00.0.0. 008... .00.00. 0.0.. 000.. .0008 0... .8602... 00.8.... c. 00.30.... 2.. .8602... 2.. 2... .5...» .0..) a... ..0 000800000< 00002 000.000— 80800000< .000... 00.800..— "AHA—GE 0.008.35— 00002 .0:—«08> 06000.— 134 Research Question Results In this section each of the eight research questions is answered in terms of the data collected for that purpose. Question #1 Who were the significant individuals and what were the significant events and/or landmark applications that figure into the evolution of Alpha and Beta type needs assessment models? This question was dealt with in detail in Chapter II under the heading ”Some Early Steps in the Development of Alpha and Beta Needs Assessment Models". Table 4 presents individuals, events, and landmark applications, their date, and how they figured into the development of Alpha and Beta type needs assessment. In earlier sections was described the development of the models and early applications of importance. Of particular significancemtfioit-he understanding of-current models is inclusion of some explanation of two early applications. These two from Temple City and Newport- Mesa, the deductive and inductive approaches, were forerunners to what might be considered the two basic approaches to current practices in needs assessment. The inductive approach represented by the assessment conducted in the Newport-Mesa Unified School 135 District (Shuck, 1968) began by deriving goals, objectives, and outcomes from the sub-communities in the district. The Temple City Unified School District, on the other hand, used an approach through which the educational program is deduced from existing goals and outcome statements. The general methods of the inductive approach can be likened to the general steps that should be taken when performing a needs assessment that might be classified "Alpha' (Kaufman, 1979). The main purpose of such an assessment is to gather data from the environment external to the educational system, from which the purposes, goals and objectives are to be derived. The deductive approach, like those types in Kaufman's taxonomy (1979) subsequent to the Alpha, begin with pre-stated goals and objectives as givens. Such assessments, because of the assumptions made concerning pre-established goals, may be seriously lacking in external validity and utility. Most extant models are based on the “classical“ method of assessing educational needs (Kaufman, 1972; Southard, 1974) and are patterned after the early model developed by Kaufman, Corrigan, and Johnson (1969) (Trimby, 1979). Their major contribution was that the end goal of education should be the learner's contribution and survival upon exiting the educational system. Maslow (1968), Frankl (1962, 1965, 1967, 1969), Rogers (1964), and Ruckers (1969) contributed work that 136 put emphasis on the “humanistic“ requirements necessary to survival and contribution. From the Hannah Formulation (1966) came the logical entry point into an assessment. This would be through the dimension “nature of society“. Stufflebeam's CIPP model (1968) pointed out the inextricable link between planning and evaluation. He further emphasized the necessity of including the determination of context and inputs to a program before beginning. Sweigert contributed the idea of education "partners” (1969, 1971). Rucker (1969) added a concern for the individual's uniqueness and jpotential, beyond minimum requirements. Trimby (1979) called attention to the relationship of needs assessment to evaluation and Kaufman suggested a taxonomy by which extant needs assessment models might be classified (1979). Table 4 summarizes these early applications and other significant individuals and' events that figure into the development of the current state of the art of Alpha ad Beta type needs assessment. 137 w individual, Event { mte Significance Application Accountability l960's Hindreds of nodels, kits, tools and procedures developed Hannah 1966 Three foci of curriculun upon which Formllation rmdel was based CIPP wodel of 1968 Planning and evaluation inextricably Stufflebeanr related Mesa-Newport Needs 1968 inductive approach to needs Assesmnt assesmnt Tarple City Needs 1968 Deductive approach to needs Assessment assessn'ent Kauinan, Corrigan 1969 "Classical” nodel upon which and Johnson npst extant nodels are based Sweigert Model 1969 Need for ”partners” RuckerA 1969 Concern for individual's uniqueness and potential Frankl 1962 Work 1an which humnistic Rogers 196'! requirements for nodels were Nhslow 1968 derived Rucker 1969 ' Kauinan 1972 Placed discrepancy rrodel in context of systematic educational planning Kaulnan 1979 Taxoncmy of needs assessment nodels Trinby 1979 Relationship of needs assesmnt to evaluation A Table ll individuals, Events and Landmrk Applications 138 Question {2 What are the names and description of established Alpha and Beta type needs assessment models? This question's answer is found in two sections of this work; the first is the matrix presented in Table 3 of this chapter and the second is Appendix A. Twenty-three questions are used on the vertical axis of the matrix on Table 3 and all of the models included in this study are listed on the horizontal axis. The data gathered in answering these matrix questions lends to answering research question number two in the following three categories: 1) those that describe the processes used, the purposes, and outputs associated with the models (matrix questions 2, 3, 4, 5, 6, 7, 8, 9, 10, ll, 17, 18, 19, and 20), 2) those that relate the availability of the resources for the accomplishment of a needs assessments (matrix questions 1, 12, 14, 15, 16), and 3) those that deal with the models' ability to produce the information for which they are designed (matrix questions 19, 21, 22, 23). In Appendix A is offered a narrative and, where available, a graphic presentation of each model. The questions on the matrix were designed chiefly to elicit information from model developer/marketers. The following questions, either because they were more readily presented in narrative rather than matrix form 139 or because they deal with subject matter assumed unique to this study and unfamiliar to many of the developer/marketers, are answered in Appendix A: 1) 2) 3) 4) 5) Into what category on Kaufman's taxonomy does the model fall? What are the particular strengths/weaknesses of the model when viewed from the standpoint of that taxonomy? What are the general components and processes connected with each model? Where, why, and by whom were the models developed? What source might a prospective user contact for further information on a particular model? When taken together, the matrix and the appendix are designed to provide a description of each model sufficient to answer research question number two. 140 Question #3 What types of information could ideally proceed from the application of Alpha and Beta type needs assessment models? This question has been dealt with extensively in Chapters I and II. The "Problem" and ”Purpose” sections of Chapter I contain particularly relevant material. Chapter II addresses the use of information derived from such assessments as it relates to evaluation, other instructional design and development considerations, and Education as a whole. The following is a synopsis of those sections. It has been suggested that the information proceeding from the Alpha and Beta type needs assessment processes can be used to: 1) help learners measure their own success 2) help educators plan on the basis of ends rather than means 3) help instructors gain more results from efforts 4) improve the image of the educational profession (Chapter I, p. 8). Needs assessment can further provide information that identifies gaps between the ideal and the real, suggests staffing patterns and resource allocations, provides accountability, improves organizational direction finding, assesses the validity of means used 141 and suggests appropriate timing for changing educational goals (Kaufman, 1979) (Chapter I, pp. 8 and 9). These types of assessments have also been credited with providing a "clear cut approach to the determination of needs" including data useful for both formative and summative evaluation (Heldge and Marrs, 1978), a process "by which the unfulfilled educational requirements of a population of students are identified” (Lee, 1973), a process that allows for the coordination and articulation of curricular programs for kindergarten through the twelfth grade (Buhl, 1978). ‘Needs assessment provides for a positive involvement of those other than the professional educator (English, 1977) (Chapter I, p. 9). Kaplan, in the 1974 handbook Needs Assessment in Education, suggested that a needs assessment can: 1) provide necessary community pressure calling for citizens to express their views, leading to a systematic tapping of- these—viewsr-and providing for community involvement. Such information can prove helpful for not only gathering data to set educational objectives and goals, but also to gain information that may be beneficial if a school is suffering from such ailments as school budget defeats. 2) help pin-point problems precisely. 3) show how real problems were not being attacked, if for instance, a new program isn't working. 4) provide data which can lead to well defined, verifiable educational goals. 142 5) clear a smoother path toward goal achievement. 6) provide a data base for future educational decisions. 7) establish a resource bank for information of a district. 8) generally, provide a means for: a) reducing internal and external problems, b) developing an organizational structure and process for continued self- evaluation, and c) a basis for future planning and problem solving. Question #4 Are there major variations among the Alpha type needs assessment models? A concerted attempt was made to group together all the models of the Alpha type on the matrix for comparison's sake. If an Alpha model is located among others of the Beta classification, it is because the physical set-up of the matrix would allow for only three models on a page. To introduce—the- variations among—the ’models‘fand by way of review, this discussion will begin with the characteristics these models have in common that lead to their classification as Alpha type needs assessment models. Each of the models either attempted, or suggested, deriving learner needs by gathering data that reflects the demands a learner would face upon exiting the educational system. ‘This was accomplished, partially, by involving the community external to the 143 school in the assessment (matrix question 6). Though not specified in literature relating to the Atlanta project, and done by way of invitation to the community in Fresno, the VENAP and Kaufman models rely upon a random selection of participants based upon community demographic characteristics (matrix question 7). All Alpha assessments, to varying degrees, sought the input‘of the various participants in carrying out functions of the assessment beyond that of being merely sources of information (matrix question 8). It should be noted at this point that Kaufman's appears a much more complete and generic model than the others. Several of the matrix entries are "all of the above” under the ”other” category. This is not necessarily a short coming on the part of the others, but may be a result of the fact that all, expect Kaufman's, were project specific models. Because of purpose, all of the possible applications of any given model may not bese.reporte&.--'~For example when asked at what educational level the assessment is aimed, the Atlanta project lists only “secondary“ (matrix question 3). The model may be appropriate to a broader application but was not applied in that way. On the other hand, Kaufman's model is not to be considered lacking because several of the questions were recorded as ”other: not specified” (matrix questions 12,13, 17, 23). Kaufman dealt with each of these areas but in a 144 much less prescriptive way than. many of these other models. The others could not avoid prescription, to a certain degree, because they were dealing with specified settings and applications. Along the same line, Kaufman, partly because of the lack of desire to become involved with the production and marketing aspect, does not offer certain services and materials which are, because they sprang from application's demands, a part of, for example, Atlanta's, Fresno's, and VENAP's (matrix questions 1, 12, 14, 15, 16). It should be noted again, that unless they result from a specific application, pre-specified materials may inhibit the gathering of appropriate data. When looking over the answer to those questions that deal with the validation of needs and the models' ability to produce appropriate information (matrix questions 9, 19, 21, 22, 23), a couple of distinctions among the Alpha models surface. The validation of needs is accomplished partially through objective means in, for example, the .Atlanta, VENAP and Illinois ‘Problem Index models. Fresno's is decided upon by the admin- istration's arbitrary choice. Finally, only the Atlanta and VENAP models report any form of research conducted in connection with them. The results of that research is dealt with under "Question 8'. 145 Question 5 Are there major variations among Beta type needs assessment models? As in the previous section concerning Alpha type models, illustrative examples citing specific models will be used to point out the variations among Beta needs assessment models. There were many more similarities among the Beta type models than distinctions. Even the differences are only of degree. That is, almost all of the models contained all the components and addressed all the concerns appropriate to the Beta process (Kaufman, 1979). But some considerations were more well represented in some models than others. The following list is an accounting of the more notable distinctions. Immediately after, is a more detailed account of the comparisons among the Beta type models. -resources and services offered with the models -purpose of the individual models -education levels at which the models are aimed -pre-assessment considerations -extent of participant involvement -cost of the assessment -time needed for completion -objective validation of needs 146 -field testing -connected research -industrial applications The Beta models that appear to be the most complete in the considerations with which the matrix questions are concerned are the Ohio Department of Educational Needs Assessments Guidelines (ODENAG), the Phi Delta Kappa (PDK), and A Comprehensive Needs Assessment Module (ACNAM). The resources and services offered with the models (matrix questions 1, 12, 14, 15, 16) are pretty much the same, with the main tools for carrying out the assessments being surveys and questionnaires. The PDK model offers a needs prioritizing game (matrix question 15) and the surveys for the ACNAM and Consolidated Application's Needs Assessment Guide (CANAG) are offered in bi- or multi-lingual versions (this is reported in -more detail in Appendix A). ‘ The purpose of most of the models (matrix question 2) is generally the determination of discrepancies between current outputs and desired goals. Two exceptions were Coffing's, whose purpose was decision making, and the Quality Education Program Study (QEPS) which was carried out for the purpose of “goals definition“. 147 The levels at which the assessments are aimed (matrix question 3) are generally elementary, middle, and secondary education. The Mott Foundation Community College Model, the Battelle Survey, and the Institutional Goals Inventory (161) are aimed at higher education. The Coffing and Harless model's chief use is in industry. Though all models deal with pre-assessment considerations (matrix question 4), Coffing's model and the Illinois Problem Index emphasize them most heavily. Most models used participants other than professional educators and administrators (i.e. learners, parents, conununity members) merely as sources of data for determining needs. Exceptions to this were the Illinois Problem Index (IPI), ODENAG, QEPS, ACNAM, and Lee's model which called for these “partners“ (Sweigert, 1969, 1971) to engage in a variety of processes beyond being information sources. The cost of the assessment (matrix question 12) ranged from that incurred by such expenses as copying materials, time and acquiring substitute teachers, to all of these plus paid consultants. Only the CANAG, ACNAM, Mott and Battelle models suggested the requirement of outside consultation. The time for the completion of the processes ranged from less than a year for most models to one year and six months. 148 The ODENAG, PDK and CANAG models rely heavily upon objective means to validate perceived needs. Most others decide needs by participant consensus. Field testing (matrix question 21) ranged from once for Sensing Educational Needs in the Far West Region (SENFWR) to extensive application throughout the U.S. for the PDK model. It might be of interest to note that the purpose of the SENFWR assessment was to increase that organization's responsiveness to that region's needs. Research (matrix question 22) was reported in connection with only four models,. the ACNAM, PDK, Atlanta Assessment Project (AAP) and Lee's model. The topics of the studies were very limited and 'will be dealt with at the end of this chapter under "Question #8”. Industrial application (matrix question 23) is the main purpose of the Harless and Coffing models. In many of the questions on the matrix the “other” category is checked and followed by ”not specified” or "determined by user's purpose" under, for instance the Coffing model. The reason for this 18' that the processes of these are very open to modification according to each particular application. Options and suggestions relating to these matrix questions are often present in materials related to the models but little is done in the way of prescription. 149 Question #6 What characteristics could be included in idealized Alpha and Beta type needs assessment models? The characteristics of the Alpha and Beta assessment processes presented in this section are not unique to the researcher but are a synthesis of the best components of extant models of the two varieties. For comparison's sake, the two processes might be portrayed in this manner: Components Alpha Beta 1) Pre-Assessment Coonsiderations x x 2) Determination of Goals x 3) Set Desired Output Levels x x 4) Measure Current Outcomes x x 5) Determine Discrepancies x x 6) Prioritize Gaps x x 7) Select Gaps for Closure x x 8) Continue to Monitor‘SyStemw‘-."Wm'x"~”" “a" Figure 4 A Comparison of the Alpha and Beta Processes As has been reported in earlier sections of this study, the major distinction between those models classified Alpha and those Beta is the starting assumptions upon which the assessment is conducted 150 (Kaufman, 1979). Both models may begin with a referent set of goals. These may be in the form of the philosOphies, goals and objectives under which the education system is «currently’ operating (see Illinois Problem Index or Kaufman's model in Appendix A), a set of goals offered by an organization (see Worldwide model, Appendix A), or a set of goals developed by a State (or other) Board of Education (see Quality Educational Program Study in Appendix A). The distinction between the Alpha and Beta approaches to an assessment, is the assumptions made about these goals and the processes dictated by these assumptions. The Alpha model will start the assessment at the point of goal determination, go on to compare these goals (determined as independently as possible from the influence of pre-established set of goals), note the discrepancies between the two sets of goals and label these discrepancies ”needs“. The Beta process, on the other hand, begins with the assumption that the set of pre-determined goals is valid and useful, compares the systems current outputs to these and labels ensuing discrepancies as “needs". It should be noted, that the Beta type assessment is not looked upon as inferior in any way to the Alpha. If a set of goals has been established in such a way that the assumption can be made as to their validity and utility, an Alpha assessment would be superfluous. If a 151 set of goals is dictated from a higher authority, an Alpha assessment may be out of the question. The conditions under which an assessment is conducted, not the mode in question, determines whether or not one of these modes is appropriate or inappropriate (Kaufman and English, 1978). To illustrate the two approaches, it could prove helpful to compare two of the models included in Appendix A. The Fresno model is a prime example of the Alpha variety and the Phi Delta Kappa of the Beta (Appendix A). It will be necessary to look only at the graphic presentations of the two models to determine that a marked difference exists in the approaches followed by the two. The Fresno model begins with the determination of goals and goes on to determine needs by comparing the discrepancies between these newly derived goals and the system's current outputs. The Phi Delta Kappa model, on the other hand, begins .with a setsof goals developed by that organization and derives needs by noting the discrepancies between these goals and current outputs. Useful light can be shed on the understanding of the Alpha and Beta approaches by looking also at the Quality in Educational Program Study undertaken in Pennsylvania schools (see Appendix A). A set of goals was determined by the Pennsylvania Board of Education. It was the function of the ensuing needs assessment to 152 determine the validity of these needs (goals) as well as the discrepancies that existed between current outcomes and these established goals. Depending upon the methodology used (Kaufman and English, 1979), the validation of these goals might be considered an Alpha assessment. If an external assessment is conducted independently of these goals and the results compared to the goals, an Alpha assessment would likely have taken place. If the goals themselves were merely rated, for instance, on a scale of l to S that supposedly reflected their importance, then the assessment might be a Beta type. This is especially true if data is not gathered that might indicate needs that are not included in the original set of goals. Although the gathering of information and the participation of those considered non-professional educators has not been used as criteria for the classification of models as either Alpha or Beta, it might help to clarify the term "external' at this point by looking at the matrix (Table 3). Both the Fresno and Phi Delta Kappa models enlist the cooperation of individuals representative of the community outside the school's “educators', gathering information that, hopefully, reflects the viewpoints and conditions of that external community (matrix question 6). Yet the Fresno model is labeled Alpha, or external, and the Phi Delta Kappa a Beta, or “internal“. The distinction is 153 that in the Fresno model, the goals are first determined externally and then compared to internal conditions. The PDK model's goals are accepted without this external assessment. The PDK then uses external involvement to determine ”needs”. It is important to emphasize that to qualify as an Alpha type, a process must conduct an external assessment. External involvement is optional to the Beta approach (see matrix question 6 under the Institutional Goals Inventory and Coffing's model). The Unified Needs Assessment Model The information gathered and presented in this study seems to indicate that some differences between the Alpha and Beta modes of needs assessment do exist. However, there is reason to believe that these differences can be dealt with effectively within a single model. This section presents a synthesis across these models examined here. A schemata of this model is presented on the following two pages. In the narrative explanation that follows, this schemata will be referenced where helpful. Construction of the Model When constructing a model, and in particularly a model or models picturing the Alpha and Beta type needs 154 assessment processes, at least two main aspects of modeling should be taken into consideration. First, the components comprising the assessment processes must be chosen and described completely enough to be of benefit to the reader. Second, those components must be presented in such a way as to appropriately describe the interrelationships among the individual components to each other and to the process as a whole. It might prove helpful at this point to review some of the ideas presented in Chapter II in the section ”Models, the Systems Approach and Alpha and Beta a Needs Assessments“ and apply those ideas to the development of a specified model. When choosing the components used to represent a process it should be kept in mind that the choice is an arbitrary one from among an almost infinite variety of possibilities (Churchman, 1968). Which components to eliminate is as important a consideration as which to include. The inclusion of a characteristic, activity, or strategy, or too much detail may rob a needs assessment model of flexibility, rendering it inappropriate for broad application (Silverman, no date). Conversely, the inclusion of even desirable components, if not properly emphasized, may result in the exclusion of the component's beneficial results in actual practice. Because, by definition, the processes pictured in 155 an Alpha model should include the participation of representatives from among educational administrators, professional staff, teachers, parents, learners and the community at large (Kaufman, 1979: Kaufman and English, 1979; Witkin, 1975), and by practice the majority of the Beta models opt to do so, the model should be designed and presented so that it can be understood with reasonable ease by this broad audience (Kaufman, 1979: Alexander and Yelon, 1969). From the discussions in the literature, it can be determined that the needs assessment process, which can serve as the foundation for all further levels of assessment and educational planning (Alkin and Fitz- Gibbon, 1975: Kaufman, 1979: Witkin, 1977), is usually broken down into four major components (Witkin, 1977): 1) statement of goals, or desired states, which are ranked in order of importance, 2) procedures for ascertaining the present status of those goals or states, 3) methods for identifying and describing discrepancies between the goals and the present status, and 4) methods for assigning priorities to the discrepancies found in step 3. These components might be modified and expanded into the following eight, if a generic model presenting both the components of the Alpha and Beta processes is to be developed: 156 l) Pre-assessment considerations. 2) The determination of the system's desired goals. 3) Setting minimum output levels. 4) The measurement of the educational system's current production of those outputs derived from the desired goals. 5) The determination of the gaps between the goals and the current outputs. 6) The assignment of priorities to those gaps for closure. 7) The selection of solutions for closure. 8) Continual monitoring of the system. By its nature, the Alpha type needs assessment should be such an integral part of those activities herein labeled "Continual Monitoring of System", that a useful assessment of this type, could likely not be conducted without relating its processes to the product that it is designed to produce (Lee, 1973: Kaufman and English, 1979). This type of assessment is a front end analysis procedure that should be periodically repeated. When and how often this is necessary can be decided by a continual monitoring of the educational system and its external environment. As society's demands change so will the 'needs' of the educational system. Monitoring should provide the necessary information that leads to the modification of the system's programs. Each of the eight steps incorporated into the Unified Model, are graphically portrayed and presented 157 narratively in order below. Noted are: their derivation from extant models, their place in the overall assessment process, and the options that a user might consider for specific application. Pre-Assessment Considerations The configuration in the model (see Table 3) labeled 'PRE-ASSESSMENT CONSIDERATIONS" represents concerns and decisions common to both the extant Alpha and Beta modes of assessment. The answers derived at this stage of the process determine whether the assessment will be of the Alpha or Beta variety. The decisions made and planning done prior to a needs assessment have a good deal of influence on the final outputs of the process (Witkin, 1975: Coffing, 1975: Kaufman and English, 1979). Most current models, as can be seen in the matrix (question 4), offer little in the way of suggestions in this area. Three notable exceptions to this are models of Coffing, Kaufman, and the Illinois Problem Index. The information gathered during the pre-assessment phase of the needs assessment will not only lend assurance that the approach adopted and applied will be suitable to the educational system and its community, but that the data gathered will be useful in meeting the purpose for which the assessment was undertaken. It stands to reason that 158 mm a: .w m l . . . m m i. H a... .2. H ." J1 .. .......................... A" h. 1 m a e z. a ._ a , _ The Lhiiied Needs Assessment Model 159 --------d L J._-..__--..._.._......_-___-_-L._.-_-__-..--_.._- Unified Model (cont.) 160 a well defined purpose and direction will more likely produce a useful result. Monitoring the System Upon entering the system, the first process addressed by the model is the ”monitoring" of the system (1.0). At this point the monitoring may not be particularly active as far as needs assessment is concerned. That is, the input received may simply be from a source that has direct influence on the educational system, which could result in a needs assessment. It is possible that a member of the educational community (in or outside its professional educational staff) has come into contact with information that leads to the idea that the conduction of a form of needs assessment might be a viable option. This may result from an awareness of the possibilities of such assessments, being involved in related problem solving or decision marking or, simply because evidence seems to suggest the need for program evaluation of the type with which such assessments are concerned. This ”monitoring“ may be of a more active nature than that which proceeds many of the assessments evidenced by the models included in this study. It is more likely that a needs assessment is undertaken in order to demonstrate accountability to a higher 161 authority or for the sake of funding. Such was the case of the assessment involved with the original Alameda Country Needs Assessment Model (ACNAM). ACNAM's original applications were to gain funds for special projects under ESEA Title III. In other instances, organizations such as State Boards of Education, may establish goals and design assessments for the sake of stabilizing, improving and standardizing state-wide educational programs. It may not be dictated that local schools conduct such assessments, but, as in the case of the Quality Educational Program Study developed by the Pennsylvania Board of Education, goals were developed and a service offered to local schools. In the latter cases, ”monitoring” takes a less active form than in the first. The potential users of needs assessment do not seek but receive input. The question of the desirability of a needs assessment, represented in box 1.1, as a result of this input, arises. Decision points and activities 1.2 through 1.23 are a combination of the concerns compiled from extant models that are here categorized "Pre-assessment Considerations“. The physical set-up of the model depicts decision points and activities 1.2 through 1.16 as very interactive in nature. These steps are portrayed in a likely order of execution but, because of their closely 162 knit relationship, may not necessarily be considered or performed in this same order. It is likely, also, that many of these steps will not be completed simply because they have been dealt with initially. A decision made at one point along this path may make it necessary to evaluate and change others that have been made. For example, many of these steps may have to be considered before characteristics necessary to determining the surveying of similar projects are studied, many of the ”preceding" steps may be re-conducted because of new considerations raised by the input derived from these other projects. If some of the following descriptions of these processes are not prescriptive, it is partly because of this consideration. Setting the Parameters A possible next step in the pre-assessment process is to define the parameters of the needs assessment (1.2). Such assessments can be conducted from the level of an individual classroom to an assessment of needs that can guide the design of entire educational systems. It should be kept in mind that any educational setting does not stand in isolation, but, being a part of a larger system, will be affected by the learners prior experiences and will, in turn, create an effect on future learning. If a decision is reached to assess the 163 needs of learners as related to their ultimate “survival and contribution" in society (Kaufman, 1979) the more extensive the assessment, the more valid and useful. The learning of higher level skills taught in the high school, will be directly influenced by the foundations laid in the primary grades. Whether a full scale assessment of the general needs of learners is undertaken or a narrower one aimed at specific subject matter areas, an assessment is going to be limited by the capabilities of a system. These capabilities are reflected by such considerations as time, budget, available jpersonnel, extant information, and other resources such as the presence of persons trained in the assessment process and subject matter experts. There may exist a wealth of such individuals as trained interviewers and necessary area specialists in university centers. Because a needs assessment is not an end in itself, but is carried out as the prelude to further educational planning/decision. making, planning ‘must, take ‘place to help insure that the data gathered will be functional for these processes. The types of decisions to be made and those responsible for them should be considered (Coffing, 1975). Information can be gathered, tailored and directed toward those to whom it will be most pertinent. Input from these individuals during the planning stages may prove helpful also in gaining 164 acceptance of the needs assessment's results. It is usually good politics to keep those in positions of power involved and well informed. Before choosing the model and methods to be used, a careful study of the community from which the data will be drawn is necessary (Witkin, 1975). The demographic characteristics of the community (including the community at large, educators and learners) will enable planners to choose a model and design instruments and techniques suitable to that particular setting and audience. Users may have to consider educational levels, cultural questions and language when designing and administering the process. Some models, for instance the Alameda model, have developed multi-lingual instruments. The size of a community may influence decisions as to whom will be involved in an assessment and what methods will be employed. In a small community, it may be possible to survey a large percentage of the population, if not all. A larger community might have to be randomly surveyed so that a representative sample can reflect the views of the much larger group. Likewise, the methods used will be dependent upon such variables as size of the sample to be employed, time and demographic characteristics. A largely non- reading population may have to be interviewed in person or by telephone. A large sample might prohibit the use 165 of one-on-one or even group interviews, as would time. Whatever method chosen, materials. and 'methods can be designed that will give credible results (see SENFWR model, Appendix A). Who will participate and in what aspects of the assessment is a crucial question effecting the validity of the resultant data (Lee, 1973: Kaufman and English, 1979). If information is sought that reflects the views of the total community, an assessment should be designed that gathers input from all concerned. Educators alone are not necessarily in a position to know all of the demands that society will place upon a learner upon exit from the educational system (English, 1977). To insure the optimum value of the information gathered, it should be drawn from as widely a representative audience of the community as possible and validated by objective means. The extent of involvement is also an important issue. If non-educators are used only as sources to determine areas of need, a valuable resource may be lost when it comes to refining and prioritizing those needs (Kaufman and English, 1979) and budgeting (see Dallas Planning Model, Appendix A). Public Relations Public relations (1.4) should be kept in mind throughout the whole process. Whether it is the members 166 of the communities who will be surveyed or those of the educational system whose activities will be affected by the results of the assessment, proper handling of comunication as to such questions as the purpose for which the data will and will not be used, can prove helpful in alleviating such fears as the invasion of privacy or domain which such groups as educators may feel (English, 1977). In order to carry out such a process the cooperation and involvement of a very broad spectrum of individuals and groups will be required. How the needs assessment is introduced and the resulting comprehension of the assessment and its purposes may have a good deal of influence on how the process is received and acted upon by the administration, the educational staff and general public (Sarthory, 1977). It might be necessary to convince the general public, parents and learners that their input is not only desirable, but important to the determination of a truly useful curriculum. Keeping their involvement and interest at a level that will encourage involvement may call for substantial effort. Decision makers should be kept well informed and given any information pertinent to their involvement. Boards of education and teachers, like the rest of the public, may have to be convinced that the process is useful and worth their time. It is not uncomon for 167 educators to feel threatened by such an assessment (English, 1977). In a sense, a system's present outcomes, methods and goals are being evaluated. It may be only normal to assume that the purpose of an assessment is to judge a system "good" or 'bad". The positive outcomes of such an assessment should be stressed for such groups. The user's manual that accompanies the Illinois Problem Index, Establishing Educational Priorities Through the Illinois Problems Index, presents a good rationale for and schedule of a public relations program. It states that "The extent to which different constituencies in your district are involved in the IPI program may determine the extent to which the results of the program will be supported by the community." (p. 14). "The individuals who will respond to the survey instruments or who will receive the recommendations that result from the IPI process must be informed about the process from the beginning." (p. 19). The advice offered in the IPI users manual focuses upon three areas: 1) selection of the target group, 2) type of dissemination, and 3) public relations schedule. The issue of target group selection revolves around the decision as to whom information should be disseminated. Depending upon the assessment, the representative groups from which committee members are 168 selected may be included as well as those targeted for later surveying. Also, those who are to receive the results might receive information. These might be informed of the project's purpose, means of accomplishment, the end results and the desirability of their input. The type of dissemination concerns the best means for reaching the target groups. The means of contact will likely depend on the group to whom information is sent and the content. The general community might be informed of the purpose, methods, and anticipated date of completion through a local newspaper news release. A newsletter might be used for parents. A letter might also be used to inform potential respondents of the need for their cooperation in providing information. What must be born in mind is the ”nature of relationships already existing between the school and the groups targeted for publicity." (p. 21). The scheduling of public relations is an important matter. Early contact can encourage those concerned to feel more involved, cooperative and receptive. Early contact can also aid in identifying public relations problems before they become unmanageable. Of course, timing will be dependent upon the purpose of the release. 169 Determine System Capabilities The interrelationships among these first few steps becomes very evident when the system's capabilities (1.6) are enumerated. Here such things as time, money, available personnel and other probable resources should be noted. It may prove a little difficult to meaningfully carry-out this step without a frame of reference that suggests the possible demands of the assessment process. It might be helpful to use an available model for this. The model used at this stage may be beneficial only for this purpose but it can provide a starting point and a primer for those who are not familiar with many of the requirements of needs assessment. Decision Makers and Pertinent Data At each stage of the assessment, decision makers will be involved, whether it is those who make the decision to accept or reject the original plans for the assessment or those to whom the results of the assessment are given for program design and development. To be of greatest benefit, the information gathered should be tailored to those individuals responsible for planning and implementation decisions (1.8) (Coffing, 1977). This will involve not only the determinatimn of 170 those by whom information will eventually be used, but also the type of data (1.10) that should be gathered and how it might best be presented. Survey Similar Projects After parameters have been established (1.2), public relations planned accordingly (1.4), the systems capabilities surveyed (1.6), and decision makers (1.8) and information descriptions (1.10) derived, it could prove very useful to survey projects that have been conducted of a similar nature (1.12) (Kaufman and English, 1979). These should include not only needs assessments that have been conducted by other systems, but projects that the present system has undertaken that might lend to a better understanding of the system's capabilities and even receptiveness to the carrying out of such a process. At this stage, again, can clearly be illustrated the close interrelationship and interaction among these first steps. If needs assessments are studied that took place under seemingly similar circumstances, it is likely that new information and questions will arise that had previously’ not been considered or had been dealt with inadequately. The more complete understanding, that the study of these projects may offer, might lead to a re-evaluation of the system's 171 capability to perform it. Added awareness of the ramifications may further lead to a reconsideration of whom to involve in the process and to what extent, a redefinition of the assessment's parameters, and ensuing public relations. An internal look at projects previously undertaken in the same system might also surface information very valuable to such an undertaking. Even if needs assessment, per se, has not been conducted in the system, a look at other planning, budgeting, implementation, or related procedures could give clues about attitudes of boards of education, administration, or teachers toward the involvement of the public in curriculum decision making, educators and the public's understanding of the process, and influences of outside agencies such as state and federal government (English, 1977: Sarthory, 1977). This information could directly guide not only the setting of the assessment's parameters, but, public relations. If certain system limitations are found that will not allow for particular aspects of a needs assessment, such as lack of trained staff or other resources (Sarthory, 1977), or political considerations such as uncooperative or mistrusting teacher's unions or boards of education (English, 1977), definite modifications may be called for. A good deal of public relations (1.4) may be involved in educating both professional educators, administration and the 172 general public in the benefits and conducting of such an assessment (Sarthory, 1977). Present Plan to Decision Makers The next step in the process is to present the plan for the assessment to decision makers (1.14). It is possible, that up to this point, those in a position to decide whether or not a needs assessment of the variety recommended will be implemented, have not been involved. The idea of an assessment might have been introduced by a group of interested community members, some segment of the educational staff, or an outside organization such as the state board of education. The "decision makers" in this case may be a local board of education, superintendent. or other administrator whose responsibility it is to make such go/no go decisions. The proposed assessment would next be turned over to such a decision maker for approval. It is possible that the decision maker might recommend a revision of any of the pre-assessment steps before the choice of an appropriate model can be made and the process implemented. If any of the parameters need reconsideration (1.16), it may be necessary to once again repeat any or all subsequent steps. Even if the decision has been made to carry out an assessment, it may be beneficial, at this point, to 173 present the plan to those in positions of power whose cooperation and input are needed. It could prove helpful in the long run, for political reasons and/or valuable input, to inform these individuals, pass ideas by them and take into account their feedback. Choosing and Modifying the Model Enough information should have been gathered at this point to intelligently review and evaluate the model that guides the assessment process (1.17 through 1.21). A single model may not be appropriate to the needs of a specific application. One model may be used to guide the process as a whole. The model developed and presented in these pages should present the considerations with which any needs assessment of this variety should be concerned. But, even this generic model does not attempt to cover every question that might arise during an assessment. Materials have been developed, ideas and approaches discussed, and situation-specific concerns dealt with, with which a single model can not likely deal. The Unified Model does not deal directly with many of the "how to's" that relate to many of these areas. A system may want information on higher education applications (Mott Community Model), budgeting (Dallas Planning Model), development of instruments in many languages (Alameda 174 Country Needs Assessment Model), the use of alternate surveying techniques (Sensing Educational Needs in the Far West Region) or any number of assessment related questions. Many of the extant models presented in this study contain suggestions and ideas, and offer materials and methods, that may prove invaluable to practitioners. Some of these methods and models could likely be used in conjunction with the processes presented in the Unified Model for a particular application. The Unified Model, itself, is open to modification. There is no attempt in these pages to prescribe the steps that should be undertaken when conducting such an assessment. The purpose of the model, instead, is to present those considerations that a study of extant models and literature have suggested as necessary. By the omission of such considerations, a needs assessment might prove much less effective than it might be. But these steps should be omitted as a result of careful consideration, not as a matter of oversight or ignorance. Training/Orientation Once a model has been chosen for application, the training and orientation demands that will arise can be planned for and carried out (1.22). Decisions about such training and orientation needs will be deduced by 175 looking at both the processes presented by the model and the current ability of the system to perform them. Though the idea of needs assessment has been around for some time, it should not be assumed that it is understood by either educational professionals or the public served (Sarthory, 1977: English, 1977). Needs assessment can prove to be a process which demands a much higher level of sophistication than is present in the vast majority of educational systems (Sarthory, 1977) . Determination of Goals Two arrows proceed from the box 2.0 in the figure of the model. They are the "yes' and 'no" options to the question ”Are the desired goals, or the goals by which the outputs of the educational system will be measured already determined?" The 'yes" option arrow can be correlated with the current most common decision (Witkin, 1977) to opt for a Beta type assessment such as the Phi Delta Kappa or the Alameda Model (ACNAM). The choice of this process may be made for a number of reasons: the ignorance of alternate approaches (Kaufman and English, 1979: Witkin, 1977), the assumption that the goals in question are valid and useful, or the arbitrary decision by such groups as boards of education as to which goals will be established (see ACNAM, Appendix A) . 176 Collection of 'Felt Needs“ The 'no" arrow leads to an Alpha type assessment that seeks to first determine the desired goals by which current outputs of the system will be measured. This will be done by an external assessment of goals represented by boxes 2.1 through 2.11. The assessment that chooses the path in the ”no" direction begins with the collection of 'felt needs" (Kaufman and English, 1979). “Felt needs” are basically the concerns of the participants, or "partners" in education (Kaufman, 1972) (administration, teachers, parents, learners, and other community members) as related to what should be taught by the system in question. Processes, or how (methods, means) the system operates, should be avoided. It has been suggested (Kaufman and English, 1979) that one way to avoid concentration on processes and I'pet solutions“, is to begin the assessment without the use of a referent set of goals or a broad educational philosophy. Participants might be asked merely to brainstorm in small groups or to reSpond by questionnaire to very broad categories of possible outputs. The focus of the goals that are to be developed should be the condition of the learner (the skills, knowledge, and attitudes) upon exiting the educational system (Kaufman, 1979). The Critical Incident Technique (Flanagan, 1954, 1970) has proven a useful tool to the uncovering of 177 needs. It has been successfully implemented in such assessments as that undertaken under the Ohio Needs Assessment Guidelines (1980). The “critical incidents” might be used as symptoms of or clues to the discovery of needs in a system's outcomes. These incidents, if investigated, might lead to the uncovering of deeper causes, or learners needs. Reach Consensus At this stage of the assessment, validation usually means consensus among the participants (2.4) (Kaufman and English, 1979). Data gathered up to this point usually produces a list of the community's perceived or "felt needs". Consensus is often reached by exposing the participants to the input of the various other members. Being exposed to the input of other participants may help everyone involved better understand not only the needs assessment process but also to better evaluate their own thinking concerning what should be taught, and the overall import of each goal (Kaufman and English, 1979). Through various methods, such as averaging scores of ratings of these needs by participants, consensus is reached concerning needs. 178 Validate Needs As can be seen by looking over the areas of the previous matrix concerning validation of perceived needs (question 9) and from reading the models' descriptions in Appendix A, most extant procedures consider a need “validated" when consensus is reached among participants. Perceptions, even of experts, are only as valid as the objective tests by which they are measured. A wealth of information may be elicited by soliciting appropriate people's Opinions, but to insure the validity of the derived needs and the utility of the resultant educational programs to learners, these ”perceived needs” should be exposed to objective tests. Such data as the results of *well designed criterion referenced tests may demonstrate that the need, as it is presently stated, does not actually exist. The study of such materials as employment records and other documents might further point to a modification of the ”perceived needs" list. Calling upon noted experts and those directly influenced by certain educational outputs, such as law enforcers, businessmen, those of the medical profession, and previous graduates, might give insight into the refinement or general acceptability of ”felt needs“. Any information that can shed light on and lend a more comprehensive understanding of the environmental demands for which the learner is to be prepared can help 179 in making the needs more valid and useful. A look at the matrix (#9) will demonstrate that the majority of models rely heavily upon consensus among participants for validation. Among the Alpha models, VENAP, AAP, the IPI and Kaufman's emphasize the need to use many of the objective sources to determine needs. Reach Consensus on Validated Needs After needs have been validated, it is necessary to once again go through the process of reaching consensus among participants, or at least to turn the findings back for input (Kaufman and English, 1979). If explanation is not made as to why certain needs were or were not included in the list, it may be difficult to maintain the support of the participants. It will be helpful to assure all segments of the system that their input is taken seriously. Also, if a need was mistated or misunderstood, that can, at this point, be corrected, the need objectively evaluated, and added or rejected. Two Levels of 'Need' It might be helpful here to explain the use of the word 'need", in this step, that is unique to the Alpha assessment. 'Need" at this point does not suggest an educational system's shortcomings or the discrepancies 180 between 'what is' and 'what should be“ (Kaufman, 1972). Rather, “need", at this stage of the assessment, refers to the knowledge, skills and attitudes learners should possess upon exit from the educational system's current practices and programs. "Needs", or discrepancies between current outputs and desired outputs, will be derived in step five of the eight step process. "Needs” derived in the former step will actually form the ”desired goals” by *which the outputs~ of the current system will be measured. Futuristic Input Futuristic input (2.5) might now be considered, and the goals modified. The Delphi technique, in spite of some drawbacks, has proven useful for this purpose. The Atlanta Assessment Project (AAP) (Sweigert, 1970) is probably the most notable example of extensive use of this technique in a needs assessment. Input from Previous Goals To insure that the final output of the ”Determination of Desired Goals" aspect of the needs assessment is as complete as possible, the "new“ goal statements might now be compared to the system's former set of educational goals (2.6 through 2.10). An earlier 181 introduction of these into the process might have adverse effects upon the validity of the assessment (Kaufman and English, 1979). It is likely that many of the ”new” goals are currently a part of the system's current goals. Once the two sets are compared, and omissions are noted in the newly validated set, these might be added to the list, once they are validated, and the resultant goals prioritized. Once needs are reasonably validated, they should be prioritized (2.11). Politics may be an important factor here. When resources are allocated for educational expenses, it is more likely that goals that have been ranked according to importance will receive a commensurate share of the available resources. Without prioritization, all goals may be deemed ”important” and the division of resources will be left to political groups who will steer money, staff and facilities, toward goals that they "feel" should be priorities. This, more than likely, will result in the preservation of the status quo and be a very serious deterrent to the effectiveness of the whole needs assessment (Kaufman and English, 1979). All of the extant models, with the exception of Coffing ' s , make provision for goal or need prioritization (Matrix question #18). Coffing's gathers the information necessary to a specific decision. The information is given to the apprOpriate decision maker, 182 and decisions may be made without the involvement of other participants. Goals should next be stated in specific performance terms. The more specific goals can then be turned back to the participants for redefinition and re- prioritization. It is possible that more specification will cause participants to reconsider their importance. It is also possible that the objectives developed from the goals were not at all what the participants had in mind. It must be decided if this and subsequent stages in the Alpha needs assessment should, because of the logistics involved, be performed by only a narrower representation of the educational community (partners). Again, if committees or individuals replace broad participation, it is important that all members of the community are adequately’ represented (Kaufman and English, 1979). Determine Minimum Output Levels Before discrepancies between the desired goals and current learner performance can be determined, levels of expected achievement should be set (3.0). These levels should be a direct reflection of the stated goals, realistic, and written in terms that give measurable definition of what is expected. The most complete treatment of some of the practical considerations of 183 doing this is offered in the manual that accompanies the Ohio Department of Educational Needs Assessment Guidelines (ODENAG). This is a step common among extant models of both types. Almost all of the models included the measurement of current learner abilities and compare these to the information derived from their surveys to determine discrepancies ("needs"). But few deal with the subject of setting the desired levels of achievement which might lead to a higher specification of needs. If pertinent data is to be gathered indicating the actual discrepancies between current outputs of the educational system and the newly derived goals ('needs'), standards of achievement must be written that have direct reference to the goal determining information that has been gathered up to this point. It is reasonable to assume that many of the goals that result from needs assessment efforts are a part of current educational programs. A need in the area may still. be indicated because existing offerings do not sufficiently cover the necessary knowledge, skills, or attitudes connected with that goal. Perhaps certain courses in mathematics, for example, are included in a curriculum. A needs assessment indicates that local business and colleges maintain that the learners are ill prepared in the related mathematic skills. Though the courses are offered, the skill levels produced do not match those 184 needed in the world external to the learning environment. If information from the needs assessment is gathered in such a way that the needs can be developed into measurable criteria, the present program can be evaluated with greater utility. These criteria will lose their validity if they do not directly reflect the specific goal determining information from the assessment. The criteria should, of course, be realistic. All students should be able to perform to minimal levels in all those areas necessary to their survival and contribution to society. For many learners, this standard will be far too low. Achievement may be much higher in areas of particular strength or interest. But it would be unrealistic to expect all students to perform at high levels (above that which is socially necessary) in all areas. To set these levels, educational offerings would be [evaluated as to their necessity in surviving and contributing, and "desirability“ for special cases such as the college bound or those desirous of occupations and activities involving physical/manual skills. For example, a learner with plans for a college education, may be benefited by a 40 word per minute typing speed, whereas a secretarial candidate would likely’ need much ‘more. Levels indicated from the assessment and input from professional educators can be helpful when designing 185 realistic expectations. Without writing these levels of expectation in specific measurable terms, it could prove a very difficult task when it comes to determining gaps in current practices. The more specific the criterion, the more measurable, the greater the facility for discovering exactly what is lacking and, hence, precisely what must be done to fulfill that particular need. To suggest a "lack of mathematic skills" in a system is much less easily dealt with than ”55% of our graduates are unable to balance a check book." The setting of these levels and the specification may not prove easy in some cases, but a more intelligent effort can be exerted when information from a well conducted needs assessment is coupled with input from educators and experts in evaluation. Current Outputs There should now exist a set of validated, prioritized goals stated in specific terms by which the current educational outputs can be measured (5.0). The use of standardized tests for such measurement, for the purpose of the Alpha needs assessment, has been strongly argued against (Kaufman and English, 1979). There is little guarantee that the knowledge, skills and attitudes measured by such tests, reflect the 186 prioritized goals of the system in question. The needs pointed out may not be at all pertinent. If tests can not be found that reflect the validated goals, then it may be necessary to develop the best test possible based upon that newly developed criteria. To determine the existence of appropriate measurement instruments (4.1), existing instruments should be compared to the specified desired outputs (4.0). If the instruments do not measure these outputs sufficiently or if test items measure skills, knowledge and attitudes superfluous to the newly derived output standards, then the best possible criterion referenced tests should be constructed. These criterion referenced instruments, based upon the desired output levels, can now be administered to determine the current program's ability to produce the desired ends (4.3). Determine and Prioritize Gaps The next step in the assessment is' the determination of the gaps ("needs“) between 'what is" and 'what should be" (5.0) and their prioritization for closure (6.0) (Kaufman, 1972). This is accomplished by comparing the validated set of goals, or desired outputs, with the current outputs of the educational process. The discrepancies between the two are the "gaps" which should be given attention. These must next 187 be prioritized for closure. First they must be related to the prioritized goal of which they are a part. The priority of a gap can be decided by two indices. One is the severity of the gap. To what extent is the skill, knowledge or attitude missing from the present outputs? The second consideration is the priority of the goal with which the gap is associated. A severe gap in a low priority goal may not call for closure, whereas, a much less severe gap in a high priority goal, might (Kaufman and English, 1979). The final step that Kaufman includes in the Alpha needs assessment is to publish the findings of the assessment: a list of prioritized gaps chosen for closure. Depending upon whom will use the data, it is next collated in appropriate tables, graphs, charts and other summative and descriptive materials. This is then turned over to the appropriate decision makers. Continue Monitoring The final decision point of the model related to the Alpha model of assessment, is to either continue or discontinue the monitoring of the system external to the educational program. The 'yes' option simply involves setting up a mechanism in the organization that periodically gathers information pertinent to the updating of current curricular offerings. 188 To be classified Alpha, a needs assessment ought to be an on-going process continually monitoring the society of which the educational system is a part, gathering data to refine and redefine the needs resulting from the original assessment (Kaufman and English, 1979). Being a systems approach, the process is continually open to all of its findings being updated according to subsequent activities (Kaufman, 1983). Previously described activities have been presented in such a way as to emphasize a systems approach to the Alpha assessment. The repeated re-stating and re- prioritizing of needs as a result of new input should take place. Such instances were presented as those when goals having been turned into more specific objectives, have to be turned back to participants to see if the results were those intended. Again, after “futuring' and comparing “new" needs to the educational system's previous goals, another prioritization would take place. After a program has undertaken the changes suggested by a needs assessment, information should be gathered from two perspectives. First, the outcomes of the program, or the effects upon society (Kaufman, 1983), should be measured to determine if they are indeed positive. Second, because society is not stagnate, but continually in a state of flux, if an educational program is to keep abreast of the changes in its environment, there must be a continual gathering of 189 data. A needs assessment can never be complete as long as society's demands change and an educational system's purpose is to meet those demands. The Interrelationship of Alpha and Beta Needs Assessments and the Other Process of the Instructional Development Procedures The connection of the information gathered from the needs assessment processes and its use in the subsequent educational development activities are dealt with, at best, sketchily by most of the models' developers. The exception is Kaufman who has dealt with most aspects of the assessment process in detail in books and articles scattered throughout educational literature. A much fuller treatment of these interrelationships, therefore, is dealt with in Chapter II where needs assessment is related to evaluation, instructional development and Education as a whole. The following is a brief synopsis. Being part of evaluation (Alkin and Fitz-Gibbon, 1975), the Alpha process's results can be used as the referent for all subsequent evaluation, both formative and summative. If a needs assessment has been conducted in such a way that the findings reflect the abilities that a learner needs to survive in and contribute to society and the goals and objectives of the system are directly derived from these needs, an educational 190 program's content can be designed with a direct link to society's demands (Kaufman and English, 1979). The goals and objectives developed from the needs assessment are the results toward which the activities of the educational system are to proceed. If a program is coordinated from kindergarten through the twelfth grade (and beyond) there will, of necessity, need to be check points along the way. A twelve year long education might prove difficult to correct if its success is only measured at its completion. This is the purpose of formative evaluation (Cronbach, 1963: Scriven, 1967). The evaluation measures are planned to measure and guide the learner's progress and to gather information for educational decisions concerning that progress toward a final goal. Summative evaluation (Scriven, 1967) may be even more readily traced to the data resulting from the needs assessment. If the purpose of summative evaluation is to judge the final results of a program, except in form, there should be little distinction between the data of the assessment (the abilities of the learner upon exiting the educational system) and the standards used in the summative evaluation. The summative evaluation is an integral part of the needs assessment in that, to compare the present outcomes of the system with the goals derived from an Alpha. needs assessment a form of summative evaluation must be conducted. Even if a 191 system is based upon an external assessment, the summative evaluation's findings should be used as feedback to the assessment for comparison's sake to determine still unmet needs. As has been mentioned above, the information from the needs assessment is used to form the purposes, goals and objectives that form the educational system (Mager, 1962: Kaufman, 1972: Kaufman and English, 1979: Hannum and Briggs, 1982). Specific objectives should foster the apprOpriate methods that lead to their accomplishment (Kaufman, 1972). The Beta processes in particular analyze the educational programs that might be used for achieving the desired goals (Alkin and Fitz- Gibbon, 1975). Prioritized needs can help in such decisions as staff/facility allocations and budgeting. If needs are not prioritized according to objective means, but by the feelings of decision makers, there is little assurance that the new program will reflect the true needs of learners rather than the ”feelings" of the decision makers (Kaufman and English, 1979). Appropriate instructional strategies (Bretz, 1977: Briggs and Wager, 1981) and media (Peterman, 1981: Jamison, Suppes and Wells, 1974: Schramm, 1977) can also proceed from specific objectives drawn from needs assessment data. In short, a well conducted needs assessment forms the foundation for most subsequent decisions related to 192 the educational process. Question #7 How do the characteristics of extant models compare to the characteristics considered ideal for such models? In answering question number six in the previous section, an eight component model was offered in an attempt to synthesize the best characteristics of extant Alpha and Beta type models into a generic one. It would be a very cumbersome task to relate all of the models in the study to all eight components in a narrative fashion. Instead, as each of the eight components was discussed, reference was made to extant models and their contribution to the component in question. Although all eight components can be accounted for if extant models are compiled, they are not completely represented in any single previous model. On the following matrix (Table 5), the numbers on the horizontal column represent the eight main processes of the Unified Model: 1) Pre-Assessment Considerations 2) Determination of the System's Desired Goals 3) Setting Minimum Output Levels 4) Measurement of Current Outputs 5) Determination of the Gaps Between the Desired and the Real 6) Prioritize Gaps for Closure 7) Select Gaps for Closure 8) Continue to Monitor External System 193 The vertical column is composed of a list of the available models included in this study. The 'x's“ in the boxes indicate that the model at the column's left takes the process at the top of the column into consideration. At lanta Assesmnt Pronect Battelle Survey Dilas Planning sbdel \bt t Camunity Col lege Abdel A Illinois Problcn Index A_* \\ Oiio Dparmnt ol Educa- tion hbeds Msesmnt Caidelines Quality Education Pro- gran Study A Cmprehensive heed: Assesmnt nodule Institutional Goals inven- tory Phi Delta Kappa Ibdel A \\ Sensing Educational hbeds in the Far ks! Region for ldwide Planning node! Coil ing' s Client Needs Assesmnt A Lee's Needs Assesmnt Abdel A Consolidated Application's heed: Asseimnt Canide \\\\ \ Fresno Planning Abdel Vocational heds Assess-rent Project \\ Kaufman's hbeds ksesmnt Nbdel "\ \ \\\\\\\\\ \\\_\\\\\ \\\\\\\\\\\\\ \\\\\ Skyline I'est Educational Plan \\\ \ \ Hat less' Front-end Analysis t/l/ \ Table 5 Carparison oi the Twenty-one Models to "idealized" Characteristics 195 Question #8 What research evidence exists as to the ability of these models to produce the information they were designed to acquire? The matrix lists only five models that reported any research in connection with them (#22). These models were the Phi Delta Kappa, A Comprehensive Needs Assessment Model, Vocational Education Needs Assessment Project, the Atlanta Assessment Project, and Lee's Needs Assessment Model. The following is a brief reference to and description of that research. The Phi Delta Kappa Model's research included "Evaluating Needs Assessment: Reactions to Phase I of the iPDK iEducational. Planning Model". This research, reported in Education ‘Vol. 101, No. 3, surveyed the participants who attended the representative community meeting that constitutes Phase I of the model. The survey questioned their attitudes toward needs assessment and their opinions on the Phase procedures. It was indicated that the participants viewed needs assessment as important and that the representative community was a "pleasant and productive process." But, certain aspects of the process, it was added, should be improved. It was reported on the questionnaire that the components of the A Comprehensive Needs Assessment Model were statistically analyzed by Charles WOodsen, 196 University of California at Berkeley. The Vocational Education Needs Assessment Project was subject to research conducted by Hannah Mayer, reported in the Performance and Instructional Journal, Oct., 1983. The study's purpose was to investigate the "missing link” between the demand for such planning tools as needs assessment models and the successful implementation of such procedures. Dr. Sweigert, of the Atlanta Assessment Project, reported that ”all of the extensive batteries of test items were field tested in the Atlanta Public Schools and statewide in Georgia. Item analyses were conducted and faulty items were revised or discarded.” ‘Walter S. Lee answered that, though research ‘had been conducted in connection with his model (1973), there is currently none "directly available." Summary The main purpose of this chapter is to reference or provide the answer to the eight research questions posed in Chapter III, which have guided this study. Data that resulted from a questionnaire and the study of user's manuals and other literature related to extant Alpha and Beta type needs assessment models is presented in a matrix (Table 3) that allows for comparison of extant models according to a list of twenty-three questions. 197 All eight research questions are either answered according to the information offered or reference is made to where they have been answered in other sections of this work. A Unified Alpha/Beta model is synthesized from the characteristics found in extant models. Chapter V Summary, Findings, Conclusions, and Implications for Research Summary It has been the main purpose of this study to gather and present information concerning the use of educational needs assessment models, in general, and the Alpha and Beta types (Kaufman, 1979) in particular. This study has been carried out in the hope that educators and non-educators, alike, will be enlightened as to: l. the benefits of conducting needs assessments, 2. the description and availability of models useful in guiding the process, and 3. further information that could be helpful in choosing a model for a particular application. The information necessary to the study was gathered primarily from educational and related literature. Since answers to many of the questions asked to analyze the models were not reported in the literature (including the manuals that accompanied some of the 198 199 models), it was necessary to design and distribute a questionnaire. The questionnaire was sent to model developers and those who market both the models and materials necessary to a needs assessment. The descriptive and evaluative data is presented in a matrix that allows for a reasonably easy comparison among the models. A narrative description of each model, offering a more indepth look at the individual models, particularly emphasizing the questions asked in this study, is included in Appendix A. An opening assumption of this work was that much of what is amiss in the condition of public and private education in the United States today, can be traced to a lack of a well documented systematic approach to the definition, deve10pment and evaluation of the purposes, goals and objectives that underlie the programs of our educational institutions. It was further assumed that a major cause of current conditions was that legislators, administrators, educators and other involved community members base their educational planning. . .on unvalidated information rather than upon well documented data...reflecting requirements of the society that the educational system was established to serve. The root of the problem was suggested to be that these individuals were unaware of the source of the problem and the means available to its solution. Needs assessment was offered as a partial solution 200 to some of the problems that currently plague education. Such assessments have been claimed to offer a 'clear- cut...approach to the determination of needs“ (Heldge and Marrs, 1978), a formal set of tools and techniques...for putting means and ends into proper perspective (Kaufman, 1978), and a "process by which the unfulfilled educational requirements of a population...are identified and educational objectives determined (Lee, 1973). Needs assessment offers a process through which programs from the kindergarten to the twelfth grade can be coordinated (Buhl, 1978) and further, promises to pursue the development of “educational partners" (Kaufman and English, 1979) the involvement of cormnunity, industry, parents, learners and educators in the educational planning process. It was reported that in the fourteen year period between 1968 and 1982 that educational outputs in the U.S. took a marked decline. SAT scores dropped from 466 out of a possible 800 in 1968 to 424 in 1981 (Freidrich, 1982). The 1981 Gallup Poll, measuring the public's Opinions on education, reported that the 'bad" grades outweighed the 'good" by eighteen percentage points (54% to 36%) (Weiler, 1982). The model upon which most extant models are based is Kaufman's (1972) (Trimby, 1979). From about 1972 to 1980 the major proliferation of currently available models and materials (Witkin, 1975) for the conduction 201 of needs assessments took place. As will be discussed in more detail in subsequent paragraphs, the means to the solution of the above problems were as much available during education's twelve year decline (from 1970 to 1982) as now. It seems almost ironic that it is during this period that the greatest efforts took place in the develOpment of the proposed partial solution, and still the decline continued on to what has been termed an educational "unilateral disarmament” (Bell, 1983). It seems unlikely that such development of a solution took place without at least the hint of a related problem. Yet, the problem continued, and worsened. The systems view will tell us, of course, that the problem was not traceable to the mere absence of sound needs assessment technique application. There are far too many influences that effect education as a whole to narrow the cause to any one. But a look at many of the criticisms leveled at current graduates could lead one to think that a major problem with current education's output is content. It is claimed that learners are simply not being taught what they need to know to survive in, and contribute to society. Whether or not the 'fault" lay in the accusation that teachers are ill prepared, the fact remains that even well prepared instructors, teaching the wrong subject matter, are bound to turn out an inferior product. 202 Perhaps the answer partially lies in the suggestion that even though available, educators are unaware of the potential of such assessments and the means of their (conduction (Sarthory, 1977: Trimby, 1979: Kaufman, 1977). This can be tied to the fact that information on needs assessment is scattered throughout the literature, leaving the area in a state of underdeve10pment (Witkin, 1975) . Findings, Conclusions and Suggestions for Research This section contains first a list of the eight research questions that guided this study. Following those are the findings that resulted from answering those questions, conclusions drawn from those findings and suggestions for further study based upon those findings and conclusions. These are ordered, as strictly as possible, according to the eight research questions. Findings Finding L1: 1. Who were the significant individuals and what were the significant events and/or landmark applications that figure into the evolution of Alpha and Beta type needs assessment models? 203 Historically, the development of the Alpha and Beta type needs assessment processes involved relatively few individuals, events and landmark applications. Most extant models are based upon the “classical“ method of assessing educational needs (Kaufman, 1972: Southard, 1974) and patterned after the early model developed by Kaufman, Corrigan, and Johnson (1969), (Trimby, 1979). Maslow (1968), Frankl (1962, 1965, 1967, 1969), Rogers (1964), and Ruckers (1969) contributed work that put emphasis on the “humanistic” requirements. From the Hannah Formulation (1966) came the logical entry point into an assessment, the ”nature of society“. Stufflebeam's CIPP model pointed out the inextricable link between planning and evaluation. Sweigert contributed the idea. of educational. “partners“ (1969, 1971) and Rucker the concern for the individual's uniqueness and potential beyond minimum requirements. Trimby called attention to the relationship) of needs assessment and evaluation (1979). Kaufman suggested a taxonomy for classifying needs assessments (1979). Two land mark applications emerged as representa- tive of two possible, distinct approaches to the assessment of educational needs. These were the applications in the Newport-Mesa Unified School District (Shuck, 1968) and the Temple City Unified School District (Kaufman, Rand, English, Conte, Hawkins, 1968). 204 The inductive approach, which first gathered data from the environment external to the educational system from which to derive the system's goals, was exemplified by the Newport-Mesa application, and might be called the forerunner to Alpha type needs assessments. The deductive approach, beginning with pre-stated goals as given, may be typified by the Temple City assessment, and relates to those approaches found in Beta type assessments 0 Finding #2 2. What are the names and descriptions of established Alpha and Beta type needs assessment models? A dedicated search of the educational and related literature, surfaced twenty-six models. A continued search may have produced additional models, but the researcher is confident that the ones presented here are more than representative. Five of the models recorded herein are classified as belonging to the Alpha variety, twelve to the Beta, and three to the ”other” category. Five of the twenty-six are either currently unavailable or their sources unlocatable. "11:11 'JILL.’.I_.L- .. i' * '- ' "' ‘mTIII'P'J? firm-mum HUI] .lfll" ifimlfl'm 205 Finding #3 3. What types of information could ideally proceed from the application of Alpha and Beta type needs assessments? It was suggested by many authors and developers of models that the information proceeding from the proper application of the needs assessment process could provide foundation for many of the subsequent educational design and development activities. Among these were problem solving , decis ion mak ing , accountability, budgeting, coordination of programs, allocation of staff and facilities, community involvement, and evaluation. After further development, the information proceeding from needs assessment can result in marked effect on any or all of these areas and should, since providing data upon which future educational planning can be based is the purpose of such assessments. The information that immediately results from a needs assessment of the varieties represented in the extant Alpha and Beta models Chiefly consists of: l) a validated, prioritized set of requirements the learner faces in society upon exiting the educational system. 2) a validated, prioritized set of educational goals by which the educational system's educational planning and decision making can be guided. 206 3) a description of the educational system's current outputs in relation to the goals established, according to an external referent. 4) a validated, prioritized set of needs or discrepancies that exists between the externally derived goals and the system's current outputs. Finding #4 4. Are there major variations among Alpha type needs assessment models? Although in most ways very similar, there are some variations among needs assessment models of the Alpha type. All of the models categorized as Alpha, sought to determine learner needs through the participation of both professional educational staff and the general community. These participants also had involvement beyond the data gathering stage of the assessment. All Alpha models, except Kaufman's, were developed as a result of actual applications. The applications, from which they sprang, lent to the distinctions among the models because of the demands present in the varying situations and settings. These were applied at different educational levels. Unique surveys, questionnaires and other tools and techniques were used. The most outstanding variation is the means by which needs are validated. All of the Alpha type models, except Fresno's, rely to differing degrees upon ' _- 207 objective information to decide upon needs. Fresno's needs are decided by administrative choice. Only the Atlanta and VENAP models reported research conducted in connection with them. Finding #5 5. Are there major variations among Beta type needs assessment models? As in the case of Alpha models, Beta models are also in many ways very similar. Likewise, there are some marked distinctions among the needs assessment models classified as belonging to the Beta variety. Although mainly differences of degree, variations can be noted in the following elements among Beta type models: -resources and services offered with the models -purpose of the individual models -educational levels at which the models are aimed -pre-assessment considerations -extent of participant involvement -cost of the assessment -time needed for completion -objective validation of needs -field testing -connected research -industrial applications These are dealt in detail with appropriate models named as examples in the previous chapter under "Research Questions #5“. 208 Finding #6 6. What characteristics could be included in idealized Alpha and Beta type needs assessment models? A distinct difference exists between Alpha and Beta type modes of assessment. A model need not contain only those characteristics unique to one mode of assessment but may, and several do, cross boundaries of Kaufman's taxonomical scale. The seven characteristics that could be included in an "idealized” Alpha and Beta type needs assessment model are: l. Pre-assessment considerations, 2. Determination of goals, 3. Determination of desired output levels, 4. Measurement of current outcomes, 5. Determination of discrepancies (“gaps“) between desired and current outputs, 6. Prioritization of those gaps, and 7. Continued monitoring of the system to update programs according to external demands. The one major process that distinguishes the Alpha model of assessment from the Beta is the external assessment of needs used to determine the system's goals or final proposed outputs. Other characteristic differences were suggested by Kaufman's taxonomy (1979) 209 such as the Alpha dealing with products rather than the processes (as are dealt with in the Beta). The major distinguishing characteristic of the Alpha model is its concentration on Ella}. should be taught rather than h_o_w_ to teach. Since the 31135 of instruction is a given in the Beta process, optimizing output according to methodology (”how") is the emphasis of this approach. The one process that the researcher found unique to those models herein referred to as Alpha, was the external determination of the system's goals or outputs. At least one model, the Quality Education Program Study, had the purpose of defining and clarifying goals. This might be construed as an intermediate step between the actual setting of goals in an Alpha and the acceptance of pre-determined goals in a Beta. If the utility of an educational system's program is a major concern, then a crucial step in a needs assessment will be the goals that help to define the final outputs of that educational undertaking. How those goals are derived may be the crucial question in a needs assessment. An Alpha assessment proposes that a concerted effort be made to help insure the validity and utility of the system's guiding goals by collecting information from a representative sample of the entire community served by the school. The Beta assessment begins with these goals as givens. Both modes may attempt a community wide determination of the "gaps' or 210 discrepancies between the guiding or desired goals and the current outcomes. This is a similarity, not a distinction. An Alpha assessment might not deal with these processes. As soon as it does, it enters the realm of a higher category of the taxonomy (but its beginnings remain Alpha). Finding #7 7. How do the characteristics of extant models compare to characteristics considered ideal for such models? A look at Table 2 can lead to the conclusion that among all the models reported in current literature, only the Kaufman model accounted for all the characteristics and processes that might be included in an ”idealized" needs assessment model. To derive the information needed to fully understand this approach, even from Kaufman's writings, would take considerable effort on the reader's part since they were developed and presented over an eleven year period from 1972 to 1983. From Table 3, it can be seen that almost all models dealt in some detail with the measurement of current outputs, the determination of discrepancies between desired and current outcomes, and the prioritization of these discrepancies (“gaps“). 211 Although some dealt with pre-assessment considerations, others did not. The detail necessary for understanding this phase of the needs assessment process was lacking in almost all models, it might be added, except the Ohio Department of Educational Needs Assessment Guidelines. The vast majority of models fell into the Beta or "other” categories as can be determined by looking at the lack of entries under number 2, the ”Determination of Goals". This may not, of course, be a shortcoming. It simply indicates that these models began with the assumption that a current set of goals was valid and useful. This may be considered a shortcoming although, since these models did not provide for the Option of checking to see if these goals were in fact externally valid. Only a few of the models suggested why and how minimum output levels might be set. Although mentioned in passing by several of the manuals and other articles researched, only Kaufman dealt in detail with the necessity of continually monitoring the society external to the educational system to update the curriculum. The 'whys' and “how to's" of needs assessments are amply represented in educational literature. To be sure, the information is piecemeal and scattered. It would assuredly take a concerted effort on the part of 212 any school board, or other organization, to gather it together into a coherent package---one that would answer the questions that the “untrained“ would need answered to insure a needs assessment that would produce well documented needs that actually reflect the demands that the external society places upon the system's learners. The Unified Needs Assessment lModel presented in Chapter Four is an attempt to demonstrate the salient and necessary' elements and interrelationships involved in needs assessment models of the Alpha and Beta types. There is nothing in the model that could not be found in extant literature. Hopefully, this study has provided a broad enough base of explanation and reference to better develop the framework, initially suggested by Kaufman's Taxonomy (1979), to help the reader tie together this important area. Materials or tools needed to conduct the assessment are not Offered. The researcher, at this time, has no desire to become involved in their develop- ment and marketing. It might be argued, also, that tools and materials designed and developed by an agency outside the social environment of the educational system undertaking the assessment, might influence the outcome of the process, and bring into question the validity and utility of its findings. An Alpha assessment should be carried out using data derived from the system's envi- ronment, data that reflects the needs of that system as uniquely and as unadulterated as possible. 213 Finding #8 8. What research evidence exists as to the ability of these models to produce the information they were designed to acquire? Very little research evidence exists as to the ability of extant needs assessment models to produce the information they were designed to acquire. Only five of the models reported any research in connection with their development or application (see matrix question 22). Research on the Phi Delta Kappa Model sought feedback from participants as to their Opinions and attitudes concerning Phase I of the process. The components of the Alameda County Needs Assessment Model were statistically’ analyzed. Research concerning the Vocational Education Needs Assessment Project was concerned with the considerable lack of implementation of available tools such as needs assessment models. The Atlanta Assessment Project's extensive battery of test items were field tested and faulty items revised or discarded. The research on Lee's Model is no longer available. 214 Conclusions Conclusion #1: Needs assessment models of the Alpha and Beta types are well established in educational literature and applications of both types have taken place in actual practice. The Unified Model, presented in Chapter IV, has been established in this study incorporating successfully the best elements of both Alpha and Beta models. The components chosen for that model were singled out because it can be demonstrated, either through reason or application, that each was a desirable part of the overall process. Since no extant model accounts for all of the characteristics present in the Unified Model, it is concluded that extant models are individually incomplete in their attempts to portray a combination of the Alpha and Beta varieties of needs assessments . Conclusion #2: The comparison of extant models of the Alpha and Beta categories shows that a distinct difference between those types can be demonstrated. A better distinction or classification than a model taxonomy might be a needs assessment process taxonomy. Kaufman (1972) attempted to develop a model that 215 emphasized only the processes that deal with the assessment of the products or outcomes of education, deliberately eliminating any consideration of the processes ("how to's") needed to actually accomplish those outputs. He did this to avoid confusion between the two and to emphasize the need to externally determine a system's goals. Other models, that begin as an Alpha assessment, go on to processes that are characteristic of subsequent processes of the taxonomy. The World Wide Model, for instance, goes through the Alpha process and then gives some direction for deciding upon a strategy for solution. It is conceivable that a model could be produced that should begin with the Alpha processes and continue on through to the summative evaluation of a program. Conclusion #3: Because a logical and reasonable case has been presented for the use of Alpha types assessments, the lack of measuring the impact of applications of these on actual educational systems, leaves doubt as to the utility and validity of such models. One of the main tenets for the use of such assessments (Kaufman, 1972: Lee, 1973: Kaufman and English, 1979) was that professional educators, simply by the virtue of their position as teachers, are not necessarily in a position to know the precise demands society will place upon a 216 learner. Conventional wisdom and expert opinion, unvalidated by objective data, may not necessarily, accurately reflect a true picture of educational needs. This same argument against the use of only Opinion to determine the needs of an educational system might be advanced to support the applications of an Alpha type needs assessment. Logic and strong reason do not constitute objective proofs. A strong theoretical case can be, and has been, presented for the benefits of the application Of such assessments. If understanding leading to a fuller realization of the potential of this mode of assessment is to come about, it is likely that empirical evaluation of the processes in question, their outputs, and the outputs called for, should be conducted. Conclusion #4: It may not be possible to design an "ideal' needs assessment model until some of the research suggested in the next section is conducted and conclusions reached. The Unified Model offered by the researcher is the most comprehensive model currently available. It is a compilation of the processes of extant models arranged and designed according to the input from users, model developers and other related experts. The possibility exists that research may uncover other elements that should be included or even, possibly excluded. Com- 217 ponents beyond currently available models, results of field tests or input from experts are not included in the Unified Model. Even if all necessary elements and processes are currently available, research can lend them credibility and further understanding. Conclusion #5: Misunderstandings concerning the purpose and processes of needs assessment are an important inhibiting factor in the successful implementation of these assessments. Needs assessment has been fairly common in the literature over the last fifteen to twenty years. Many administrators, educators and, to some extent, the public have an awareness Of the processes involved. The mistaken Opinion that this awareness constitutes a working knowledge of the complicated process can lead to at least two undesirable consequences (Sarthory, 1977). The first is that administrators, feeling comfortably knowledgeable, may not seek to acquire further information, skills, knowledge and training in the methodologies involved. Lack of understanding of the power of the process may lead to a decision not to carry out the assessment. Perhaps worse, an insufficient needs assessment might be implemented, poor results experienced, and the technique abandoned for lack of promised results. 218 The misunderstandings involved with the definition of 'need" lend to an unsuccessful implementation of needs assessment. Needs are not a desire or wish but a quantifiable, measurable gap in performance. One of the major reasons Kaufman (1972, 1979) emphasized the distinctions between the Alpha and other processes of his taxonomy was to stress the necessity of beginning with the products of an educational system before the processes for their accomplishment are considered. Discrepancies between current outputs and desirable outputs can not be noted and dealt with if desired outputs are not first determined. Extant models put little emphasis on improving the understanding of users concerning either the promise that needs assessment holds or the processes useful in achieving those desirable ends. Conclusionng: In many models too little emphasis is put upon training participants both in the purposes and use of extant models and methods of needs assessment. Without training, the "untrained” remain just that. As can be seen from the matrix, only a few of the models' developers and marketers suggested the necessity Of outside consultation (question #14). Most suggested that little formal training was needed to manage the process. Unfortunately, the systematic design and 219 development of the tools, data collection and analysis, management, and other skills needed to accomplish an assessment of the nature described in these pages require a high level Of sophistication and training. Needs assessment has been described as a relatively complex, arduous, and time consuming task (Heldge, and Marrs, 1978). It is possible that all the necessary skills are a part of the ”typical" educational setting and that these are easily tapped and-coordinated into a program. It is also possible that some of these skills are lacking or, if available, that the depth of understanding about the requirements of the needs assessment jprocess are not developed to a sufficient level to enable a ”typical" educational system the effective application Of even available expertise for this particular purpose. It is a suggestion that more training and, even, public relations publicizing the process, are needed. Conclusion #7: In many models "felt needs“ rather than objectively validated data, are Often the outputs of extant assessments. Such well established and broadly applied models as the Fresno and Dallas Planning Models rely very heavily upon the perceptions of participants to determine both the system's desired goals or outputs and the discrepancies between these and current outputs. 220 Without objective validation, there is little reason to assume that needs that reflect merely the feelings of participants will result in little more than a “wish" list. There may be little reason, also, to expect that the “new” program will deviate much from the status quO and, hence, will have little more chance of reflecting the real world needs of the learners in the system. Conclusion #8: In many models, problem oriented assessments may result in the loss of valuable program elements.“ Activity box 2.7 on the Unified Model represents a procedure that is not represented in extant needs assessment models. That is the comparison of the system's previous goals to the newly derived goals. This should be done to insure that important areas that did not surface during the external needs assessment, but were a part of the previous set of goals, are not set aside and forgotten when planning takes place. Through an interview with Kaufman, the point was made that there exists the possibility that when focus is put upon problem areas, or needs, in the form of discrepancies between what is and what ought to be, that the newly derived program may eliminate some Of those previously included elements that are also needs. These may be the 'met needs“ or those skills, attitudes and 221 knowledge needed by the learner that the current educational system is satisfying. As Scriven stated in his Evaluation Thesaurus (1981): A second fatal flaw in the discrepancy definition is its fallacious identification of needs with one particular subset of needs, namely unmet needs. But there are many things we absolutely need--like oxygen in the air, or vitamins in our diet--which are already there. To say we need them is to say they are necessary for e.g. life or health, which distinguishes them from the many inessential things in the environment. Of course, on the discrepancy definition they are not needs at all, because they are part of “the actual”, not part of the gap (discrepancy) between that and the ideal. It may be useful to use the dietary terminology for met and unmet needs-- maintenance and incremental needs. People sometimes think that it's better to focus on incremental needs because that's where the action is required: so maybe--they think--the discrepancy definition doesn't get us into too much trouble. But where will you get resources for the necessary action? Some Of them usually come from redistribution of existing resources, i.e., from robbing Peter's needs to pay for Paul's where Peter's (the maintenance needs) are just as vital as Paul's (the incremental). This leads to an absurd flip-flop in successive years: it is much better to look at all needs in the NA. Prioritize them (using apportioning methods not grading or ranking) and then act to redistribute Old and new resources. 222 Implications for Research Implication #1: Applications of the various Alpha and Beta models can be studied to determine if the resultant information is indeed varied as a result of the variations among their component parts. If the outcomes do differ, it could be decided if these variations result from. the lack of certain components that are not accounted for in the model, or as a matter of other considerations such as purpose. Implication #2: A primary consideration might be the nature of the information actually needed to enable an educational system to develop goals that accurately reflect the knowledge, skills and attitudes needed by the learner upon exit from the school. This may prove difficult since the methods needed to determine this may be the very modes of assessment that are under question. That is, in order to determine what constitutes a life of "survival and contribution”, the very processes that characterize the Alpha mode may have to be used. At the outset, the assumption might be made that these methods are valid. 223 It might be more fruitful to determine the type of data that must be gathered to enable decision makers to make goal setting decisions. The type of data needed, the detail, and the amount, can likely be used to determine the methods for procuring it. Implication #3: Once the parameters in the above implication have been set, the outputs of Alpha assessments can be compared to see if they are one and the same. IIf significant discrepancies exist, possibly the extant assessment processes can be modified to close that gap. Implication #4: It may be necessary to conduct both Alpha and Beta assessments in the same or very similar circumstances to find out if outcomes are actually discrepant enough to warrant the application of one or the other, for specific outcomes. Implication #5: Along the same lines as #4 above, it could be very useful to determine if the data gathered from non- professional educators actually lends enough valuable input to warrant the extra time, energy and expense that the applications of such methods demand. 224 Implication #6: The desirability of any of the steps in the models might be determined by conducting assessments under controlled conditions with different combinations of these applied. The differences in output might signify the value of the present, and absent, steps. Implication #7: After carefully describing those concepts and processes that have been proven successful in assessment projects, those who might be consumers of the technique might be surveyed to determine their understanding. Possibly the results might be used to shed light as the whether such consumers have affected past less successful attempts at application of the technique or if those with misconceptions are also unreceptive to the technique. Implication #8: After determining that knowledge and those skills necessary to carry out a typical assessment, several questions might be asked: To what degree are these skills and knowledge represented in various types Of school systems? What knowledge and skills are present in those electing to carry out an assessment? What 225 training is offered in these situations? If training differs in similar system's applications, what is its effect? Implication #9: The Objective validation of felt needs can be a time consuming and difficult task. Evidence that it is necessary might prove a valuable incentive to its undertaking. Perhaps one way to gain support for this process would be to check past assessments that have applied it and compare the ”felt needs” with the final objectively validated needs. If, in fact, a distinct difference can be demonstrated between the felt and objectively validated needs, the effort involved in the process might be considered worthwhile. Implication #10: Since the application of the data derived from a needs assessment is to cure ills, not to cause them, it could be beneficial to ascertain just what has been lost as the result of a needs assessment that may have been of value to the learners in that system. If an Alpha type monitoring of the system is continually carried out, this could be detected. But a Beta type, that deliberately singles out these deleted items and 226 determines their current value as compared to what is presently a part of the system, might be more useful. Implication #11: A fuller look might be taken at the application of these educationally oriented :models in industrial settings. A starting place might be the study of extant models that have already been applied in that setting (see matrix question 23). The value of such studies is not only to determine what can be traded between the practices of the two areas, but to better understand the potential of the models as their abilities are demonstrated in different settings. First, it might be useful to describe any pertinent differences between the demands and characteristics of the two settings. Are there differences in effectiveness when the models are applied in these settings because of these characteristics? What modifications of models or methods have to be made? What do these modifications tell us of the settings in which the application took place and of the model that was applied? 227 Implication #12: After the necessary processes and data to be gathered through Alpha assessments have been established through objective research, it could prove helpful to identify and standardize some of those techniques that might be used for carrying out those processes. This could prove helpful in at least two ways. First the untrained, even if they have a basic understanding of what do do, do not necessarily know how to do it. Secondly, after sufficient data has been gathered as to what to do, the “how to's' of the process may lend to an understanding of just what that process consists. Summary It has been the main purpose of this chapter to sumarize the procedures used to carry out this study and it's major findings. Conclusions from these findings are alSo offered. These conclusions imply research that might be conducted to further illucidate this developing area of needs assessment in education. Several suggestions for further research are also suggested. APPENDIX A Individual Model Descriptions 228 229 Model: Source of Information: Atlanta Assessment Project Dr. Ray L. Sweigert, Jr. 5239 North Peachtree, NE Atlanta, Georgia 30338 Type: Alpha Under the guidance of Dr. Ray L. Sweigert, the Atlanta Assessment Project was designed as a comprehensive assessment of minimal learner objectives for successful OOping in the society of 1985. The project, now completed, and materials and services for the most part unavailable, was designed under Title III of the Elementary and Secondary Education Act, administered by the Georgia Department of Education and Operated in the Atlanta Public Schools. The assessment was designed to obtain the answers to three basic questions: 1. What will the young people in Atlanta area need to know, be able» to do, and value --to the extent possible to determine--— in order to be able to OOpe with life in the society of 1985? 2. Where are the young people of Atlanta today in achieving these things? 3. To what extent are the assessments and tech- niques developed to answer the first two ques- tions applicable to statewide testing in Georgia? 230 The project used the Goals for Education in Georgia adopted by the Georgia Board of Education in 1970 which were developed tO reflect learner needs in 1985. These goals were developed from position papers by scholars and experts on future conditions in Georgia regarding 19 areas of concern. Delphi studies were used to validate these goals involving 1100 community leaders, educators and high school students. Each goal was clarified and defined by 90 teachers, curriculum specialists, cOllege and university professors, professional staff of the State Department of Education, and representatives of various social agencies. Tests were developed to measure the attainment of these goals by contract bidding to test development agen- cies on a national level. These were administered to 3000 high school students in the Atlanta Public Schools, items were revised as necessary, statewide tests were conducted to test appropriateness for a statewide assessment and the outcome was eventually administered to 15,000 12th grade students in sixty-five Georgia systems. Though at last report unavailable, this model's de- scription is included in this study because it is a rare large scale application of many of those aSpects of needs assessment that qualifies as Alpha. 231 Model: Source of Information: Battelle Survey of Educa- Center for Improved Edu- tional Needs cation Battelle Memorial Insti- tute Type: Beta 505 King Avenue Columbus, Ohio 43201 Battelle's Survey assesses the perceptions of the broad educational community in terms of 1) the way ser- vices and programs currently Operate and 2) the way they should operate in the future. Instruments, designed for use in elementary, sec- ondary, junior and community colleges, gather perceptual data in a systematic way from the community within a 958 confidence level. Data is analyzed and reported back by numerical rank ordering of each item as it is perceived by the various participants. The elementary and secondary items are organized under sixteen function clusters: 1) Personal DeveIOp- ment of the Student, 2) Educational Program, 3) Individ- ualizing Instruction, 4) Instructional Management, 5) Guidance and Counseling, 6) Instructional Management, 7) Formulating Policy, 8) Planning, 9) Innovating, 10) Com- municating, ll) Supervising, 12) Solving Problems, 13) Staff Development and Board Organization, 14) Managing Facilities and Resources, 15) Budgeting, and 16) Eval- uating. 232 The Community/Junior College instruments classify the items under twelve function clusters: 1) Communi- cating, 2) Community Services, 3) Evaluating, 4) Goals, 5) Guidance, 6) Instruction, 7) Managing Personnel Relations, 8) Managing Resources and Materials, 9) Participative Decision-Making, 10) Planning, 11) Staff Development, and 12) Student Services. The procedures for the process are basically as follows: 1. The Center for Improved Education (CIE) pro- vides instruments specifically designed with the categories of participants (e.g. learners, parents, support staff). 2. A CIE consultant provides detailed instruction to the local coordinator. 3. The local team, consisting of a district co- ordinator and approximately 30 volunteers, distribute and pick-up the completed forms. 4. CIE codes the data and statistically analyzes it by computer. 5. CIE staff analyzes open-ended comments and correlates them into the printed report. 6. CIE returns to school with agreed to number of printed forms and explains how to interpret and use data for local analysis. The Battelle surveys seem to present an assessment process of the Beta variety that provides a rank- ordering and comparison of needs. A 'need' is defined as the difference between the participants opinion about 233 what currently exists and what should exist. The process seems very heavily dependent upon the perceptions of participants. Since the main purpose seems to be one of ”concerns", the instrument might prove a valuable starting referent to a more comprehensive and objectively based assessment. 234 Model: Source of Information: Dallas Planning Model Lawrence Ascopf Communications and Community Relations Department m a... 3233.???{33232 Designed as an aid in the annual budgeting process, the Dallas planning process seeks to involve educators, students, parents, the community and the Board of Educa- tion in assessing needs, assigning priorities and allo- cating resources (See figure). The process is to be initiated so that it coincides with the budget cycle. The assessment is meant to be a continuous process that evaluates each year's progress toward the completion of established long range (ten year) goals. Program managers, appointed to insure con- tinuous progress of the planning process for each of the district's seven long range goals, first evaluate accom- plishments of the educational program against the previous year's goals. These findings are reported to Operation Involvement groups, consisting of represent- atives from each school's teacher advisory comittees, students, parents, principal representatives, central office staff, other employee groups, and citizens who are not parents. The Operations Involvement groups, meeting monthly, evaluate the condition of current pro- grams and activities and the desired future condition for each item in question. Both current and desired 235 conditions are rated on a fifteen point scale. After survey results are compiled for each interest group and a group as a whole, priorities desired and the greatest gaps are indicated and presented to the Board of Education. Representatives of each Operation Involvement group meet with the Board on a weekend retreat to respond to and revise compiled needs. Gaps are identified, representing the discrepancy based upon these ”new needs". From these needs, goals are established for the coming school year, plans are drawn for their accom- plishment, and an accompanying budget is estimated. Each goal is examined for payoff and risk. This information is communicated to the Operation Involvement groups. At a second retreat, representatives and the Board assign rank order to the goals, according to previously established goals, and assign the necessary resources for their accomplishment until funds are ex-hausted. Each goal is restated into operational objectives by Managers, using standard district budget terms, the proposed budget is presented for public hearing, adjust- ments are made, and it is adopted by the Board. The process is designed to include a broad repre- sentation from among professional educators, students and community. Its costs are relatively low. Its only major time constraint is that it be initiated so that 236 it coincides with the budget cycle. Little special training and outside consultation are necessary. It has proven very successful in broad Dallas area application. It should be noted however that there is little indication that the establishment of the initial long range goals were based on little more than the perceptions of those involved. These goals' validity and utility should be Open to some question. The strong emphasis on previously established goals and priorities used as the standard against which progress is measured might indicate that the standards themselves are not being continually subjected to questioning. Also, the fact that the assessment is based upon the previous year's progress (possibly evaluating sub-goals, not goals), making it program-oriented, may further indicate a lack of assessing needs that may exist in those overall goals. Even with broad representative input, if data is analyzed according to a limited set of sub-goals, those areas of need, or gaps between currently established educational outputs and the outputs necessary to “contribution and survival" in the society external to the school, may not be uncovered. For these reasons, the Dallas model could likely be classified a Beta type needs assessment model. i I}! . . . Ill-III; i I i . iv! ‘ I! case's-saline. 1— — 3.1.3.38..- 3. slog-”la Ila-Bola _ fl easel-ea. 81.: a. “ eels-"II: . , "i i‘ l is. all! .- gulls-IA on. all-Illa 1: 4 Tel-Jello...- in!!! go. no}; . to"! one!" a ell-I all!!! - ..I-atlas 15.5.6. cal-xi.- — ell: sisal-II... 3 III! Illegal-Ia oils-Ill]. > call“) 3‘ I..." '9 ‘0“! .a Ii {lids-alas 0 is...» l alien III-..I} l ales-Ilsa..- nae-Ila! -_ E'U'i iii 1! l . i la 13' oil calla-all... cafes-lase- lies! ..I-.80.] g a ll» . In} In: 238 Model: Source of Information: The Mott Foundation Com- David Feldman, Dean munity College Model School of Business United States Inter- national University San Diego, Ca. Type: Beta Made possible by a grant from the Charles Stewart Mott Foundation, Flint, Michigan, this needs assessment sought to perform a sort of educational “market analy- sis“ in conjunction with the services offered by the San Diego Community College District. The assessment's def- inition of 'need", considered based upon Kaufman's, was the discrepancies (gaps) between perceived current out- comes and desired outcomes of the partners in the educa- tional community. Its purpose was the gathering of in- formation necessary for making decisions regarding plan- ning District programs, facilities and services to meet the needs of the community. Outside consultants were used to help design the necessary surveys. Specific questionnaires were devel- oped for the general community, community leaders, the staff of the Community Colleges, current students, and former students (graduates). These were based on the following requiremental guidelines: 239 -Current students--you've selected our college. How might our programs be improved to reflect your needs? -Former students-- you've experienced our programs. How do you rate the training and education you received? -Community--your tax dollars help support our college. What can we do to make our services more readily available to you? -Staff--you're our delivery subsystem. You have direct contact with our students. What are your perceptions of the needs of those students and the needs of the community from which they come? Community and student participants were chosen by a stratified, random sampling technique. A cumulative sampling of community leaders was derived from standard sociological indices and various advisory groups. The former students surveyed were those most recently gradu- ated. All regular-monthly staff of the District were included. Care seems to have been taken in not only the development of the questionaires used but in survey technique. Only experienced surveyors and those well trained by the District were allowed to administer the surveys. Earlier in this description of the Mott, or San Diego, Community College Model, the definition of 'need' upon which this process was based was offered: The dis- crepancies between perceived current outcomes and desired outcomes. The outcomes of such an assessment may mereley result in a ”wants list" rather than a well documented list of needs based upon an external 240 referent. This model might be classified an Alpha but it should be noted that the original surveys developed were done solely by the educational staff and outside professional consultants. This, depending upon the extent to which the survey limited the input Of the participants, could have been considered a referent set of goals moving the process more into the realm of Beta. 241 Model: Source of Information: Illinois Problem Index Illinois State Board of Education 100 North First Street Springfield, Illinois 62777 Type: Beta Directed in 1975 to develop an uncomplicated approach to needs assessment characterized by ease and rapidity of administration while sound in theory and practice, the Illinois State Board of Education devel- oped the Illinois Problem Index in 1977. In 1977-78 alone it was used by approximately 60 school districts. Not intended to ensure the accountability of the dis- trict to the State Board of Education, the IPI Offers a very flexible approach to the needs assessment process exploring alternative techniques. Incorporating data collected from a broad cross section of the community from a series of three surveys, the IPI process attempts to: -identify the educational problems currently per- ceived by various constituencies -eva1uate these perceived problems in the light of existing evidence -assess the desirability of expending available re- sources to solve these problems. 242 The IPI requires seven steps (see figure). The steps are clearly defined and involve organizing a com- mittee, conducting five meetings, administering three assessment instruments, and acting on the basis of find- ings. These can be completed in a two month period. The IPI is an unusually well presented and complete assessment process of the Beta variety. The accompanying materials and methods are reasonably comprehensive, easily understood, and appear to be a good means for accomplishing the purposes for which they were designed. Included in each stage of the assessment is a step-by-step discussion of why and how the process or instrument can be used. The Illinois State Board of Education offers assistance with any or all of the steps. 243 — «co-la!— tsaiea e... 85258 auto '38:; «E mmmocfla— m:.—. i mam—Q7: mESmOmh m—OZE— mum—h. 244 Model: Source of Information: Ohio Needs Assessment Division of Educational Services Block Grant and Basic Skills Section Type: Beta Room 802 65 South Front Street Columbus, Ohio 43215 Developed to enable school district staff members in planning and conducting a needs assessment in compli- ance with the eligibility requirements for federal fund- ing under Title III, ESES, the Ohio Needs Assessment Guidelines can also be useful in the areas of educa- tional planning and school-community communications. The procedures presented in the explanatory materials are based upon the surveys of literature on needs assessment practices and capabilities of Ohio school districts. The following criteria for the development of the needs assessment procedure were suggested by the surveys: -use the ”discrepancy” approach to determine needs -include a method for assigning priorities to iden- tify needs -obtain input from parents and other coumunity mem- bers in addition to input from professional educa- tors and students -begin with broad statements of educational goals -assess the needs in the affective and psychomotor domains as well as cognitive areas -involve the setting of expected levels of student 245 achievement of goals -use both objective and subjective data to determine needs The following figure, Assessment Guidelines booklet, in the Ohio process, gested by the surveys:- Step One -- Step Two ~- Step Three -- Step Four -- Step Five -- Step Six -- Step Seven Step Eight A Needs Assessment Procedure Establish a Needs Assessment Committee Prepare statements of educational goals Conduct a survey to determine perceived educational needs Assign priorities to perceived educational needs Set desired levels of student achievement Determine actual status of student achievement Compare actual status with desired levels Assign priorities to educational needs Because the survey is developed based upon existing general goals and the philosophy of the educational sys- tem in question or those of another district, taken from the Ohio Needs presents the major steps designed to meet the criteria sug- the Ohio 246 model could be classified a Beta model. The explanatory materials available with this model are among the most complete and easily understandable of any reviewed for this study. It should be noted, however, that though a lot of care was taken to enumerate and explain the above suggested steps in the assessment procedure, that the results of the process are dependent upon the “perceptions” of the participants almost to the exclusion of objective validation and on the decisions of a ‘Needs Assessment Committee whose dictates may or may not be truly reflective of the community's needs. 247 Model: Source of Information: Quality Education Program Bucks County Schools Study Intermediate Unit NO. 22 Cross Keys Building Route 611 and 313 Type: Beta Doylestown, Penn. 18901 Designed to define and clarify the Ten Goals of Quality Education recommended by the Educational Testing Service and adopted by the Pennsylvania State Board of Education in 1965, the Quality Education Program Study, undertaken in 1968, was funded under E.S.E.A. Title III. Q.E.P.S. used the Critical Incident Technique (Flanagen, 1954) to collect information from students, teachers and parents to enable their staff to write the goals into specific performance terms that could be used to develOp valid indicators for l) assessing student behaviors, 2) providing rationale or narrative explanations, and 3) assessing strategies. One booklet was developed for each of the Ten Goals which provides instruments by which the "practitioner" (learner, parent, teacher, administrator) can measure performance Of the individual learner or instruction. The assessments can be applied to determine the needs of from one learner to a total school system. There is a general and an individual needs assessment instrument for each goal. Both rely upon a five point rating scale to determine needs and assign them a priority. 248 The needs are determined according to the individ- ual's or system's current output as measured by the Ten Goals. This places the model in the Beta category. Although care was taken to design and carry-out a study to clarify and define those goals under the auspices of consultants from U.C.L.A., the American Institutes of Research, the University of Alabama, and the University of Nevada, Las Vegas, there is no mention of the valida- tion of the original Ten Goals. The booklets themselves Offer some good general explanations concerning some of the rationale behind and the methods for conducting a needs assessment but are seriously lacking in some of the detailed explanations and suggestions that a practi- tioner would likely require. 249 Model: Source of Information: Alameda County Needs Joy Richardson Assessment Model Instructional Support Services Office of the Alameda County Superintendent of Schools Typg: Beta 313 W Windon Hayward, Ca. 94544 Originally designed by Dr. Ruth Belle Whitkin to assist California schools to assess needs for planning programs funded under a consolidated application for ESEA Title I and other Federal and State funds, the model's use has been expanded for use in the planning Of any program (K-6). Needs are assessed in nine basic areas: language development, reading, mathematics, multicultural education, bilingual/bicultural education, staff development, parent participation, parent educa- tion, and health/auxiliary services as well as affective and psychomotor domains. The data gathered yield numer- ical indices of criticality of need. Materials, devel- oped for both reader and non-reader, are available in English, Spanish, Chinese, and Tagalog for students. Materials available are: l. a manual with directions for sampling pro- cedures, a decision matrix for data reduction, and an action graph to facilitate decision making, 250 2. a supplement sequencing the major tasks of the assessment from start to "ready for planning", 3. data forms for putting statistical facts to- gether in usable form, 4. machine scorable surveys in the four languages, 5. explanation of the data. Produced is a computer print-out with three types of summaries (component, area, and item) separately for parents, students, and staff. The process is reportedly easily administered, practical and offered as a low-cost processing service (.45 each for parent and staff survey and .36 for student's). The ACNAM appears to be a well devised and complete model of the Beta variety. 251 Model: Source of Information: Institutional Goals ETS College and Inventory University Programs Box 2811 Princeton, N.J. Type: Beta The Institutional Goals Inventory, developed by the Educational Testing Service for the purpose of defining educational goals and establishing priorities among those goals, is a needs assessment aimed at planning and evaluation on the college level. The process, admin- istered to the college community and the general commu- nity in which the college is situated, can be conducted with available staff, especially an Office of Planning or Office of Institutional Research experienced in the collection and interpretation of data. Information, in the form of a five point rating scale (from 'Of No Importance“ to 'Of Extremely High Importance“), is gathered concerning 90 goal statements. Up to 20 additional goal statements can be written locally. Up to five sub-groups (e.g. faculty, students, administrators, alumni, and trustees) can be included in the 161 report. The goals focus on the 'Is' and ”Should Be' of both present and future conditions of the college. The process, of the Beta variety, consists mainly of the following steps: 252 Institution orders IGI booklets/answer sheets at least four weeks in advance. Institution selects sample, subgroups, writes Optional goal statements and supplementary questions, distributes, collects and returns completed forms to ETS for scoring and report- ing. In two to three weeks a bound report is returned to the institution containing: -Interpretive Guide -Information and Supplementary Information Questions -Twenty Goal Areas summarized for both Is and Should Be responses with means, standard deviations, and discrepancies -Goal Area Summaries rank ordered by Should Be means -Goal Area Summaries rank ordered by Is means -Goal Area Summaries rank ordered by discrepancies between Should Be and Is means -Ninety Goal Statements with Is and Should Be responses tabulated by percentage Of responses to each alternative with mean, standard deviation, and discrepancies A planning committee, involved since the beginning of the process, uses the data from the report for planning. The process should be supported by the insti- tution's president and publicized widely to encourage response. The results should be widely’ distributed and used as the basis for continuing discussion of institution's goals and planning. The costs of the IGI are machine scorable booklets (.45 each) and scoring ($1.50 per booklet). There is a $200 minimum for scoring of booklets that appears on any one report. 253 Model: Source of Information: Phi Delta Kappa Wilmer K. Bugher, Asc. Planning Model Executive Secretary for Administration Phi Delta Kappa Eigth Street and Type: Beta Union Avenue Box 789 Bloomington, IND 47402 The main purpose of the Phi Delta Kappa process (see figure) is to determine how well the current out- comes of an educational system compare to the eighteen educational goals developed in 1969 by the California School Boards Association. PDK believes these goals to be all encompassing. At an initial community meeting, the goals are ranked through a forced—choice technique by 40-80 commu- nity members. The group represents a broad cross section of the community members. Individuals prior- itize the goals and re-rank them through a group consensus process. Scores are averaged to arrive at an overall ranking. .At a second meeting, a fifteen point scale is used to rate how well the schools are currently meeting the prioritized goals. At. similar meetings, professional staff and students do the same. An analysis of variance is taken to indicate the agreement or disagreement on each goal. Mean scores are taken of the individual rat- ings to aid teachers, administrators and the board in determining needs. If there is incongruence between 254 standardized test scores and other data and individual perceptions, this information is communicated to the participants. When goals are identified and prioritized, and it is decided which goals are not being met as well as others, they are turned into performance objectives. PDK provides a programmed course for writing such objec- tives. Since these Objectives state how the desired outcomes of learning will be :measured, evaluation is facilitated. Though the PDK kit contains all the materials needed to implement the needs assessment, workshops, training district representatives in the process, have been found very useful. ‘No further consultation is needed. The goal ranking and school performance rating seem better validated than that undertaken in the Fresno model, which bases its own consensus. Like the Dallas model, the PDK model begins with a pre-determined list of goals. This would likely qualify it as a Beta type model. Though there is allowance for the addition or deletion of the goals on a local basis, there seems to be little assurance that the resultant program, based upon centralized goals, will reflect the distinct needs of every local district. The goals may prove to be all ”encompassing", but if care is not taken to periodically re-evaluate and up—date these, there is no guarantee 255 that they will remain so when applied to a broad spec- trum of settings in a changing world. 256 3.355111.— _ In: 380l— 15 is; a):- Sloln if; 8‘51“- } 8'02" 3 gill! — _ _ 3...... 133 III-c in... lav-lid :- 1.2-In ..lo. Issac-.4812: ¢ I; 3%} . 3'" ii] 1F lilo: III-oi iv — — 11- 33 .921 ii will. Biol-Ni ..I—18:5 .on—III." — 53.3 Ilsa!— ilsso 1.8] ales-.3 g Ila-oi 3 1.31; H — i is _ ’3 Ii: '3 all!!! sals- is: Inst—3.51 alga-Ila: 18"! 1h: all; 1}: 5: plaice _ i i is i-El in I i C! I 3 awn—OE UZ—ZZSQ (was (HAM—G :mm 257 Model: Source of Information: Sensing Educational William G. Spady, Needs in the Far West Laboratory Director Region Far West Laboratory for Educational Research and DevelOpment Type: Forerunner to 1855 Folsom Street other assessments San Francisco, Ca. 94103 Sensing Educational Needs in the Far West Region (SENFWR) was a needs assessment undertaken by the Far West Laboratory for Educational Research and DevelOpment (FWL) to increase the Laboratory's responsiveness to educational needs in California, Nevada, and Utah (1980). It was then not concerned with problem-related or problem-domain-related needs. Though needs may relate to these domains, regional needs may also include needs that cut across programs and problems dealing with needs with which the Laboratory is not working. Once the needs were identified they were ranked by the FWL Board of Directors and considered in relation to institutional strengths and weaknesses. The Institutional Support Program (ISP) is a branch of the department established in 1979, for the purpose of sensing needs by “identifying effective approaches that could be implemented." The method implemented in this case was Flanagan's Critical Incident Technique (1954) which has been used for many years in solving industrial, community and educational problems. Infor- 258 mation was gathered first on a trial basis from FWL staff and then field tested with operators of regional educational programs and individuals who benefit from these programs. Tryouts were conducted to determine if this tech- nique was useful in generating information in the field. A second objective was the usefulness of three different methods for collecting critical incident data. Face to face data collection proved expensive, so group meetings and telephone interviews were also tested. One hundred and three incidents were recalled by forty administra- tors, teachers, students, parents, and community agency personnel at three regional sites. Tryouts indicated that the Critical Incident Tech- nique was useful in gathering information concerning needs (McGrail and Chow, 1980). Negative incidents were more useful in providing action-oriented information than were positive. Factors contributing to success were often not apparent. Therefore, emphasis on gather- ing negative incidents, using this technique, seemed warranted. It was further indicated that face-to-face inter- views, group meetings, and telephone interviews were all useful methods. Group meetings, were considered most productive because of the participants included during a given period of time. Some difficulty was experienced in reaching individuals by telephone. The in-person 259 meetings, both individual and group, resulted in greater description of events than did telephone interviews. But it was concluded that if in-person interviews are not feasible, that the telephone method was useful in gathering need-related information. This model was more a forerunner to a possible Alpha type needs assessment. It sought more to validate the means to an assessment of actual educational needs than to gather information for an actual program evaluation. Plans were under way to conduct an actual assessment of the Laboratory's needs in 1983. 260 Model: Source of Information: Worldwide Planning Dr. Geraldine H. Plumb Model Coordinator, Federal Programs Boise Schools Type: Alpha Boise, Idaho The involvement of teachers, administrators,‘ stu- dents and community members in needs assessment is the main purpose of the Worldwide Planning Model (see figure). Representatives from the community implement and manage the assessment. Unlike the PDK and Dallas models, the Worldwide model does not begin with established goals but suggests starting with a stated philosophy or very broad educa- tional goals. The needs assessment is first conducted and goals are written based upon these findings. Needs are assessed through five sources of infor- mation: 1. A Survey of Opinions. These tap the commu- nity's perceptions and concerns. 2. Existing test scores, data and other eval- uation measures to assess student achievement. 3. Community Concerns Conferences to discuss con- cerns about schools. 4. Speak-Ups which give students an Opportunity to voice and discuss concerns. 5. Administrative Data and Evaluation (e.g. growth projections, drOp-out rates, parent participation) that may indicate needs. 261 After the data is collected, organized into common areas, documented with policies and facts, it is stated in terms of learner needs and the needs prioritized. From the list of prioritized needs, goal statements are written, problems are defined and analyzed, and al- ternative solutions are suggested. After possible solu— tions are selected, implementation begins. Evaluation, based upon measurable objectives, is an integral part of the WOrldwide model. Clear and easy to understand manuals are provided describing in full detail all of the steps outlined in the process. The model can be classified as an Alpha type and, unlike the Fresno model, is not merely problem-oriented. It appears that more care is taken also to validate needs and to check on their utility in terms of the community outside the school. It might be noted, though, that some discrepancy between the narrative and graphic presentation of the model seems to exist, in that, in the graphic, "Goal Determination” proceeds the ”Needs Assessment". The "Goals" represented here, though, may be the suggested starting “philosophy“ or “broad goals'. There also seems to be lacking the necessary feedback between all the steps subsequent to the needs assessment and the needs assessment itself. Such feedback relates both a 262 continuous monitoring of needs, from both the perspec- tive of detecting “new” needs and the evaluation of those previously indicated. 833.5 e... base} a. 1.. s... 53.. J areas-n «salad 18 axes. in? «.83 a .88 cl... 80.38 ‘1 .1... .see 08.3.3...“ 003.22 0056390 w 0055.30 .8880 4. in. sham 11 1.6 21.6 4 o 0 so Ed .38.. auto 33.2 3301.0: 2.2m 500.6 a... 4L :5... 3...: (A... in,” .32. 568.: I... 838 If 3 £02.10. .838 243—OSA— tEthfZax—h «EGGS 02—225.— mun—EEO? 264 38: «.+ J 1 3800+ : 3 . z 23; fi .132 A . — shew... ~ soap... H 682 2059 EEHU- of Mastery —->- Possible Indicators Effects Performance Causes Cost! Solution Evidence 4" (bjectives Effectiveness/ '4— Alternativee ‘— and Test of Time. etc. Remedy Hypothesis (Tr imby , 1979) 285 Model: Source of Information: Skyline West Dallas Independent Educational Plan School District Office of the Gen- eral Superintendent 3700 Ross Avenue Type: Futuring Dallas, Texas 75204 The mission of the SWEP project (1974) was to "examine, conceptualize, and describe the secondary school in the years 1980-2000." It was necessary to first attempt a description of the society of the society of the 1980's with a special emphasis on the impact of that society upon education. It was assumed that societal changes would require educational atten- tion concerning worker's skills, citizen attitudes, cer- tain traditions, knowledge and values. The school was looked upon as only a part of that necessary educational system. Through a modified DELPHI technique, information was gathered through two survey instruments. One instrument was used to assess the future events that would confront the high school graduate in the 80's and beyond. The other was used to generate high priority goals for that same time period. The results of the needs assessments was future oriented data that was presented to those responsible for making planning decisions for the two county Dallas/Fort WOrth area. Information was gathered and analyzed concerning demographic and legislative trends, economic trends, 286 both locally and nationally, government and regulatory agencies, social trends, employment, pOpulation trends, enrollment expectations, cost of schooling (payroll, energy costs, non-classroom expenses, legislation and policy), technology and international demands. This data was used to delineate priorities in six goal areas: improved student achievement, increased employee per- formance, to provide accountability, to promote school- community relationships, to maintain an effective gov- ernance system, and to ensure adequate funding. The reports generated by the SWEP project were not intended as a ”blueprint“ for specific schools to be established. It was hoped that it would, instead, pro- vide input to those responsible for decisions that had long term consequences. The use of such input might provide for the removal of much of the uncertainty that accompanies decision making. The methods and procedures involved in this ”futur- ing' needs assessment, could provide very useful input to an Alpha type needs assessment. Its purpose is not to establish goals in the area of educational outputs or to point out any discrepancies between current and desired system production. Resultant information could prove beneficial in discovering such needs but does not in itself provide such an assessment. APPENDIX B Cover Letter and Questionnaire for Survey 287 288 GARY B. WANAMAKER 334 E. Elm Mason, MI. 48854 Feb. 26, 1984 Dr. David Feldman, Dean School of Business United States International University San Diego, CA. Dear Dr. Feldman, I am currently completing my dissertation at Michigan State University in the Educational Systems Development program under Dr. Castelle Gentry. My topic concerns needs assessment models. Of the 36 questions I used to analyze the Mott Foundation Community Model, reported in Educational Technolo , November, 1977, I was unable to answer the following 10 from the literature that I have gathered. I would appreciate it *very’ much, and it would be very helpful, if you could take a few minutes to complete the accompanying questionnaire and return it to me. It has been designed to take as little of your time as possible. If you check the category marked “other“, it would be useful if you would provide a little explanatory information on the lines provided. You need answer only those questions circled. Thank you, 838M 289 Are the services and/or materials connected with this model still available? If yes, to whom? What is the major purpose of the model? At what educational level is the assessment aimed? ‘What pre-assessment planning stages, or considerations, are addressed? yes no inhouse use only general public other involve parents, learners, educa- tors, and commu- nity problem detection decision making budgeting other elementary middle secondary higher adult industrial train- ing other introduction of process to public and educational system. review of simi- lar, previous efforts review of models locate ap ropri- ate decis on makers identify resources needed identify resources avail- able locate data already avail- able other 290 What are the final outputs of the assessment? Who are the sources of information used to de— termine needs? How are the information sources chosen? In what aspects of the assessment do the various categories of participants (e.g. learners, parents, community) engage? What methods are suggested for deciding which ”per- ceived needs“ are in fact needs? list of priori- tized needs list of priori- tized goals list of needs (gaps) selected for closure other administration educators parents learners general community experts in perti- nent fields other randomly according to com- munity demo- graphic charac- teristics other sources of infor- mation aid in deciding upon needs validation of needs prioritize needs select needs (gaps) for clo- sure budgeting other consensus among participants arbitrary choice by administration majority vote committee other 10. ll. 12. 13. 14. 15. “291 What staff or committees are required to carry out the assessment? Are the staff/committee responsibilities outlined in materials available to users of the needs assess- ment model? What costs are involved in the assessment? What formal training is required to conduct the assessment? Are outside consultants necessary to the assessment? If yes, are they available? What are the instruments of the assessment? one coordinator more than one co- ordinator a committee con- sisting of repre- sentatitive members of the comunity other yes no printing/copying materials mailing substitute teach- ers under one day per month of adminis- trator or teacher time kit or materials (amt: ) other workshop(s) training of coor- dinator(s) general orienta- tion other yes no yes no survey questionnaire interviews Delphi technique other 16. 17. 18. 19. 20. 292 Do you provide the neces- sary instruments? How long does it take to complete a typical needs assessment with your model? Are needs prioritized? Are needs validated through empirical means? If yes, which means? What interrelationships between the needs assess- ment and other stages of the educational process are specified? yes no other less than a week less than six months less than a year other yes no yes no standardized test scores criterion refer- enced test scores related experts observations related docu- ments (e.g. gov't employment re- cords) other setting of goals setting of objec- ives budgeting staff/facility allocations evaluation selection of alternative pro- grams selection of methods basis for future assessments other 21. 22. 23. 24. 293 To what extent has the model been field tested? Has research been conducted yes concerning the model or no any of its aspects? If yes, what were the results of the findings? Approximately what percentage Education of the model's use has taken Industry place in 294 BIBLIOGRAPHY Alexander, Lawrence T. and Stephen L. Yelon, "The Use of a Common Experiential Reference in Instructional Systems Design“, The Educational_Technol Review Series: Instructional Systems. Englewood Cliffs, New Jersey, 1969 Alkin, M.C. "Evaluation Theory Development“, Evaluation Comment, 2(1), 1969, 2-7 ---------- ”The Evaluation of Educational Technology“, in Improving Productivity of School Systems Through Educational Technology: Final Report of Symposium. Philadelphia: Research for Better Schools, 1973 ---------- , and Carol T. Fitz-Gibbon, ”Methods and Theories of Evaluating Programs', Journal of Research and Development in Education, Vol. 8, No. 3, 1975 Alameda County School Department International Society of Educational Planners, National Institute of Education. "Educational Needs Assessment: Theme and Variations", Alameda County Office of Education, Publications, 1984 A Nation at Risk; the Imperative for Educational Reform, A Report to the Nation and the Secretary of Education by the National Commission on Excellence in Education. April, 1983, or The Chronicle of Higher Education, May 4, 1983 R.C., Anderson, R.J. Spiro, W.E. Montague, (eds.) School and the Acquisition of Knowledge. Hillsdale, New Jersey: Erlbaum Associates, 1977 Armsey, James W. and Norman C. Dahl, An Inquiry into the Uses of Instructional Technology. New York, Ford Foundation, 1973 Axtelle, G.E., ”The Humanizing of Knowledge and the Education of Values", Educational Theory, XVI No. 2, April, 1966 Ausubel, D., J. Novak, and H. Hanesian, Educational Ps chl : A Cognitive View. (2nd ed.). New York, Heft, Rinehart, and Winston, 1978 Banathy, Bela. Instructional Systems. Palo Alto, CA: Fearon, 1968 295 Bell, Terrence A Nation at Risk: the Imperative for Educational Reform, A Report to the Nation and the SEcretary of_§ducat1on by the National Commission on Excellence in Education. April, 1983 or The Chronicle of Higher Education, May 4, 1984 Bertalaffy, Ludwig Von. ”General Systems Theory, a Critical Review", in General Systems. New York, G. Fraziller, 1968 Bleau, Barbara Lee, “Planning Models in Higher Education: Historical Review and Survey of Currently Available Models”, Higher Education, Vol. 10, 1981, 153-168 Bloom, 8.3. “Learning for Mastery", UCLA Evaluation Comment, 1968, 1(2) Boroser, Samuel E. and Eleanor R. Robinson, “Needs Assessment Study of Vocationally Related Needs for Secondary Special Education", Educational Technology, November, 1977, 43-46 Bouch, Gary D., 'A State of the Art Assessment of Educational Evaluation“, (ERIC Document Reproductive Service, E.D. 187 717) Branson, R. K. "Military and Industrial Training" , in L. J. Briggs (ed. ), Instructional Design:P rinciples and Applications. Englewood Cliffs, New Jersey: Educational Technology Publications, 1971 Bretz, R. A Taxonomy of Communications Media. Englewood Cliffs, New Jersey: Educational Technology Publications, 1971 Briggs, L. J., and W’.W. Wager, Handbook of Procedures for the Design of Instruction, (2nd edition). Englewood C11ffs, New Jersey: Educational Technology Publications, 1981 Brodinsky, Ben, "The New Right: The Movement and Its Impact“, Phi Delta Kappan, October, 1982 Buhl, Dr. A.U. ”Assessing the Assessment Movement”, School and Community, February, 1978, 17-19 Bureau of Technical Assistance. Rhode Island Department of Education Needs Assessment Composition of Abstracts, (Providence, R.I., ERIC Document Reproduction Service, ED 153 351, 1978) 296 Burton and Merrill, Instructional Design: Principles and A lications. Educational Technology Publications, 1977 Calder, J.R., ”In Defense of the Systematic Approach to Instruction and Behavioral Objectives", Educational Technology, May, 1980 Churchman, C. West, The Systems Approach. Delacorte Press, New York, 1968 Carroll, J.B. "A Model of School Learning“, Teachers College Record, 1963-64, 535-549 Carron, Andrew S. "Needs Assessment in Government: Rearranging the Deck Chairs in the Ship of State“, Educational Technology, November, 1977, 27-28 Coffing, T.R., "Client Needs Assessment" in D namic Educational Change. (Zaltman, B. et al., ed1tors). Free Press, 1977 Cooke, Robert A. and Denise M. Rousseau, "Problems of Complex Systems: A Model of Systems Problem Solving Applied to Schools“ , Educational Administration Quarterly, Vol. 17, No. 3, Summer, 1981, 15-41 Cortright, Tom D. “A Systems Approach to Managing Complex Educational Projects“, Educational Technology, January, 1979, 30-32 Corrigan, R.E., A S stems Approach for Education (SAFE). Garden Grove, CaIIfornia, R. . orr1gan ss., Cronbach, L.J. “Course Improvement Through Evaluation”, Teachers College Record, 1963-64, 672-83 Crumbaugh, J. C., and L. T. Maholick, Manual of Instructions for the Purpose in Life Test. Brookport, Ill. Psychometric Affiliates, 1969 De Cecco, J. P., and W. R. Crawford, The Psychology of Learning and Instruction (2nd edition). Englewood Cliffs, New Jersey: Educational Technology Publications, 1977 Deutsch, Karl W. "Toward a Cybernetic Model of Man and Society“, From some notes on Research on the Role of Models in the Natural and Social Sciences. Synthese: Vol. 7, 1948-49, 506-533 297 Dick, W. “Formative Evaluation“ in L. J. Briggs (ed. ), Instructional Design: Principles and Applcations. Englewood Cliffs, New Jersey: Educational Technology Publications, 1977 ---------- , and Lou M. Carey, “Needs Assessment Instructional Design", Educational Technology, November, 1977, 53-59 Dick, W., and L. Carey, The Systematic Design of Instruction. Glenview, 111: Scott, Foresman and Company, 1978 Dixon, George, Rev. ”The Deliberate Sabotage of Public Education by Liberal Elitists', Phi Delta Kappan, October, 1982 Duncan, James K., “Traditions in Educational Evaluation-- America Style", The Educational Forum, January, 1980 Educational Technology: Definition and Glossary of Terms Vol.1. AECT Task Force, Association for Educational Communications and Technology, 1126 16th St., N .W., Washington, D.C. 20036, 1977 English, Fenwich W. "Politics of Needs Assessment", Educational Technology, November, 1977, 18-23 Fessler, Ralph, “Moving from Needs Assessment to Implementation: Strategies for Planning and Staff Development“, Educational Technology, June, 1980 Flanagan, John C., "The Critical Incident Technique“, Psychological Bulletin, LI, No. 4, July, 1954 ---------- , 'Individualizing Education“, Education, XC, No. 3, February-March, 1971 Frankl, V.L. Man's Search for Meaning: An Introduction to Logotherapy. Boston: Beacon Press, 1962 ---------- , The Doctor and the Soul: from Psychotherapy to Logotherapy. (2nd ed.) New York: Alfred A. Knopf, 1965 ---------- , Psychotherapy and Existentialism: Selected Papers on Logotherapy. New York: Washington Square Press, 1967 ---------- , The Will to Meaning: Foundations and Applications of Logotherapy. New York: the World Publishing Company, 1969 298 Frederich, Otto, ”Five Ways to Wisdom“, Time, September 27, 1982 Freidman, Milton 'Gammon's 'Black Holes", Newsweek, November, 1977, 90-94 Gage, N.L., and D.C. Berlinger, Educational Psychology, (2nd edition). Chicago: Rand McNally, 1979 Gagne, R.M. Essentials of Learning for Instruction. Hinsdale, Ill: Dryden Press, 1975 ---------- , The Conditions of Learning, (3rd edition). New York: Holt, Rinehart, and Winston, 1977a ---------- , "Analysis of Objectives", in L.J. Briggs (ed.) Instructional Design: Principles and Applications. Englewood Cliffs, New Jersey: Educational Technology Publications, 1977b ---------- and L.J. Briggs, Principles of Instructional Desi n, (2nd edition). New York: Holt, Rinehart, and W1nston, 1979 Gerlach, Vernon S., and Donald P. Ely, Teaching and Media: a Systematic Approach. Englewood Cliffs, New Jersey: Prentice Hall, Inc., 1971 Glaser, R. 'Components of a Psychology of Instruction: Toward a Science of Design”, Review of Educational Research, 1976, 46, 1-24 ---------- (ed.), Advances in Instructional Psychology, Hillsdale, New Jersey: Erbaum Associates, 1978 Greenwald, H. Direct Decision Therapy. San Diego: Edits, 1973 Hannum, W.E. “Task Analysis Procedures”, NSPI Journal, 19 (3): 1980: 6’14 ---------- , and Leslie J. Briggs, "How Does Instructional Systems Design Differ from Traditional Instruction?" Educational Technology, January, 1982 Harsh, J.R., R.A. Kaufman, and L.E. Shuch, “A Proposed Continuation Model for ESEA Title III Projects", Task Force 8, ESEA Title III Office, California State Department of Education, Sacramento, California, 1968 Harless, J.B. An Ounce of Analysis sz Worth a Pound of Objectives). McLean, Val, Harless Performance Guild, 1975 299 Hartman, Ann, 'But All Knowledge is One: A Systems Approach to the Dilemmas in Curriculum Building”, Journal of Education for Social Work, Vol. 16, No. 2, Spring, 1980 Heldge, Doris J., Ph.D. and Lawrence W. Marrs, Ph.D., l'Needs Assessment--A Catalyst for Systematic Residential School Program Planning and Evaluation", American Annals of the Deaf, 1978, 123(4), pp., 478- 485 House, Ernest R. 'Assumptions Underlying Evaluation Models”, Educational Researcher, Vol. 7, No. 3, March, 1978' PP. 4-12 Huba, Mary E., Elaine F. McNally-Jarchow, Anton J. Neutsh, 'Evaluting Needs Assessment: Reactions to Phase I of the PDK Educational Planning Model", Education, Vol. 101, No. 3, Spring, 1981, 243-245 Jamison, D., Suppes, P., and Wells, S. ”Effectiveness of Alternative Instructional Media”, Review of Educational Research, 44 (l), 1974, 1-67 Kaplan, Bernard Needs Assessment in Education, A_P1anning Handbook for School Districts. Trenton, New Jersey, Bureau of Planning, February, 1974 Kaufman, Roger A. “A Holistic Planning Model. A System Approach for Improving Organizational Effectiveness", Performance and Instruction Journal, October, 1983, 3- 12 ---------- , 'A Possible Taxonomy of Needs Assessment“, Educational Technology, November, 1977, 60-64 ---------- , 'A System Approach to Education: Derivation and Definition”, AV Communication Review, Vol. 16, No. 4, Winter, 1968 ---------- , Educational System Planning. Englewood Cliffs, New Jersey, Prentice Hall, 1972 ---------- , “Needs Assessment in Perspective: Introduction to Special Issue", Educational Technology, November, 1977 ---------- , R.E. Corrigan, and D.W. Johnson, ”Toward Educational Responsiveness to Society's Needs--A Tentative Utility Model”, Journal to Socio-Economic Planning Sciences, Vol. III, August, 1969 ---------- , and F.W. English. Needs Assessment. Educational Technology Publications, Englewood Cliffs, New Jersey, 17632, 1979 300 ------ ¥---, M.J. Rand, F. English, J.M. Conte, and W. Haskins, 'An Attempt to Put the Ten Objectives of Education Developed for Pennsylvania by Educational Testing Service into Operational Definitions“, Temple City, CA., Unified School District, April, 1968 ---------- , and Robert G. Stakenas, "Needs Assessment and Holistic Planning", Educational Leadership, April, 1981 Klahr, D. (ed.) Cognition and Instruction. Hillsdale, New Jersey: Erlbaum Associates, 1976 Konczal, Edward F. "Planning for Training at ATGT”, Training and Develgpment Journal, April, 1978, 3-7 Kornfeld, L.L., Farris, R.G. and Olson, D.N. “Simulation Models technique for juggling the what ifs”, College and Univeristy Business, 53 (5), November, 1972, 31-37 Laswell, H.D. Eye Communication of Ideas. New York: Harper and Row, Publishers, 1948 Lee, Walter S. ”The Assessment, Analysis, and Monitoring of Educational Needs in Post Secondary Education“, Educational Technology, April, 1973, 29-32 Lenning, Oscar T. "A Conceptual Framework for Identifying and Assessing Needs in Post Secondary Education“, (Air Forum Paper, 1978, ED 161 341). Lessinger, L.M., and Connors, Thorough and Efficient. Raleigh, N.C.: Stewardship Press, 1978 Lippitt, Gordon L. Visualizin Chan e1 Model Building and the Change Process. NTL-Learning Resources Corporation, Fairfax, Virginia, 1973 MacDonald-Ross, M. Janus "Consultant: Educational Technology at the Open University“, British Journal of Educational Technology, 7 (1), 1976, 65-75 Mager, R. Preparing Instructional Objectives. Palo Alto: Fearon Publishers, 1962 Marshner, Connaught C. The Blackboard Tyranny. Arlington House, Westport, Conn., 1978 Maslow, A. Toward a Psychology of Bein , (2nd edition). Princeton, New Jersey: D. Van Nostrand Company, Inc., 1968 McGraw, Onalee, ”Where is the Public in Public Education?“ Phi Delta Kappan, October, 1982 301 Mayer, Hanna “Implementing Needs Assessment. Some Important Considerations", Performance and Instructional Journal, OctoEer, I953, 21-23 Melton, Raymond G. ”Applications of Needs Assessment in Public Schools, Three Case Studies", Educational Technology, November, 1977, 36-41 Miller, Lunn and Thomas E. Wolf "Staff Development for School Change: Theory and Practice”, Teacher's College Record, September, 1978 Morgan, Larry and David Feldman, “Needs Assessment in Higher Education: The Mott Foundation Community College Model“, Educational Technology, November, 1977, 48-52 Murphy, Norman and Margaret Mattin, “Needs Assessment in Higher Education: Starting up in a Small College”, Educational Technology, November, 1977, 46-48 "Needs and Priorities", in L.J. Briggs (ed.), Instructional Design: Principles and Applications. Englewood Cliffs, New Jersey: Educational Technology Publications, 1977 New Jersey State Department of Education, Trenton Division of Research. “Planning Handbook for Districts", Handbook Series on Compulsive Planning for Education Districts, (Trenton, NJ ERIC Document Reproduction Service, ED 189 405, February, 1974) Newsweek, 'Can the Schools be Saved?', May 9, 1983, 50-60 Norlen, Urban, Simulation Model Building. Stockholm: Almguist and Siksell, 1972 Peterman, Thomas A. 'A Compendium of Selected Research on the Application of Educational Media in the Classroom“, Dissertation, Michigan State Univeristy, 1982 Peters, Patrick Vincent, "Task Development and Evaluation of a Systematic Training Needs Analysis Model for Business/Industrial Training“, Dissertation for Degree of Ph.D., Michigan State Univeristy, 1980 Popham, W.J., and E.L. Baker, Systematic Instruction. Englewood Cliffs, New Jersey: Prentice Hall, 1970 ---------- , Educational Evaluation. Englewood Cliffs, New Jersey: Prentice Hall, Inc., 1975 302 Provus, M. Discrepancy Evaluation. Berkeley, Calif.: McCutchan Press, 1972 Ragland, Robert ”Models and Maps: the Lost City of Mars“, The Personnel and Guidance Journal, Vol. 60, No. 8, April, 1982 Randall, Robert S. “An Operational Application of the CIPP Model for Evaluation", Educational Technology, Vol. 09, No. 7, July, 1969 Roberts, Wesley K., Dathleen M. Daubek and John C. Johnston, "The Use of Needs Assessment Techniques for Establishing Training Programs Responsive to the U.S. Army's Role“, Educational Technology, November, 1977, 41-42 Rogers, C., “Toward a Modern Approach to Values: the Valuing Process in the Mature Person', The Journal of Abnormal and Social Psychology, LXVIII, No. 2, 1964 Ross, Peter E., Howard E. Freeman and Sonja R. Wright, Evaluation: A S stematic A roach. Sage Publications, Inc., Beverly Hills, CA., London, England, 1979 Rowntree, D., Educational Technology in Curriculum Development. London: Harper and Row, 1974 Rucker, W.R., ”A Value-Oriented Framework for Education and the Behavioral Sciences”, The Journal of Valgg Inguiry, III, No. 4, Winter, 1969 ---------- , V.C. Armspiger, and A.J. Brodbeck, Human Values in Education. Dubuque, Iowa: William C. Brown Book Company, 1969 Saba, Farhad, “Conceptual Development of Educational Technology“, International Journal of Instructional Media, Vol. 7 (4), 1979-80 Sarthory, Joseph A. "Needs Assessment and the Practitioner: Problems and Prospects“, Educational Technology, November, 1977, 24-26 Schramm, W}, Big Media, Little Media. Beverly Hills, CA., Sage, 1977 Schwals, Joseph J., “What Do Scientists Do?" Behavioral Science, January, 1980 Scriven, M., Evaluation Thesaurus. (3rd edition), Edgepress, Inverness, California, 1981 303 ---------- , The Methodology of Evaluation, AERA Monogram No. 1, Chicago: Rand McNally, 1967 ---------- , and Jane Roth, ”Needs Assessment: Concept and Practice“, New Directions for Program Evaluation. San Francisco, CA., 1978 Shuck, L.E., “The Instructional Tasks Project“, Newport- Mesa Unified School District (California), 1968 Silber, K.H. “Instructional Development: Deliverance.“ Journal of Instructional Development, 2(1), 1978, 2-5 Silvern, Leonard C. Evolution of Systems Thinking in Education. Education and Training Consultants Co., Los Angeles, CA., 1968 Silverman, Robert E., (contributing editor). "Theories and Models and Their Utility", Educational Technolo Review Series, Englewood Cliffs, New Jersey, no date Sobel, I. “Human Capital Revolution in Economics DevelOpment: Current Status, Expectations and Realities”, Comparative Education Review, November, 1980 Southard, M., "A Framework for Evaluating Needs Assessment Models", A Preliminary paper for doctoral dissertation, Tallahassee, Florida State University, 1974 Stake, R.E. I'The Countenance of Educational Evaluation", Teachers College Record, 1967-68, 523-40 Stanton, Charles M. “A Perception-based Model for the Evaluation of Career and Value Education Within the Liberal Arts”, Journal of Higher Education, Vol. 49, No. 1, 1978 Steadham, Stephen V., "Learning to Select a Needs Assessment Strategy“, Training and Development Journal, Jan., 1980 Stein, Jonathan A., Rabbi, 'No Real Solutions to a Very Real Problem“, Phi Delta Kappan, October, 1982 Struening, Elmer L., and Marcia Guttentage, (eds.), Handbook of Evaluation Research. Sage Publications, ever y s, ., 304 Stamas, Spelios Theodore, 'A Descriptive Study of a Synthesized Operational Instructional Development Model, Reporting It's Effectiveness, Efficiency, and the Cognitive and Affective Influence of the Development Process and Client“, Dissertation for the degree of Ph.D., Michigan State Univeristy, 1972 Stufflebeam, et al., Educational Evaluation Decision Making. Itasca, Illinois: F.E. Peacock Publishers, 1971 Sweigert, R.L., "Assessing Educational Needs to Achieve Relevancy", Education, XCI, No. 4, April-May, 1971 ---------- , “The First Step in Educational Problem Solving--A Systematic Assessment of Student Benefit”, California State Department of Education, Bureau of Elementary and Secondary Education, PLEDGE Conference, October, 1969 I ---------- , “Assessing Learner Needs with Criterion- Referenced Tests: A Working Approach", Educational Technology, November, 1977, 28-35 Trimby, Madeline J., "Needs Assessment Models: A Comparison", Educational Technology, June 8, 1979 Tucker, Kate D. "Educational Needs Assessment, A Simulation Model for Humanities Planning”, (New Orleans, LA, ERIC Document Reproduction Service, ED 087 413, April, 1973) Tyler, R.W. Constructing Achievement Tests. Columbus, Ohio: Ohio State University, 1934 ---------- , “Some Persistent Questions of the Defining of Objectives", in Definin Educational Objectives. C.M. Linduall (ed.), Pittsburgh: University of Pittsburgh Press, 1964 Urwin, Deuch and Ray McAleese, (eds.), Encyclopedia of Educational Media, Communications and Technology, Greenwood Press, Westport, Connecticut, 1978 Vichery, B.C. "Conceptual and Mathematical Models“, in Information Systems. London, Bulleworths, 1973 Weiler, Has N. “Education, Public Confidence, and the Legitimacy of the Modern State: Do we Have a Crisis?', Phi Delta Kappan, September, 1982, 9-14 Wight, A.R. ”Beyond Behavioral Objectives", Educational Technology, 12(7), 1972, 9-14 305 Witkin, Belle Ruth, "Needs Assessment Kits, Models and Tools“, Educational Technology, November, 1977, 5-18 ---------- , “Communication Strategies in Public Policy Decision Making: An Analysis of Processes in Major Needs Assessment Models from a Systems Point of View“, (Chicago, Ill. Annual Meeting of the Communication Association, April, 1975, ED 120 848) Wittich, W.A., and C.F. Schuller, Instructional Technology: Its Nature End Use (Sixth Edition). New York: Harper and Row, 1979 Wittrock, M.C. and D.E. Wiley (eds.), The Evaluation of Instruction. New York: Holt, Rinehart and Winston, 1970 Zaltman, Gerald, David H. Florii, and Linda A. Akorski, Dynamic Educational Change. The Free Press, New York, 1977