Sigkmlpfflw pm; . ‘———’——_ \ . l . 1 x r . , . r , g! L.‘ r ; - { art-mi,» ‘4 ("Mr-.3“. 1 .» I \ taps-11am -.r.o,. hugs ' V ’ _ ' ".-u .V J . A DEVELOPMENTAL STUDY OF TEACHERS’ ABILITY TO ‘lDENTi'FY AND WRITE PERFORMANCE OBJECTIVES Dissertation for me Degree of; 9h. 9. TE 'UNWERSITY U ; . _ ‘ . .. . . ’ .. u,‘ .. r.’ - ‘ ......'a.¢: 1 .;;» 319'th .n"! '1’" 'I."’ Mr 1 {5,53% 1:11?!) 'I'I: I!" I. 11:?“ ... . 75.53 Jr'cy-ru w-u " '::~ €1.13 RARY ' Michigan State University w "l!“llllllllllulll1|an , This is to certify that the thesis entitled A DEVELOPMENTAL STUDY OF TEACHER' ABILITY TO IDENTIFY AND WRITE PERFORMANCE OBJECTIVES presented by CHERYL ANN RABIDEAU has been accepted towards fulfillment of the requirements for Ph. D. degree in Education /) ’leynQ (bf: :/Q‘\}( Htjfsl'n * Major professor Date ///7/’7\37 0-7639 If BINBING BY -- , 1- sons in mm INB. LIBRARY BINDERS ‘m iiifiiiiiol'r. mellfll ABSTRACT A DEVELOPMENTAL STUDY OF TEACHERS' ABILITY TO IDENTIFY AND WRITE PERFORMANCE OBJECTIVES BY Cheryl Ann Rabideau There is much emphasis today on the use of performance objectives in curriculum development and evaluation. The utilization of such objectives in classroom activities creates a new pre-service and in-service need. If teachers are expected to use performance objectives to their maximum benefit they must be instructed in the identification and development of properly stated performance objectives. In addition, there must be some means of evaluating whether one can identify and write performance objectives. At the present time there is no instrument available to assess this skill. This study was therefore concerned with developing and validating an instrument to provide an estimate of one's ability to identify and write performance objectives. Answers to the following research questions were sought: 1. Is the test developed a reliable measure of whether or not teachers and teachers-in-training can identify and write performance objectives? 2. Is the test developed a valid measure of whether or not teachers and teachers-in-training can identify and write performance objectives? Cheryl Ann Rabideau 3. Do teachers in the field teaching score significantly higher at the .05 level of confidence on the first administration of a test concerned with performance objectives than teachers-in-training? 4. Do teachers and teachers-in-training who state that they have received prior instruction in performance objectives score significantly higher at the .05 level of confidence on the first administration of the test than those who have received no instruction? 5. Do teachers and teachers-in-training score significantly higher at the .05 level of confidence on the second administration of a test concerned with writing performance objectives after formalized instruction? Once the instrument‘was developed, it was administered at least twice to four different groups of teachers or teachers-in-training in the state of Michigan. The same form of the test was administered each time, with a six week lapse in between each session. The test - retest method was used to investigate the reliability of the instrument. Content validity was established through an analysis of course content and review by an authority in the field of performance objectives. For research questions III and IV, a t-test for two independent samples was used to compute the analysis of variance. A t-test for two related samples was used to compute the gains score analysis of research question V. Within the limitations of the study, the following conclusions seem reasonable based on the findings: Cheryl Ann Rabideau The instrument developed to measure whether teachers and teachers-in-training can.write performance objectives indicates a positive attitude toward reliability. At this point it is impossible to state that the test is a reliable instrument because content validity has not been tested as yet. Repeated testing of the instrument, however, does yield consistent results. Based upon a thorough analysis of course content related to performance objectives and review by an authority in the field, the instrument developed appears to be both reliable and valid in terms of content validity. Teachers in the field do not score significantly higher at the .05 level of confidence than teachers-in-training on the first administration of a test concerned with performance objectives. Teachers who have received prior instruction in performance objectives do score significantly higher at the .05 level of confidence on the first administration of the test, than do those teachers who have received no prior instruction. Teachers-in-training who have received prior instruction, do not score higher on the first administration of the test, than do those teachers-in-training who have received no prior instruction. Both teachers and teachers-in-training scored significantly higher at the .05 level of confidence on the second admini- stration of the test, after receiving formalized instruction. Cheryl Ann Rabideau Literature in the area of performance objectives has shown us that teachers need specialized training in performance objectives. The development of this instrument and its initial testing verify the need for specialized training, and suggests ways of instituting the performance objectives thrust in our educational systems today. A DEVELOPMENTAL STUDY OF TEACHERS' ABILITY TO IDENTIFY AND WRITE PERFORMANCE OBJECTIVES BY Cheryl Ann Rabideau A DISSERTATION Submitted to Michigan State University in partial fulfillment of the requirements for the degree of DOCTOR OF PHILOSOPHY College of Education 1973 é a. Q 6*... C57” A?) Copyright by Cheryl Ann Rabideau 1973 ACKNOWLEDGMENTS For their guidance, time, and assistance, I thank my committee chairman Dr. Lawrence Borosage, and committee members, Dr. Norma Bobbitt, Dr. James Page, and Dr. Robert Poland. For sharing their knowledge and expertise in the area of testing and statistical design, I thank Dr. Maryellen McSweeney, Associate Professor, Counseling and Personnel Services, Michigan State University and Dr. Willard Warrington, Director of Evaluation Services, University College, Michigan State University. In the area of performance based instruction, special thanks goes to Dr. Stephen Yelon, Assistant Director Learning Services, Michigan State University for his encouragement and assistance. I sincerely appreciated the cooperation of Dr. Daniel Brown, Miss Jan Danford, and Dr. Walter Hapkiewicz who permitted the instrument to be administered to their students or faculty. ii TABLE OF CONTENTS ACKNOWLEDGMENTS . . . . . . . . . . . . . . . . . . . . . . . . LIST OF TABLES . . . . . . . . . LIST OF FIGURES . . . . . . . . . . . . Chapter 1. OVERVIEW OF THE PROBLEM, . . . . . . . . . . . . . . . Introduction . . . . . . . . . . . . . . . . . . . The Problem . . . . . . . . . . . . . . . . . . Need for the Study . . . . . . . . . . . . . . Research Questions to be Answered . . . . . . . . . Definition of Terms . . . . . . . . . . . . . . . . Delimitations . . . . . . . . . . . . . . . . . . . Limitations . . . . . . . . . . . . . . . . . . . . Study Procedure . . . . . . . . . . . . . . . . . Overview of the Chapters . . . . . . . . . . . . 2. REVIEW OF RELATED LITERATURE . . . . . . . . . . . . Objectives in Accountability and Evaluation . . . Definition and Composition of an Objective . . . . History of Objectives . . . . . . . . . . . . . . . Impact of Objectives . . . . . . . . . . . . . . . Objectives and the Behavioral Domains . . . . . . . Questioning the Validity of Performance Objectives 3. PROCEDURES UTILIZED IN THIS STUDY . . . . . . . . . . Purpose and Nature of the Test . . . . . . . . . . Development of the Test . . . . . . . . . . . . . . Scoring the Test . . . . . . . . . . . . . .‘. Selection of the Study Population . . . . . . . . Description of the Study Population . . . . . . . Collection of Data . . . . . . . . . . . . . . 4. ANALYSIS OF THE DATA . . . . . . . . . . . . . . . . Research Question I . . . . . . . . . . . . . . . Research Question II . . . . . . . . . . iii Page ii vi H (DNNVUIU'Ir-‘l-‘H 10 10 14 17 18 22 24 29 29 29 33 35 35 39 41 42 44 iv Chapter Page Research Question III . . . . . . . . . . . . . . . 45 Research Question IV . . . . . . . . . . . . . . . 47 Research Question V . . . . . . . . . . . . . . . . 49 5. SUMMARY, CONCLUSIONS AND RECOMMENDATIONS . . . . . . . 52 Summary . . . . . . . . . . . . . . . . . . . . . . 52 Conclusions . . . . . . . . . . . . . . . . . . . . 54 Implications . . . . . . . . . . . . . . . . . . . 56 Recommendations for Further Research . . . . . . . 59 LIST OF REFERENCES . . . . . . . . . . . . . . . . . . . . . . 61 APPENDICES O O O O O O O O O O O O O O O O O O O O O O O O O O 65 Table LIST OF TABLES Correlation Between Test Scores on the First and Second Administration of the Performance Objectives Test . . . . . . . . . . . . . . . . Correlation Between Test Scores on the Second and Third Administration of the Performance Objectives Test . . . . . . . . . . . . . . . . . . Correlation Between Test Scores on the Second and Third Administration of the Performance Objectives Test . . . . . . . . . . . . . . . . . . Analysis of Variance Table on the First Administration Scores of Teachers and Teachers-in-Training . . . . Analysis of Variance Table on the First Administration Scores of Those with Prior Instruction and Those with No Prior Instruction (Teachers-in-Training) Analysis of Variance Table on the First Administration Scores of Those with Prior Instruction and Those with No Prior Instruction (Teachers) . . . . . . . Gains Score Analysis Table for Teachers-in-Training Gains Score Analysis Table for Teachers . . . . . . . Page 43 43 44 46 47 48 50 LIST OF FIGURES Figure Page 1. Test Structure . . . . . . . . . . . . . . . . . . . . 31 2. Test Items Breakdown . . . . . . . . . . . . . . . . . 32 3. Percentage of Each Group Having Received Previous Instruction . . . . . . . . . . . . . . . . . . . . 37 4. Previous Instruction Breakdown . . . . . . . . . . . . 38 5. Test Administration Schedule . . . . . . . . . . . . . 40 vi CHAPTER I OVERVIEW OF THE PROBLEM Introduction There is much emphasis today on the use of performance objectives in curriculum development and evaluation. As in many other activities, the teacher that is involved in developing or modifying these objectives to meet the needs of his program, is more likely to make the maximum effort in helping his students reach these objectives. The utilization of such objectives in classroom activities creates a new preservice and inservice need. If teacher are expected to use performance objectives to their maximum benefit they must be instructed in the identification and develoPment of proPerly stated performance objectives. The Problem The purpose of this study was to: (l) develOp an instrument designed to provide an estimate of one's ability to identify and write performance objectives; and (2) validate the instrument among four different groups of teachers or teachers-in-training in the state of Michigan. Need for the Study For many years the educational community has been involved in preparing academic goals as a tool in citing learning outcomes. These goals have been used as an aid to planning curricula, teaching strategies and course outlines. The idea of performance objectives, with an agreed upon frame of reference, is however a new concept. Although performance objectives have been successfully employed for a number of years by various branches of Armed Services, business and industry, in the academic world there is still considerable confusion over what constitutes a performance objective, how it is developed or written, and how it might be used. The need and interest in performance objectives can be seen across the United States. The Southern Association of Colleges and Schools (1973) is in the process of deve10ping an interstate consortium which would be responsible for producing performance objectives and criterion measures in occupational education. The following list of states have been invited to participate in the initial planning stages. Alabama Missouri Colorado North Carolina Florida Ohio Georgia Oklahoma Kentucky South Carolina Louisiana Tennessee Michigan Texas Mississippi Virginia At the present time the state of Michigan is in the process of implementing a major thrust for the use of performance objectives in all curricula. Both the United States Office of Education and the ‘Michigan.Department of Education have very recently been requiring that schools applying for categorical financial support state their plans in terms of performance objectives. In the state of Michigan, performance objectives are seen not as an entityWunto themselves, but as an.important facet of a total accountability model. Briefly, this accountability model, which is explained in Chapter II, includes the need for common goals of education, development of performance objectives, assessment of needs, analysis of teaching techniques, and provisions for outside audits to determine if change has indeed taken place, in addition to providing guaranteed inservice professional development. In the area of general education a Grade Level Commission has been developed for each grade K-12. Objectives for each academic area have been written and evaluated by teachers, teacher-educators, administrators, parents, and students. In the coming academic year (1973-74), these objectives will be field tested and revised. The Vocational Education and Career Development Service of the Michigan Department of Education has further stated that by September 1974 every vocational program receiving reimbursement will be reaponsible for submitting performance objectives for each program subsidized. As part of the master plan for developing performance objectives in Vocational Education, the Vocational Education.and Career Development Service has inserviced over 300 vocational instructors in the philosophy and writing of performance objectives. When objectives have been written and validated for each vocational area, local teachers will have the option of: (l) accepting and using the objectives as written for their vocational area; or (2) accepting the objectives, but making additions of their choice; or (3) rejecting the objectives as written, and submitting their own objectives for approval to the Vocational Education and Career Development Service. All of the above mentioned plans for the development and utilization of objectives demand that teachers are able to identify and/or write an objective and determine its appropriateness for the respective program or grade level. Farr (1969) pointed to the teacher need for training in the development and use of behavioral objectives.* He believed this training must take place before teachers could select appropriate measuring instruments, materials or procedures that would be fundamental to evaluating the growth of students and/or programs. Glaser (1967), another proponent of objectives, believed that the analysis and specification of objectives and learning outcomes is the most important factor in improving any educational program. Many other authors have discussed the significance and role of performance objectives, in addition, suggestions have been given for educating the teacher in developing performance objectives. Few authors have cited ways to measure this knowledge. Since no instrument was available to assess this skill, this study is therefore concerned with developing and validating an instrument designed to provide an estimate of one's ability to identify and write performance objectives. *For purposes of reviewing literature, the following terms are used synonymously with performance objectives: goals, behavioral objectives, educational objectives, enabling objectives, terminal objectives, training objectives, and instructional objectives. Research Questions to be Answered The questions which this study attempted to answer follow: 1. Is the test developed a reliable measure of whether or not teachers and teachers-in-training can identify and write performance objectives? 2. Is the test developed a valid measure of whether or not teachers and teachers-in-training can identify and write performance objectives? 3. Do teachers in the field teaching score significantly higher at the .05 level of confidence on the first administration of a test concerned with performance objectives than teachers-in-training? 4. Do teachers and teachers-in-training who state that they have received prior instruction in performance objectives score significantly higher at the .05 level of confidence on the first administration of the test than those who have received no instruction? 5. Do teachers and teachers-in-training score signi- ficantly higher at the .05 level of confidence on the second administration of a test concerned with writing performance objectives after formalized instruction? Definition of Terra For purposes of this study, the terms are defined as follows: Elements--the component parts of a performance objective. Test items pertaining to this aspect of the test requested that the participant name, define and select the component parts of an objective. Identification--recognition and selection of a performance objective. Test items pertaining to this aspect of the test requested that the participant recognize and select those statements written in performance terms. Participants were also asked to identify the appropriate domain of stated objectives. Performance objective--a statement of what the learner is like or able to do when he has successfully completed the learning experience. Literature reveals a variety of other terms which refer to performance objectives as defined by this study. The following terms have been used synonymously with performance objectives: goals, behavioral objectives, educational objectives, enabling objectives, terminal objectives, training objectives, and instructional objectives. Reliability--as applied to educational instruments, reliability may be defined as the level of consistency of the measuring device. This consistency reflects the degree to which the test may be considered stable or may be depended upon to yield similar test results (Borg, 1963, p. 84). Teacher--one presently possessing an elementary or secondary teaching certificate and/or vocational certificate. This person is presently teaching in an elementary or secondary school or vocational center; or has recently completed at least one year's experience the previous academic year at one of the above schools. Teacher-in-training--one who is presently enrolled in a College of Education and has the intended professional goal of teacher. Validity--as applied to educational instruments, validity may be defined as the degree to which a test measures what it claims to measure (Borg, 1963, p. 80). Writing--composing and listing performance objectives. Test items pertaining to this aspect of the test requested the participant to rewrite and compose sentences in performance terms. 1. Delimitations The population tested was limited to four groups of teachers or teachers-in-training in Michigan. The testing instrument, which was developed by the researcher, was operational only as defined. Subject matter content was not considered in the administration or grading of the instrument. Limitations The majority of the population tested consisted of vocational instructors, and did not include other school functionaries. Educational background of the population varied from high school graduate to Ph. D. degree within and among the different groups tested. The test was taken on a voluntary basis. It is assumed that each participant worked to his best ability during the time limit established. Study Procedure A review of literature was conducted in the areas of goals and objectives etc., testing; accountability and evaluation; teaching technology; and curriculum planning. 2. A test was researched, developed, pre-tested, modified and administered two or three times to the various groups. 3. Background information was gathered from the participants in the following areas: occupation, whether or not previous instruction in performance objectives had been received, and the nature of the instruction. 4. The test was administered at least twice to four different groups of teachers in the state of Michigan. a. Graduate students at Wayne State University - Detroit, Michigan (June, 1972). Sample size - twenty participants. b. Conference participants at the Career Education Conference in Grand Rapids, Michigan (August, 1972). Sample size - fourteen participants. c. Graduate and undergraduate students at Michigan State University - East Lansing, Michigan (October, 1972). Sample size - nineteen participants. d. Faculty members of the Capitol Area Career Center - Mason, Michigan (December, 1972). Sample size - thirty faculty participants. 5. The resulting data was tabulated, analyzed, and. interpreted. 6. Conclusions and recommendations were drawn. Overview of the Chapters Chapter II provides a review of related research in the areas of: goals and objectives; testing; accountability and evaluation; teaching technology; and curriculum planning. Chapter III discusses the procedures utilized in the investigation by examining: the development of the test (items, test form, answer sheet): selection of the study population; description of the study population; and collection of data. In Chapter IV derived statistics for testing the research questions are presented and a summary made of the results.. Chapter V includes a summary of the investigation; conclusions; implications; and recommendations for further research. CHAPTER II REVIEW OF RELATED RESEARCH The related research is presented in six categories: (1) the role of objectives in accountability and evaluation; (2) definition and composition of an objective; (3) history of objectives; (4) impact of objectives; (5) objectives and the behavioral domains; and (6) questioning the validity of performance objectives. Objectives in Agcountability and Evaluation Accountability has a historical frame of reference found in Ontario, Canada from 1876 to 1882. Sciara and Jantz (1972) give the following account. During those years, payments to high schools were largely dependent on the number of students who passed an intermediate exam after a year or two of attendance. Although standards were raised according to the adopted criterion, this practice was abandoned in 1883 after a protest against the sacrifice of all other educational values for the attainment of this goal (p. 6). The demand for accountability in education has been growing rapidly in the past decade. According to Wickline (1971) people became concerned when a large percentage of the draftees were rejected because they were functionally illiterate. People began to question whether schools*were really doing their job. As a result of such inquiry, the following types of evaluation arose; the National Assessment Program‘which gives some idea of the status of schools and whether or not progress is being made; Title I program.evaluation, 10 11 *which includes apprOpriate objective measurement at least once a year. Today words such as performance contracting, audit, goal, performance objective, and accountability are familiar vocabulary to many administrators, teachers, and even students. Throughout the United States performance objectives are playing an integral role in evaluation of education. In the state of Michigan, the Department of Education has developed an overall accountability model in public education. John W. Porter, Super- intendent of Public Instruction identified the six general categories; (Michigan Department of Education-Developing Performance Objectives, undated). 1. Identification, discussion and dissemination of common goals for Michigan Education. 2. Approaches to educational challenges based on performance objectives consistent with these goals. 3. Assessment of educational needs not being met, and which must be met to achieve performance objectives and goals. 4. Analysis of the existing (or planned) educational delivery system in light of what assessment tells us. 5. Evaluation and testing within the new or existing delivery system to make sure it serves the assessed needs. 6. Recommendations for improvement based upon the above (p. 1). Performance objectives play either a direct or indirect role in achieving the goals of this accountability model. State departments of education are not the only ones to become involved in performance objectives. Many college instructors, teachers, 12 administrators and teacher training institutions have recently been involved in developing objectives for their needs. If curriculum program evaluation is to be relevant, it must contain a clear explanation of the behaviors which are being measured. Ammons (1962) supports this belief in the statement that educational objectives benefit the classroom teacher in the following: (1) "in selecting activities appropriate to the achievement of the objective; (2) in selecting evaluation techniques suitable for assessing both student progress toward the objective and the general quality of the program (p. 433)." McAshan (1970) concurs that writing behavioral objectives provides educators with a guide to the evaluation of programs and also to the intended processes of instruction. Mazur (1969) who believes objectives are fundamental to the evaluation process, emphasized the role of objectives with the following statement: The adequacy with which objectives are stated is a critical factor in determining the quality of the information derived from evaluation. The presence of an assessable objective provides a foundation on which to build a systematic evaluation which in turn, would provide information that will be useful to administrators and project managers in both facilitating and improving decision making (p. 49). Tyler (1938) who has been a proponent of objectives and evaluation through the years believes testing should measure the extent to which program objectives are being reached. Testing based on behavioral objectives should provide information that will aid in improving the instructional process. Many years later Tyler (1951) suggested that the teacher should provide the learner with the opportunity to demonstrate ability by writing many test items that measure the l3 attainment of the particular objective. Engman (1968) agreed with the need for teachers to develop learning experiences around specific course objectives. Consalvo (1969) too believed that performance objectives are a prerequisite for valid assessment. The road to specifically stated performance objectives has been long in the making. Four instructional movements have been intricately involved in the growth of such objectives. Tyler (1938, 1950, 1951) and his associates were some of the earliest researchers in this area. Their basic interest was in developing goals of education that would be relevant to the teacher. As a result of this effort, many educators began to describe objectives in terms of student performance. The second movement was spearheaded by the Mid-Century Committee on Outcomes in Elementary School. Kearney (1953) reported that the task of this group was to describe the ‘measurable goals of instruction in American elementary schools for educators, test-makers, and interested citizens. The second movement also consisted of a Survey Study of Behavioral Outcomes of General Education in High School (French, 1957). The purpose of this study 'was to describe the objectives of general education in American secondary education for all educators. A third movement started about a decadeago. This movement resulted from the Armed Services' need to prepare people to operate and maintain large, complex military weapons systems. Robert B. Miller (1953, 1962) developed procedures for job task analyses. Tasks and skill performances represented the behavioral requirements. The fourth movement is denoted by the concept of programmed instruction. Writers of instructional programs demanded specific guidance on instructional objectives. (Mager, 1961; 14 Gagne, 1963; Mager, 1962; and Miller, 1962) agree that in reference to programmed instruction, the more specific the objectives, the more easily they can be transformed into instructional materials. All three educators agreed that behavior must be precisely described . with the level of performance stated for recognition of successful student performance. Definition and Composition of an Objective With so many people writing and discussing objectives, it would seem feasible that there might be as many definitions of the term "objective" as authors of the subject. In-as-much as objectives do come under a variety of titles (behavioral, performance, terminal, enabling etc.), this is sometimes the case. Since the 1960's however, proponents of behavioral objectives have, despite their differences, developed some common bonds. Both Huffman (1973, p. 26) and Byers (undated, p. 3) define a performance objective as an "educational objective that clearly states observable and measurable performance and that identifies for the student and teacher the conditions under which the events or steps in the learning will take place". Kibler, Barker, & Miles (1970) and Cohen (1970) have similar definitions. Kibler et a1., believe "Behavioral objectives are statements which describe what students will be able to do after completing a prescribed unit of instruction (p. 1)." Cohen, similarly believes "An objective is a specific observable student action or product of student action (p. 6)." Bernabei and Leles (1970) have a similar definition. "Objectives 15 are desired outcomes toward which effort or behavior is directed or aimed with a definite purpose in mind (p. 1)." According to McAshan (1970) a performance objective is "any specifically stated objective that identifies a goal and specifies some type of performance, instrumentation, or evaluation strategy that will furnish evidence that the intended outcome of the goal has or has not been achieved (p. 15)." The Vocational Education and Career Development Service of the Michigan Department of Education defines performance objective as a "communication device which provides a precise description of a testing (criterion) situation (Bailey, Bland, Brown, 1972, p. 1)." Although each one is an extension of modification of Robert Mager's (1962) early definition. An objective is an intent communicated by a statement describing a proposed change in a learner--a statement of what the learner is to be like when he has successfully completed a learning experience. It is a description of a pattern of behavior (performance) we want the learner' to be able to demonstrate (p. 3). In much the same way authors modeled Mager's definition of performance objective, component parts of a performance objective too have a similarity. Meger (1962) gave us three basic component parts: (1) Conditions (2) Behavior (3) Criterion. Conditions refer to relevant materials, supplies, prdblem, or situation. Behavior refers to the verb, or what is to be accomplished. Criterion refers to the level of proficiency (time constraints, quality statements, or percentages). Most proponents of behavioral objectives have either accepted'Mager's component parts "as is" or have made further additions or refinements. Kibler et a1., (1970), Bernabei and 16 Leles (1970), Cohen (1970), Williams (1973) and Vogler (1973) are just a few proponents who accept the components with no modifications. Byers (undated) includes the following components: A. B. C. D. E. F. G. H. Givens: people, objects, information Sources: locations, records, activities Purposes (to do something) Methods: following, developing, or refining asmwmme Outputs: documents, interaction, intangibles Quality measure or error tolerance Time limit or priority rating Prerequisite: ability to learn task (p. 2). The Michigan Department of Education - Developing Performance Objectives 1. (undated) has developed the following elements: The performer. The individual or individuals who will be involved in the instrucational task. This might be a single student, an entire class or school, or even the entire state. This variable is called WHO. The behavior to be demonstrated. What it is the performer is to do - complete, comprehend, leap, construct, etc., etc. This is the HOW variable. The object of the behavior may be charac- terized as WHAT variable. In other words, are we specifying electronics, French, mathematics or composition? The element of time will establish the essential of WHEN. This might be expressed as a future date, number of days or months or even in terms of necessary prerequisite to another level or phase. HOW MEASURED provides data as to the techniques to be employed to verify that the objective has been met. This might be a normative instrument, a teacher-made test or an observational check list. HOW WELL is the criterion for success relative to the objective. This is the element of proficiency or degree of accomplishment (p. 7). 17 The two previously cited examples have taken the Mager elements and made some additions. The basic components of conditions, behavior, and criteria are included. History of Objectives Just as evaluation is rich in tradition, so too, is the history of objectives. Concealed under many guises, literature dealing with objectives may be identified as: performance objectives, behavioral objective, instructional objectives, and educational goal. Franklin Bobbitt, who has been called "the father of curriculum theory" was one of the earliest proponents of objectives. Bobbitt (1918) stated: . . education that prepares for life is one that prepares definately for specific activities. However, numerous and diverse they may be for any social class, they can be discovered. This requires that one discover the particulars of which these affairs consist. These will be numerous, definite, and particularized . . . The curriculum will then be that series of experiences which childhood and youth must have by way of attaining these objectives (p. 42). Bobbitt (1924) listed objectives needed in various phases of education. "Ability to use language in all ways required for proper and effective participation in community life (p. 11)." "Ability to utilize music for a healthful, abundant, and varied awakening of one's emotional nature (p. 19)." Although not listed in behavioral terms, these goals or objectives were the rudiments of today's behavioral objectives. During the 1920's several attempts were made to develop objectives in the area of general education. Billings (1921) developed 18 888 objectives for the teaching of social studies. Pendleton (1924) developed 1,581 objectives for English instruction and Guiler (1926) develOped 300 objectives for arithmetic. Although the basic premise on which these objectives were written had merit, the large number .of objectives tended to frighten and discourage teachers. Without training in the writing and utilization of objectives, this effort was temporarily stifled. A revival of interest in objectives came about in the 1940's and '50's along with the interest in evaluation. Impact of Objectives Behavioral objectives involve all people in the teaching- 1earning process, this includes students, teachers, school boards, administrators and parents. Kibler et a1., (1970) in discussing how objectives affect students has written: "By being given behavioral objectives, students do not have to guess what is expected of them in the learning setting. Learners may spend their time acquiring behaviors specified by the teacher rather than attempting to infer what the teacher expects of them (p. 106)." A study by Doty (1968) tested a hypothesis suggested by the American Institute for Research that prior knowledge of educational objectives affects the practice and performance of students. The results of this study indicated that students' knowledge of educational objectives before study of a unit increased the efficiency of student learning. Miles, Kibler, and Pettigrew (1967) similarily found that when students are given specific behavioral objectives for a course, they tend to score better on an objective test than when they are not aware of specific objectives. Bryant (1970) researched a similar area to determine 19 whether or not stating course objectives in behavioral terms had a significant effect on the achievement of students. He concluded that pupils taught by teachers trained in the use and development of behavioral objectives performed better on the criterion measure. He also discovered that providing students with the objectives improved their understanding of what was expected of them. Ojemann (1969) believes that students may lack understanding of course material because of misdirected learning experiences, inappropriate evaluative measures, and confusion over what is expected. He believes ambiguity can be avoided if curriculum objectives are expressed in specific behavioral terms. Kibler et a1., (1970) states another benefit of utilizing behavioral objectives. It is the sense of security a student experiences when he knows what specifically is expected from him in a course and the conditions under which he will be expected to exhibit his competencies. Psychologists suggest that generalized fears cause greater emotional anxiety than specific well-defined fears. Behavioral objectives can help students understand specific requirements of a course and also reduce the amount generalized anxiety about course requirements (p. 106). Teacher training--the advantages of performance objectives have been discussed, yet without proper instruction in the development and utilization of such objectives, the possible benefits will be minimized or non-existent. According to Mager (1962) "when clearly defined goals (or performance objectives) are lacking, it is impossible to evaluate a course or program.efficiently, and there is no sound basis for selecting appropriate materials, content, or instructional methods (p. 3)." Popham and Baker (1970) reinforce this belief, but point out what has been happening in many schools: 20 "Teachers have always been concerned with the importance of instruc— tional objectives, yet the kinds of objectives which they have endorsed usually made little difference in the nature of their instructional programs. The principal reason for this is that these objectives were stated in terms too broad and ambiguous to allow any one to agree upon.what the objectives meant (p..23)." Sullivan (1971) a division head for the Southwest Regional Laboratory for Educational Research, believes that despite verbal commitment of many educators to instructional objectives, precise objectives are not commonly employed in major efforts to plan, evaluate, and improve classroom instruction. The reason stated for this behavior is obvious: "Teachers in general have neither the time nor the training to design and develop objectives-based programs of instruction for their own use, and such programs have not been provided for them by other agencies (p. 55). Nerbovig (1956) researched teachers' knowledge of objectives in regard to awareness and utilization. He found the major function to be performed by objectives in the teaching-planning activities, was least considered by experienced teachers. In contrast, when performance objectives were used with student teachers in an experimental study by Moffett (1966), the following results were discovered: (1) Goals of instruction were more readily achieved by pupils of student teachers in the experimental groups, (2) The supervisor was considered more helpful by the experimental group teachers. Lapp (1970) conducted a study which determined the ability of elementary teachers to write performance objectives. Results 21 revealed that only eight percent of the total population of teachers achieved satisfactory criterion level performance. Those with one to ten years of teaching experience scored higher than those with more than ten years experience. Gilpin (1962) believes educators have many ideas on preparing behavioral objectives, yet little has been done to prepare teachers to utilize objectives in curriculum development. He believes the following three questions must be answered if a teacher is to write worthwhile objectives: 1. What is it that we must teach? 2. How will we know when we have taught it? 3. What materials and procedures will work best to teach what we wish to teach (p. viii)? Baker (1967) attempted to compare the effect behavioral and non-behavioral objectives have on learning. The results showed no significant differences on mean test scores. Baker stated this may have been because of the teacher's inability to discriminate items that related to specific objectives. He also felt this may have been because of the teacher's inability to discriminate items that related to specific objectives. He also felt this may have been due to the teacher's lack of motivation to promote high pupil performance. Baker implies a need for training in the use of performance objectives. Popham.and Baker (1970) agreed that although everyone may be talking about behavioral objectives, teachers still do not know how to utilize them in classroom planning. McAsham.(l970) stated that: Much in-service training time is necessary to teach school administrators, staff members, and teachers how to write objectives in their special areas. This also requires time for cooperative planning. Without the in-service training and cooperative planning time, the results obtained by the 22 teaching teams may be mediocre at best. And any program that is based on mediocre program objectives will produce results and educational products that may be classified by the same name (p. 9). The research concerning the teacher and performance objectives suggests a need for training in the develOpment and use of objectives before the optimum results can be realized. According to Kibler et a1., (1970), the teacher who utilizes objectives in curriculum planning and evaluation is the teacher who is confident 1) that the subject matter being presented is of prime importance and 2) that measurement of achievement is efficient and appropriate to course goals, is more secure in his position and, consequently, is usually more satisfied with his professional contribution (p. 107). Objectives and the Behavioral Domains Benjamin Bloom developed a Taxonomy of Educational Objectives. This work divided behavioral objectives into three domains: cognitive, psychomotor and affective. The cognitive domain includes those objectives which deal with the recall or recognition of knowledge and the development of intellectual abilities and skills. Psychomotor deals with the student's physical ability to perform a skill. The affective domain is concerned with the student's character traits and willingness to perform. It includes objectives which "describe changes in interest, attitudes, and values, and the development of appreciation and adequate adjustment. Bloom (1956) believes the following about objectives in the affective domain. Objectives in this domain are not stated very precisely; and, in fact, teachers do not appear to be very clear about the learning experiences which are appropriate to these objectives. It. is difficult to describe the behaviors apprOpriate 23 to these objectives since the internal or covert feelings and emotions are as significant for this domain as are the overt behavioral manifes— tations. Then, too, our testing procedures for the affective domain are still in the most primitive stages (p. 107). Although objectives written for this domain are considered difficult to develop and often receive criticism, several authors have offered suggestion to alleviate these problems. Ridenour (1971) reported on work completed by an ad hoc committee of Future Farmers of America. This group identified performance objectives on student personal growth. Their work investigated the areas of leadership, communications, citizenship, service to others, social skills, management of financial resources, and individual adjustment. Gronlund (1969) gives specific examples of changing general instructional objectives into action verbs for writing corresponding affective objectives. Vargas (1972) states that in addition to teaching specific skills, we must strive to create positive attitudes. It is his belief that outlining specific behaviors that usually indicate positive attitudes is one way of measuring student attitudes, but only if students are under no pressure to produce the behaviors specified. According to Vargas, "Behavioral objectives for attitudes, in contrast to objectives for achievement, should not, therefore, be set as requirements for a course (p. 26)." Kibler et a1., (1970) agrees with Vargas and suggests using questionnaires or interviews to determine what students might do under specified conditions. 24 Questioning the Validity of Performance Objectives Even though there are many proponents of performance objectives, there are still those who question their value. Performance objectives are no panacea for all the ills of evaluation and it is important to look at all sides of the issue. Ebel (1973) challenges the entire process of directing instruction according to specified behavioral objectives. "Detailed specification of educational objectives is not essential to effective education", he has stated. Ebel called it "dangerously misleading to hold that educational objectives ought to be stated in behavioral terms." Part of the purpose of education, he said, "is to equip the individual to behave more effectively of his own volition in response to future needs and circumstances . . . We cannot teach the students how to behave in response to these needs and circumstances. We can only give him the general tools he will find useful in doing a particular job." "It is the cultivation of intellectual resources, not the cultivation of specific behavior patterns, that ought to be regarded as the essential purpose of education (p. 37)." Georgia and Eldon Scriven (1973) hold similar beliefs as Ebel. According to them, behavioral objectives tend to concentrate on the end product, not the means of acquiring that end product. It is their contention that the "means" may be the most important aspect in learning certain behaviors. In addition, the Scrivens ‘have noted that the Taxonomy of Educational Objectives, although an understandable and manageable categorization of man's behavior it has never been proven to be an accurate or complete categorization. It follows 25 then, that "Accepting this categorization as truth and basing all learning upon it in the form of behavioral objectives, is an act which can only be as accurate as is the accuracy of the categorization schema (p. 530)." The Scrivens further believe that all learning does not result in an overt change of behavior. They question compensation for those learnings which cannot be communicated within the framework of behavioral objectives. The Scrivens, however, are not totally Opposed to objectives. It is their belief that objectives should be used to organize learning tasks, where they lend themselves to this kind of organization. Time should be left however for flexibility: random learning, group process, discussion and inspiration. They also believe that the process used by the learner to reach the and objective may either change the end objective, or may be as important as reaching the objective. According to the Scrivens, ‘not to recognize these possibilities . . . "relegates formal school learning to a series of leaps from end objective to end objective with, perhaps, little in between (p. 531)." Vonk (1973) expresses some other concerns that should be noted. Vonk believes that measurable objectives in a classroom means teacher-made paper and pencil tests. Since the tests which are often fact-centered, should reflect the teaching, the teaching will soon become fact-centered, not insight-centered or attitude- centered or even student-centered. Vonk agrees with the Scrivens in fearing the "glorification of superficial and trivial fact at the expense of thoughtful subtlety. Moreover, some teachers will wind up teaching the test--especially if the test is used as a tool of 26 supervisory control (p. 544)." Vonk further concurs with the Scrivens and Ebel in the following paragraph. It is my belief that the behavioral objectives position is a mistake: that it furthers an empty-organism approach to boys and girls: that it engenders a screwed-down, authoritarian curriculum; that it will deteriorate into a teach-the-test, beat-the—test gamesmanship; then it can become a snoopervisory device for those who do not trust teachers; that it elbows subtle thinking and reflection of the path of learning to make way for fact and action; and that a teaching machine could do non-sense syllable education better and cheaper (p. 544). The process of stating objectives in performance terms has its shortcomings, and certain criticisms are valid. In looking at objectives it is important to discern the legitimate contentions from the unwarranted excuses. P0pham (1968) cites the following examples of dubious validity. l. Trivial learner behaviors are the easiest to operationalize, hence the really important outcomes of education will be underemphasized. Rebuttal--instead of encouraging unimportant outcomes in education, the use of explicit instructional objectives makes it possible to identify and reject those objectives which are unimportant. 2. Prespecification of explicit goals prevents the teacher from taking advantage of instruc— tional Opportunities unexpectedly in the classroom. Rebuttal--when one specifies explicit ends for an instructional program there is no necessary implification that the means to achieve those ends are also specified. Unexpected instructional opportunities in the classroom should always be justified in terms of its contribution to the learner's attainment of worthwhile objectives. 27 Measurability implies behavior which can be objectively, mechanistically measured, hence there must be something dehumanizing about the approach. Rebuttal--A broadened conception of eval- uation suggests that there are diverse and extremely sophisticated ways of securing qualitative as well as quantitative indices of learner performance. That isn't really the way teaching is; teachers rarely specify their goals in terms of measurable behavior; so let's set real- istic expectations of teachers. Rebuttal--There is a difference between identifying the status quo and applauding it. Instructors must begin to identify their instructional intentions in terms of measurable learner behaviors. The way teaching really is at the moment just isn't good enough. In certain areas, e.g. fine arts and the humanities, it is more difficult to identify measurable pupil behavior. Rebuttal--Yes, it is more difficult, but those subject specialists should not be allowed to escape this responsibility. Criteria lurk whenever this teacher does make a judgment, and these criteria must be made explicit. Measurability implies accountability; teachers might be judged on their ability to produce results in learners rather than on the many bases now used as indices of competence. Rebuttal--Teachers will no longer be judged on the idiosyncratic whim of visiting supervisors. Rather, he can amass evidence that, in terms of his pupils' actual attain- ments, he is able to teach efficiently. Even.though this is a striking departure from the current state of affairs, and a departure that may be threatening to the less competent, the educator must promote this kind of accountability rather than the maze of folklore and mysticism‘which exists at the moment regarding teacher evaluation (p. 391-396). 28 Once we state objectives in performance terms we must examine student change rather than teacher performance. We must look beyond the implied abilities of students to their specific actions, beyond their unknown attitudes to their observed behaviors. Performance objectives must become a major portion of the instructional, as well as evaluation process. There are a variety of opinions as to the extent and role, performance objectives should play in our modern educational systems. However, if performance objectives are to be used minimally or extensively, those involved in all phases of this endeavor must be well informed and trained in each aspect of development and implementation. CHAPTER III PROCEDURES UTILIZED IN THIS STUDY Purpose and Nature of the Test A review of literature revealed that there was no existing instrument or standardized test developed to ascertain whether or not teachers and teachers-in-training could write and identify performance objectives. The purpose of this study then was to: (1) develop an instrument designed to provide an.estimate of one's ability to identify and write performance objectives; and (2) validate the instrument among four groups of teachers and teachers-in— training in the state of Michigan. As a result the researcher developed the Performance Objectives Test. The following sections of this chapter examine: the development of the test; scoring procedures; selection of the study population; description of the study population; and collection of data. Development of the Test A chart depicting the overall plan for development of the test is found in Figure 1. This chart includes the fundamental processes followed in developing the instrument. The test booklet was constructed as a product of the test plan, test items, item pool and the final test form. Accessory information was concurrently developed including the answer sheet and scoring procedure. In planning the Performance Objectives Test, the first task of the 29 30 researcher was to state the objectives of the test and the study. (1) The researcher will develop a written test designed to measure whether or not teachers and teachers-in-training can identify and 'write performance objectives. (2) The researcher will test this instrument for validity and reliability among four groups of teachers or teachers-in-training in the state of Michigan. The content of the test was concerned with two specific areas: (a) identifying objectives; (b) writing objectives. Since a review of literature has shown that both identifying and writing objectives are crucial in effective management of learning, approx- imately half of the test items deal with identification and half with the elements and/or writing of objectives. Although many facets of identifying and writing objectives overlap one another, Figure 2 shows a breakdown of the test items. Questions 1-11, and 22-24 deal specifically with the identification of an objective. (Definitions of the terms identification, elements, and writing are clarified on page 5 and 7). Questions 13-21 relate to the elements of an objective, and questions 26-30 are concerned with the actual writing of an objective. Questions 15 and 17 encompass both the elements and the writing of an objective. Concurrently an answer sheet and test directions were developed. Capies of these are presented in the appendix. In a symposium on "content validity", Ebel (1956) made the following point: -Hl MMDnmoomm UZHMOUm a r 41 ououosuum umoH H ousmne mZMHH HmMH a Hammm mmzmz< lemon - , goon may saga L 31 _ I”, Aoum oo>wooom wcw>mm macaw zoom «0 ammunoouom m ounwwm NON mmezmu 558 . <52 noHHmHZD HHHz= whmfim mzw<3 Z001 Z56 Z06 Z58 Z08 Z5£ ZOL Z59 Z09 Z55 Z05 Z5? Z0? Z58 ZOE Z52 ZOZ Z51 Z01 Z50 Mbcmo 38 caooxooum coauosuumaH mnofi>oum e muste .cowuoouumaw msofi>oun mo noun oeo easy once ooxoomo o>w£ hue ouemnfioauuomw mmazmo mummao N m «H s OH “u amm< o n SEAS "nun 3 a as shaggp m H m m N x 3 Beam 1. zaonoH: N 0 nu J T. .d .1"; mnHmae ozamo o N m H H 30 mgfifiéo w .4 75:52am c.q. «sumac a waHmmmsta o H a o m maHZD mHHZD mH