ABSTRACT LEARNING FROM EXTANT MATERIALS: THE EFFECTS OF PERFORMANCE OBJECTIVES AND ADJUNCTIVE PRACTICE QUESTIONS BY L. Denis Hlynka The purpose of this study was to investigate the separate and interactive effects of adding performance objectives and/or adjunctive practice questions in the form of an adjunct program, on learning from extant instructional materials. The two dependent variables of concern were (a) cognitive learning, as measured by a researcher—constructed achievement test; and (b) attitude, as measured by a researcher-constructed attitude measure. Three sections of Educational Media 831-A offered by Michigan State University were selected as the target population for the research. Each subject was randomly assigned to one of four treatment groups. Treatment one received only the extant materials, followed by an achievement and attitude test. Treatment Egg received lists of performance objectives, plus the treatment one materials. Treatment three received an adjunctive program in addition to the treatment one materials. Treatment four received bg£h_adjunctive program plus performance objectives, as well as treatment one materials. The extant materials consisted of three articles from various journals selected for their relevance to a unit on film music as a variable in instructional product deve10pment. This topic was selected because it was felt that it was one with which students L. Denis Hlynka would have little initial familiarity. Also, the topic could be justified as relevant to a course in educational media. Lack of familiarity with the content was important to this study so that a pre-test could be omitted, since it was felt that a pre-test might confound the effects of objectives and adjunctive program. Six hypotheses were generated and examined. The hypotheses tested (a) an objectives main effect; (b) an adjunct main effect; and (c) an objectives-by-adjunct interaction effect. These hypotheses were examined in terms of both achievement and attitude. The experiment was set up as a fully balanced 2 x 2 factorial design. An analysis of covariance was run using study time and total time as covariates, since it was considered possible that the factor of time might be able to account for any differences which occurred. Neither study time nor total time, however, were fbund to predict either dependent variable. Therefore, the analysis of covariance was dropped from further consideration, and an analysis of variance was considered to be the proper test. Conclusions Analysis of the data supports the following conclusions, significant at the .05 level. 1. Student cognitive performance on extant instructional materials is improved when those students are supplied with performance objectives, assuming no adjunct questions are present. 2. Student cognitive performance on extant instructional materials is improved when those students are supplied with an adjunctive program, assuming no objectives are present. L. Denis Hlynka 3. The addition of bgth_adjunctive questions and performance objectives does not increase student learning beyond what is learned when either objectives or adjunct questions are available alone. 4. Students show a more positive attitude towards extant materials when those materials are accompanied by performance objectives. 5. The addition of adjunct questions to extant instructional materials does not result in the development of a positive attitude towards those materials. 6. Study time and total time used as covariates were not able to explain the results on the achievement or attitude dimensions, in this study. LEARNING FROM EXTANT MATERIALS: THE EFFECTS OF PERFORMANCE OBJECTIVES AND ADJUNCTIVE PRACTICE QUESTIONS By L. Denis Hlynka A DISSERTATION Submitted to Michigan State University in partial fulfillment of the requirements for the degree of DOCTOR OF PHILOSOPHY Area of Instructional Development and Technology 1975 ACKNOWLEDGEMENTS Only one name appears on a dissertation...the name of the researcher. Yet it is only through the combined input of many individuals that such a study is possible. First and foremost, I am especially indebted to Dr. Kent Gustafson, who headed the doctoral committee. His guidance, encouragement, and assistance at every stage of the study were invaluable. Second, I wish to thank the other members of the committee for their time, patience, advice and willingness to assist: Dr. Erwin Bettinghaus, Dr. James Page, and Dr. Stephen Yelon. Although not a member of the committee, Dr. Paul Witt must be singled out for providing constant encouragement. Several individuals gave invaluable statistical assistance. These include Dr. John Schweitzer and Dr. William Schmidt, both of Michigan State University; and Dr. Brian MacPherson of the University of Manitoba. Other individuals who participated, and to whom thanks are graciously acknowledged, include Dr. Castelle Gentry, Dr. Curtis McCarty, Dr. Barry Bratton, Mr. Robert Price, and Mr. 5 Mrs. Daniel Huzyk. Special thanks go to Mr. Geoffrey Saville and Dr. Fritz Kramer...friends and colleagues who made the entire experience both ii enjoyable and memorable. Advisors, committee members, friends, colleagues, family... all these and many more fbrm an integral and significant part of what has become this dissertation, and the culmination of this doctoral program. iii ACm TABLE LIST LIST Chap1 TABLE OF CONTENTS ACKNOWLEDGEMENTS . . . . . . . TABLE OF CONTENTS . . . . LIST OF TABLES . . . . . . . . . LIST OF FIGURES . Chapter I INTRODUCTION. . Overview . . . . . Background of the Problem. . . Statement of the Problem . . . . Hypotheses to be Tested . . . . Delimitations of the Study . . Limitations of the Study . . . . Assumptions . . . . . . . Chapter Summary . . . . . . II REVIEW OF LITERATURE AND RELATED RESEARCH Adjunctive Programming Techniques. Review of Relevant Practice. . . An ID Perspective . . Review of Research: Chapter Summary . III DESIGN OF THE STUDY . Introduction . . Experimental Design Variables . . Treatments . . Hypotheses . . Instrumentation . Review of Research: Adjunctive Questions. Performance objectives Construction of the Instructional Unit . Construction of the Performance Objectives . Construction of the Adjunct Program Materials Other Instructional Materials . . . iv Page ii iv vi vii 13 14 15 16 17 17 19 19 23 25 28 33 38 40 40 4O 40 41 42 44 45 47 48 49 Chapter Development and Validation of Criterion Instruments Reliability of Instruments. . . . Collection of Time Data . . . . Collection of Other Data. . . . . Subjects. . . . . . . . . Loss of Subjects . . . . . . . Missing Data . . Preparation for Computer Analysis of. Data Chapter Summary. . . . . . . . IV ANALYSIS OF DATA . . . . . . . . Introduction. . . . . . . . . Findings. . . . Regression Analysis and Analysis of Covariance Controlling for Study Time. . . . Regression Analysis and Analysis of Covariance Controlling for Total Time. . . . . Analysis and Interpretation . Interpretation of the Cognitive Hypotheses. Interpretation of the Affective Hypotheses. Other Findings . . . . . . . . V SUMMARY, CONCLUSIONS, RECOMMENDATIONS . . . Summary . . . . . . . . Conclusions . . . Discussion . . . . . . . . . Implications. . . . . . . Implications for Future Research . . . Implications for Instructional Developers . Implications for Classroom Teachers . APPENDICES . . . . . . . . . . . . BIBLIOGRAPHY . . . . . . . . . . . . Page 50 51 51 52 53 53 S4 55 55 S7 S7 58 64 66 67 67 69 72 77 77 78 79 82 82 83 84 85 . 117 Table Table Table Table Table Table Table Table Table Table Table LIST OF TABLES Summary of Hypotheses . . . . Two Way Analysis of Variance Table fOr Cognitive Test Scores . . . . . . Two Way Analysis of Variance Table fer Affective Test Scores . . . . . . Test for Significance of Relationships between Covariate (Study Time) and Dependent Variables . Test fer Significance of Relationships between Covariate (Total Time) and Dependent Variables . Appendix K Least Square Estimates of Effects: Effects X Variables . . . . . . . . . . Standard Errors of Least Square Estimates . . Analysis of Covariance for Achievement Test Scores Using Study Time as Covariate . . . . . Analysis of Covariance for Attitude Test Scores Using Study Time as Covariate . . . . . Analysis of Covariance for Achievement Test Scores Using Total Time as Covariate . . . . . Analysis of Covariance fer Attitude Test Scores Using Total Time as Covariate . . . vi Page 43 63 6S . 66 . 114 . 114 . 115 . 115 . 116 . 116 LIST OF FIGURES Page Figure l Treatments. . . . . . . . . . . 41 Figure 2 Cell Means: Achievement . . . . . . . 68 Figure 3 Cell Means: Attitude . . . . . . . . 68 Figure 4 Interaction: Achievement . . . . . . . 70 Figure 5 Interaction: Attitude . . . . . . . . 70 Appendix K Figure 1 Cell Means: Time Spent Studying in Minutes. . . 113 Figure 2 Cell Means: Total Time Spent in Minutes . . . 113 vii CHAPTER I INTRODUCTION Overview The research reported in this dissertation was conducted at Michigan State University during the summer and fall quarters of 1974. The purpose of the study was to investigate the effects of performance objectives and an adjunctive program on cognitive and affective learnings from extant print materials. The term "perfbrmance objectives" or "behavioral objectives" is used here to identify a list of information given to a student prior to instruction. The list informs the student as to his anticipated learning outcomes in terms of behavior which he should be able to perform after the instruction (Peterson, 1971). The term "adjunct program" or "adjunctive program" (Briggs, 1970) is used here to refer to a set of questions to be used adjunct to, or in conjunction with, existing materials in order to assist the student in practicing responses appropriate to the instructor's objectives. Normally, these adjunctive materials would be presented and used concurrent with the extant instructional materials. The term "extant materials" or "extant instructional materials" refers here to existing materials, originally prepared 2 fer purposes other than use within a specific learning package. The original use of the materials may have been instructional, but within a different context; the original use may have been generally informative (and not specifically instructional); or the original use may have been primarily entertainment. In addition, as used in this study, the term "extant materials" implies the combination of materials which have not been used together befbre, and not specifically designed as a package or unit. The results of this study may provide theoretic support for certain activities of the instructional developer, as well as practical data of relevance to practicing teachers. Finally, the study should add to the body of research knowledge dealing with the topic of performance objectives and adjunctive questions and/or programs. Background of the Problem Instructional development has been defined variously as "a process for improving the quality of instruction" (Gustafson, 1971, p. 18); a "systematic process of bringing relevant instructional goals into effective learning activity" (Hamreus, 1971, p. 70); and as being concerned with the "design, validation, revision, dissemination, installation, operation, and evaluation of instructional products or systems" (Stowe G Schwen, 1973, p. S). Implicit in the instructional development (ID) concept, seems to be the idea that ID starts "at the beginning" and leads towards the development of "new" instructional products and/or lirograms. These resulting products or programs are expected to be directly relevant to pre-determined objectives, student needs, careful cost-effective analyses, and other identified constraints, limitations, and opportunities. Countless models of ID have been developed over the past few years, all based on a feedfbrward "define-develop-evaluate" paradigm (Stamas, 1974). What seems to have been lost in these ID models is the use of extant materials, that is, those materials originally developed for other audiences and other purposes than those of the specific product or program under design. Thus, as related to a specific learning package, such extant materials would probably have vague objectives, be based on little or no needs assessement, and be characterized by an absence of tryout, validation, or revision. This tendency of developers and instructors to overlook extant media is pinpointed in a recent mediated (slide-tape) presentation (Esseff, undated) which begins: When instructors talk about selecting a textbook or a film, or any other material, it is often in terms of selecting something new. However, as the present tight economy restricts the growth of school budgets, it becomes important to make the best use of existing materials and media. Indeed, thousands of textbooks, articles, reports, papers, tapes, films, games, and other materials which may be used for instructional purposes are available on today's market. These are often not designed so much to teach, as to convey general 4 infbrmation. Commenting on the effectiveness of textbooks as extant instructional materials, Espich and Williams (1967) write that "the effectiveness of these books as teaching books depends upon...the student's ability to differentiate the important material in the book from that which is of lesser importance" (p. 83). The problem of effective utilization of extant materials would appear to have at least two distinct dimensions. First is the problem of retrieval. How do we find out what's out there, and how can we obtain it? Second, given that the extant material can indeed be located and obtained, how can we use it most effectively and efficiently for classroom teaching and learning? The first question has been addressed many times, and sophisticated information handling systems are daily becoming yet more sophisticated. To date, most of these systems, such as ERIC, MEDLINE, and at a theoretic level, the work of Havelock (1971) within the educational diffusion literature, all tend to stress identification of extant materials, but ignore the problem of actual retrieval. Among current examples focusing upon the latter concern (of which there are very few indeed) are the Educational Information Consultant project of the Far West Regional Laboratory (Banathy, 1972) and the Resource Utilization Project (Nord, 1971). However, the second question...how can we most effectively use the materials once located...has not received the attention of the first. It is assumed that once a teacher obtains the necessary materials, he will know how to use them effectively. Research, unfertunately, seems to suggest otherwise. Thus, Zaccaria and Adams (1964) point out that "hundreds of lml' millions of people have been learning knowledges and skills in the past...from books, classroom lectures, by trial and error, and such other methods as all experts in programmed instruction abhor" (p. 180). Given the above, the same writers ask, "How can we improve the efficiency of education and training that will continue to be accomplished with unprogrammed instructional materials" (p. 180)? In most general terms, the problem is to determine whether extant teaching materials can be taken and develOped, or redeveIOped, so as to increase the effectiveness of the material for the learner in some new context, not necessarily intended by the producer of the original material. An implicit limitation would be not to alter physically the extant material itself. While there is, of course, nothing "wrong" with physically altering any extant materials, the reader is reminded that such a strategy would be moving once again to a more time consuming and expensive approach. On the other hand, the intention of this study is that the processes being researched here would be fairly easy to implement by teachers (with a little training) within existing facilities, under present administrative organizations, and given the current budget "crunch". In short, the processes of instructional redevelopment, if effective, are quite feasible, compared to many new innovations requiring massive changes. At this point, perhaps an example is appropriate. Suppose a teacher of high school physics discovers an article in Scientific American which is of direct relevance to a unit he is teaching. The «original article may not have the same goals as the teacher, and indeed will probably have an entirely different purpose and intended audience. Yet this is potential teaching material. The question, at its basic level, is what can that teacher do to incorporate this article into his teaching in such a way that the student will benefit maximally from it? Research suggests at least two strategies which may increase student learning from extant materials. First, a list of performance objectives may be used to focus the student's attention upon what the teacher expects him to learn (which may or may not correspond to the original intent of the author). However, results of the behavioral objectives research seem at best to be uncertain. In particular, two recent and thorough research reviews (Duchastel 6 Merrill; 1973; Walbesser 8 Eisenberg, 1972) both show about a fifty-fifty split as to whether or not behaviorally stated objectives do make a significant difference. Some studies suggest that objectives do make a difference (Dalis, 1970; Blaney 5 McKie, 1969; Doty, 1969); others suggest that it doesn't much matter either way (Cook, 1969; Jenkins 8 Deno, 1971); and a very few studies, such as a fascinating one by Yelon and Schmidt (undated) suggest that, under some circumstances, objectives may even have a somewhat negative or interfering effect. A second strategy suggested by research comes from the programmed instruction field. Most users of programmed instruction are familiar with the two most common formats...linear and branched. But Pressey, since about 1926 has advocated yet a third variety... adjunctive programming...a technique which as Briggs (1970) remarks, has new "may 31' laborat been ex buSines a5 infc and Pro “SQarc repreSe Mre P0 It is has never really caught on. To Pressey (1967), adjunct programming "may aid study, facilitate teaching, greatly aid in learning laboratories or other plans for guided independent study” (p. 239). Zaccaria and Adams (1964) suggest that: There are many possible ways of developing adjunct programs to already existing texts. The main service these adjuncts would provide is to call attention to the educational or training objectives. One method of presenting an adjunct is to list all of the relevant test questions for a text. A refinement of this is to indicate on which pages and in which paragraphs of printed materials the answers can be found. A further refinement could be to deveIOp a linear program to give students basic knowledges such as terminology; this could be used in conjunction with questions on the text. (p. 180) Most recently the concept of adjunctive programming has ‘been extended and applied to make technical communication in business and industry easier and quicker using a technique known as information mapping (Horn, 1974). Closely related to developments in adjunctive programming and providing relevant research support for the adjunct approach, is Imesearch on learning from written materials. This research, irmepresented by names such as Rothkopf, Frase, and Anderson, appears ulure positive and definite than the behavioral objectives research. It is generally concluded that questions can and do increase both intentional and incidental learning depending upon the type of question, location of question in text, and contiguity of question to information to be learned (Frase, 1967). However much of this research has been carried on in carefully controlled laboratory settings. The research also involves physical manipulation of the materials utilized by inserting questions directly into reading passages. Further, whether the transition from adjunct question to adjunct program is merely one more step, or is indeed more like a quantum leap, remains to be researched. Whether or not equally dramatic results as Rothkopf and Frase have obtained would appear in a "real world" classroom setting, and without "tampering" with the original extant materials, is not known and is the focus of the present study. If one strategy for improving student learning from extant materials is to employ performance objectives; and if a second strategy is the development of an accompanying adjunct program; then a third potential strategy would simply be a combination of both objectives and adjunctive questions. To the instructional developer, it is important to know whether the effectiveness of his product might be improved by either objectives, questions, or a combination of both. To a teacher, it is similarly important in terms of strategies which are most effective for utilization of the vast numbers of extant materials which are already so easily accessible for use. Some research has attempted to document the effects of combining two or more components or strategies other than those under which [1971: organ: treat] organ. tepic postt 2 x 2 or it only Furtl of ti to b. note time to I Sch. the C01 °Pe under consideration in this study. Such studies have used a methodology which stresses potential interactions. One such study by Peterson (1971) looked at separate and interactive effects of advance organizers, post organizers, and behavioral objectives. Eight treatments varying each of the possible combinations of advance organizers, post organizers, and objectives were prepared on a tapic in Grade 8 mathematics. Data were collected for an immediate posttest and a delayed retention test, then analyzed within a 2 x 2 x 2 factorial design via analysis of variance. 0n the immediate posttest, none of the main effects or interactions were found to be significant. On the retention test, only a significant main effect for the post organizer was found. Further, an examination of the data indicated that the presence of the post organizer seemed to hinder the retention of the concept to be learned. Several reasons might be advanced to explain the essentially "no significant differences" findings of this study which served as a guide and model fer the present dissertation. First, Peterson noted that students were rushed and may not have had sufficient time to study the lesson adequately. The present study proposes to present the treatment materials within an open-ended time schedule, allowing students to work at their own rate and to take the posttest when they feel ready. As a result, time data will be collected as part of the study. Second, there is a possibility that Peterson did not (mperationalize his perfbrmance objective sufficiently. The one 10 objective was stated as: Upon completion of the study of this booklet you are expected to know the general rule for tracing networks and to be able to solve simple problems based upon the application of the rule. (p. 3) The present study proposes to present performance objectives at a much greater level of specificity than did Peterson. Several positive features from this study will also be incorporated into the present research. Specifically, the idea of a unit which may be completed within one class sitting will be retained. Similar procedures will be utilized in terms of selection of subjects, random sampling, and preparation of treatment materials. Another somewhat similar study of direct relevance was conducted by Doty (1969): The purpose (was) to investigate and to provide evidence relevant to the effectiveness of two strategies of instruction as measured by immediate learning: (1) Providing or withholding prior knowledge of behavioral objectives, and (2) Providing practice on the actual referent (object) and the symbolic referent (written description). (Abstract) Doty found the following: 1. "There was a tendency for the students receiving prior knowledge of educational objectives before a unit is taught to have higher test scores than those students who did not receive prior knowledge of...objectives." (Abstract) impel Ha 11 2. Practice made no difference. 3. No interaction was indicated. 4. Individual differences due to environment and schools were isolated. The present study will differ from Doty's in several important respects: 1. Emphasis will be on the use of extant material, while Doty used a rigidly structured unit taught by teachers. 2. Doty used a population of grade 7 students. The present study will fbcus on college level students. 3. Doty administered a pretest. This study will not, on the assumption that a pretest may be a confounding variable. 4. Doty emphasised problem solving skills. This study will concentrate on knowledge and comprehension levels of the cognitive domain. 5. "Practice" as used by Doty is not congruent to the adjunctive programming technique to be examined in this study. 6. Doty used elementary science (values and tolerance of carbon axial resistors) as a subject area. This study will use educational media (film music as a variable in instructional product design) as a subject area. A third study of direct relevance to the present research was conducted by Cook (1969). He examined the effect of behavioral objectives and a learning heirarchy on both learning and retention. 1 vacuum rm " At 1e: additi Final be cg aPPTC 12 At least two of Cook's "Implications for Current Practice" deserve additional investigation, namely the following: 1. The findings of this study lend no support to the assertion that telling students the behavioral objectives...will increase their performance on immediate achievement tests. 2. The stating of behavioral objectives in textbook fermat without explanations by the teacher do not significantly help the student to perform higher on an immediate achievement test. There is some basis to believe that such infermation may even confuse the student. (p. 122) Finally, several of Cook's "Implications for Future Research" will be considered and adopted into the design of the present study where appropriate: 1. "Studies should be conducted...without the use of programmed materials" (p. 123). This potentially confuunding factor so often used in a surprisingly large number of studies will not be used in this study. The implication of Cook's statement is that programs especially of the linear type will cue in the student to the objectives such that it will later be impossible to determine whether any effect was due to the objectives or to the program. "Future studies might more effectively be conducted if the possibility of sharing treatments outside of 13 class were minimized" (p. 123). Apparently this was a problem which Cook encountered. However, the present study will present the stimulus material and posttest within a single class period (of approximately one hour) thereby allowing no opportunity fer such interaction to take place. 3. "Studies should be conducted in a typical classroom situation" (p. 123). This study will be conducted in actual classroom settings, as part of the onegoing instruction of the selected course. 4. "Studies should be conducted in other academic areas and academic levels" (p. 123). The present study will emphasise college level graduate education majors, extant materials, and the topic of film music. Statement of the Problem It is in the context of the above discussed factors...(l) the need for a more detailed examination of how to use extant materials more effectively and efficiently; (2) the need to further explore the potential contributions of performance objectives to learning; and (3) the need to further explore the potential contributions of adjunctive programming techniques to learning...that the fbllowing problem statements have been fermulated. What are the separate and interactive effects of performance objectives and adjunctive programming on immediate cognitive learnings from extant print materials? objecti adunis nateria indicat results spent 5 on the l the pop a gradtl 14 What are the separate and interactive effects of performance objectives and adjunctive programming on an attitude score administered immediately fellowing presentation of extant print materials? If differences between objectives and adjuncts are indicated in either cognitive or affective domain, can these results be explained by consideration of the amount of time students spent studying the material, or the total time the students spent on the entire package? In each of the preceding questions, as defined in this study, the population may be characterized as mature adult learners taking a graduate course in educational media. Hypotheses to be Tested From the above stated research questions, a total of six hypotheses were derived. Stated here in their research ferm, these hypotheses are: H : Giving students statements of performance objectives before 1 extant instructional materials will result in higher achievement scores than when no objectives are given. H : Giving students an adjunctive program to be used with extant materials will result in higher achievement scores than if no such program is given. H : There will be an interaction between objectives and adjunct program on the achievement measure. H:GW ext ma lat: pm} use stud) for thes between V "9 defixl by the n "Strict, mama 15 H : Giving students statements of performance objectives befOre extant materials will result in a more positive attitude score than if no objectives are given. H : Giving students an adjunct program to be used with extant materials will result in a more positive attitude score than if no such program is given. H : There will be an interaction between objectives and adjunct program on the attitude measure. In addition to the above hypotheses, it is proposed to use study time and total time as covariates, in order to control fer these variables. Delimitations of the Study In this and the following section, a distinction is made between the terms "limitations" and "delimitations". Delimitations are defined here as the workable limits (Doty, 1968, chap.l) selected by the researcher. In other words, delimitations are those restrictions which are researcher-selected in order to keep the study to a reasonable size and scope. The following are the delimitations of this study: 1. The study was delimited to college students enrolled in a course in educational media (831-A) offered for credit by Michigan State University. 2. The extant materials used were delimited to a print only mode of communication. Other forms of extant material...films, audiotape, etc...were not considered. aSpects (Doty. A Study: 16 3. The posttest was administered immediately after treatment. Long term retention was not analyzed. 4. The content area was delimited to a unit on film music as a variable in instructional product design. 5. The objectives, adjunctive questions, and posttest questions were prepared to test for lower level cognitive learnings only. Limitations of the Study Limitations, in contrast to delimitations, are "those aspects of the study which can be identified but not controlled" (Doty, p. 14). The following are the limitations of the present study: 1. The learning package was written primarily fer the purpose of this study, and thus may be out of context with the on-going activities of the course. 2. To obtain a sufficient sample size, it was necessary to call upon three instructors of the same course, conducted in three different cities (Lansing, Jackson, and Grand Rapids...all in Michigan) and at three different points in time. 3. Results of this study may not be generalized beyond the specific population type, and subject area selected. 17 Assumptions The following assumptions were made in order to design and conduct this study: 1. Since the total sample size of 80 subjects will be achieved by combining three classes of the same course taught at different locations and at different times, it is assumed that location, the instructor variable, and the point in time at which the study was administered to each group will have no effect on the results. Any loss of subjects which may occur will not be due to the nature of the study, and will be random. The achievement and attitude tests are valid and reliable. Entry level of all subjects on the topic of film music is at or near zero. Chapter Summary_ This chapter has attempted to present the framework around which the present study was constructed. Briefly, the researcher began with a concise statement of the problem. This was followed by an overview of the relevant background, which put the need for the study in proper perspective. Next followed a detailed problem statement, along with the six hypotheses to be tested, given in the research form. Finally a list of delimitations, limitations, and assumptions were set forth. 18 Chapter II will review relevant research and literature of interest to this study. Chapter III will outline the design and methodology of the study. Chapter IV will present the results of the statistical analysis of the data, along with an interpretation of these results. Finally Chapter V will present a summary, conclusions, and implications. CHAPTER TWO REVIEW OF LITERATURE AND RELATED RESEARCH This chapter will review research relevant to the present study. The several areas of interest which impinge upon the present research problem have been categorized and reviewed as fellows: (a) Review of non-research literature dealing with adjunctive programming techniques, (b) Review of research on adjunctive questions, and (c) Review of research on performance objectives. Adjunctive Programming_Techniques Programmed instruction has developed over the years into a significant and sophisticated form of instruction. In Britain, the term "programmed learning" is more commonly used, which Callendar (1969) explains is preferable as it emphasises a learner orientation, as opposed to an instructor orientation. Regardless of the term used, programmed instruction in its broadest sense is usually considered to be one of two standard formats. The first is linear programming associated with the name of Skinner. The second is the branching method associated with the name of Crowder. There is also a third, less formal, and usually neglected form known as adjunctive programming, introduced in 1926 by Sidney Pressey. As Briggs comments (1970), "Oddly enough, this simple technique has never 'caught on' to the extent that programmed instruction has" (p. 109). Briggs attributes this lack of interest 19 20 in the adjunct approach as a programming technique to the fact that it is more often considered a testing procedure (p. 109). Indeed an early Pressey title for a journal article, "A Simple Apparatus which Tests and Scores...and Teaches" (1926) would appear to give credence to Briggs' suggestion that there might be an overemphasis on the testing side of the coin. A second reason why adjunctive programming has never caught on might be the emphasis and seeming dependence upon mechanical devices to make the system work. In recent years, Pressey himself has returned to his adjunct programming concept, still coupled with its mechanical features. Thus in "Re-program Programming?" (1967) he reiterates that "brief incisive feedback materials adjunct to organized subject matter can aid study, guide and reduce need for instruction, (and) make feasible individualized progress greatly increasing efficiency" (p. 237). Pressey then goes on to suggest three "feedback devices" for making adjunctive programming a reality. Unfortunately, although much simpler than his earlier cumbersome machines, these also rely on essentially mechanical devices. The three devices specifically recommended by Pressey are the erasure card, the punch board, and the chemo card. Fortunately however, interest in adjunctive programming techniques has proceeded on other fronts without the mechanical outlook. Callendar, a British programmer, is one of several writers to devote several pages of her short basic text on programming to adjunctive techniques. Her definition is concise and useful: "Adjunctive programming is a means of utilizing existing teaching 21 materials and textbooks by preparing a program which instructs the student how to organize his learning" (1969, p. 75). She further suggests that adjunctive programs "because of their use of existing material are less expensive and less time consuming to prepare" (p. 75). The format of an adjunctive program, according to Callendar, may be in booklet form, or on audio tape. The adjunct program has three characteristics. First, it will refer the reader to information contained in the text or reference material. Second, it will ensure that the student is actively responding. And third, it will supply him with knowledge of results. Langdon (1973) in a guidebook for individualized learning has also included one chapter in which he discusses an "adjunct study guide". He comments that most existing instructional materials generally lack an interactive component, and that therein lies the strength of the adjunctive technique. Langdon's operational definition of the study guide approach includes four components: 1. Instruction is provided according to behavioral objectives. 2. Since this is an "adjunct" study guide, sources of existing instructional and informational material are required. 3. A means of structured interaction is provided, so that the student can assess if he is learning. 4. A means of confirmation to the interaction is provided so that students will know the correctness or incorrectness of their responses. (p. 65) VT.- adva Unfc adjt repc tex1 stm Pro} “It Pro; 22 Langdon concludes his chapter with a discussion of the advantages of the adjunctive technique: I. A much higher degree of learning effectiveness and efficiency should be achieved. 2. Once the objectives of a lesson are achieved, they will be "seen" in context. 3. A higher level of student preparation for in-class instruction can be expected. 4. A real cost savings can be expected in terms of development and use costs. Existing instructional materials constitute the bulk, if indeed not all, of the content. (pp. 79-80) Unfortunately, Langdon provides no data to support his claims. The military has also reported experimentation with adjunctive techniques. For example, Meyer (Pressey, 1967) has reported success in using self-instructional tests adjunct to textual materials. Zaccaria and Adams (1964) report that "recent studies at Lackland Air Force Base have indicated that an adjunct program can be less costly to develop and administer and yet produces as much learning as a conventional program" (p. 180). Especially relevant to the present study is their description of an adjunct program as a series of objectives in question ferm: This (adjunct) program asked questions of the student for each paragraph of text. The learner was forced to analyze the printed material and to write his answer to a question that tested his knowledge of the paragraph. 23 Student completion time was half that of the corresponding linear program. The questions in effect became the course objectives. (emphasis mine) We hypothesize that the student realized that the questions were his objectives and motivated himself to learn them. (p. 180) Finally, Briggs (1970) summarizes the relevant empirical research which he has reviewed, concluding that: An increasingly large body of research data indicates that a terse text accompanied by such self-test review items is an economical and effective mode of learning and is often a less expensive way to achieve results than to prepare new material in other media. Evidence also indicates that adjunct methods achieve equal results in less time than programmed instruction. (p. 110) The research review which Briggs has conducted has also led him to suggest several guidelines for the improvement of textbooks. These guidelines are: 1. State objectives. 2. Sequence materials to match the learning structure of the objectives. 3. Intersperse self test items at appropriate intervals. (p. 110) Review of Relevant Practice. While the writings discussed above all show a basic similarity of purpose and procedure with respect to adjunct programming, there is nevertheless much room for individual 24 perception of what, in practice, such adjunct programs will look like to the reader. For this reason, this section will examine three different approaches actually employed in textbooks. These approaches also serve as a model for the construction of the adjunct materials for this study. Ivor Davies (1971) introduces each chapter of The Management of Learning with a list of some five to ten objectives, and repeats these at the end of each chapter as a posttest. Knowledge of results is not given. A typical Davies objective might be "After carefully reading this chapter, you will be able to state the five functions of a criterion test" (p. 207). The corresponding posttest question appears as "State the five functions of a criterion test" (p. 214). A similar method is used by DeCecco (1968) in his Educational Psychology text. Typically, each DeCecco chapter lists some twenty-five to thirty-five objectives such as "Define problem solving in terms of the definition of learning in Chapter 7" (p. 429). In addition to the objectives, three and at most four questions appear for each chapter, one at the end of each section, ostensibly to provide review for each subsection of that chapter. However, whether one question alone is sufficient to cover approximately ten objectives is certainly open to debate. Knowledge of results is provided for the adjunctive questions, but not for the objectives, and there is no posttest. Yet a third approach is employed by Gerlach and Ely (1971) in Teaching and Media: A Systematic Approach. Truly the most systematic of the three reviewed approaches, each Gerlach-Ely chapter 25 begins with approximately five objectives asking the students to identify, name, describe, order, or construct...operations which the authors claim will provide a strategy "to write objectives for nearly all elementary and secondary school learning" (p. 79). Adjunctive practice questions are liberally interspersed within the text material in the farm of a linear program section. Each of these sections is color coded to distinguish from the main body of text. Knowledge of results is provided. Finally, a summary is given which concludes the chapter verbally and sends the student back to the original objectives as a self check, or refers him to selected adjunctive questions. While many other effective examples might also be cited here, the above three serve to illustrate the wide variety of available strategies to follow. An ID Perspective. The above review of adjunctive programming techniques suggests an interesting alternative to the typical instructional approach. Instructional development as currently practiced implies, if not actually states that the development of relevant instructional media start "from scratch". While the concept of new development of materials to achieve Specified objectives is sound and certainly necessary, there are certain limiting factors which operate against the instructional deveIOpment approach. At least three essentially pragmatic limitations stand out. First, ID is expensive. Even on a small scale a new development effort can tax one's budget, and this is especially true for large scale 26 projects. Second, ID is time consuming. The actual product development alone takes much time, and product development is only one stage of the multi-stage process of instructional development, whichever model one fellows. Third, ID requires considerable expertise. This expertise is usually provided by a single developer, or by an instructional development team. These limitations are significant, whether one is talking about large scale or small scale projects. Rothkopf (1973) sees an alternative to new development of instructional materials in a redevelopment process: Rational improvement in instruction can be approached not only through the systematic design of instructional products and methods, but also through the enlightened creation of instructional environments that are designed to foster effective learning activities. (p. 126) To Rothkopf, the term "instructional environment" implies the fostering of effective mathemagenic activities. This latter term is one coined by Rothkopf (1970) to mean "the student activities that produce the desired learning outcomes" (Rothkopf, 1973, p. 126). Examples of methods which foster mathemagenic activities would include the use of questions, adjunct programming, and statement of goals and objectives. With the above definition, Rothkopf is able to make a distinction between the development of instructional products, and the development of instructional environments: 27 Two major philosophies for the improvement of education vie with each other for our attention. Both of these demand careful analysis of instructional goals. The first of these philosophies emphasises the deve10pment of stable and effective instructional means through empirical techniques, such as tryouts, careful measurement of results, and repeated revisions of the instructional package. This philosophy requires substantial investment in instructional development. Experience indicates that this approach is effective but is fairly expensive in the current state of the art. Furthermore, it appears to be better suited to instructional situations in which the future use of the relevant skills can be accurately anticipated. The alternative approach is to treat the nature of the instructional materials and other instructional forms as givens. Emphasis is instead on the creation of an instructional environment that will help students to select and process suboptimal materials to his best advantage and to maintain effective study activities. Such an environment involves...the festering of the student's active participation in the instructional process. (p. 127) While the above lengthy quote is quite general and might apply to several kinds of mathemagenic activities as solutions, it nevertheless supports the proposed strategies of this study. Instructional development is indeed one solution, but it is a solution which is 28 often not feasible for the practicing teacher. If, then, one cannot develop new materials whenever necessary, is there some way to utilize existing materials and media more effectively? This study has proposed to examine the roles of performance objectives and adjunctive programming techniques as a step to answering that question. Next, however, it is necessary to review relevant research within these two domains. Review of Research: Adjunctive Questions Although current deve10pments in the field of adjunctive questions are most associated with Rothkopf, Frase, Anderson, and others several related areas and studies predate and provide the groundwork for contemporary interest. They include the entire programmed instruction movement led by Skinner, Crowder, Pressey, and others; the incidental learning studies of Postman and associates; and the entire domain of "how to study" perhaps most popularized by the famous 803R method, and its many imitators and variants. A 1947 study by Carmichael and Dearborn (Anderson, 1970) examined the effect of reading six hours at a time. Subjects were high school and college students. Under study was the factor of attention, determined by monitoring of student eye movements. When the reading was interrupted by tests every twenty-five pages, the experimenters were able to report more constant performance and a virtual absence of fatigue on the parts of experimental subjects as compared to no-questions subjects. A second study which fereshadowed in many respects the 29 current work of Rothkopf and others concerned the value of note taking during film viewing (Ash 8 Carlton, 1953). Four groups were given the following treatments. Group A saw a film and took a posttest. Group B saw the film, took notes while viewing, then took the posttest. Group C saw the film, took notes while viewing, reviewed their notes for ten minutes, then took the posttest. Group D was a control group which took the posttest only. Test data fer the sample of 216 subjects indicated that: Taking notes while watching the film resulted in reduced test scores. Review of notes befere taking the test resulted in slight gains over note-taking without review. The highest test scores were made by the group that watched the films without taking notes. (p. 124) The above conclusions are interesting and deserve replication. The relevance of the Ash and Carlton experiment to the current one lies particularly in potential extension of mathemagenic activities into other media, in this case film. In addition, the results suggest that there may be times when certain activities hinder learning. A related study reported in AV Communication Review (Roberts and Parchert, 1962) asked the similar question, "Do worksheets improve film utilization?" and concluded that : l. worksheets can produce increased learning. 2. Worksheets impress on pupils' minds definite aims for the viewing. 3. Werksheets present a task that eliminates the time- conditioned idea that motion pictures are only for 3O entertainment. 4. Worksheets tangibly anchor certain information, thus eliminating the usual film disadvantage...presentation of fleeting information. 5. (The worksheet) helps make learning permanent. (p. 109) While this study lacked the more rigorous controls which one would like to have seen in research of this type, it nevertheless provides some intriguing intellectual fedder. Of course, differences between worksheets, taking notes, tests, and adjunctive questions are obvious, and generalizations in any direction should await further studies. Modern interest in the use of questions began in the middle 1960's. In 1966 Rothkopf took a 5200 word passage from Rachel Carlson's popular non-fiction book The Sea Around Us, divided it into sections of three pages each, and had volunteers answer questions based on the sections, followed by a criterion posttest. The questions were placed either before or after each section of prose, and with or without knowledge of results. Rothkopf found that intentional or specific instructional effects were found when questions were presented either before or after the reading, and were greatest when knowledge of results was given. General facilitative learning, or incidental learning, was greatest when questions were placed after reading. Knowledge of results was not a factor. Rothkopf's study has since been replicated and confirmed in several variants by other researchers (Frase, 1967; Rothkopf G Bisbicos, 1967). Anderson summarizes these, stating: 31 Subsequent research confirmed that interspersing questions in reading material either before or after the section of the passage on which they are based, improves performance sharply relative to a no-question group when the same questions are again repeated on the criterion test. These studies also confirm that performance on non- practiced criterion test questions improves only when questions are inserted in reading material after relevant passages. (p. 360) Frase (1967) replicated Rothkopf's findings, and added the length of the written passage as an additional variable. Further, he used multiple choice questions, in contrast to Rothkopf who used completion questions. His design was a 2 x 3 x 2 factorial design. The three factors were questions before or after reading passages, length of passage, and knowledge of results present or absent. Posttest analysis fecused on retention questions, or questions which had occurred during reading; and incidental questions, or related questions not specifically tested by the retention questions. Frase found a main effect which showed three factors as being significant for retention questions: length of passage, position of questions, and presence of knowledge of results. For incidental learning, a main effect was feund only fer position of questions. Specifically questions occurring after the passages fostered incidental learning. A 1973 study by Rothkopf and Billington has extended these series of studies to the concepts of indirect review and priming. Findings indicated no significant differences for priming (that is, 32 perfbrmance on a test question if a related question has been studied), and a significant effect fer indirect review (that is, answering one question will facilitate response to other questions not directly related). The "pay off" from the above reviewed research findings lies in the promise of an alternative to more expensive and time consuming instructional design. Rothkopf states (1970): The concept of mathemagenic activities tends to shift emphasis from investment of resources in the development of instructional materials, to the investment in the instructional environment. Instructional materials are accepted within some limits as givens. Emphasis in instruction is on promoting those activities in the student which will allow him to achieve instructional goals with available materials. (p. 334) This statement is precisely the rationale behind the present study... the identification and examination of a practical strategy to improve utilization of extant materials and media. Rothkopf does however issue one caution to those who would implement his findings and emphasis on the instructional environment as an alternative to instructional development: How to manage this poses many practical problems. But it may be more economical than expensive concentration on the detailed design of instructional material. (p. 335) The challenge to operationalize the above research findings is clear. The present study focuses upon one aspect of that problem. 33 What happens when adjunct questions are superposed upon performance objectives? The final section of this chapter will look at the relevant behavioral objectives research. Review of Research: Performance Objectives According to Davies (1971) and Nord (1969), the behavioral objectives movement had its origins within four distinct areas: "programmed learning (Mager, 1962), military instructional design (Miller, 1962), school examinations and the measurement of achievement (Bloom, 1956), and changing emphases in management philos0phy (Drucker, 1954)" (Davies, p. 36). Since then the behavioral objectives movement has burst full strength upon the educational community. Numerous writers have stressed the necessity of behaviorally stated objectives in the construction of instructional programs and products.(Mager, 1961; Burns, 1972) And a seemingly equal number have warned us against their dangers (Atkins, 1968; MacDonald Ross, 1973; Ebel, 1970). The ultimate resolution of the objectives controversy, one hopes, will not be resolved by polemics, but by empirical evidence obtained from research. There are several rationales for expressing objectives in education. Duchastel and Merrill (1973) suggest that objectives serve three major functions. First, objectives provide direction for teaching and curriculum deve10pment. Second they provide guidance in evaluation. And third, they facilitate learning. The present 34 review is concerned with the third of these categories, that is, objectives used for the express purpose of assisting the learner. Tieman (1967) investigated the effects of general or specific objectives on two versions of televised instruction. The subject area was college economics, and the treatment period was of four weeks duration. On an immediate posttest, no differences were feund for the types of objectives. However, on a delayed posttest, differences were found in favor of the specific objectives group as compared to the general objectives group, significant at the .05 level. Duchastel and Merrill (1973) have pointed out that Tieman's differentiation between types of objectives was limited to include a specific measureable behavior, such that behaviorally stated objectives asked subjects to "Recognize that..." while the generally stated objectives asked subjects to "understand the relationship..." As a second finding, Tieman also reported that students favored the behaviorally stated objectives as indicated by a more favorable attitude associated with this group. Weinberg (1970) studied the effects of presenting objectives at four levels of specificity for a ten-week course in beginning bowling. The four levels were: a no objectives group, a general objectives group, a terminal objectives group, and a group which received intermediate and terminal behaviorally stated objectives. No significant differences were reported between groups for bowling skill, knowledge, or form. However, like Tieman, Weinberg was able to report more positive attitudes in favor of 35 behaviorally stated objectives. Boardman (1970) looked at the effects of prior knowledge of objectives and no prior knowledge of objectives in a remedial chemistry class and found no differences in learning due to objectives. While the above three studies do not support the hypothesis that objectives improve learning, several studies do support this hypothesis. Blaney and McKie (1969) studied conference attendees who were placed into one of three treatment groups. One group received performance objectives, then attended the conference. A second group received a general orientation, then attended the conference. The final group received a pretest, then attended the conference. This Canadian.study found significant differences in favor of the group which had been given performance objectives over the general orientation group. The pretest group fell between the other groups, but differences were not significant in either direction. Dalis (1970), too, found significant differences in favor of groups given objectives in a study conducted in grade ten health education. He concluded that: It is possible to enhance health education classroom achievement by using precise instructional objectives in advance of instruction with high school age learners. These objectives however, must be stated precisely, otherwise their value to learning efficience is doubtful. (p. 22) This emphasis on precisely stated objectives could indeed be one 36 factor in explaining at least some of the no significant differences reported in similar studies. Dalis' operational definition of precise objectives further elucidates his point: The precise objectives contained explicit specific content, the kind of overt behavior expected of the learner when he is demonstrating mastery of the objectives, and the inclusion of what will be acceptable performance. (p. 21) Jenkins and Deno (1970) have suggested that there are at least feur levels of objectives, characterized by their degree of abstractness. Level A objectives are, in fact, general goal statements. Level B objectives suggest hypothetical dispositional states and are characterized by such terms as knows, understands, and appreciates. Level C objectives represent student capabilities, and are characterized by terms such as classifies, defines, and predicts. Level D objectives are most specific and related to test items, such as, states, marks off, checks, and points. Unfbrtunately, the authors have apparently not conducted research directly on this question. However, Jenkins and Demo (1971) have conducted research to test the effects of two levels, general objectives and specific objectives when presented to teachers alone, students alone, and both. The general objectives appear to be equivalent to their earlier Level B designation, while specific objectives approximated their Level D type. Results showed no significant differences, "neither knowledge of objectives, nor the type of objective differentially influenced performance on a criterion test" (p. 67). In another study, Kaplan and Rothkopf (1972) employed 37 a complex 3 x 2 x 3 x 2 factorial design to examine three prose passages, two levels of objectives (general and Specific), three levels of density (20%, 40%, 60%), and two kinds of learning (intentional and incidental). Intentional learning was defined in terms of questions relevant to objectives, while incidental learning was defined in terms of learning not specified by objectives. Density was defined as the ratio of relevant sentences to the total number of sentences, where a relevant sentence was a sentence directly rele- vant to an objective: Results showed that: (1) Intentional learning was greater than incidental. (2) Specific objectives resulted in higher performance than general objectives for intentional items. Specificity of direction had little or no effect on incidental learning. (3) Increases in density were accompanied by decreases in the proportion of intentional items that were correctly recalled. There were no measurable effects of density on incidental learning. (p. 2) Most interesting to Kaplan and Rothkopf was the implication that: carefully Specified instructional objectives will not interfere with the serendipitous discovery of information not directly relevant to instruction. This finding is reassuring because serendipity in education Should be a concern among educational technologists. (p. 2) While reassuring, the latter statement is somewhat confusing and possibly contradictory, especially when made by Rothkopf, who in other studies reported that questions placed before a passage 38 (as indeed objectives are), may tend to depress incidental learning. Finally, two excellent research reviews are available which thoroughly examine the state of the art concerning performance objectives in education. Duchastel and Merrill (1973) grouped some 40 extant studies into several categories, of which one is of direct interest to the present study. This was a grouping of ten studies which investigated whether or not objectives made a difference. Five studies Showed such a difference, while five showed no difference. The second major review was conducted by Walbesser and Eisenberg (1972). For the hypothesis that telling the learner the behavioral objectives increases achievement, the authors recorded eight studies supporting, and seven studies not supporting the hypothesis. Thus both research reviews bring out the rather uncertain conclusions and implications which may be drawn in this respect. For the hypothesis that students would Show a more positive attitude towards instructional objectives stated as behavioral objectives, Walbesser and Eisenberg found two studies supporting the hypothesis. Chapter Summary_ This chapter has summarized research and literature related to the present study. Techniques of adjunctive programming were examined and much variation was found in the construction of such materials. In particular three specific approaches were examined and used as models for this study. 39 Research on adjunctive questions was examined also. This research seems to be quite clear in its implications, namely, that the use of questions can and do increase both intentional and incidental learning, depending upon the type of question, the location of the question, and the contiguity of the questions to information to be learned. Finally, research on performance objectives was reviewed and analyzed. This research appears to be ambivalent. No firm conclusions are possible since some research suggests that objectives have little or no difference, while other research suggests that objectives do make a difference, in terms of improved student learning. Two major research reviews have also come to similar conclusions. There does, however, seem to be some agreement that objectives will lead to improved attitude on the part of students receiving such objectives. The present study will reexamine and extend some of the concerns brought out in the literature review. Specifically, what is the effect of providing specific performance objectives on learning from extant materials in a college level class? What are the effects of providing an adjunctive program on learning from extant materials? And finally, are there interaction effects when both adjunctive program and performance objectives are combined, and if so, what is the nature of the interactions? These questions are of primary interest to this study in terms of cognitive achievement and overall attitude. CHAPTER III DESIGN OF THE STUDY Introduction Chapter III presents a detailed description of the various phases of the research procedure used in this study. The first section presents the experimental design. The second section describes the development of the treatment materials and criterion instruments. The third section focuses on the collection of time data. The fourth section discusses the collection of other data of interest to the study. The final section covers procedures used in the selection and assignment of subjects. Experimental Desigfl_ This experiment studied the effects on learning and attitude of the presence or absence of performance objectives and adjunctive practice questions with extant learning materials. To this end, a 2 x 2 factorial design was selected "which best permits the study of the effects of two treatments each of which is varied in two ways" (Isaac and Michael, 1971, p. 51). Figure 1 illustrates this design. Variables. There were two independent variables to be manipulated in this study. They were: 1. presence or absence of performance objectives. 2. presence or absence of adjunctive questions. 40 41 ADJUNCT absent present 0 B J ABSENT TREATMENT ONE TREATMENT THREE E C T I V E S PRESENT TREATMENT TWO TREATMENT FOUR Figure l. Treatments The two dependent variables of interest to the study were: 1. 2. Treatments. follows: T : scores on a researcher-constructed achievement test. scores on a researcher-constructed attitude reactionaire. The experiment consisted of four treatment groups as Control group. This group received a learning package consisting of three extant articles on the topic of film music. (The specific components of the package are discussed in the second section of this chapter.) Objectives_group. This group received lists of perfermance objectives plus the extant materials. Adjunct group. This group received an adjunct program to accompany the extant materials. Combination group, This group received the total 42 package consisting of performance objectives, adjunct program, and the extant materials. All groups took the same posttest and attitude measure. A fuller description of each treatment, including both development and administration of all instruments is discussed later in this chapter. The format of the study fellowed a randomized control group posttest only design (Campbell 6 Stanley, 1963) for the following reasons: 1. No interaction between pretesting and treatments would be possible within this design. 2. A pretest was considered not to be necessary due to the believed low entry level of the selected population on the topic to be studied. (However, Students were given an "Infermation Sheet" befere treatment to check this assumption. 3. The prerequisite of random assignment would be satisfied. The decision to avoid a pretest was based on the assumption that such a pretest could be a confeunding factor in this study, since any gain score might then be attributed to the pretest as well as to objectives and adjunct questions. Hypotheses. As stated in Chapter 1, six hypotheses were generated to answer the research questions posed. Table 1 summarizes the derivation of these hypotheses in terms of the dependent and independent variables involved. 43 Table 1 Summary of Hypotheses Achievement Attitude Objectives Main Effect H H l 4 Adjunct Main Effect H H 2 5 Interaction H H 3 6 Stated in the research form these hypotheses were: H : Giving students statements of performance objectives before 1 extant instructional materials will result in higher achievement scores than when no objectives are given. H : Giving Students an adjunctive program to be used with extant materials will result in higher achievement scores than if no such program is given. H : There will be an interaction between objectives and adjunct program on the achievement measure. H : Giving students statements of performance objectives before extant materials will result in a more positive attitude score than if no objectives are given. 44 H : Giving students an adjunct program to be used with extant materials will result in a more positive attitude score than if no such program is given. H : There will be an interaction between objectives and adjunctive program on the attitude measure. Instrumentation Two types of instruments were constructed fer this study. These were (a) instructional instruments, and (b) testing instruments. All instruments were developed by the researcher for the purposes of this experiment. All of these materials are reproduced in the appendices.. The instructional instruments were those materials used in presenting the content to students in the various treatment groups. The instructional instruments included: 1. Extant instructional material in the farm of three articles from three different sources as the core of a learning package on the topic of film music. 2. A set of performance objectives to accompany each article. (These materials were given only to treatment groups two and four.) 3. An adjunct program to accompany each article, presented together at the beginning of the total package. (These materials were given only to treatments three and four.) The next section discusses the construction and administration of these instructional instruments. The criterion instruments, on the other hand, included an 4S achievement test and an attitude reactionaire. Both were developed by the researcher, and were administered to all treatment groups immediately following completion of the treatment materials. The construction and validation of these instruments follows the discussion of the instructional instruments. Construction of the instructional unit. Three extant articles from popular, entertainment, and research journals were selected due to their relevance to a unit on film music as a variable in instructional product design. Each article presented a different aspect of the topic. The topic of film music was selected for two reasons. First, film music may be considered a valid portion of a course dealing with educational media ( which was to comprise the setting for this study). Second, film music was assumed to be a topic usually ignored in an educational media course, and would thus suit the purposes of this study that subjects be unfamiliar with the topic used. It was on the basis of student unfamiliarity with the topic that a pretest was excluded. To confirm this assumption of unfamiliarity, a six question "Information Sheet" was completed by all students at the beginning of the experiment. This information sheet, asking students to indicate their familiarity with film music, is reproduced in Appendix A. In nearly all cases, the assumption that students have little knowledge of that subject was supported. The very few instances where students indicated some familiarity with the topic were examined separately by the experimenter. In each case, test 46 scores did not seem to bear out the supposed familiarity, and the assumption was made that even if some familiarity did exist, this had no effect on the results. As a double check of student's predictions of their own knowledge of film music, a specific question was included on the information sheet, designed as a spot check on such knowledge. The question, chosen by the researcher as one which was quite simple fer those knowledgeable in the field was: "Name as many films as you can, scored by composer Miklos Rozsa." Only one student, in the pilot study, identified one film, that film coincidentally having been shown the night befere on the television late movie. A second question on the information sheet asked students to summarize what research says about music in film. No correct answers were elicited here. The three articles, their sources, approximate length, and relevance to the total unit are described below: 1. "Movie Music Set the Meod." Washington Post, 1974. 950 words. Provides a light general overview with emphasis on the use of music by the entertainment industry, specifically Hollywood. 2. "Scoring Music: Into audio and audio-visual productions." Audio-Visual Communications, 1973. 1390 words. Provides a practical "how to do it" approach for adding music to small scale productions in business, industry and/or education. 47 3. "Effects of Familiar Background Music upon Film Learning." Journal of Educational Research, 1959. 2650 words. Provides a summary report of one research study which suggests strategies for future direction. Construction of the Performance Objectives. The performance objectives for each article were selected to focus on what the researcher considered the important components of that article. However, the objectives were limited to lower level cognitive learnings which would probably fall within Bloom's taxonomy at the "knowledge" or "comprehension" levels, or in the domain classified by Ausubel (1968) as "meaningful verbal learning". Most crucial was the problem of operationalizing the objectives. There are many methods of writing objectives, with the possibility thus arising that different methods might ultimately result in different findings. This researcher examined several techniques for writing objectives before selecting a technique modeled after the works of Davies, DeCecco, and Gerlach G Ely as described in Chapter II. These objectives were written as precisely as possible, so may be also compared to what Jenkins and Demo (also in Chapter 11) called Level D objectives. Not more than ten objectives were prepared for each article, so that each article had one page of objectives. These pages were inserted into the learning package just in front of the appropriate article, and presented to treatment groups two and four. The objectives, as used in this study, may be feund in 48 Appendix B. Construction of the Adjunct Ppggram Materials. The adjunct materials were constructed as a series of questions requiring various kinds of responses but reflecting the objectives as nearly as possible. Thus an objective which stated that the student should be able to "define film music" would be coupled with an adjunct practice question which asked the same question: Define film music. Sufficient space would be left for the student to write in his answer. This one-to-one correspondence was continued into the achievement test with an equivalent posttest question. Each objective, and therefore each adjunct question appeared restated in the posttest using as nearly as possible identical words. The adjunct questions for each article were combined into a single "program" which was presented to the appropriate treatment groups. Instructions on how to use the program were included. This adjunct program was inserted into the learning package following the third introductory page. The adjunct materials do not contain knowledge of results. This lack of direct student feedback would cause some writers to object to using the term "program" to describe the sets of questions. However, if one accepts the position of Espich and Williams (1967), feedback is not a prerequisite for adjunct programs. They feel this is especially the case if the answers are provided within the text material and are in reasonable proximity to the questions. More important was the issue of keeping the objectives and t-m 49 adjunct questions as nearly identical as possible. Feedback would have had the effect of separating the objectives and adjunct program, and further differentiating them from each other, since feedback would have then been supplied for the adjunct program but not for the objectives. The distinction is an interesting one, and perhaps provides a potential for future research. However such a direction was not the intention of this Study. The adjunct materials will be feund in Appendix C. Other Instructional Materials. To round off the learning package three introductory pages were necessary. The first of these was labeled "Instructions". This page listed in point farm the procedure the students would follow in working through the materials. All students read this page. No mention was made of different treatments. The second introductory page was the "Infermation Sheet" discussed earlier in this chapter. It served to take the place of a pretest, and to alert the researcher should any subjects prove to be knowledgeable in the field of film music. The third introductory page was entitled "Introduction: An Exercise in Formative Evaluation." On this page, subjects were told that they were participating in the development of a learning package. They were not told that different subjects had received different treatments, and were not aware of this difference during the duration of the experiment. tHowever, so as not to deliberately hide information from the students, a comment was embedded within this section which stated: "So that we may try out some different ideas of presentation, 50 others will be doing somewhat different activities than you." Each of these three introductory pages is found in Appendix A. Development and Validation of Criterion Instruments. As described earlier, the present study required two testing instruments which needed development and validation. The two needed instruments were (a) an achievement test, and (b) an attitude reactionaire. Both instruments were developed in the fellowing manner: 1. Generation of a pool of items. 2. Initial tryout on small group. (N82) 3. Revision on the basis of tryout data. 4. Large scale pilot test (N220) and final revisions. For both instruments, revision proved to be minor. In addition, another small scale pilot was conducted (N=3) to determine whether the cognitive posttest was complex enough so that naive subjects would score at or below chance level without seeing the treatment materials. That is, it was important that students not exposed to the materials would score poorly on the cognitive test. In each case scores of the naive subjects were sufficiently low. Scores were 3, 4, and 4 out of a possible 39. Therefere it was concluded that the treatment materials were necessary in order to achieve on the posttest. The final version of the achievement instrument consisted of 24 items for a total of 39 points. The final version of the attitude instrument consisted of 11 items. Both instruments are reproduced in Appendix E. 51 Reliability of Instruments. A measure of the reliability of the two tests was obtained using the Kuder-Richardson Formula #20. Computed reliability values were .8806 for the achievement test and .8086 for the attitude reactionaire. (These calculations were made not on the pilot groups, but on the experimental sample on which the study itself was conducted.) These values are more than satisfactory for a teacher made test, and allow one to accept the findings of the experiment with a reasonable degree of confidence. Although no formal validity coefficient was computed, the high reliability values indicate that some construct was being measured consistently. Also, examination of the content and analysis of the data suggest that these constructs are indeed knowledge of subject matter and attitude to the overall process. Thus content validity is assumed on the basis of the high reliability. Further, a test is assumed to have content validity when it requires the learner to perform the same behavior under the same conditions specified in a learning objective (Alexander, 1973). Since in this study each objective is precisely matched with a posttest question, content validity may be claimed for the achievement measure. Collection of Time Data Since it was considered possible that the factor of time might be able to account for any differences which occurred, an analysis of covariance was planned using study time and total time as covariates. Study time was defined as the amount of time the student used to read and study the treatment materials. Total time was defined 52 as the total time spent by the student, including testing time. To collect these data, students were simply asked to record the times at which they began and completed the posttest. The first time recorded represented study time, while the second indicated total time. Since students were allowed to proceed at their own rate, it was important to collect time data for each individual involved in the experiment. All rooms had wall clocks to facilitate the collection of . these data. Collection of Other Data In order to obtain attitudinal information with regard to specific treatments, an additional set of questions was developed and added as a second page of the attitude scale. Treatment One received no additional questions. Treatment Two received three additional questions asking for reactions with regard to performance objectives. These additional questions are feund in Appendix F. Treatment Three received three additional questions asking fer reactions with regard to the adjunct program. These questions are found in Appendix G. Treatment Four received seven additional questions asking for reactions with regard to both objectives and the adjunct program. Appendix H contains these questions. Data for these sets of questions were summarized and are reported in Chapter IV. 53 Subjects The sample chosen fer this research was the set of all graduate education students enrolled in a graduate course in educational media (831-A) conducted by Michigan State University during the fall of 1974. The sample consisted of three groups. One class was taught at Michigan State University, East Lansing, as an evening course. A second class was taught at Jackson, Michigan, while the third was held in Grand Rapids, Michigan. The latter two courses were both evening extension courses offered for credit by Michigan State University. All three courses were evening courses, and all ran for three hour periods, once a week, for ten weeks. Each course was taught by a different instructor. The study was conducted during week nine fer groups two and three, and during week six for group one. As stated in Chapter I, it was assumed that this time difference did not affect the final results. The total sample size available to the researcher without attrition was 104 subjects. Group one contained 55 subjects; group two contained 28; while group three had 21. Each of the four treatments was administered randomly to each of the three classes. Then the data were pooled and analyzed as if one large group had been available. Loss of Subjects. Of the potential 104 subjects, only 80 attended class on the day of the experiment. In group one, 41 out of 55 attended. It was assumed that this is a normal attrition rate fer a large night class, since these individuals had no advance knowledge 54 of the study. In the second group, only 14 out of a potential 28 attended on the day of the Study. The reason for the large 50% attrition was again in no way related to the study about which the subjects knew noth- ing in advance. The class had been scheduled to meet on November 27, 1974, the eve of Thanksgiving. While this day is not usually a holiday, Jackson community College (where the course was being conducted) made an administrative decision to close its doors early that day. As a result, the regular course instructor was obliged, at the last moment, to reschedule the class to a different day. Monday November 25 was finally selected, but too late to avoid confusion. Thus the attrition rate may be attributed in this case to the last minute readjustment in the time and location of the class. No subjects were missing from the final group held in Grand Rapids. It is assumed that all attrition was random. Missing Data. The resulting sample left exactly 20 subjects per cell for a total of 80. However, two individuals turned in unusable tests due to missing data. In one case, the individual involved approached the researcher to explain that she was new to the country and did not speak or read English well. Although she attempted to complete the materials, over half the attitudinal items were not completed, as well as the final page of her achievement test. In the second case, the individual involved was apparently not sufficiently motivated to participate in the study, as evidenced by pictures drawn on the 55 adjunctive program and final posttest. This individual also failed to fill out any attitudinal questions. Since the Finn program used for the study cannot accomodate missing data, these two subjects were dropped from further consideration. In order to maintain an equal balance of subjects per cell, two additional subjects were randomly discarded. The result was a matrix containing 19 subjects per cell fer a total of 76 subjects. Prgparation for Computer Analysis of Data Test materials were collected, hand scored, coded, and transfered onto computer cards. Data were then analyzed on the Michigan State University CDC 3600 computer using Finn program fer univariate and multivariate analysis of variance, covariance, and regression, version 4. Chapter Summary, A 2 x 2 factorial design was selected to study the effects of the presence or absence of perfermance objectives and adjunct practice questions on learning from extant materials. The two dependent variables were (a) scores on an achievement test, and (b) scores on an attitude reactionaire. Criterion instruments and instructional instruments were developed, field tested, and validated. Reading time (study time) data and total time data were collected to be used as covariates. For each extant article selected, a set of performance 56 objectives and an adjunctive program guide were developed. The study was administered to a sample of 76 subjects in a course in educational media, during the Fall of 1974. This allowed for feur treatment groups with 19 subjects per cell. Data were collected, coded, and computer analyzed on the MSU CDC 3600 using the Finn program for univariate and multivariate analysis of variance, covariance and regression, version 4. The next chapter will present the results of those data. CHAPTER IV ANALYSIS OF DATA Introduction In this chapter, the data obtained during the experiment will be presented and analyzed. The first section will restate the research hypotheses and report the findings for each. The second section will analyze and interpret these hypotheses. The third section will present other findings not subjected to computer and statistical analyses. Figures 2 and 3 present the cell means and marginal means which together summarize the raw data obtained for testing the six hypotheses. Figure 2 presents the cell means on the achievement measure, while Figure 3 presents the cell means for the attitude measure. Figures 4 and 5 use the cell means data to illustrate the nature of the two interactions studied. The results of each of the two way analyses of variance are summarized in Tables 2 and 3. The results of the regression analysis using covariates of study time and total time are summarized in Tables 4 and S. In addition, Appendix K presents supplementary statistical data not discussed directly within this chapter. These tables and figures include further data on study time and total time in terms of cell means (Appendix K, Figures 1 a 2); least square estimate of effects and appropriate standard errors from which an estimate of confidence intervals may be calculated (Appendix K, Tables 1 6 2); and the 57 58 results of the analyses of covariance when study time and total time are used as covariates (Appendix K, Tables 3-6). Findings Six hypotheses were presented in Chapter I. They are repeated here, first in the research form, then in null ferm, and finally in alternative statistical form. The hypotheses will be followed by a statement of the findings obtained using the univariate results obtained from the Finn program. All hypotheses stated below were tested at the .05 level of significance. In each hypothesis analyzed: p, = mean score of treatment groups which received 1 experimental conditions. Au - mean score of groups which received control 2 conditions. The first three hypotheses dealt with cognitive learning measured by an achievement test administered immediately after treatment. Hypothesis one was: H : Giving students statements of perfbrmance objectives 1 befbre extant instructional materials will result in higher achievement scores than when no objectives are given. (Objectives main effect) For the purposes of statistical analysis, the appropriate null hypothesis is: H : Giving students statements of perfermance objectives 0 befbre extant instructional materials will result in 59 similar achievement scores as when no objectives are given. (H : ‘u = u ) 0 l 2 This null hypothesis is tested against the alternative hypothesis: "‘P"}' l 2 a The purpose of this hypothesis was to provide support for proponents of perfbrmance objectives in the objectives argument. As summarized in Chapter 11, research reported in the literature is inconclusive and even contradictory on this question. An analysis of variance of the data in this study indicates a p value less than .0259. Thus the null hypothesis is rejected, inferring support for the objectives main effect hypothesis. The second hypothesis tested was: H : Giving Students an adjunctive program to be used with 2 extant materials will result in higher achievement scores than if no such program is given. (Adjunct main effect) The appropriate null hypothesis is: H0: Giving Students an adjunctive program to be used with extant materials will result in no different achievement scores than if no such program is given. This null hypothesis is tested against its alternative hypothesis: H: J1“(F2 a This hypothesis was meant partly to parallel the first and partly to 6O replicate other research which appears to support this hypothesis. The null hypothesis was rejected, based on a p value of less than .0006 obtained in this study. Thus support is inferred for the research hypothesis. This is consistent with research supporting the claim of an adjunct main effect. The third hypothesis tested was: H : There will be an interaction between objectives and 3 adjunct program on the achievement measure. The null version of this hypothesis is: H : There will be no interaction between objectives and 0 . adjunct program on the achievement measure. (H=p=p) 0 l 2 The alternative statistical hypothesis is: H‘Pi‘P l 2 a The purpose of this hypothesis was to determine whether or not any interaction between these strategies would occur. An interaction was indicated by the data. The null hypothesis was rejected on the basis of a calculated p value of less than .0394 obtained in this study, thus inferring support for the research hypothesis. The significance of this finding, coupled with hypotheses one and two, will be interpreted in the next section of this chapter. Table 2 summarizes the analysis of variance results for the first three hypotheses dealing with the cognitive domain. Hypotheses feur, five, and six examined the objectives “Rain effect, the adjunct main effect, and interaction effect but 61 Table 2 Two Way Analysis of Variance Table for Cognitive Test Scores Source of variation df Mean Square 5 p_ Adjunct 1 552.96 13.0684 .0006** Objectives 1 218.96 5.1748 .0259* Interaction 1 186.33 4.4036 .0394* Error 72 42.31 * Significant at .05 level ** Significant at .01 level in the affective domain, as measured by a researcher constructed attitude scale administered immediately after treatment. The feurth hypothesis tested was: H : Giving students statements of perfbrmance objectives 4 before extant materials will result in a more positive attitude score than if no objectives are given. (Objectives main effect) The null version is: Ho: Giving Students statements of perfbrmance objectives before extant materials will result in the same attitude score as when objectives are not given. (H=p=}x) 01 2 62 This is tested against the alternative statistical hypothesis: H=pr a l 2 As reported in Chapter II, only a few studies have examined the role of perfermance objectives from the point of view of the affective domain. These Studies appear to offer cautious support to the research hypothesis stated here. Its inclusion here is to explore the validity of that hypothesis further. The calculated p_value was less than .0062 in this study, allowing one to reject the null hypothesis and to infer support for the research hypothesis. Thus support appears to be given to the statement that the use of performance objectives results in a more positive attitude than no objectives. The fifth hypothesis tested was: H : Giving students an adjunct program to be used with 5 extant materials will result in a more positive attitude score than if no such program is given. The null version states: H0: Giving students an adjunctive program to be used with extant materials will result in similar attitude scores as when no such program is given. (H : ‘u =‘u ) 0 l The alternative research hypothesis is: “=11” 1 2 a The purpose of this hypothesis was to extend knowledge of the effects of adjunctive practice questions into the affective domain. Calculated p value was less than .1827, so the null hypothesis was not rejected. .I' 63 Table 3 Two Way Analysis of Variance Table for Affective Test Scores Source of variation df Mean Square E. p_ Adjunct l 88.47 1.8103 .1827 Objectives 1 389.26 7.9651 .0259* Interaction l 23.21 .4749 .4930 Error 72 48.87 * Significant at the .05 level No support can be given fer the research hypothesis in this instance. The sixth hypothesis tested was: H : There will be an interaction between objectives and 6 adjunct program in the affective domain. The null version states: H : There will be no interaction between objectives and 0 adjunct program in the affective domain. (Hoz‘p1 a‘u ) The alternative version is: H=p#p a l 2 No interaction was found. Calculated p value was less than .4930. Therefore it was not possible to reject the null hypothesis. Table 3 summarizes the analysis of variance results for hypotheses four, five, and six dealing with the affective domain. 64 The significance of these findings will be interpreted in the second section of this chapter. ‘Regpession Analysis and Analysis of Covariance Controllipg for Study Time; In addition to the basic hypotheses, additional data were collected on the amount of time the students used to study the learning package. (Study time was not controlled but left to the discretion’ of each student.) It was hypothesized that if differences appeared on the time dimension, that an alternative explanation of differences in achievement and attitude might not be the effect of the adjunct questions and the performance objectives p§£_§g, but rather that those students who spent more time did better. Appendix K, Figure 1 shows the results of the raw study time data in terms of cell means. These data suggest that the explanation that students who studied longer did better might well be a correct one. Therefore, an analysis of covariance was planned, controlling fer study time, to answer the question as to whether or not attitude and achievement results could be explained by the difference in study time. Two regression analyses were performed using study time as the independent variable. The first regression used achievement, ‘while the second used attitude as dependent variables. Table 4 jpresents the results of those analyses. Those results show that only .0011 of the total variance can be explained by Study time on the hachievement dimension, and only .0283 on the attitude measure. In each 65 Table 4 Test fer Significance of Relationships between Covariate (Study Time) and Dependent Variables Variables M1111. 3 Square Mult. 5 §(df=l,71) p_ Achievement .0339 .0011 .08 .78 Attitude .1683 .0283 2.07 .15 case p_is less than .78 and .15 respectively. These results show that study time cannot predict either dependent variable. Since study time cannot predict either dependent variable, that is, study time cannot explain the variance in attitude or achievement, the analysis of covariance was dropped from further consideration, and the original analysis of variance was considered the appropriate test. Appendix K, Tables 3 and 4 shows the analysis of covariance results with study time as a covariate. These results are substantially the same as the analysis of variance results, except that the interaction effect no longer appears significant on the achievement dimension (p is less than .07). However this difference is probably not due to the effect of the variable, but to error due to the loss in degrees of freedom. In summary, on the basis of the finding that study time was rust related to the dependent variables under study, it was expected tllat controlling fer this variable (study time) would not substantially 66 Table 5 Test fer Significance of Relationships between Covariate (Total Time) and Dependent Variables Variables Mult. R Square Mult. 5 £(df=l,7l) p_ Achievement .0077 .0001 .0043 .95 Attitude .1574 .0284 1.8092 .18 alter the analysis of variance findings. Table 4 confirms this expectation. Regression Analysis and Analysis of Covariance Controllipg fer Total Tips, A correlation of total time with study time revealed a value of .936 suggesting that the conclusions reached for study time would be substantially repeated if total time were used as a covariate. Table 5 repeats this essentially same infermation, except with the dependent variable now being total time. The analysis of covariance tables are Appendix K, Tables 5 and 6. The practical significance of this and the preceding section is that the analysis of covariance gives added support to the initial hypotheses that any effects are probably due to the nature of the vari- ables (adjunctive questions and performance objectives) themselves. Time does not explain these results. 67 Analysis and Inteppretation This section will examine and interpret the findings reported in the preceding section. For the purposes of this discussion, two sets of three hypotheses each will be examined, based on the two dependent variables of (a) achievement scores, and (b) attitude scores. The results will be discussed at the .05 level of significance. Interpretation of the Cognitive Hypotheses. Table 2 and Figure 4 summarize the results of this study with re5pect to the three cognitive hypotheses. Considered at the .05 level, all three hypotheses are significant. Figure 4 both confirms and illustrates the interaction which is present. The large difference between points A and B coupled with a smaller difference between points C and D Show the reason fer the objectives main effect. The definite upward slape of the two lines AC and particularly 80 suggest the presence of an adjunct main effect. And finally it is obvious from the visual that the lines BD and AC are not parallel, indicating an interaction effect. The Significant interaction means that any main effect may not be interpreted alone, but must be stated in conjunction with the other independent variable. Thus a logical interpretation would appear to be as fellows: Assuming that there are no adjunct questions present, objectives make a difference. If adjunct questions are already present, then the addition of objectives will have little or no additional effect. Conversely, if there are no objectives present, mm<-inmuwo mm<-anmuwo absent present absent present 68 A D J U N C T absent present 14.21 22.74 20.74 23.0 17.47 22.87 Figure 2 Cell Means: Achievement A D J U N C T ‘ absent present 35.53 32.26 29.89 28.84 32.71 30.55 Figure 3 Cell Means: Attitude 18.47 21.87 33.89 29.37 69 then adjunctive questions will make a difference. But if objectives are already present, then adjunctive questions will have little or no additional effect. This statement of the interaction is in fact a logical one in light of the initial definitions presented in earlier chapters. Since both objectives and questions were constructed so as to mirror each other as precisely as possible in a one-to-one correspondence, it seems reasonable to eXpect that each would have its major effect when the other is absent. Further, the use of both would be much like stating the same objective twice, or the same question twice, which again one might hypothesize, shouldn't be particularly likely to increase learning. In summary, it would appear that either objectives or adjunct questions are indicated, but not both. Inteppretation of the Affective Hypotheses. Table 3 and Figure 5 summarize the results of the study with respect to the three affective hypotheses. At the .05 level, no interaction is indicated. The adjunct main effect hypothesis is not significant, however there is a significant objectives main effect (p less than .0064). Figure 5 graphically illustrates these results. The nearly parallel lines AC and 80 show an absence of any interaction between the two main effects. The small or weak slope of both lines indicates the absence of an adjunct main effect. On the other hand, the "objectives jpresent" line AC and the "objectives absent" line BD are strongly .separated from each other, and therefore suggest an objectives main Score on Achievement Test Summed Attitude Scores LOW JAtt e High Att. 70 25"" C (23.0) ,0 D (22.7) A . 20" (20.7) ,x ,I' _______ objectives present __a. ," 15 B.’ .--.-.. objectives absent (14.2) 1; I g ‘1 Adjunct Adjunct Absent Present Figure 4 Interaction: Achievement 40— B ...‘-._- (35.5) "“----.-.-._ "“0 D (32.2) 30"" A37* g; (29.9) C (28.8) ._____. objectives present 20___‘ .---_., objectives absent 1 . . f l I Adjunct Adjunct Absent Present Figure 5 Interaction: Attitude 71 effect, which the data confirm. Figure 5 also indicates the direction of the attitude. Since the attitude scale was constructed so that a low score meant a positive attitude, while a high score meant a negative attitude, it is clear that those subjects who received objectives also had a more positive attitude than those who did not receive objectives. In addition, it may be noted that the attitude scale used a 5 point Lickert-type scale where l a most positive; 3 a neutral; and 5 a most negative. Thus an overall neutral score for the eleven items would be 11 x 3 a 33. Referring back to Figure 5, one can quickly see that attitudes associated with objectives were definitely on the positive side of 33, i.e., were positive attitudes. On the other hand, attitudes associated with adjunct questions were also positive, although not as strong. Taken together, these three hypotheses show that, within the limits of the present study, providing performance objectives to students prior to extant instructional materials results in more positive attitudes than when objectives are not provided. Adjunct questions seem to have little or no effect on attitude. No interaction is indicated, which means that whether or not adjunct questions are present, performance objectives will positively affect student attitude. It is interesting to speculate as to why the addition of objectives to the learning package influenced attitudes in this study. One suggestion might be that graduate Students in education...all practicing teachers...have already experienced enough in the use of 72 objectives, or are so well informed as to their potential value, that such familiarity has resulted in a positive attitude. A second possibility might be that objectives make students feel more secure. A third possibility might be that objectives make an achievement demand on the student resulting in the deve10pment of a positive attitude. And a feurth, potentially confbunding factor might be that by the time this study was run (weeks six and nine of a course in educational media) students had already been exposed to the theory and practice of writing objectives within that course. Different findings might have occurred had the study been run earlier in the course. Thus several possibilities may be speculated upon as to why objectives resulted in a positive attitude in this study. The question remains one for future research to confirm or resolve. Other Findingg In order to obtain an indication of specific student reaction to the objectives and adjunctive questions, several additional questions were constructed and appended onto the attitude reactionaire, appropriate to each treatment group. Thus treatment groups two and fuur, both receiving objectives, were asked to respond to questions 1, 2, and 3 below. Treatment groups three and feur, both receiving adjunctive programs were asked to respond to questions 4, 5, and 6 below. Treatment group four, receiving the total package, was asked one additional question, that being question 7 below. And treatment group one, the control group, received no extra questions, 73 since they received neither objectives nor adjunct program. These additional questions, along with their reSponses, are reported here: I. I feund the statements of performance objectives: Response Number of Respondents very helpful 17 somewhat helpful 16 uncertain . 3 not too helpful 1 useless 1 Total 38 In the final revision of this package the statements of objectives should be: Response Number of Respondents retained 38 excluded 0 Total 38 Which statement best describes how you used the objectives: Response Number of Regpondents ignored 2 skimmed 12 read in depth 6 used as review 8 other 10 Total 38 4. 74 I feund the adjunctive questions: Response Number of Re5pondents very helpful 22 somewhat helpful 11 uncertain 2 not too helpful 2 useless 0 no response 1 Total 38 In the final revision the adjunctive questions should be: Response Number of Regpondents retained 27 excluded 8 no response 3 Total 38 Which statement best describes how you used the adjunctive program: Resppnse Number of Respondents ignored 1 skimmed 9 read in depth 20 used as review 2 other 5 no response 1 Total 38 75 7. I found the combination of objectives and questions: Response Number of Respondents very helpful 9 somewhat helpful 4 uncertain 4 not too helpful 1 useless 0 no response 1 Total 19 A brief discussion of these findings is appropriate. In general, responses to these questions seem to confirm and support the findings from the hypotheses statements. Question 2 shows that all students who received objectives felt that objectives should be retained. This appears to support the reported finding of favorable attitudes for objectives. A curious inconsistency is apparent between questions 1 and 2, since one person found the objectives "useless" and a second person found them "not too helpful", yet they both felt that nevertheless objectives should be retained! Another inconsistency occurs between questions 4 and 5. Here eight students felt that adjunct questions should be excluded, although only four found them not helpful or were uncertain. It would therefOre appear that four individuals did find the adjunct questions helpful, but nevertheless thought that they should be excluded anyway. This may partially support the finding that adjunct questions did not result in a positive attitude. An interesting comparison may also be made between questions 76 3 and 6. It appears that there was a much more consistent use of adjunct questions than there was of objectives. That is, most individuals who received adjunct questions reported using them "in depth". On the other hand, the use of objectives was variable: twelve students simply skimmed the objectives; eight used them as review; only six used them "in depth"; while ten students indicated "other" uses. In summary, the comments obtained from students here seem basically consistent with overall results. CHAPTER V SUMMARY, CONCLUSIONS, RECOWENDATIONS Summapy The study described in this research was designed to analyze the separate and interactive effects of adding performance objectives and/or adjunctive practice questions on learning from extant instructional materials. The two dependent variables of concern were (a) cognitive learning, as measured by a researcher- constructed achievement test; and (b) attitude, as measured by a researcher—constructed attitude measure. Three sections of Educational Media 831-A offered by Michigan State University were selected as the target population for the research. Each subject was randomly assigned to one of four treatment groups. Treatment one received only the extant materials, followed by a criterion and attitude test. Treatment two received lists of performance objectives in addition to the extant materials. Treatment three received an adjunctive program in addition to the extant materials. Treatment four received both adjunctive questions and lists of performance objectives along with the extant materials. The extant materials consisted of three articles from various journals selected for their relevance to a unit on film music as a variable in instructional product development. This topic was selected because it was felt that it was one with which students would have little initial familiarity. Also, the topic of film music 77 78 could be justified as relevant to a course in educational media. Lack of familiarity with the content was important to this study so that a pretest could be omitted, since it was felt that a pretest might confound the effects of objectives and adjunctive program. Six hypotheses were generated and examined. These hypotheses tested (a) an objectives main effect; (b) an adjunct question main effect; and (c) an objectives-by-adjunct interaction effect. These hypotheses were examined in terms of both achievement and attitude. The experiment was set up as a fully balanced 2 x 2 factorial design. An analysis of covariance was run using study time and total time as covariates, since it was considered possible that the factor of time might be able to account for any differences which occurred. Since neither study time nor total time were found to predict either dependent variable, the analysis of covariance was dropped from further consideration, and analysis of variance was considered to be the proper test. Conclusions Analysis of the data supports the following conclusions, summarized at the .05 level: 1. Student cognitive performance on extant instructional materials is improved when those students are supplied with performance objectives, assuming no adjunct questions are present. 2. Student cognitive performance on extant instructional materials is improved when those students are supplied with an adjunctive program, assuming no objectives are present. 79 3. The addition of pp£p_adjunctive questions and performance objectives does not increase student cognitive learning beyond what is learned when either objectives or adjunct questions are available alone. 4. Students show a more positive attitude towards extant instructional materials when those materials are accompanied by performance objectives. 5. The addition of adjunct questions to extant instructional materials does not result in the development of a positive attitude towards those materials. 6. Study time and total time used as covariates were not able to explain the results on the achievement or attitude dimensions in this study. Discussion This section will be devoted to a discussion of three points. First, an interpretation of the results of this study will be examined in terms of utilization of either or both objectives and questions. Second, the use of study time and total time as covariates will be examined. Third, the results of this study will be reviewed in light of previous findings from other studies. It is clear that a strong case can be made for the inclusion of either objectives or adjunct questions within the instructional design of learning packages, according to the data presented in this study. If the instructional developer wishes to stress cognitive gain, adjunct questions or objectives appear to be indicated. If the 80 instructional developer wishes to develop a positive attitude in his subjects, the use of objectives is indicated. A case for the inclusion of 2252 objectives and adjunct questions is not quite as obvious, nor as definite, from the results of this study, as is the inclusion of each alone. However, one may speculate as to such a possibility on two counts. First, adjunct questions showed a main effect significant in the cognitive domain only. Performance objectives showed significant main effects in both cognitive and affective domains. Thus it may be, although not confirmed in this study, that each is making a different contribution to learning. Second, it must be remembered that for the purposes of this study, adjunct questions and objectives were constructed so as to be as nearly identical as possible. This may or may not reflect a "real world" situation. Follow-up studies need to be conducted to determine what happens when adjuncts and objectives substantially differ from each other in dimensions not considered by this study. For example, what happens when feedback is given for the adjunct questions? What would happen if different types of feedback were supplied? Answers to these questions await future research. Early in the study it was determined that if significant results appeared on the achievement and attitude dimensions, it would be necessary to determine whether the cause of these differences was not the independent variables, but that perhaps those who spent more time would do better. Therefore, two analyses of covariance were planned using reading time and total time as covariates. The findings of this part of the study are especially important, since the results 81 allow one to refute the suggestion that either study time or total time causes the results. In this study at least, the time a student spent had no bearing on how well he achieved or on what his attitude was. Thus further support is gained for the hypothesis that what did make a difference was the nature of the objectives and adjunctive materials themselves. Tables 3 to 6 in Appendix K Show the results of the analysis of covariance, controlling for study time and total time. These data are of interest since no studies have yet shown how objectives work or how the adjunct program works. At least in this Study, the answer cannot be attributed to the amount of time used by subjects. The results of this study are of interest when compared to other previous research. In general, research on adjunctive questions was supported. That is, previous studies have been reasonably unanimous in reporting that adjunct questions influence learning. The present Study supports such findings, and extends them, adding the qualification that adjunctive questions do not have an effect when in the presence of objectives. This study also suggested that adjunct questions have no particular influence on attitudes. This question has not been a focus of attention in previous studies which this researcher reviewed. Objectives research has in the past been somewhat ambivalent. This study supports those who claim that objectives do make a difference in cognitive learning. However, that statement must be qualified in light of the interaction with adjunctive questions. Studies dealing with the affective gains from performance 82 objectives are very few, although all seem to have reached a similar conclusion that objectives result in a positive attitude. This study confirms and supports those findings. Implications Implications for Future Research. 1. Future studies should concentrate on determining why differences exist between performance objectives and adjunctive questions. The present study focused on whether such differences existed, and on the nature of those differences. 2. Future studies might concentrate only on the adjunctive program aspect. For example, one study might aim to determine whether it is the active response component or the adjunct program pgp_§g_ which results in improved learning. One way to carry this out would be to construct adjunctive programs in which half of the questions call for active response, while the other half are merely statements which summarize and/or highlight significant objectives within the reading. An analysis somewhat akin to the split-halves procedure might then be conducted to determine whether there is a difference between the two approaches. 3. Future studies should be extended to using media other than print sources of extant materials. For example, it is assumed that the results of this study should be generalizable to other extant media...films, filmstrips, audiotapes, television, etc...but this may not be so. 4. This Study should be replicated for other subjects, with 83 other content, and at other age and grade levels. 5. Studies should be conducted which can more carefully integrate the content area being tested with the overall context of the course in which the study is conducted. In the present study, some Students indicated an inability to relate the tapic of film music as an instructional design variable to the specific course in educational media. 6. Future studies should stress different kinds of learning above and beyond the "knowledge" and "comprehension" levels of Bloom's taxonomy. 7. Future studies should be conducted stressing interactions between objectives and adjunct questions with other instructional design components such as advance organizers, post organizers, pretests and posttests. 8. Longer studies should be conducted which go beyond the one class duration time of this experiment. 9. Studies Should be conducted in which all the contents of the extant materials are not relevant to the objectives. In this study, the objectives were made to fit the materials. Ipplications for Instructional Developers. 1. This study supports the instructional development strategy of stating explicit performance objectives before each unit of a learning package or textbook. AS such, the Study re-emphasizes the need to establish objectives for students. 2. This study supports the strategy of providing accompanying 84 adjunctive questions to support text materials. 3. Developers should conduct a search for extant materials which approximate their objectives. 4. The findings of this study suggest an alternative to more expensive and time consuming development of new materials. The strategy of using objectives and/or adjunct programs implies that extant instructional materials can be redeveloped such that their cognitive and affective impact will be greater than if such extant materials are used alone. Cost-effective studies to compare the results of inexpensive "instructional redevelopment" to new deve10pment would be in order to determine the benefits and limitations of each approach. Implications for Classroom Teachers. 1. Teachers Should continue to search for extant materials to supplement their teaching. 2. Teachers should redevelop extant materials used in teaching so that they are accompanied by performance objectives and/or adjunctive programs. APPENDICES APPENDIX A Introductory Pages of Learning Package Instructions STOP! Do not proceed until instructor gives signal. Complete information sheet (page 2) Read "Introduction" page (page 3) BEGIN the learning package. RULES: a. You may refer back to any page or article as you wish. b. This is not a Speed test. There is no time limit. However a gross ballpark figure would be anywhere from 1-2 hours to complete entire learning package including tests. When you feel that you are ready to take the posttest: a. Raise your hand and have the instructor collect your learning package material. b. Open the manila enve10pe and begin the test. When you are finished, return the test to the manila envelope and hand it to the instructor. 85 86 Page 2. Information Sheet Booklet code number: Have you ever studied the tOpic of film music, either formally or informally? YES NO Do you in any way consider yourself knowledgeable in the area of film music YES NO Do you have any idea what research says about music in films? YES : Summarize: NO Name as many films as you can think of with music by Miklos Rozsa: Have you ever taken a course in the area of motion picture production? YES NO Have you ever taken a course in motion picture appreciation? YES NO STOP! WAIT FOR FURTHER ORAL INSTRUCTIONS BEFORE PROCEEDING. 87 Page 3. Introduction An Exercise in Formative Evaluation You are about to participate in the development of a "learning package". The stage at which you will contribute is the formative evaluation stage. All relevant materials have been selected, collected, tests prepared, etc. Now it remains to see whether or not the material teaches; whether or not it works. Your job will be to complete the learning package as directed. So that we may try out some different ideas of presentation, others will be doing somewhat different activities than you. When you finish, you will take a posttest in order to assess how well the package did its job. WE ARE NOT TESTING YOU, BUT THE PACKAGE. In addition to the test, you will be asked to complete an attitude reactionaire so that we may obtain an indication of your feelings towards the materials. The learning package will then be revised on the basis of your participation and comments. The topic to be presented is film music within entertainment, business and research settings. The unit is designed as part of a course in educational media production or instructional product development for teachers. Three articles have been selected to convey the basic content. APPENDIX B Performance Objectives List of Performance Objectives for "Movie Music Set the Mood" After reading the material in this episode you will be able to: l. 2. Name three film composers. Recognize from an array of choices why the term "film music" is preferable to "background music". Select from a list the era which most closely approximates the "heyday" of great film music. Identify from three alternatives the current "state of the art" of film music. Differentiate from four alternatives the kind of film music recordings you are most likely to find in record Stores today. Identify from an array what was responsible for the change in attitude towards the use and purpose of film music by the movie companies and movie producers. State the major source of movie music for silent films. Suggest two reasons for the high price of rare sound track albums. Name the LP recording responsible for renewed interest in film music. 88 89 List of Performance Objectives for "Scoring Music" After reading this episode you will be able to: 1. List the three main components of the sound track. 2. List the six ways music can be used in a slide tape presentation. 3. Recognize a definition of the term "dead air". 4. Recognize from a list of four choices the historical antecedent of film music. 5. State four reasons for using a music production library. 6. Order the four steps in preparing music for a presentation. 7. Given a list of alternatives, recognize the significance and potential relationship of this article to the teacher and to teaching. 90 List of Performance Objectives for "Effects of Familiar Background Music upon Film Learning" After reading the material in this episode you will be able to: 1. 2. State who conducted this study. Name the date indicating in what year the study was conducted. Identify from a list of four alternatives, Lindgren's beliefs on learning from films with and without music. Relate (from four possible choices) Lindgren's "theory" stated above to the use of film music in 2001: A Space Odyssey. Summarize in one sentence the basic finding of this study. List three limitations of the study. State the purpose of the Nuchols and Abramson study of 1949. State the finding of the Nuchols and Abramson study of 1949. APPENDIX C Adjunctive Questions Adjunctive Program for "Movie Music Set the Mood" The questions below are presented to guide your reading through the relevant points contained in this episode. As you read, watch for the correct answers and record them in the appropriate spaces. 1. Name three film composers identified in this reading. 2. The author implies that the term "background music" is less than sufficient. Why? a. It is not in the background. b. The music often reaches equal status with the visuals which it enhances. c. The music is most often in the foreground. d. Music helps make the movies bigger and better than life. 3. The "heyday" of film music is/was: 3. today c. 30's/40's/50'S b. 20's/30's/40's d. 40's/50's/60's 4. The current "state of the art" of film music is characterized in this article as: a. on the decline c. roughly the same as always, b. on the rise a neglected but significant art form 91 92 What kinds of film music recordings are you most likely to find in record stores today? a. jazz c. romantic-symphonic b. nostalgia d. pop What, according to the article, was responsible for the change in the attitude of the movie companies towards the use and purpose of film music a. new trends in music b. acceptance of pop sounds by the movie studios c. introduction of sound into the movies d. the LP record. The major source of movie music for silent films was . State two reasons for the high price of sound track rarities as collector's items: 1. 2. Today there is a resurgence of interest in the old film scores written when film music was considered a significant and viable art form. What is the name of the LP record which began this resurgence of interest? 93 Adjunctive Study Guide for "Scoring Music" The questions below are presented to guide your reading through the relevant points contained in this episode. As you read, watch for the correct answers and record them in the appropriate spaces. 1. List the three main components of the sound track. 2. List the six ways music can be used in an audio-visual production. 3. A blank portion within a sound track containing no sound whatsoever is technically known as . 4. Opening music to set the theme dates back to: a. early opera b. Greek drama c. silent movies d. the first talkies. 5. Four reasons for using a professional music library are: 6. Order the following, according to occurance from first to last. "clear" the music distribute the program license the program select the music 7. 94 To the classroom teacher about to prepare a slide tape audio-visual program for classroom use, this article has many direct implications. Mark "D" in front of those implications which are indeed directly related to this article: a. Don't ignore the music element of the sound track. b. Don't take music off commercial recordings. c. Familiar music is just as effective as unfamiliar music. d. An advantage of using familiar music is that it is more cost-effective than using original music for each production. e. All of the above are correct. f. None of the above are correct. 95 Adjunctive Prpgram for "Effect of Familiar Background Music upon Film Learning" The questions below are presented to guide your reading through the relevant points contained in this episode. As you read, watch for the correct answers and record them in the appropriate spaces. 1. Who conducted this study? 2. When was this study conducted? 3. Lindgren believes that: a. music is distracting when it is too good. b. well known music is distracting. c. neither d. both. 4. Stanley Kubrick in 2001: A Space Odyssey and A Clockwork Orangg_ used familiar musical compositions taken from the classics. According to Lindgren (see your response to the above question) this practice is: a. effective b. poor c. neither: it doesn't matter either way. d. neither: it depends on the film and how the music is used. 5. What was the basic finding of the study reported in this reading? 0‘ List three limitations of this study: 96 7. State the purpose of the Nuchols and Abramson 1949 study. 8. What were the findings of the Nuchols and Abramson study? APPENDIX D Sources of Extant Materials Used Shales, Tom. Movie music set the mood. The State Journal, Lansing, Michigan, July 22, 1973, p. F-l. Valentino, Thomas J. Jr. Scoring music: Into audio and audio-visual productions. Audio-visual Communications, 1973, 7(11), p. 12. Freeman, John, 8 Neidt, Charles. Effect of familiar background music upon film learning. Journal of Educational Research, 1959, 53(3) p. 91. 97 APPENDIX E Posttest Please record the time at the start of test: 1. Name three film composers noted in reading #1. The LP record which marked the resurgence of interest in classic film scores is a. Now Voyager c. The Seventh Voyage of Sinbad b. The Sea Hawk d. Vertigo What kinds of film music are you most likely to find in record stores today? a. jazz c. romantic-symphonic b. nostalgia d. pop The current "state of the art" of film music is characterized in reading #1 as a. on the decline b. on the rise c. roughly the same as always...a neglected but Significant art form. The "heyday" of film music is/was: a. today c. 30's/40's/50's b. 20's/30'S/40's d. 40's/50's/60's 98 99 6. State two reasons for the high prices of sound track rarities as collector's items. 1. 2. 7. The major source of movie music for the silent films was 8. What, according to article #1, was responsible for the change in attitude of movie companies towards the use and purpose of film music? a. new trends in music b. acceptance of p0p sounds by the movie studios c. introduction of sound into the movies d. the LP record 9. Article #1 implies that the term "background music" is less than sufficient. Why? a. It is not in the background. b. The music often reaches equal status with the visuals it enhances. c. The music is most often in the foreground. d. Music helps make the movies bigger and better than life. 10. List three main components of the sound track. 11. Opening music to set the theme dates back to a. early opera c. Silent movies b. Greek drama d. the first talkies 12. Reading #2 lists six ways music can be used in a slide tape or film presentation. Two are listed for you. Name the other four. 13. 14. 15. 16. 17. 18. 100 opening_j closingfi A blank portion within a sound track containing no sound whatsoever is technically known as Four reasons for using a professional music library are: To the classroom teacher about to prepare a mediated presentation for classroom use, article #2 has several direct implications. Mark "D" in front of those statements which are direct implications from the article. a. Don't ignore the music element of the sound track. b. Familiar music is just as effective as unfamiliar music. c. Don't record music directly from commercial records due to copyright. d. All of the above. e. None of the above. Order the following from first to last: "clear" the music license the program distribute the program select the music Who conducted the "Effects of Familiar Background Music..." study? When was it conducted? 101 19. State the basic finding of the study: 20. List three limitations of the study: 21. What was the purpose of the Nuchols and Abramson study? 22. What were the findings of the Nuchols and Abramson study? 23. Lindgren believes that: a. music which is too good can be distracting. b. well known music is distracting. c. both. d. neither. 24. Stanley Kubrick in 2001: A Space Odyssey used familiar musical compositions taken from the classics. According to Lindgren, this practice is a. effective. b. poor. c. neither:it doesn't matter either way. d. neither:it depends on the film and how the music is used. 10. 11. 102 Learninngackage Reactionaire Considered as a whole, the entire learning package was: very clear 1 2 3 4 5 very confusing Considered as a whole, the entire learning package was: very interesting 1 2 3 4 5 very boring What I learned was: very important 1 2 3 4 S a waste I think I learned: a great deal 1 2 3 4 5 nothing I was not sure exactly what I was supposed to be learning. strongly agree . l 2 3 4 5 strongly disagree The learning package was well organized. strongly agree 1 2 3 4 5 strongly disagree The package contained many unfamiliar terms which were not adequately explained. strongly agree 1 2 3 4 5 Strongly disagree Many of the things I was asked to do seemed like busy work. strongly agree 1 2 3 4 5 strongly disagree I found the topic of film music: extremely interesting 1 2 3 4 5 extremely boring I found the posttest to be: very relevant 1 2 3 4 5 very irrelevant I found the posttest very difficult. strongly agree 1 2 3 4 5 strongly disagree APPENDIX F Additional Attitude Questions fer Treatment Two 12. I found the statements of performance objectives: a. b. c. d. 8. very helpful. somewhat helpful. uncertain. not too helpful. useless. 13. In the final revision of this package, the statements of performance objectives should be: a. b. retained. excluded. 14. Which statement best describes how you used the performance objectives? a. b. I ignored them. I skimmed the article with the objectives in mind. I read the article carefully with the objectives in mind. I used the objectives as a review by trying to answer them without referring back to the articles. Other: Please describe: Please record time at the end of the test: 103 APPENDIX G Additional Attitude Questions for Treatment Three 12. I found the adjunctive program guide (i.e., the review questions): a. very helpful. b. somewhat helpful. c. uncertain. d. not too helpful. e. useless. 13. In the final revision of this package the adjunctive program guide should be: a. retained. b. excluded. 14. Which statement best describes how you used the adjunctive program? a. I ignored it. b. I skimmed the articles looking for the answers. c. I read the articles carefully looking for the answers. d. I used the questions as a review by trying to answer them without referring back to the articles. e. Other: Please describe: Please record the time at the end of test: 104 APPENDIX H Additional Attitude Questions for Treatment Four 12. I found the statements of performance objectives: a. very helpful. b. somewhat helpful. c. uncertain. d. not too helpful. e. useless. 13. In the final revision of this package the statements of objectives should be: a. retained. b. excluded. 14. I fOund the adjunctive program guide (i.e., the review questions): a. very helpful. b. somewhat helpful. c. uncertain. d. not too helpful. e. useless. 15. In the final revision of this package the adjunct questions should be: a. retained. b. excluded. 105 106 16. Which statement best describes how you used the objectives? a. I ignored them. b. I skimmed the articles carefully with the objectives in mind. c. I read the articles carefully with the objectives in mind. d. I used the objectives as a review. e. Other: Please identify: 17. Which statement best describes how you used the adjunct program? F a. I ignored them. b. I skimmed the articles looking for the answers. c. I read the articles carefully looking for the answers. d. I used the questions as a review. e. Other: Please identify: 18. I found the combination of objectives and questions to be: a. very useful. b. somewhat useful. c. uncertain. d. somewhat confusing. e. very confusing. Please record the time at the end of this test: 10. ll. 12. 13. 14. 15. 16. 17. 18. 19. 20. APPENDIX I Answer Key Green; Herrmann; Bernstein; Korngold. b d a C Few printed; Nostalgia; Considered an art form. the classics. d b music; effects; voice. b background; bridging; time marking; mood setting. dead air. quality; cost; protection; acceptability; convenience; variety. a 5 c 2;3;4;l. Freeman and Neidt 1959 No significant differences as to whether familiar or non-familiar music was used. 10 minute film; University students only; Only factual information 107 108 21. To test value of music vs no music in film learning. 22. No significant differences. 23. c 24. b Class Time 1 minute 3 minutes 3-5 minutes 2 minutes 2 minutes 1 minute 60-90 minutes APPENDIX J Detailed Procedural Instructions a. Activity Flow Student Activity Settling down Listen Listen Listen and read Complete "Information Sheet" on page 2. Complete learning package. Complete tests. Hand in tests. 109 Teacher Activity Introduction by regular teacher. Introduction by experimenter. Random distribution of materials. Read over page 1. Collect packages as hands are raised. Collect tests. 110 b. Oral Introduction by Researcher For the next hour, you are going to participate in an exercise in instructional development. Let me orient you to what you will be doing by referring to the Lasswell communication model as an outline: Who says what to whom in what medium with what effect. WHO: That's me. SAYS WHAT: There are two stresses here: content and methodology. a. Content: Components of effective mediated presentations is the general topic. This package will focus on one component only, music as a variable in instructional product design. b. Methodology: This is an exercise in "formative evaluation". As you know, in order to deve10p effective learning packages, it is necessary to try them out, then revise. Your job will be to go through the package and try to learn as much as possible, then write the posttest. Remember that you are not being evaluated, but the materials are. TO WHOM: You. Who are you? In general, you are a. teachers. b. potential instructional designers. c. users and developers of mediated presentations for classroom use. 111 IN WHAT MEDIUM: Print. Specifically three articles. The total package may eventually contain other media as well. The purpose here is only to teach some of the basic factual information which is best conveyed through print. WITH WHAT EFFECT: That is what the formative evaluation will tell us. As a first step, I will now hand out the learning packages to be field tested. Please do not read ahead. ( 2 minutes allowed here to distribute materials.) Everyone should now have a manila envelope in which is contained the posttest. Also, attached to the enve10pe is a learning package. The page facing you, page 1, labeled "Instructions" provides you with the general flow of activities. Let's read it over together. (Read "Instructions" page) Now turn to page 2 and complete the "Information Sheet". I'll give you just one minute. (Allow one minute pause.) You are now ready to turn to the next page and begin the learning package. 112 £:__£2212£. For the purposes of later identification and analysis, and to preclude the possibility of mixing information during the analysis stage, each learning package was separately coded in five places as follows: a. A number on each manila envelope of l to 54. (54 represents the size of the largest subgroup.) b. A color code on the front of each learning package. Color coding was done with felt pen and located next to the staple fastening the pages together. BROWN: control group (Treatment One) BLUE: objectives group (Treatment Two) GREEN: adjunct group (Treatment Three) BLACK: combination group (Treatment Four) c. Information Sheet: Coded as follows: (1) Initial of instructor: G for Gentry P for Price 8 for Saville (2) Number of package: 1 to 54 (3) Treatment number: 1 to 4 d. Adjunct program: Coded same as Information Sheet. e. Posttest: Coded same as Information Sheet. EXAMPLE: $153 meant that this student was in Saville's class (ie. Grand Rapids); he received the 15th package; and this was Treatment Three (ie. adjunct group). mm<-eomhwo mm