THESlB A . . . . nfimyrsa“... a; .r This is to certify that the thesis entitled The Development and Formative Evaluation of Multi-media Learning Packages in Supplementary Woodwind Techniques for Use in Teacher Training presented by David C. Sebald has been accepted towards fulfillment of the requirements for PhD . Music degree in %j@W Major professor Date 7/29/81 0-7 639 -h—r "' Illlllllllllllllllllllll||l|||llllllllllllllllllllllll “ 3 1293 10418 3292 MSU LIBRARIES “— RETURNING MATERIAL§z filace in book drop to remove this checkout from your record. FINES w1ll be charged if book 15 returned after the date stamped below. __‘ THE DEVELOPMENT AND FORMATIVE EVALUATION OF MULTI-MEDIA LEARNING PACKAGES IN SUPPLEMENTARY WOODWIND TECHNIQUES FOR USE IN TEACHER TRAINING BY David C. Sebald A DISSERTATION Submitted to Michigan State University in partial fulfillment of the requirements for the degree of DOCTOR OF PHILOSOPHY Department of Music 1981 {(4 )J/ \_)l[ ABSTRACT THE DEVELOPMENT AND FORMATIVE EVALUATION OF MULTI-MEDIA LEARNING PACKAGES IN~SUPPLEMENTARY WOODWIND TECHNIQUES FOR USE IN TEACHER TRAINING BY David C. Sebald The purpose of this study was to create and formatively evaluate three automated slide-cassette and workbook instructional packages covering several aspects of woodwind knowledge which do not readily lend themselves to in-class live presentation in courses designed for instrumental music teacher trainees. Formative evaluation was defined as assessment of strengths and deficiencies for the purpose of suggesting beneficial product revisions. The topics of the packages were: (a) oboe reed knowledge and adjustment techniques, (b) single reed knowledge and adjustment techniques, and (c) single reed mouthpiece knowledge. Prototype packages were based on research of an appropriate level of content and media theory and on the researcher‘s background in woodwind pedagogy and educational product development. These prototypes were submitted to two levels of evaluation. In the first level, the packages were verbally assessed by authorities in media product design, evaluation techniques, and each area of content. Revisions were completed based on the suggestions of these authorities. In the second level, each package was submitted to a group of eight to thirteen instrumental music methods students at Michigan State University for viewing and evaluation. Data were gathered from pre- and post-tests, student reactionnaires, and discussion sessions. Analysis of the data suggested the effectiveness of the materials in teaching the content and a positive student attitude toward learning supplementary woodwind information through these combined media. A list of fifteen revisions based on the student trial data analysis was suggested for the further refinement of the packages. To Cindy and Angus McAsgill ii ACKNOWLEDGMENTS The author wishes to thank Dr. Albert LeBlanc for his guidance and encouragement which enabled me to complete this project. Appreciation is also extended to Dr. James Nord, Professors Daniel Stolper, Fred Ormand, and James Forge: for their assistance in various stages of the research. Further thanks are extended to committee members Dr. Robert Erbes, Dr. James Niblock, Dr. Conrad Donakowski, and Professor Stanley DeRusha. iii Chapter I. II. III. IV. TABLE OF CONTENTS LIST OF TABLES . . . . . . . . . . . BACKG ROUND 0 O O O O O O O O O O O 0 Introduction . Purpose . . . Problem . . . Definitions . Assumptions . Limitations . Need for the Study .0 O O O O O O O O 0 RELATED LITERATURE . . . . . . . . Overview . . . . . . . . . . .- Development and Evaluation ProjeCts . . . . . . . . . . . . Formative Evaluation . . . . . . . Media Product Development . . Oboe Reed, Single Reed, and Single Reed Mouthpiece Knowledge . . . PROCEDURE 0 O O O O O O O O O O O 0 Introduction . . . . Research Phase . . . Development Phase . Evaluation Phase . . RESULTS OF EVALUATIONS . . . . . . . Introduction . . . . . . . . . . . Consultant Reviews . . . . . . . . Student Trials . . . . . . . . . . DISCUSSION, REVISION RECOMMENDATIONS AND CONCLUSIONS . . . . . . . . . Discussion . . . . . . . . . . Package Design Revisions . . . Technical Revisions . . . . . Conclusions . . . . . . . . . iv Page vi OOQO‘U‘IU'IH H h‘ la F‘ is to 1» w NM \lU‘ 28 29 29 30 33 36 39 39 40 46 71 71 73 78 81 APPENDICES Page A. The Abedor Model for Formative Evaluation of Multi-media Instructional Products . . . . . . . 86 B. Student Reactionnaire . . . . . . . . 87 C. Oboe Reed Pre- and Post-test . . . . . 90 D. Single Reed Pre- and Post-test . . . . 94 E. Mouthpiece Pre- and Post-test . . . . 97 mFERENCES . O O C O O O O O O O O O C O O O C 9 9 FOOTNOTES 0 O O O O O O O O O O O O O O O O O 1 0 7 Table 1. LIST OF TABLES Oboe Reed Package Pre-test and Post-test Raw Scores, Percent Correct Scores, and Criterion Acquisition Assessment . . . . . Single Reed Package Pre-test and Post-test Raw Scores, Percent Correct Scores, and Criterion Acquisition Assessment . . . . . Single Reed Mouthpiece Package Pre-test and Post-test Raw Scores, Percent Correct Scores, and Criterion Acquisition Assessment . . . . . . . . . . . . . . . . Post-test Item Difficulty Indices for Oboe Reed Package . . . . . . . . . . . . Post-test Item Difficulty Indices for Single Reed Package . . . . . . . . . . . Post-test Item Difficulty Indices for Mouthpiece Package . . . . . . . . . . . . Student Reactionnaire Likert Scale Mean Scores Reflecting Agreement with Preferred Response . . . . . . . . . . . . vi Page 50 51 52 54 55 56 61 CHAPTER I BACKGROUND Introduction In recent years, the educational community in the United States has begun to experience a tendency toward more practical, product-oriented research to complement the traditional scholarly studies which have typically had as their goal the investigation and reporting of some new facet of educational knowledge. This current inclination has been an indirect result of increasing societal pressures for evidence of concrete changes in education coupled with the widening recognition that much of the knowledge gained through traditional research has been considerably more than one step removed from its utilitarian application in every-day teaching and learning. Borg and Gall (1979) state, One of the most promising recent advances in education has been the emergence of significant programs of educational research and development.... Educational R and D effectively bridges the gap that has long existed between research and classroom practice. (pp. 34-35) To clarify their distinction between this recent style of research and traditional scholarly inquiry, they further state, In contrast [to traditional research] the objective of educational R and D is a finished product that can be used effectively in educational programs. The product is typically in the form of textbooks, audio-visual materials, training manuals, and possibly equipment of some sort. (pp. 34-35) Significantly, all of these products are actually examples of educational media, and among them the audio-visual media figure prominently. . As examples of research leading to these concrete products have accumulated throughout the last decade, the academic respectability of this form of endeavor has likewise grown. Psychological aspects of learning through media have been explored, compiled, and increasingly applied; unvalidated methods of production have given way to models of demonstrated effectiveness; techniques of product evaluation have become more standardized and based on thorough research. With these developments, the educational community has begun to recognize substantial merit in the new research and to see its potential for making significant contributions to the future of education. Those involved in music teacher training have also seen the benefits, even the necessity, of developing a body of effective media products to assist in future college and university curricula. Robert House (1976) gives an example: Shifting philosophies, experimentation, new hardware, and cost pressures will inevitably produce basic modifications in instructional approach.... ...Theory students will do most of their work alone, with tapes and coordinated visual materials which will explain the principles to be followed and learned; aural and keyboard assignments will then be practiced and self-recorded. (pp. 70-71) The many recent examples of music education research aimed at creating products such as House describes lend credence to his predictions. Although many of the early research media products attempted to explore the relative effectiveness of learning through a particular product versus traditional classroom instruction or another product, these general comparative studies often proved to be inconclusive. Many recent examples have eschewed this comparative approach, choosing instead to focus only on whether or not a product can be practically developed and demonstrated effective for a particular learning situation. Often these Studies have been based on the researcher's realization of a particular need in his own circumstances. Such has been the case with this researcher. Begin- ning in 1977, he has taught classes in combined woodwind techniques at Olivet College and Michigan State University. From the beginning of this experience, it has been obvious to him.that the time limitations of such classes do not usually allow for effective coverage of the tremendous diversity of information needed by students preparing to become instrumental music educators. Time spent demon- strating problems and solutions specific to one instrument is lost to those who have already adequately mastered that instrument; time spent covering general woodwind knowledge consumes time which is needed for individual attention and the development of psychomotor skills. In addition, it has been obvious that the physical settings of these classes do not permit effective presentation of the many :minute or subtle visual details needed for thorough understanding of some essential facets of woodwind knowledge. It has seemed evident throughout this time that one answer to overcoming these limitations might lie in the development and application of a series of short, single topic, multi-media packages dealing with the cognitive aspects of woodwind knowledge. These would be used by students as supplements to regular classroom instruction. Despite the potential magnitude of such an endeavor if every suitable topic were to be encapsulated in this manner, this researcher has felt that a beginning would prove useful if it provided even a few well-designed products. Such a beginning would also expedite future development of a larger body of such materials. It was to this purpose that the present study was directed. Purpose Specifically, the purpose of this study was to create and formatively evaluate three multidmedia instructional packages covering several aspects of woodwind knowledge which do not readily lend themselves to effective in—class live presentation for instrumental music teacher trainees. Subject matter for these packages was selected on the basis of: (a) the researcher's own experience of need, (b) informal surveys with others involved in similar situations, and (c) the researcher's desire to explore the feasibility of packaging different topics in units of essentially similar formats. Thus, rather than developing several packages dealing with the same subject--tuning' . for instance--on each of the major woodwinds, the researcher planned to develop products dealing with the following topics: (a) oboe reed knowledge and adjustment techniques, (b) single reed knowledge and adjustment techniques, and (c) single reed mouthpiece knowledge. Problem The problem presented by this study can best be summarized in the following question: Can multi-media packages in supplementary woodwind techniques for use in teacher training he created and refined by an individual developer through accepted methods of instructional development and formative evaluation? An analysis of this question suggested its division into the fullowing sub- problems: 1. Can valid content be effectively incorporated into a standard format in these instructional packages? 2. Can known principles of audio—visual instruction be effectively incorporated into the design of these . pacakgeS? 3. Can products of acceptable technical quality and a suitable delivery system be created by an individual developer? 4. Can effective procedures and instruments for the formative evaluation of these products be created or adapted from existing models? 5. Can conclusions be drawn from the evaluation process which will generate ideas for the beneficial revision of these materials? ‘Definitions Formative EValuatiOn An early authoratative definition of formative evaluation was proposed by Scriven (1967) as "outcome evaluation of an intermediate stage during development of a teaching instrument...to discover the deficiencies and successes in intermediate versions of the new curricula" (p. 51). For the purpose of this research, the term was construed as a process of gathering and analyzing data concerning deficiencies and successes in prototype audio- visual materials and subordinate materials including tests, delivery system, and written matter: then devising appropriate revisions of these materials before their application to a large body of students. The terms "develOpmental testing" and ”developmental evaluation" are used as synonyms for formative evaluation in this report. Multibmedia‘Package In this study, "multi-media package" refers to an educational kit containing subject-related stimuli in the form of thirty-five millimeter slides and cassette tapes plus dependent materials in the form of tests, unit guides, and directions. Deliveryjsystem As used in this research, "delivery system" refers to the combination of machines which transmitted aural information stored on cassette tapes while simultaneously displaying visual images without the aid of an operator. Specifically, the combination consisted of: (a) two Kodak Carousel slide projectors, (b) an Audiotronics cassette recorder having seperately operable channels for audio track and projector synchronizing signals, and (c) a Clearlight dissolve unit designed for programming various synchronizing signals onto the cassette and translating these signals on playback into slide projector functions. Validate In this report, the term "validate" and its grammatical variations are used in an open sense to refer to the justification of content, materials, and procedures by their appropriateness to the stated purpose of the project--to create, through a process of development and formative evaluation, effective multi-media packages in supplementary woodwind techniques for use in teacher training. ‘Assumptions The purpose underlying most educational research is to provide knowledge of a product which can be applied usefully to a population larger than that directly involved in the study. Although the scope of this project does not permit validation of its products' effectiveness outside Michigan State University, it nevertheless seems logical to assume two conditions which would make this study valuable to a wider field: 1. It is assumed that the class woodwind and instrumental methods situations with which the researcher is familiar and which have led to the development of these products are reasonably comparable to those at other higher education institutions preparing instrumental music teachers for the public schools. 2. It is assumed that the population of instrumental music teacher trainees at Michigan State University which provided subjects for the formative evaluation of these products is reasonably similar to that at other higher education institutions preparing instrumental music teachers for the public schools. Limitations This study-was limited to the development and formative evaluation of a single group of educational products; no attempt was made to compare their instructive potential to other methods of education. Although formative evaluation can be an almost continual process of trial and revision, this study was limited to two cycles of evaluation. Revisions were completed after the first cycle, and suggestions for further revisions were given after the second. The entire formative evaluation process is this study was conducted at Michigan State University and involved instrumental music teacher trainees. Therefore, although potential users of the products are free to 10 draw inferences concerning their applicability in other situations or at other institutions, the conclusions and revisions generated by this research can be specific only to current instrumental methods and woodwind methods courses at Michigan State. . .Materials created by this project were designed to supplement but not replace regular classroom instruction in woodwind techniques. Consequently, students involved in the developmental testing were limited to those who possessed the woodwind skills and knowledge stated by the researcher as prerequisite for the understanding of the concepts in each package. Need for the Study Discussing self-instructional multi-media learning systems, Allen Abedor (1972b) articulated a growing concern in the educational community by stating that many colleges and universities were beginning to recognize the need to improve efficiency and effectiveness of instruction. Throughout the seventies, the growing number of research projects, articles, and symposia dedicated to studying the potential of media technology for increasing educational success reiterated this concern either through direct declaration or implication. 11 That many products of these endeavors were created without sufficient attention to their technical or educational quality was expressed by Alkin and Fink (1974), The problem of establishing standards for product development is not new....However, with the passage of time and the increasing number of available products, the problem becomes more obvious. (p. 103) The need in education has not been for instructional products per s2 but for products of demonstrated quality and effectiveness. In music education, Rainbow (1973) expressed his desire to see research leading to more effective preparation of instrumental music teachers. This writer believes, however, that instrumental music and music education might benefit more if the emphasis of future research were placed on the investigation of some of the basic problems related to instrumental performance and improved training of teachers of instrumental music. (p. 16) Others in the field have directed Rainbow's general statement of need specifically toward research in developing products for increasing effectiveness in this area of education. Dvorak (1973), in his thesis on the use of filmstrips in instrument repair instruction suggested "that further research be completed utilizing technology as an approach to the learning of additional musical skills" (p. 70). Studies of instrumental methods training have shown 12 widespread deficiencies in this particular area of music teacher preparation. In a survey of 198 public school ‘music teachers, Coleman (1979) found that teaching a beginning vocalist or instrumentalist received below average ratings in the "effectiveness of collegiate preparation" category. Duncan (1979), surveying woodwind methods classes throughout the United States, found that time provided for these courses was often insufficient and recommended that further research “explore the uses of computers and A-V in woodwind techniques instruction" (7208A). From these expressions of concern and calls for further study, it can be clearly seen that a current need exists for technological product development in general higher education and more specifically in music teacher training and training in woodwind techniques. Research directed toward developing multi-media packages in supplementary woodwind techniques will contribute to filling the need for useful products in this area and to providing further knowledge in effective media design for-music education. CHAPTER II RELATED LITERATURE 222m This chapter is limited to pertinent writings within four areas: (a) product development and evaluation projects in music education, (b) formative evaluation, (c) media product development, and (d) oboe reed, single reed, and single reed mouthpiece knowledge. Development and Evaluation Projects A researcher investigating previous product oriented studies for guidance in methodology soon finds that most examples can be classified into one of two categories: non-experimental projects in which the main effort has been to demonstrate a product's effectiveness without comparison to traditional teaching or other products and formal experimental studies in which an effort has been made to compare the new product's effectiveness to that of other methods of presenting the same content. Numerous examples of both can be found.. In this section, attention is given primarily to studies of the past decade which have concentrated on the development and evaluation of visual and/or aural l3 14 instructional products for use in music education. Stress is placed on those studies which, because of procedural or content smmilarities, have a close bearing on the present project. No coverage is given to computer or broadcast television research in music education or to those investigations in which the use of an audio-visual product has been ancillary to another purpose. Following the review, a summary of research results is given and conclusions are drawn concerning the relative appropriate- ness of the two methodologies to this project. Non-comparison Studies Miller (1972) developed video-taped simulation tests of woodwind embouchure problems using non-comparison procedures similar to those used in the present research: prototype consultation with authorities in content, simulation, and evaluation; several revisions of the prototypes: and field evaluations with students. On the basis of data from his content validation procedures and student evaluations, he concluded that the revised products were valuable in helping students solve embouchure problems. Similarly, Diamond and Collins (1967) developed loop films to teach the identification of clarinet embouchure, fingering, and hand position errors. Showing the films 15 to twenty-nine students, these researchers used a filmed pre-test and post-test and an attitudinal survey to evaluate the effectiveness of their products. Significant results at the .01 level from a correlated t-test of pre-test and post-test scores, together with data from the attitude survey, led them to conclude that their products effectively accomplished their purpose, especially with students experienced on clarinet. Using a somewhat different evaluation procedure, Dvorak (1973) created sound filmstrips to aid instrument repair instruction. A panel of expert judges suggested revisions and rated the repairs made by twenty-three students exposed to the final products. Composite scores given by these judges were compared to students' scores on a Bennett Mechanical Comprehension Test. Results indicated no significant relationship between the two sets of scores, but since eightyneight percent of the students completed satisfactory repairs, Dvorak concluded that his materials were effective. Although Dvorak selected his subjects on the basis of their stated inexperience in repairing instruments and included a test of mechanical comprehension, he did not use a pre-test to clearly demonstrate prior inability to make specific repairs. The inclusion of such a measure might have offered firmer support to his conclusion. 16 Williams (1978) designed video-tapes to teach oboe embouchure formation and reed adjustment after his public school survey indicated that these were two of the most common areas of problems on the instrument at that level. Based on evaluations by public school teachers and their students, he concluded that his materials were effective in teaching those skills. McClintick (1976) produced video-taped packages to assist pre-service teachers in learning to teach expressive properties to primary school children. Using six teacher trainees to evaluate verbally the products both before and after revision, she concluded that the packages did fulfill their purpose. Both the Williams and McClintick products were judged effective solely on the basis of participant evaluation. Although such assessment certainly has value in revealing particular areas of strength and weakness in a product, one could wonder if the inclusion of pre-test and post-test would make their overall conclusions more substantial. Some developers have eschewed any attempt to formally evaluate their products, concentrating instead only on the creation of materials. Lindelien (1979) developed color video-cassettes for teaching care and maintenance of nine primary wind instruments using published materials, 17 interviews, and personal experience for determining content validity. However, he reported no attempt to evaluate the effectiveness of his products. Likewise, Beck (1979) concentrated only on development, making sixteen millimeter films and video-tapes showing recreations of Renaissance madrigal performances for use in college level fine arts classes. Christopherson (1975) similarly created video-tapes of authentic African music performances. Interestingly, he claimed validity for his products on the basis that the performers themselves approved them. In addition to the studies which have developed visual-aural products for music education, many similar non-comparison examples of aural-only product development exist. Following a procedure identical in many respects to the present study, Ellis (1973) made prototype audio—tapes, tested them with individual students, and irevised them. He then used a pre-test/treatment/post- test design and an attitude measure to evaluate them with sixty-three students. He found that the tapes were effective in helping students of at least high-average to above average cognitive ability achieve an eighty/ eighty criterion level on the post-test (eighty percent of students achieving at least eighty percent correct responses). In addition, ninety-two percent of all 18 students involved felt that the packages were helpful and enjoyable. Husak (1978) developed similar tapes to help in jazz ensemble arranging. His procedure involved content validation by authoratative consultants, pilot testing and revision, and pre-test/treatment/post-test evaluation with fifteen college level students. A correlated t-test showed better than a .001 level of significance in performance gains from using the package. An opinionnaire indicated further that students felt the packages were effective in self-study of jazz arranging. Shaw (1971) used audio-tapes in packages on the elements of snare drum technique, validating their effectiveness by significance testing the pre-test to post-test differences and by interpreting compiled data from an attitude survey. In addition, he correlated measures of personal, academic, and musical characteris- tics of his subjects finding that musical aptitude correlated most strongly. Krueger (1974) used essentially identical procedures to validate cassette/workbook packages on musical timbres and to correlate elementary grade level to achievement. 19 FOrmal Comparison Studies Hansuld (1971) developed eight millimeter sound films to teach selected wind instrument skills at the college level. The films included bassoon reed adjustment, clarinet alternate fingerings, and trumpet embouchure formation. They were developed in two parallel formats. demonstration only and demonstration-participation. Randomly selected student groups using each format and students having only traditional lecture-demonstration were subjected to a pre-test/treatment/post-test experiment with resulting data submitted to analysis of covariance in which the pre-test was the covariate.‘ ANCOVA was employed to statistically equate students classified by sex, grade level, and previous wind instrument experience. From the analysis, Hansuld concluded that no significant difference existed between filmed instruction and traditional instruction, but students did score higher when using the participation format than when using the demonstration only format. Boley (1970) similarly produced sound films to teach playing fundamentals to beginning students. His comparison of regular instruction to film supplemented instruction showed no overall benefit from using the film. 20 Rees (1976) and Burgess (1974) designed video-taped lessons to supplement string bass and violin instruction respectively. Experimental to control group comparisons were used to evaluate them. Burgess found no significant difference in achievement, but Rees found that traditional instruction was more effective than his products when they were substituted for it. Rees implied a lack of precise controls and inadequate video-tape technology as possible reasons for the relatively poor effectiveness of his products. Hill (1972) produced filmstrips and Robbins (1979) produced video-tapes to supplement more cognitively oriented courses in music--music appreciation and theory respectively. Neither found that the addition of these materials to regular instruction caused higher test scores than regular instruction alone. Several researchers have developed audio-visual materials to improve teacher trainees' rehearsal skills. Gonzo and Forsythe (1976) created video-tapes to teach rehearsal techniques and principles. They found that the use of these products significantly increased students' correct responses on observation tests compared to students who had only lecture based instruction. Stuart (1979) developed a variety of materials (video- tapes, slides, and texts) to increase error detection 21 skills on string instruments. Through her experiment, she found that the use of these materials positively affected conducting students' recognition of specific errors compared to instruction in which the products were' notrused. Many researchers in music education have produced audio-only products as an integral part of programmed self-instructional packages and have compared the use of these to traditional instruction. Drushler (1972) found no significant difference in teaching beginning wind fingerings and notation. Anderson (1979) also found no difference when using aural models to guide home practice for sixth grade instrumentalists. Miller (1973) found no difference when using aural models to teach rhythm at the sixth grade level. However, Lawrence (1973) found that taped models helped seventh and eighth grade students learn music reading. In similar comparison studies at the college level, Wyatt (1974) found that auto-tutorial tapes helped choral conducting students learn phonation, resonance, and breathing. Damron (1973) found improvement in jazz improvization skills through the use of self-instructional packages which included cassettes. Fritz (1979) found no improvement, however, when he used tapes to supplement 22 instruction in basic listening skills for prospective elementary classroom teachers. Sherry (1975) found that taped self-instruction in selected musical concepts was an effective substitute for regular in—class instruction at the college level. Sidnell (1971) found that self-instructional audio-tape packages aided conducting students in developing aural error detection skills. SumMary and Conclusions The task of the educational product developer is not merely to create materials but to demonstrate that they can effectively aid learning. The foregoing examples of media product development research in music education show basically two approaches to the second task: (a) some form of non-comparison evaluation and (b) a formal treatment/ control group comparison of the product to another form of teaching. Unfortunately, an investigation of these studies shows that neither approach would be universally accepted as ideal. A researcher who adheres closely to strict, scientific methods of evaluation would surely find the effectiveness claims of many of the non-comparison studies unconvincing. Claims based primarily on evaluations by those for whom the products were intended, represented by 23 the McClintick and Williams studies might be considered tenuous at best because the judges by definition were not experts in the field. The post-test only design, shown in the Dvorak study, could be criticized for an inadequate assessment of entry level skills directly related to the content of the product. This too would make claims of effectiveness questionable. Even the seemingly sound significance testing of pre-test to post-teSt_gains--which played an important part in the Diamond and Collins, Husak, Shaw, and Krueger evaluations-- has been denounced by some measurement authorities as a deceptively inflated measure. Most questionable of all, of course, would be developments in which the researchers reported no attempt at all to demonstrate effectiveness. On the other hand, the more formal treatment to control group comparison studies seem contradictory or inconclusive in establishing the relative effectiveness of the product versus traditional teaching methods. Studies by Hansuld, Burgess, Drushler, and Sherry found no significant difference between the use of their products and traditional instruction. Gonzo and Forsythe, Stuart, Lawrence, wyatt, Damron, and Sidnell found that the use of their respective products as supplements aided instruction; however, Boley, Hill, Robbins, Fritz, and Rees found either no benefit from the use of their 24 products or that traditional instruction alone was more effective. One could easily question the appropriateness of comparison research in product development and, indeed, whether or not it is worth the effort. Leonhard and Colwell (1976) called attention to the earlier suggestions of Walter Ihrke, a pioneer in applying technology to music education, "that to compare automated training with that offered by the traditional approach may hold little profit because the human approach has not been very effective" (9. 17). Although the more formal design of comparison research lends an air of scientific credebility to its results, this too could be questioned. Michael Apple (1977) states, It should be clear, however, that in general educators have appropriated the reconstructed logic of science rather than the logic-in—use of scientific investigation. Their view of scientific activity as the expert and efficient means to . guarantee certainty of results has been fundamentally inaccurate. It represents a picture drawn from technological models of thought, whereas accounts of significant scientific investigation show a more sophisticated posture, in which the complex blend of technique, art, and personal commitment is highly evident. This appropriation of an inaccurate model produces considerable difficulty. It leads educators to practice poor research, and, most importantly, it is a major component of their tendency to confirm the conceptual paradigm under which they are working even though substantive 25 progress may require a new disciplinary matrix in place of the current one. The numerous findings of "no significant difference" might just point to this conclusion. (p. 478) To the author of this study, the answer to the question of the appropriateness of comparison designs in product development research is clear. If the objective is an effective product to aid instruction, the developer need not ask, "Does it work better than or as well as something else?" but simply, "Does it work adequately in its intended setting?" and, "What refinements will make it work better?" Used collectively, the techniques of non-comparison research should provide substantial data to answer these two queStions. ‘ Formative EValuation While authoratative writings on formative evaluation in general education are copious, those specifically related to music education are sparse and have only a minor bearing on this project. Of major interest to the author's research is Abedor‘s (1972a) dissertation which describes the develop- ment and validation of a model for the formative evaluation of multi-media instructional packages. That the description of this model is highly detailed and that it was validated using media identical to those proposed by the author (slides and cassettes) make this thesis a 26 prime source of the formative evaluation procedures used in this endeavor. Borich's (1974) EValuating Educational Prggrams and Products provides a concise source of information on a number of other, more generalized developmental testing techniques. The procedures described, although not as closely related to the researcher's project as that found in the Abedor thesis, still provide a background in differing practices of formative evaluation. Schwen's (1972) Four Views 9:;F0rmative Evaluation and Sanders and WOrthen‘s (1972) Descriptive Summary pf Frameworks for Planning‘Evaluation Studies serve similar purposes. As the recognized source of major philosophies and definitions underlying formative evaluation methodology, Scriven‘s.(l967) Methodology gf Evaluation is often quoted. Other informative writings on the subject include Bloom's (1971) HandbOok'gn Formative and Summative Evaluation in Student Learning, Lawson's (1974) Formative Instructional Product EValuation, and Sanders and Cunninghamis (1974) Techniques and Procedures for Formative Evaluation. 27 Magia Product Development Three areas of educational media knowledge are important to the developer of instructional media products: (a) educational media theory, (b) designs for efficient product development procedures, and (c) specific techniques of production. Numerous studies have been conducted to assess the characteristics of media in education and how to use these most effeCtively. Two summaries of these studies important to the design of the researcher's products are Levie and Dickie's (1972) The Analysis and Application 9f Media and Kirschner's (1973) Doctoral Research in Educational Mgdia, Of particular interest to the production design of this study has been Gerlach and Ely's (1980) Teaching and Media: a Systematic'Approach which presents a model for efficient product development in educational settings. Information on specific slide-cassette package production techniques is most readily obtained from commercial sources. Planning and Producing Slide Programs, published by Eastman Kodak Co. (1978), provides information on a number of related techniques. More specialized information useful to this project was obtained from the same company's (1978) Basic Art Techniques for Slide Production and‘Legibility--Artwork tg_Screen. 28 Oboe Reed, Single Reed, and ”Single Reed Mouthpiece Knowledge Information in these areas is plentifully available, but this study calls for content appropriate to the level of teacher trainees. Thus, scientific studies on reed and mouthpiece acoustics, information on historical develop- ment, and similarly advanced knowledge were not considered relevant. Of particular importance to the oboe reed package were Hedrick's (1972) ObOe Reed Making and Sprenkle and Ledet's (1961) A35 9_f_ 31395 Playing. For information on single reeds, Opperman's (1956) Handbook fOr Making and Adjusting Single Reeds and Larry Teal's (1963) Ar; 2; Saxophone Playing were valuable. Information on single reed mouthpieces is contained in Teal's A£g_g£ saxophone Playing and in Stubbin's (1965) éEEflE: Clarinetistry. Educational publications from commercial sources like G. LeBlanc Co. are concise and valuable. Such a publication is Arlie Richardson's (1962) The Clarinet MouthpieCe. All of these sources are comprehensible to the non-woodwind major and are readily available at the present time. CHAPTER III PROCEDURE Introduction Many strategies have been proposed within the last fifteen years for the development and/or evaluation of educational media products. These include plans by Sanders (1972); Stake (1967); Bash (1970); Paulson (1969); Cunningham (1972); Walker (1974); Barson (1965); Hamreus (1969); Tyler and Klien (1967); wright and Hess (1974); Bertram and Childers (1974); Katz and Morgan (1974); Gerlach and Ely (1980); and Luft, Lujan, and Bemis (1974). Although there was obviously no dearth of models for the researcher to follow in this project, the problem was that they were not specifically applicable to this project. As Abedor (1972) stated concerning many models of instructional development, " They are either too general for direct application to complex multi-media treatments or, if specific, provide techniques applicable to simple stimulus configurations" (p. 11). Therefore, in planning the procedure for this study, it seemed wiser for the reSearcher to select and adapt ideas from several sources rather than limit procedures fo a single model. This endeavor followed basically a three phase plan 29 30 to which several of the above models contributed in whole or in part. The steps were: (a) research of content and media design principles, (b) development of prototype packages, and (c) evaluation and revision of the packageS. Research‘Phase The first phase of the project entailed three sub- steps: acquisition of consultants and student evaluators for the later evaluation phase, research of content information, and research of media design principles. Discounting some sporadic content and media design inquiry in which the researcher had been engaged for the previous two years, this phase took place during late December of 1980 and throughout January of 1981. The first sub—step began with the researcher approach- ing expert consultants, one each in educational media and evaluation and three—~in oboe reeds, single reeds, and mouthpieces-~for content. Each potential consultant was briefed on the nature of the study, the particular role he would be expected to play, and the probable time requirements involved. Those who agreed to help were provided with details and a set schedule of when their services were to be needed. The media consultant, James Nord, was a professor in 31 the education department at Michigan State University and was highly respected by the researcher for his work in the fields of educational product design and educational media. The evaluation consultant was Albert LeBlanc, a professor of music education at Michigan State University with substantial background in music education research and measurement techniques. All three content consultants were professors of applied music at Michigan State University: an oboist, Daniel Stolper, for the oboe reed package; a clarinetist, Fred Ormand, for the single reed package; and a saxophonist, James Forger, for the single reed mouthpiece package. The researcher also contacted instrumental music methods students at Michigan State University to arrange for a pool of volunteers to assist in evaluating the packages after technical and content revisions had been completed. The nature of the project and the benefits they-might derive from it were explained, and encouragement given by allowing volunteers to substitute participation in package evaluations for a required methods class project. Following recommendations set forth in the Abedor dissertation, the researcher set a target number of subjects at between six and ten per package, but the actual number of students who volunteered was ' between eight and thirteen per package. 32 Based on the volunteers' schedules, the researcher made arrangements for running the package trials in the first week of March, 1981. In addition to the minimal considerations of locating a room and setting suitable times, a doctoral student in music education was acquired as a test analysis assistant so that post-test problems could be noted during the debriefing sessions at the end of each trial. Also, a tentative interview schedule was created for the debriefing and refreshments planned for the volunteers. Concurrently with the search for consultants and student subjects, the researcher entered the second sub-step, collecting and consolidating information pertinent to the topic of each package from available resources including books listed in the content section. of Chapter II, in field surveys of persons engaged in teaching woodwind courses and in public school instrumental music, and his own professional experience. . Inferences concerning content validity were made on the basis of agreement among sources and the researcher's perception of authority. Since in some areas the information from one source conflicted with that of other sources, the expertise of the content consultants or, at last resort, that of the researcher himself served as arbitrator. 33 When it was concluded that sufficient content material had been gathered, the information was reduced to what the researcher considered an appropriate level of detail and complexity considering the audience for which it was intended. This distilled information was then outlined for each package. With his own previous experience in media development techniques and several graduate courses in educational product design serving as a basis of information, the researcher began to accumulate and digest results of media research to develop a list of potentially useful product design principles. Since the purpose of the study was not to search out and document all original studies in this area, information from compilations and secondary sources was accepted as well as single studies particularly pertinent to the project. All of this information was then organized into single statement principles and distilled to those which promised to be most useful to the conditions of this endeavor. From these, a general package format was designed. ‘DevelOEment Phase The second major stage entailed three sub-steps: development of a list of objectives and preliminary evaluation instruments for each instructional unit, creation of aural and visual elements and instructional 34 sequence for each unit, and organization of all these elements into complete packages. This phase was carried out through February, 1981. The researcher began by producing a list of behavioral objectives based on the outlined content previously made for each package and on the projected characteristics of its intended audience. From these objectives, equivalent test prototypes were developed to serve as pre and post measures of objective attainment. These instruments in their final form would be designed to be quickly analyzed so that collective data could be fed back to the researcher during the debriefing session immediately follOwing administration of the post-test for each trial. In addition to the instruments for measuring knowledge attainment, a Likert scale attitudinal measure was adapted from the Abedor dissertation for use with all three packages. (See Appendix B for Abedor's student reaction- naire.) Following completion of the first sub-step, the researcher began development of all audio-visual and accompanying materials. The first task involved writing a rough draft narration based on the content outlines and from this creating notes or sketches for complementary visuals. These were intended not only to illustrate the 35 first draft narration but to influence its revision. Thus, if an idea for a visual seemed to demonstrate a concept differently but better than the previously written narration, the script was revised. Together the script and sketches served as a storyboard to direct production of the audio—visual materials. At this time, the researcher began to collect all necessary physical samples for later photography including examples of numerous reed brands and reed problems and mouthpieces. Necessary aural demonstrations which were later to be dubbed into the narration on the master tape and visuals which could not be photographed in the studio were also created at this time. Following these tasks, the final draft of the narration was taped and the compiled aural examples dubbed in to create the master tape for each package. When the previously taken photographs were returned from processing, all further photography was completed including composites, studio shots, and copy stand visuals. These were then sent for processing. The last procedure of this sub-step entailed collecting or creating all materials which would accompany the packages including written directions, unit guides, and evaluation instruments. The final sub-step in the production phase was the organization and packaging of all created materials. This 36 process involved mainly sequencing all slides in appropriate trays, duplicating the master audio track onto cassettes, and programming the cassettes with slide synchronizing signals. These tasks completed the development of the prototype packages. EValuation Phase :3. The third phase of the project adhered closely to a model of developmental testing created and validated by Allan Abedor in 1971. In its broadest conception, the *model consists of two major steps: technical review by expert consultants and student trial. (See Appendix A for the Abedor model.) In the first step, the researcher displayed the complete prototype packages to appropriate consultants with a matching written script for noting particular areas in need of revision. Audio-visual materials were shown as they would be in the student tryouts, and each consul- tant was asked to make oral and written comments concerning his own area. Although directions concerning the focus of information desired from each consultant had been given to him when he volunteered to assist, the face-to-face interaction between the researcher and the consultants was kept open both to suggestions concerning the consultants' areas and to serendipitous information. Two such consultations were carried out with the 37 media authority and one each with the evaluation, oboe reed, and single reed authorities. Scheduling pressures did not permit the mouthpiece consultant to view his package prior to its scheduled student trial; however, it was subsequently shown to him and his comments incorporated into plans to revise future packages. Following these reviews, the researcher organized all comments into priority lists. weighing the necessity for change with the feasibility of alternate methods of improvement, he revised the packages accordingly. Student trials began with the researcher reminding subjects involved of the role they were expected to fill in the project and giving procedural directions. The pre-test was then applied, the presentation shown, and the post-test administered. These tryouts were held during the first week of March, 1981. When all students had completed the post-test, they were given a short break to relax, have refreshments, and begin formulating comments on the merits or deficiencies of the instructional unit. The debriefing session was started with several broad questions from the researcher's interview schedule, but as students warmed to the discussion in each case, the format was opened to include comments on other, more technical aspects of the package as well as instructional problems they had encountered. 38 The only guidance at this time was an attempt to keep the discussion casual, limited to the package, and progressing from point to point. As soon as the evaluation assistant had completed a cursory item analysis of the post-test, questions concerning particular concepts in the package were entered into the discussion. During the first trial, for the oboe reed package, this analysis was completed by the evaluation assistant while the researcher began the discussion session. However, this method proved to be too time consuming and subsequent post-tests were revised so that results could be self-tabulated by students and immediately analyzed by the researcher during the discussion. The entire debriefing session was audio- taped for future detailed analysis. CHAPTER IV RESULTS OF EVALUATIONS Introduction The purpose of this research was to create and formatively evaluate three multiemedia instructional packages covering supplementary woodwind techniques for use in teacher training. The topics of the packages were: (a) oboe reed knowledge and adjustment techniques, (bl single reed knowledge and adjustment techniques, and (c) single reed mouthpiece knowledge. There were two major levels in the evaluation phase of the project. In the first, prototypes of the packages were submitted to several expert consultants, one in educational media, one in testing procedures, and one in each area of content. At the end of these evaluations, the prototypes were revised on the basis of the consul- tant's recommendations and economic feasibility. In the second, the revised packages were submitted to trial and critique by teacher trainees at Michigan State University. For each package, this trial consisted of a pre-test and a post-test, a written student reactionnaire, and an oral discussion. 39 40 The purpose of all evaluative procedures was to gather data from.which inferences could be drawn for revising the packages. No attempt was made to compare them experimentally to other methods of instruction or to summatively validate them. Contained in this chapter are the evaluation results and subsequent revisions at the first level of evaluation and the evaluation results at the second level. consultant Reviews Media Assessment Because of the media authority's schedule commitments and because all three packages were to be alike in format and media characteristics, the researcher concluded that it would not be necessary to show all three packages to him. Instead, an initial prototype of the oboe reed package was submitted for his evaluation, and the single reed package, revised on the basis of these comments, was later shown for further comment. His critique and suggestions for improvement were delivered both orally and in written notes at the end of each showing and later organized into a priority outline for revising the materials. The consultant's overall reaction to the first presentation was favorable; however, he pointed out a 41 number of individual areas of weakness, particularly in the format. Organization of these comments into categories suggested that four basic changes would enhance the materials' overall instructional potential. These were: (a) division of the presentation into smaller, more discrete parts which would be easier for the learner to process mentally, (b) inclusion of better motivational and relaxational devices to alleviate the tension which might accompany viewing a continuous half-hour instructional unit, (c) more repetition of major concepts, and (d) incorporation of better mnemonic devices. To implement the first recommendation, the researcher planned modifications to more clearly separate all I presentations into five-to-ten minute segments, each dealing with an individual objective of the package. Because the initial prototypes already dealt with these objectives in sequence, the only changes planned at first were the incorporation of title slides at the beginning of each section and a narration which plainly specified which objective would be covered next. However, consideration of the second and third areas of weakness suggested that these separations might be made even more beneficial by allowing the viewer to actually stop the presentation at these points and complete a self-correctable review quiz on the concepts of the 42 segment just shown. This approach, which would not only divide each presentation but would also provide more repetition of important points and relieve the tedium of continuous viewing, was incorporated into all three packages at appropriate points. Although the researcher assumed that these breaks would help the viewer relax, he decided to integrate music and cartoons at appropriate points in the format to further lighten the presentations. These were to be placed at the beginning of each section and at the beginning and end of each presentation. In practice, however, time limitations permitted their incorporation in only two of the three packages, oboe reeds and single reed mouthpieces. In addition to completing the two major format changes detailed above, the researcher attempted to increase concept repetition by analyzing the number of repetitions of each major point and, where needed, inserting more. The goal was to make each major idea at least doubly redundant while at the same time maintaining enough variety of presentation to assure the viewer's interest. The media consultant had also noted several aural and visual inadequacies in the first prototype. Categorization of theSe comments led the researcher to review each package for: (a) improper match of words used 43 in the narration with words projected on the screen or specifically implied by the simultaneously shown picture, (b) visuals or sound track excerpts not thoroughly identified by the narration, (c) insufficient number of visuals for complete understanding of ideas, and (d) technically inadequate visuals. Instances of these problems were noted in each prototype and, where feasible, corrected. Following completion of all corrections, the new prototype of the single reed package was shown to the media consultant who gave thorough approval to the revisions. COntent Assessment Each of the three revised packages was next submitted to review by a consultant whom the researcher considered an authority on the subject matter of the prototype shown to him. Although the researcher's own background and previous investigation assured some degree of content Validity, differences of opinion inherent in these areas of woodwind knowledge in addition to the possibility of -misinformation or oversight assured the usefulness of these critiques. As with the media assessment, comments were delivered in written notes and oral discussion follow- ing each showing and were later organized into priority lists for package revisions. 44 An analysis of the specific comments given by the oboe reed consultant led to changes in three areas: (a) modification_or improved presentation of several testing and adjusting techniques, (b) incorporation of several new testing and adjustment techniques, and (c) clarification of certain technical terms for reed parts. In addition to his content critique, the oboe reed consultant also expressed doubt about the use of several colloquial words in the narration. Although these had been incorporated purposefully during the previous revision to keep the presentation informal, further consideration by the researcher led to their removal. A few minor suggestions by the consultant were not included in the second revision because, in the researcher's judgement, their importance did not justify the expenditure of time and money needed to correct them in the next level prototype. Comments by the single reed consultant led to revisions in two areas: (a) modification of several testing and adjusting techniques and (b) improvement of aural and visual concept demonstration. This consultant also expressed concern over the use of informal language which the researcher later deleted. Time limitations did not allow the single reed 45 mouthpiece consultant to view this package prior to its scheduled student trial. However, those potential problem areas which had been discerned in the other presentations were checked as carefully as possible by the researcher and revised before this trial. These revisions included removal of colloquialisms and improvement of several visual demonstrations. Following its student tryout, this package was submitted to the content authority. His comments were broad and favorable with only one specific criticism concerning the lack of aural demonstrations for several tonal concepts. A few new pieces of product information which emerged from his discussion would also be included in future versions of the package. Evaluation Assessment After content revisions had been completed for the first two packages and before the next level of evaluation was to begin, the researcher discussed the general testing strategies to be used during the student trials and specific testing techniques with the evaluation consultant. He approved the researcher's methods and suggested no major changes in the data collection instruments or the evaluation design at this level 46 Student Trials Description At the second level of evaluation, each package was shown to a small group of music education teacher trainees at Michigan State University, all within the first week of March, 1981. The groups consisted of eleven, eight, and thirteen students for the oboe reed, single reed, and.mouthpiece packages respectively. Each student saw two of the three packages although one of those involved in evaluating the mouthpiece package arrived after the pre-test had been given and most of the presentation had been shown. As a result, data from this student was not included in the evaluation of this package. Information was collected from three major sources. Identical pre- and post-tests were analyzed for indications of knowledge acquisition and possible areas of instructional weakness in the presentations. Student reactionnaires, identical for all three packages, were analyzed for agree- ment with various statements incorporated into a Likert scale and for written comments concerning pre-determined aspects of each package. Oral comments from taped debriefings following the post-tests and reactionnaires were organized and studied for clarification of previous data and for further, unanticipated information. 47 Pre-test and Post-test Evaluation The pre- and post-test for each package consisted of multiple choice and matching questions. Fifty-three items were included in the first pre/post-test, for the oboe reed package, but thereafter the number of items was reduced to allow more time for oral discussion at the end of each trial. From these tests, the following data were derived to give the researcher feedback for further package revisions: (a) raw and percentage gain scores, (b) criterion acquisition, (c) item difficulty, and (d) test reliability. The researcher considered gain scores important to the package evaluation as an indication not only that the packages successfully imparted information to the students involved but also as a means of identifying which packages were most successful and whether they were successful for their intended users, teacher trainees. In all cases, students registered gains between pre- and post-tests. From a low percentage gain of 9.5 percent to a high of 57.7 percent, the mean gain throughout all three trials was 33.2 percent. Some gain may be attributable to the sensitization involved in the proximity of the pre-test to the presentation and post-test, but since the pre-test had been intended to be used as an integral part of each package, this was interpreted as a 48 valid gain.l A comparison of mean percentage gains among the three packages represented in Tables 1, 2, and 3 indicated a progressive increase from first to third, 26.61 percent for the oboe reed package, 32.59 for the single reed package, and 40.41 for the mouthpiece package. Several factors may explain this. First, whereas none of the students exposed to the first package had had previous contact with a similar unit in this series, three of the eight students who viewed the second package had seen the first, and all who saw the last package had seen one of the others. Thus, progres- sively more students may have had increased receptivity to concepts presented in this format. A second possibility is that the information presented in each package was progressively less familiar to the students, and therefore they had more to learn from each. Judging from the pre-test scores, this could be a valid explanation for the high gain in the third package tests, as the pre-test scores here were lower than those of the previous packages. However, the pre-test scores for the first two packages, 59.17 percent for oboe reeds and 60.61 percent for single reeds, were so similar that such reasoning could not be justified for the increase between them. 49 A third reasoning could be offered saying that the instructional effectiveness of the packages improved from first to last. To the researcher, this seems to be the most logical explanation as he did make modifications in the second and third packages based on test results and comments of the previous presentations. When each group of gain scores was broken down into means for majors and non-majors on instruments related to each presentation's topic, it was noted that for every package nonemajors registered the highest mean gain scores. This may indicate that the level of information contained in the packages was correctly targeted toward teacher trainees rather than applied music majors. Interestingly, pre- and post-test scores for majors using the mouthpiece package were lower than for non- majors, the opposite of what would be expected. This was explained when it was found that the lowest scores, for student number twelve, were made by a major who later. indicated that he had misunderstood the directions on both tests. Prior to testing the students, the researcher had adopted a common criterion level for each package of eighty percent of students achieving an eighty percent correct response rate on the post-test. This standard was to be used as a broad indicator of whether each .mm mo ouoom ucoosumrumo oaoemmoo Houou o co comma mouoom umoulumoo one room "meoz 50 NN.m «m.h mh.HH NN.¢ mH.v NN.w u Om wMN.Nh Hm.mN mb.mm ha.mm mo.vH m¢.m¢ om.Hm n m 02 m.ON m.mh h.vm AH ov aN Ha mm» m.NN m.vm m.Nm MH mv mm OH no» m.m m.Nm o.mm m av vv a mo» m.mN N.oa m.ho ma Hm on m mm» m.mm o.om h.¢m 5H mo aN 5 Oz N.cm ¢.hh N.h¢ ma Hv mN 0 Oz h.NN m.mn m.om NH mm hN m mow H.Nm m.mm b.¢m ma we mN v mow m.mN m.Nm N.vm ma me «m m m0» m.mm H.Hm m.Hv HN mo NN N mow h.NN m.Nm m.mm NH mv mm H cofiumuauu cwmw unmouom ucmouom came ouoom 3mm oncom 3mm ucmosum ummulumom mom ucoouom unculumom umouloum 3mm unworumoo ammunoum ucoammommd coauwmwsood cofluoufino one .mmsoom uoounou ucoonom .monoom 3mm umoulumom poo smoulmnm mmmxomm comm mono H manna .hm mo whoom ucoosumluoo oaowmmoo Hobo» o co combo monoom amoulumoo new room ”meoz mm.m mo.m He.ss oa.~ H~.n o~.s 51 u om wom.sm mm.~m m~.ma Ho.om m~.~H .om.¢m m~.- ,u m we» o.m~ _ G.em G.mm Ha . mm , «N. m mm» «.mm o.oos c.5m NH em mm 5 mm» s.qm m.sm ~.~m ma mm mm m we» s.sm c.4m «.ms ma mm or m was s.m~ s.Hm «.mm so am mm s mm» «.mm G.¢m «.mw «H mm mm m oz m.a~ o.ms «.ms as am as a new m.v~ o.oos s.ms m an aw H cowuouwuu neon unmoumm accouom cwmw oncom 3mm oncom 3mm acronym umourumom wow usoouom ummulumom Humorous 3mm unculumom umoolonm .ucmsmmommr coHuwmflsoom coonoufino oco .monoom uoosnoo ocoonom .mouoom 3mm amourumom one unculoum oomxomo room mamcflm N GHQMB .mN mo choom scoosumiumo oHonmoo Hmuou o co comma mouoom umoorumom can lose "meoz NN.NH, .mH.NH HN.mH oN.N VHs.m s¢.N 52 u no. NN.Nm Hs.ov oo.mm NH.ms mm.oH mN.NN mN.HH u m oz N.oN o.om H.NN N mH o NH mos N.oN N.om o.om m HN NH HH mos N.NN N.oN m.Nm OH mN mH oH mo» N.Nm N.Nm o.sm NH «N a N mo» o.om N.om N.om NH HN N a mo» o.om N.om N.os NH mN NH N oz N.oN N.oN o.om 5 ON NH o mos m.mm .m.NN o.om oH NN NH m mos H.o¢ o.sm m.mm NH NN OH 4 mos «.mH m.NN H.NN a NN NH N no» N.Nm N.om m.mN mH mN oH N mos N.o« N.oa o.om NH mN NH H cOeuouHHU cwmu usoouom ucoouom came ouoom 3mm ouoom 3mm osmosum umoorumom wow ucoouom umoulumom oncoloum 3mm unculomom umouroum .ucosmmommd coHuHmHsoom coHHoueHU ocm .moHoom uooHHoo ucoosom .mmsoom 3mm unculumom one Humorous mooxomm mooworusoz comm onch m canoe 53 package could be judged successful at this level of development. Both the single reed and mouthpiece packages passed this criterion, single reeds attaining 87.5 percent passing students and mouthpieces attaining 83.3 percent. As is indicated in table three however, the oboe reed package failed to pass this criterion, attaining only 72.73 percent. Two factors were thought to be most likely causes of this relatively poor showing. First, the same student Who had misunderstood the directions on the mouthpiece tests apparently misread the directions on the oboe reed tests; this placed his score below the criterion level. The second possible factor is that several serious flaws in information pacing of the oboe reed package were corrected in the subsequent presentations. To reveal specific areas of weakness in the packages, the researcher completed an item difficulty analysis for (each post-test using a standard formula d = grin which R equals the number of correct responses and N equals the total number of responses on a given item. The results are shown in Tables 4, 5, and 6. From these tables, the researcher isolated all items yielding indices below .80 and examined both the item itself and its referent concept in the presentation for possible weaknesses. It was assumed that inadequacies 54 Table 4 Post-test Item Difficulty Indices for Oboe Reed Package Item Index of Difficulty 2 .82 3 .91 4 .82 6 .91 9 .82 10 .91 15 .82 16 .82 17 .73 19 .73 20 .73 21 .73 22 .36 . 23 .73 25 .82 26 .82 27 .82 28 .64 3O .91 31 .91 32 .91 34 .64 37 .91 38 .73 39 .82 42 .46 47 .91 49 .73 52 .82 53 .91 NOTE: Items not included had zero variance; all students responded correctly. 55 Table 5 Post-test Item Difficulty Indices for Single Reed Package Item Index of Difficulty 1 .88 3 .88 4 .88 7 .88 8 .75 10 .88 12 .88 13 .88 16 .88 21 .63 22 .75 24 .63 25 ' .88 26 .88 NOTE: Items not included had zero variance; all students responded correctly. 56 Table 6 Post-test Item Difficulty Indices for Mouthpiece Package Item Index of Difficulty 2 .83 4 .83 5 .92 7 .92 9 .92 10 .92 11 .83 12 .83 13 .92 15 .83 17 .58 18 .92 19 .83 20 .67 21 .83 22 .33 24 .58 25 .83 NOTE: Items not included had zero variance; all students responded correctly. 57 indicated by this analysis could be attributed either to poor aural-visual presentation or to a poorly constructed test item. In the oboe reed package, misidentification of reed parts caused the highest number of scores below .80. The fact that items 17, 19, 20, 22, and 28 all required the ability to name various parts of a reed indicated that the presentation of these visuals or the items on the post-test were inadequate. Since later discussion of the package did not reveal complaints of unclear or dim slides, the problem could be most easily attributed to: (a) too fast a pace in presenting the information, (b) inadequately specific arrows to point out small'parts, or (c) poor match between the projected images and the drawings used on the identification section of the post-test. In a related problem, a low score on item 34 indicated an inability to identify proper reed tip openings. This was most easily explained by a lack of visual redundancy in tip opening comparisons during the presentation; however, a poor match between the projected images and the post-test drawings could also account for the problem. Low scores on items 8, 42, and 38 revealed confusing test questions rather than inadequate audio-visual presentation. In nearly all cases, students who erred on these items chose the same distractors. Subsequent 58 diagnosis of these distractors showed that their content or wording actually did make them reasonable substitutes for the preferred answer. Analysis of item 49, concerning reed hardness and its corrections, showed neither a noticeably bad test question nor a confusing visual presentation. The most likely alternative hypothesis therefore was that the narrative coverage of the concept in the presentation was either unclear or insufficient. An analysis of the single reed post-test errors revealed that three of the four items with difficulty indices lower than .80 (21, 22, and 24) could be attributed to the same problem: inadequate oral clari- fication of reed resistance and its causes. The fourth item (8) showed a similar problem: inadequate explanation of the causes of a buzzy tone. On the mouthpiece post-test, three items under .80 (20, 22, and 24) seemed attributable to either unclear narrative explanation or insufficient repetition since the concepts involved were not visual and there were no obvious faults in the test questions. These questions all concerned mouthpiece parts and their functions. It is notable that far fewer items under .80 were to be found in the second two tests than in the first. Although this may in some part be attributed to the fact 59 that these tests were shorter than the first, they, like the presentation with which they were associated, had been revised on the basis of the results of the first trial. It is also notable that the reliability coefficients, computed through SPSS subprogram RELIABILITY were higher for the second two post-tests than for the first. Alpha for the oboe reed test was .76,mfor the single reed test .85, and for the mouthpiece test .84. ReactiOnnaire Evaluation Following the post-test during each trial, students were asked to complete a reactionnaire in which they assessed various aspects of the package they had just seen. This form, identical for each package, consisted primarily of a Likert scale of twenty-seven statements with which students were asked to rate their agreement on a scale of 1 = strongly agree to 5 = strongly disagree. Four open questions were also included, asking students to express what they felt were the most difficult and the easiest parts of the package, what were the three worst aspects of the package, and what improvements they would suggest. Ten of the statements on the Likert scale were positive e.g., "The examples used to illustrate the main points were excellent," while the rest were negative e.g., "I think this whole procedure of trying out new materials with students is a waste of time." Ratings for the 60 negative statements were reversed during analysis to reflect agreement with the preferred reaction; thus a 1 would always be favorable, a 5 unfavorable. Table 7 shows that the means of student scores for statements in each reactionnaire indicated a favorable response to all aspects of every package. However, in order to isolate those responses which were least favorable, the researcher averaged the ratings within the reactionnaires for each package and compared individual statement ratings to these means. The mean ratings were: oboe reeds 1.64, single reeds 1.66, and mouthpieces 1.67. Any statement rating above (worse than) these means was studied for possible revision implications. Five responses were less favorable than average in all three packages. These indicated that students felt: (a) they did not have sufficient prerequisite knowledge (2.68), (b) they needed to go back over part of the lesson (2.50), (c) the narration moved too fast (2.48), (d) a professional speaker should be used (2.11), and (e) extensive modifications should be made (2.03). Less favorable than average ratings for the oboe reed package showed that students felt: (a) equipment operation or breakdown was distracting (2.10),2 (b) the presentation did not make them feel better toward the subject (2.00), (c) the unit guide was used too frequently 61 Table 7 Student Reactionnaire Likert Scale Mean Scores Reflecting Agreement with Preferred Response Oboe Reed Single Reed Mouthpiece Statement Package Package Package Means 1 3.00 2.13 2.92 -2.68 2 1.82 1.50 2.08 1.80 3 1.36 1.62 1.23 1.40 4 2.10 1.40 1.38 1.63 5 1.55 1.75 1.54 1.61 6 1.09 1.36 1.54 1.33 7 2.10 2.53 1.69 2.11 8 2.82 2.25 2.38 2.48 9 1.37 1.37 1.38 1.37 10 1.55 1.37 1.46 1.46 11 1.45 1.75 1.85 1.68 12 1.82 1.37 1.54 1.57 13 1.55 1.37 1.21 1.38 14 1.23 1.37 1.28 1.29 15 1.36 1.75 1.54 1.55 16 1.10 2.00 1.77_ 1.62 17 1.09 1.25 1.46 1.26 18 1.36 1.63 1.38 1.45 19 1.28 1.62 1.34 1.41 20 1.64 1.50 1.85 1.66 21 1.18 1.25 1.31 1.24 22 2.00 1.87 2.23 2.03 26 1.36 1.29 1.82 1.49 27 1.19 1.14 1.27 1.20 28 1.28 1.43 2.00 1.57 29 2.64 2.86 2.00 2.50 304 2.00 2.00 1.54 1.85 i = 1.64 1.66 1.67 1.65 - SD = .52 .42 .40 .39 62 (1.82), and (d) they were unsure of what they should be learning (1.82). For the single reed package, lower than average ratings indicated that students felt: (a) some vocabulary was unfamiliar (2.00), (b) the presentation did not make them feel better about the subject (2.00), (c) examples used to illustrate main points were less than excellent (1.75), (d) the unit guide was less than excellent (1.75), and (e) the lesson was tedious or boring (1.75). Mouthpiece responses showed that students felt: (a) unsure of what they were learning (2.08), (b) a text- book or lecture presentation would have been preferable (2.00), (c) they had to guess on post-test questions (1.85), (d) they did not understand all of the concepts presented (1.82), and (e) some vocabulary was unfamiliar (1.77). The reactionnaire sections in which written comments were requested concerning the three easiest and three most *difficult parts of each package were misinterpreted by many students who felt that they had called for best and worst aspects. In spite of this confusion, a reduction and tabulation of these verbal responses to check for concensus or disagreement yielded useful information for planning revisions. For all packages, there was a noticeable dichotomy of 63 opinion concerning the sections on reed or mouthpiece parts and their respective tonal/response functions. Nine students overall found these sections easiest while twelve judged them to be hardest. Similar disparities could be seen for the single reed adjustment section and for the mouthpiece brand and facing recommendation sections. With eight related comments, concerns about the fast pace of the presentations were by far the most prevalent negative expression, especially in the adjustment section of the oboe reed package. For the single reed package, two students commented on the difficulty of understanding reed density concepts. Other noticeable concerns included unfamiliarity with the subject for both the single reed package and mouthpiece package and too much information in the mouthpiece package. The most common favorable expressions concerning the easiest parts or best aspects of the packages were listenability, organization, and reinforcement for the single reed presentation and flow for the mouthpiece package. Three students commented that the single reed mastery test was one of the easiest parts of that package. Two students remarked that the section on materials was one of the easiest parts of the mouthpiece package. The final page of the reactionnaire contained space 64 for students to list comments or suggestions for the improvement of the packages. Because of the misinterpreta- tion on the easiest/hardest part section of the reactionnaire, many statements here duplicated those listed previously. students offered numerous favorable general comments in this section including, for one or more packages: "good," -"well organized," "held my interest," "very professional," "smooth," "better than any text," "great for methods students," and "nice job." The cartoons used in the oboe reed and mouthpiece packages received specific praise with the most numerous comments applied to the color cartoon sequence used as a motivator at the beginning of the latter package. One student commented favorably on the oboe reed pre-test as an aid in preparing him for the presentation. Another student commented favorably on the voice used for the narration of the single reed presentation. A third expressed liking for the slides on how mouthpieces were made. The most common suggestions for improvement of all packages pertained to pace with statements including: "slow the pace a bit," "...talk too fast," "brands and facings too fast." A related suggestion advised slowing the pace by spreading the presentation over a number of days. One student mentioned in the mouthpiece 65 reactionnaire that the pace of that presentation was much better than that of the oboe reed package which he had seen previously. Several suggestions were made for improving the visual quality of slides: "arrows not clear," "dark to light slides bothersome," and "confusing graphs." One student was bothered by the lack of aural demonstrations in the mouthpiece package. ”Debriefing_Evaluation Fifteen to thirty minutes of open discussion followed the reactionnaire at the end of each package trial. These conversations were kept informal and open in an attempt to elicit any useful, unanticipated data not covered by the tests or reactionnaire. Audio tapes of each discussion were later reduced to lists of suggestions. These lists were organized into topics and ranked according to students' concensus and the number of individual references. Comments on the oboe reed package fell most easily into five categories: pace, audio-visual presentation, tests and quizzes, efficiency, and overall reaction to the medium. The first of these, in students' opinions, was the most serious. Many students criticized the speed of the entire presentation while several singled out the last section, on adjustment, in particular. One student suggested the 66 inclusion of a review handout to accompany the package as a way of compensating for the fact that many students would not be able to absorb all the information presented at such a speed. Another student allowed that a fast pace could be beneficial at times as a method of forcing attention. Comments related to various aural or visual aspects of the presentation were most common after pace. All students expressed-favorable reactions to the use of cartoons between major sections and before major points. Several students noted that they did not see clear relationship between slides of oboists in a band and the narration explaining reed care techniques to be used during rehearsals. Others complained of a lack of synchronization between the audio track and slide changes in various sections. Individual comments included the need for better fidelity in aural reed demonstrations and more specific depiction of various reed adjusting tools. Students offered several unanticipated comments concerning the tests and unit quizzes accompanying the package. Favorable remarks were expressed on the retention value of using identical tests immediately before and after the showing. One student questioned the need for latent image self-correction on the quizzes. The wording of the second question of review 67 quiz two was criticized as misleading and the directions to question seven of the pre- and post-test were criticized as unclear. Several suggestions were made concerning the efficiency of the package. Students generally agreed that it would be better for retention to recommend just one book on reed adjustment rather than several. A few questioned the need to know the names of reed parts. Others mentioned that the presentation of information in the last section took too much time. Overall comments concerning the medium revealed that students felt an audio-visual presentation of this kind of woodwind information was much more effective than a live lecture—demonstration or a textbook. Several suggested that the same media should be applied to other areas-of their music education training. The single reed package discussion included primarily comments in four areas: pace, audio-visual presentation, testing, and content. Although fewer students in this discussion commented on the pace of information presentation than had in the oboe reed debriefing, several suggestions showed that it was still a problem. In addition to general criticisms of speed, various students recommended specific remedies like slower speech, more repetition, or pauses in 68 explaining unfamiliar concepts and handout sheets post-presentation review of the concepts. Several noted audio-visual problems including an inconsistency of handwritten review slides with the more professional quality of the rest of the presentation, a lack of exact relationship between reed pictures and their explanatory diagrams, and a too-subtle distinction in diagrams of French and American cut reeds. Students who had seen the oboe reed package missed seeing cartoons in this-preSentation. In response to a question from the researcher, students generally agreed that the inclusion of physical samples--reeds and adjusting tools--with the package would not be helpful. Whereas no observations had been made on the content of the previous package, several single reed majors expressed opinions on reed choice and maintenance techniques differing from those stated in the package. Majors and non-majors alike, however, commented favorably on the organization of specific tests for choosing a potentially playable reed. Commenting on the tests and quizzes, students said that forced writing of answers during unit quizzes was probably better for retention than merely selecting or visualizing the answer. However, one student involved in the oboe reed trial previously did note that users of a 69 series of similar packages would soon become tired of the quizzing format and tend to skip writing the answers. Another student who had been involved in the oboe reed package trial appreciated the improvement in the wording of questions . Areas of concern derived from the mouthpiece package discussion were identical to those of the single reed package debriefing although the priority of concern had changed. Far more comments were offered on specifics related to audio-visual presentation than to pace. Positive remarks on the presentation showed that students felt it to be superior in general to the oboe reed package, that they appreciated the shortened length, and that they approved the re-introduction of cartoons. One student involved in the single reed trial, however, stated that the former package had a better quality presentation. A feeling of confusion was expressed on part names and their functions. Review frames were criticized for being handwritten and not relating exactly to the narration. Several students missed hearing an aural demonstration of mouthpiece tonal differences explained by the narration. Comments on pace of instruction were generally favorable although some students stated that brand names 70 were presented too fast. Comments on instructional content were also positive. Single reed majors as well as non-majors stated that the information was valuable. A few-negative opinions were expressed about the tests and quizzes. One student felt that the alternation of one and two answer questions on the post-test was confusing; a second stated that the unit quiz questions on mouthpiece facings were unclear. CHAPTER V DISCUSSION, REVISION RECOMMENDATIONS, AND CONCLUSIONS Discussion The sources of evaluative data described in the preceding chapter were: (a) media authority comments, (b) content authority comments, (c) evaluation authority comments, (d) gain scores, (e) criterion acquisition assessment, (f) item difficulty analysis, (9) test relia- bility analysis, (h) Likert scale analysis, (i) written student comments, (j) oral student comments. A further valuable source was the researcher's own subjective perception of student responses during the package trials. In order to formulate suggestions for the further, prOjected revisions of the three packages, the researcher listed, compared, and classified data from each of these sources. For suggesting revisions, some data were valuable only in the most general way. For example, mean gain Scores supported each product's effectiveness as a whole yet they gave no indication of what particular improvements could make it more effective. However, other data--from item analysis, written and oral comments, and Likert scale analysis--generated more specific ideas for revisions. 71 72 In almost all cases, data from one source tended to be corroborated by those of other sources. For example, individual suggestions to slow the pace of particular parts .of the presentations, written by several students on the reactionnaire, were supported by notes from the oral debrief- ing, by Likert scale analysis, by item analysis, even by a review of pfevious consultant comments. Since this confirmation usually provided better insight concerning the eXact nature of the problem, it was a valuable factor in the researcher‘s attempt to derive the most potentially valid suggestions for package revisions. Still, the nature of such a process is necessarily subjective. Although the researcher might assume that most of the revisions he institutes will improve the packages, their actual effects will remain indeterminate until the reSults of another cycle of evaluation are analyzed. Some revisions may correct more than one area of inadequacy in the prototype, but some may be ineffective in their purpose or have unanticipated deleteriOus side-effects. Others may prove to be impractical in the chosen media. Also, the revision process itself will eventually reach a point of diminishing returns at which further refinement is uneconomical. With these points in mind, the researcher will, in this chapter, present a list of revision suggestions based on 73 the organized results of the evaluations. They will be listed as general suggestions, relevant to all packages. Data sources and specific applications will be given under each heading. They are categorized as either suggestions for the improvement of overall package design or for the improvement of particular technical aspects of the presentations. Following this list, conclusions will be offered concerning the formative evaluation process used in this study and suggestions presented for its application to further research. Package Design Revisions Slow the pace'gf important concept presentation. Comments related to pacing were the most numerous and most emphatic statements made on the student reactionnaire and in the oral debriefing sessions. Pacing criticisms aimed at specific areas of packages were reinforced by generally lower post-test scores in those areas. Future packages should incorporate narrative pauses after important concepts and, in varied forms, more repetition of these concepts. Somewhat more deliberate speeCh should also help in these areas. Particular areas of concern are: (a) oboe reed adjustment techniques, (b) oboe reed tip openings, (c) single reed density and tonal variances, (d) mouthpiece brand names, (d) mouthpiece 74 dimensional-tonal relationships. Shorten the presentations. Verbal comments approving the shortening of the third package were reinforced by suggestions advising the division of all packages into shorter, separate presentations. Any plan to shorten the packages as they now stand would conflict with the necessity of slowing the pace of presentation. The most feasible method of accomplishing both goals would be to separate the oboe reed and single reed packages into two units apiece, one dealing with basic reed knowledge and selection and the other with reed adjustment and maintenance. The mouthpiece package can be kept intact although an attempt should be made to improve the efficiency of content presentation without destroying necessary repetition. Include review sheets. In both the oboe reed and single reed debriefings, students commented that a review handout would be an effective method of assuring recall of important concepts since the pace of presentation and the slide-cassette media did not permit adequate note-taking. This need was also indicated by agreement in each package evaluation with the Likert scale statement, "I often needed to go back over a portion of the lesson to fully understand it." 75 Future versions of each package should incorporate a one or two page review handout containing all major points of the presentation in brief outline form. In addition, important visual discrimination concepts should be included in the form.of screened photographic reproductions. Provide advance organization. Likert scale reactions indicating that students felt that they did not have sufficient prerequisite knowledge to prepare them for the lesson and that they often felt unsure of what they were supposed to be learning demonstrate that some form of preview would make the presentations more meaningful. A student preview sheet, in addition to the review sheet proposed previously, seems unnecessarily redundant. Instead, future versions of each package should contain a direction sheet for its incorporation into typical wood- wind techniques or instrumental methods courses. These directions, designed for course instructors rather than for students, should be brief statements concerning mainly what skills and concepts students should possess before viewing the unit and some general, flexible suggestions for attain- ing them. Similar suggestions should be directed toward student motivation. 76 Simplify'brand name and reference book recommendations. Several comments from debriefing sessions indicated confusion with alternate choices of reed brands, mouthpiece brands, and reference book titles. Future versions of the packages should not eliminate these alternate choices but, even at the risk of oversimpli- fication, should stress one preference to a greater degree through more reiteration and visual emphasis. Usgmore”cartoOns. Unexpectedly, the use of cartoons in the oboe reed and mouthpiece packages received numerous favorable comments from students in both opinionnaires and debriefings while the lack of cartoons in the single reed package was noted with some disappointment. The researcher also noticed the relaxing and attention focusing effects of these drawings in the package trials. The large difference in technical quality between the first and second sets of cartoons (oboe reed cartoons were simple line drawings while mouth- piece cartoons were full color and personality developed) was not seen as important. Future packages should include cartoons at appropriate points througout the presentation. Their demonstrated potential for attracting attention and providing humorous, relaxing comment on selected points of the narration can 77 be used effectively in making the packages more enjoyable and may provide a further mnemonic device to promote retention of important concepts. Keep test guestiOns in_gng format. One student failed to notice the change in directions between one and two response questions on the post—tests of both packages he helped evaluate. This significantly lowered his scores and, in one instance, prohibited a package from achieving its pre-set criterion level. In debriefing sessions, other students criticized the alternating one and two question format used in two of the post-tests. Future post-tests should contain only single response items. This will not only eliminate confusion but will allow more flexibility in designing pertinent evaluation items. Although such a format would make the tests slightly longer, this should be mdnimized by designing the questions to be shorter and more easily comprehended. Eliminate latent image self-correction. One student questioned why the researcher had used latent image self—correction on review quizzes. Although this technique had been thought to be an efficient method of assuring immediate concept reinforcement, the expense of the required materials and their inconvenience in actual 78 use coupled with the availability of easier methods for accomplishing the same purpose made an answer to this student's inquiry difficult. Future packages should incorporate less cumbersome and less costly methods of self-correction in review quizzes. Traditional formats of framed programmed instruction can probably be substituted with equivalent effect. Technical Revisions Improvematgg between narration and_complementaryvisuals. Oral criticisms pointing out a lack of relationship between slides showing oboists playing in a band and a narration explaining oboe reed maintenance during rehearsal led the researcher to review all packages for similar potential problems. Future packages should contain more specific visuals in which aural-visual relationships are clarified. These should be incorporated particularly in the areas of oboe . reed maintenance and mouthpiece dimensional-tonal relationships. Show reed areas more precisely. Item difficulty analysis and students' written comments indicated a deficiency in the use of arrows to point out areas of oboe reeds and single reeds. Future packages should demonstrate areas with colored shadings rather than arrows. 79 ImproVe match between photographs and representative graphics. written comments from the single reed package evaluation mentioning confusing graphs in general and a lack of relationship between vamp photographs and their explanatory diagrams in particular indicate that future versions should be more carefully equated. This may be best accomplished by superimposing graphs directly on darkened copies of the photographs. Emphasize aural and visual distinctions. Debriefing comments indicated some confusion in distin- guishing graphic representations of French and American cut single reeds. Single reed post-test item analysis showed an inadequate understanding of tone buzziness. Oral and written comments criticized the lack of sound-track demon- strations of mouthpiece tonal concepts like bright and dark, compact and diffuse. The fidelity limitations of most school audio equipment make these aural distinctions extremely difficult to relay in a slide-cassette format. In future packages, such concepts must be clearly demonstrated even at the expense of grossly exaggerating the distinctions. Aural demonstrations must accompany visuals. 80 Improve test and review quiz questions. Likert scale analysis showed that students felt that the unit guides were generally less than excellent. Oboe reed post-test analysis indicated several weak questions with more than the intended number of defensible answers. Further investigation by the researcher revealed a potential weakness in the relationship of photographic representations from the presentations to the drawn representations of the same concept on tests and quizzes. Future versions of the packages should incorporate improved test questions and screened photographs rather than drawings. Improve technical quality g£_slides. Specific student objections to visual quality included: (a) the use of some handwritten review slides, (b) some unclear graphs, and (c) distracting changes in light intensity from one slide to the next. Further examination by the researcher added slides in which camera reflections or set-up equipment were visible. All slides in future versions of the packages must show professional production quality. Simplify equipment operation. Several written comments criticized a lack of synchronization between narration and slide changes in the 81 oboe reed presentation. The use of expensive image dissolving projection equipment may be more of a hindrance than a benefit in typical school applications. Future packages, once judged adequate, should be converted to cassette-fibmstrip format in which a common 1000 cycle seperate track synchronizing tone, an inaudible fifty cycle synchronizing pulse, and an audible tone for manual filmstrip advancement are incorporated. Cbnclusions The problem presented by this study was to discover whether or not an individual developer could create and refine multi-media packages in supplementary woodwind teChniques for use in teacher training using accepted methods of instructional development and formative evalua- tion. This problem was divided into the five sub-problems which are restated below with pertinent conclusions drawn by the researcher: 1. Can valid content be effectively incorporated into a standard format in these instructional packages? Yes. All three packages used essentially the same format but different content. Each content consultant approved the content of the package he saw. Also, analysis of data from the student evaluations revealed gains in knowledge and positive student attitude for each package. 82 In addition, these data revealed higher mean gains for non-majors on instruments related to each presentation's topic than for majors. This indicates that the level of content in each package was targeted correctly toward teacher trainees rather than applied music majors. 2. Can known principles of audio-visual instruction be effectively incorporated into the design of these packages? Yes. Approval by the media authority of the-media techniques employed, especially after revision of the prototype format indicated that these techniques would be effective. Positive results of the student evaluations both in knowledge acquisition and attitude support this conclusion. 3. Can products of acceptable technical quality and a suitable delivery system be created by an individual developer? Yes. The researcher himself financed and/or created all hardware and software connected with the products. Both the technical consultants and student evaluators generally approved the quality of these materials. Furthermore, all revision suggestions generated by the data analysis seem to be within the reach of the researcher's financial resources and skill. 4. Can effective procedures and instruments for the 83 formative evaluation of these products be created or adapted from existing models? Yes. The Abedor model of formative evaluation proved effective when applied nearly intact to the products generated by this study. Although the pre and post evalua- tion instruments were necessarily created by the researcher and individualized to these packages, they were based on suggestions in the Abedor dissertation. The student reactionnaire was a direct copy of that in the Abedor dissertation. The amount of mutually corroborating data generated using the Abedor model indicate its reliability for this type of product evaluation. 5. Can conclusions be drawn from the evaluation process which will generate ideas for the beneficial revision of these materials? Yes. In almost all cases, evaluative data from one source was corroborated by data from other sources. These corroborations were used to generate revision suggestions for future versions of the packages. The agreement of data indicates that these suggestions will improve the products rather than merely change them. The methods employed in this developmental research were effective in creating and revising multi-media packages in supplementary woodwind techniques. In addition to this conclusion, several others emerged from the study 84 which were not covered by the sub-problem list. First, it is difficult to limit the development, evaluation, and revision processes to discrete, pre- determined steps when creating several packages in the same period of time. The researcher found, for example, that one package could be revised on the basis of data from evaluations of preceding packages. The creative process in any application, including educational product development, must retain some flexibility within designated structures. To function strictly within the constraints of any detailed model of instructional development would cause an-inefficient use of time. Second, the developer of prototype materials should not be overly concerned with professional polish in the preliminary versions of his products. Because of his own familiarity with the materials, he will notice far more numerous technical flaws than will those who are assisting with the evaluations, and attempting to perfect these in versions not designed for publication or final application is not worth the expense. Finally, the process consumes an unexpectedly vast amount of time and energy. Although the schedule for this project had been planned to include ample leeway for unanticipated problems, the researcher found that the total amount of time actually spent at least doubled that 85 originally planned in addition to consuming the time planned for setbacks. Others interested in developing similar products should be forewarned to allow plenty of time. The products developed through this research are considered by the researcher to be a beginning rather than a culmination of study in this area. Further research is recommended in the development and application of media products in other areas of woodwind instruction. Similar products should be developed and evaluated in other secondary instrument courses for music teacher trainees. Finally, additional research should be directed toward the application of similar products in public school situations. APPENDICES APPENDIX A The Abedor Model for Formative Evaluation of Multi-media Instructional Products court: TECHNICAL "VI“ “TA llflll . . PI! TEST MTTEST [.Ylf‘l'l’t TESTS TVAL TASl'IPIS CIH'P DEBI IDS b EOl’l P- UM. TTY DATA CAUSAL [mm LIST DEF lL‘XEXCTES IS RANK film! ‘Izll ‘0‘ C! CO"- mo TRADIOIT MY! IS .2 MK! W404” DEC!!!“ Fm EACH OISCIEPW "mus REV IS! 7 ”mum: cost or mum 6.2.: Y” “It.“ "It“ C-platcy Achieve-t can 6 P00: tut lama COCO. Redundancy Guantana- :JJJ om. Idol 86 APPENDIX A The Abedor Model for Formative Evaluation of Multi-media Instructional Products cm TECHNICAL mm “TA mm? PCTTBST I!!! ."I 1'! TEST! mu. OIJECTIVB VASI‘NIS cm-vv 2 1 [ON 5 EOL‘ IP- QUALITY AMI-Y2! OVA CARS“. FACTOIS LXST DEF lC IEXC It! IS IAN! DIM! .2.x ‘-‘ arcan- CT” .2.2 TWIN? ANALYSIS 5 RESOUICIS M 0040-00 DEC!!!“ ton EACH 03$!!!le nmu IEV [8t IST “no COST 7 0f mums 5.2.: “5 Achieve-It can 0 hot use Inna“ not. Organ-t un- 0 one. "u u: 86 APPENDIX B Student Reactionnaire 0: fl ‘0 l‘ch'lE LESSON TITLE DATE Please be frank and honest in answering the following questions. Remember, you are our prime source of information regarding what needs to be revised. ' KEY: 0) l0. 1_ means you strongly agree; g_means you agree; 3 means you are un- certain; g_means you disagree; and §_means you strongly disagree. I had sufficient prerequisites to prepare me for this lesson. I was often Ensure of what, exactly. I was supposed to be learning. After completing the lesson, I felt that what I learned was either directly appli- cable to my major interest, or provided important background concepts tc me. Manipulating the equipment, or equipment breakdowns often distracted my attention. LiStening to the tapes and watching the slides became tedious, or boring. This lesson was very well organized. The concepts were highly related to each other. A professional Speaker (announcer) should be used to make the tapes. The audio tape moved too fast for me, there was too much information. There was too much redundancy. I was bored by the repetition of ideas. There was a lot of irrelevant infor- mation in this lesson. 87 'T" '1'" .1 “i “i . .| '2” '7?" "3" '75" .4 4 4 _1 "I 444 4 4 4 4 '4" ‘3" 4 4 4 4 4 4 4 4 4 a. oi 64' 61 4 61 16. 17. 19. 20. 24. 88 The workbook was excellently designed. I could easily follnw the instructions and perform the exercises. Frequent reference to and use of the workbook was distracting. Often the tape and slides seemed unrelated to each other. This lesson had very serious gaps and lacked internal continuity. The examples used to illustrate main points were excellent. The vocabulary used contained many un- familiar words. I often did not under: stand what was going on. _ The pre-test and final exam questions did a good job of testing my knowledge of the main points in the lesson. The questions during the lesson gave me valuable feedback on how I was doing. Many of the things I was asked to do, or questions I was asked to answer during the leSson seemed like needless busy work. At the end of the lesson I was still un- certain about a lot of things and had to guess on many of the final exam questions. I believe I learned a lot, considering the time Spent on this lesson. I would recommend extensive modifications to the lesson before using it with other students. For you, what was the most difficult part of the lesson? ll 4 4 4 4 4 4 _.| x1 - 4 .1 n+ 6i _i 6* 6i 4 4 4 4 4 4 “T" T- 4 4 4 4 “i' 6+ T T 44444 4 4 4 4 4 T T .1 4 4 4 4 4 61 61 oi 0‘ 61 What was the easiest part of the lesson? 27. ' 28. 29. 30. 31. 89 What were the three worst things about this lesson? I understood most of the concepts and vocabulary immediately after completing the lesson. 'T- T T T T I think this whole procedure of trying out new materials with students is a waste of time. ‘1'" T T T “'5" I would prefer a textbook or lecture version of this lesson rather than the slide/tape/workbook version. ‘7- T T T T I often needed to go back over a portion ____ of the lesson to fully understand it. T 2 T T, T After completing the lesson, I was more interested in and/or favorably impressed ' with the general subject matter than I was before the lesson. '1— T T T T Please write below any comments, suggestions. or changes which you believe will improve this lesson. Thank you. APPENDIX C Oboe Reed Pre- and Post-test Please respond by putting a check (a’) on the lines beside the best answers. IF you are unsure about your answer, put the check in anyway but indicate your insecurity with a solid question mark beside thar check like this 1? ‘// Remember that the test is to see if the materials are able to teach the concepts. It is not an evaluation of you. Please be as honest and serious as if it were a final exam, though, so we can make good decisions for revisions. 1. Two good sources of further information on oboe reeds are: (two responses) Frederick Westphal's Guide £2 Teaching Headwinds. Evelyn Rothwell's [he Oboist's Cogpanion. Peter Hedrick's Oboe Ree Mak1_ng_ Kalman Opperman's Making gag Adjusting Double Reeds. I». . Two correct ideas about soaking reeds before playing are that: (two rosponscsl Soaking in the mouth is better for the cane than soaking in warm water. About 1 minute is a good average soaking time for most reeds. Too little soaking will encourage cracks to develop. Too little soaking keeps the cane too bowed and resistant to.blowing. 01d reeds often need more soaking than new reeds. 3. The cork should he greased: (two responses) To keep it from deteriorating in humid weather. Every two weeks on average. Only with lanolin treated vaseline. Because trying to pull a dry reed from the oboe well can rip the cork. But not too much. 4. The inside of the reed should be cleaned: (two responses) By turning the reed around and blowing sharply through the cork end. By twirling a feather inside it and flushing with water. By running a pipe cleaner from the tip through the tube. Infrequently, because it deteriorates the critical curvature of the gouge. To remove sma11.globs of grease and particles of food. S. Storing the reed: (two responses) In a sealed tube can cause the cane to mildew. In commercially available plastic envelopes is a good option. In a sealed tube will prevent the tip from becoming too open. In a professional reed case or the hard plastic case supplied with many oboe reeds are the best options. In an old cigarette pack -- like a Marlboro hardpack -- is a recommended option to the expense of professional reed cases. 90 91 6. Reeds should be: (two responses) Kept moist throughout rehearsal or practice sessions either by soaking in water or at least in the mouth. Allowed to dry out periodically during a long rehearsal. Washed under a faucet periodically to prevent buildup of saliva deposits. Thrown out when they become too wet. Soaked cork end down in a vial of warm water to insure that no bubbles form on the critical gouge area. 7. Match each part or area name with the proper lettered arrow if one is provided. Blade . Bark Back Binding Fishskin Spine. Staple Stock Plateau Integration Wire *6 Es Scoring Gouge Rail Heart Table 8. Select two commercially available reeds that are closest to the recommended scrape. (These reeds will be passed around during the test period.) and 9. The two tip openings that appear to be best are: 0—— 4+— 0— o —— «=>—— 0— 10. 11. 12. 13. 14. 15. 92 Two names for materials commonly used in making quality oboe reeds are: (two responses) Bamboo. Sugar Cane. Arundo Donax. Reed Cane. Wood. Fibercane. Two areas that lend strength to the back of the blades and keep them from collapsing when played are: (two responses) Binding. Spine. Tip. Rails. Lay. Heart. Two names commonly given to the style of scrape preferred by most professional oboists in this country are: (two responses) French Style. Long Scrape. Short Scrape. Double cut. American Style. European cut. The two most distinguishable aural differences between the preferred scrape and the non-preferred scrape are: (two responses) The preferred style sounds more open. The non~preferred style sounds reedier. The non-preferred style can't play loud. The preferred style gives a preferred Mallard sound. The non-preferred style sounds more open. The ONE group of brand names whose scrape comes closest to an ideal which would—most likely produce an acceptable American concept oboe sound is: (ONE response) VanDoren, Mitchell Lurie, Rico. Straubinger, LaVoz, Chartier. Prestini, VanDoren, Nielsen. Nielsen, Jones, Straubinger. Gower, Meason, Stevens. .Two comments that can be made about a reed with a too open or rounded tip opening are: (two responses) It won't be able to play loud. It is more common on an older reed than a new one. One cure for it is letting it stand overnight with a buffered clothespin holding the blades shut. One cure for it is squeezing about % inch of the blades together for a few minutes. It will usually sound thin and squeaky. 93 16. Two comments that could be made about a reed with a too closed or flat tip opening are: (two responses) _Letting the blades stand in just boiled water will often round out the _tip and make the reed play better. It is a common problem on new commercial reeds. —A good, quick solution is to gently squeeze the sides of the blades between thumb and finger for a minute or two. Another solution is to gently squeeze the top and bottom of the last millimeter of the binding. (or the wire if the reed has one) It's more common on the non-preferred style of commercial reeds. 17. Two common causes of a reed that plays too hard are: (two responses) Too heavy a binding (on commercial reeds). A too open tip. Too much cane on the tip area. werms in the wood. A too flat gouge. 18. Two useful corrections involving the use of adjusting tools on oboe reeds are: (two responses) Scraping the back of the reed with a reed knife if it's too flat-sounding. Clipping less than a millimeter from the tip with concave toenail clippers to raise the pitch. Scraping the tip with a reed knife (and plaque) if the reed blows too hard. Sanding the sides to narrow the reed if it plays flat in the upper register. Clipping less than a millimeter from the tip with a sharp knife and cutting block to make the reed harder. 19. Two useful tests that don't involve a developed embouchure are: (two responses) Seeing if the reed is apt to play in tune by inserting it up to the binding and trying to crow at or near a "C" pitch. _Allowing the reed to soak overnight in white vinegar to check for calcium —deposit buildup from saliva. Checking for air leakage between the blades by covering the end of the staple, putting the blades between sealed lips, and blowing. Running the thumbnail over the back of the reed to check cane density. Looking at the gouge to determine probable resistance problems. APPENDIX D Single Reed Pre- and Post-test Please respond by butting a check iv’) on the lines beside the besl answers.. If you are unsure of your answer, put the check in anyway but indicate that It was a guess with a solid question mark beside the check like this 73 v’ gnu—‘- Remember that the test is to see if the materials are able to teach the concepts. lt is not an evaluation of you. Please be as serious as iF it were a final exam though so we can make valid decisions For future_revisions. . Two sources of further information on single reeds are: (two responses) Kalman Opperman's Handbook for Making and Adjusting Single Reeds. Larry Teal's Art of Saxophone Playing. Evelyn Rothwell's-Teac er's .ui e to the Clarinet and Saxophone. Hovey and Voxman's "Single Reed Adjustment" (lnstrumentalist, May, 1977). Frederick hastgood's Teaching Single Reed instruments. . Two correct statements about reed care are: (two responses) Even a short period of excess jaw pressure can damage a reed. Reeds are best stored in airtight containers to prevent drying and cracking. Soaking in warm water is better for the reed cane than soaking in the mouth. Periodic cleaning with hydrogen peroxide helps to restore resilience. It's best to use one reed for most playing so that the embouchure can adjust to its particular characteristics. . Two correct statements about breaking reeds in are: (two responses) Massaging the vamp and rubbing the table help keep a reed from waterlogging and losing its resilience. Reeds should be played for only a few minutes the first day they are used and gradually longer on succeeding days until ready for full usage. It takes at least a month to break in a reed. A reed's table won't warp if the reed is broken in carefully. Dipping a new reed in melted paraffin is a recommended trick to assure that the reed won't waterlog too quickly. . A soft, buzzy tone in a reed can be caused by: (two responses) A too thin tip. A too long heart. A table that isn't flat. Mottled bark on the stock. A too long tip. ' In comparison to American cut reeds, French cut: (two responses) Usually play with a freer, lighter tone. Usually give the player more dynamic control without tonal distortion. Usually show a thicker heart. Usually have a slightly thicker tip. ____ Usually require less adjustment to make playable. 911 6. 10. 11. 95 For the instrumental director, choice of single reed brands and cut styles: (two responses) Should be based on those of the serious music professional. Is more commonly American cut than French cut. Is best centered around Vandorens (for clarinet). Usually settles For consistency at the expense of better sound. Should seriously consider the new synthetic material reeds like Ricoplex for the younger student. In choosing reed strengths, the director should: {two responses) > Stock mostly about a 2 with a few strengths on either side of that for those set-ups which demand them. Consider that larger single reed instruments usually require softer reeds. Allow l'or the. fact that more open mouthpieces often require harder reeds. Understand that reed strength gradings vary from brand to brand. Stock only harder reeds and adjust them for individual set-ups. 3 far as choosing an individual reed is concerned: (two responses) Clear tan bark on the stock usually produces a better sounding reed than a mottled appearance. A slight split in the tip indicates that the reed should be discarded. The are and thickness of the heel can be an indication of the reed's density and balance. Color of the vamp and table doesn't matter much. Greyish green coloration on the table is O. K. as long as the vamp is ivory colored. The two heart contour - tip fiber patterns indicating the best potential reed are: (two responses) ll Two useful techniques for making a soft reed harder are: (two responses) Clipping it less than a millimeter. Sanding a warped table lightly on carborundum paper. Sanding or scraping the vamp near the scoring to restore balance. Moving the tip slightly over the tip rail of the mouthpiece. Moving the reed on the mouthpiece slightly toward the weaker side. Two possible causes of a reed that is too resistant are: (two responses) A tip that has been bent down by too much jaw pressure. A warped table. Mushy green cane. Too much wood on the tip area. Too long a tip. (two 96 1°. Two useful techniques for making a hard blowing, resistant reed softer are: responses) Scraping or sanding across the heart about 3 inch behind the tip. Moving it on the mouthpiece slightly down toward the body of the instrument. Scraping or sanding the corners of the tip. Lengthening the tip area slightly by scraping or sanding the front part of the heart. Sanding a warped table lightly on 220 to 320 grit carborundum paper. Ii. A reed that is unbalanced: (two responses) Can be helped by moving it on the mouthpiece toward the weaker side. Can he helped by sanding the heavier side of the table. Can be tested by alternately dampening each side of the reed when it is on the mouthpiece with the lower lip and listening for the difference in sound and feeling the ease of response differences. Is very difficult to fix; it's better to discard it. Can be helped by physically bending down the rails of the reed. 14. Match each part or area name with the proper lettered arrow if one is provided. Blade A Rail Integration Heart Scoring Heel Wiffle Vamp Stock Table Tip APPENDIX E Mouthpiece Pre- and Post-test Please respond by putting a check (V/)'on the lines beside the best answers. If you are unsure about your answer. put the check in anyway, but indicate your guess with a solid question mark beside the check like this a? / Remember that the test is to see if the materials are able to teach the concepts desired. It is not an evaluation of you. Please be honest and as serious as if it were a final exam though so we can make good judgements for future revisions. 1. Two good sources for information on single reed.mouthpieces are: (two responses) Kalman Opperman's deadbeat for; .S n 6 3.9.9.4 Instruments- Larry Teal's Art 2; Sagonhqng P Educational booklets from Instrument manufacturers. John Hansuld's dissertation "A study of factors influencing tone and response in selected single reed mouthpieces.“ ___ Himie Voxman' s 139 _lggigg_t Mogthnlggg. §_le t o and Mainteng_ge. (Booklet published by CEMREL A good test for table flatness is: (one response) “Put a few drape of water on the table and press against a piece of flat glass. Look for areas that don't spread water. Lay a straightedge across the table and check against a light. Use metal shims between the table and a piece of flat glass. Play the mouthpiece. Check the angle with an angular micrometer or vernier caliper. Rails can be checked: (two responses) By visual inspection. With metal shims when pressed onto a piece of flat glass. By placing a few drops of water on them and pressing against a piece of flat glass. By rplling the moistened rails carefully over a piece of flat g ass. With an accurate straightedge. most acceptable material for quality mouthpieces is: (one response) Glass. Crystal. Grenadilla. Bakelite. Rod rubber. 97 10. 98 . Match each part with the apprOpriate lettered arrow. Rail Table Integration Bore Baffle Pacing Throat Tone chamber Well Window Cylindar 0-3 :7 0 two groups with the most potential for quality are: Vandoren Bhu (Clarineth Sumner BPOZ (Bass Clarinet) Selmer 0* (Saxophone), Vandoren RVDZ (Clarinet) Selmer HS’ (SaxOphone). Selmer HS’ (Clarinet) Selmer HS’ (Clarinet). Vandoren Bub (Clarinet) Selmer 0* (Bass Clarinet). Selmer C* (SaxOphone) In general. exterior parts of a mouthpiece affect _;_____ extent than interior parts. (one response) ___’Tone and resistance. Focus and darkness/brightness. Feel to the embouchure.and response. Dynamics and Openness. Tuning and quality. A baffle with a close angle: (two responses) Will cause a diffuse sound. Will make the sound bright. Will raise the pitch. Will lower the pitch. Will encourage projection. A spacious tone chamber: (two responses) Will make the sound bright. Will sound darker and more Open. Will feel more resistant. Is preferable to a wide throat. Facing designations: (two responses) With *‘s indicate a wider window. With *‘s indicate a narrow window. one brand to the next. the same brand. (two responses) to a larger Will have a similar effect to outwardly bowed chamber walls. Don't equate in numerical designation or letters well from Equate roughly with numerical reed strength designations. Usually show larger tip Openings with larger numbers within REFERENCES REFERENCES Abedor, A. J. Development and validation of a model explicating the formative evaluation process for multimedia self-instructional learning systems (Doctoral dissertation, Michigan State University, 1971). Dissertation Abstracts International, 1972 ‘23, 4195A. (University Microfilms No. 72—8621) Abedor, A. J. Second draft technology. Viewpoints: Bulletin of the School of Education, Indiana University, I972, lg, 9:14. Alkin, M. C. & Fink, A. Evaluation within the context of product development: a user orientation. In G. D. Borich (Ed.) Evaluating educational programs and products. Englewood Cliffs, N. J.: Educational \ Technology Publications, 1974. Anderson, J. N. An investigation of the effect of using tape recorded aural models for home practice on selected sight reading and performance skills of sixth grade clarinet students (Doctoral dissertation, University of Texas at Austin, 1979). Dissertation Abstracts International, 1979, 19, 3176A. (University . Microfilms No. 7928236) Apple, M. W. The process and ideology of valuing in educational settings. In A. A. Bellack & H. M. Kliebard (Eds.) Curriculum and evaluation. Berkley, CA: McCutchan Publishing Co., 1977. Barson, J. Instructional systems development: a demonstration and evaluation project(Contract No. OE-5-16-025). Eastfiiansihg, MI: M chigan State University, June 1965. Beck, J. G. Educational film for the fine and performing .arts-(Doctoral dissertation, St. Louis University, 1979). Dissertation Abstracts International, 1979, 12, 1822A. (Univeristy Microfilms No. 79223639) Bertram, C. L. & Childers, R. D. A multistage model for evaluating educational products. In G. D. Borich (Ed.) Evaluating educational ro rams and products. EnglewoodCliffs, N. J.: Educational Technology Publications, 1974. 99 100 Bloom, M. S., Hastings, J. L. & Madaus, G. F. Handbook ‘gg formative and‘Summative evaluation in student learning. New York: McGraw-Hill, 1971. Boley, H. B. The development and evaluation of the effectiveness of a pilot instructional sound film for teaching beginning trumpet students with a brief survey of the history of trumpet instruction and the history of educational film research (Doctoral dissertation, University of Iowa, 1970). 'Dissertation' Abstracts International, 1970, 31, 1035A. (University Microfilms No. 70-15,583) Borg, W. R. & Gall, D. Educational research, an introduction (3rd ed.) New York: Longman, 1979. Borich, G. D. (Ed.) EValuatin educational programs and ‘ roducts. Englewood—Cli fs, N. J.: Educational— gecfinology Publications, 1974. Burgess, N. E. The application of a self-instructional multi—media approach to teaching violin in a college string -techniques course (Doctoral dissertation, Indiana University, 1974). Dissertation Abstracts ~International, 1974, 34, 4219A. ’TUniversity Microfilms NO- 73-301663) — Christopherson, L. L. Teaching African music with the aid of video taped performances and demonstrations by African musicians (Doctoral dissertation, North- western University, 1973). Dissertation Abstracts International, 1974, 34, 6681A-6682A. (UfiiVersity Mi crofi Iris No. 74-7727)- Coleman, H. Perceptions of music teacher competencies through a survey of public school music teachers in selected school districts: a positive response to accountability for higher education (Doctoral dissertation, Memphis State University, 1979). Dissertation Abstracts International, 1980, 42, 3861A. (University Microfilms No. 8001179) Cunningham, D. J. Comments on the case studies of formative evaluation--the source of information. yiewpoints: Bulletin of the School of Education, Indiana University,I972}_§§,_III:II7T 101 Damron, B. L. The development and evaluation of a self- instruction sequence in jazz improvisation (Doctoral dissertation, Florida State University, 1973). Dissertation Abstracts International, 1974,35, niversity Micfafilms No. 74- 18, 028) Diamond, R. M. & Collins, G. The use of 8 mm loop films to teach the identification of clarinet fingering, embouchure, and hand position errors. Journal of ‘Research in Music Education, 1967,15, 224-228. Druchler, P. F. A study comparing programmed instruction with conventional teaching of instrument fingerings and music pitch notation for beginning students of clarinet, flute, and trumpet in a flexible scheduled curriculum (Doctoral dissertation, State University of New York at Buffalo, 1972). Dissertation Abstracts International, 1972, 3;, 1185A. (University MicEEfilms No. 72-23519) Duncan, D. J. Practices and standards in the teaching of woodwind technique classes in the music education curriculum in selected colleges and universities in United States (Doctoral dissertation, Indiana University, 1978). Dissertation Abstracts International, 1979, 32, 7208A. (University MiCrofilms No. 7912466) Dvorak, D. D. The use of film-strip sound materials in instrument repair instruction (Doctoral dissertation, Arizona State University, 1973). Dissertation Abstracts International, 1973, 34, 6428A. (University Microfilms No. 73-3I§86) Eash, ,M. J. Developing an instrument for the assessment of instructional materials. Paper presented at the [meeting of the American Educational Research Association, Minneapolis, March 1970. Eastman Kodak Company. Basic art techniques for slide production. Rochester, N. Y. : Author,41978. Eastman Kodak Company. Legibilit¥—-artwork_ to screen. Rochester, N. Y. Aut or, . Eastman Kodak Company. Planning and producing slide programs. Rochester, N. Y.: Author, 1978. 102 Ellis, F. N. The production, development, and validation of an original self-instructional program.and supplement on musical note and rest values in common time (Doctoral dissertation, University of Maryland, 1973). Dissertation Abstracts International, 1974, 33, 238A. (University Microfilms No. 74-16526) Fritz, R. A. The development and evaluation of individual learning units in basic listening skills for prospective elementary classroom teachers (Doctoral dissertation, Ohio State University, 1979). Dissertation Abstracts International, 1980, 39, 3614A-3615A. (University Microfilms No. 8001732) Gerlach, V. S. & Ely, D. P. Teaching and media: a systematic a roach (2nd. ed.) Englewood Cliffs, N. J.: Prentice-Hall, Inc., 1980. Gonzo, D. & Forsythe, J. Developing and using videotapes to teach rehearsal techniques and principles. Journal 9: Research in Music Education, 1976, 31, 32—41. Hamreus, D. G. Pretotypg development. Monmouth, OR: Teaching Research Division of the Oregon State System of Higher Education, 1969, Hansuld, A. M. The development and evaluation of two filmed methods for teaching principles of selected musical wind instrument skills on the college level (Doctoral dissertation, University of Miami, 1971). Dissertation Abstracts International, 1972, 33, 5826A. (University Micrafilms No._72-1292) Hedrick, P. & Hendrick, E. Oboe reedmaking: a modern method. Ithaca, N. Y.: Swift-Dorr Publications, Hill, J. R. An investigation. and evaluation of the use of visual aids for teaching music appreciation at the college level (Doctoral dissertation, University of Oklahoma, 1972). Dissertation Abstracts International, 1972, 33, 5826A. (University MicrOfilms No. 72-1292) House, R. The future of music education in higher education. In A. Motycka (Ed.) Music education for tomorrow's society: selected topics. Jamestown, R. I.: GAMT Music Press, 1976. 103 Husak, T. J. The development and evaluation of programmed instruction in the techniques of jazz ensemble arranging (Doctoral dissertation, Ohio State University, 1978). 'Dissertation Abstracts gpternational 1978, 33, 741A. (University Microfilms No. 7812346) Katz, D. S. & Morgan, R. L. A holistic strategy for the formative evaluation of educational programs. In G. D. Borich (Ed. ) Evaluating educational programs ' and products. Englewood Cliffs, N. J.: Educational Technology Publications, 1974. Kirshner, D. D. Doctoral research in educational media 1969—1972. Boulder, CO: University of Col lorado, Krueger, V. R. The development and evaluation of a self-instructional program for teaching recognition of selected mmsic timbres at the elementary level (Doctoral dissertation, University of Southern Colifornia, 1974). Dissertation Abstracts International, 1974, 35, 3039A. (University Microfilms No. 73-30,943) Lawrence, S. L. Development and evaluation of a self- instructional program in music reading for instrumental music students (Doctoral dissertation University of Iowa, 1973).' Dissertation Abstracts International, 1977, 33, 3452A. (UniVersity Microfilms No. 73-30,943) Lawson, T. E. FOrmative instructional product evaluation. Englewood CIiffs, N. J.: Educational Technology Publications, 1974. Leonhard, D. & Colwell, R. Research in music education Bulletin of the Council for Research in Music Education, “1976 49, 1— 30. Levie, W. H. & Dickie, K. E. The analysis and application of media. In R. M. Travers (Ed.) Second handbook of research on teaching. Chicago: Rand-McNally Co. 1973. 104 Lindelien, D. R. The design and use of video tape in teaching the basic care and maintenance of the primary wind instruments (Doctoral dissertation, University of Oregon, 1979). Dissertation Abstracts International, 1980, 40, 4793A. (University MicrofiIfis No. 80057897 Luft, M., Lujan, J., & Bemis, K. A quality assurance model for process evaluation. In G. D. Borich (Ed.) Evaluatin educational programs and products. Englewood Cliffs, N. J.: Educational Technology Publications, 1974. ’ McClintick, J. L. An instructional package for pre- service teachers to teach expressive properties to primary children (Doctoral dissertation, Columbia Teachers College, 1976). ‘Dissertation Abstracts International, 1976,37, 857A. (University Microfilms No. 76-17,285) Miller, J. K. The development and evaluation of an experimental approach to teaching rhythm to beginning wind instrumnnt players (Doctoral dissertation, Pennsylvania State University, 1973). Dissertation Abstracts International, 1974, 33, 1688A-1689A. (University Microfilms No. 74:20,949) Opperman, K. Handbook for makin and ad'ustin single reeds. New York: Chappel & Co., I956. Paulson, C. F. Evaluation of instructional systems. In J. Crawford (Ed.) National research trainin manual. Monmouth, OR: Teach1ng Research D1vision of the Oregon System of Higher Education, 1969. Rainbow, E. Instrumental music: recent research and considerations for future investigations. Bulletin of the Council for Research in Music Education, I973, 33,-8:19T_———' _—'_—_—— Rees, F. J. The exploratory study into the development and evaluation of a self-instruCtional supplement videotape program.for the beginning double bass player. Unpublished doctoral dissertation, University of Southern California, 1976. 105 Richardson, A. The clarinet mouthpiece in relation to tone and pitch. Kenosha, WI: GT Leblanc Corp., I962. Robbins, D. E. The development and evaluation of a series of video-tape lessons to supplement a college course in advanced music theory (Doctoral dissertation, North Texas State University, 1979). Dissertation ’Abstracts International, 1979, 32, 1338A. (University Microfilms Non 7919747) Sanders, J. R. Comments on the case studies of formative evaulation—-search for structure. Viewpoints: Bulletin of the School of Education, Indiana University: 1972, 48, 1194132. Sanders, J. R. & Cunningham, D. J. Techniques and procedures for formative evaluation. Protland, OR: Northwest Regional Educational Laboratory, 1974. (A document prepared for the Chrysler Institute.) Sanders, J. R. & WOrthen, B. A descriptive summary of frameworks for planning evaluation studies. School Research Information Service Quarterly, 1972, 10-14. Schwen, T. M. (Ed.) Four views of formative evaluation in instructional development. Viewpoints: Bulletin of the School 9: Education, Indiana UniVersity, £3, 19727— Scriven, M. The methodology of e alution. In R. W. Tyler R. M. Gagne, & Scriven, M. (Eds.) Perspectives of Curriculum Evaluation. AERA monograph series on curriculum evaluation, vol. 1 Chicago: Rand-McNally, 1967. Shaw, A. C. The development and evaluation of a programmed learning approach in teaching the elements of snare drum technique (Doctoral dissertation, Indiana University, 1971). Dissertation Abstracts International, 1971, 31, 6653A. (University Microfilms No. 71-15,?54) Sherry, M. W. The development and evaluation of a Self- instructional system for teaching selected musical concepts to general college students (Doctoral dissertation, University of Southern Californiaq 1975). Dissertation Abstracts International, 1975: 33, 21A. (University Microfilms No. 75-15576) 106 Sidnell, R. G. Self-instructional drill materials for student conductors; JOurnal 22 Research 33 Music Sprenkle, R. & Ledet, D. The art of oboe la in . Evanston, IL: Summy-Birchard_Co., 1961. Stake, R. E. The countenance of educational evaluation. In A. A. Bellack & H. M. Kliebard (Eds.) Curriculum and EValuation. Berkley, CA: McCutchan Pfib'l 191/- Stuart, M. The use of videotape recordings to increase teacher trainees’ detection skills. Journal pf Rheearch 32 Music Education, 1979, 31, 13-19. Stubbins, W. H. The art pg clarinetistry. Ann Arbor, MI: Ann Arbor Publishers, 1966. Teal, L. The art 9; saxo hone playing. Evanston, IL: Summy-Birchard Col, . Tyler, L. L. & Klein, M. F. Recommendations for curriculum.and instructional materials. Los Angeles: University of California. Los Angeles, 1967. (mimeo) walker, J. P. A quality assuranCe model for educational research and development. In G. D. Borich (Ed.) Evaluating educational programs and products. Englewood CIiffs, N. J.: Educational Tec o ogy Publications, 1974. Williams, E. C. The use of videotape in the solution of selected problems in oboe-playing--a field study (Doctoral dissertation, University of Illinois, 1978). Dissertation Abstracts International, 1979, 32, 7210A. (University Microfilms No. 7913653) wright, W. J. & Hess, R. J. A criteria acquisition model for educational product eValuation. In G- D. Borich (Ed.) Evaluatin educational programs and products.‘ Englewoo C 1 s,-N'T'J_.':'—EHucationaI Te—chnology Publications, 1974. Wyatt, L. D. The development and testing of auto-tutorial instructional ‘materials for choral conducting students (Doctoral dissertation, Florida State University , 1974), Dissertation Abstracts International, 1975, 35, 42223. (University Microfilms W 75-984) '— FOOTNOTES 107 FOOTNOTES 1Students who are subjected to a pre-test just prior to an experimental treatment may be sensitized in that they know what will be expected of them after the treatment. In this case, they would know what concepts they would be expected to deal with after the presentation. 2Sophisticated equipment does not guarantee faultless operation. Several slide advancing signals on the sync track failed to operate the dissolve unit causing the researcher to re-synchronize the visuals to the narration manually.