INFORMATION TO USERS Thia malarial was produced from a microfilm copy of the original document. While the moat advanced technological meant to photograph and reproduce this document have bean used, the quality i* heavily dependant upon the quality of the original submitted. The following explanation of techniques is provided to help you understand markings or patterns which may appear on this reproduction. 1. The sign or "target" for pegas apparently lacking from the document photographed is "Missing Page(s)". If it was possible to obtain the missing pega(s) or section, they are spliced into the film along with adjacent pages. This may have necessitated cutting thru an image and duplicating adjacent pages to insure you complete continuity. 2. Whan an image on the film is obliterated with a large round black mark, it is an indication that the photographer suspected that the copy may have moved during exposure and thus cause a blurred image. You will find a good image of the page in the adjacent frame. 3. Whan a map, drawing or chart, etc., was pert of the material being photographed the photographer followed a definite method in "sectioning" the material. It is customary to begin photoing at the upper left hand corner of a large sheet and to continue photoing from left to right in equal sections with a small overlap. If necessary, sectioning is continued again — beginning below the first row and continuing on until complete. 4. The majority of users indicate that the textual content is of greatest value, however, a somewhat higher quality reproduction could be made from "photographs" if essential to the understanding of the dissertation. Silver prints of "photographs" may be ordered at additional charge by writing the Order Department, giving the catalog number, title, author and specific pages you wish reproduced. 5. PLEASE NOTE: Some pages may have indistinct print. Filmed as received. Xgrox University M icrofilm s 300 North Z n b Road Ann Arbor, Michigan 4*100 73-29,771 ROTH, Arthur Raymond, 1928THE DEVELOPMENT OF A COGNITIVE MUSIC ACHIEVEMENT TEST TO ASSESS HIGH SCHOOL INSTRUMENTAL MUSIC STUDENTS IN MICHIGAN. Michigan State University, Ph.D., 1973 Music University Microfilm s. A XEROX C o m p an y. A nn Arbor. M ichigan THE DEVELOPMENT OF A COGNITIVE MUSIC ACHIEVEMENT TEST TO ASS&SS HIGH SCHOOL INSTRUMENTAL MUSIC STUDENTS IN MICHIGAN By A . Raymond Roth A THESIS Submitted to Michigan State University In partial fulfillment of the requirement for the degree of DOCTOR OF PHILOSOPHY Department of Music 1973 ABSTRACT THE DEVELOPMENT OF A COGNITIVE MUSIC ACHIEVEMENT TEST TO ASSESS HIGH SCHOOL INSTRUMENTAL MUSIC STUDENTS IN MICHIGAN By A . Raymond Roth This study involves the development of a cognitive music achievement test and its administration to 2714 tenth, eleventh and twelfth grade instrumental music students in Michigan. Eight variables plus a control group were utilized as a basis for comparison of scores. Procedures Three pilot tests were constructed and administered to selected students in Michigan, Iowa, Wisconsin and Illinois in the development of the final test. The test items were constructed from a list of objectives determined by Michigan school band and orchestra directors to be logical outcomes of a student's participation in high school band or orchestra. Twenty-three items were aural and twenty-three were visual. The aural items were presented on a pre-recorded tape prepared by the investigator. The entire test was forty-two minutes in length. A . Raymond Roth During the development of the test, each Item was subjected to Item analysis techniques to improve the reliability of the test. The reliability of the final test was .80 as determined by the test-retest method utilizing seventy-one students and .81 as determined by the Kuder-Rlchardson #20 Method utilizing the final test. The test was administered to instrumental music students in fifty-one randomly selected Michigan high schools in May, 1972. Hypotheses and Results 1. There Is no significant difference in the cog­ nitive achievement of tenth, eleventh and twelfth grade instrumental music students. Rejected. 2. There is no significant difference in the cog­ nitive music achievement of students in the various MSBOA school classifications. Rejected. 3. There is no significant difference in the cog­ nitive music achievement of those students who participate in MSBOA Band and Orchestra Festivals and those whose groups do not. Rejected. 4. There is no significant difference in the cog­ nitive music achievement of those students whose groups rate first division in MSBOA Band and Orchestra Festivals and those whose groups do not. Rejected. 5. There is no significant difference in the cog­ nitive music achievement of students who play instruments in the four separate families. Rejected. 6. There is no significant difference in the cog­ nitive music achievement of those students who have taken private lessons and those who have not. Rejected. 7. There is no significant difference in the cog­ nitive music achievement of those students whose parents, one or both, have participated in a school band or orchestra and those students whose parents have not. Rejected. A . Raymond Roth 8. There Is no significant difference in the cog­ nitive music achievement of those students who have taken a formal music theory course in high school or in a stunner music camp and those who have not. Rejected. 9. There Is no significant difference in the cog­ nitive music achievement of those students who have taken Instrumental music in the secondary school and those who have not. Rejected. In addition to the above results, the test provided further information: 1. It is possible to develop a cognitive music achievement test which will measure many factors, which can be administered in a normal high school class period and which will be sensitive to measure the varying abilities of students. 2. There is considerable variance in the cognitive music achievement of students as shown by a comparison of the mean scores of the fifty-one schools tested. 3. There is considerable variance in the cognitive music achievement, as shown by mean scores of stu­ dents who form the different populations as determined by the variables. PLEASE NOTE: Acoendix B has v e r y blurred and indistinct tyoe. Best available c o d v . Filmed as received. UNIVERSITY MICROFILMS. I ACKNOWLEDGEMENTS I 4m grateful and deeply Indebted to: Cldney, who has been a patient and understanding co-worker throughout this project, Dr. Robert Sldnell who has provided helpful, thought­ ful and stimulating suggestions, the many members of MSBOA who have, In many ways, assisted In the testing process, Lisa and Linda who, though too young to understand, provided a certain amount of Indifference which furnished a needed degree of balance to our lives. 11 TABLE OF CONTENTS Page LIST OF TABLES ................................... vt LIST OF GRAPHS ................................... ix Chapter I. THE PROBLEM............................... 1 Need for Study.......................... I Purpose of the S tudy.................... 7 Hypotheses .............................. 9 Definitions of Ter m s .......................10 Limitations of the Study................... 12 Scope of the S tu d y ........................ 14 Overview of the Thesis.....................15 II. REVIEW OF LITERATURE........................ 17 Achievement Testing— An Historical Perspective.............................. 17 Published Music Achievement Tests ....... 24 Elementary Music Achievement Test . . . . 24 Watkins-Famum Performance Scale . . . . 26 Aliferis Music Achievement T e s t ..........27 Snyder Knuth Music Achievement Test . . . 29 Studies Involving Music Achievement Tests. . 31 Colwell .............................. 31 litis ................................ 32 Swinchoski .......................... 34 M a n s u r ................................ 36 Folstrom ................. 38 Sunsoary.............. 40 III. DESIGN AND ADMINISTRATION OF THE STUDY .... Development of the Testing Instrument . . . Test Construction, Pilot Test No. 1 . . . . Item Construction........................ Administration of Pilot Test No. 1 . . . . . Data Analysis, Pilot Test No. 1 . . . . . . . Test Construction, Pilot Test No. 2 . . . . lit 42 42 44 46 48 49 51 Chapter Page Administration of Pilot Test N4>. 2 ......... 52 Data Analysis, Pilot Test No. 2. . . . . . . 53 Test Construction, Pilot Test Ho. 3 . . . . 56 Administration of Pilot Test N«i>. 3 57 Data Analysis of Pilot Test No*, 3 ......... 57 Test Construction, Final T e s t ............. 58 Random Selection of Schools ............. 59 Administration of the Final T e * t ........... 63 Processing and Analysing the D * t a ..........64 Validity.................................. 66 Reliability.............................. 68 Factor Analysis ........................ 72 IV. FINDINGS OF THE S T U D Y ........................ 74 74 Introduction ............................ Level of Significance..................... 74 Frequencey Distribution, Statewide Test . . 74 Hypotheses Tested . . . . . ....... . . . 78 Interaction of Variables................... 92 Supplementary Findings.................... 100 V. SUMMARY, CONCLUSION AND IMPLICATIONS FOR . . FURTHER STUDV ............................... 103 Introduction ............................ 103 S u n m a r y ................................. 103 Conclusions............................. 104 Implications for Further Research ....... 110 BIBLIOGRAPHY ..................................... 113 APPENDICES............. 117 A. TEST OBJECTIVES............................. 117 B . PILOT TEST NO. 1, FREQUENCY DISTRIBUTION, ITEM DIFFICULTY AND DISCRIMINATION INDICES .. . 120 C. PILOT TEST NO. 2, FREQUENCY DISTRIBUTION, ITEM DIFFICULTY AND DISCRIMINATION INDICES . .133 D. PILOT TEST NO. 3 (FINAL TEST), INSTRUCTIONS FOR TAPE PREPARATION, FREQUENCY DISTRIBUTION. . 145 E. LIST OF RANDOM NUMBERS FOR SCHOOL SELECTION PROCESS.....................................172 iv Chapter Page F. FORMS AND MATERIALS FOR ADMINISTRATION OF FINAL T E S T ............................... 173 G. SUMMARY STATISTICS--ITEM ANALYSIS OFFINAL T E S T ...................................... 185 H. RESULTS OF INDIVIDUAL SCHOOLS.................188 I. FACTOR ANALYSIS— FINAL T E S T .................. 191 v LIST OF TABLES Table 1. 2. 3. Page Distribution of Item Difficulty and Discrimination Indices, Pilot Teat No. 1 . . . Expected Error of the Standard Error of Measurement........................... 49 51 Distribution of Item Difficulty and Discrimination Indices, Pilot Test No. 2 . . . 53 Distribution of Item Difficulty and Discrimination Indices, Pilot Test No. 3 . . . 57 5. Reliability of Pilot Tests ................. 69 6. Test-retest, Reliability ................... 71 7. Frequency Distribution, Statewide Test. . . . 76 7a. Summary Statistics of Statewide T e s t ..... 8. Summary Statistics of Variable--Grade .............. in School 4. 3a. 8b. 9. 9a. 9b. 77 78 One-Way Analysis of Variance. Dependent Variable--Score of Test, Category Variable-Grade in S c h o o l ........................ 79 Scheffd Post Hoc Method of Multiple Comparisons Mean Differences (x-y)....... 79 Sumnary Statistics of Varlable--SchoolClassi­ fication 80 One-Way Analysis of Variance. Dependent Variable--Score of Test, Category Variable-School Classification .................... 81 Scheffd Post Hoc Method of Multiple Comparisons Mean Differences (x-y)....... vi 81 Table 10. 10a. 11. Page Sunmary Statistics of Variable — Festlval Attendance ....................... One-Way Analysis of Variance. Dependent Variable— Score of Test, Category Variable — Festival Attendance.................... Sunmary Rating 11a. lib. 82 . 82 Statistics of Variable--Division ....................................83 One-Way Analysis of Variance. Dependent Variable— Score of Test, Category Variable — District Rating ........................... 84 ScheffA Post Hoc Method of Multiple Comparisons Mean Differences (x-y) .......... 84 12. Sunmary Statistics of Variable — Instrument F a m i l y .......................... 85 12a. One-Way Analysis of Variance. Dependent Variable--Score of Test, Category Variable — Instrument F a m i l y .......................... 85 12b. Scheffd Post Hoc Method of Multiple Comparisons Mean Differences (x-y)........... 86 13. Sunmary Statistics of Variable— Private Lessons .............................. 87 13a. One-Way Analysis of Variance. Dependent Variable--Score of Test, Category VariablePrivate L e s s o n s ............................ 87 14. Sunmary Statistics of Variable— Parent Participation ............................. 88 One-Way Analysis of Variance. Dependent Variable--Score of Test, Category VariableParent Participation ....................... 88 14a. 15. Sunmary Statistics of Variable--Music Theory Course..................................... 89 15a. One-Way Analysis of Variance. Dependent Variable— Score of Test, Category Variable— Music Theory C o u r s e .........................89 vil Table Page 16. Summary Statistics of Variable-T r e a t m e n t .................................. 90 16a. One-Way Analysis of Variance. Dependent Variable--Score of Test, Category Varlable-T r e a t m e n t .................................. 90 16b. Supplementary Information Relative to Hypothesis 9. MeanDifferences.............. 91 17. Sunmary Statistics, Grade--Classification Interaction................................ 93 17a. Two-Way Analysis of Variance with Fixed £ffect Model. Interaction of Grade and Classification ............................. 94 18. Sunmary Statistics, Private Lessons-Instrument Familyinteraction................ 94 18a. Two-Way Analysis of Variance with Fixed Effect Model. Private Lessons-Instrument FamilyInteraction................ 95 19. Sunmary Statlstlcs--Dlstrict Rating, Private Lessons, Music Theory Course Interaction................................ 97 19a. Cell Means, Music Theory Course--Private L e s s o n s .................................... 97 19b. Three-Way Analysis of Variance with Fixed Effect Model. District Rating--Private Lessons--Music Theory Course Interaction . . . 20. 98 Comparison of the Influence of the Independent Variables upon the Dependent V a r i a b l e s ................................. 100 viii LIST OF GRAPHS Graph Page A. Interaction of Private Lessons— Instrument F a m i l y .......................... 96 B. Interaction of Private Lesaons-Muslc Theory C o u r s e ........................ 99 lx CHAPTER I THE PROBLEM Need for Study Instrumental music in grades K-12 has evidenced enormous growth In the past fifty years. It has pro­ gressed from a beginning in which a few schools permitted the teaching of instrumental music on an after-school basis to present practices where the vast majority of schools include instrumental music of the curriculum. sb an Integral part During these fifty years there has been a tremendous outpouring of effort and money to instruct, equip, maintain and evaluate these Instrumental music departments. The first National Band Contest was held June 5, 6, and 7, 1923. Although its purpose was primarily for entertainment and publicity,* in a relatively short period of time such contests, and later festivals, became conmonplace and served to provide serious subjective evaluations of performances, both solo and group. School instrumental music evaluation has shown a similar record of growth in the State of Michigan. Cur­ rently thousands of young musicians are Involved in solo *Emil A. Holz, '*The National Band Tournament of 1923 and its Bands," Journal of Band Research. Autumn, 1966, p. 17. and group festivals. From the results of these festivals It would be possible to develop lists which would show the ratings that individual schools or students have accumulated over a period of years. Although these ratings may be somewhat unreliable in measuring the total musicianship of a student, they would provide a basis for compiling lists of first, second, third, and fourth division evaluations of performances. There is constant evaluation of this type of student musical achievement throughout Michigan. BenJamine Bloom defines three domains of learning 2 skills as cognitive, affective and psychomotor. If we accept these three domains, it becomes apparent that the current evaluation of musical skills is centered largely with performance skills and, thus, with only one general classification of outcomes. Although some cognitive knowledge is required for these skills, and aspects of the affective may be present, a musical performance can rely primarily upon the psychomotor skill. The evaluation of instrumental music student achievement over the past forty years in Michigan has been, primarily, In these psychomotor skills. In recent years music educators have placed consider­ able emphasis upon increasing the amount of cognitive knowledge taught to members of large performance groups. 2 Benjamin Bloom, Taxonomy of Educational Objectives. Handbook I: Cognitive Domain twew York: David mc k s y Co .. inc., 1936}, pT 7. 3 It has been theorized by many that quality student per* formance In bands and orchestras will require more than the rote learning of psychomotor skills. This will require knowledge about music which will enhance these student performances and help students to become better all-round musicians, not merely performers. Woodruff states that successful performances are products of four kinds of learning: Concepts--of harmony, form, and other things. Symbols--notes, signatures, rests, technical terms. Motor abilities— muscular performance skills. Feelings--likes and dislikes for various musical effects and their Influence on choice.3 If students are to be given musical experiences which will remain with them and if they are to retain knowledge about music beyond their school years, they will require more than psychomotor skills. Colwell recognises the Importance of cognitive knowledge as a part of the teaching process as he states: There is hardly an area of living In which cognition is not Important, and though we are prone to emphasise the subjective and emotional nature of tousle, know­ ledge is essential to valid experiences with music. Only with knowledge can the emotional response be transformed Into the truly aesthetic response, the performer's skills create honest and appropriate interpretations of the composer's intent, the passing musical Interest of the public school student be retained as a lifelong pursuit for recreation..^ o Asahel D. Woodruff, "Concept Teaching In Music," In Perspectives In Music Education Source Book III, ed. by Bonnie C. Kowall (Washington, D.C.: HKNII,' I9bb), p. 220. 4 Richard Colwell, The Evaluation of Music Teaching and Learning (Englewood Clitts, N.J.: Frentice-Hail, inc., 1970), p. 80. 4 If we are going to teach cognitive skills, we need to evaluate these efforts. Empirical evidence reveals that evaluation occurs, to varying degrees, within indi­ vidual schools; but no district, regional or statewide programs exist which might inform us as to cognitive knowledge achievement, or any of the factors which affect this achievement. As Bloom states, in reference to the cognitive domain; It is the domain in which most of the work in curric­ ulum development has taken place and where the clearest definitions of objectives are to be found phrased as descriptions of student b e h a v i o r . 5 Thus the cognitive domain lends itself more readily to observation, analysation, evaluation and course redevelop­ ment . However, before this can be accomplished, we need to measure present cognitive music achievement accurately and, hopefully, to determine some factors which affect this achievement. Such a test must be both a reliable and a valid measurement. It must lend itself to ease in scoring and it must be short enough to be administered in one normal high school period of approximately fifty minutes. A test which requires multiple periods for administration Is cumbersome to administer and disrupts the normal operation of a school program. In addition, a multiple period test will more likely provide an opportunity for varied student reaction to the testing ^Bloom, Taxonomy of Educational Objectives, p. 7 5 situation and experimental mortality will affect the reliability of the teat. There are few cognitive muaic achievement teat a which fit all of the above attributea and none which la baaed upon specific objectives determined by Michigan instrumental music teachers. Over the past decade the Michigan School Band and Orchestra Association has held music education workshops in which problems peculiar to instrumental music are discussed. Considerable time has been spent In developing publications such as the Handbook of Music Theory and Music History and Literature. A Handbook for Rehearsal Enrichment. Both publications were developed as enrich­ ments to band and orchestra rehearsals and are presently in use throughout Michigan. Through these publications the state organization has attempted to effect curriculum changes in individual schools; however, there has been no effort, either on a regional or a statewide basis, to assess student achievement levels in these areas. The Michigan State Board of Education has made clear its position that all education within Michigan must be organized, delivered and assessed in such a manner that all Involved with this process are held accountable for success or failure. It Is the board's hope that the implementation of accountability will result in an improve­ ment of educational services to the youth of Michigan. In the preparation of the model for building accountability 6 the board has indicated six steps to be accomplished by educators. Step These six steps are: 1. Common goals for children Step II. Performance objectives areas Step 111. Child-school needs assessment Step Step Step IV. New delivery system plans V. Evaluation VI. Recommendations Steps II and III state more specifically: Step II. There are, by conmon consensus and by def­ inition, certain things it Is assumed children ought to know at various stages In their development. This information must now be translated Into performance measures. While much work remains to be done, the performance objectives fall naturally Into skill areas and attltude-aaplratlon areas which are, psychologically speaking, In the cognitive domain, the psycho-motor domain or the affective domain. Step III. Having identified the goals for children, and having articulated the performance objectives for schools. It la necessary to assess the existing relationship between them. This analytical chore must utilize all the knowledge at hand: research, testing, resource distribution and personnel avail­ ability and a host of others. The objective Is to give local school officials some notion of the variance between desirability of performance objec­ tives and what the child or children can do (needs assessment).° Ralph Tyler stated at the AFRA-NCME symposium In Chicago, Illinois, in February of 1966, in a presentation of the plans for the National Assessment of Educational Progress: Because education has become the servant of all our purposes, its effectiveness is of general public Michigan State Board of Education, A Position State­ ment on Educational Accountability (Lansing, Michigan, 19721, pp. 4-7. 7 concern. The educational tasks now faced require many more resources than have thus far been avail* able, and they must be wisely used to produce maximum results. To make these decisions, dependable Infor­ mation about the progress of education Is essential, otherwise we scatter our efforts too widely and fall to achieve our goals. Yet we do not now have the necessary comprehensive and dependable data. We have reports on numbers of schools, buildings, teachers, and pupils, and about the moneys expended, but we do not nave sound and adequate Information on educational results. Because dependable data are not available, personal views, distorted reports, and journalistic Impressions are the sources of public opinion and the schools are frequently attacked and frequently defended without having necessary evidence to support either claim. This situation will be corrected only by a careful, consistent effort to obtain valla data to provide sound evi­ dence about the progress of American education.' With the preceding information In mind, the need for a preliminary study of certain cognitive music achievement of Instrumental music students In Michigan seems apparent. Purpose of the Study The purpose of this study Is to assess, through the use of a music achievement test, the cognitive musical knowledge of tenth, eleventh and twelfth grade instrumental music students in the State of Michigan. A secondary purpose is to make comparisons of scores on the basis of eight nominal category variables: 1. Grade in school 2. School classification 3. District band and orchestra festival participation 7 Ralph W. Tyler, "The Objectives and Plans for a National Assessment of Educational Progress," Journal of Educational Measurement. Volume 3, No. 1 (Spring, I9bb) , p. 7~. 4. District bahd and orchestra festival rating 5* Instrument family 6. Private lesions 7. Previous patent participation in a school band or orchestra 8. Music theory course This study has been undertaken with the expectancy that the results will help dramatise the need for greater emphasis on cognitive music achievement in Instrumental music instruction. It is hoped that such emphasis will assist in causing a greater awareness on the part of Michigan instrumental music teachers of the need for a general musical knowledge by students in our bands and orchestras. Still another purpose of this study is to develop and pilot test a measure of cognitive music achievement. Since the test used In this research will be available for use following completion of the study, Individual Instrumental music instructors may wish to administer the test to their own students and to compare the results of their students with those found In this study. This should provide the instructor with a core of knowledge by which he may assess the achievement of his students and give impetus to future improvements and changes within the instrumental music curriculum of his school. 9 Hypotheees Although considerable time and effort was expended In developing and validating an Instrument for teatlng purposes, a major thrust of this study was pointed toward the comparison of scores on this test through the use of the eight variables. This list of eight variables was developed by the investigator through consultation with other instrumental music teachers In Michigan. Other variables could have been added to this list; however, machine scoring procedures for this study allowed a maxi­ mum of eight variables. Since the control group did not respond to the eight variables, a ninth possibility was added. The hypotheses (stated In null form) are: 1. There Is no significant difference In the cog­ nitive achievement of tenth, eleventh and twelfth grade Instrumental music students. 2. There Is no significant difference In the cog­ nitive music achievement of students In the various MSBOA school classifications. 3. There Is no significant difference In the cog­ nitive music achievement of those students who participate In MSBOA Band and Orchestra Festivals and those whose groups do not. 4. There Is no significant difference In the cog­ nitive music achievement of those students whose groups rate first division In MSBOA Band and Orchestra Festivals and those whose groups do not. 5. There Is no significant difference in the cog­ nitive music achievement of students who play instruments In the four separate families. 6. There Is no significant difference in the cog­ nitive music achievement of those students who have taken private lessons and those who have not. 10 7. There Is no significant difference in the cog­ nitive music achievement of those students whose parents, one or both, have participated in a school band or orchestra and those students whose parents have not. 8. There is no significant difference in the cog­ nitive music achievement of those students who have taken a formal music theory course in high school or In a suraner music camp and those who have not. As the study developed, a control group was added which consisted of randomly selected students from the schools tested who had had no secondary school Instrumental music experience. A target figure of ten per cent of the treatment group was not realized for the non-music group; however, the findings are reported within this document. Thus, a ninth hypothesis is added: 9. There is no significant difference in the cog­ nitive music achievement of those students who have taken Instrumental music in the secondary school and those who have not. Definition of Terms In order to clarify later statements, it is necessary to define the use of some terms. MSBOA--Michigan School Band and Orchestra Association is an organization of instrumental music teachers through­ out the State of Michigan. Membership is voluntary and is obtained by the teachers through the office of the Managing Secretary of MSBOA. INSTRUMENTAL MUSIC STUDENTS IN MICHIGAN— Tenth, eleventh, and twelfth grade students who are members of a band or orchestra in a school which is a member of MSBOA. 11 COGNITIVE MUSICAL KNOWLEDGE— Specific facts that a student has accumulated, and can recall, about music such as: definition of terms, the number of flats or sharps in a specific key signature, Interval recognition and music reading. This also Includes the ability of aural perception which enables a student to make decisions on the basis of acquired knowledge relative to sound stimuli such as: determining Intonation errors of several musical pitches, determining rhythmic errors from a printed score and making other musical decisions which require response to a visual and/or an aural musical stimuli. GRADE IN SCHOOL*-Tenth, eleventh or twelfth grade students in Michigan high schools. SCHOOL CLASSIFICATION— Classification by MSBOA standards is determined by total high school enrollment. These figures are: Class Class Class Class Class AA A B C D 1400 or more students 850*1399 students 500*849 students 300*499 students 299 students or less DISTRICT BAND AND ORCHESTRA FESTIVAL PARTICIPATION— For purposes of this study, participation Is determined if a school has participated in a MS BOA sponsored Band and Orchestra Festival in two of the past three years. DISTRICT BAND AND ORCHESTRA FESTIVAL RATING— The average rating of the group in MSB0A sponsored Band and Orchestra Festivals in the past three years. 12 INSTRUMENT FAMILY--Either brass, percussion, string or woodwind Instruments used In Michigan MSBOA sponsored festivals. PRIVATE LESSONS--Those students who have taken ten or more private lessons on a band or orchestral Instrument In the past five years. PARENT PARTICIPATION--Students whose parents, either one or both, have previously played In a school band or orchestra. MUSIC THEORY COURSE--A course of study which Includes music terminology, structure and symbology which is a part of a high school curriculum or a summer music camp program. TREATMENT GROUP--A population of 2563 tenth, eleventh and twelfth grade students enrolled In a band or orchestra in a randomly selected Michigan school who took the test. CONTROL GROUP--A population of 141 tenth, eleventh and twelfth grade students from some of the above schools who have not taken band or orchestra In their secondary school experience. Limitations of the Study There are no known data at present, which states how many senior high schools in Michigan Include either a band or orchestra In their curriculum. no known data which Indicate Similarly, there is how many Michigan schools that do include these subjects do not belong to MSBOA. 13 Empirical evidence does Indicate that the vast majority of schools that have a band or orchestra in their curric­ ulum are members of MSBOA. There were S3S such schools in 1972. Further statements in this study which may refer to instrumental music students will refer only to those whose schools are members of MSBOA. Even though some high schools are organized other than on a 10-12 system, it was decided that this study would use only sophomores, juniors and seniors who were enrolled in a band or orchestra at the time of the test. No attempt was made to test those students who had been members of the school group previ­ ously but had not continued in band or orchestra. Since the testing of fifty-one schools took place over a span of eighteen days, May 8-25, 1972, it was impossible for one person to administer all of the tests. Seven college band members were chosen and trained to perform this task In such a manner that conditions during the testing period would be as similar as possible. Even though strong efforts were made toward this end, it was impossible to duplicate all testing situations in fiftyone separate schools by seven different test administrators. Part of the original goal was to obtain a control, or non-music, group of ten per cent of the number tested in each school. Plans were made for each test administrator to randomly select the control group from the school office records. Any student who had participated in instrumental music in secondary school would be rejected as a member 14 of the control group. In actual practice the test admin­ istrators were able to obtain a control group in thirty-two of the fifty-one schools. Since the school administrators in some schools did not allow the selection of control group students from the permanent records, the selection process was not random. As an example, In one school, the principal selected the control group from the study hall. The total number of students in this non-music group did not attain the initial size desired and finally became slightly over five per cent of the total tested rather than the ten per cent anticipated. It will not be within the scope of this study to answer the question of what constitutes music achievement. It is assumed that music achievement is a construct made up of a number of observables and that this study will be concerned with some of these observables. Although the test Instrument was made up of both an aural and a visual part no comparisons, other than statewide means, were made of achievement in relation to each separate part. Many educators agree that aural perception is affected, to varying degrees, by aptitude more than by achievement; however, it is not the Intention of the investigator to become Involved in this controversy. For purposes of this study the test was treated as a whole. Scope of the Study This study has grown in size since the inception of the original project due to the interest and support of 15 two professional educational organisations: Michigan School Band and Orchestra Association and Michigan Education Association. Since both groups supported the project with grants-in-aid and MSBOA offered the use of facilities in its Managing Secretary's office. It was possible to test a larger number of students. Those tested comprised approximately nine per cent of the total tenth, eleventh and twelfth grade instrumental music student population in Michigan during the 1971-1972 school year. Thus, this large N and the random selection process have provided results which can be generalised on a statewide basis. Overview of the Thesis In the following chapter literature and research relating to this study will be reviewed. This will include an historical perspective of achievement testing, four standardised music achievement tests now in use--their description and construction, and a review of some dis­ sertations which have used achievement tests in their studies. In Chapter III the design of the study will be dis­ cussed. This will Include the development of the testing instrument, validation and reliability procedures, plus the test administration. An analysis of the data is presented in Chapter IV with the hypotheses presented in similar order as in this 16 chapter. An Interpretation of the data and the hypotheses plus the inclusion of supplementary data will conclude this chapter. The fifth and final chapter contains a sumnary and conclusions. Implications for further research, based upon findings of this study, are presented. The appendix Includes the measuring device, forms used in test development and administration, item analysis, and other pertinent material used in this study. CHAPTER II REVIEW OF LITERATURE Achievement Testing--An Historical Perspective The adage, "If you wish to know something about a person, just ask him," can be applied not only to personal facts but also to Indications of an individual's Interests, expectations, aptitudes and achievements. It becomes obvious, by surveying the literature and through personal observation, that there is extensive use of achievement testing in our present-day society. Likewise, a surge of achievement measuring was evident in this country in the beginning third of the twentieth century. However, the search for the origin of the achievement test points to a much earlier date. Ebel reports that an extensive system of written examinations of educational achievement existed In China as early as 2357 B.C. under Emperor Shun. It provided a basis for admission to and promotion In the ancient Chinese civil service and was a part of the merit system which rewarded effort for achievement. This system, which gave the lowliest subject the opportunity to become one of the ruling elite, existed until this century. It was at least partly responsible for maintaining a 17 18 society with a high degree of Internal stability as well as a relatively high level of culture for over two mlllenla.^ At a much later date the Society of Jesus, founded In 1540, placed a high value upon educational achievement. Rather than utilising the oral examination method common to that period, they developed a detailed set of rules for the conduct of written examinations which "...apart from the fact that it is in Latin could be used in an examination room today. The Jesuit influence on testing did not find universal acceptance, for In the middle of the nineteenth century Horace Mann became Involved in a controversy over oral versus written examinations. As secretary of the Massachusetts Board of Education, Mann recognised the need for extensive Improvement in education. He was troubled over the use of oral examinations because they often degenerated into sessions In which unhappy students chose the occasion to settle differences with the schoolmaster. Mann felt the need for more objective evidence of pupil achievements and listed the following advantages of written examinations: g Robert L. Ebel, Essentials of Educational Measurement (Englewood Cliffs, New Jersey: F1rent Ice-Ha II, Inc., IWzj , p. 5. Q James Petch, "Examinations." Encyclopedia Brittanies. 1960, VIII, 937. ------ -------------- 19 1. More evidence concerning the achievement of each pupil could be obtained. 2. A written report of those achievements would be produced. 3. Each pupil would be asked the same questions; thus, all would be treated alike. 4. There would be less possibility of favoritism for, or bias against, particular pupils or teachers.10 Through Mann's influence the oral examinations began to disappear and be replaced by written examinations. Today, except for the oral examinations given by some colleges to graduate students upon completion of their work, the written examination is used for achievement testing. Ebel regards Edward L. Thorndike as the "father of modem educational measurements."^ Thorndike was the author of the first textbook in educational measurement which was published in 1903. Evidently the book fulfilled a need in the field but was not sufficiently understood by students, for Thorndike states in the preface to his second edition, "In fact, an elementary introduction to the theory of mental measurements, treating the simpler general problem in the logic of quantitative thinking, is needed now more than ever."12 ^Ebel, Essentials of Educational Measurement, p. 7. U Ibid., p. 11. *^Edward L. Thorndike, An Introduction to the Theory of Mental and Social Measurements. Aid ed. (New York; Teachers college, Columbia University, 1913), p. vii. 20 In this book. little attention Is given to the philosophy and theory of measurement; rather, elementary statistical and computation aids are offered to the student. Discussion centers about such basic measures as central tendency, distribution, variability, frequencies, relationships and correlations, plus the use of tables, graphs and formulas. By present standards the book would be of limited value but, for its time, the book's contribution to educational measurement was far-reaching. The work of Thorndike was soon followed by other attempts to measure systematically the outcomes of various subjects in the school's curriculum: Stone's arithmetic test in 1908, Courtis's arithmetic computation test In 1909, Thorndike's handwriting scale in 1910, the Ayres Handwriting Scale In 1911 and the Hillegas Composition scale in 1912.^ The first state-wide achievement testing program was initiated by New York in 1865 and was known as the Regents examinations.^ Although the early Regents examinations were designed primarily as screening tests for college admission, in later years the tests became broader eval­ uation instruments. Another achievement test series given on a state-wide basis was the Iowa Every Pupil Teats, ^^Henry Chauncey and John E. Dobbin, Testing. Its Place in Education Today (New York: Harper and Row, i9o j >, p . 1 2 . 14Ebel, Essentials of Educational Measurement, p. 8. 21 begun at the University of Iowa in 1929.*^ These tests began as an academic contest in which local winners moved on to regional and finally to state contests. Often high-scoring students in the state contest were awarded scholarships for their achievement. Testing of even larger groups of subjects took place during World War I. When the United States entered the war, the Army requested the American Psychological Association to help devise a method for classifying recruits according to their mental ability. Two tests were developed and given to over two million soldiers: the "Army Alpha Test" and the "Army Beta Test" ( a test designed for illiterates which utilised drawings and non-verbal materials). The "Army Alpha Test" was a good test instrument to choose promising officer candidates, to assist in the assignment of recruits with different Intellectual skills, and to reject those who lacked sufficient mental ability for the service. The use of achievement, aptitude and psychological tests for large groups has been conmonplace in the twentieth century. In recent years colleges have shown great interest in 3tudents who score high marks on such tests as the Scholastic Aptitude Test, the National Merit Scholarship Qualifying Test and others. Project TALENT, 15Ibid., p. 16. ^Chauncey and Dobbin, Testing Today, p. 7. 22 a program Intended to provide nationwide data on the aptitudes, achievements, backgrourds and career plans of secondary school students, was supported by the United States Office of Education, and tests with accompanying surveys were administered to approximately one-half 17 million students in March, 1960. One of the most recent developments in large group achievement testing is the National Assessment of Educa­ tional Progress. The development of NAEP was brought about when a number of individuals concerned about educa­ tion were searching for comprehensive and dependable data to use in answering questions regarding the progress of education in this country. After numerous discussions and conferences, John W. Gardner, President of Carnegie Corporation, asked a number of individuals to form a committee, known as the Exploratory Committee on Assessing the Progress of Education, chaired by Ralph Tyler. This committee was given three charges: (1) to explore ways In which a national assessment of educational progress might be structured in order to provide needed information without harming present educa­ tional efforts, (2) to develop methods and procedures 17 Ebel, Essentials of Educational Measurement, p. 24. 23 to secure such information, and (3) to suggest the makeup of a group which would implement such a plan. 18 A large number of professionals and laymen was Involved in identifying important educational objectives In the ten subject areas to be assessed: literature, science, social studies, writing, citizenship, music, mathematics, reading, art and vocational education. In selecting the objectives, three criteria were used: (1) the objectives should be those which scholars in the field feel are worth­ while, (2) the objectives should be those which schools are currently attempting to attain, and (3) the objectives should be those which thoughtful laymen consider important for youth to learn. iq The plan was to assess approximately 130,000 Individuals using the following subdivisions: male and female, race, two Bocio -economic levels, four age groups (9, 13, 17 and young adults between the ages of 26 and 35), four geographic locations, and four different types of communities.20 18J. C. Merwin and F. B. Womer, "Evaluation in Assessing the Progress of Education to Provide Bases of Public Under­ standing and Public Policy," Educational Evaluation: New Roles. New Means. The Sixty-eighth Yearbook ot the National Society tor the Study of Education, Part II (Chicago, Illinois: The University of Chicago Press, 1969), pp. 305334, 19 Carolyn Hightower, How Much Are Students Learning?. Report from the Comnittee on Assessing the Progress of Education (Ann Arbor, Michigan: November, 196s), p. 7. 20Ibid., p. 18. 24 The Assessment began in 1969 with the testing of three subjects--science, citizenship and writing. The remaining seven subjects consisting of literature, social studies, music, mathematics, reading, art and vocational education were tested In the following two years. It is planned to continue the cycle so that constant evaluation of educational achievement may take place and results may constantly be updated. Published Music Achievement Tests It is obvious that the number of music achievement tests is constantly changing with new tests being developed while others fade in use and are dropped from print. Investigation by the writer Indicates a need for more testing Instruments inasmuch as it is often difficult to find more than one published achievement test avail­ able for a specific purpose, such as cognitive music achievement on the secondary level. Elementary Music Achievement Test Richard Colwell has developed n series of four music achievement tests for grades 3-12 which are Intended to measure certain basic skills in aural perception. 21 These tests have been developed over a period of four years and require some two hours and twenty-five minutes for the entire battery. 21 They have been constructed so that Richard Colwell, Music Achievement Tests (Chicago: Follett Publishing Company, l9b!>) . 25 a teacher may give all parts at the same tine or nay administer then individually at the teacher’s convenience. Content validity was established by the use of objectives stated or implied in textbooks and curricula along with consultations with leading educators. Colwell does not attempt to establish concurrent validity between student scores and teacher ratings because he calls teacher ratings 'Victoriously unreliable.1122 Reliability of the MAT is generally high (between .85 and .91) with the reliability of the subtests being somewhat lower. All tests utilize a 33-1/3 rpm record with aural stimuli provided by piano, violin, viola and cello. The following areas are covered within the battery of tests: pitch, Interval, meter and mode discrimination; feeling for tonal center; pitch and rhythmic auditory-visual discrimination; tonal memory; melody, pitch and instrument recognition; musical style; auditory-visual discrimination; chord and cadence recognition. In the opinion of this writer the MAT is the best available comprehensive music achievement teat for use within grades 3-12. In the Buros Seventh Mental Measurement Yearbook Lehman suggests that the tests are useful for: 1. assessing the progress toward meeting the curriculum objectives 2. program evaluation 22 Richard Colwell, "The Development of the Achievement Test Series." Council for Research in Music Education. No. 22 (Fall, 197077V. 59.-------- ----------------- 26 3. student guidance 4. group diagnostic work 5. Identifying student s.who may profit from Instru­ mental instruction. Watkins-Famum Performance Scale A contrasting test for Instrumental students is the Watkins-Farnuro Performance Scale 24 for it measures musical achievement objectively through individual perform­ ance. The test is not designed for a specific grade level but rather it measures a student's performance level on a band Instrument. The scoring is accomplished in the following manner: (1) the examinee plays the exercises from the test booklet, (2) the examiner marks the errors within measures, (3) the final score is determined by the number of correctly played measures. Although the reported reliability is high, it is somewhat open to question since it involves a personal decision on the part of the person scoring the test. Validity coefficients, based upon teacher ratings with a small number of subjects, vary from .68 to .87. There are several disadvantages with the test: 1. What constitutes an error is an individual 23Oscar Krlsen Buros. Mental Measurements Yearbook. Seventh Edition (Highland Park, New Jersey: Press, 1972), p. 248. The Gryphon 24John G. Watkins and Stanley E. Famum, WatklnsFamum Performance Scale (Winona. Minn.: Hal Leonard” Music, 1954).---------- 27 judgment despite the attempt by the authors to give clear Instructions. 2. The test requires large segments of time for it must be administered individually. 3. Since the statistical data Is incomplete, published norms are unavailable. The test has been developed from one objective: to be able to determine different levels of achievement through performance. It is a serious attempt to solve the age-old problem of objectively measuring musical performance, ■which is highly subjective In nature. Allferis Music Achievement Test The Allferis Music Achievement Test 25 measures students' ability to make auditory-visual discriminations of melodic, harmonic and rhythmic examples at the college entrance level or at the end of high school. While many other tests require several class periods for administration, the Allferis Test can be administered in forty minutes. The test is entirely aural Inasmuch as the stimuli are provided by piano and have been prepared on tape. The test items were devloped from a search of freshman theory and harmony books plus consultation with music instructors. The materials are separated according to melody, harmony, and rhythm. On this basis, Allferis 25 James Allferis, Allferis Music Achievement Test (Minneapolis: University ot Minnesota press, 19^4.). 28 claims content validity. Predictive validity is obtained by correlating test scores to college course grades earned during four years. Generally high coefficients prevail with the highest correlation shown In music theory grades (.66) and the lowest In performing organisation grade. Reliability coefficients vary from .67 to .84 with the combined figure of .88 for all three test sections. Each of the three elements of music (melody, harmony and rhythm) is presented as an element and as an Idiom. The element Is presented to depict the smallest unit out of musical context. The Idiom presents each of the elements in a context by grouping two or three elements so that the example is closer to a practical musical experience. The rhythmic section of the test contains examples which have Included melodies to the rhythms. The author states that these were added so as to avoid "the tapping of rhythms, a procedure In conflict with the declared objective of creating musical test Items."26 Extensive norms are given In the test manual for various grades, types of Institutions and different geo­ graphic regions. Such data offers teachers, counselors, administrators and students the opportunity to make numerous comparisons of scores, locally, regionally, Instit­ utionally and nationally on the basis of the Allferis Test. 26 James Allferis and James E. Stechie in, **The Development of a College Entrance Test in Music Achievement," Journal of Research in Music Education. No. 2 (Washington, b.C.: Music 'Educators National Conference, 1953), p. 88. 29 Snyder Knuth Music Achievement Teat The Snyder Knuth Music Achievement Test was developed from a Doctoral Thesis, '‘The Development, Construction and Standardization of a Test of Musical Achievement" by Alice M. Snyder. 27 The purpose of the study was to develop an instrument which would measure general college elementary educations students' acquired knowledge and skill In certain aspects of elementary school music. It was designed as a test to screen those students who are knowledgeable in basic music fundamentals. An arbitrary cut-off point (at the 55th percentile) is suggested for those students to be enrolled In an elementary music methods course. Test items were constructed from material derived by analyzing the content of several elementary music series textbooks. Equivalent forms of the test were prepared, each containing four parts: (1) Listening and Seeing, (2) Listening, (3) Analogies or Music Comprehension, and (4) Recognition of Familiar Melodies. There Is a question as to the equality of the equivalent forms, for the mean score for Form A is 75.5 while the mean score for Form B is 82.5. The piano provides the musical stimulus and the entire test, including directions, is tape recorded. A filmstrip which is required for test administration presents 27 Alice M. Snyder, "The Development and Standardization of a Test of Music Achievement" (Unpublished Ph.D. Disser­ tation, University of Oregon, 1958). 30 the potential problem of coordinating the tape and film­ strip. The need of a darkened room may likewise be a hindrance In test administration. A pre-pilot test was administered to Dr. Snyder's classes. She utilized the piano as the sound stimuli for items while students listened, watched notation In the test booklets and marked the answer sheets. Content validity Is claimed by the test constructor and, although no statistical evidence is available, the claim appears justified. Reliability is reported as extremely high (.99+). Both Gordon and Colwell state that an obvious error was made in computation; 28 therefore, no accurate reliability estimate is available. Norms were derived from 311 elementary education majors at San Francisco State College. Certainly this is a small number of subjects on which to base norms. Both Slagle and Garder report mean scores from nine to sixteen points lower than Snyder. 29 * As is true with so many achievement tests, the test has strengths along with weaknesses, but It is the only one which measures achievement for this specific purpose. It Is evident that additional tests of music achievement are needed to establish outcomes in a variety of areas. 28 Buros, Mental Measurements Yearbook, pp. 250-251. 29Ibld., p. 251. 31 Studies Involving Music Achievement Tests Colwell Colwell reports a study conducted In the Sioux Falls, South Dakota, Public Schools in which he compared music achievement among approximately 4000 vocal and instrumental students In grades five through twelve. No hypotheses were used In the study, for the purpose was to evaluate objectively the skill of audltory-vlsual discrimination achievement In students. The study utilized the Allferis Music Achievement Test, the Famum Music Notation Test and the Knuth Achievement Tests as measuring Instruments. 30 Results consistently showed that students whose back­ ground Included a variety of musical experiences were superior In achievement, even when the amount of time spent was equal. Instrumental students were superior except that vocal students with piano training were superior to instrumental students without piano training, and instrumental students with piano training scored highest of all. It must be noted, however, that the Intelligence quotients of the above groups corresponded directly to the achievement levels; therefore, we must raise the question of whether training or Intelligence produced the greatest effect upon achievement. The author of the study did not address himself to this point. Another factor which may ^Richard Colwell, "An Investigation of Musical Achievement Among Vocal Students. Vocal-Instrumental Stu­ dents and Instrumental Students, Journal of Research in Music Education. Volume 11 Fall,19t>3, pp. 1^3-130. 32 have affected the results of this study Is the fact that since the instrumental music was entirely elective, it may have attracted the better students. Some interesting supplementary information was pro­ vided through the study. It was found that the amount of individual practice was not a factor in achievement as measured by the Knuth Tests. Further Information indicated that at the end of the summer months students had retained almost all of the music knowledge acquired the previous school year. In the high school level little or no improvement was shown on the test as a result of an additional year's training in music. Instrumentalists gained only slightly while vocal students actually declined. Although not stated specifically by Colwell, the most significant factor in achievement seemed to be private piano lessons taken by students in all levels. Finally, Colwell concluded that Kwalwasser's statement that "training attracts the talented" Is true and that musical aptitude is not so important as other factors in predicting musical achievement. Iltis A somewhat different achievement test, constructed by John Iltis, 31 measured the ability of high school 31 John L. Iltis, "The Construction and Validation of a Test to Measure the Ability of High School Students to Evaluate Musical Performance (Unpublished Mus. Ed.D. Dissertation, Indiana University, 1970). 33 students to evaluate musical performance from short taped examples of wind ensemble literature which contained performance errors. Students were asked to make Judgments of the example within five categories: Intonation, tone quality, interpretation, ensemble and technique. Similar responses were made by professional musicians and teachers and the results were compared. Several problems were evidenced in the development of this test. The use of subjective words or phrases to describe a musical performance, or even a single tone, produced differences of opinion and some confusion. Item difficulty was a problem, although Improvement seemed to be apparent as the test was developed. Furthermore, the attempt to provide three levels of adjudication pro­ vided for greater lack of objectivity. Dr. Iltis found that preparing taped examples of purposeful errors by trained musicians is extremely taxing and difficult. A questionnaire provided Iltis with Information regarding individual students and the opportunity to hypoth­ esize from these reponses. Over a period of time 1637 students took either a part of or the entire test. On the basis of this study the following conclusions were reached: 1. There is a positive relationship between amounts of performance experience and the ability to evaluate musical performance. 2. Some types of instrumentalists are better able 34 to evaluate musical performance than others. In this case, double reed players excelled. 3. Students with broader performance backgrounds are more sensitive to performance evaluation. 4. Students with more education are more sensitive to performance evaluation. 5. There Is a positive relationship between the number of years of private music study and the ability to evaluate performance. 6. The number of years of Instrumental experience is positively related to the ability to evaluate performance. This Investigator, on the basis of data from this study, has formed conclusions similar to points 2, 4, and 5 above. Swlnchoski If an achievement test is to be a good test which is usable, it must be both valid and reliable. The Swlnchoskl Standardized Music Achievement Test Battery for the Intermediate Grades 32 is an example of a test which is valid but contains low reliability. Extensive effort was made in securing the content of music education progams through reconmendations of leading authorities In the field, through the investigation of a number of textbooks in use, and through a survey of some 135 courses of study. 32 Albert A. Swlnchoskl, "The Development of a Standard­ ized Music Achievement Test for the Intermediate Grades" (Unpublished Ph.D. Dissertation, University of Kansas, 1963). 35 The test was designed with four subtests: rhythmic activities, listening activities, creative activities and music reading; with each subtest containing twenty to twenty-two multiple-choice items for fourth, fifth and sixth grade students. All items are presented aurally by tape and four regular class periods are required for administration of the test. Two of the disadvantages of testing on four separate days are the possibilities of varied reaction to testing and fluctuation in the size of the population. A stratified random selection of schools in the midwestern and western states provided twenty schools for the test. Since all materials required mailing and all tests were administered by the classroom teacher, the test constructor lost control of the testing situation. The study also included a survey of teacher preparation in music and the amount of classroom time devoted to music. Coefficients of reliability were low for individual subtests and ranged from .23 to .48. Combined coefficients of the four tests produced somewhat higher figures: .59 for fourth grades, .67 for fifth grades and .74 for sixth grades. This would indicate the possibility that the teat was too difficult for younger students, yet the mean item difficulty Index was 38.3 which would Indicate difficulty that was too low to provide for successful discrimination. Evidence points to the fact that, had the test constructor spent more time using item analysis 36 techniques and redesigning test items, the final result would have been a test which contained higher reliability. Items were not functioning properly to discriminate consistently the high students from the low. Alice Snyder, in her doctoral thesis, Indicates the need for a proper balance of analyzatlon of data plus observation as she states, "This experience proved to the writer the futility of trying to judge the difficulty of an item on its face value without test results to balance one's personal judgment. Mansur Paul Mansur was intrigued by the amount of time required to audition instrumental students in performing groups. He set out to develop an objective measurement which would make a valid selection using only a fraction of the time normally involved for the person holding the auditions. 34 As the basis for the study he used students chosen for the Oklahoma All State Band versus those who auditioned and were not chosen. A pilot test of fifty multiple-choice items was developed covering four factors which Mansur felt were contributors to success in musical performance achievement: (1) knowledge and understanding of musical terms, (2) experience Snyder, 'Music Achievement Test," pp. 43-44. A J Paul M. Mansur, "An Objective Performance-Related Music Achievement Test" (Unpublished Ph.D. Dissertation, University of Oklahoma, 1965). 37 and familiarity with composers, (3) experience and familiarity with music literature, and (4) the ability to analyse musical notation. An attempt to correlate scores with audition ratings was unsuccessful, as some students with relatively low test scores received excel­ lent audition ratings while some who received high test scores received low audition ratings. It is interesting to note in this test and all succeeding tests that either the percussionists were excused from the test or all their test papers were rejected. Following the unsuccessful pilot test a new test was developed utilizing printed musical examples and requiring students to make judgments about tempo, articulation, dynamics, phrasing, style, rhythm or tonality. In addition, four unmusical situations were devised and presented. All test items required that the students accept or reject a choice. This new test was given to the Oklahoma All State Band with some Improvement in reliability. Attempts to establish validity were unsuccessful due to a lack of responses by mall. A validity coefficient of .64 is reported but it must be regarded as spurious as only sixty per cent of the validating population responded. A final revision was called the 'Vflnd Instrument Inventory Scale" and consisted of eighty items which required twenty minutes to complete. Once again, It was administered to those auditioning for the Oklahoma All State Band. There were several contaminating factors in the adminis­ tration of the test, the most serious being the fact that 38 forty-nine students had taken the test the previous year. However, due to the Improvements In the test, the study operated more efficiently as Mansur was able to reject the null hypothesis on the basis of the results. It stated that there Is no significant difference between the means of a performance-related achievement test administered to students nominated and accepted for member­ ship In the Oklahoma All State Band and to students nominated but not accepted for membership. Although the "Inventory Scale" lacks validity and contains low reliability, it Is a serious attempt to develop an objective measurement for audition purposes. The study certainly Indicates that more research would be worthwhile in this area; however, the possibility of evaluating the psycho­ motor domain by testing cognitive judgment is questionable. Folstrom Considerable controversy has arisen In recent years over the value of the study of music in high school per­ formance groups. Critics that the general of performance groups declare levelof musicianship Is relatively low because rehearsals often lead only to ''playing or slnglxig notes" rather than to a thorough understanding of the composition. Supporters of performance groups contend that students involved in these organisations do, in fact, attain a certain levelof achievement and that these groups are the most successful generators of music training in our entire music curricula. Roger Folstrom addressed 39 himself to this controversy In "A Comparative Study of the Musical Achievement of Students in Three Illinois High Schools."35 In the study he evaluated the degree of musical achievement of performance groups over a nine month period; compared these groups with control groups (randomly selected groups of non-music students); compared the results between different schools Involved In the test; and, finally, compared differences in the areas of melody, harmony and rhythm. He chose the Aliferls Music Achievement Test: College Entrance Level as the testing Instrument for the following reasons: (1) validity, (2) reliability, and (3) adaptability to the testing situation. The testing procedure involved administering the test to both treatment (students in performing groups) and control groups during the opening weeks of school and again to the same students during the closing weeks of school. A total of 496 students from three high schools was Involved in the test. The results of this study indicated that students from performance groups scored significantly better than students not involved in any school band, orchestra or choir. This indicates that a learning situation existed and, in addition, all treatment groups showed a gain over a nine-month period. It is interesting to note that the 35Roger J. Folstrom, "A Comparative Study of the Musical Achievement of Students in Three Illinois High Schools" (Unpublished Ph.D. Dissertation, Northwestern University, 1968). 40 greatest difference In means, of all the schools, existed In the area of rhythm. Results further Indicated that musical achievement differed significantly between schools. Other factors which showed a possible effect upon musical achievement were soclo-cultural level of the community, student grade level, and student Interest In out-of-school musical participation. The most serious problem In this study was participation. Initially, approximately 2000 students were to be Included In the study; however, due to a number of circumstances, the final N was 496. Sumnary The previously mentioned studies are but a part of the process of evaluating musical achievement. Even though much time and effort have been devoted toward this end, greater endeavor Is needed In the future. However, our greatest effort must be placed In a slightly different area--the application of knowledge acquired through research. The following comments by Robert Ebel are appropriate as a conclusion to the survey of literature In this study: What can we learn, apart from the facts, by viewing this perspective on the history of measurement In education? Perhaps that the problems of educational measurement are persistently perennial--that the problems of what to measure and how to measure It, of objectivity, reliability, validity, and efficiency that bothered our predecessors still bother us. Perhaps that those predecessors were remarkably In­ ventive and resourceful In discovering some solutions to those problems. Perhaps that we owe a great debt to the dynamic Innovators who saw a need, and who then comnltted enough time, effort, persistence, and 41 skill to create the tools end establish the agencies that could satisfy the need. Perhaps that our technical skill In developing tools, procedures, and agencies has been greater than our pedagogical skill in getting teachers to make effective use of our the available technology. If teachers could only be taught and persuaded to use well the technology of educational measurement that has already been developed, there could be a quiet revolution in the quality of our testing and teaching almost overnight.36 36 Gbel, Essentials of Educational Measurement, p. 27. CHAPTER III DESIGN AND ADMINISTRATION OF THE STUDY Development of the Testing Instrument Planning the assessment of the population of tenth, eleventh, and twelfth grade instrumental music students in the State of Michigan provided the investigator with a number of problems: 1. If the test were to be statewide, what would be the mechanics of the test? 2. How many schools and students would be Involved? 3. What Instrument would be used In the testing? 4. What Information would be measured by the test? Fortunately, the president of the Michigan School Band and Orchestra Association became Interested in the project and asked the author to present a preliminary proposal to the executive board of that organisation. The Board decided to supply printing, secretarial and mailing services for the project. Furthermore, the Board requested and received from the Michigan Education Association a sisable research grant which was applied in support of the statewide test. With such support It was evident that the scope of the project could be widened so that a fairly large sample of the population of instrumental A2 43 music students could be measured and, using a randomised selection technique, it would be possible to generalise on a statewide basis. One of the first decisions to be made was the selection of an achievement testing Instrument. First and foremost, the test would have to be valid for measuring Michigan high school instrumental music students. The second consideration was that the test would have to be the right length. Since It would be administered to many schools throughout the state, it seemed impractical to utilise more than one class period for administering the test. A greater degree of cooperation could be expected from the school administrators and music directors if the test could be given during a single band or orchestra period and would not interrupt the normal operation of the school day. A test which would fit the above criteria was un­ available; therefore, it seemed advantageous to develop a test which would not exceed forty-five minutes in length. Such a test would be less reliable than one of twice the length, but some reliability was sacrificed for greater participation from the selected schools. The final test, which was administered to fifty-one schools, was designed for greater usability at a negligible sacrifice of reli­ ability. A music achievement test which can be administered in less than sixty minutes provides the constructor with a dilemma: Should the test be a cursory examination of many musical factors or should it examine a small number 44 of factors In depth? An in-depth examination of a few factors will likely provide greater reliability, but the question remains whether music achievement measurement can be based upon a small number of factors. Conversely, a test which contains a few items on each of many factors will likely result in lower reliability but should provide the examiner with more knowledge relative to the total musical achievement of the examinee. It was decided to adopt the second procedure. Test Construction. Pilot Test No. 1 Tests are built upon objectives, curriculum objectives, objectives from textbooks or from consultation with wellknown educators within the field. Gronlund speaks of test construction and objectives when he states: The first order of business, then, In constructing an achievement test Is not the construction of test items, but rather the Identification and definition of learning outcomes to be measured. These should logically grow out of the instructional objectives of the course In which the test Is to be used. The sequence of steps for determining outcomes Is as follows: (a) Identify the Instructional objectives of the course. (b) State the objectives In terms of general learning outcomes. (c) Under each objective list the specific learn­ ing outcomes you are willing to accept as evidence of the attainment of that objective.37 In many cases, schools do not have specific objectives stated for all courses, music curricula differ from one 37 Norman E. Gronlund, Constructing Achievement Tests (Englewood Cliffs, N.J.: Prentice-hall, Inc. 1^58), p . 3. 45 school to another and some directors do not use specific textbooks In their bands and orchestras*-how does one construct an achievement test for all these groups? The approach was to ask the music teachers to provide opinions as to the course content of band and orchestra classes which they felt to be Important. This information would provide a basis for the development of the test. Ebel confirms this direction when he suggests: Publishers of standardised tests of educational achievement and directors of wide-scale testing pro­ grams that use achievement tests face the special problem of how to make a single test suitable for students who have been taught by different teachers using different textbooks and learning materials in courses having different orientations. The usual solMtlon Is to base the tgst on elements thought to be roost commonly taught.**” Over a period of time this Investigator compiled a list of forty testable objectives which are possible outcomes of instruction In bond and orchestra classes In Michigan schools. The objectives were constructed on the basis of personal observation, experience, consultation with other instrumental music teachers and the content of the two MSBOA handbooks on music theory and music literature. This list was given to music directors at a spring MSBOA meeting with instructions to respond to each of the objectives according to a weighted scale. The directors were given the opportunity to make additions to the list. 38 Ebel. Essentials of Educational Measurement, pp. 57 -58 . ---------------------------------- 46 The results were tabulated and the top twenty objectives were used as a basis for writing test questions. See Appendix A for the list of these objectives and the weighted scale for each objective. At this point it became necessary to determine the type of test questions to be used in the test. Multiple- choice Items offered the best alternative for the follow­ ing reasons: 1. The effect of guessing by students is minimised. 2. Multiple-choice questions can be adapted to measure almost any educational learning. 3. If items are constructed carefully, there is less likelihood of ambiguity within items. 4. Multiple-choice items are easily scored. Ebel sums up the status of the multiple-choice test when he states: 'Multiple-choice test Items are currently the most highly regarded and widely used form of objective test item. 39 Cronbach reports that multiple-choice items offer the opportunity to test the ability of the student to generalize data.**® Item Construction A total of seventy Items was constructed by the investigator and constituted the first item pool. 39Ibid.. p. 87. 40Lee J. Cronbach, Essentials of Psychological Testing (New York: Harper & Brotners, 1^49), pp. i/b-2/y. 47 Music is an aural art; therefore, this author felt that any testing instrument involving music achievement should Include items which would require the student to make decisions based upon musical aural stimuli. The next problem was to develop some type of recording which would present the aural portion of the test effi­ ciently and correctly. It was decided to utilise a pre­ recorded tape because better fidelity of sound could be accomplished in the administration of the test using a portable tape machine rather than a portable disc machine. With the use of the recording studio of station WFBE in Flint, Including their high quality recording equipment and an excellent engineer, a tape could be pro­ duced which would be of the highest quality. Since the population to be tested was comprised of band and orchestra students the author decided to utilise instruments from these groups rather than electronic sound sources in the preparation of the tape. Although using electronic instruments would provide an easier method for constructing errors, utilisation of band and orchestra instruments would provide the medium in which students perform, thereby establishing one form of content validity for the test. It was this point, however, which caused great difficulty in the preparation of the tape. Trained musicians react on the basis of their preparation. This came to light in recording sessions, for: 1. Trained musicians find it very difficult to play planned errors. 48 2. When playing a planned error, trained musicians often change the quality or timbre of their tone, thereby causing a second and unwanted error. 3. Trained musicians find It difficult to sustain a satisfactory degree of planned Intonation error. 4. Trained musicians attempt to make decisions con­ cerning the degree of error which can and should be made only by the test developer in conjunction with the recording engineer. These points, plus the author's Inexperience in determining the degree of discrimination required for a good aural test item, caused many lengthy and tedious recording sessions. Unfortunately, correct judgments In tape preparation come only from actual recording experience and gaining such experience Is time-consuming. Administration of Pilot Test No. 1 The pilot test, consisting of twenty-four visual and twenty-six aural items, was constructed from a pool of questions which had been developed from the previously mentioned objectives. During July, 1971, it was administered to 230 students at the summer music camps at Ferris State College and Michigan State University. The students' responses were placed upon machine scored answer sheets (see Appendix F) , and were processed through Evaluation Services at Michigan State University. 49 Data Analysis. Pilot Test No. 1 An item analysis of Pilot Test No. 1 provided the following Information: Table 1.--Distribution of Item Difficulty and Discrimination Indices. Item Difficulty Indices 81-90 71-80 61-70 51-60 41-50 31-40 21-30 11-20 00-10 1 9 10 3 8 3 9 4 3 Mean Item Difficulty 47 Discrimination Indices Less than 61-70 51-60 41 -50 31-40 21-30 11-20 00-10 00 2 3 13 11 6 6 7 2 Mean Item Discrimination 31 Other information on which was provided by the item analysis included: Mean Score of Test Standard Deviation Kuder Richardson Reliability #20 Standard Error of Measurement N - 230 26.52 6.34 .77 3.04 In interpreting the above data, primary consideration was given to item difficulty, discrimination power and point blserial correlation. An indication of a target figure for the first two is given by Gronlund: Other things being equal, we should favor items at the fifty per cent level of difficulty and those with the highest discriminating power. However, the tentative nature of our data requires that we allow for a wide margin of error. If an item 50 provides a positive index of discrimination, if all of the atematlves are functioning effectively, and if the item measures an educationally significant outcome, it should be retained and placed in an Item file for future use.^1 Although the mean Item difficulty neared fifty, it was obvious that a number of items had difficulty indices which were too low while, at the opposite end, a number of items were high in difficulty. Two Items had both difficulty and discrimination Indices of less than seven. Five items contained a difficulty Index of above seventy while the discrimination index was ten or below. In addition, there were two items which had negative discrim­ ination indices. Since all of the above items were in the aural portion of the test, it was evident that the greatest concentration of effort in refinement should be applied in this area. See Appendix B for data relative to difficulty and discrimination for the complete test. The frequency distribution indicates a normal dis­ tribution of scores with approximately two-thirds of the scores falling one standard deviation on each side of the mean. The standard error of measurement is well within the expected error stated by Gronlund in the following table: 41Gronlund, Constructing Achievement Teats, p. 88. 51 Table 2.--Expected Error of the Standard Error of Measurement. Number of Items In the Test Standard Error 2 3 4 5 6 7 Less than 74 24-47 48-89 90-109 110-129 130-150 Source: Gronlund. ConstructinK Achievement Tests, p. 88. Test Construction. Pilot No. 2 Through personal observation and analyzation of the data, It became obvious that refinements were needed for Pilot Test No. 2. A number of ambiguities in the pre­ recorded tape seemed to provide Items with excessively high or low difficulty and too many Items which contained low discrimination Indices. In the first recording session the musicians were asked to replay musical examples which were to be heard twice on the tape. They found It virtually Impossible to play even the simplest passage exactly the same two tiroes. This problem was solved by recording many examples for each test item, selecting the roost valid example and dubbing the tape where more than one performance of the same example was required. The order of the test was changed from visual-aural to aural-visual thereby providing more continuity in that students who finished before the time limit on the 52 visual part would not have to wait for resumption of the test. Twenty minutes for completing the twenty-four visual items proved too long so the time limit was reduced to fifteen minutes. The test was also shortened from fifty to forty-six Items. The visual appearance of the test booklet was improved through better reproduction methods, more space between items, and clearer written examples on the music staves. Other refinements Included improved terminology, better dlstractors for some items, an attempt to vary the diffi­ culty of some items, and dropping some items while adding others. Administration of Pilot Test Wo. 2 It had been decided at an earlier date to make a random selection of schools in Michigan for the final t*>, »sts. Therefore, so as not to contaminate the selection, i.., was decided to try to obtain a population outside Michigan to test Pilot Test No. 2. A national American School Band Directors Association Convention provided the author with the opportunity to search for several schools to serve as the testable population for Pilot Test No. 2. The search was successful and North High School and South High School in Sheboygan, Wisconsin; Collinsville High School, Collinsville, Illinois; and Boone High School of Boone, Iowa were chosen to form the pilot population. 53 The new pre-recorded Cape, the restructured test of forty-six Items, the answer sheets plus Instructions for administration of the test were sent to the four schools in late fall of 1971. Pilot Test No. 2 was administered to 424 students by the four directors in December and all materials were returned to the author in January, 1972. Data Analysis. Pilot Test No. 2 The answer sheets were again processed by Evaluation Services at Michigan State University. An item analysis provided the following information: Table 3.--Distribution of Item Difficulty and Discrimination Indices. Item Difficulty Indices 81-90 71-80 61-70 51-60 41-50 31-40 21-30 11-20 00-10 4 10 7 9 6 4 3 2 1 Mean Item Difficulty 56 Discrimination Indices Less than 71-80 61-70 51-60 41-50 31-40 21-30 11-20 00-10 00 2 3 1 7 11 5 11 4 2 Mean Item Discrimination 31 Other information which was provided by the item analysis included: 54 Mean Score of Test Standard Deviation Kuder Richardson Reliability #20 Standard Error of Measurement N - 424 20.13 5.63 .72 3.00 The mean Item difficulty rose some nine points on this test while mean item discrimination remained the same. In general, the visual portion of the test functioned well Inasmuch as only one Item contained an obvious low discrim­ ination Index. See Appendix C for data relative to difficulty and discrimination for all Items on the teat. Again the aural portion provided the greatest concern, for there still remained two negative discriminators and four Items with an Index between 00 and 10. improvement was shown In the standard error. A slight A frequency distribution table (see Appendix C) reveals less variability on this test with a greater concentration of scores one S. D. from the mean. The well-informed personal observation la one of the roost valuable tools In the kitof the researcher. Van Dalen supports this view and states: Some theories are evolvedIn a less formal manner. Many Investigators believe that an undue, pre­ mature concern with ordering facts and structuring highly formalized theories may cause them to terminate their exploratory activities too soon and may blind them to other facts and ordering possibilities. To them, a theory Is a provisional tool They place less emphasis on elegant conceptualisations and logical-deductive procedures and more explicit emphasis upon observation and data-oriented explana­ tions. They believe that the interaction of 55 observational and conceptual processes Is necessary for scientific progress.43 The need for such Interaction became evident during the period of analysation of data on Pilot Test Mo. 2. The following points came under scrutiny during this period of observation and analyzatlon. 1. For the most part, the Items In the visual part of the test were functioning well as test Items. 2. The drop of mean score of almost six and onehalf points was attributed to the greater selectivity of students In Pilot Test No. 1 for they were members of summer music camps. Furthermore, both camps provided a required music theory course as a portion of the camp curriculum. This alone should provide a basis for a higher mean score. 3. It was determined that a number of items on the aural portion of the test were too difficult, Indicating further need for greater refinement In the pre-recorded tape. 4. Ten per cent of the testees omitted three or more test Items which would Indicate some problem in test administration. There was ^Deobold B. Van Dalen and William Meyer, Understanding Educational Research (New York. N. Y. : McGraw-Hill, T5b27, pp. 64-65. 56 little or no control over this fact since the Individual band directors administered the test to their own students. Test Construction. Pilot No. 3 With the above Information in mind Pilot Test No. 3 was redesigned. Another pre-recorded tape was constructed during a new recording session at the same studio and with the same musicians as for the previous pilot. Through experience In developing tapes the Investigator realised that the use of college or advanced high school musicians was superior to the use of professional musicians for recording purposes. First, their sound more closely approximated the sound common to the students to be tested. Second, they were more willing to allow the test constructor to initiate decisions regarding the construction of a tape with planned errors. The format of the test was again Improved with even more space between Items and with musical examples written by a professional copyist. Each section which dealt with a new factor was begun with a heading and the action portion of each Item was Included In the form of written Instructions so that the student would both hear the directions on the tape and read the directions on his test booklet. The test was typed by a professional typist and copies were printed at the University of Michigan Copy Center In Ann Arbor. The test booklet was now in Its final form and ready for Pilot Test No. 3. 57 Administration of Pilot Test No. 3 The problem of choosing a group to test for the final pilot was solved by making the statewide random selection at this time and choosing a school which was not selected for the actual test administration as the school for Pilot Test No, 3. This test was administered In March, 1972, to Flint Northwestern High School and the results once again provided the basis for an item analysis' by Evaluation Services at Michigan State University. Data Analysis. Pilot Test No. 3 Table 4.--Distribution of Item Difficulty and Discrimination Indices. Item Difficulty Indices 81-90 71-80 61-70 51-60 41-50 31-40 21-30 11-20 00-10 Discrimination Indices 7 6 14 9 3 3 2 2 - Mean Item Difficulty 60 Less Less than 81-90 71-80 61-70 51-60 41-50 31-40 21-30 11-20 00-10 00 1 1 4 2 11 8 7 6 5 1 Mean Item Discrimination 34 Other information which was provided by the item analysis included: Mean Score of test Standard Deviation Kuder Richardson Reliability #20 Standard Error of Measurement N - 111 18.18 6.36 .78 2.96 58 A frequency distribution table (see Appendix D) revealed a larger variability of scores on this test and, as can be seen, the scores did not produce a normal curve due to predominancy of low scores and an abnormal spread of higher scores. Through observation and analyaation It was felt that the abnormal central tendency was caused by the testing population rather than by the test Itself. error once again made slight improvement. The standard Both difficulty and discrimination mean indices were higher than previous pilots although one negative discriminator still remained. Items In the visual portion of the test were reacting well to analysis which indicated that a change In this area was unnecessary. Test Construction. Final Test At a final recording session seven Items were re­ recorded. One fact was apparent during the preparation of these tapeB— the slightest variation In performance at the wrong instant can produce unwanted results. The greater degree of discrimination required greater finesse in preparation. The final test consisted of forty-six Items, twentythree of which were visual and twenty-three of which were aural. The aural items utilized the pre-recorded tape and required twenty-five minutes for completion. Discount­ ing the time required to place the information required for the variables on the anwer sheet, the test proper required forty-two minutes for administration. 59 Both the test booklet (see Appendix D) and the tape were now in final form. In preparation for the final test, ten copies of the original tape were dubbed at the studio of Station WFBE, Four thousand coplea of the test booklet were printed at the University of Michigan Copy Center In Ann Arbor. Four thousand answer sheets obtained from the Scoring Center at Michigan State Uni­ versity completed the materials needed for the administration of the test. Random Selection of Schools The random selection process was made by choosing schools on the basis of random numbers. A list of 225 random numbers from 1 to 999 was printed by computer at the University of Michlgan-Flint (see Appendix E). The 1971-1972 list of senior high schools which were members of MSBOA was numbered from 1 to 535. The first number on the random list was 746 and was rejected because It was greater than 535. The second number was 207. The corre­ sponding number on the MSBOA membership list provided the initial school for the testing list. This process continued until a list of seventy randomly selected schools had been constructed. Following the random selection process, letters were sent to the band and orchestra directors of the first sixty schools chosen stating that their school had been selected randomly and requesting participation In this statewide test (see Appendix F) . The additional ten 60 schools were held in reserve should a significant number of the first sixty not participate, but none was needed. The second and third weeks of May were selected as the target period for administration of the test. This seg­ ment of time was deemed the most advantageous, for It was somewhere between the state band and orchestra festival and the close of school. Two weeks gave di­ rectors some latitude In working around concerts or other planned activities which could not be changed. A self- addressed return postcard was Included In this letter (see Appendix F) with the directions that It be returned within five days. Also Included In this mailing was a note to the principal requesting his cooperation in the study (see Appendix F). A follow-up telephone call was required for those directors who did not return the enclosed post­ card. Approximately forty per cent of the sixty schools were called In this manner. An exceedingly high degree of cooperation was shown on the part of these schools, for fifty-seven of the sixty schools contacted responded positively to the query. For various reasons, six of the directors In the fifty-seven schools later found that their groups would be unable to cooperate In the testing. The six schools were not similar In slse or location, indicating a random dropout. A second letter providing Information relative to the test and Its administration was sent to the band and orchestra director in each of the fifty-one schools 61 (Bee Appendix F). It was In this letter that a control group was requested (five to ten per cent of the total number taking the test from each school) to take the test along with the Instrumental music students. The control group was to be randomly selected from the entire school student population of tenth, eleventh and twelfth graders. Any student who had had band, choir or orchestra experience In Junior or senior high school would be rejected. A self-addressed postcard was Included In this mailing (see Appendix F). On the card the director was requested to Indicate how many separate bands or orchestras would be tested, what time the classes met and which days would not be available for testing In hla school. The vast majority of schools had only one group to test or combined two groups Into one period. In the actual administration of the test, forty schools required one test period, six required two test periods and five required three test periods. As for the previous mailing, additional telephone calls were required to accumulate all of the Information. The stage was now set for the scheduling of the fiftyone schools within the two week testing period. A highway map of the State of Michigan was placed upon a bulletin board with the location of each of the schools Indicated by a colored pin. It then became a matter of mechanics In coordinating class meeting times with distances between schools. In an effort to save expenses, every effort was 62 made to schedule two schools in one day per test admin­ istrator or two schools which could be reached by two administrators traveling in one car. Overnight accom­ modations for test administrators also became a determining factor in scheduling, again due to the expense Involved. Two consultants at the Testing Bureau of the Michigan Department of Education suggested that the author train test administrators rather than attempt to send all materials by mall and ask each music director to administer the test to his own students. The latter method would result in loss of control of the testing situation; there­ fore, seven college students who were actively Involved in instrumental music were trained as test administrators. This method would give maximum assurance as to the uni­ formity of the testing situation In all cases. The seven that were selected as test administrators were members of the University of Michigan-Flint Wind Ensemble. The students were briefed as to the scope of the study, their duties and their part In the administrative process. Later a training session was held In which all attended and participated. The following forms (see Appendix F) were distributed and discussed. 1. "Check List for Test Administrators". This was a list of duties for each administrator from the time he arrived at the school until he left. 2. "Checklist of Materials". This was designed so the administrator might check the materials 63 he would need for the testing before he entered the school. Additional materials such as an extra tape, extra pencils, power cord, etc. were listed, hopefully to meet any normal situation. 3. The "Variables". These were to be read to the students by the test administrator before the test began so that they might correctly mark their answer sheets. 4. The "School Information Sheet". This was to be filled out by the test administrator during the test and to be Included In the envelopes con­ taining the completed answer sheets. 5. Mark sense scoring sheets, Form 0-7928, from Michigan State University. Following this discussion, a trial testing situation was held in which the author acted as the test administrator while the students actually took the test. Following the test, additional discussion was held to assure unanimity of understanding toward the testing situation. Each student was then given an ample supply of materials plus the schedule to follow for the succeeding weeks. Administration of the Final Test During the period of May 8th to May 25th, 1972, the test was given to fifty-one schools by seven test admin­ istrators who traveled a total of 5521 miles. Thirteen individual overnight accommodations were required during this period. No more than five test administrators were 64 testing at one time. Five Wollensack portable tape recorders were used to provide the stimuli for the aural part of the test. Ten copies were made of the original pre­ recorded tape so that each administrator had two tapes at all times. The testing ran very smoothly during the seventeen days, for but one long-distance phone call was needed to solve a last-minute emergency. Since only tenth, eleventh, and twelfth grade students were to be tested, some directors were confronted with the problem of what to do with the ninth grade students in their bands or orchestras. This situation was handled in a variety of ways and on only four occasions did the ninth graders actually take the test. The test administrator collected these answer sheets separately and marked them so that they were not processed by Evaluation Services. Each of the four directors was provided with an answer key at a later date which allowed him to hand score the tests of his ninth grade students. Processing and Analyzing the Data The answer sheets from each individual school were kept separately and were taken to Evaluation Services at Michigan State University. The tests were scored on the Op Scan 100 Optical Scanner and the responses to each item, the variables and the mean score of each in­ dividual student were placed upon computer cards. In addition, an item analysis printout was secured for each school plus an item analysis for the combined 65 population of 2714 students from the fifty-one schools. See Appendix H for information relative to the test results of the individual schools. in order of raw mean score. following Information: These schools have been ranked Appendix H also Includes the K. R. #20 reliability, number of students tested, standard error of measurement, high and low score, standard deviation and whether or not a control group was obtained from that school. With the aid of the Consultation Service of the Department of Education and models 3600 and 6500 of Data Control Computers at Michigan State University, extensive data were accumulated and are presented in the following chapter. Nine one-way Analysis of Variance programs pro­ vided data for the eight variables plus the control versus treatment comparison. Four Post Hoc Scheffd Multiple Comparison Techniques were used for the four variables which contained more than a yes-no response. A two-way Analysis of Variance with Fixed Effect Model provided information between grade and classification variables while a similar analysis was completed between the variables of private lessons and instrument. A Three- Way Analysis of Variance with Fixed Effect Model provided information relative to the interaction of the district rating, music theory course and private lesson variables. In addition, Factor Analysis programs rotating eight, eleven, fourteen, fifteen, seventeen and eighteen factors were also completed. 66 Validity For any test to be worthwhile It must be valid; i.e., it must measure what the constructor has set out to measure. This statement Is somewhat unscientific for It relies to varying degrees upon the judgment of the person who con­ structs the test. Cronbsch states: No matter how complete the test author's research, the person who Is developing a selection of classification program must, In the end, confirm for himself the validity of the tests in his partic­ ular situation. And the person who is evaluating a training program must determine the content validity of the tests for his program.^ The author claims validity of this test on the basis of three factors: (1) The test is constructed from objectives which instrumental music teachers determined were Important outcomes of the students' band or orchestra experience, (2) the items on the test make up a number of observables which are a part of the construct of cognitive music achievement, and (3) the pre-recorded tape which is used as a stimulus for the aural portion of the test utilizes actual band and orchestra instruments for all examples. The author recognizes that there are other dimensions in the construct of cognitive music achievement which were not measured by this test. Achievement tests are often constructed from objectives stated in textbooks, opinions of well-known educators, curricula or course objectives. However, 44Cronbach, Psychological Testing, p. 105. 67 what is stated by or from the above may not, in reality, correspond to what is actually being taught in the class­ room: This final responsibility Is in the hands of the individual teachers. Thus, this test has greater validity than those which are constructed from the above-mentioned objectives, for the constructor has gone to the primary source--the classroom teacher--for information upon which to base the construction of this test. Band and orchestra directors selected the objectives they considered to be the most important as logical outcomes of a student's participation in band or orchestra. It may be argued that what teachers say is important and what they actually teach may not be the same; however, it may be argued further that their opinion is much closer to what students are actually learning in their classrooms than opinions of sources removed from the classroom. As previously mentioned, it is not within the scope of this paper to define music achievement; however, it is logical to assume that cognitive music achievement consists of a number of observables which are measurable. The combination of observables within a particular domain is a construct. Although all observables of the construct of music achievement were not measured by this test, the fact that some were measured and that they are a part of the construct of music achievement lends construct validity to the study. when he states; Nunnally confirms this viewpoint 68 Because constructs concern domains of observables, logically a better measure of any construct would be obtained by combining the results from a number of measures of such observables than by taking any one of them individually..... Thus any particular measure can be thought of as having a degree of construct validity depending on the extent to which results obtained from using the measure would be much the same if some other measure, or hypo­ thetically, all the measures, In the domain had been employed in the experiment. Similarly, the combined scores from a number of measures of observables In the domain can be thought of as having a degree of construct validity for the domain as a w h o l e . * * ' The Instruments which were used in the construction of the pre-recorded tape were flute, oboe, clarinet, violin, viola, cello, trumpet, French horn, trombone and snare drum. Thus the sources of the aural examples are those which students use and are familiar with from their band and orchestra rehearsals. Reliability A good test must not only be a valid measure of the subject but also must be consistent. It is difficult to determine a specific degree of reliability required for a test. Many authors do not give specific figures but state that reliability limits may well vary due to such factors as the use of the test, the number of subjects tested, the length of the test and others. Therefore, It is wise to treat reliability within each test and what may be quite reliable in one situation would not be sufficient in another. N.Y.: ^Jum C. Nunnally, Psychometric Theory {New York, McGraw-Hill, 1967), p. 86. 69 Two methods were used to determine reliability of this test: Test-retest and Kuder Richardson #20. The aural portion dictated, to some degree, what types to use and what not to use. When recording errors of live Instru­ mental performance, It Is virtually Impossible to obtain parallel forms. Split-half was likewise discarded as a measure of reliability for this test because of the multiplicity of factors tested within the forty-six questions. Kuder Richardson #20 was used throughout the develop­ ment of the test as one measure of reliability. The following table will show the progress of the three pilot tests: Table 5.--Reliability of Pilot Tests. Standard Deviation Reliability Mean Score 230 .77 26.52 6.34 Pilot No. 2 424 .72 20.12 5.63 Pilot No. 3 111 .78 16.18 6.36 N Pilot No. 1 The population of Pilot No. 1 Included students from two simmer music camps In grades eight through twelve. The fact that these students were voluntarily enrolled In a summer music camp which required a sizable fee would indicate a somewhat abnormal population. This Is evidenced by a 70 substantially smaller variability in this group when com­ pared to the scores of the population in Pilot No. 3. The population of Pilot No. 2 consisted of four widely separated high school bands, grades nine through twelve, from smell to medium-Bleed towns in Illinois, Iowa end Wisconsin. The bands of on urban high school, grades ten through twelve formed the population for Pilot No. 3. All three pilot groups were chosen primarily because of their availability. Throughout the development of this test it was evident that the aural portion required more refinement end re­ designing than the visual. Often items which appeared as valid questions during the recording sessions later per­ formed poorly in the actual testing. The construction of aural perceptive items using live performers Is quite difficult, for It requires an "Inexact" person to produce an "exact" degree of error. The Judgment and observation of the constructor are of extreme Importance. These points come to light when an item analysis was run on the twenty-three aural items of Pilot Test No. 2 and the reliability was found to be .29 while the reliability of the twenty-three visual items was .76. After a careful redesigning of taped items and re-recording of the tape for Pilot Test No. 3, the figure rose to .44 and .79. Following Pilot Test No. 3, seven Items were re-recorded for the final test. Two additional Item analyses were run following the final test on items one through twentythree and twenty-four through forty-six. The results 71 Indicated a reliability of .63 for the flrat twenty-three items and .76 for the second twenty-three Items while the reliability was .61 for the combined portions of the test. The second method used for determining reliability was the Test-retest method. Seventy-one tenth, eleventh and twelfth grade students were tested with an Intervening period of two weeks. Table 6 is useful in comparing the two tests. Table 6.--Test-retest. Reliability. N Mean Score S. D. K. R. #20 S. E. M. Test 76 24.77 6.30 .76 2.96 Retest 71 26.07 6.24 .76 2.98 Reliability of Test-retest .80 T(z1 Using the formula r - x *2) ■ ^ the computed reliability was .80 which compares favorably with .81 with the K. R. #20 of the master test. It Is the opinion of this author that in light of (1) the length of the test and (2) the multiplicity of factors tested by the test, the above reliability is the best that can be expected. Leonhard and House, in summarising opinions regarding degrees of reliability, state that: ".80-.84 Fairly high: of some value in individual measurement and highly 72 satisfactory for group measurement. In light of all factors this test is the most valid and reliable for the purpose which it has served--to measure large groups of Instrumental music students in cognitive music achievement. Factor Analysis Seemingly, the most inconclusive evidence of the entire study was derived from the attempt to arrive at a successful factor analysis of the final test. Six factor analysis programs were run at the Computer Center at Michigan State University rotating eight, eleven, fourteen, fifteen, seventeen and eighteen factors. Of these six runs, the seventeen factor analysis provided the highest loadings on the Individual factors and is shown in Appendix I. In addition, empirical observation determined the seventeen factor rotation to be closer to the factors measured by the test. By comparing the Intercorrelation matrix in Appendix I with a list of factors, by Item, prepared by the author in constructing the test one can readily see a low correlation. As an example, items ten, eleven and twelve of the final test are Items which all require the listener to determine how many measures contain rhythmic errors or If all are played correctly. None of the six factor analyses provides this similarity of these three items. Closer analyzatlon will provide one with similar 46Charles Leonhard and Robert U. House, Foundations and Principles of Music Education (New York: McGrswH i l i , 1959), pp. "341-342. 73 problems throughout the forty-six items. The facts that the final test is multi-factored and that the construct of music achievement depends upon many factors would lead one to expect that no factor analyeie would show conclusive results on this test. Therefore, this seemingly inconclusive evidence may be quite conclusive, for it points to the complexity of cognitive music achievement and the difficulty of measuring it accurately with a single test. This fact is brought to light by consideration of the previously mentioned items ten, eleven, and twelve. Although, on the surface, these items are measuring rhythmic achievement, another factor or factors may be present. It is obvious to this investi­ gator that it is difficult to Isolate any one single cognitive musical factor. This difficulty is compounded when one attempts to measure many factors, of which some are likely interacting with each other, in determining the construct of cognitive music achievement. CHAPTER IV FINDINGS OF THE STUDY Introduction This chapter is a presentation of the findings of this study. The frequency distribution of the statewide test will be presented. be: Following this, the format will the statement of each hypothesis, data and discussion relative to that hypothesis, and finally a rejection or a failure of rejection for each hypothesis. Also Included will be data collected from 2 two-way and 1 three-way analyses of variance between variables of grade In school, school classification, music theory course, private lessons and Instrument family. The chapter will close with supplementary findings resulting from the study. Level of Significance Since each Individual Is used eight times with the variables, the alpha level Is therefore cumulative. To split up the alpha level and to help account for this fact, 0.005 will be used as the level of significance. Frequency Distribution. Statewide Test Table 7 provides the distribution scores of the total population tested, both treatment and control groups. An investigation of the entire deck of 2714 computer cards 74 75 revealed ten on which an exceaslve amount of Information was missing. These were dropped from the study but are Included In the frequency distribution in Table 7 and Bunmary statistics of Table 7a. It should be noted that of the remaining population of 2704 students, the treatment group numbers 2563 and the control group 141. Some students did not respond completely to the question relative to the variables; therefore, the total N of 2563 will not be reached and the total will differ in the suanary statistics of each of the first eight hypotheses. The distribution of scores approximates a normal curve, skewed positively, which indicates a few relatively high scores. This can be observed in Table 7a which shows that twenty-six of the total 2704 scored in the ninetyninth percentile. Their scores ranged from forty-four to thirty-eight. This situation, plus a high frequency of low scores, has caused both the mode and median to be smaller than the mean. Since the test is rather diffi­ cult and the lower scores of the control group are Included In this distribution, It is not surprising that the mean Is somewhat low. The standard deviation is slightly larger than any of the previous three pilot tests, thereby signifying a more heterogeneous population. Table 7a indicates the frequency of raw scores and the percentile rank of the entire population. 76 Frequency Distribution, Statewide Test IOC 29 ye -25 D N - 2714 Mean ■ 19.18 Median - 18.33 Mode - 17 B. D. - 6.79 +* yc> 3$ V4 77 Table 7a.--Sumnary Statistics of Statewide Test. Raw Score 44 43 42 41 40 39 38 37 36 35 34 33 32 31 30 29 28 27 26 25 24 23 22 21 20 19 18 17 16 15 14 13 12 11 10 9 8 7 6 5 4 0 Frequency 3 2 3 o 4 4 8 10 10 22 24 23 44 45 44 44 57 47 78 97 93 102 100 134 144 160 168 176 166 167 165 143 127 113 76 48 26 22 5 3 3 2 Percentile Rank 99 99 99 99 99 99 99 98 98 97 97 96 94 93 91 90 88 86 83 80 77 73 69 65 60 54 48 42 36 30 23 18 13 8 5 3 1 0 0 0 0 0 78 Hypotheses Tasted Hypothesis 1. There la no significant difference in the cognitive music achievement of tenth, eleventh or twelfth grade instrumental students. Table 8. --Summary Statistics of Variable--Grade in School. N % of Population Mean S. D. 1184 47 18.58 6.19 Eleventh Grade 799 32 20.07 6.74 TVelfth Grade 528 21 21.49 7.42 2511 100 19.67 6.74 Category Tenth Grade Total Table 8 Indicates a nearly equal improvement of 1.5 pointfi per grade. Although the meaningful difference Is small, the test does indicate differences across these years. Since the populations of eleventh and twelfth grades become much smaller it may be theorised that the weaker students are dropping out, leaving only the better students in the twelfth grade. On the other hand, it could also be theorized that the stronger students are dropping out because of schedule conflicts or lack of interest. If either supposition were correct, a smaller S. D. should develop; however, the larger S. D. Indicates both suppositions are incorrect since both weaker and stronger students remained, causing greater variability among scores. Like­ wise, if the majority of weaker students remained, the 79 mean score would become lower end, conversely, If the stronger remained, a higher mean score would result. Therefore, since both weaker end stronger students remain, the findings Indicate that there may be some justification for hypothesising a gain In cognitive musical achievement across grades. Table 8a.--One-Way Analysis of Variance. Dependent Varlable-Score of Test, Category Variable--Grade In School. Source of Variation Grade In School Residual Total d.f. Mean Square F p. leas than 2 1644.81 37.29 0.0005 2508 44.10 2510 Table 8b.--Scheffd Post Hoc Method of Multiple Conq>arlsons Mean Differences (x-y) Eleventh Grade (y) Twelfth Grade (x) 1.43* Eleventh Grade (x) Tenth Grade (y) 2.91* 1.49* ^Denotes significance at < 0.005 level. The Scheffd Post Hoc Method Indicates that there Is a significant difference between eleventh grade and twelfth grade, between tenth grade and twelfth grade and also tenth 80 grade and eleventh grade scores. of the above information: Therefore, on the basis Hypothesis 1 is rejected. Hypothesis 2. There is no significant difference in the cognitive music achievement of students In the various MSBOA school classifications. Table 9.--Summary Statistics of Variable--School Classi­ fication. Category N 7. of Population Mean S. D. Class D 123 5 17.94 5.82 Class C 688 27 17.55 5.38 Class B 341 14 19.01 6.70 Class A 566 22 20.96 6.90 Class AA 806 32 21.00 7.23 2524 100 19.63 6.73 Total The mean score gain between AA and D Classifications indicated in Table 9, is approximately .5 of a point higher than the gain in Hypothesis 1 between tenth and twelfth grade, indicating a similar gain of mean scores. Not only do students in Class AA schools form the largest popula­ tion with the highest mean score, but they also show the highest amount of variability. Such variability is likely attributable to another factor or factors other than school classification, such as socio-economic conditions, avail­ ability of private teachers and availability of quality musical concerts. 81 Table 9a.--One-Way Analysis of Variance. Dependent Variable--Score of Test, Category Variable-School Classification. Source of Variation Classification Residual Total Mean Square F p. less than 4 1493.27 34.73 0.0005 2519 43.00 d.f. 2523 Table 9b.--Scheffd Post Hoc Method of Multiple Comparisons Mean Differences (x-y). Class AA (x) Class A (x) Class A (y) Class B (y) Class c (y) Class D (y) .04 1.99* 3.45* 3.06* 1.95* 3.41* 3.02* 1.46 1.07 Class B (x) Class C (x) -.39 ^Denotes significance at < 0.005 level • Results of the Scheffd Post Hoc comparison determine that there is significant difference between Class AA and B, between AA and C, between AA and D, between A and B, between A and C, between A and D; however, no significant difference exists between Class AA and A, between B and C, between B and D or between C and D. If Class AA and A are considered large classifications and B, C, and D as 82 small, then significance exists between large and small classifications, but not within. Thus, students in larger schools scored higher on the test than students from smaller schools. As school size diminished, mean achievement decreased. On the basis of the above information: Hypothesis 2 is rejected. Hypothesis 3. There is no significant difference in the cognitive music achievement of students who partic­ ipate in MSBOA Band and Orchestra Festivals and those who do not. Table 10.--Summary Statistics of Variable--Festival Attendance. Category N Festival Attendance Hean S. D. 2460 97 19.72 6.74 76 03 17.28 5.96 2536 100 19.64 6.73 Festival Non-attendance Total 7o of Population Table 10a.--One-Way Analysis of Variance. Dependent Variable--Score of Test, Category Variable-Festival Attendance. Source of Variation d.f. Festival Attendance Residual Total Mean Square 1 438.46 2534 45.09 2535 F p. less than 9.72 0.002 83 The effect of festival attendance upon the dependent variable, although statistically significant to < 0.002, carries less emphasis than the seven other variables utilized in this test, for the other seven hypotheses are proven at the significance level of < 0.0005. Since the sample of the non-attendance group is comparably smaller, addi­ tional population samples may produce somewhat different results. However, on the basis of the above data: Hypo­ thesis 3 is rejected. Hypothesis 4 . There is no significant difference in the cognitive music achievement of those students whose groups rate first division in MSBOA District Band and Orchestra Festivals and those whose groups do not. Table 11.--Summary Statistics of Variable--Division Rating. Mean S. D. 31 22.31 7.17 1396 57 18.81 6.29 292 12 17.60 5.92 2430 100 19.73 6.76 Category N X Rating 742 II Rating III Rating Total % of Population Information in Table 11 illustrates that while less than one-third of the bands and orchestras that attended district festivals received a first division rating, the scores of the students in these groups generated the great­ est achievement plus the greatest variability. 84 Table 11a.--One-Way Analysis of Variance. Dependent Variable--Score of Test, Category Variable-Dlstrlct Rating. Source of Variation District Rating d. f. 2 Mean Square F 3715.22 87.19 p. less than 0.0005 2427 Residual Total 2429 Table lib.--Scheffd Post Hoc Method of Multiple Comparisons Mean Differences (x-y). III Rating (y) II Rating (y) I Rating (x) 4.71* 3.49* 1.21 XI Rating (x) ♦Denotes significance at < 0.005 level. The Scheffd Post Hoc comparison reveals In Table lib that there Is a significant difference between first division and second division, between first division and third division; but no significant difference between second and third division ratings. of the above information: Therefore, on the basis Hypothesis 4 is rejected. Hypothesis 5 . There is no significant difference in the cognitive music achievement of students who play instruments in the four separate families. 85 Table 12. --Summary Statistics of Variable •-Instrument Family. Category N % of Population Mean S. D. Brass 855 34 19.61 6.72 Percussion 237 9 16.11 5.54 1248 50 19.97 6.43 162 7 24.07 7.25 Total 2502 100 19.74 6.69 Woodwind String The Information In Table 12 Illustrates the fact that students who play Instruments In the four separate families scored differently on this test. While woodwind and brass players scored similarly, the percussion players scored perceptibly lower and the string players perceptibly higher. This table reveals the somewhat surprising fact that there are two per cent more percussionists than string players In this sample of Michigan high schools. Again, as In the past hypotheses, the highest mean score also contains the greatest variability. Table 12a.--One-Way Analysis of Variance. Dependent Variable--Score of Test, Category Variable-Instrument Family. Source of Variation Instrument Family Residual Total d.f. Mean Square F 3 2084.49 49.23 2498 42.34 2501 p. less than 0.0005 86 Table 12b.--Scheffd Post Hoc Method of Multiple Comparisons Mean Differences (x-y). Woodwind (y) A. 20* String (x) Woodwind (x) Brass (y) Percussion (y) 4.46* 7.96* .36 3.88* 3.50* Brass (x) ^Denotes significance at < 0.005 level. The Scheffd Post Hoc comparison reveals in Table 12b that significant differences exist between string and woodwind, between string and brass, between string and percussion, between woodwind and percussion, between brass and percussion players but none exist between brass and woodwind players. The greatest meaningful difference exists between string and percussion players, for the string players scored approximately fifty per cent higher than the percussion players. This Is the largest mean difference of score thus far on the test. If a one-way analysis of variance had been made using only the scores of string players and percussionists, the effect of instru­ ment upon the dependent variable would have been more pronounced. Thus, on the basis of the above information: Hypothesis 5 is rejected. Hypothesis 6 . There is no significant difference in the cognitive music achievement of students who have taken private lessons and those who have not. 87 Table 13.— Summary Statistics of Variable--Private Lessons. Category N Private Lessons No Private Lessons Total % of Population Mean S. D. 999 40 22.46 7.21 1500 60 17.90 5.66 2499 100 19.72 6.71 Table 13a.--One-Way Analysis of Variance. Dependent Variable--Score of Test, Category Varlable-Prlvate Lessons. Source of Variation d. f. Mean Square 1 12,458.43 2497 40.01 Private Lessons Residual Total F 311 .36 p. less than 0.0005 2498 A study of data In Tables 13 and 13a will reveal that private lessons do present a strong effect upon the dependent variable. Although there Is considerably more variability in the private lessons group, the mean score is more than 4.5 points higher than the score of the non-private lessons group. There would appear to be strong meaningful difference between the two groups. On the basis of the above Information: Hypothesis 6 Is rejected. Hypothesis 7. There is no significant difference in the cognitive music achievement of those students whose 88 parents, one or both, have participated in a school band or orchestra and those students whose parents have not. Table 14.--Summary Statistics of Variable--Parent Participation. Category N Parent Participation No Parent Partlc ipat ion Total 7. of Population Mean S. D. 892 36 20.78 7.03 1599 64 19.11 6.44 2491 100 19.71 6.70 Table 14a.--One-Way Analysis of Variance. Dependent Variable--Score of Test, Category Varlable-Parent Participation. Source of Variation Parent Participation Residual Total d.f. Mean Square 1 1597.26 2489 44.27 F p. less than 36.08 0.0005 2490 The statistics from Tables 14 and 14a Indicate slgnif Icance at less than 0.0005; however, as in Hypothesis 3, the meaningful difference is less pronounced. There is a small variation of group means around the grand mean which would indicate less effect of the Independent variable 89 upon the dependent variable. above information: However, based upon the Hypotheala 7 la rejected. Hypothesis 8 . There is no significant difference in the cognitive music achievement of those students who have taken a formal music theory course in high school or summer music camp and those who have not. Table 15.--Summary Statistics of Variable--Music Theory Course. Category N J. of Population Mean S. D. 376 15 24.69 7.89 No Theory Course 2132 85 18.86 6.07 Total 2508 100 19.74 6.69 Theory Course Table 15a.--One-Way Analysis of Variance. Dependent Variable— Score of Test, Category Variable-Music Theory Course. Source of Variation Theory Course Residual Total d.f. Mean Square 1 10,858.10 2506 40.46 F p. less than 268.38 0.0005 2507 Statistics In Tables 15 and 15a indicate a strong effect of the Independent variable upon the dependent variable. The mean of 24%69 is the largest mean score of 90 any grouping thus far on the test. Furthermore, the difference between the two groups is almost six points, which is likewise the largest thus far. Such a result is expected since the test is based primarily upon know­ ledge of music theory and its application. of the above information: On the basis Hypothesis 8 is rejected. Hypothesis 9 . There is no significant difference in the cognitive music achievement of those students who have taken Instrumental music in the secondary school and those who have not. Table 16.--Sumnary Statistics of Variable--Treatment. Category N Treatment 2563 Mean S. D. 95 19.58 6.74 141 05 12.82 3.57 Total 2704 100 19.22 6.78 Control % of Population Table 16a. --One-Way Analysis of Variance. Dependent Variable--Score of Test, Category Variable— Treatment. Source of Variation Treatment Residual Total d.f. Mean Square 1 6093.67 2702 43.74 2703 F p. leas than 139.33 0.0005 91 Statistically, Table 16a reveals a significance at < 0.0005 which would indicate a strong Influence of the treatment group upon the score. However, in the light of a meaningful difference, one would expect the mean score difference between the two groups to be larger. This situation has been affected by the difficulty of the test and by the large number of relatively low scores, which is evident from the frequency distribution in Table 7. Stated in another manner, the scores of some Michigan band and orchestra students Indicate considerable cognitive achievement while other scores are low. There is a large variability in achievement scores by Michigan band and orchestra students. On the basis of the statistical information provided in Tables 16 and 16a: Hypothesis 9 is rejected. Table 16b.--Supplementary Information Relative to Hypothesis 9. Mean Differences. Test Items Control Group Treatment Group Difference 1-23 8.12 11.94 3.82 IT, 24-46 4.70 7.64 2.94 12.82 19.58 X, Total Mean The information in Table 16b indicates a corresponding gain by the treatment group in both aural perception and in visual cognitive knowledge over the control group. Since 92 the mean score for the first twenty-three items is higher In both groups than the last twenty-three items, some factor or factors are present to cause this difference. This investigator theorises that, since all the items from one to twenty-three are aural, this factor is aptitude. It is worthy of note that by pure chance one would answer correctly 4.52 questions on the aural part of the test and 4.60 questions on the visual part. Thus, the achieve­ ment is 3.70 points above chance by the control group and 10.66 points above chance by the treatment group. Interaction of Variables It was decided to analyze the interaction of six of the above variables. The number of possible combina­ tions of interactions utilizing the eight variables is staggering; therefore, those six which had the greatest effect upon the dependent variable were used in the study. Grade, school classification, district rating, instrument family, private lessons and music theory course were used in three Interaction analyses. There Is a constant gain of mean scores, with three exceptions, in tenth, eleventh, and twelfth grade students, as reported in Table 17, within all classifications. The highest mean score of 22.86 by twelfth grade Class AA students and the lowest mean score of 17.12 by tenth grade Class D students reveals a mean difference of 5.74. The three mentioned exceptions Involve eleventh and twelfth grade Class C and D where D students scored higher and 93 Table 17.--Sutnnary Statistics, Grade-"Classification Interaction. Tenth AA (N) Mean Score A (N) Mean Score B (N) Mean Score C (N> Mean Score D (N) TVelfth 369 246 173 19.65 22.03 22.86 253 176 128 19.66 21.87 22.64 155 105 63 18.24 19.20 21.92 334 225 126 17.18 17.34 18.94 59 Mean Score Eleventh 17.12 34 17.76 30 19.80 Pooled within-cell standard deviation ■ 6.45 Class A and AA where Class A tenth graders scored .01 point higher than ClasB AA tenth grade students. It Is interesting to note that the greatest mean gain from tenth to twelfth grade occurs in Class B with a gain of 3.68 while Class C Is the lowest with a gain of 1.76. 94 Table 17a.--Two-Way Analysis of Variance with Fixed Effect Model. Interaction of Grade and Classification. d.f. Source of Variation Mean Square F p. less than Grade (G) 2 1006.62 24.21 0.0001* Classification (C) 4 1439.48 34.62 0.0001* (G) - (C) Interaction 8 66.21 1.59 0.1219 2461 41.58 Residual Total 2475 ^Denotes slgnlflcance at < 0.005 level. The information contained In Table 17a Indicates significance of both grade and classification as a signif­ icant effect upon the dependent variable; however, this had been proven earlier. The Interaction of the two variables produces a significance at the .12 level. Table 18.--Summary Statistics, Private Lessons--Instrument Family Interaction. Woodwind Private Lessons Brass Percussion Yes (N) Mean Score 303 21.95 93 18.01 483 22.70 116 26.40 No (N) Mean Score 544 18.33 136 15.03 756 18.22 45 18.33 Pooled within-cell standard deviation ■ 6.17 String 95 The data contained in the Interaction of private lessons and instrument family provide some of the most revealing Information of the entire study. From a study of Table 18 It Is determined that there Is a large meaning­ ful difference between the scores of those percussion players who did not take private lessons and the string players who did. The percussion players mentioned scored only slightly less than six points above pure chance while the string players provided a mean score seventyfive per cent greater than the percussion players. It may also be noted from this table that those percussion players who took private lessons scored lower than students In all other Instrument groups who did not. This fact becomes more marked when one recalls the strong effect of private lessons upon the dependent variable. Table 18a.--Two-Way Analysis of Variance with Fixed Effect Model. Private Lessons— Instrument Family Interaction. Source of Variation d.f. Private Lessons (P) 1 6371.91 167.65 0.0000* Instrument Family (I) 3 1216.43 32.00 0.0000* (P)-(I) Interaction 3 217.37 5.72 0.0007* 2467 38.01 Residual Total Mean Square F 2474 ^Denotes significance at < 0.005 level. p. less than 96 Graph A.--Interaction of Private Lessons--Instrument Family. Score 25 23 21 19 17 15 Brass Perc. W.Wlnd String By observing Graph A, one can see that the effect of private lesaons on scores of brass, percussion and wood­ wind players Is similar, In that the increase is approxi­ mately 3 to 4.5 points, whereas the addition of private lessons for the string players provides a difference of over 8 points. It should also be noted that the difference between string players who took private lessons and percussion players who did not is nearly 11.5 points. A final Interaction was made with district rating, music theory course and private lessons as Independent variables. From the Information In Table 19 one can determine that approximately a thirteen point spread exists from highest to lowest mean group scores. Third division students who had not taken private lessons or a music theory course scored a mean of 16.34, while first 97 Table 19.--Sutnnary Statistlcs--Dlstrlct Rating, Private Lesaons, Music Theory Course Interaction. Independent Variables District Rating 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. I I I I II II II II Sunsnary Statistics Private Lessons Theory N Mean Yes Yes No No Yes Yes No No Yes Yes No No Yes No Yes No Yes No Yes No Yes No Yes No 101 292 57 282 95 398 66 799 19 51 15 201 29.24 22.96 22.77 19.50 27.11 20.02 19.47 17.36 23.53 19.75 20.27 16.34 Fooled wlthln-cell standard deviation “ 5.90 Table L9a.--Cell Means, Music Theory Course--Private Lessons. Theory - Yes Theory - No Private Lessons Yes Cell #1,5,9 Mean Cell #2,6,10 Mean Private Lessons No Cell #3,7,11 Mean 20.96 27.79 21.16 Cell #4,8,12 Mean 17.67 98 division students who had taken both scored a mean of 29.24. From observation of the same table It would appear that the combination of private lessons and a music theory course would, In fact, produce a strong Interaction, for the score of the second division group with both private lessons and music theory course produces the second highest mean score and the third division group with both private lessons and music theory course produces the third highest score. Table 19b.--Three-Way Analysis of Variance with Fixed Effect Model. District Rating--Private Lessons--Music Theory Course Interaction. Source of Variation d.f. Mean Square F District Rating (D) 2 1115.46 32.09 0.0001* Private Lessons (P) 1 3476.61 100.01 0.0001* Theory Course (T) 1 3370.13 96.95 0.0001* (D)-(P) Interaction 2 38.38 1.10 0.33 (D)-(T) Interaction 2 9.50 0.27 0.76 (P)-(T) Interaction 1 296.45 8.53 0.004* (D)-(P)-(T) Interaction 2 86.07 2.48 2364 34.76 Residual Total p. less than 2375 ♦Denotes significance at 0.005 level. In Table 19b it can be seen that there is an inter­ action at the level of significance with private lessons 99 and music theory course. There Is a weaker Interaction of all three independent variables not at the signif­ icant level. There la little Interaction between district rating and private lessons and practically none with district rating and music theory course. Graph B.--Interaction of Private Leaeons--Music Theory Course. Score 28 26 24 22 20 18 16 Yes No Theory Graph B Indicates the Interaction of music theory course and private lessons without the effect of district rating. Table 20 indicates that In this study knowing whether or not a student takes private lessons accounts for eleven per cent of the total variance of the dependent variable. 100 Table 20.--Comparison of the Influence of the Independent Variables Upon the Dependent Variable. Multiple Correlation Coefficient (R) Coefficient of » Determination (R ) Private Lessons .33 117. Music Theory Course .31 107. District Rating .26 077. Instrument Played .24 067. School Classification .23 057. Grade in School .17 03(7. Parent Participation .12 01* Festival Attendance .06 00.1* Source of Variation It should be noted that these percentages are not cumulative; therefore, one may not add them to determine a total per­ centage. This table Indicates that relative to cognitive music achievement the effect of whether or not a student participates in a band and orchestra festival Is less than the effect of the rating his group attains when they do participate. Supplementary Findings In addition to the previous findings other information was compiled on the basis of the results of the test. information Indicates the status as of May, 1972. This 101 1. Enrollment in band or orchestra by grade: Tenth grade--1184 students (47%) Eleventh grade— 799 students (32%) Twelfth grade--528 students (21%) 2. Enrollment in band or orchestra by class sice: Class AA--806 students (32%) Class A— 566 students (22%) Class B — 341 students (14%) Class C--688 students (27%) Class D— 123 students (05%) 3. Enrollment In band or orchestra by participation in district band and orchestra festivals: Festival attendance--2460 students (97%) Festival non-attendance--76 students (03%) 4. Enrollment in band or orchestra by division rating in district band and orchestra festivals: First division— 742 students (31%) Second division--1396 students (57%) Third division--292 students (12%) 5. Enrollment in band and orchestra by instrument family: Brass— 855 students (34%) PercussIon--237 students (09%) Woodwind--1248 students (50%) String— 162 students (07%) 6. Enrollment in band or orchestra by those who have taken ten or more private lessons in the past five years: 102 Private lessons--999 students (40%) No private lesBons--1500 students (60%) 7. Enrollment In band or orchestra by those who have at least one parent who participated In a school band or orchestra: Parent participation--892 students (36%) No parent participation--1599 students (647.) 8. Enrollment In band or orchestra by those who have taken a formal music theory course In high school or simmer music camp: Music theory course--376 students (15%) No music theory course--2132 students (85%) 9. Enrollment In band or orchestra by Instrument family and by private lessons: Brass--847 students, 303 took private lessons (36%) Percusslon--229 students, 93 took private lessons (40%) Woodwind--1239 students, 483 took private lessons (39%) Strlng--161 students, 116 took private lessons (72%) CHAPTER V SUMMARY, CONCLUSIONS AND IMPLICATIONS FOR FURTHER RESEARCH Introduction This final chapter contains the concluding statements about the study. The first section la a summation of the development of the cognitive, aural-p trceptlon achievement test and Its administration. The second section la a presentation of the conclusions drawn from the study while Implications for further study are contained In the third and concluding section. The purpose of this study was to assess the cognitive knowledge, verbal and aural, of a sample of the tenth, eleventh and twelfth grade instrumental music students In Michigan and to compare the sample and the results on the basis of eight variables. Since there was no test available which fit this specific purpose, the author developed a cognitive, aural-perception achievement test which could be administered In one normal class period. The test was developed from a list of objectives which band and orchestra directors In Michigan determined as logical outcomes from students' participation in bands and orchestras. 103 104 Three pilot tests were developed and administered in various locations in four Midwestern states. From these three pilots, the final statewide test was developed which consisted of twenty-three aural items, stimuli provided by a pre-recorded tape, and twenty-three visual items. The entire test required forty-two minutes for administration. Instrumental music students from fifty-one randomly selected schools in Michigan took the test In May, 1972. The population tested Included a control group of 141 stu­ dents who had had no secondary instrumental music experience and the treatment group of 2563 students. The results of this test were analysed by computer at Michigan State University and were presented in the pre­ ceding chapter. Conclusions On the basis of the data in Chapter IV and the rejection of all nine hypotheses the following conclusions may be stated: 1. The test was sensitive in uncovering the differentiates in cognitive music achievement among tenth, eleventh and twelfth grade students. The results suggest a linear relationship since the mean score of the eleventh graders was higher than that of the tenth graders and the mean score of the twelfth graders was higher than that of the eleventh graders. The differences were 105 too large to be attributed to chance; therefore, one is forced to conclude that achievement differs across grade levels. 2. The test was sensitive In uncovering the differen­ tiates in cognitive music achievement among the various MSBOA school classifications. The mean score of students In larger schools was higher than the mean score of those in smaller schools. Although there was little meaningful difference between Class AA and Class A students and the mean score of Class D students was higher than the mean score of Class C students, the differences, In all classes, were too large to be attributed to chance; therefore, one Is forced to conclude that achievement differs across school slse. 3. The test was sensitive In uncovering the differen­ tiates in cognitive music achievement between students who participated In MSBOA District Band and Orchestra Festivals and those who did not. The mean score of the students who participated was higher. This difference was too large to be attributed to chance; therefore, one Is forced to conclude that achievement differs from festival participation to non-participation. 4. The test was sensitive in uncovering the differen­ tiates in cognitive music achievement among students whose groups received various ratings In MSBOA 106 District Band and Orchestra Festivals. The results suggest a linear relationship since the mean score of the students in second division groups was higher than the mean score of those in third division groups and the mean score of the students in first division groups was higher than the mean score of those in second division groups. The differences were too large to be attributed to chance; therefore, one is forced to conclude that achievement differs across district band and orchestra ratings. 5. The test was sensitive in uncovering the differen­ tiates In cognitive music achievement among the players of instruments in the four families of Instruments. The mean score of the string players was highest of the four groups, while the mean score of the percussion players was the lowest. These differences were too large to be attributed to chance; therefore, one is forced to conclude that achievement differs across the various instrument families. 6. The test was sensitive in uncovering differen­ tiates in cognitive music achievement between students who took private lessons and those who did not. The mean score of those students who took private lessons was higher. This difference was too large to be attributed to chance; therefore. 107 one is forced to conclude that achievement differs from taking private lessons to not taking private lessons. 7. The test was sensitive In uncovering differen­ tiates in cognitive music achievement between students whose parents participated In band or orchestra and those whose parents did not. The mean score of the students whose parents partic­ ipated was higher. The difference was too large to be attributed to chance; therefore, one Is forced to conclude that achievement differs from parental participation to non-partlclpatlon. 8. The test was sensitive In uncovering the differen­ tiates In cognitive music achievement between students who had taken a formal music theory course and those who had not. The mean score of those who had taken a music theory course was higher. The difference was too large to be attributed to chance; therefore, one Is forced to conclude that achievement differs from election to non-election of a music theory course. 9. The test was sensitive In uncovering the differen­ tiates in cognitive music achievement between students who participated in a school band or orchestra and those who did not. The mean score of those who participated was higher. The difference was too large to be attributed to 108 chance; therefore, one la forced to conclude that achievement differs from those who participate to those who do not. Several interactions were derived from the two and three-way analyses. In general, on the basis of mean scores, older students from larger schools scored higher than younger students in smaller schools. A strong Inter­ action existed when string players took private lessons Inasmuch as their scores were much higher than those of any other Instrumentalists. A strong Interaction was also present for students with private lessons and music theory course Inasmuch as their scores were likewise much higher. In addition to the above conclusions, the test pro­ vided further Information. There was a considerable variance In the achievement of students In different schools (see Appendix H). The highest mean score of 24.88 was nearly twice as large as the lowest, 13.73. Furthermore, the mean score of the lowest school in the treatment group was less than one point higher than the mean score of the total control group who had had no secondary school band or orchestra experience. The musical cognitive knowledge gain In some schools was minimal. It Is very evident that students who achieved higher scores also listened more carefully to Instructions and filled out responses on answer sheets more completely. 109 Through a sorting of the computer cards It was observed that students in the upper ten per cent rarely omitted information while such omissions were conmonplace on the cards of the lower ten per cent. It should be noted that although there was statistical significance for all variables, grade in school, parent participation and festival participation provided the least meaningful differences. Furthermore, it would seem there should have been a larger mean difference in score between the treatment and the control groups. It seems likely that experience in a high school band or orchestra should have provided higher scores on this test. The greatest meaningful difference was provided by private lessons. The music theory course provided a strong meaningful difference but this was to be expected. While not so conclusive as these two variables, district rating, instrument family and school classification did provide a strong meaningful difference. There is a need for a larger number of music achievement tests which are both valid and reliable. Many existing tests are designed for a specific purpose; hence are not good measures for other than that one purpose. The Roth Cognitive, Aural-Perceptlon Test is both a valid and a reliable instrument to measure cognitive musical achievement, especially when measuring groups of students. Although it is a multi-factor test it has functioned well as a measuring instrument. 110 On the basis of work done in this study it seems that it is possible to develop an even more reliable multi-factor cognitive music achievement test which does not require more than fifty minutes for administration. A battery of tests is time-consuming, for it requires either several days of single-period administration or one day of multiple-period administration. In either case, the effect of testing may produce fatigue, boredom or disinterest among the students. It is likely that the students would not perform to their capabilities under such conditions. A multiple-period test is disruptive to the normal school program. In addition, mortality caused by student absences may result in a greater loss of testable population in the administration of a multipleperiod test. Therefore, a test which is both reliable and valid and requires but one normal school period for administration would be superior to those which require multiple periods. It is the opinion of this investigator that, on the basis of the results obtained in this study, it is possible to develop such a test. Implications for Further Research As a result of the findings of this study, a number of implications for further research are clear. The fact that forty-seven per cent of the instrumental music atudents in Michigan are in the tenth grade while only twenty-one per cent are twelfth graders indicates a possible dropout problem. It would be in error to state that there is Ill greater than a 1007. dropout, for such a statement would require a three-year study. There lst however, clear evi­ dence that the participation becomes less in the eleventh grade and still less in the twelfth grade. It would be of great value in the evaluation and planning of instru­ mental music programs to determine the reasons for this lessening participation. Such a study, if statewide in nature, could have great impact in curriculum reorgani­ zation. It was previously stated that a great variance of achievement existed between schools. the need for further study. cause this variance? This, too, indicates What are the factors that Do situations exist which could be changed to increase achievement? Do similar factors appear in other schools which might indicate a widespread deficiency? The large variance in the achievement of students who play different instruments indicates a need for study. Are the string players, who scored high, achieving because of the musical demands of their instruments or have they been placed upon the Instruments by teachers whose selection process places high achievers in the string family? Are teachers more efficient in their teaching of string players than they are in teaching percussionists? Does the college preparation of these teachers encourage this problem by placing greater emphasis upon the string family with little 112 emphasis on percussion? very low? Why do percussionists score so The answers to the above questions can be pro­ vided by further research. It Is possible, in the opinion of the investigator, to develop a test which will measure the construct of cognitive music achievement. However, it must be deter­ mined what factors make up this construct. It seems obvious that there are many factors at work, with varying degrees of effect, In determining individual student achievement. Likewise, it seems obvious that individual students react differently to the effects of such factors. If it were possible to determine the factors which do influence musical achievement, the profession would be in a much better position to reorganise its existing music curricula into more productive programs. Although such answers may be either elusive or difficult to analyse and put into practice, the music education profession should continually strive for such solutions. BIBLIOGRAPHY Books Barr, Arvll S.; Davis, Roberta A.; and Johnson, Palmer 0. Educational Research and Appraisal. Chicago: J. B. Lippincott company, 1953. Bloom, Benjamin S. Taxonomy of Educational Objectives. New York: David McKayCompany, Inc., 19^0. Borg, Walter R. Educational Research. An Introduction. New York: David 'McKay Company, Inc., Brown, Frederick G. Measurement and Evaluation. Illinois: F. E. Peacock, 1971. Itasca, Chauncey, Henry, and Dobbin, John E. Testing. Its Place in Education Today. New York: Harper and Row, l9b3. Colwell, Richard. The Evaluation of Music Teaching and Learning.^ Englewood cutts, N. J.: Prentlce-Haii, Cronbach, Lee J. Essentials of Psychological Testing. New York: Harper & Brothers, 1949. Ebel, Robert L. Essentials of Educational Measurement. Englewood Clifts, N. J.: Prentice-Hall, inc., 1972. Gronlund, Norman E. Constructing Achievement Teats. Englewood Cliffs, N. J. : Prentice-Hall, Inc., 1968. . Measurement and Evaluation in Teaching. New York: The Macmillan Company, iy/i. Lehman, Paul R. Tests and Measurements in Music. Englewood Clifts, N. J. : Prentice-Hall, Inc., i9b&. Leonhard, Charles, and House, Robert W. Foundations and Principles of Music Education. New York: McGrawHill Book C o ., 1959. Mager, R. F. Developing Attitude Toward Learning. Palo Alto, Calif.: Fearon, i§bB. Preparing Instructional Objectives. Palo Alto, dalif.: Fearon, l96z. 113 114 Michigan School Band and Orchestra Association. Handbook of Music Theory. Ann Arbor: Michigan School Band and Orchestra Association, 1964. Music History and Literature, a Handbook for ReKearsal Enrichment. Ann Arbor: Michigan school Band and Orchestra Asaoclation, 1970. Nunnally. Jutn C. Hill, 1967. Psychometric Theory. New York: McGraw- -- Renners, H. H. , and Gage, N. L. and Evaluation. New York: Educational Measurement tfarper & Brothers, 1943. Tate, Merle W. Statistics in Education and Psychology. New York: The Macmillan company, 196V. Thorndike, Edward L. An Introduction to the Theory of Mental and Social Measurements. 2nd ed. New Y o r k : 'teachers dollege, CoTumbla^Jnlverslty, 1913. Thorndike, Robert L . , and Hagen, Elisabeth. Measurement and Evaluation in Psychology and Education. New York: John Wiley and Sons, 1961. Van Dalen, Deobold B., and Meyer, William J. Understanding Educational Research. New York: McGraw-Hill, 1962. Whybrew, William E. Dubuque, Iowa: Measurement and Evaluation in M u sic. Wm. b. Brown Company, 190Z. Standardized Tests Allferis, James. Allferis Music Achievement Test. University of Minnesota frees, 195ft. Colwell, Richard. Music Achievement Tests. Follett Publishing Company, 1965. Minneapolis: Chicago: Knuth, Alice Snyder. Snyder knuth Music Achievement T e s t . San Francisco: Creative Arts Research Associates, 1965. Watkins, John G., and F a m u m , Stanley E. Watkins-Farnum Performance Scale. Winona. Minn.: Hal Leonard Music. Inc., 1954.----- 115 Dissertations Folstrom, Roger J. "A Comparison Study of the Musical Achievement of Students in Three Illinois High Schools." Unpublished Ph.D. dissertation. Northwestern University. 1967. Iltis, John L. "The Construction and Validation of a Test to Measure the Ability of High School Students to Evaluate Musical Performance." Unpublished Mus. Ed.D. dissertation, Indiana University, 1970. Mansur, Paul M. "An Objective Performance-Related Music Achievement Test." Unpublished Ph.D. dissertation, University of Oklahoma, 1965. Snyder, Alice M. '*The Development and Standardization of a Test of Music Achievement." Unpublished Ph.D. dissertation, University of Oregon, 1958. Swinchoski, Albert A. "The Development of a Standardised Music Achievement Test for the Intermediate Grades." Unpublished Ph.D. dissertation, University of Kansas, 1963. Articles In Journals Allferis, James, and Stechlein, James E. "The Development of a College Entrance Test In Music Achievement. Journal of Research in Music Education. No. 2 (1953), 83-96. Colwell, Richard. "An Investigation of Musical Achievement Among Vocal Students, Vocal-Instrumental Students and Instrumental Students." Journal of Research In Music Education. No. 11 (Fall, 1905), 1Z3-13U. "The Development of the Music Achievement Test Series." Council for Research In Music Education. No. 22 (Fall, 197U) , 3 7 -73.------- ---------------Ebel, Robert L. "Some Measurement Problems In a National Assessment of Educational Progress." Journal of Educational Measurement. Volume 3, No. 1 1Spring, (l9bb), 11-17. Holz, Emil A. "The National School Band Tournament of 1923 and its Bands." Journal of Band Research. Ill (Autumn. 1966), 17. ---------------------- 116 Merwin, Jack C. "The Progress of Exploration Toward a National Assessment of Educational Progress." Journal of Educational Measurement. Vol. 3, No. 1 (Spring, 1 9 6 6 ), 5 -1 0 . Tyler, Ralph W. ’'The Objectives and Plans for a National Assessment of Educational Progress." Journal of Educational Measurement. Volume 3, No. 1 (Spring, 1966), n r Woodruff, Asahel D. "Concept Teaching in Music." Perspectlves in Music Education Source Book XII, Edited "by Bonnie 0. kowaii. Washington, to. C.: MfcNC, 1966, 219-223. Miscellaneous Buros, Oscar Krisen. Mental Measurements Yearbook. Highland Park, N. J. : The Gryphon Press, 197Z. Merwin, J. C., and Womer, F. B. "Evaluation in Assessing the Progress of Education to Provide Bases of Public Understanding and Public Policy." Educational Evalation: New Roles. New Means. Sixty-eighth Yearbook of the National Society for the Study of Education, Part II. Chicago: The University of Chicago Press, 1969. Michigan State Board of Education. A Position Statement on Educational Accountability. Lansing, Mich. : Michigan State Board of Education, 1972. Hightower, Caroline. How Much Are Students Learning? Report from the Comnlttee on Assessing the Progress of Education. Ann Arbor, Mich.: November, 1968. APPENDICES APPENDIX A TEST OBJECTIVES APPENDIX A TEST OBJECTIVES Rehearsal ObjectIves Please respond as to the importance of each objective-1 is the lowest response while 11 is the highest response. On the basis of participation in band or orchestra, Michigan high school instrumental music students should be able to: 1 2 3 4 5 6 7 8 9 10 11 (1 recognise major key signatures. 1 2 3 4 5 6 7 8 9 10 11 (2 recognize minor key signatures. 1 2 3 4 5 6 7 8 9 10 11 (3 recognize major and perfect Intervals. 1 2 3 4 5 6 7 8 9 10 11 (4 recognize minor intervals. 1 2 3 4 5 6 7 8 9 10 11 (5 recognize augmented Intervals. 1 2 3 4 5 6 7 8 9 10 11 (6 recognize diminished Intervals. 1 2 3 4 5 6 7 8 9 10 11 (7 hear the difference between major minor, and perfect Intervals. 1 2 3 4 5 6 7 8 9 10 11 (8 place given composers In the correct period of music history. 1 2 3 4 5 6 7 8 9 10 11 (9 Interpret basic musical termi­ nology. 1 2 34 5 6 7 8 9 10 11 (10 detect incorrect phrasing. 1 2 34 5 6 7 8 9 10 11 (11 recognize different forms of the minor scale. 1 2 34 5 6 7 8 9 10 11 (12 spell basic triads in a given key 1 2 34 5 6 7 8 9 10 11 (13 place the periods of music in chronological order. 1 2 34 5 6 7 8 9 10 11 (14 recognize the basic style of the major periods of music. 1 2 34 5 6 7 8 9 10 11 (15 hear Intonation discrepancies in chords. 118 9 10 11 (16) hear balance problems In chords. 2 3 4 5 6 7 8 9 10 11 (17) hear rhythmic discrepancies from printed parts. 1 3 4 5 6 7 8 9 10 11 (18) hear major and perfect Intervals. 1 2 3 4 5 6 7 8 9 10 11 (1 9 ) hear minor intervals. 1 2 3 4 5 6 7 8 9 10 11 (2 0 ) hear augmented Intervals. 1 CM 3 4 5 6 7 8 9 10 11 (2 1 ) hear diminished Intervals. 1 2 3 4 5 6 7 8 9 10 11 (2 2 ) recognise modal scales. 1 2 3 4 5 6 7 8 9 10 11 (23) hear modal scales. 1 2 3 4 5 6 7 8 9 10 11 (24) hear modulations. 1 CM 3 4 5 6 7 8 9 10 11 (25) recognize the basic forms in music (three part, two part, etc.) 1 to 3 4 5 6 7 8 9 1 0 11 (26) transpose. 1 ro 3 4 5 6 7 8 9 1 0 11 (27) recognize contrapuntal writing versus homophonlc writing. 1 ro 00 3 4 5 6 7 CM 1 CM 1 3 4 5 6 7 8 9 10 11 (28) recognize the difference between half steps and whole steps. 1 2 3 4 5 6 7 8 9 10 11 (2 9 ) recognize the musical character­ istics of the major periods of music. 1 2 3 4 5 6 7 8 9 10 11 (30 recognize pentatonlc and whole tone scales. 1 2 3 4 5 6 7 8 9 10 11 (31 know the difference between parallel and relative minor. 1 2 3 4 5 6 7 8 9 10 11 (32 recognize instruments by their characteristic sound. 1 2 3 4 5 6 7 8 9 10 11 (33 recognize similar and dissimilar melodies. 1 2 3 4 5 6 7 8 9 10 11 (34 recognize cadence points. 1 2 3 4 5 6 7 8 9 10 11 (35 hear intonation discrepancies in a melodic line. 1 2 3 4 5 6 7 8 9 10 11 (36 hear intonation discrepancies between individuals. 119 1 2 3 4 5 6 7 8 9 10 11 (37) hear the difference between half steps and whole steps. 1 2 3 4 5 6 7 8 9 10 11 (38) recognize meter upon hearing a melody. 1 2 3 4 5 6 7 8 9 10 11 (39) hear a number and place it In the correct historical period. 1 2 3 4 5 6 7 8 9 10 11 (40) Interpret basic musical orna­ mentation. Please add other objectives you feel to be Important outcomes of high school instruction in band and orchestra. APPENDIX B PILOT TEST NO. 1, FREQUENCY DISTRIBUTION, ITEM DIFFICULTY AND DISCRIMINATION INDICES APPENDIX B PILOT TEST NO. 1 I. Tt>t- fe j n ■* 'A * j'* * i * ,i i 'rti A) «{Mi >) 1 •h.vp fc) 4btvti'P'* t>> t a tu tp * t> 2 2. Thu li rj» b ' j . ’ M j i ' . f I t' h j J '» I ' n u ' , 1* : Vm 3 a . * . i‘ c r ! . » r-£— ---- m tf) £■ r>l T.~ v - T h * f u l l u v ’ ti* r > r *** ♦* to t in - « m J .■ . . i / »■• " * i r * - **■»-■ —* — s i ■■ “* rSr i$=v^rrrz;i^zj(“'—s^rzrzz.— ?. ----------T-J— I---- 1 5. A) r ih a r p lt> C) »> C 0 A z> * * n * o f th tf wbnvu I!ctf ttvt fl A) #(rt t k»y av.v B ») C Cl c D) SI A o f l'. - D) B) . i' m Ji A. &) O ■■iii 120 fi) C-Plett B) A-Ho-tr C) F 0} B-Flat F) /tone of "iAc Qfcc«i>t- ; H-T V- 121 i■ fc. Ili iiCv-fi-.t *ti».;v %ii • i j a k . .•» I * tJ . . . la . 1>4£. U ui.IC'Ctn^‘ . ~e) “ £) t. tlia f i l l o w t a i i raA 'idj * a - _ -l i * % . a— fc ■— a V—1~V-1 a—*1 v> (?) b) W V ■"* 'o ih>- v 'T i.r l » v eTr fcrcrrrzrr l. IJ a T 1 •C. —kJ a k _ - L. * t A) ■> Cl t - f JL*t C I H a I) Kr.il>- o f tb a i l o i t ft) c S, 1h« ■ ij; o ^ l. n < i o f \ u l j > r A) I a lu rp C) to t i u r f i or f!:tn ^ . u ln a ; 1) A C) 3 u ita r p * D) 2 dtwk‘p« or c::a* fc fc J. Tj i in ir.-. l rr.ra n\ J 1J J j; SI}JjTS J; J. j j t>)tLikE.IL'oA E) 0. l,'h*Eh jf :-t* r> Ho* b J J J/ J A) I It I C ) !!atukklan al AM tv « v '. T w t f C..k>■r* • »OtCT.-kW \ I i 1 !! •■} D> I k ,4 -••. 4 Cl i ’ V Ara t> .'.Ta I * 122 -j- 11. Whlrh • ( ik « fa llo w in g j J. | j jj A t Maaaura 1 ■1 M m m n 9 C) Ha— ro 3 17. Which H d u t M Whfah M t n m c o n ta in h a l f o ta p a f D> K ra n u m 4 Ej A l l a ra o o rra a k c o a ta ln a h e la a t* p a t A) Haaaura 1 0) Maaaura I C)Hcaaura 3 14. 0 ) Haaaura 4 B) N e i i caw t a in « t o l» atapa Ttip f t v a c a j r r p v ilo f a o f a n ( c c h ro o o lo ® te a tlp f r a a A .B . 1433Co p n a a a t tlia a a r a i A) A) C) D) B) 13. \ r m t>) Maaaura 4 B) A l l a ra o a r ra c t 4 ) Haaa*t% 1 B) K ta a u ra 7 C) H t u u n 3 13, i w i i u r i i l i In e n rra ttl C la a a lc a l, l i m u a , P anataaanc*. I t a a a t l c , 3 0 th C entury fttn a te a a a ca , C la a a lc a l, Barcaue, A c a a a t t c . 3 0 th C entury Pauataaaace, l a re-q u a , C la a a lc a l, K o u a a tlc , 3 0 th C entury M e a la a a n c e , D a m iM , R aa w atlc, C la a a lc a l, lO th Caatw ry* S a u l i i a n c i , R a a a a ttc , garooua, C lc a a fa a l, 3 0 th C entury John V h l.ltp Iowa a I tw o* a t th a aaaa a p p ra a ta a t* c h a r aa4 w ra to I * t h a aaaa a t y la a a i A) Cadi > ) H «qt>cr#o C 't llo a a i-t D) tioaa » f th a above Uj A l t o f lh a above 123 It, hhltfa a f th a fo U o v la * tc u p o e .ra w n u ld io n l l h a l y a a a ic fa ta n r t h ■ t H M l A) »- 7 2. S e le c t th a t e n * w hich M a t e 'f r e c H y d i a - : r " v ** t 'is t a a a o t « ? o • e * r r « a a lA> I) C) Dj 23. a e lo v e r teapo a C aster t M f o th a sn ta tempo th a t i n t n p o a *4 a a p ra a i ’.v r ead wn4 w afreaelvw taC na t ^ ; r i « l w S e le c t Cite ta r s * t-h U h o o c t c o r r e c t ly d e a c r'h e •* p#i » a p o o r f t a r d a i w u r f * . 1' A) I) C) 0) 24. At At At At H ath n t i slo w e r and d j r t t t Stew 4 o m t t t t l a b y t l t r l i Slew Aaaa l i t t l e bp l i t t l a SpaaA up l i t t l a b y l i t t l e out aad t'y ln s e -jt an4 b t c - m la f leu A a r end b ic c n ta a lew der S e le c t th a terw e w h ich M a t c u r r a c t ly d e s c rib e " a lle g r o , p u l pa. A) Moderately fnct ulth h i w i'rai B) C> D) V e ry fe a t w ith h e a v tn rs e M o d e ra te ly fa e t w ith o u t p a re tu a V e ry Cate w ith o u t paaeloa w m iM STOP IS . A) Bj C) D) Z> The Tha The The The botCoat p it c h woo f l a t , a l M I * p t t c li was f l * t . to p p it c h wee f l a t . b e tto a i p it c h eaa i l a r p to y p it c h wee s h a rp - 24. A) 1) C) D) R) The b u tto n p it c h teas f l - j t . The m id d le p it c h waa f l a t . The to p p it c h «wa f l a t . Tha b o tto w p it c h waa s h a rp . Tha ta p p it c h waa sh a rp . 27. A) C) C) D) » Tha h e tte a p it c h waa f l a t . Tha m id d le p it c h waa f l a t . Tha to p p it c h waa f l a t . Tha h e ttc n p it c h waa sharp. The ta p p itc h woe ah a rp . i ite iiii 9 A I '»► scat uti far jjrl i;n i m * +* m j|Fr » i.i Hi mi ft *5 ip ! . jl ’ It irw !! w "'If "Ml u ! r r jr r;rr *r;- M 1+1 • L< I < ti/r n » *% ft« £3S?3 0 t ^ * 1 3 IJESt S ? • f ^ A >■ m ft • ft ft » tsccr * !)«■• ?r a I ViiJ $ « > riri • ^ ^ i »i •••» lilt - *L-i» ; ; ; ; Eiii Mir nwj»i UV t ri m'l m m mm inn fr*i i < • I r>. i lilt e* ■ ■Tl*f H « r l Vtw 1 i» VU|» i'W ili» i »**•• i *»ri 1* VU 125 Itt i . J. . s S it* IT V i_ H9A W ** Vfr grrrr itttt 9• >0Af» Jrtrr • n »> 9 si??? ;;ir r• *i - ;i W 3 iiH i, M M %_ j ‘ l!S* liiii, !>!! <;;;sj i 4 a • » * - - 5 !!!! S l!ll ; ?a i t k ; M 2 ** * 5? il I r i • en»*• « V w £??f W rl* , j**** }i! 1.5! 34* * *■• ^ a*,!i ri? r mi ■► 1111 liii fist tsst ii If »# w 11 mi «ft * « * » <• t s t s !!!! ii 11« *u * •» * *» :? 1 ? il m fc * * ■» * flit ft(I 3? I: * 1 * 2? « n « »w v 7777 **** It II p w ^*■ 1f!I *k•I itil I4*I • P * • m m m ^ iill 127 :i ?* s» I»l>l*j< I* ?? « s m► 1*4,- IUJ. lljlf AH'! ‘ CSt i iJflj Mill fin! it EU» ••t•* • » fe * ► • ■my* ¥ yy W ♦O* 9" i ^i«•* a 8 «« > -*• € II ‘:" if! 1 *I i. ii :u i!i> t » *i :t• m 4 ; * 11 4«ue p SjJT! l-i! r..*i’: If’* Ml * - * - fc • li*;c • c • » e * ►t « £ 3W (It c * f !i|i **»! Ilf! iJ 4* 129 130 FREQUENCY DISTRIBUTION--PILOT NO. 1 Frequency Raw Score -2S.D N - 230 Mean ■ 26.52 Median ■ 25.5 Mode * 28 S. D. - 6.34 -is.D. +IS.D. +25.D. 131 ITEM DIFFICULTY AND DISCRIMINATION INDICES PILOT TEST NO. 1 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 Item Difficulty Item Plecrlmlnatlon 41 23 23 41 61 65 70 49 68 29 42 27 44 62 20 51 34 74 67 64 21 73 27 66 57 76 68 79 71 72 73 56 39 41 49 64 44 42 59 35 46 45 50 64 47 36 50 59 39 37 25 41 15 31 35 24 19 31 6 7 19 10 132 (Continued) Item Item Difficulty Item Discrimination 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 43 20 26 11 40 27 17 64 3 3 87 40 46 10 23 42 56 72 80 33 30 20 22 47 42 34 27 1 6 9 29 31 16 -6 44 11 - 7 6 APPENDIX C PILOT TEST NO. 2, FREQUENCY DISTRIBUTION, ITEM DIFFICULTY AND DISCRIMINATION INDICES APPENDIX C PILOT TEST NO. 2 PART I . 1. A ural 1) 2) 3) 4) 5> The The The The The 2. 1 ) 2) 3) 4) 5) The The The The The 1) 2 I 3> &1 51 An An An An A ll in to n a tio n e r r o In to n a tio n e r r o In to n a tio n e r r o in to n a tio n e r r o n eaau raa are c r r r r o occur* occur* occur* occur* rre c t. in in In in m easure #1. m easure 4 2 . m easure #3. m easure *4. 1) 2J 3) 4) 5) An An An An A ll in to n a tio n e r r o in to n a tio n e r r o In to n a tio n e r r o In to n a tio n e r r o m e a s u r e s ,a r e c r r r r o occurs occurs occurs occurs rre c t. In In in in m m m m 1) 2) 3) 4) 5) The The The The The 4. flu te w as fla t. 133 easure easure 42. easure #3. easure #4. 134 - 2- 6. 1 L*Q'toI 11i 21i 31i 41 i 5)i The The The The The 7. 11i 2)i i 4) The The The A ll 8. 11 21 3) 4 The The The A ll 5 9. J aU O 2 An An An A ll 10 . erro r o ccu rred erro r o ccu rred erro r occu rred m e asu res w ere In In in co m easure #1. m e a s u r e 1*2. m e a s u r e 1i3 . rre c t. JrBo 1) 2) 3) 4 j 5) An An An An A ll e r r o r o ccu rred e r r o r o ccu rred e r r o r o ccu rred e r r o r o ccu rred m e a s u re s w ere in m easure In m easure in m easure in m easu re c o rre c t. #1. #2, #3. #4. 1) 2) 3) 41 5) An An An An A ll e r r o r o ccu rred e r r o r o ccu rred e r r o r o ccu rred e r r o r o ccu rred m e a su re s w ere in m easure in m easure in m easure in m easure c o rre c t. #1. #2. #3. #4. U. * i t - T - t" J 135 -3 - 12 P a l r #1 w a s P a i r 0 2 w as P a i r 03 w as P a i r 04 w as None w e re h 13 E x a m p l e 01 i s d i f f e r e n E x am p le 02 i s d i f f e r e n E x a m p le 03 i s d i f f e r e n B o th e x a m p le s 1a n d 3 B o th e x a m p le s 2 and 3 t. t. t. arc d iffe re n t. arc d if fe r e n t. 14 E x a m p l e 01 i s d i f f e r e n E x am p le 02 i s d i f f e r e n E x am p le 03 i s d i f f e r e n B o th e x a m p le s 1a n d 3 B o th e x a m p le s 2a n d 3 t. t. t. arc d iffe re n t. arc d iffe re n t. a h a lf-ste p a h a lf-ste p a h a lf-ste p a h a lf-ste p a lf-ste p s. . . . . 15 • 1> 21 3) 4) The e r r o The e r r o The e r r o The e r r o r r r r o ccu rred o ccu rred o ccu rred o ccu rred in in in in m easure m easure m easure m easure 01. 02. 03. 04. 16 lri?S The The The Tho e e e e rro rro rro rro r r r r 17 1) 21 3) E x cerp t E x cerp t E x cerp t 01. 02. 03. 18. 1) 2) 3) E x cerp t E x cerp t E x cerp t #1. 02. 03. 19. I) 2i 33 E x cerp t E x cerp t E x cerp t 01. 02. 03. o ccu rred o ccu rred o ccu rred o ccu rred in in in in m easure m easure m easure m easure 01. 02. 0 3. 04. 136 -A - 22 1) 2> 3) 4) 5) One m e a s u r e c o n t a i n s e r r o r s . Two m e a s u r e s c o n t a i n e r r o r s . T h ree m e asu res c o n ta in e r r o r s . F our m easures c o n ta in e r r o r s . No m e a s u r e s c o n t a i n e t r o r s . 1) One m o a su re c o n t a i n s 2j T W o measures contain errors. 3) 4t 5) T h ree m e asu res c o n ta in e r r o r s . F our m easures c o n ta in e r r o r s . No m e a s u r e s c o n t a i n e r r o r s . erro rs. . One m e a s u re c o n t a i n s e r r o r s TWo m e a s u r e s c o n t a i n e r r o r s . T h ree m easu res c o n ta in e r r o r s . F our m easu res c o n ta in e r r o r s . No m e a s u r e s c o n t a i n e r r o r s . 137 23 One m e a s u r e c o n t a i n s e r r o r s . TWo m e a s u r e s c o n t a i n e r r o r s . T h ree m e a su re s c o n t a i n e r r o r s . Four m easures c o n ta in e r r o r s . A ll m easures a re c o r r e c t. PART I I . 24. The k e y 1) sig n a tu re 4 fla ts No f l a t s 4 sh arp s 3 sh arp s 2 fla ts o f A M a jo r c o n t a i n s : o r no sh a rp s 23. The k ey s ig n a tu r e 26. The fo llo w in g m a jo r k ey 1) 2) 3) 4) 3} E -fla t A -fla t F R -fla t None o f The fo llo w in g 27. V isu a l th e o f E M a jo r Is r ^ G T i'iiil F-shnrp 2) 3) 4} 5) G D F None o f th e sig n a tu re is: above m e lo d y 1) la : above in th e m a jo r k e y o f : 138 28. Nome C h e f o l l o w i n g m i n o r k e y 1) 23 3) 4) 53 E C C A None o f th e & bove 29. The k e y sig n a tu re 30. T tio f o l l o w i n g m e l o d y i s B -fle t C E -fle t C N one o f 31. W hich o f 13 2) 3) 4) 5} 32. th e M easure M easure M easure M easure A ll a re w h ic h o f th e of B -fla t in m in o r c o n ta in s : th e m in o r key o f : above fo llo w in g m e a su re s c o n ta in an I n c o r r e c t num ber o f b e a ts ? c o n ta in an I n c o r r e c t num ber o f b e a ts ? 1 2 3 4 co rrect th e fo llo w in g m e a su re s w . j o ji I-s.. • mm 4 t v * V V Us* * \ 13 2i 3) 43 3) sig n a tu re : M easure 2 M easure 3 M easures 1 and 3 M easures 2 and 3 M easu res 2 and 4 • .s l . r i anHf l vA U ; M 33. Which o f the f o l l o w i n g nuMBur i' d c o n t a i n V -O iff 8 i- ill p i u ^j 3) All an I n c o r r e c t nu m b e r o f h e a t * m 1 # • *y i, i n v i .ire c o r r e c t 34. Which measures contain li.it r m r p * ? 1) 2) J) 4i 5) 35. 36, M c .is u rc 1 h iM S u rrN 2 anil 4 M easurea 1 and 3 M easure 4 A ll a rc c o r r e c t W h ich m e a s u r e s c o n t a i n w h o le 1) 2) 31 4j 5) ste p s M easure I M easu res 2 and 4 M easure 3 M e asu res I and 3 None c o n t a in w h o le The f i v e m a jo r p e r io d s p r e s e n t tim e a r e : 1) 21 3) 4j 5) rttc p s? o f m u sic c h r o n o l o g i c a l l y C l a s s i c a l , Ita ro q u e , R e n a is s a n c e , R o m an tic, R e n a is s a n c e , C l a s s i c a l , Ita ro q u e , R o m an tic, R e n a is s a n c e , Ita ro q u e , C l a s s i c a l , R o m an tic, R e n a is s a n c e , U a ro q u e , R o m an tic, C l a s s i c a l , R e n a is s a n c e , R o m an tic, U aro q u o , C l a s s i c a l , f r o s i A .D . 2 0 th 2 0 th 2 0 th 2 0 th 2 0 th 1450 to C e n tu ry C e n tu ry C e n tu ry C e n tu ry C e n tu ry 140 -837. W hich o f A) B1 Ci D) 11 21 31 41 5) 38. fo llo w in g sta te m e n ts A re fa lse ? Ite m Ite m Ite m Ite m Item s s s s s A B A C A and and and and and C C B L) I) W h ich o f t h e f o l l o w i n g c o m p o s e r s w o u ld you m o re w ith a fugue? 11 21 31 4) 5) 39. tlu ' The i n t e r p r e t a t i o n o f I s t h e sam e In J a z z a s In a m a rc h . In g e n e r a l , th e lo n g e r th e n o t e th e g r e a t e r th e e sg > h a sis. I n g e n e r a l , a> a m e l o d y a s c e n d s , t h e v o l u m e I n c r e a s e s . M u sic t h a t I s s o f t i s u s u a l l y s lo w e r th a n t h a t w h ic h i s lo u d . M ozart B eeth o v en Bach B e rn ste in None o f th e W hich n o t e Is lik e ly a s so c ia te above a m a jo r t h i r d a b o v e C? 0 40. 41, W h ich n o t e is a p erfect 2) 5) fo u rth *0 above F? \*Z \ ) >) W h ich n o t e is a m a j o r s e c o n d b e l o w C? W h ich n o t e is a m in o r s i x t h b elo w D? •*) »J 141 -943. B double-flat (bb) Bounds the s a m e a s : 1) 2) 3) 43 5) 44. S e le c t 1) 2) 33 43 b) 45. th e M ore L ess L ess M ore L ess th e above te rm s w h ic h m o st c o r r e c t l y m o tio n m o tio n m o tio n m o tio n v o lu m e M uch m ores lo w ly S l o w dow n l i t t l e b y S l o w do w n l i t t l e b y Speed up l i t t l e by S u d d e n ly slo w e r an d S e le c t 13 23 33 43 5) th e d e sc rib e "m eno m o ss u e d o l c e . " d e sc rib e "poco and s o f t l y and s w e e tly and s o f t l y and s w e e tly and s w e e tly S e l e c t th e te rm s w h ic h m o st c o r r e c t l y c tn o re n d o ." 13 23 3j 4) 5) 46. A -fla t B A A -sh arp None o f a poco r l t a r d and d y in g o u t little and d y in g o u t little an d b e c o m in g lo u d e r l i t t l e and b e c o m in g lo u d e r d y in g o ut te rm s w h ic h m o st c o r r e c t l y M o d e ra te ly f a s t w ith h e a v in e s s V ery f a s t w ith h e a v i n e s s M o d e ra te ly f a s t w ith o u t p a s s io n V ery f a s t w ith o u t p a s s io n M o d e ra te ly f a s t w ith o u t h e a v in e s s d e s c rib e "a lle g ro , pul p e s a n te ." 142 FREQUENCY DISTRIBUTION--PILOT NO. 2 Frequency VX Vo 6f -Jc Af # XV 22] 5^. // a (4 (X io a */ x Paw Score X i 6 i i /c tx N - 424 Mean - 20.12 Median - 19.68 Mode - 20 S. D. + 5.63 /* l( JU) XX X. ax 5» i%- 94 3k a 143 ITEM DIFFICULTY AND DISCRIMINATION INDICES PILOT TEST NO. 2 Item Item Difficulty Item Digcrlmlnation 1 62 80 86 84 56 68 44 65 5 38 30 25 53 38 75 59 52 16 17 52 61 73 59 44 33 28 51 72 70 81 82 46 17 3 10 13 11 22 11 14 11 35 17 23 32 31 36 17 31 18 - 5 21 18 - 1 24 46 61 48 61 48 48 33 33 43 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 26 29 30 31 32 144 (Continued) Item Item Difficulty Item Discrimination 33 34 35 36 37 38 39 40 41 42 43 44 45 46 44 55 57 75 73 65 49 77 77 78 40 66 47 80 44 67 71 20 21 49 71 37 38 10 55 37 40 5 APPENDIX D PILOT TEST NO. 3 (FINAL TEST), INSTRUCTIONS FOR TAPE PREPARATION, FREQUENCY DISTRIBUTION, ITEM DIFFICULTY AND DISCRIMINATION INDICES APPENDIX D PILOT TEST NO. 3 (FINAL TEST), INSTRUCTIONS FOR TAPE PREPARATION, FREQUENCY DISTRIBUTION, ITEM DIFFICULTY AND DISCRIMINATION INDICES APPENDIX D PILOT TEST NO. 3 (FINAL TEST) R O T H CO G N ITIV E A U R A L -P E R C E P T IO N M USIC A C H I E V E M E N T T E S T C o p yr ig ht © 1972 by R a y m o n d R o th 145 146 PART I . AURAL INTONATION 1. Determine If the flute or clarinet la either flat, aharp or in tune with the oboe. 1) 2) 3) 4) 5 2. Determine if the violin or trumpet is either flat, aharp or in tune with the cello. 1) 2) 3) 4) 5; 3. The The The The The bottom pitch was flat. middle pitch was flat. top pitch was flat. bottom pitch was sharp. top pitch was sharp. Determine which pitch is out of tune and whether it is flat or sharp. 1) 2) 3) 4) 5) t j j J J l i * . 4. J. IJJ-uq j I jj j j II An Intonation error occurs An Intonation error occurs An intonation error occurs An Intonation error occurs All measures are correct. 6. In In In In measure # 1 , measure measure #3, measure #4, Determine in which measure an Intonation error occurs or if all measures are correct. 1 m 1) 2) 3) 4) 5) An An An An All 3* 2 4 F f f 7J 1J J' 1 * intonation error occurs intonation error occurs intonation error occurs intonation error occurs measures are correct. in in in in measure measure measure measure #1. #2. #3. #4. BALANCE 7. Determine which tone is too loud for proper balance or if they are all in proper balance. The The The All bottom tone was too loud. middle tone was too loud. top tone was too loud. three tones were in proper balance. 148 -3 8. Determine which tone is too loud for proper be lance or if they ere ell in proper belence. n 2) 3j 4) The bottom tone was too loud. The middle tone was too loud. The top tone was too loud. All three tones were in proper balance. STYLE 9. Tell which excerpt is most characteristic of the Baroque Period. 1) 2) 3) Excerpt #1 Excerpt 42 Excerpt 4 3 RHYTHM 10. Determine how many measures contain rhythmic errors or if all are played correctly. 1) 2) 3) 4) 5; One measure contains an error. Two measures contain errors. Three measures contain errors. Four measures contain errors. All measures are played correctly. 11. Determine how many measures contain rhythmic errors or If all are played correctly. J»ss s 1) 2) 3) 4) 5) (£>1972 One measure contains an error. Two measures contain errors. Three measures contain errors. Four measures contain errors. All measures are played correctly. 149 -412. Determine how many measures c cm tain errors or 1£ all are played correctly. 1) 2) 3) 4) 5) One measure contains an error. Two measures contain errors. Three measures contain errors. Four measures contain errors. All measures are played correctly. MELODY RETENTION 13. Determine which one or two of the transpositions differ from the original melody. 11 2j 3) 4) 5) 14. Example #1 was different. Example #2 was different. Example #3 was different. Both examples #1 and #3 were different. Both examples #2 and #3 were different. Determine which one or two of the transpositions differ from the original melody. 1) Example #1 was different. 2) Example #2 was different. 3) Example #3 was different. 4) Both examples #1 and #3 were different. 5) Both examples #2 and #3 were different. STYLE IS. Tell which excerpt is most characteristic of the Twentieth Century. 1) 2) 3) <£> 1972 Excerpt #1 Excerpt #2 Excerpt #3 150 -5INTERVAL RECOGNITION 16. Tell which pair of notes contains a half-step or if there are no half-steps, 1) 2) 3) 4) 5) 17. Pair Pair Pair Pair None #1 was a half-step. #2 was a half-step. #3 was a half-step. #4 was a half-step. were half-steps. Tell in which measure the Interval error occurs. J.ao !• K i ________ ~_______ 1J 2) 3) 4) 18. The The The The Interval interval interval interval error error error error _ occurs occurs occurs occurs 3* in in in in It* i___ i measure measure measure measure #1. #2. #3. #4. Tell in which measure the Interval error occurs. 1. 2. 3. . 4 M i n i I ' n ij i J-tt 1 2 3 4 The The The The 1 Interval Interval interval Interval 2 . 3. erroroccurs erroroccurs error occurs erroroccurs in in in in measure measure measure measure #1 #2 #3 #4 151 - 6- STVXE 19. Tell which excerpt is most characteristic of the Romantic Period. 1) 2i 3) Excerpt //I Excerpt #2 Excerpt #3 INTERPRETATION OF MUSICAL NOTATION 20. Determine how many measures contain errors In performance or If all measures are played correctly. A A --- j T -i lJ- ^^ f II One measure contains an error. Two measures contain errors. Three measures contain errors. Four measures contain errors. No measures contain errors. 21. Determine how many measures contain errors in performance or if all measures are played correctly. i A P One measure contains an error. Two measures contain errors. Three measures contain errors. Four measures contain errors. No measures contain errors. Determine hew many measure a contain errors in performance or if all measures are played correctly. Je 1) 2) 3) 4) 5) One measure contains an error. Two measures contain errors. Three measures contain errors. Four measures contain errors. No measures contain errors. Determine how many measures contain errors In performance or If all measures are played correctly. f > f l t f E • rr-w. m J.<*a r JHBT >& a: fmamf \ 1) 2) 3) 4) 5) L r Q M - ll One measure contains an error. Two measures contain errors. Three measures contain errors. Four measures contain errors. No measures contain errors. f - i 153 - 8- PART II. VISUAL 24. The key signature of A major contains: 1) 2) 3) 4) 5) 25. 4 flats No flats or sharps 4 sharps 3 sharps 2 flats The key signature of E major is: 1. 26. 3* 4. 5. The following major key signature is 1) 2) 3) 4) 5) 27. 2. E-flat A-flat C B-flat None of the above The following melody is in the major key of: f e P j f f f T F f r r i ? f*r 1) F-sharp 2) G 3) D 43 F 5) None of the above (&) 1972 154 -928. Name Che following minor key signature -17^ | 1) 2) 3 4 5 29. E G C A None of the above The key signature of B-flat minor contains 1. 30. 2. 5* Which note is a major third above C? 1. 2. 3. li. 5* j fcn i iH Ha b ... i. 31. 3* 2. i 3. g i= f t e li. 5. g i Which note ^ 1. is a perfect 2. 3* i te l fourth above F? S» I *Ili;I«a I 1. ? * - 2. 3. 9 w~~t~ia - li* 5* I * « ? ll 155 - 32. The following melody is In the minor key of: 1) 2) 3) 4) 5) 33. B-flst C E-flat G None of the above Which of the following measures contains an Incorrect number of beats or are all measures correct? 1) 2) 3) 4^ 5) 34. 10- Measure Measure Measure Measure All are #1 #2 #3 #4 correct Which of the following measures contain an Incorrect number of beats? 1. 1) 2) 3) 4) 5) 1972 2. Measure #2 Measure #3 Measures #1 and #3 Measures #2 and #3 Measures #2 and #4 3* li* 156 - 35. 11- Which of the following nwaiurti contains an incorrect nxeber of beats or are all measures c o m e t ? !• 2* 3. Um r ir 1) 2) 3) 4) 5) 36. Measure Measure Measure Measure All are #1 #2 #3 #4 correct Which measures contain half-steps? 1. 2. 3. 4* & Measure #1 Measures #2 and #4 Measures #1 and #3 Measure #4 All measures contain half-steps 37. Which measures contain whole-stepa? 2m 3. 4. r r 1. 7 11 2) 3) 4) 5) * r * r 2. 3. I ,J Measure #1 Measures #2 and #4 Measure #3 Measures #1 and #3 None contain whole-steps 4. 157 - 38. The five major period* of music chronologically from A.D. 1450 Co present time are: 1) 2) 3) 4) 5) 39. 12 - Classical, Baroque, Renaissance, Romantic, Renaissance, Classical, Baroque, Romantic, Renaissance, Baroque, Classical, Romantic, Renaissance, Baroque, Romantic, Classical, Renaissance, Romantic, Baroque, Classical, 20th Century 20th Century 20th Century 20th Century 20th Century Which of the following statements are true? The interpretation of • T 3 is the same In jazz as in a march, In general, the longer the note, the greater the emphasis. In general, as a melody ascends, the volume Increases. Music that is soft is always slower than that which is loud, i; 2 3 * 5 40. Items Items Items Items Items A B A C A and and and and and C C B D D Which of the allowing composers would you more likely associate with a fugue? Mozart Beethoven Bach Bernstein None of the above 41. Which note is a major second below C? 1. 2. 3. U. 5. 1 <£>1972 158 -1342. Which note(tt a minor sixth below DY k 1« 2* 3* 4* $m 1. 2. 3* U. 5. 4 ■ l-- £ ^ 43. B double-flat (bb) sounds the same as: 1) 2) 33 4) 5) 44. Select the terms which most correctly describe "meno taosso e doIce." 1) 2) 3) 5) 45. More Less Less More Less motion motion motion motion volume and and and and and softly. sweetly. softly. sweetly. sweetly. Select the terms which most correctly describe "poco a poco rltard e morendo." 1) 2) 3) 4) 5) 46. A-flat B A A-sharp None of the above Much more slowly and dying out. Slow down little by little and dying out. Slow down little by little andbecoming louder. Speed up little by little and becoming louder. Suddenly slower and dying out. Select the terms which most correctly describe 'Snolto allegrosempre staccato." 1) 2) 3) 4) 5) Very fast--always separated, Not so fast--always separated. Very fast--suddenly separated. Not so fast--suddenly connected. Not so fast--suddenly separated. 159 INSTRUCTIONS FOR TAPE PILOT NO. 3 The test you are about to take is not one which you will pass or fall. Band and orchestra students from approxi­ mately 50 other schools throughout the state are taking the test to determine knowledge in a number of non-performance areas in music. Answer all questions. If you are unsure about a question, select the best answer on the basis of your knowledge in music. For questions which require printed notation, both bass end treble clef examples are shown. Use the one with which you are most familiar. Thank you for your cooperation. Question #1. INTONATION. You will hear a tuning note sounded by the oboe, followed by the flute end clarinet. You are to determine if the flute or the clarinet Is either flat, sharp or in tune with the oboe. now the flute and clarinet Here is the oboe . Once again, ell three Choose one of the five answers Example 160 Question #2. This time the tuning note will be sounded by the cello and followed by the violin and trumpet. You are to determine If the violin or the trunpet Is either flat, sharp or In tune with the cello. Here is the cello now the violin and trumpet Once again, all three Choose one of the five answers. Example: Question #3. You will hear a major chord consisting of three pitches played separately and In tune. Then you will hear the three pitches sounded together In a chord, but one of the three will be either flat or sharp. You should determine which pitch Is out of time and whether it is flat or sharp. Here are the three pitches, starting with the bottom, followed by the chord with one out of tune pitch ____ . And again. ______ . Choose one of the five answers. Example: Question #4. The Instructions are the same as for the previous question but with different instruments and a different chord. You should determine which pitch Is out of tune and whether it Is flat or sharp. three pitches followed by the chord. Here are the ____ . 161 And again. ____ , Choose one of the five answers. Example: Question #5. Observe the following melody as you hear it played and listen for correct intonation. Determine in which measure an intonation error occurs or if all measures are correct. first time The melody will be played twice; this is the and again Choose one of the five answers. Example: Question #6. Observe this melody as you hear it played and listen for correct intonation. Determine in which measure an intonation error occurs or if all measures are correct. The melody will be played twice; this is the first time and again Example: Question #7. Choose one of the five answers. . _ u BALANCE. »• j . l * You will hear three separate tones which will comprise a minor chord. After you hear them played together, determine which tone is too loud for proper balance or if they are all in proper balance. First, the 162 three tones, then the chord: again, Once Choose your answer from the four choices. Example: _~ft- -£Vl£rsrt: Question #8. question but changed. The Instructions are the same as Inthe previous with different Instruments and the balance Determine which tone Is too loud for proper balance or It they are all in proper balance. three tones, then the chord: again. ___ ____ First the ___ _________ • Once . Choose your answer from the four choices. Example: Question #9. compositions. STYLE. You will hear three short excerpts of Tell which excerpt Is most characteristic of the Baroque Period. You will hear each excerpt once. This Is excerpt #1 excerpt #2 excerpt #3 Choose one of the three answers. Excerpt #1: Excerpt #2: Excerpt #3: Fugue, Suite In C minor--J. S. Bach Klelne Kammermus Ik--Hindemith Divertimento #1 In B-Flat major— Haydn 163 Question #10. RHYTHM. You will be given ten seconds to study the following four-measure example of rhythm, then the example will be played on a snare drum. You are to determine how many measures contain rhythmic errors or it all measures are played correctly. Here Is the performance Study the example. Once again . Choose one of the five answers. Example: LlilJ Question #11. The Instructions are the same as for the previous question but with a new example. Determine how many measures contain rhythmic errors or if all measures are played correctly. Here is the performance of the five answers. Example: Study the example for ten seconds. . Once again, . Choose one 164 Question #12. Again the Instructions are the sane as for the previous two questions but with a new example. Determine how many measures contain rhythmic errors or If all measures are played correctly. Study the example for ten seconds. Here Is the performance Once again Choose one of the five answers. Example: Question #13. MELODY RETENTION. You will hear a melody, then you will hear it in three different keys. You are to determine which one or two of the transpositions differ from the original melody. once. Each example will be played only Here Is the original example #2 answers. Example: . example #3 Here Is example #1 Choose one of the five . 165 Question #14. Your Instructions are the same as for the previous question. Determine which one or two of the transpo siCions differ from the original melody. example will be played only once. Example #1 example #2 Each Here Is the original^ example #3 Choose one of the five answers. Example: E==a tgaP5 . Question #15. STYLE. of compositions. You will hear three short excerpts Tell which excerpt is most characteristic of the 20th century. You will hear each excerpt once. Excerpt #1___ , excerpt #2____ , excerpt #3 Choose one of the three answers. Excerpt #1: Toccata--Frescobaldi Excerpt #2: Overture for Band--Mendelssohn Excerpt #3: March, Divertimento for Band--Perslchettl 166 Question #16. INTERVAL RECOGNITION. You will hear four pairs of notes either a whole-step or a half-step apart. Tell which pair contains a half-step or if there are no half-steps. First time , and again Choose one of the five answers. Example: Question #17. Examine the melody. The flute will play the melody and you are to determine in which measure the Interval error occurs. Here is the melody and again In which measure did the error occur? Example: Question #18. Examine this melody. The trombone will play the melody and you are to determine in which measure the interval error occurs. Here is the melody . In which measure did the error occur? , and again Question #19. STYLE. of compositions. You will hear three short excerpts Tell which excerpt Is most characteristic of the Romantic Period. You will hear each excerpt once. This Is excerpt #1____, excerpt #2___ , excerpt #3 Choose one of the three answers. Excerpt #1: Excerpt #2: Excerpt #3: Eine Kleine Nachttmislk, 1st Movement— Mosart Serenade for Strings, 2nd Movement— Tschalkovsky Suite #1, Foriane— J. S. Bach INTERPRETATION OF MUSICAL NOTATION The following four question are concerned with correct interpretation of musical symbols. Determine how many measures contain errors in performance. Each melody will be played twice. Question #20. Here is the melody many measures contain errors? Example: . and again . How Choose one of the five answers. 168 Question #21. Here Is a new melody How many measures contain errors? and again Choose one of the five answers * Example: Question #22. Here Is another melody and again How many measures contain errors? 'choose one of the five answers. Question #23. Here Is the last melody How many measures contain errors? answers. Example: k^ci- and again Choose one of the five 169 The remaining 23 questions do not require a tape. will have fifteen minutes to complete them. Take your time in answering. your answers. You Do not hurry. If you finish early, check You may begin with question #24. 170 FREQUENCY DISTRIBUTION--PILOT NO. 3 Frequency IX lo fcaw Score A V S "ZS.D to a H -IS.D N - 111 Mean - 18.18 Median - 17.20 Mode - 1 6 S. D. - 6.36 ti, //j JU X* JU J* V | J A 3f Jfc +ISD *2S.D. 4^ **. v y 171 ITEM DIFFICULTY AND DISCRIMINATION INDICES PILOT TEST NO. Item 1 2 3 4 5 6 7 8 9 10 II 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 Item Difficulty 25 69 59 70 23 33 62 14 55 54 61 68 56 50 15 71 86 33 38 59 71 88 67 63 62 47 55 83 77 63 82 82 81 61 46 58 68 78 68 62 79 75 55 82 55 68 Item Discrimination 28 10 24 13 52 49 13 41 45 14 52 17 21 42 24 17 10 31 3 41 45 - 3 0 38 62 41 45 38 38 44 35 21 38 65 62 66 76 24 45 32 24 7 83 14 45 35 APPENDIX E LIST OF RANDOM NUMBERS FOR SCHOOL SELECTION PROCESS APPENDIX E LIST OF RANDOM NUMBERS FOR SCHOOL SELECTION PROCESS 225 Three-Digit Random Numbers 746 919 745 46 484 49 566 979 782 494 3 185 227 549 467 207 533 152 34 5 212 885 608 299 533 534 775 266 554 863 495 152 234 778 862 410 832 380 622 976 331 86 716 556 320 195 775 597 256 959 797 790 212 779 246 516 974 949 66 479 894 557 774 882 448 721 848 227 422 836 679 211 403 196 210 903 701 920 839 143 900 940 80 893 835 392 292 798 154 101 864 209 492 563 395 35 150 92 402 239 626 595 766 58 38 917 508 648 837 470 85 381 343 421 348 305 927 534 240 655 172 662 541 433 314 884 794 853 725 30 692 995 549 218 588 800 424 549 401 544 148 341 20 263 746 258 177 563 863 359 697 400 475 25 160 857 945 568 304 167 746 552 469 212 643 626 203 92 926 620 846 351 74 139 817 379 542 182 410 588 978 660 64 905 871 761 554 279 844 58 499 164 479 668 255 442 89 634 502 559 360 751 420 435 750 499 631 856 497 525 898 492 13 302 142 510 401 744 921 992 14 487 827 348 619 119 APPENDIX F FORMS AND MATERIALS FOR ADMINISTRATION OF FINAL TEST First letter sent to Michigan high schools: MICHIGAN SCHOOL BAND AND ORCHESTRA ASSOCIATION Dear This letter is written to inform you that your school Is one of 50 randomly selected schools in Michigan being asked to take part in a statewide test. The test Is being administered to tenth, eleventh, and twelfth grade instru­ mental music students through the cooperation of the Michigan School Band and Orchestra Association and the Michigan Edu­ cation Association. It will measure what cognitive knowledge of music our students gain from our band and orchestra re­ hearsals plus a number of comparisons. Please keep in mind the following: 1. You will be asked to have your students take a 40 minute test, that requires no special equip­ ment or "pre-teaching." 2. The test will be given by a trained testadminlstrator. 3. No results of the test will ever be made public. No school names will be listed, not even In the credit section. HOWEVER, IF YOU ASK, YOU CAN OBTAIN INDIVIDUAL SCORES OF YOUR STUDENTS. All I need at this point is an Indication from you that we may test your students. All other necessary Information will be sent to you after we determine a mutually agreeable test date. PLEASE FILL IN THE ENCLOSED CARD AND RETURN WITHIN FIVE (5) DAYS. THANK YOU. Sincerely, Raymond Roth TO THE PRINCIPAL: I sincerely hope your school will be able to take part In this test. Mr. Roth, a Ph.D. candidate at Michigan State University, has devised a test to determine results of instrumental music teaching in our schools. The questions all deal with that subject alone. Naturally, his validity 173 174 will suffer If we do not receive your cooperation. Additional Information is available In the enclosed reprint from the MSBOA Journal. Sincerely, Bruce W. Galbraith, Managing Secretary Michigan School Band and Orchestra Association University of Michigan Bureau of School Services 401 South Fourth Street Ann Arbor, Michigan 48103 (313) 764-8242 175 Text of first postcard: [ | Yes, our school will be available for testing. | | No, our school will not be available for testing. Director's Signature School School Address City and Zip School Phone Please mall to: Home Phone Ray Roth, 3636 Brentwood, Flint 48503 176 Text of letter sent to participating schools: Dear Colleague: Thank you for the positive response to my recent query regarding the statewide test. The overall response has been very cooperative and Is Indicative of the type of cooperation one can expect with MSBOA. There are several points 1 wish to bring to your attention: I need to know the dates your school would NOT be available for testing. Please circle those dates on enclosed card and return to me as soon as possible. I need to begin to schedule testing dates and this is not possible until I have all the cards with all available dates. Each student should bring a soft lead pencil on the day of the test. Ball point pens should not be used. Since each student will have a test booklet plus an answer sheet, It would be advantageous to have a flat surface on which to write. The test need not be given In your rehearsal room--this Is left to your discretion. I shall not administer the tests personally. Test adminis­ trators (college students) are being trained for thla purpose. The test will take 46 minutes. Including Instructions. As you know, however, we should plan for several extra minutes to take care of unexpected situations. There Is no need to prepare your students other than for the above points. We hope to have the testing situation as normal as possible. If you wish to receive the results of your students, please tell the administrator when he arrives at your school. We also would like to test a control group within each school. We need your cooperation If we are to accomplish this part of the test. Our test administrator would randomly select several students from your total school population (5-101 of the total number taking the test from your school--not of the total school population--would be ample)'.. If any of these randomly selected students had ever taken either Band, Orchestra, or Choir In high school or Junior high school, they would be rejected and another student randomly chosen. It would be necessary for these students to be excused from their classes to take the test with your suslc students. As you can see, we need to rely heavily on you 177 to contact your administration on this matter. If they refuse, O.K.: we shall be without a control group from your school. IMPORTANT Sincerely, Return postcard to: Ray Roth 3636 Brentwood Drive Flint, Michigan 48503 Phone: (313) 234-2590 178 Text of second postcard: Please circle those dates in which your school will NOT be available for testing. May 8 9 10 11 12 16 16 17 18 19 (grades 10, 11, 12 only) Time Group Meets 1st Band _ _ _ _ _ _ _ _ _ _ 2nd Band Orch. _________________ Other B /O_____________ _ _ _ _ _ _ _ _ _ _ Director's Signature School City and Zip Please mall to: Ray Roth, 3636 Brentwood, Flint, 48503 179 CHECK LIST FOR TEST ADMINISTRATORS 1. Find the Instrumental director and Introduce yourself. 2. Ask to see the room where the test will be given. 3. Find out if the director wishes to know the results for his school. 4. Determine answers to Variables #58, #59, #60. 5. Check for electrical outlet and the possible location of tape recorder. 6. Ask If It Is possible for a control group (10X of those taking the test would be ample). a. Make the random selection from main office records. Reject any student who has been a member of a Junior or senior high school band, choir or orchestra. b. Make arrangements for these students to take the test with the band and orchestra students. c. If you are testing in a school for more than one hour, It is not necessary to have a control group for each hour--one will suffice. 7. As testing time approaches, be ready with: answer sheets and test booklets. tape recorder, 8. Determine the fastest, most efficient and least dis­ turbing method of passing out answer sheets and teat booklets. 9. Assure the control group they are to answer questions to the best of their ability. Control group masfeers do not fill out variables 57-64. 10. Have students mark answer sheets: Names, then variables. Ask them not to mark test booklets. 11. Begin tape. Be sure they begin with answer #1. 12. When tape is completed, begin timing for a 15-minute test Interval for visual part of the test. 13. Complete school information sheet. 180 14. Give a "three minute warning." 15. Collect answer sheets and test booklets. 16. Later--after you leave, mark In the school code number. Place answer sheets In a separate, marked envelope. 181 CHECKLIST OF MATERIALS HEEDED BY TEST ADMINISTRATOR 1. Answer sheets 2. Test booklets 3. Tape recorder 4. Recorder power cord 5. Extension cord 6. Take-up reel 7. Extra pencils 8. "Variables" sheet 9. School Information sheet 10. Two testing tapes 11. Manila envelopes for answer sheets 12. Know the name of director and principal 182 VARIABLES Item #57 If you are presently In the 10th grade, mark re­ sponse #1; If In the 11th grade, mack response #2; and If In the 12th grade, mark #3. Item #58 Mark response___ . (D-#l C-#2 B-#3 A-#4 AA-#5) Item #59 Mark response (Yes-#1 Item #60 Mark response___ . (I-#l II-#2 III-#3 IV-#4 V-#5) Item #61 If you presently play in this organisation a No-#2) brass Instrument, mark response #1; if percussion, mark response #2; if woodwind, mark response #3; and if string, mark response #4. Item #62 If you have taken ten or more private lessons on a band or orchestra Instrument within the past 5 years, mark response #1; If not, nark response #2. Item #63 If either of your parents have played In a school band or orchestra, mark response #1; If not, mark response #2. Item #64 If you have ever taken a formal music theory course either In high school or Sumner camp, mark response #1; if not, mark response #2. 183 SCHOOL INFORMATION SHEET Nome of administrator ^ _ _ _ _ _ - School tested Addre ss City_______________________________________________ Director's name Number of students tested Test begun at Test completed at Were there any unusual distractions during the test? Yes No PLEASE RELATE Does the director wish to know the results of his students? Yes____ No___ What groups were tested? Band I Band II BandIII Orchestra Describe the attitude of the students. Serious Quiet 1 1 2 2 Interested 3 3 4 1 2 3 45 6 7 not serious 56 7 talkative 4 6 5 7 disinterested Describe the attitude of the director. Cooperative 1 2 34 5 6 Did a control group take the test? How many were in the control group? 7 not cooperative Yes No APPENDIX G SUMMARY STATISTICS— ITEM ANALYSIS OF FINAL TEST *WW(!* 1» -. 1 mi l— iX M (i uki •* » •* ill * 4 * » * » .!* U .f L [ it d » , 4 |i r < i . i* * i n m i * «|i >•***4*0*11 ,'• * t * * i4 0 « * * M ft.v ■»! • & TtTTT .^tm n n 4*J cvl Ml »«».’ij«:tfi:mi I **• *, MJ r / f r / m ; ! IA - * * ! * < £ 1 * U >1 «Jk»> u i ■ u u < ■ • O ft ift« I > ) 0 ( 1 * IMo n a n a n a n Ml -A ■ft ^ * * * * * • IIMPIW 1 «U 1 »> *IUAtAV*O l f t * , 'l l l M l l 1 IA*A» m - («) Ml Ml *»1»*'1*- 1*1U'ltln*lii*1*1> Ml •ao iftj C-l wj (-1 rU*f (— J < 0 * L > « 0 » f Jl*.p*0* J*0» C ««* 0*©» Oi \ ) * l \ 8 M i * l u o r i a n a n ia iiii CM .-,J Ml i-a C-l „i*- *\«» la < |* n *M* n *m » i . < M rn * « > « *« ft ft * *1*11 U V l *1* I n u n u i i u i i 1 CM Ml Ml 0 I I W I 1*1 ***• *1***1 r«! .*) 4«) f i f * f « f » r trtI f * ( If* c tf»r*f»|. i i i i i i » i ft ftM t-ft1 t-> ft 1 J .I.A 184 W- s CO otf u 3 f t * n t n * n * « * n *rtr n * / ( * ft i *• n r , tj) > nn i— n w i — t. 111111111 lowii*(—i* i< w * i^ . t * ft ? I •— ’ft M t]Q —J* Cft.au■'*A ( - i l i j l a< ^ ii* —• { * - ; trmiV* «* «***• r nf, i-i ►> ! I i-i* ’h | I I - « -i • » -• I III -* ft * 1 MU' >titikic**o*a»ei cm’ '»1 *4 tH; , •* •* H\tvs '. Off i^iaik/i , J i-, M i} ti • i;c» i*-®1 1A1 | * t V i * t iA M **• ■« | f t &t l ( * ) , U1 .bl * t » )iM ^13 I M ;* * T r f« j f '* * ; *■! VO M * ‘v n w o1 w o ,►' I —I U, c-l 1-1 I 1 Vl ? sM1 « s 4.« ^ *-•* T ca 4 -i ip l' i-1 tWJ m i H " (-i * m fto m i f«r ^ 9U tf# im ' m b t | 3 * S J 3 § I f a ft, u f im ift A ft Ml' IK ' tm iM*Cft**MS .*1■Ml t*1 CM * (ft» 1*1' r*i . •ftB **1 a*u i'-l CM ft) M> T#»l Mi (ft. Ift i« Ml 4 *>• ftfj p f t w tr*; I /O - ; - o > Ift! l-l f-» *-' Ml *-! Ml i-. i*l t-l f~f <-* M l C-J t-f I- ■ c«* £ 1 f X 3 t £ • * 1 3 ! ? 5 3 * * 1 IVl r*t CM (V, ' IO Ifti CM CM •fti CM La J r«a Mi M) Iff t* ift) if 4ft) iftl ^Ift* (•t •«l tftl Cftl tfti ■1*1 tftl c«l •vl Ml ■_ft4 Ml tft l»l tftr ft*; CM Ml Ml Cftl ‘Ml in M £*•j »' Ml1 n ; (ftj ;r> i*« .*% tm. tfti (ft tsi ft*) U>1 INI EV Eh! ► u*l V*' Ml A'1 C c-a . M ft :-» Ml L— ; c-« ► 1 t— -l I-1 Ml :-! Ml (-1» ft! *ftN £ 6 8 a 5 « £ a ft* i 8 ft ft 5 aft v 3 A ft ft 1 .A- Ml ■4, c. eft: tft' CM (a* UV .A, CM i- ; to i- » CM Iftl IK 1 Ml (fl t*: (*l Cft> 4*1 »ft' *. fftl c«: Cftl Cftl (ft) £*: (VI : Ml f ■ tm CM V’ i CM4•> vn •’ (4 i*f* t m i^j <*.• Ml ■r. f .'a, Ml i**i (ft) r*n tftj c*o ■s IN i IN IN IN] . Ml k-. i-i f*i Ml M i Ml .-j l-l *-i -r Ml i-1 M* - 1 c-l r— 8 f 4 3 o IS R 1 a 8 s c a s 8 ft « 8 It 1 Lw -T Ml C#t ;44 ttf. tft) LA Ml - 309 141 i«l M l i--; 141 ft i4fft IfA Ml r- (ft) tv; ift* M- CV1 *•I MS (V. Ml Ift! -■*; iftl ,fl Ml wn l-l M l i tm r- 9*1 VII Ml I"l Ml t«. tftj 1 M S tftl Ml **•« Mk ftl Cftl l«- Ml ftl ftl i^al ft*! (SI MJ f*.i Ln) »».; (O t'j M L -1 Ml l (*ft. M l *•-i C -l l-i M l M* (-1 Ml C-l Ml (-. Ml i- i MO ft 1* ** «* ftf ft R 1 ft f ft £ & £ S s t 1ft t ftt s 3 iftj C tm ftO Ml £41 Ml 4<-i tJO (•■1 MS ivi Ml Ml Ml M3 . Cftl Iftj Cl) l iftl ifti M l If1 Ift) i« i Cftl Mi tftt (41 Ift ,«> Cftl i— *■ rva ( “k fft! i»- i-*" E— I tn * M3 r% M1 «W t-*j *•*1 .*** ft! Mi tftj .s> Cftl ;f.i •ft! 4— iftl 1ft! 1^0 MS. :s* 9 0 ( C'O (■: M l II.) tm r. : E-l v' (-1 . i (-» .- : Ml *-: Ml -1 C-l t— - l-l *- : I-) * R a t-a mt * £ 1 «ft 1 J 1 ftft 3 S > 8 f * X 3 j i«) mj wt ift tm m Ml M (ft! IN t*o IN »a ft*1 ad M l ( M INJ Ml I^i Ml u*j Ml 1*0 «—1 M ) 1 -1 C -l b -1 M l C -l Ml c -> i-i M l M i M l (-1 Ml 1-1 to i-l t - « ft ft • pft* X nft • t & 8 5 * I <.k * * i M » H UK* iUlllvn m u ft « m i i*: f*l M l ^ M l m i - ' w W i v) 1 *1 M i M l N 1 m i r-i M l M : (ft u ti M l cm ! I i n m i O IQ o n m u n n a l M i l m ‘ I** *m ------- mmimtontim m J 1H0U03S' im m K W 103 » unoA z t o APPENDIX G SUMMARY STATISTICS--ITEM ANALYSIS OF FINAL TEST Rtra Index of Difficulty Index of Discrimination Maxinun) Discrimination Discriminating Efficiency Point Biserial Correlation 1 25 35 55 63 .41 2 59 46 82 56 .44 3 68 25 65 38 .29 4 74 19 53 35 .26 5 19 38 46 82 .49 6 50 45 97 46 .44 7 23 3B 54 70 .46 e 32 19 69 27 .20 9 55 30 96 31 .32 10 65 20 68 29 .20 11 62 45 83 54 .47 12 71 24 58 41 .30 13 48 23 97 23 .24 14 39 49 81 60 .48 15 21 23 44 63 .37 16 39 47 81 58 .48 17 51 52 96 54 .51 18 40 35 81 43 .37 19 48 6 100 5 .06 Index of Difficulty Index of Discrimination Maximum Discrimination Discriminating Efficiency Point Biserial Correlation 20 55 31 89 34 .30 21 68 49 71 69 .54 22 58 39 89 43 .40 23 51 44 100 43 .43 24 67 49 81 60 .58 25 65 59 87 67 .68 26 51 51 95 53 .52 27 69 46 74 62 .58 28 84 29 43 67 .59 29 80 23 45 51 .44 30 67 52 80 64 .63 31 85 20 34 58 .45 32 82 22 40 54 .42 33 84 26 40 64 .53 34 50 55 97 56 .52 35 49 43 97 44 .45 36 56 58 98 59 .62 37 72 59 71 83 .74 38 77 19 51 37 .31 (Continued) Item Humber Index of Difficulty Index of Discrimination Maximum Discrimination Discriminating Efficiency Point Blscrlal Correlation "T" of Point Biserial Correlation 39 74 24 56 42 31 40 54 43 97 44 44 9.31 4.12 2.19 78 22 52 42 42 62 11 37 29 20 7.23 43 50 68 94 72 66 2.33 44 74 15 53 28 18 6.86 45 49 39 97 40 40 7.34 46 63 34 76 44 37 5.95 187 41 37 APPENDIX H RESULTS OF INDIVIDUAL SCHOOLS APPENDIX H RESULTS OF FIFTY-ONE SCHOOLS School K. R. Reliability 1 87 88 2.85 2 86 102 3 81 4 Mean of School Standard Deviation Control Group 44 7 24.88 7.85 Yes 2.94 43 9 23.43 7.76 Yes 91 2.99 40 7 22.57 6.92 Yes 85 60 2.88 38 9 22.39 7.57 Yes 5 83 66 2.95 41 7 22.22 7.18 No 6 82 58 2.94 38 8 21.79 6.94 No 7 73 50 2.95 38 9 21.65 5.73 No 8 83 109 2.98 42 4 20.77 7.24 No 9 79 43 2.95 34 11 20.58 6.49 Yes 10 82 101 3.00 38 5 20.32 7.11 Yes 11 77 16 2.90 30 10 20.18 6.01 Yes 12 81 80 2.95 38 9 20.09 6.82 Yes 13 87 106 2.94 44 5 20.00 8.12 Yes 14 71 108 3.06 34 9 19.98 5.68 Yes 15 85 108 2.97 37 7 19.88 7.55 Yes 16 82 24 2.93 39 11 19.66 7.00 No No Standard Error of Measurement High Score 85 17 2.84 34 7 19.29 7.31 18 69 33 2.90 35 9 18.96 5.19 No 19 70 42 2.99 31 10 18.90 5.46 No 20 78 112 2.99 42 8 18.88 6.44 Yes 17 168 Low Score Nuober of Student* (Continued) School K. R. Reliability Number of Students Standard Error of Measurement High icore Low Score Mean of School Standard Deviation Control Group 21 79 52 2.87 40 9 18.65 6.21 Yes 22 59 8 2.80 27 14 18.62 4.37 No 23 75 114 2.97 37 7 18.42 5.98 No 24 74 39 2.89 32 4 18.41 5.70 No 25 64 12 2.86 31 14 18.41 4.75 Yes 26 76 87 2.94 35 8 18.33 5.95 Yes 10 18.27 4.85 Yes 27 64 37 2.93 34 28 79 44 2.89 33 7 18.22 6.37 Yes 29 84 33 2.87 35 7 18.18 7.17 Yes 30 59 23 2.87 26 11 18.04 4.49 Yes 31 72 75 2.99 42 8 18.02 5.66 Yes 32 78 45 2.93 35 7 17.86 6.19 No 33 70 50 2.92 33 8 17.33 5.35 Yes 34 85 19 2.92 37 8 17.31 7.46 Yes 35 80 51 2.89 40 8 17.21 6.41 Yes 36 63 70 2.92 29 7 17.11 4.80 No 37 81 71 2.93 33 4 17.08 6.81 Yes 38 67 31 2.96 30 9 17.00 5.19 Yes 39 71 40 2.84 30 7 16.92 5.26 No 40 80 55 2.92 36 7 16.72 6.51 Yes £ (Continued) School K. R. Reliability Rvnaber of Students Standard Error of Measureaent High Score Low Score 46 46 2.86 23 10 Mean of School Standard Deviation Control Group 16.62 3.90 6 16.43 6.64 No 8 16.29 5.03 Yes 8 16.19 4.08 No 31 9 16.16 5.61 No 2.89 29 10 16.06 4.51 Yes 45 2.91 25 8 15.61 4.34 Yes 68 40 2.83 34 7 15.14 5.01 Yes 49 46 18 2.78 22 9 14.50 3.77 Yes 50 55 30 2.83 21 6 13.89 4.22 No 51 44 26 2.84 21 7 13.73 3.81 Yes 41 81 25 2.87 33 43 68 37 2.86 27 44 50 47 2.88 25 45 73 30 2.89 46 58 30 47 55 48 No 190 42 APPENDIX I FACTOR ANALYSIS— FINAL TEST APrnroix i »-T»TIO% 5SLJTIM Tfttfi JrAV s ' : *T1 J* . : M t T S ! i . -C T*? r " r* c» j* r i d . 1 T \* - z ' \ 1 * fc • i ' ‘- * -.1 * * 7 ..lie * -..1 * 1 .U 't t .1 1 * * * * z H -, - ■ 1 A» * 1 i* * 1 ' f *. ii * ir , W fc, 1* ’ ‘ fc, ;* \ A** i 1 ; t \ fcfc. 7 i* 5 ■, *'■. 4 n i’ i' ,: i* : .1 4 4 2 •i* -: - . : : s* .1 2 .9 .15*5 ..7 't • . 16 ' 6 .7 '! 5 - . ‘ * ') ..7 * 1 -.1 1 7 4 . .tn . 1*- 4 *'7 ;• - .s7 * t 1* . i ; .2 .• c ,7 1 1 1* - .C S 'l - *7 -.1V*1 1; ..7 4 3 1= .:: u 71 71 •i *• it 27 v ifi. 7 7 ------------- . ! ? * : *1 l* H r 21 2* 1*9. 24 j - .1 * 0 .I t '4 -.1 * 0 5 * .S 2 ?l .1 1 7 1 -.*1.71 • .t i n ~ J* wMf 2* .1 2 1 9 . *2* Yiff, 31 ,0 (3 *.»42t .11*8 .125* •;ii4 4 .1)5) *.14 53 .lit ) .143* ;t247 .821* .13*2 .1377 -.1**1 .1 )** . if) *,'.47. .1J3( .lit* -.:* :i - . c;74 • 5143 * .(*75 .119* MM1 .11*7 .1*74 .>14) .9 75* -.lie ) .:=j4 .114* .1*31 tti’ i -.:i4 j -.1**4 .1*71 -.7297 .11*4 *.111) *.■51) -.3 4 *j *4]]*S .1214 .175* 4.11*3 .i:*7 . 21)9 .12*2 .1111 ■.17*9 •J17J4 lU S L * -,1 **3 .227* .171) .7177 -.71*4 .lis t •.14*8 - . Ji7t .131* -.5772 •■1*77 ».1«7# .1**2 .7/11 - -.:i’i ,2.4a-.0 i* . - a; 1-3 ,3434 .***2 * •»CI*I -.1 2 1 1 -.1 5 .) .1 H 4 . m i *.117J .7414 *.01*) •H U .1**3 *•810* '.) ] * • .8177 * .i* n *.D * i 5131* • 81*1 .n n *;«t*i» •.» *3 l .m i • 19*7 -.H 3 1 •:»7i5 4 ,1 **) * .•)* » .IW l ,233* .1173 1)729 ..I2«^ .m m t*l» * * 481*1 • i« m .,n i) .ID * •;i*8B .119) *» t» 5 .M l} *.25*7 • 1482 -.1**7 — 4 jS I* J .171* *.n*a -.211* ■ J j4 |» -.11*1 -.11 ** •O il -" .S 1 I3 — ---- - i l l * * .,11*9 -.:m • +37?4 -.1 1 3 1 .t»4| -.1771- -.-191 .1 1 9 ) .1195 - .o '; -.7*15 -.1 1 1 2 .131* .8243 .U 3 * -.1 2 1 1 *.117* -.1913 •ii* * * .+ l‘ 2 - — - - |W M + **, - 2 * i* -.1*11 ,1.49 .1 * 2 1 ;•**» n c io * .12*2 .C l.'J -.* 4 1 1 - • .*754 -.1*21 -.2 7 1 4 rtctua .71*1 .,1?) .ti 't * - .: 2 * i .t its .J1 3 3 ~ neTija .7443 .1*44 -.1 1 : * 7 -.139* . I 4 ** .■<*1 .1 0 7 -il,, ,D -.7 1 * 5 n s T j* .15*1 -.1 1 ’ ! -.:i* : .> ««• * •i:*.7 .-4*1 - ,1 V * ,:« !! -,:r* l s f*Crus ■*t;4. -.7 4 ; 7 2* -Z * -.* 9 4 2 -.5 4 * 9 * ..2 » 5 ..i* r a * TJ» -.3591 . - * * ’7 ' 3 7 ; «. e 11.H I * t « : ' Tt *T'' i — -.tS7T .1*11 ,t)*t in n ,» w -sr ^ .*» *) m u Vl«l* 4 ll* M • ;* i» * m e * Bate* . m t * fate* - m i»ro* - « « ' — Bite* ;s!i‘ m * bii'i cm:- (ill* Idlt* Hff* lit** ttit*. n t ii* t t ir * m t' M v* tin * tcii* Jitt** »nu*. ♦ Hi* ii'tUf.- - its*.'— /tig* )*Z*- p3 i1 1 * 4* Uti' >.13* (ift* titl*- sja-, *- r t.&l* iait" r / z -.- * i*f:*- 9* * ‘i* A >» d i l ? ’ tttT* m - * *-.a:** it * •!* * i* *.»:•- ta+I* till* ‘( H £9 t/H** t;c3* liat* *tri> l> ta/3*. it£y* «»:■• Vi«l* Ctil*- tig:- Tit-!* !««•• it« * i/s.** ltd1* r n i'» W J * » M PT* zll“ . ;» Sat:* lit Jf of tail** t a i l -* uti' ism** £*»;•• t in* «.!:■ 6^t : - i a t : -* *E *i 1 - it t » B 5* B i t * -* T;TI*. itsB*- !;/'■ sit:- f t .fl* •ft°a** if V T if f* fi;•iitf* H U * » ! 5 I' UIi*- • £ 0*1 ‘ s«ii' HH* i s i i -* Bttl*- t»n** tktt* PatC* t i * 5* iatv t t ii ! otic** n ir > vjti*. tilfl*- lift' S 5S T * eiBi* iftc1 - site** fBS)** *»afa* » i f 0* TttB*. t/St* lit:* •far*l * tilt** ft t l - Tia e -- f/iS*- «•£!*- ttt!*. lit)* ttigt* t t»r ~ t M T* * d l l! ♦hi* l i t I!- ■ m o * * w lit)! C)lg‘> Bill- o»»e* iiig*- itll' — HU! (°i) • T i 7 : -- Bale* Ilii* ♦»H** fit)* i» Sift** tcrt* <«()*• r Igli'- ♦raa* hti'< **' lit/*- Bit** *t«* .- far nit'- -■* mi*- :,t:* e t * •j'-af n«T / tan -— [,tv tui! - £//:•— atti* t»R* M l ' • ill* Tate* ttii* - ----tat>* tm * ttii' - (Sl3* tiii7> Itii* tlti'. 192 i(ii- g i t )1 < i ;** ilf'*- *(!=• st.t* . . . ittt* «itt* :*- u u :> *-l i «9 s i ’ -* -r 1 ■:* Cl »ft>* :.;:* »? * i# ' it ill!*- £f«l* it :tti* If” flf • .\ *r it ar‘t\ bt * Ji'-. * * • 5* 7 .*«»i'** *t:r*. •l if ' *T\ 17 ' *1 ^ tZ sttu* lift* tt»t* l.l»* ttti* :z till! till* :*»•- Till' itlt*- tii:*- iz MCI'* itt|*.* Ttt^*‘ til.*- iilt'- tit*** it«:* :c (0 ) (>') &) it }>tl* cell* #!) *• / * z* TI|C** w T* * "! * 1 . n t f - - '* Zt S i : I* I I I ! 1- - ’* >r { £ 11* tit:** tttf ♦ i tr » * «: ■■* Sill* Ittt* «rt*» t m : ?t«ti* (•{I- *»tl* m t * M f . ‘ (ITf* m t * it ( I H 1 - n - i«:t*. •frit1 u twe;*- £t 9» i ;*■ m t * 1- •a* • V itti* M r * l ti®»* s « f t>»»* - M ** 1 * 7 1 ’. 1 •iti** . ** J V * *.f * i £»Ti* l T c * * . r* H lf' M (") ir **■1 \