‘—w— THE 93‘35.3“\4&‘5“ CF ;.ESTEMEN$ CG-Mf’iliifiifigifi‘fi T335750 FCR MCEEGAN $732.?5 COLLEGE FRESfiMEN 312m {at m Dayna of Eé. Z}: MICHEfiM-i‘ $TA‘E‘E COLLEGB Slycée Wakon Saw "£952 0-169 This la to certify that the thesis entitled The Development of Listening Comprehension Tests for Michigan State College Freshmen presented by Clyde Walton Dov has been accepted towards fulfillment of the requirements for JAIL—degree inwucation Major professor Date Angget 1. 1252 REMOTE STORAGE PLACE IN RETURN BOX to remove this checkout from your record. TO AVOID FINES return on or before date due. ' DATE DUE DATE DUE DATE DUE 2/17 208 Blue FORN S/DateDueForms_ZOI7.indd - pg.5 THE DEVELOPMENT OF LISTENING COKPREHENSION TESTS FOR MICHIGAN STATE COLLEGE FRESHMEN By Clyde Walton Dow A THESIS Submitted to the School of Graduate Studies of Michigan State College of Agriculture and Applied Science in partial fulfillment of the requirements for the degree of DOCTOR OF EDUCATION School of Education 1952 H 5813 -‘ ... -:-- v;‘_ “.5. a »l I a a . r .n: - v“.| U, I a: H C . oval-n u‘ 'Ag ’0». j "V “-5.. I “: 3"» “' v..-, u D O. ‘ rh- s.‘ h‘ae‘ . I.'...‘:. ~~. 3: “.'Q \ § “a. C an: ‘ b'. e._ 0‘ i ,. a. . ‘s .5 “~ '90 . 0‘. ‘l 1" § ‘| -_ ‘b e I'L e u... 'e -. ' I I . \.~' ._ ‘ v ACKNOWLEDGMENTS The writer wishes to eXpress his sincere thanks to the many persons who have given their time and effort so gener- ously to assist him in the various phases of this study. Their assistance has been eXpressed in numerous ways: far too numerous to recount in complete detail. For advice, suggestions, guidance, and physical aid in carrying out the various parts of this study, the most appreciative thanks are expressed to: Professor George Angell for assistance in planning the completely auditory test. Professor Victor Noll for advice in the development of the completely auditory test. Professor Merton Babcock, Professor Charles Irvin, and Professor Arthur Heilman for suggestions for setting up the first study. Professor Paul Bagwell for permission to use departmen- tal facilities in carrying out the several parts of the tests. Miss Maxine Eyestone for recording the completely audi- tory test. Professor Merton Babcock and Mr. Judson Perkins for per- mission to have their classes assist in writing the prelimi- nary test items. 3 0 {$3125 l ass 5 v , n u .. Pa n. e. I... Rub «av a \ 3 e In: an: A u ‘U I ‘ r I H .C.-S . 11 Mrs. Edith Kaiser, Mrs. Anita Benson, and Miss Jean Heron for duplicating materials. Professor Osmond Palmer, Professor Paul Dressell, and Professor Jack Sohmid for assistance in developing the seven foils. Professor John Auston, Professor Merton Babcock, Pro- fessor Osmond Palmer, and Miss Beatrice Hartman for evalu- ation of the preliminary form of Test 1. Professor Charles Irvin for assistance in classroom trials of the preliminary forms of Test 2 and Test 5. Mr. Robert Jackson, Mrs. Mary Edgerton, Mrs. Janice Sines, and Miss Gwendolyn Norrell for scoring answer sheets, computing key analyses, and locating psychological and reading test scores of the students used in this study. Professor Harold Dahnke and Professor Jack Schmid for advice and aid on statistical procedures. Professor Harold.Anderson, Professor Ralph Nichols, and Professor James Brown for evaluation of the tests. Professor Byron VanRoekel for advice and books on the subject of reading, and reading improvement. Hrs. Henrietta Alubowicz, and Miss Ada E. Biester, reference librarians, who obtained theses and lost period- icals needed for the literature surveyed. Professor Milosh Muntyan, major adviser, Professor Clyde Campbell, Professor Whiter Fee and Professor‘Wilson Paul of “: 3a" day 0' ‘ .64 ‘3‘: u: ,- iii the writer's committee, for empert guidance and valuable ad— vice during the entire project. Mrs. Ruth Dow for inspirational and physical aid during the development, execution, and writing of the entire project. The students in the writer's classes who served as ex- perimental subjects, and who gave suggestions and assisted in revising all parts of each of the three tests. -__ "Luv Cal-0'1 /' at» C1 ...al exa:-n ~10? O \1 t l M‘ ..sser.at-on: .m Te: Mn)“ "we of Studi e: :’A.~ ‘- ...,.a,’hical Item t I" am, Septetr.‘ Undergraduatr are dusts Stu #151 .T w '1 163cc: iv VITA Clyde Walton Dow candidate for the degree of Doctor of Education Final examination: August 1, 1952, 10:00 A.M. Dissertation: The Development of Listening Comprehension Tests for Michigan State College Freshmen Outline of Studies: Major subject -- Higher Education Cognate field -- Speech Biographical Items: Born, September 18, 1907, Wakefield, Massachusetts Undergraduate Studies, Emerson College, 1927-51. Graduate Studies, University of Massachusetts, 1955-57; University of Denver, Summers 1959, 1946, 1947; University of Wisconsin, Summer 1940; Teachers College, Columbia University, Summer 1942; Michigan State College, 1947-1952. Experience: Graduate Assistant, University of Massa- chusetts, 1955-57; Instructor in English, University of Massa- chusetts, 1957-41; Assistant Professor of English in Charge of Speech, University of Massachusetts, 1941-45; Associate Professor of Speech, University of Denver, Summer, 1941; Assistant Professor of written and Spoken English, and Speech, Drama and Radio Education, Michigan State College, 1945-47; ~ Associate Professor of'Written and Spoken English, and Speech, Drama and Radio Education, Michigan State College, 1947-51; Associate Professor, Communication Skills, Michigan State College, 1951- uh ~wnn -nn-<1¢ A -.o'.:'_ 4 i ‘mi- ’1'“ n -- --~. nfi‘fi‘? " ‘:‘:‘ y... o-obv can.“ .. -- . VO‘ ube a “see: to t‘fl; .CL "5‘ A“ 'Mu l r cu" .Y‘ fast “‘81 THE DEVELOPMENT OF LISTENING COMPREHENSION TESTS FOR MICHIGAN STATE COLLEGE FRESHMEN By Clyde Valton Dow AN.ABSTRACT Submitted to the School of Graduate Studies of Michigan State College of Agriculture and Applied Science in partial fulfillment of the requirements for the degree of DOCTOR OF EDUCATION School of Education Year 1952 Approved WflWém ’ :erials; (3) Adapt 3:5:53011 go; (4) ~;:';j.s of student? ‘fiftj' minute 11!: station at difre: Pizza: material ' Eterial to be he: ’ A; and. Wed: (10) r: 3:323:33: namely ‘ “3-, "Aviation. vi ABSTRACT OF THESIS BY CLYDE W. DOW Purpose: The purpose of this study was to develop a test of listening having the following characteristics: (1) Limited to comprehension; (2) Limited to expository ma- terials; (5) Adapted to first-term freshmen at Michigan State College; (4) Capable of being administered to large groups of students; (5) Short enough to be administered in a fifty minute listening "laboratory"; (6) Constant in pres- entation at different times; (7) Including a minimum of printed material to be read; (8) Including a maximum of oral material to be heard; (9) Capable of being easily and rapid- ly scored; (10) Meeting the requirements of good test con- struction, namely: a. Reliability, b. Validity, and c. Dis- crimination. Procedure: First, Three passages ("Instructions for Taking the Test“, "A Definition of Listening", and ”Listen- ing Can Be Taught") were prepared. Second, implications from a speech text and four reading manuals suggested seven "standardized” foils as indicative of the major aspects of listening comprehension. Briefly, they are: (A) These concerned with the central idea: (1) Quote. Exact repetition of the central idea. (2) Same central idea, but stated in different words. (5) False, twisted, or wrong statement of central idea. (B) Those concerned with details: U) L: \ .. & "‘ I4, ‘3'6. 38C irc‘eteil, but Tlsted or wrong; :erzec' with both 3:: given nor in; Tnird, passe "ters and stud nts it; to deternin ea "A n- «.3315 revision a: :he final form siest‘ts - T". N; (“Is "“3: and the SC is...“ ! “ «we exercise 4 Q: m 980.11 te 75"“; “'7' (e) C ‘ .1: ~ ) ‘. 4‘. a ‘I 12“”91 ‘ ‘.v‘anA'Ic 4.4“ vii ABSTRACT OF THESIS BY CLYDE W; DOW (4) Quote. Exact repetition of a detail. (5) Same or sim— ilar detail, but stated in different words. (6) False, twisted or wrong statement of a detail. (C) The one con- cerned with both central idea and details: (7) Unrelated. Not given nor implied in the passage. Third, passages and foils were submitted to staff mem- bers and students in various trial forms to obtain items, and to determine their discrimination and difficulty. Nmmerous revisions and item analyses were made in arriving at the final form. Results - Three Tests: Printed instructions, printed foils, and the scripts for materials to be recorded (a. Practice exercise, b. Passage, c. Thirty-two items) are given for each test. Analysis: Reliabilities of the tests obtained by the test-retest method were: Test I .70, Test II .77, and Test III .71. validity was studied by several methods: (a) Correlations with two parts of the Nichols Test were found to be .50 for Test I, .51 for Test II, and .81 for Test III. (b) Two of three experts reported favorable inspectional validity. (c) Correlations with A.C.E. psychological scores showed .44 for Test I, .51 for Test II, and .52 for Test III. (d) Correlations with A.C.E. reading scores were Test I, .55, viii ABSTRACT OF THESIS BY CLYDE W} DCW' Test II, .59, and Test III, .61. (e) Sex differences were found to be lacking in significance. (f) The tests were found to emphasize factors pertinent to comprehension as taught in the course and as derived from the speech text and reading manuals used in conjunction with the course. (g) The present tests compare favorably with other listening comprehension tests in the factors of reliability and valid— ity considered. The average difficulty of the tests was found to be 58.90, 46.87, and 52.45 respectively. Therefore two of the three tests, in their present form, meet the purpose for which they were developed. TABLE OF CONTENTS ACKNOWLEDGMENTS........................................ VITA................................................... ABSTRACT............................................... TABLE OF CONTENTS...................................... LIST OF TABLES......................................... CHAPTER I. INTRODUCTION..................................... A. Listening in General........................ B. Listening at Michigan State College......... C. The Need for An Instrument to Measure Listening Skill........................... D. The General Statement of the Problem........ E. Factors in the Course Influencing the General Nature of the Measuring Instrument and the Specific Statement of the Problem. F. Factors of Listening Influencing the General Nature of the Measuring Instrument and the Specific Statement of the Problem..;...... G. Specific Statement of the Problem........... H. Definitibn of Listening Comprehension....... I. Basic Assumptions Involved in the Study..... J. Preview of Organization of Thesis........... K. smary or ChapterOOOIOOOOOOOOOOOOOOOOOOOOOO ix PAGE iv ix 10 13 14 21 22 23 CHAPTER II. A BRIEF SUMMARY OF TESTS MEASURING LISTENING COWREIENSION...0.00...OOOOOOOOOOOOOOOOOOOOOOOI A. B. C. Studies in Which the Development of Listen- ing Comprehension Tests Have Been A Rela- tively Minor Part......................... Studies in Which the Development of A Lie- tening Test was A Significant or Major Part0...OOOOOOOOOOOOOO0.0000IOOOOOOOOOOOOC smary or Chapter...OOOOOOOOOOOOOOOOOOOO... IIIOTEDEWLOPMNT OF THE FOILSOOOOCCOOOOOOOOOOCCOOO A. B. C. The Development and Failure of A Completely Auditory Test............................. The Concept of Printed Foils................ Implications From Selected Literature on Speech as A Guide to the Development of The Foils................................. Implications From Selected Literature on Reading as A Guide to the Development of The Foils................................. Influence of Two Articles on the Development of The Foils.............................. The Assumption of Minimum Reading........... Presentation to Staff Members............... Presentation to Students.................... PAGE 25 26 50 32 55 55 39 4O 42 53 54 56 56 xi CHAPTER PAGE I. The Seven Foils Used in The Tests........... 57 J. Summary of Chapter.......................... 58 IV. THE SELECTION OF MATERIALS FOR THE PASSAGES...... 61 A. Selection Methods Used by Some Other Studies 61 B. Selection Methods Used in This Study........ 62 C. The Three Passages............. ..... ........ 65 D. Summary of Chapter.......................... 70 V. THE SELECTION OF ITEMS IN THE TESTS.............. 72 A. Method Used to Obtain the Trial Items....... 72 B. The Development and Refinement of Items for Test 1.................................... 76 C. The Listening Laboratory Trial for Test 1... 84 D. Selection of Items to be Included in Test 1. 88 E. Procedures Used in Selecting the Items for Test 2 and Test 5 Which Vary From Those Used For Test 1........................... 91 F. Summary of Chapter.......................... 99 VI. THE RELIABILITY OF THE TESTS..................... 101 A. Method Used................................. '101 B. Test-Retest Correlation..................... 102 C. The Reliability of The Tests in Relation to The Reliability of Other Listening Tests.. 105 D. Summary of Chapter.......................... 107 xii CHAPTER PAGE ' VII. THE VALIDITY or THE TESTS 109 A. Validity Studied by Correlation with the Nichols Test.............................. 109 B. Validity Studied by Expert Inspection....... 112 C. validity Studied by Relation to Psychologi- cal Scores................................ 118 D. validity Studied by Relation to Reading Ability................................... 122 E. Validity Studied in Relation to Sex Dif- ferences.................................. 126 F. Validity Studied by Reference to Course Content................................... 128 G. Validity Studied by Reference to Materials in the Speech Text and in Reading Manuals Used in the Course........................ 150 H. Summary of Chapter.......................... 152 VIII. SUMMARY - THE THREE TESTS 155 Test 1 Responses................................... 156 Emplanation of Responses.................... 159 Emphasizing That Test is to be on Instruc- tions..................................... 142 Passage..................................... 145 ItemSeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee 145 xiii CHAPTER PAGE Test 2 Instructions for Taking Listening Comprehen- sion Test.................................. 148 Responses.................................... 150 Practice Session............................. 155 Passage...................................... 156 Items........................................ 158 Test 5 Instructions for Taking Listening Comprehen- sion Test.................................. 162 Responses.................................... 164 Practice Session............................. 167 Passage...................................... 170 Items........................................ 172 BIBLIOGRAPHY........................................... 177 APPENDICES APPENDIX A. A Description of Tests Used to Measure Listening Comprehension... 191 APPENDIX B. Individual Test Scores.............. 255 xiv LIST OF TABLES TABLE PAGE I. Item Analysis of Test 1.......................... 89 II. Item.Ana1ysis of Test 2.......................... 95 III. Item Analysis of Test 5.......................... 97 IV}.Average Difficulty of the Three Tests............ 99 v; Test-Retest Correlations......................... 102 VI. Mean and Standard Deviation for Test and Retest.. 105 VII. Correlations Between Test and Nichols Test....... 110 VIII. Mean and Standard Deviation of Same Students on Test and Nichols Test.......................... 111 Ix. Correlations Between Test and Psychological Scores......................................... 120 X. Correlations Between Test and Reading Ability.... 125 XI. Difference Between Males and Females on The Test. 127 CHAPTERI CHAPTER I INTRODUCTION A. Listening in General Listening in our society. The citizen in our society is expected to acquire knowledge, skills and attitudes not only for the discharge of his political responsibilities, but also for the purpose of earning a satisfactory living, raising a healthy family, and participating happily in general social and recreational activities. In our society, reading and listening constitute the basic tools of learning as well as the prime media of social intercourse. In the fulfillment of these roles, the importance of reading has never been ques- tioned. More recently, particularly since the advent of radio, the significgnoe of listening is receiving increased recognition. Today, the “average" person in our society acquires in- formation not only through the media of the printed page and the radio as Goldstein mentioned, but also through the in- creased use of telephone, recordings, sound-motion pictures and television. As he lives his regular life he will not only read, write, and talk, but he will also spend a sig- nificant part of his time in listening. He will, according ___‘ 1 Harry Goldstein, Readi and Listenigg Comprehension ét Vari ous Controlled Hates, Teachers 0 ege, on r u one 50 EducatIon, No. 521, Bureau of Publications, Teachers College , Columbia University, N. Y., 1940,1. to one report,2 distribute his time among the several come nunication activities in the following proportions: 9 per cent in.writing, 16 per cent in reading, 50 per cent in speaking, and 45 per cent in listening. Ligtening in education. When our young future citizen begins attending school he will be subjected to a veritable barrage of talk to which he is expected to listen: and from which he is expected to begin acquiring some of the knowl- edge and some of those characteristics which will enable him to operate effectively in our society. In the elementary school he may be expected to listen 57.5 per cent of the total school day.3 At the secondary school level teachers talk, and stup dents are apparently supposed to listen, an average of well above 50 per cent of the time.4 The extremes range from 20 per cent to 95 per cent. 2 Paul T. Rankin, ”Listening Ability: Its Importance, measurement anerevelopment", Chicago Schools Journal, XII (January, 1950), 179. 3‘Miriam.E.‘Wilt, ”A Study of Teacher Awareness of Listening As A Factor in Elementary Education", Ph.D. Thesis, Education De artment Research Contribution, Pennsylvania State CoIIege, State CoIIege, PennsyIvanIa, 1949, 5. D.W. Morris and A.W. Huckleberry, "The Student Teacher's Speech", guarterlg Journal of Speech, XXIX.(December, 1944), 48 - 8 . s is a report of self-ratings from apprgximately 1,000 elementary and secondary public school teac ers. "" 4 No studies of the amount of time college students spend in.listening are available at present, but a casual consid- eration of the number of lectures, discussions, and recita- tions in the schedule of the average college student will suggest that both the amount of time spent in listening, and the amount of information expected to be acquired through listening is not insignificant. B. Listening at Michigan State College Stated course objectives. The importance of listening as a part of education at Michigan State College was given concrete form when the committee responsible for the organ- ization of Basic Education included among the stated objec- tives a specific reference to skill in listening. ”The Report of the Committee Appointed 'for Study and Recommen- dation' Concerning Basic Education at Michigan State Col- lege",5 stated that the broad objective of the "Area of _Written and Oral Communication"6 is: To improve the student's ability to communicate clearly in speech and in writing and also to comprehend spoken and written discourse. 5 ”Report of Committee Appointed 'for Study and Recommenda- tion' Concerning Basic Education at Michigan State College”, Faculty Meeting, May 22, 1944. 6 The name originally recommended for the course was "written and Oral Communication.” This was changed to "written and SpokeniEnglish” at the Faculty Meeting of may 22, 1944. Thgztitle "Communication Skills" became effective July 1, 19 e ‘ 7‘Underline is the writer's. mg the 335°C is followifls T Listen with co; The lists: tires stated 1: karteent of C ’lsbcratory” . lazily lecture ‘5 553) listene 2359:: of the 4 mication. The 71! lecture and c1538 period. T113 method ”direction abou :1 "ale-5y, 31330:? 134.7, “9 head 0 :59 113 t"mi-'18 pr Among the specifically listed objectives of the course is the following pertaining exclusively to listening: "To listen with comprehension and critical evaluation". The listening program. To achieve the listening objec- tives stated in the "original charter" of the course the Department of Communication Skills established a listening "laboratory”. This "laboratory" consisted of a fifty minute weekly.lecture period in which large groups of students (up to 550) listened to forty to forty-five minute lectures by members of the department on some aspect of language or com- munication. The students took notes on this lecture material. The lecture and notes were later discussed in some regular class period. This method of ”practicing” listening without receiving instruction about the skill was generally unsatisfactory. There was little if any difference between this practice, and similar lecture situations which the student attended in Biology, History, or other disciplines. During the fall of 1947, the head of the department expressed a desire to have the listening program improvedt At that time several members of the department, including the writer, met and began a search for studies and teaching materials on the subject of listening. Eventually an improved listening program was developed.8 ‘While a member of this group the writer became interested not only in improving listening instruction, but also in developing a test instrument that would measure the amount of listening skill possessed by students in Communi- cation Skills classes. C. The Need for An Instrument to Measure Listening Skill If listening skill was to be taught much information the needed concerning it: What was it? What were its most important constituents? How much of it. existed in college freshmen? How much improvement could be made by instruction? What kinds of instruction would bring about the greatest ‘ improvement? would improvement be permanent or would re- gression take place? Basic to an understanding of many of these questions was some type.of measuring instrument that would give an indication of the amount of listening skill possessed by the students in the Communication Skills classes. D. The General Statement or the Problem In 1947, a great deal of information was needed con- cerning the subject of listening. One small area of needed information was selected by the writer. He undertook the limited objective of developing a test which would give some indication of the amount of listening skill possessed by 8 See Charles E. Irvin, "An Analysis of Certain.Aspects of A Listening Training Program Conducted Among College Freshmen at Michigan State College," Uhpublished.Ed.D. Thesis, Michigan State College, 1952, for one report of the pro- gram. freshmen students in the Michigan State College Communica- tion Skills program. E. Factors in the Course Influencing the General Nature of the Measuring Instrument and the Specific Statement of the Problem Egposition. The first two quarters of the course in Communication.Skills are devoted to the study and improve- ment of reading, writing, speaking and listening in the area of exposition. The third term is devoted primarily to per- suasion. As a first objective, it seemed desirable to con- centrate on the develoPment of a measuring instrument par- alleling the first two quarters of the course, and thus limit the content of the test to expository materials. ggmprehension. The primary objective of expository speaking and writing as stated in.the Syllabus of the course is clear understanding on the part of the receiver. In view of both.the organization of the course and.the end ob- jective of expository communication, it seemed desirable to limit the test to comprehension. Persuasive communication, which may require critical analysis, calls for a measure of critical listening. The writer proposes to develop such a test, but it is outside of the scope of the present study. gigs, Since the normal listening ”laboratory” is com- posed of groups of students ranging up to 550 in number, the test should be so constructed that it could be admin- istered to large groups of subjects at one time. Time, The first important time factor that influenced the construction of the test was the requirement that a maximum amount of time be devoted to listening instruction and to practice in listening. Those factors suggested that the test should require a relatively short time to admin- ister: It should not consume several periods. The second important time factor was the fifty minute class period. If large numbers of students were to assemble in one room, receive the necessary materials, take a test, and be dismissed in time for the next class, then the test would have to be timed to be administered in considerably less than fifty minutes. The third important time factor was related to size of the total freshman class. If large numbers of students (from 2,500 to 5,000) were to receive results of the test in reasonable time, or if the staff members were to receive results in a reasonable time, then the test had to be capa- ble of rapid and easy scoring. This factor lead to the desirability of an objective examination which was capable of being rapidly hand scored or of being machine scored. Motivation. Because the primary objective of the listening ”laboratory” is to have students improve their listening skill, a motivation toward that end might result if students could know how their listening ability compared with that of their classmates. Stroud has said, The examination holds possibilities for motivating pupils, realization of which will depend upon the way it is used. . . . Knowledge of attainment based upon an examination whose validity the pupil respects should, in the light of psychological data, be an important motivational force. Properly used, the examination may also possess diagnostic value in that it infgrms the student where his strength or weakness lies. This factor of possible motivation suggested that the test might be short enough to administer and.easy enough to score to permit students to both take and score the test within the fifty minute ”laboratory" period. Student level. All regular freshmen students are re- quired to enroll in the Communication Skills course during their first quarter at the college. The general level of student development is therefore representative of a rela- tively uniform group. There are some transfer students, and some more mature students, as well as a few foreign students, but these latter constitute a small percentage of the total. In view of the predominance of ”typical” first quarter fresh- men in the course, the test material was prepared for begin- ning freshmen students. ___ 9 James B. Stroud, Ps cholo in Education, Longmans, Green and 00., Inc., N. Y., l§1§, 373. 10 F. Factors of Listening Influencing the General Nature of the measuring Instrument and the Specific Statement of the Problem The act of listening. The process or the act of listen- ing as a part of the process of communication is an extremely complex one. In essence, some stimulus within or without one person causes a sensory impression which after going through pro-verbal states takes on symbolic forms which are trans- formed into verbal formulations (a) air waves and (b) light waves, which stimulate the listener or receiver who in turn, through a similar complex process, in his own nervous system, translates these stimuli into meaning something near that of the person.who spokze.lo Too simply stated this means that listening is concerned primarily with the aural (ear) re- ception and translation by one person of oral (spoken) language symbols (words) of another person. The implication of these factors for the construction of a test to measure listening meant that a speaker as well as a receiver had to be involved. This speaking could be done in at least four ways: (1) The speaker actually could be present, (2) A sound-motion picture could be made of a speaker, (3) A recording could be made of a speaker, or (4) A speaker lo‘Wendell Johnson, ”The Spoken'Word and the Great Unsaid", Quarterly Journalqof Speech, XXXVII (December, 1951), 42. 11 could talk over a public address system and not be visible to the listeners. Standardization. To prevent variations in presentation of the spoken material that would normally result from having a speaker deliver the same test material ten differ- ent times in each listening "laboratory" the test had to be presented either on sound-motion picture film or by record- ing. Primarily because of the expense involved in producing synchronized sound.pictures, the test was recorded. The recording on tape or records maintained a relatively stand- ard presentation for all ten sections of the listening "laboratory”. The emission of the visual cues (opportunity to see the speaker) have been shown to result in some loss of comprehensionl1 but the gain in constancy of presentation and.the reduction of cost would compensate for the omission of visual cues. Moreover, since the administration of test materials was relatively constant, an adjusted interpreta- tion of the results could be made if desired. Minimum Reading, Since listening as here used is pri- marily an auditory receptive skill, the introduction of reading material in the form of printed questions or multi- ple-choice answers would introduce another variable into the 11 Edward J.J. Kramer and.Thomas R. Lewis, "Comparison of Visual and Nonvisual Listening," Journal of Communication, I (November, 1951), 16-20. 12 measuring situation. Standard reading tests are administered to all incoming freshmen, and some of the students were known to have low scores on the reading rate and comprehension tests. Furthermore, Goldstein,12 in a very carefully con- trolled study, indicated that some of his college students were definitely superior in their understanding of either spoken or printed language. Although investigators13 have found correlations ranging from .27 to .82 with an average of .70 between listening and reading skills, the less reading there was in the test the more valid the test would be. Therefore, the test was constructed to keep reading materials at a minimum. The ideal would probably be no reading, but that ideal was not attained. Simple answer sheet. In relation to reading and answer sheets both Villarreal14 and K'aulfers,15 although they were writing about aural comprehension of foreign languages, point out the great discrepancy that may exist between understanding spoken language and being able to read it or 12 Harry Goldstein, op. cit., 59. 13 Ralph G. Nichols, ”Needed Research in Listening Communi- cation", Journal of Communication, I (May, 1951), 48. 14 Jesse J. Villarreal, "A Test of Aural Comprehension of English for Native Speakers of Spanish", Speech Mong- 15Walter V. Kaulfers, "wartime Developments in Modern- Langua e Achievement Testing", Modern Language Journal, XVIII 1944), 139-150. Ileutilvnlcn I... i at . u’wfl A e. Lmtarn E. . in 13 to write it. Although the comparison between reading English and hearing English by native Americans is not identical with that of Americans learning a foreigh.lan- _guage, a degree of similar difficulty can well exist between the person.who may possibly possess a high degree of under- standing of what is said in English and at the same time possess a low understanding of what is written in English. Therefore, questions, foils, and answer sheet were con- structed to contain the least amount of reading possible. G. Specific Statement of the Problem The specific problem for this investigation.thus be- came: To develop a test of listening having the following characteristics: 1. Limited to comprehension. _ 2. Limited to expository material. 3. Adapted to first-term.freshmen at Michigan State College. 4. Capable of being administered to large groups of students. 5. Short enough to be administered in a fifty minute listening "laboratory”. 6. Constant from section to section (recorded). 7..Including a minimum of printed material (to be read). 8. Including a maximum of oral material (to be heard). 9. Capable of being easily and rapidly scored. 14 10. Conforming to the requirements of good test con- struction: and therefore possessing: a. Reliability b. Validity c. Discrimination H. Definition of Listening Comprehension The terms "listening” and ”listening comprehension" have been subjected to numerous interpretations. Rankin16 defined listening as follows: Listening ability is the ability to understand spoken language. I do not mean the readiness to pay attention, as teachers often mean.when they say, ”He won't listen”. Rather, I mean the trait that has the same relation to speech that reading has to writing. Ability to listen well is to understand.what one hears. Nichols17 stated: The present study was concerned with aural assimilation when visual and oral cues were both present, when speakers confronted audiences in face-to-face situations, being seen as well as heard, when the time of testing comprehension of speeches immediately followed their delivery, . . . For the sake of clarity, the term listening comprehension is arbitrarily used throughout this paper to refer only to this kind of performance. Similarly, auditory comprehension is hereafter arbi- trarily used to refer only to aural assimilation in which oral cues alone are operative. 16 Paul T. Rankin, op. cit., 177. 17 Ralph G. Nichols, ”Factors in Listening Comprehension“, Speech Monographs,‘xv (No. 2, 1948), 155. 15 In another place Nichols-.18 has further clarified his ume of the term listening by dividing it into three main types: 1. Discriminative listening to informative materials which is limited to comprehension. This is some- times referred to as discriminatory listening. 2. Critical listening to persuasive materials. This involves evaluation as well as comprehension. 5. Appreciative listening to any aural presentation gratifying to the senses. Villarreal19 gave an ”Qperational definition of the Egrm 'aural comprehension", as follows: By aural comprehension of English is meant the ability to extract appropriate meaning from connected samples of the English system presented orally. It is recog- nized that there are many levels of comprehension, de- pending upon.the type of material presented and the measure of comprehension applied. Comprehension may conveniently be thought of as a continuum, grading from very simple instances (for example, a horse stop- ping on command) to much more complex situations (such as the interpretation of highly metaphorical poetry). This study is concerned primarily with that segment of the continuum represented by the communication of fac- tual information. At this level of understanding, it may be assumed that the listener understands if he is able to say what happened, describe objects and persons talked about, and repeat explanations. 18 Ralph Nichols, ”Teaching of Listening", Chicago Schools Journal, XIX (June, 1949), 274. 19 Jesse J. Villarreal, op. cit., 122. 16 James Brown?0 said: By listening comprehension is meant the aural assimila- tion of spoken symbols in a face to face speaker-audi- ence situation, with both oral and visual cues present. Listening comprehension assumes the ability to hear, but hearing and listening are not to be considered ‘synonymous. The medical doctor's concern is with hear- ing: the instructor's concern is with listening. John Caffrey21 objected to the term listening compre- hension, and proposed instead the term audipg coined by Don Brown of Sequoia (California) High School. .As long as we confuse auding with ”listening”, we shall achieve results comparable to those which we might ob- tain by confusing reading with "looking”. Reading is not ”looking comprehension;” I can comprehend the pur- pose of objects such as revolvers or medicine droppers without being able to read. Similarly, I can comprehend the probable purposes or sources of barking, slapping, or shooting without being able to ”sud”. Blewett22 defined it as follows: Listening is the process of attaching meanings to the spoken word. According to this definition hearing is not necessarily equivalent to listening. One may hear, that is to say apprehend, an unfamiliar foreign tongue, ‘but he may be unable to comprehend, understand, or at- tach meaning to that which he hears. 20 James I. Brown, "The Construction of A Diagnostic Test of Listening Comprehension”, Journal of erimental Educa- tion, XVIII (December, 1949), 115. 21 John Caffrey, ”The Establishment of AudingeAge Norms”, 9 School and Society, 70 (November 12, 194 , 310. 22 Thomas T. Blewett, ”An.Experiment in the Measurement of Listening at the College Level”, Journal of Communication, I (May, 1951), so. Irving:5 de: 0 s . In th new! from {'0 Synonym; You my heal; of 70m. c” mortal“: t co rehens". Listening t for somethi‘ e e ' This ‘! 'reception 1 of era11y p.“ Per Purpos' in considfl‘able {313101133 1. W m afiisn of 80m sand meso W. hermit” or 2 11011 the 0” ° :tion pictures :11} auditory i" L3393310118 (11" 13‘: be present n ending; but V} tépeople can fiz‘lirelent 18: )2 'V” 0 17 Irving:5 defined listening along similar lines: . . . In the first place, listening is to be differen- tiated from hearing, although the layman often uses the two synonymously. Hearing is the awareness of sound. You may hear a foreign language; you may hear the motor of your car. Listening is more than hearing. It is important to remember that. . . listening involves the co rehension of what is heard. Listening to a foreign anguage infBlves the understanding of that language. Listening to the motor of your car involves listening for something: a knock, or a jumping spark. . . . This writer has, .‘. . defined [listening] as "reception and comprehension; reception and evaluation of orally presented materials”. For purposes of the present study, listening is defined in considerable detail. The definition contains six major divisions: l. The first important factor in listening is the re- ception of sound. There must be the physical presence of sound waves. When people listen to the telephone, radio, recordings, or a person not visible, they depend exclusive- ly on the ear. When people ”listen” to television, sound- motion pictures, or a speaker before them, they receive not only auditory impressions (sound waves), but also visual impressions (light waves). The visual element may or may not be present. If present, it is a further aid to under- standing; but visual impressions may be entirely excluded and people can still listen. The primary (but not exclusive) requirement is, therefore, the reception of sound. 23 Charles Irvin, op. cit., 4. 18 2. The human ear is sensitive to a multitude of sound vibrations. An extremely broad classification of these sounds ranges from simple noise to complex musical tones. The term listening as used herein is arbitrarily limited to, those speech sounds produced by human beings. Excluded are all other auditory impressions such as telegraphic code, tom-toms, instrumental music, train whistles, the drumming of horses' hOofs, etc. It is not that these auditory sensa- tions do not conmiunicate meaning to those who know how to interpret them, but that the chief act of listening in com- munication is with these sounds produced by human beings. 3. Human beings produce a wide variety of speech sounds. Humming may be cited as one example that may or may not be colmnunication. The babbling stage of language development in children is another example. In listening we are general- ly concerned with those sounds produced by human beings that we call language symbols. In reading, the communication is received through printed language symbols; in listening the commication i‘s'received through spoken language symbols, or word sounds. Singing also makes use of ”spoken" language symbols, but generally with greater and more abrupt varia- tions in pitch, force and rate; and these variations normal- ly conform to a set of pro-determined notes. However, the major type of human sound involved in the act of listening is the reception of spoken language symbols. 4. In the It seems unneco fills and there and? If no 0 spoken language Listening inclu 5. Observe ll'm 'pay stt ' niches their a: tention', or 'I L’zits. the act ‘: 4' ,I "3‘45 - The 1 'listeningu is ht I “Wt ll 391‘.” the req' me factor in 1 internal Bet" 6' Lifltem a! thg heard 1! CQMpreh 6. I together?“ VOIVBS b t E‘\ 3r}; e Str 2 . a 13:10:18 2‘3 “(,5 ’ *3: ’ 19 4. In.the act of listening there must be a receiver. It seems unnecessary here to debate the argument, If a tree falls and there is no one present to hear it, is there any sound? If no one is present to hear a human being producing spoken language symbols, there is no act of listening. Listening includes a receiver. 5. Observation will confirm the fact that people do not always "pay attention" to the flow of spoken language which reaches their ears. The common phrase, ”I wasn't paying at- tention”, or ”I wasn't listening", suggests that within limits, the act of listening involves some degree of "at- tending“. The distinction between the terms "hearing" and ”listening” is sometimes drawn by the statement, "I heard it, but I wasn't listening”. This latter statement indirectly implies the requirement of attention. It is assumed that one factor in listening is at least some degree of a ”physi- caldmental set'I of attention on the part of the person re- ceiving the orally produced language symbols. 6. Listening, as herein used, includes the comprehension of the heard language symbols. Comprehension, as Latin students know, means "taking together”, or as we would say, "taking in”. This in- volves both memory and the organézation of thought, as well as recognition of words. 24 Ruth Strang, Stud T e Readi Exercises, Bureau of Publi- cations, TeacEers Cottage, CotumEIa Uthersity, N.‘Y., 1935, 1. 20 The common synonyms ”get”, ”catch", ”see", and "understand” tend to convey the idea of "meaningfulness" on the part of the listener who comprehends. The question, "What is the meaning of the term 'reading comprehension'"? brought the following reply from a reading clinic director:i "Reading comprehension involves inter- preting the word symbols and getting the meaning the writer intended. ‘When you define reading, you define comprehen- use” "Comprehension", as Villarrealz5 noted, "may conveni- ently be thought of as a continuum, grading from very simple instances . . . to much more complex patterns. . . ." p In the sense of this study, comprehension is concerned with that segment of the continuum devoted to the under- standing of factual expository material. At this level of understanding it was assumed that if the listener ”got the meaning” intended by the speaker as confirmed by colleagues and students in previous test trials, then the listener had ”comprehended". Summary of definition. Listening may thus be defined as the act of an attentive person, receiving and under- standing spoken language symbols. 25 Jesse J. Villarreal, op. cit., 122. I. Basic Inherent i listening ebili to following a 1. Listen: 'fferent p60“: 2. Listeni ltad among the a! norm probe 3. Any me mm use of WI citing. 4. Listen* “’53: Provide vised. 5. Since either Overt 0‘ can be measure.» 5- The ab: 31' In ”all: p. “1133 the abi‘ that has been Zr (:11: ty to dig: »\' IJ fl .QOmaa Bl _ 323310113 8] 21 I. Basic Assumptions Involved in the Study Inherent in the development of a-test to measure the listening ability of Michigan State College freshmen were the following assumptions:25 1. Listening ability exists in different degrees in different people. 2. Listening ability, like other abilities, is distrib- uted among the general pOpulation according to the principle of normal probability. 3..Any measure of listening ability should involve a minimum use of other means of communication such as reading and writing. 4. Listening ability can be measured in quantitative terms, provided a suitable measuring instrument can be de- vised. 5. Since listening involves some form of response, either overt or covert, the amount and quality of listening can be measured in terms of some observable behavior.' 6. The ability to identify clearly the major thought of an orally presented passage, as well as some of the de- tails; the ability to distinguish clearly between exactly what has been heard and.what has not been heard; and the ability to distinguish relevant from.irrelevant materials, 26 Thomas Blewett, o . cit., 50, listed a set of basic as- sumptions similar to several underlying the present study. 22 indicate the degree of listening comprehension skill pos- sessed.by freshmen college students.27 7. Material which is of instructional value can also be used as test material. 8. Listening ability, as understood in this study, as- sumes the ability to hear: auditory acuity is assumed to be ”normal”. J. Preview of Organization of Thesis Chapter II of this study gives a brief review of the tests that have been developed to measure listening compre- hension, Chapter III explains the development of the seven standardized foils, Chapter IV presents the material selected for the three oral passages, Chapter V reports the method used for obtaining the items used in the three tests, Chapter VI presents the study of reliability, Chapter VII explains the procedures used to make the tests valid, and Chapter VIII presents the complete scripts for all three tests. Appendix A contains a description of tests used to measure listening comprehension. The individual test scores for all students participating in the final phases of the study are given in.Appendix B. 27 See Chapter III, "The Development of the Foils" for a more detailed statement. 23 K. Summary of Chapter Chapter I has presented (a) a general picture of lis- tening in our society, (b) listening at Michigan State Col- lege, (c) the need for an instrument to measure listening skill, (d) a general statement of the problem, (e) factors in the course influencing the general nature of the measur- ing instrument, (f) factors of listening influencing the'. general nature of the measuring instrument, (g) a specific statement of the problem, (h) a definition of listening, (i) basic assumptions involved in the study, and (j) a pre- view of the organization of the thesis. CHAPTER II CHAPTER II A BRIEF SUMMARY or TESTS MEASURING LISTENING COMPREHENSION Nearly all students of listening have reported some of the research preceding their own studies. Nichols,1 as part of his work, compiled a bibliography of studies relating to listening. Schneider2 reviewed some of the literature, and Brownfi gave a brief description of tests together with a summary of the findings. No previous research, however, has devoted space to a detailed description of the teats used in the several studies to date. Because this test material is difficult to obtain, and because it is widely scattered among various periodicals, in unpublished theses, and in mimeographed, unpublished form, the present writer undertook a detailed description of many of the tests developed for earlier research. Thirty-four 1 Ralph G. Nichols, ”Factors Accounting for Differences in Comprehension of Materials Presented Orally in the Class- room”, unpublished Ph.D. thesis, State University of Iowa, 1948, 298 pp. 2 Wallace A. Schneider, “A Review of Some of the Literature Related to Listening", Unpublished MA. thesis, University of Denver, 1950, 86 pp. 3 James I. Brown, "The Construction of a Diagnostic Test of Listening Comprehension”, unpublished Ph.D. thesis, Uni- versity of Colorado, 1949, 184 pp. sheen h‘" be describad- 515 reliability 8"” Ligation '3‘ £13955“ " 3: trial: “wk: :15 those whose :iitted since 3 the lizite of t at listening CC In this of .._...'nensicn ? 31251253 from 6 the body of the cerning the re) Prim-11: in C": tenth]: of on: in: reading ab: 331% descripti end; 1. l. Stud ie: I Comp: 1. 26 studies have been reported, and thirty-two tests have been described. Also details of administration, and analyses of reliability and validity have been recorded whenever such information.vas reported. The material has been compiled in Appendix.t. Studies concerned primarily with persuasive ma- terial, shift-of-opinion, note-taking‘zg. no note-taking, and those whose objective was aesthetic listening, have been omitted since they were considered as being largely outside the limits of the present study emphasizing the measurement of listening comprehension for expository lecture material. In this chapter, the tests used to measure listening comprehension have been classified and listed. Pertinent findings from earlier research have been incorporated into the body of the study at appropriate places. Material con- cerning the reliability of earlier studies is concentrated primarily in Chapter VI; and material concerning the rela- tionship of earlier findings to such factors as intelligence, and reading ability is concentrated in Chapter VII. The major description of previous tests has been placed in.tp- pendix.A. A. Studies in.Which the Development of a Listening Comprehension Test has been a Relatively Minor Part 1. Tests based on adaptations of silent reading and vocabulary tests. Among the studies utilizing silent reading 27 and vocabulary tests are those by.Anderson and Fairbanks,4 Burton,5 Goldstein,6 Johnson,7 Larsen and Feder,8 and Sims and Knox.9 2. Tests based on classroom lectures. Six studies have used a type of classroom lecture as basic materials for a listening test. These are represented in studies by Corey, 10 4 10 Irving H. Anderson and Grant Fairbanks, ”Common.Differ- ential Factors in Reading Vocabulary and Hearing Vocabur lary ' JOurnal of Educational Research, 30 (January, 1937 , 3I73354. nary Burton, "The Hearing and Reading Comprehension of Vocabulary'Among High School Seniors , School Review, 52 (January, 1944), 47-50. Harry Goldstein, Reading and Listening Comprehension at Various Controlled Rates, Gentribfitions to Educationi:flo. , ureau o PublicatIOns, Teachers College, Columbia University, New Yerk, 1940, 69 pp. Kenneth C. Jehnson, "The Effect of Classroom Training Upon Listening Comprehension,” Journal of Communication, I (May, 1951), 57-62. Robert P. Larsen and D. D. Feder, "Common and Differential Factors in Reading and Hearing Comprehension”, Journal of Educational Psychology, XXII (April, 1940), 241- . V. M. Sims and L. B. Knox, ”The Reliability and‘Validity of Multiple-Response Tests When Presented Orally", Jour- nal of Educational Psychology, XXIII (December, 1932), Stephen M. Corey, ”Learning from Lectures ya, Learning from.Readings”, Journal of Educational Psychology, 25 3:eeno,n L056: 5. Tests 1 539ml aituat. anon” and h: hgs, mile Kn anvard B. c: and Individ' Genetic Psv‘ o . 12 Edvin J. Lo 28 Greene,11 Loder,12 Russell}:5 Stump,14 and'Young.15 3. Tests based on special types of material or utilizing special situations. Nelson and Moll16 used sound-films, Roulon17 and his associates used standard, published record- ings, while Knower, Phillips and Koeppel18 used specially 11 Edward B. Greene, ”The Relative Effectiveness of Lecture and Individual Reading as Methods of College Teaching", Genetic Psychology Monographs, IV (December, 1928), 457- 12 Edwin J. Loder, "A Stud of Aural Learning With and With- out the S eaker Present , Journal of erimental Educa- tion, VI (September, 1937), - . - 13 R. G. Russell, ”A Comparison of Two Methods of Learning", Jburnal of Educational Research, XVIII (October, 1928), 14 N. F. Stump, ”Oral Versus Printed Method in the Presenta- tion of True-False Examination”, Journal of Educational Research, XVIII (December, 1928),-I§3¥424. 15 William E. Young, "The Relation of Reading Comprehension and Retention to Hearing Comprehension and Retention” Journal of Experimental Education, V (September, 1936 , 35:39. 16 Harold E. Nelson and Karl R. Moll, "Comparison of the ,Audio and Video Elements of Instructional Films", Journal of Communication, I (May, 1951), 62-66. 17 Phillip J. Roulon and others, ”A Comparison of Phono- graphic Recordings with Printed Material in Terms of Knowledge Gained through Their Use Alone”, Harvard Educa- tional Review, XIII (January, 1943), 63-76. 18 Franklin.H. Knower, David Phillips, and Fern Koeppel, ”Studies in Listening to Informative Speaking", Journal of Abnormal and Social Psychologyk 40 (February, T945), 5’2 '88 e 29 prepared informative speeches. Stanton19 used orally pre- sented advertising material for checking ability to remember trade names. Selover and Porter20 had material from a re- cent book read aloud to subjects. K'reuger21 used a type of printed word list, and Carverzz prepared materials empha- sizing selected words, phrases, and sentences. Studies by Erickson and King,23 and by Spachez4 give no descriptive details of the tests used. 19 Frank N. Stanton, ”Memory of Advertising Copy Presented Visually vs. Orally", Journal ongpplied Psyghology, XVIII (February, 1934), 45-64. 20 Robert B. Selover and James P. Porter, "Prediction of the Scholarship of Freshman Men by Tests of Listening and Learning Ability", Journal of Applied Psychology, XXI (October, 1937), 5831588. 21 David H. Kreuger, "A Study of the Results of Teaching Factors of Listening Comprehension to College Freshmen in the Basic Communication II Course", Unpublished M.A. Thesis, Whittier College, 1950, 176 pp. 22 Merton E. Carver, "Listening Versus Reading”, Chapter IX, 159-180, in Hadley Cantril and Gordon'W. Allport, The Psychology of Radio, Harper and Brothers, N.‘Y., 1935. 23 C. I. Erickson and Irving King, "A Comparison of Visual and Oral Presentation of Lessons in the Case of Pupils from the Third to the Ninth Grades”, School and Sociegy, VI (August, l917), 146-148. 24 George Spache, "The Construction and Validation of a Work-Type Auditory Comprehension Reading Test", Educa- tional and Psychological Measurement, X.(1950), §4§-§53. 30 B. Studies in.Which the Development of A Listening Test was A Significant or Major Part Among the studies in which the development of a test of listening comprehension was a major objective are the follow- ing which have been arranged in chronological and then alpha- betical order: Rankine5 (1930) was apparently the first person who was interested in measuring listening for itself rather than for some related problem. Sullivan?6 (1937) although primarily interested in reading, developed a test of auditory compre- hension for children. Nichols27 (1948) appears to have been the first person to make a carefully controlled attempt to build a test of listening ability at the college level. Following him in rapid order came tests by Prince28 (1948), 25 Paul T. Rankin, "Listening Ability, II, Its Importance, Measurement, and DeveloPment“, Chicago Schools Journal, XII (June, 1930), 417-420. 26 Helen Blair Sullivan, "The Construction and Evaluation of a Measure of Auditory Comprehension", Unpublished.MeA. thesis, Boston University, 1937, 44 pp. 27 Ralph G. Nichols, "Factors in Listening Comprehension", Speech Monographs, XV (No. 2, 1948), 154-163. 28'Bernice L. Prince, ”A Study of Classroom Listening Ef- fectiveness in Basic Communication and its Relationship to Certain Other Personal Factors", Unpublished M.A. thesis, University of Denver, 1948, 93 pp. 31 Brown29 (1949), Harwoodso (1950), The Nashville (Tennessee) Public Schools31 (1950), Stromersz (1950), manor-35 (1950), Blewett34 (1951), Heilman55 (1951), Kramar and Lewiss6 (1951), and the Stephens College Test of Listening Abiltgy, Form a.” (1951). 29 3O 31 32 33 34 35 36 37 James I. Brown, "The Construction of A Diagnostic Test of Listening Comprehension", Unpublished Ph.D. thesis, Uni- versity of Colorado, 1949, 184 pp. Kenneth.A1bert Harwood, ”An.Experimenta1 Comparison of Listening Comprehensibility with Reading Comprehensibil- ity”, Unpublished Ph.D. thesis, University of Southern California, 1950, 157 pp. No author, "Listening Abilities of Pupils in the Nash- ville Public Schools, A Survey: Grades 1-12”, Nashville Public Schools, Nashville, Tennessee, September, 1950, 20 pp. walter F. Stromer, "Strength of Opinion, Auditory Memory Span, and the Ability to Gather Meaning from Contextual Clues as Factors in Listening“, Unpublished M.A. thesis, University of Denver, 1950, 44 pp. RalthWilliam Widner, Jr., ”A Preliminary Study of the Ef- fects of Training in Listening”, Unpublished M.A. thesis, University of Oklahoma, 1950, 152 pp. Thomas T. Blewett, "An Experiment in the Measurement of Listening at the College Level", Journal of Communication, I (May, 1951), 50-57. Arthur‘l. Heilman, ”Measuring and Improving Listening A- bility”, Speech.Monographs, XVIII (No. 4, 1951), 302-308. Edward J. J. Kramar and Thomas R. Lewis, ”Comparison of Visual and Nonvisual Listening”, Journal of Communication, I (November, 1951), 16-20. Ste hens Colle e Test of Listening Abilit Form A, (MImeograpEed) Preparedfiby the Division 0; CommEEIcation, Stephens College, Columbia, Missouri, 1951, 20 pp. C- 32 C. The Present Tests in Relation to Previous Tests No tests identical to the present tests appear to have been developed. Particularly, no tests appear to have been, as the present tests were, based upon an analysis of the im- plications for listening comprehension as derived from speech texts and reading improvement manuals; and no tests utilizing seven "standard” foils for all items appear to have been used previously; nor does subject matter material about listening appear to have been used in earlier tests. D. Summary of the Chapter The purpose of this chapter was to give a brief listing of studies that have included a test to measure listening comprehension. .A detailed description of such tests is given in Appendix A. ' Among the twentyrone studies in which the development of a test was of secondary concern, only one mentioned an at- tempt at item analysis, only three mentioned validity, and nine mentioned reliability. In thirteen studies having the development of a listen- ing test as significant part Rankin appears to have been the first to be interested in exploring the possibilities of de- veloping a test to measure listening itself. Sullivan did careful work in developing a test to measure the auditory comprehension of younger children. It was nearly ten years later that Nichols did the first careful work of constructing 33 a test to measure the listening comprehension of college students. Since Nichols, studies in which the construction of a test to measure listening was a significant part have increased rapidly. The present tests differ from previous tests in their emphasis upon factors in listening comprehension as derived from.speech and.reading manuals, in the use of seven ”stand- ard” foils for all items, and in the use of material about listening as subject matter passages. CHAPTER III CHAPTER III THE DEVELOPMENT OF THE FOILS A. Development and Failure of A Completely Auditory Test The first attempt to develop a test of listening compre- hension for Michigan State College freshmen was begun in the spring of 1948. This test was a completely auditory (no reading) test. Even though this particular attempt failed, because it brought to light a number of significant factors that were important to the tests finally developed, and be- cause these testing methods may be capable of refinement by future scholars, a brief description, as well as a brief statement of their trial, has been included here. Brief description. The completely auditory test con- sisted of five parts: Part 1 was composed of digits for recall. A series of numbers composed of from three to eight digits were presented. Then five other numbers also composed of from five to eight digits were presented. One of the five numbers was the same as the original. The purpose of this part of the test was to measure the listener's auditory memory span for numbers. There were five questions in Part 1. 36 Part 2 was composed of four short passages of about 150 to 175 words each. Each passage was deve10ped by one of the common methods of speech development such as details, sta- tistics, analogy, or illustration. Immediately after hear- ing the passage the listener heard five statements, one of which was a correct representation of the central idea of the passage. The purpose of this part of the test was to see if the listener could grasp the main idea of a passage when the idea was presented in varying develOpmental tech- niques. There were four questions in Part 2. Part 3 was composed of words and sounds. The listeners heard a list of from five to eight words or sounds. Then they heard five separate words or sounds, only one of which was in the original list. They were asked to mark the space on the answer sheet which indicated the repetition of the one word or sound that occurred in the original list. The purpose of this part of the test was to see if the listener could understand and carry in mind exactly what he had heard. There were four questions in Part 3. Part 4 presented lists of either words, or sounds, or numbers composed of from five to nine words, sounds, or num- bers. Then the listener heard five other words, or sounds, or numbers, four of which were exactly the same as the words, sounds, or numbers heard originally; but the fifth had not been included in the original list. Listeners were asked to mark the space on the answer sheet corresponding to the one 37 word, sound, or number not originally presented. The objec- tive was to determine whether the listener could distinguish exactly between what he had heard and what he had not heard. There were five questions in Part 4. Part 5 consisted of four short expository passages of ‘about one hundred words each. Each selection was followed by three questions. The first question asked, "What was the selection mainly about?" Then five statements were heard. One was correct. Then two questions about details in the selection followed. Five responses were heard, and the listener marked the appropriate space on the answer sheet. The objective was to determine whether the listener could comprehend the central idea of the selection, and also to comprehend and remember two details. There were twelve questions in Part 5. The entire test consisted of twenty-seven questions. Recording. The test was first tape recorded and was then put on both sides of two twelve-inch records. Trial. The recorded test was tried out in two sections of the listening "laboratory" during the fall quarter of 1950. Analysis. An item analysis of the answer sheets re- vealed that the test was far too "easy”. The test also failed to discriminate between "good" and "poor" listeners. 38 Observations. Observations made by the writer while students were taking the test indicated that there were cer- tain.weaknesses and strengths in the test. 1. Students could be influenced too easily by other students when the five foils were presented orally. It was amusing to watch two hundred students hear the foils read; and then all bend over and mark answer sheets at the same time. 2. Students can apparently ”hold in mind" five to eight numbers, sounds, or words, while they also hold in mind five possible foils. gpteresting. One interesting aspect of this test was the fact that all material was presented orally. The lis- teners received all instructions, all materials, and all foils for the five-choice multiple-response answers from the recording. The listeners had only a standard IBM answer sheet and a regular scoring pencil. Students apparently found this completely oral method easy to understand and follow. Discard. In view of the failure of the test to be sufficiently difficult and to discriminate satisfactorily; and in view of the impossibility of removing from the group test situation the influence of one student on another when the foils were presented orally, this approach was discarded. 39 B. The Concept of Seven Printed Foils After the failure of the completely auditory test of listening, a new approach to the problem of measuring lis- tening comprehension in the specific listening "laboratory" situation became necessary. Since one of the major diffi- culties in the "completely auditory test” had been the op- portunity for students to observe the reactions of other students when the one correct answer was heard, some method of preventing such "communication by bodily behavior” had to be evolved. The solution finally chosen was to have a common set of seven printed foils, and present the material and questions orally. Then each.student responded to each state- ment or question by referring to the set of standard printed foils and marked the appropriate place on an answer sheet. All students were then responding to all statements rather than waiting for the correct one of five. The Opportunity for one student to observe the behavior of another student was thus reduced to the average situation of group examina- tions. Such a procedure, furthermore, permitted the increase of the number of possible test questions or items in the same amount of time, since a student was responding to each question rather than waiting to hear five statements follow each question. This chapter reports the sources examined, and the methods used in developing the seven printed foils used for the tests. 40 C. Implications from Selected Literature on Speech as A Guide to the Development of the Foils Since there is a considerable amount of material in speech texts on the subject of informative speeches (reports, instruction, lectures, etc.), the speech text1 used in the course on Communication Skills was first examined for sug- gestions important to comprehension. The significant factors gleaned from this text suggest the following: 1. The purpose (central idea) should be clearly stated. 2. There should be from two to four divisions of the material. 3. Fasten these divisions or topics in the listeners' minds. 4. Be specific rather than abstract. 5. Transitions between materials should be made clear to the listener. 6. The purpose and main divisions must be elaborated by fact, example, illustrations, etc. (details). Under a heading, "Ways and Means of Making Ideas Clear" Brigance,2 emphasizes the value of specific words, simple words, illustrations, specific instances, figures and sta- tistics, testimony, restatement and.repetition. (The writer 1‘William N. Brigance, Speech Communication, F. S. Crofts, N. Y., 1947, 220 pp. 2 Ibid., 104-114. 41 refers to such material as developmental material, or de- tails e ) Igplications for listening comprehension. Restating these factors from the listener's point of view produced the following: 1. The listener should comprehend clearly the central idea of the talk. 2. The listener should clearly identify the main divi- sions of the talk. a. He should know where he is at all times. b. He should know when he moves from one division to another. 3. The listener should remember the main divisions of the talk. 4. The listener should separate the essential from the non-essential: a. Select and remember the major aspect of the in- formation. b. Distinguish clearly between the various uses of developmental material (details): those for clarity, emphasis, interest. 5. Remember such developmental materials (details) as are significant to the comprehension of the ideas. 42 D. Implications from Selected Literature on Reading as A Guide to the Development of the Foils Werking on the assumption that some parallels might ex- ist between the devices used to test reading comprehension and those which might be used to test listening comprehen- sion, an examination of selected materials on the subject of reading was next made. Guides from reading_manua1s. Four different reading manuals are used by the Reading Improvement Service of the Communication Skills Department to help students improve their reading rate and comprehension. Therefore, these four manuals were selected for sources of devices and.methods used to measure reading comprehension, with the thought that they might offer some suggestions in the objective measure- ment of listening comprehension. The first manual consulted was Improving Reading Ability.3 An examination of the "General Directions" and the specific sections of the manual reveal that reading com- prehension is measured in the following observable ways: 1. Speeded comprehension of word meanings is achieved by first presenting a single word, then following this by five different words. The reader is directed to mark the word closest in meaning to the first word. 3 James B. Stroud and Robert B. Ammons, Improvipg Reading Ability, Appleton-Century-Crofts, N. Y., 1949, 200 pp. 43 2. Speeded comprehension of phrases is indicated in the same manner as that used for words. 3. Speeded comprehension of sentences consists of: . . . a general statement followed by ten statements, some of which are similar and some different in meaning. Write the letter S after each of the ten sentences that agrees with - is similar to - the original statement. Write the letter D after each that is different in meaning from the original statement. Sentences should be marked S when they say the same thing in the same or different words or express the same general idea or an idea closely related to the original statement. Sen- tences should be marked D when they are unrelated in meaning or are contradictory in meaning, although they pertain to the same topic.4 4. Speeded comprehension of connected text is measured by a series of questions designed to test the reader's com- prehension. These true-false or multiple-choice questions cover facts, explanations, positions, purposes, methods, reasons, and relationships. 5. That part of comprehension called "Discerning General Significance" is objectively indicated by having the reader choose one of four best general statements: Encircle the number of the statement that contains the most accurate, comprehensive account of the general significance of the paragraph. The correct statement may take the form of the best general statement of the content, the best summary, or the most complete state- ment of the main idea presented. 6. A section labeled "Supporting and Non-supporting Paragraphs", is similar in form and pattern to speeded com- 4 Ibid., 10. 5 Ibid., 13. 44 prehension of sentences: A short paragraph is presented, and this is followed by several other paragraphs which are to be marked as supporting or not supporting the initial paragraph. The authors advise the reader to: Read the stem paragraph thoughtfully, trying to form an idea of its central thought or purpose. It will be good practice to ask yourself what the purpose of the paragraph is. Igplications for listening comprehension. l. Comprehension includes the understanding of indi- vidual words, phrases, and sentences. 2. Comprehension of connected text includes the under- standing of such factors as facts, explanations, positions, purposes, methods, reasons, and relationships stated or im- plied in the text material. 3. The ability to "discern the general significance” of what has been read is a part of comprehension. General sig- nificance is represented by such things as the best general statement of content, the best summary, or the best state- ment of the main idea. 4. The ability to detect relationships (similarities) and lack of relationships is a part of comprehension. 5. Comprehension of connected text may be evaluated by multiple-choice questions involving various factors. 5 Ibid., 14. 45 6. General significance of text material may be evalu- ated by multiple-choice statements of content, summary, or main idea. ‘ 7. The ability to distinguish between supporting and non-supporting ideas or material may be evaluated by indi- cating whether a statement, or paragraph is related (sup- ports) or is unrelated to (does not support) a given text. The second manual consulted7 lists the following read- ing exercises which appear to be concerned with comprehen- sion: l.‘Word Perception. 2. word Meaning, Simple Contexts. 3. Werd Meaning, Complex Contexts. 4. Equivalent Statements. 5. Paragraph Scanning. 6. Full Comprehension. The new or different ways of measuring comprehension presented in this manual are "werd Meaning, Complex Contexts" and ”Paragraph Scanning". The first is a sort of analogy type exercise, and the second raises a question with two possible answers, ”a" and "b”. The reader is to scan the paragraph until he arrives at the correct answer. 7 Christian 0. webber, Reading and VocabularyDevelopment, Prentice-Hall, Inc., Ni'YZ, 1951, v, vi. 46 Implications for listeninggcomprehension. As a means of testing listening comprehension the two new types of ma- terial referred to above seem to offer the following possi- bilities: 1. If a question is raised in advance of orally pre- sented material, listeners might listen to a passage until they heard the correct answer and then indicate the answer on some type of answer sheet. 2. Analogies might be used, but the writer sees no way at present of transferring this device in a "simple" way to testing listening comprehension. The manual by Wilking and.Webster8 includes the follow- ing major aspects of reading comprehension9 all preceded by the statement ”Thorough Reading”: 1. Summarizing. 2. Main Ideas. 3. Details. The authors define "Thorough Reading" as follows: Therough Reading is so called because the term closely describes the skill which will be developed. Before you can read critically or evaluatively, it is of primary importance that you understand thoroughly the meaning of what the author is attempting to say. ‘You must not only be able to understand his meaning, but you must also be 8 8. Vincent Wilking and Robert G.‘Webster, A College Develo - mental Reading Manual, Houghton Mitflin Co.,*Boston,*194 , 336 PP- 9 Ibid., vii, viii. 47 able to communicate to others the proof that you under- stood what you have read. The most convincing pr$8f is a complete, written rdcall of what you have read. The exercises and the ways in which the comprehension of the material is measured are stated as follows:11 —____ The first exercise you will be required to do demands only that you pick out the correct summary sentence of a paragraph. There will be three summaries given, one of which is just right, and two others which may be too broad or too narrow. ‘Your job will be to discriminate between these choices and, if possible, give your reasons. A few additional words on the concepts of ”narrow" and "broad" may be helpful. A summary is con- sidered as being too narrow if it presents only some of the details given in the original paragraph. These de- tails may really occur in the paragraph but the summary presents too few of them to be considered adequate. A summary is too broad if it goes beyond the data which is given in the paragraph. This means that any over- statement or false statement in a summary is a signal that the summary in question is too broad for the para- graph with which it deals. The second type of exercise is a list of main ideas (central thought) and details (minor ideas) of the para- graphs in a selection. ‘You will be asked to label the main ideas and the details. Once you can discriminate between the two you will be ready for the next step. ‘You.will be introduced to a device called the ”Idea line". This is nothing more or less than your old friend (or enemy) the outline. On the first horizontal line you will place the main idea or central thought of the para- graph. Directly below this main idea you will list in numbered order the ideas or details which support the main idea. In the early lessons you will be given the main ideas of the paragraphs and will be asked to fill in the supporting details. The job will then "step up" a little and you will be given the minor ideas or details and be asked to fill in the main idea which covers all the details. The final step will be to fill in the idea line in toto. 1° Ibid. , ix. 11 Ibid., x, xi. 48 Implications for listening comprehension. There appear to be numerous implications applicable to listening compre- hension in the foregoing manual. They are: 1. Convincing proof of understanding is obtained by a complete written recall of what has been heard. 2. Identifying a correct summary sentence indicates com- prehension. 3. Identification of correct central ideas is necessary for, and is indicative of,comprehension. 4. Identification of correct details is necessary for, and is indicative of, comprehension. 5. Discrimination between central ideas and details is necessary for, and is indicative of, comprehension. 6. The ability to derive a central idea from details is necessary for, and is indicative of, comprehension. 7. The ability to combine details to form a central idea is necessary for, and indicative of, comprehension. The manual by Strang12 designed to improve reading speed and comprehension states four important reading skills,13 three of which apply directly to comprehension: 1. To discover the central thought of what we read. 2. To recognize and remember the important facts. 12 Ruth Strang, Stud T e of Reading Exercises, Bureau of Publications, Teachers College, Columbia University, N. Y., 1945, 111 pp. 13 Ibid., 3. 49 3. To see where the facts lead, or, in other words, to draw conclusions. In another place14 Strang says, One group of skills is used to answer the question: "What is the book or article about?" These skills are: l. The ability to get facts and details. 2. The ability to get the central idea. 3. The ability to make a summary of the content. In clarifying these skills she points out that readers must guard against confusing facts with illustrations used for interest and clarification. In other words, they must separate main points from the details which develop them. 14 15 She further tells the reader that: In some cases, we need only to get the central idea of what we read. In order to do this, we must ask our- selves: "What is the main thought?" The answer to this will be the most important idea in a paragraph, a chap- ter, or a book. If we have selected what is called the central thought, we will find that the other facts in the passage explain or prove it. In this way, we can test whether or not we have selected the central thought in our reading. The central thought is sometimes ex- pressed in the ”topic sentence". . . . . In our school work, we are often required to write a summary of what we have read. A summary is a fuller answer to the question "What is it about? than either the facts alone or the central thought alone can be. If we have learned to select the central thought and to remember the facts and details which explain and prove it, we shall have pg difficulty in writing a summary of what we have read. Ibid., 26. Ibid., 27, 28. 50 The method of having the reader test himself to see whether he has acquired the skills for comprehending is by the use of objective multiple-choice questions: One question will ask for the central thought of the exercise; one will ask for facts; one will ask for egg- culsions which may be drawn from whatlias been read. Implications for listening comprehension. The impli- cations for listening comprehension suggested in the Strang manual do not differ greatly from those given by Wilking and webster. Strang does, however, propose a more objective way of evaluating the specific factors of comprehension. They may be summarized thus: 1. Comprehension includes the ability to grasp the cen- tral idea of what is read or heard. 2. Comprehension includes the ability to select impor- tant details. 5. Comprehension includes the ability to separate facts (main points?) from developmental material. 4. A summary statement of comprehended material is based on remembering central idea and details.. 5. The conclusions from material read are significant to comprehension. 6. Memory (remembering) is important to comprehension. 7. The comprehension of central idea may be tested ob- jectivelye 16 Ibid., 3. 51 8. The comprehension of details may be tested objec- tively. 9. Conclusions from what has been read (or heard) may be tested objectively. Summary of implications for listening derived from reading manuals used by the Reading Improvement Service. 1. 2. 4. 5. 6. 7. 8. 9. The ability to identify and remember the central idea of material is important to comprehension. The ability to identify and remember the details of ma- terial is important to comprehension. The ability to derive a central idea from details, or to combine details to form a central idea is important to comprehension. The ability to distinguish between central idea and de- tails is important to comprehension. The ability to distinguish between important and unim- portant material is important to comprehension. The ability to distinguish between related and unrelated material is important to comprehension. The ability to draw conclusions or inferences from ma- terial is important to comprehension. The ability to make or evaluate a summary statement of material is important to comprehension. The ability to make or evaluate a summary statement of comprehended material is based on remembering central idea and details. 52 10. The ability to remember is important to comprehension. 11. The ability to understand individual words, phrases, and sentences is important to comprehension. 12. The ability to discern the general significance of ma- terial is based upon either a summary statement or a statement of the central idea. Summagy of implications for measuripg listening as de- rived from reading manuals used by the Reading Improvement Service. The examination of the four reading manuals sug- gested definite procedures for the construction of multiple- choice questions which might be used for measuring listening comprehension. The following things appeared to be capable of identification or/and evaluation by the use of multiple- choice items: . 1. The listener's ability to remember and/or identify a central idea. 2. The listener's ability to remember and/or identify a detail (or details). 5. The listener's ability to differentiate between central idea and detail. 4. The listener's ability to correctly identify a central idea derived from stated details. 5. The listener's ability to determine relationship or lack of relationship. 6. The listener's ability to select the best summary statement. 55 7. The listener's ability to identify the correctness, and/or relatedness of conclusions of a factual na- ture derived from total material. E. Influence of Two Articles on the Development of the Foils Two articles which were influential in developing the seven foils were encountered in the literature surveyed. The first, by Ewing17 reported the use of a very simple de- vice used primarily for the purpose of improving speaking. . . . This study deals with one aspect of the speaking- listening problem: Finding an index of the degree of accuracy with which the theme and main ideas of the speaker are communicated to the listener. In order to clarify the purpose of the study, it should be explained that the word "theme" here means a sentence summary or gist of the talk. The main ideas mean the main statements or divisions in the body of the talk which support, describe or develop the theme. . . . Since the students were concerned only with speeches to clarify or to inform, the purpose of this study was to find a simple measure of the extent to which the speaker made his theme and main.points clear. The procedure was very simple. Each student gave a five—minute clarifying talk based on a prepared out- line which included theme, main ideas, and supporting details. At the end of the talk each listener was asked to write on a prepared form what he considered were the theme and main ideas of the speaker. Thus we obtained a written record of what the speaker intended and what thl listener understood to be the theme and main ideas. 8 1'7 William H. Ewing, "Finding a Speaker-Listening Index”, guarterly Journal of Speech, XXII (October, 1945), 368- 18 Ibid., 568. 54 This article emphasized the importance of what the read- ing manuals called summarizing, and significance; and also suggested the possibility of having listeners give a more ob- jective report by preparing specific statements which repre- sented the ”theme" of the talk and which could be reacted to 'as being ”exactly as stated by the speaker”; "saying approx- imately the same thing, but in different words"; and "not the same idea at all". It suggested furthermore the possi- bility of using approximately the same approach with sup- porting details. The second article which influenced the final specific statement of the foils was Sondel's.19 Her presentation and clarification of the "controlling idea", and its relation- ship to the essential and non-essential developmental ma- terial (details), was largely responsible for the idea of emphasizing the ability to distinguish between related and unrelated material; and to make the further distinction be- tween a central idea and details. F. The Assumption of Minimum Reading The assumption that there should be a minimum of read- ing served as a significant guide in developing the seven foils finally selected for the test. Whereas some earlier ‘- 19 Bess Sondel, "Everybody's Listening", National Parent Teacher, 46 (January, 1951), 14-16. 55 tests20 had included samples of materials to be "looked at", and had then been followed by such oral directions as "There are three different kinds of figures in this problem, what is the number of the figure that precedes a square and fol- lows a triangle?"; and Whereas many other tests21 had used printed or oral questions with many varying printed multiple- choice responses, the present test attempted to reduce the amount of reading and the variation in reading to a minimum in all parts of the test. The printed multiple-choice ques- tions normally used were therefore unsatisfactory for the objectives and assumptions of the present test. A true- false test could have been developed, but in view of some of the criticism involving this type of test22 and the stated values of multiple-choice items, it was deemed desirable to 20 See Arthur W. Heilman, "Measuring and Improving Listening Ability", Speech Monogra hs, XVIII (No. 4, 1951), 302-308. The sample question is rom one part of the test, the "Manual of Test Directions for Directions and Statements", and was given to the writer by Dr. Heilman. 21 See Appendix A ”Description of Tests Used to Measure Listening Comprehension". 22 James B. Stroud, Psychology in Education, Longmans, Green and 00., N. Y., 1946, 482. and Herbert E. Hawkes, E. F. Lindquist, and C. R. Mann (Edi- tors), The Construction and Use of Achievement Examina- tions, Roughton‘Mifflin co,, Boston,‘1935, 136-147, 152- and William J. Micheels, and M. Ray Karnes, Measuri Educa- tional Achievement, McGraw-Hill Book 00., N. Y., I955, Chapters FandV. 56 attempt the construction of a multiple-choice test. The final form was a multiple-choice-true-false combination. After numerous exploratory attempts to solve the prob- lem of varying choice response and yet use a minimum amount of reading and variation of reading, a set of eight standard foils was evolved. These eight foils appeared to include the significant aspects of listening comprehension as implied from studies of the speech text used in the course and from the reading manuals used in the Reading Improvement Service. When used in conjunction with orally presented passages and orally presented items (or questions) these eight printed foils offered combinations to test nearly all significant aspects of listening comprehension. G. Presentation to Staff Members In a preliminary evaluation of Test 1 by four staff members, one member suggested that two foils were in need of modification. If an item was unrelated to the passage heard it was probably impossible to determine whether it was un- related to the major idea of the passage or to a detail in the passage. Therefore, these two foils were combined, and the other foils re-numbered. H. Presentation to Students During the fall quarter of 1951, Test 1, with the seven foils, was tried out in three classes of Communication Skills. Students commented on the way in which the foils were worded, 57 arranged, and the amount and kind of explanations. In some instances students revised and re-worded the foils to convey the ideas more accurately to them; in other cases they gave suggestions to the writer, who revised the written form of the foils. I. The Seven Foils Used in the Test 1. This is a quotation (exact repetition) of the central iggg of the passage. It was actually stated in these same words in the passage. 2. This statement implies or suggests the same central idea as the passage. It ”truly" represents the central idea of the passage, but it is stated in different words from those used in the passage. 3. This is a pie-statement or mis-representation ('false', twisted, deceiving, partially true, partially false) of the central idea of the passage. 4. This is a quotation (exact repetition) of one of the g3- tailp_in the passage. It was actually stated in these same words in the passage. 5. This statement implies or suggests one of the details, or a similar detail, stated in the passage. It "truly" repre- sents a detail in the passage, or a similar detail, but it is stated in different words from those used in the pas- sage. 58 6. This is a mis-statement, or mis-representation ("false", twisted, deceiving, partially true, partially false) of a detail in the passage. 7. This statement is unrelated to either the central idea or a detail of the passage. It is neither a "true" nor ”false" quotation or implication of the central idea or a detail; but is rather entirely unrelated to either the central idea or a detail of the passage. The information is neither given in nor implied by the passage. These seven responses may be stated in a shorter form as follows: (A) Those concerned with the central idea. 1. Quote. Exact repetition of central idea. 2. Same central idea, but in different words. 3. False, twisted, or wrong central idea. (B) Those concerned with the details. 4. Quote. Exact repetition of a detail. 5. Same, or similar detail, but in different words. 6. False, twisted, or wrong detail. (0) The one concerned with both central idea and details. 7. Unrelated. Not given or implied. J. Summary of Chapter This chapter has indicated how the seven foils were de- veloped by tracing: (a) the trial and failure of a completely auditory test which lead to (b) the concept of seven printed 59 foils, (c) implications from selected literature on speech as a guide to developing the foils, (d) implications from selected literature on reading as a guide to developing the foils, (e) the influence of articles by Ewing and Sondel on the development of the foils, (f) influence of the assump- tion that there should be a minimum of reading, (g) revisions by staff members, (h) revisions by students, and (i) the final statement of the seven printed foils which were used for all three tests. CHAPTER IV CHAPTER IV THE SELECTION OF MATERIALS FOR THE PASSAGES When connected, meaningful material is presented for a test of comprehension, whether the objective be to measure reading or listening, some care must be exercised to assure, as far as possible, lack of previous acquaintance with the material by the subjects to be tested. If such care is not exercised then some testees may have an undue advantage over other testees simply because they have had an opportunity to become familiar with the material at an earlier time. A. Selection Methods Used by Some Other Studies _ Many of the research workers1 in listening have used adaptations of standardized reading test materials on the assumption that such materials are reasonably secure from examination prior to the test situations. Some investigators who have selected their own materials have used different ways to reduce the influence of possible knowledge of materials in advance of the testing situation. 1 See Appendix.A, "Description of Tests Used to Measure Listening Comprehension”. 62 Nichols2 selected six different subject matter areas ”in order to neutralize the advantage which would accrue to listeners who had already had college training in one or two of them". Blewett5 chose five informative passages that "were fic- titious in content in order that no advantage would accrue to those subjects possessing specialized knowledge". Kramar and Lewis4 chose an expository lecture on New Zealand "because it was believed that it would limit the factor of previous knowledge as a variable". B. Selection Method Used in This Study The passages of meaningful material for the present tests were selected not only with a view of preventing pre— vious knowledge, but also of contributing some significant information and motivation to the students in the listening "laboratory“. Conversations held by the writer with Michigan high school teachers indicated quite clearly that high school teachers were not presenting any information about nor giving any instruction in listening. Reports from students 2 Ralph Nichols, "Factors in Listening Comprehension”, pp, cit., 156. 3 Thomas Blewett, op. cit., 51. 4 Edward J. J. Kramar and Thomas R. Lewis, op. cit., 16. 63 enrolled in the writer's own classes as well as those en- rolled in classes of his colleagues, confirmed the judgment that students were relatively unacquainted with information about listening. Therefore, subject matter on listening was chosen for the content of the test passages on the as- sumption that such material would be reasonably unfamiliar to most students. Secondly, subject matter on listening was chosen be- cause it offered information of some value to the freshmen students in a Communication Skills program. C. The Three Passages Following are the three test passages of connected, meaningful material. Each passage was revised approximately seven times before reaching the form reported here. The re- visions were based on trials with students, and comments by students and colleagues. It should be noted that the passage for Test 1 consists of the instructions, or directions, for taking the test. This conforms to those life situations in which students must understand and carry out instructions they have received. The passage for Test 2 consists of a definition similar to the kind that might be given in a carefully prepared lecture. The passage for Test 3 consists of reports of two studies similar to those stated in numerous classroom lectures. 64 Passage for Test 1 INSTRUCTIONS This is a test of listening comprehension. Just as you take reading comprehension tests to learn how much you under- stand when you read, this test is designed to tell you how much you understand when you listen. This passage tells you how to take the test. First, do not make any noise or you.may prevent your- self and others from hearing the necessary material. Second, do not take any notes; just listen attentively. Third, lis- ten carefully for you will need the information you are now hearing to do this test. 'After you hear this passage on instructions you will hear a number of statements. Any statement p51 or pg: 223 be related to these instructions you are now hearing. YOu are to assign each statement to one of seven possible re- sponses. As soon as you hear a statement you should decide which of the seven possible responses that statement belongs in, and mark the appropriate space on your answer sheet‘yp- mediately. For example, if you should hear the statement, "How to listen to a classroom lecture”, you would mark space 7, for that statement is entirely unrelated to this passage on instructions for taking the test. Each statement will be given only once; It will not be repeated. 65 {Throughout the test you should mark only ppg_space for each statement. If you don't know, guess. Your score will be computed on the number of correct answers. This completes the instructions for taking the test. No additional information will be given. Do not ask any questions. See how well you can classify some statements about the instructions you have just heard. End of Passage for Test 1. 66 Passage for Test 2 DEFINITION As human beings we are constantly bombarded with all kinds of stimuli from the world around us. We see things with our eyes, taste with our tongues, smell with our noses, and feel with our senses of touch. For the next few minutes we will confine our attention to those stimuli we receive through the ear - those called sound - in an attempt to answer the question, What do we mean by the term listening? The first important factor in our definition of listen- ing is the reception of sound. When listening to television, sound-movies, or a speaker before us, we receive both audi- tory and visual impressions. ‘When we listen to the tele- phone, radio, recordings, or a person talking even though we cannot see him, we depend exclusively on the ear. The visual element may or may not be present. If present it is a fur- ther aid to understanding; but visual impressions may be ex- cluded, and we can still listen. Second, we should note that we receive a great many different kinds of impressions through our ears. A broad classification of sound ranges from noise to music. We shall limit these impressions to word sounds or language symbols. So, in this definition we include only a person talking; and we exclude all other auditory impressions such 67 as train whistles, instrumental music, telegraphic code, tom-toms, etc. Third, we must have an attentive receiver. The most significant factor of listening is not that it involves sound, nor language symbols, but that these symbols are in- terpreted by a receiver to have meaning. If a recording of a speech is playing, but you are not present to hear it, we say you are neither hearing nor listening. If you are present, but looking out the window day-dreaming you may be hearing, but you are not listening. we can summarize our definition in this way: Listening is the act of an attentive person interpreting word symbols received primarily through the auditory sense. End of Passage for Test 2. 68 Passage for Test 3 LISTENING CAN BE TAUGHT One of the basic assumptions about the skill of listen- ing is that listening can be taught. Following are brief summaries of two research studies testing that assumption: In 1949, Professor Arthur Heilman did a study to measure the improvement of listening brought about by direct instruction. Heilman met students in the usual small class- room situation. The students received direct instruction in listening for six separate periods of about ten minutes each. Tests were given these 450 students before and after in- struction. Results showed that the listening ability of college freshmen could be significantly improved through a program of training. Such training appeared to be particu- larly effective with those students whose listening ability was originally lower than average. In the fall of 1950, Professor Charles Irvin conducted a study involving large groups of students at Michigan State College. Seven ten-minute units of listening instruction 'were given to about half the freshmen students. The instruc- tion was given in the regular large lecture situation. Stu- dents were tested prior to training and immediately after- ‘ward. The 1400 students who received listening training did ' from 9 per cent to 12 per cent better than the 1400 who did not get training. However, the major gains were made by 69 those students who were classified as "poor" listeners at the beginning of the term. Over 50 per cent of the poor listeners who received training raised themselves from the category of "poor” up to the category of "above average"; while only 11 per cent of the poor listeners who did not receive training raised themselves to "above average". From these two studies it appears that the assumption "listening can be taught", is a safe assumption to make. End of Passage for Test 3. 70 D. Summary of Chapter This chapter has discussed the selection of materials for the passages to be used in the three tests, and has given: (a) a brief summary of some methods used by previous investigators in selecting their material, (b) the method used for this study, and (c) the three passages of connected, meaningful material used in the present tests. CHAPTER V CHAPTER V THE SELECTION OF ITEMS IN THE TESTS A. Method Used to Obtain the Trial Items After the preliminary foils or responses (Chapter III) and the preliminary passages (Chapter IV) had been deveIOped, it was necessary to obtain items that would be suitable for each passage. The items could have been constructed by the author, but he desired to have them represent the student viewpoint as much as possible. Therefore, the trial items were obtained from first term freshmen students enrolled in the Communication Skills course during the summer of 1951. Duplicated sheets were prepared giving the following information: This is an exercise in listening and.writing. A short passage will be read to you, then you will be asked to write eighi short statements about the ma- terial you have heard. Read the following eight classifications of possible statements. After you hear the passage write one short state- ment for each of the eight classificatIons that will apply to the passage you have heard. 1 In the early development of the foils (responses) eight, rather than seven, foils were used. See Chapter III. 73 Write each statement on a separate card, and number the cards from 1 to 8. 1. 3. 7. 8. write a statement or quotation of the central idea (Other terms for central idea are main—idea, mainfi' point, controlling idea, thesis sentence, topic sen- tence, purpose sentence, etc.) that was actually stated in the paragraph you heard. write a short statement that represents or conveys the central idea that is im lied or suggested, rather than actually stated, in the passage you heard. Write a short statement that is a mis-statement or mis-representation (That is, one that is "twisted", or deceiving, or:"fa1se") of the central idea of the passage you heard. ‘Write a short statement of the pgntral idea that was not actually_made, nor could be‘derived_fhhm the passage heard. That is, write a statement that is neither a true nor false statement or implication of the central idea, but is rather entirely unrelated to the material in the passage. ‘Write a statement or quotation of one of the details (Other terms for detail are: sub-point, sub-idea, supporting material, deve10pmenta1 material, etc.) that was actually stated in the passage you heard. write a short statement of a detail similar to one given in the material. Although this detail would be true” according to the material, it would be implied or s ested rather than actually stated in the same words In the passage you heard. write a short mis-statement or mis-representation (”false”) statemenh of a detail in‘the passage you heard. Write a statement of a detail that was neither actu- ally made, nor could be derIved from the material heard. That is, write a statement of a detail that is neither a true nor false statement or implication, but is rather entirely unrelated to the material in the passage you heard. 74 Small cards, about three by four inches in size, were prepared. Each card was lettered in the upper left corner: A for Test 1, B for Test 2, C for Test 3, etc. Each student received eight cards for each test passage. The writer met with three separate sections for a regu- lar two hour class period of the summer program. The students were told that a test to measure listening comprehension was being constructed, and that they were asked to help prepare the specific questions. The duplicated sheets (listening- writing exercise) were passed out and explained. Questions asked by students concerning the nature of their responsi- bility were answered. Students were told that a passage would be read twice at the beginning, and again half way through the first hour, so that they could correct the quota- tions they were asked to write. 1 After the passage was read once students were permitted to raise further questions. Then the passage was read a second time, and students began writing the items. About half way through the hour, the passage was repeated. Two passages were used with each class. Five different passages were used, but only three were selected for the final tests. From this listening-writing exercise, approximately 400 item cards were obtained for each passage, or a total of 1,200 item cards for the three passages. These 400 item 75 cards gave the kinds of responses for a passage that fresh« men devised of their own choice when writing under the guid- ance of the foils. They represented student-constructed, rather than teacher-constructed items. The item cards were sorted, and all duplicate statements were removed. Then those which appeared to be very similar in meaning but not identical in wording were removed. Those which remained were examined in the light of the opinion of some test experts.2 About 100 item cards for each test were selected for further consideration. These 100 item cards represented approximately twelve items for each_of the eight foils with the exception of Foil No. 1 (exact quotation of the central idea): There were only two of these for each passage. The item cards for each passage (test) were shuffled several times and the items were numbered and recorded in that order for the first trial use. 2 Dorothy c. Adkins, and others, Construction and Analysis of Achievement Tests, U. S. Government Printing Office, Whihington,'D. 0., I947, 291 pp. and Herbert E. Hawkes, E. F. Lindquist, C. R. Mann, The Con- struction and Use of Achievement Examinations, Houghton, in co., Boston,1936, 497 pp. and William J; Micheels and M. Ray Karnes, Measuri Educa- tional Achievement, McGraw-Hill Book 00., N. 3., 1955, PP- 76 B. The Development and Refinement of Items for Test 1 Criticism by colleagues. Four complete sets of all ma- terials for Test 1 were prepared. Each set consisted of: (1) An instruction sheet which asked the reader to read the entire material carefully, to mark the items as he would key them, and to write comments and criticism of any part of the test; (2) The passage; (3) The foils; (4) The items, with a space before each for recording the reader's response; and (5) Blank pages for comments and criticisms. Four members of the department read the materials, marked the items, wrote comments, and discussed the test with the writer. The keying of the items was examined, and all items that had not been keyed the same by at least three of the four readers were discarded or revised. Also, in those few cases in which the keying of the three teachers differed from that of the writer, the items were either discarded or revised. The number of items was reduced to seventy-five, and the number of foils was reduced to seven, through the evaluations made by the four teachers. These seventy-five items were regrouped according to the foil which was now considered their correct response. A die was cast, and each time a number was read, an item was se- lected from that group of foils. The newly arranged order of items was then ready for the classroom trial. 77 Classroom trial. Duplicated sheets of the "Responses"3 were prepared for student use. A complete script for Test 1 was prepared. This included the passage and the seventy- five items. During the fall quarter of 1951 Test 1 was tried out with three regular first-term class sections of the course. Students were told the main aspects in the development of the test up to that time. They were requested to take the test and comment on it so that it might be improved. They were told that their test scores and comments would not in- fluence their grades. Determining the time needed for students to respond to the orallyppresented items. A set of numbered forms was made out prior to the classroom trial. These were very simple forms consisting of a number, followed by a space for the time-keeper to record the elapsed time: Time Sheet (Section____) 1. 2. 3. O O 0 etc. 0 O O to 75. 3 The writer has used the term ”foils” in talking about the test in this study, but has used the term "responses" in materials for student use. This choice was made because students indicated that the term "responses" was more mean- ingful to them than was the test experts' term "foils”. 78 One student was selected to act as time-keeper, and he was given the Time Sheet and a stop-watch. He was instructed to begin keeping time as soon as the instructor finished reading, and to stop time upon signal from the instructor. In the first classroom trial, after the project had been~ explained, students were given IBM answer sheets, and the ”Responses Sheet". The passage was read by the instructor. Then the first item was read. Each student was instructed to mark his answer sheet and then raise his hand as soon as he had finished, and to keep it up until the instructor in- dicated that all twenty-six students in the class had marked their answer sheets. The instructor signaled the time- keeper, who wrote down the elapsed time. This procedure was followed for the first twelve items. Then a 10 second interval was used between items until ap- proximately half the items had been read. The instructor again read the passage, and time was kept for the next twelve items. The range of time for the first twelve items was from 15 seconds to 30 seconds, with an average of 20 seconds. The second twelve items ranged from 8 to 16 seconds, with an average of 12 seconds. In the second classroom trial the timing was conducted in a manner similar to the procedure just mentioned, but this time the passage and items had been recorded on tape. The recorder used.has a lever that permits the machine to be stopped at any time. Immediately after an item had been 79 played, the machine was stopped until all students indicated they had marked their answer sheets. The range of time again was from 15 to 30 seconds,'with.an.average of 19 seconds. The second group of twelve items ranged from 10 to 15 seconds with an average of 14 seconds. In the third classroom trial the tape recorded test was presented in the following manner: A 20 second pause was made after the first twelve items, then a 10 second pause after the next twenty-six items. The passage was then played again, and a 5 second pause was made between each of the re- maining items. Students expressed the Opinion that they could respond to the orally presented items and mark the answer sheets within the 5 second period, but that they felt ”rushed”. Most students expressed the opinion that after they had become familiar with the procedure, 10 seconds was ample time. A few thought 10 seconds was too long. Time intervals chosen. The results of these timed trials and student comments were incorporated into the final test by allowing 20 seconds after the first item, 19 seconds after the second item, and decreasing each pause by 1 second until a 10 second period was reached. Thereafter each of the remaining items was followed by a 10 second pause to per- mit students to choose the response and mark their answer sheet. This time schedule gave students an opportunity to have more time at the beginning of the test when the proce- 80 dure was new to them, and to have ample time for checking their answer sheets for the remainder of the items. Recommendationsppypstudents. In all three classes used for the classroom trial, the students gave criticism and recommendations for revision of all parts of the test. These recommendations were arrived at by the following class- room procedure: After the test had been taken the correct answers were read, and students scored and totaled their own answer sheets. The instructor put the distribution of scores for each class on the blackboard. Informal discussion among the students themselves and between students and instructor resulted in refinement of the foils, revisions of the pas- sage, and changes in the items. Some items were discarded as a result of student interpretation of them. The chief difficulty in all the classroom trials was expressed by students as one of grasping the idea that the instructions on taking the test constituted the passage on which they were to be tested. Although numerous revisions were made in the passage by students and by the writer in light of student and staff comment, this factor was not completely clarified even for the final presentation of Test 1. Key analysis of test items. Answer sheets for Test 1 were collected from students who had participated in the classroom trials, and these answer sheets were machine 81 scored. Twenty-seven per cent of the answer sheets having the highest scores and twenty-seven per cent having the low- est scores were selected from the total and a key analysis compiled.4 From these tabulations a very rough item analysis was made. Items which showed poor or little discrimination were eliminated or revised. Items that had been answered correctly by more than two-thirds of the high group were considered "too easy", and were revised or eliminated. Dif- ficult items were retained. Selection of items for the final trial test. On the basis of the classroom trials and the key analysis, sixty items were chosen for test form to be used in the listening "laboratory". Those retained items that had been marked correctly by the largest number of students and were con- sidered "easy" were put at the beginning of the test. Recording. A complete script, involving all the sug- gestions and implications of the previously reported pro- cedures, was prepared and recorded on tape. One voice was used for all materials recorded. 4 The term ”key analysis” as used by the Scoring Office at Michigan State College is explained as: A tabulation of the number of times an item is answered correctly on the highest twenty-seven per cent of answer sheets; and the number of times an item is answered correctly on the low- est twenty-seven per cent of answer sheets. 82 The script was read aloud and rehearsed until a rate of approximately 100 words per minute was attained for the pas- sage and the items. When rate had been adequately controlled, the material was recorded on a tape recorder. Time was kept on each part of the test, the reading of the items, the pauses between the items, the pauses between parts of the test, and total time for the complete test. Then the tape was played back and re-timed to check on the pauses between items and the total time of the test. Rate of speaking. Simple observation confirms the in- ference that college lecturers differ in their base rate of speaking. In a test to measure listening comprehension for the college lecture-listening situation, what may be con- sidered a ”normal” or base rate around which other rates will vary? An investigation of some of the studies using a ”natural" and "created" lecture situation revealed the fol- lowing: Among the apparently "natural" lecture—listening situa- tions Coreys indicated that after familiarizing himself with his materials he found that his delivery averaged very close to 100 words per minute. Greene6 found that his rate of speaking varied considerably when lecturing to two different 5 Stephen M. Corey, op. cit., 466. 6 Edward B. Greene, op. cit., 457-463. 83 college classes: In one he averaged seventywtwo words per mdnute, and in the other he lectured at the rate of ninety- two words per minute. Among the "created" lecture-listening situations, Kreuger'7 "kept” the rate of presentation as close as possi- ble to 100 words per minute. Blewett,8 and Heilman9 had the rate of material controlled at 120 to 140 words per minute. Nichols10 who had his speakers present material in a nearly normal situation (The speakers familiarized them- selves with the material and spoke extemporaneously.) deter- mined the speaking rates used for each of the six lecture excerpts as: Literature - 138 Economics - 100 Biology - 101 Sociology - 102 Psychology - 109 Chemistry - 85 Average - 105 words per minute There are, as was noted at the beginning of this sec- tion, variations in rates of speaking among college lecturers, 7 David H. Kreuger, op. cit., 35. 8 Thomas T. Blewett, op. cit., 51. 9 Arthur w. Heilman, op. cit., 304. 10 Ralph G. Nichols, "Factors Accounting for Differences in Comprehension of Materials Presented Orally in the Class- room", Unpublished Ph.D. thesis, Iowa State University, 1948, 31. 84 but there seems to be reasonable evidence for assuming that a rate of approximately 100 words per minute is near the rate that students will encounter in the various lectures that they will hear. Therefore, a rate of approximately 100 , words per minute was used in the recorded passages, and practice passages of the present tests. The rate of speak- ing for the items was also done at approximately this same rate, but the timed pauses were excluded in determining the over-all rate for speaking the items. 0. The Listening Laboratory Trial for Test 1 Factors influencingpthe administration. Prior to the administration of the test materials in the listening ”laboratory", three decisions had been reached that in- volved the administration of the test: (1) The tests were to be administered to first term freshmen in the "off- quarters”, namely winter and spring. The largest number of freshmen enroll in fall quarter, but students are admitted during the other three quarters of the year in much smaller numbers. Since it was considered more desirable to give all students the same instruction, and to avoid interrupting the program of the ten listening laboratories of the fall quarter students, the students entering in winter and spring were chosen for the trial group. These "off-quarter" groups usually consist of about four "laboratory" sections with not more than 100 students to a section. (2) To check the reli- 85 ability of the test it was decided to use a retest one week after the original administration. In view of the numerous aspects of comprehension being measured,11 the Pearson-Brown, split-half and Kuder-Richardson formulas seemed less appro- priate than the test-retest procedure.12 (3) One check on validity was to be made by correlating scores on the test being developed with scores by the same students on the Biology and Economics parts of the Nichols test.15 To accom- modate these last two aspects of the study the following pro- gram of administration was followed: During the first week, students in Listening Laboratories A, B, C, and D, all took the trial test. During the following week groups A and C were retested, and groups B and D took the Nichols test. Administration. Test 1 was administered during the third and fourth weeks of the winter quarter 1952. Each student picked up a standard IBM answer sheet, special pencil, and duplicated "Responses Sheet” on entering the room. The blackboard at the front of the room indicated that a listen- ing test-exercise was scheduled for the hour, and gave the directions for filling out the top of the answer sheet, and 11 See Chapter III.- 12 Dorothy c. Adkins, and others, op. cit., 148-160. 13 These two parts were chosen because they have two of the highest reliability coefficients of any part of the test: .48 and .66 respectively. 86 told students that the results of the exercise would be posted on the room bulletin board at the next meeting. The instructor went over the directions for filling in the top of the answer sheet; announced that the test-exercise was to measure listening skill, and that the scores on the exer- cise would not influence the student's grade. Students were told to read the ”Responses Page" carefully. Students with a hearing loss, and students who were from foreign countries were asked to indicate such on their answer sheets. (These answer sheets were not considered in any further analysis of the test items, although the scores were reported back to these students.) The recorder, which had been tried out and adjusted prior to the time the students assembled, was started, then stopped, and students were asked if the volume was satis- factory for hearing. The estimates of needed volume were apparently adequate for no additional volume was needed at the beginning of any of the four periods. The conditions . for administering this test were best labeled "typical” rather than "ideal" since at several times during each period the ”banging” of steam pipes in the room tended to interfere with listening. At such times the volume of the machine was raised to compensate for the interference. After the students had completed the test they were asked to write two comments about the "listening exercise”; (1) Did you consider this test ”easy” or ”hard"? (2) If 87 you considered it "easy", what would you recommend to make it "harder“; if you considered it "hard", what would you recommend to make it ”easier”? An examination of these written replies indicated that those students who considered the test "hard" wanted two additional aids: (1) Further clarification that the instructions for the test constituted the passage on which they were to be tested, and (2) addi- tional clarification of the seven foils on the "Responses Sheet". Administering_retest. Prior to the retest a short pas- sage, again emphasizing the fact that the instructions were to be the test material, was written and recorded on the first part of the tape. All other factors were the same as in the preceding test situation. Administering Nichols test. Information written on the blackboard stated, "This is a test of listening comprehension. You will hear parts of two recorded lectures. After you hear each lecture you will be asked to answer twelve questions. Do not take notes." Students were told that this test would not influence their grades, and that the results would be posted for the next week. Each student picked up an answer sheet and a pencil on entering the room. The volume level of the record player had been adjusted before students arrived. The record was be; m f d‘. in i d! s i e er. I . H. :1... IJ‘IDIV!.L\..,0. stirs... Fin n 88 begun, and stopped, while students were asked if the volume was satisfactory. Then the record was played. At the end of the first record (Biology), the printed questions were distributed. Students were given as much time as they needed to mark their answer sheets. Then the second record (Econom- ics) was played, and after it, the printed questions were distributed. D. Selection of the Items to be Included in Test 1 The answer sheets for the first administration to the four sections of the listening laboratory were machine scored, and then an item analysis was made using the Flanagan14 method, of all items from the highest and lowest twenty-seven per cent of all answer sheets. From this analysis thirty-two questions were selected to be included in the actual test. The answer sheets were re-scored on the basis of these thirty-two questions, and those scores were used for all further statistical analyses. The level of difficulty of each item retained, and the discrimination index of each item retained is given in Table I. 4 1 J. C. Flanagan, "General Considerations in the Selection of Test Items and A Short Method of Estimating the Product- Moment Coefficient from the Tails of the Distribution”, Journal of Educational Psychology, 30 (1939), 674-680. 89 TABLE I ITEM ANALYSIS OF TEST 1 Number of Number of Item Correct Correct Diffi- Discrim- Number Answers in Answers in cultyb ination° Highest 27%a Lowest 27% 1 35 15 .63 .54 2 29 15 .55 .35 3 24 10 .43 .36 4 25 8 .41 .45 5 32 - 14 .58 .47 6 28 12 .50 .40 7 24 9 .41 .40 8 33 12 .56 .53 9 3O 12 .50 .45 10 24 4 .35 .56 11 3O 11 .51 .47 12 22 14 .45 .21 13 24 8 .40 .42 a There were 40 answer sheets in the highest 27%, and 40 answer sheets in the lowest 27%; a total of 80. b The lower the number the more difficult the item. ° A discrimination index of .22 is significant at the 5% level, and a discrimination index of .29 is significant at the 1% level of confidence. George W. Snedecor, Statistical Methods, Iowa State College Press, Ames, owa, 9 , 3 l. 90 TABLE I (Continued) ITEM ANALYSIS OF TEST 1 Number of Number of Item Correct Correct Diffi- Discrim- Number Answers in Answers in culty ination Highest 27% Lowest 27% 14 17 3 .25 .45 15 17 7 .30 .28 16 24 5 .36 .51 17 28 0 .35 .81 18 14 4 .23 .35 19 5 l .07 .34 20 13 1 .18 .56 21 16 6 .28 .32 22 13 9 .28 .12 23 26 6 .40 .53 24 26 7 .41 .49 25 17 5 .28 .38 26 22 8 .38 .38 27 25 8 .41 .44 28 25 6 .39 .50 29 22 ll .41 .28 30 20 7 .34 .36 31 25 7 .40 .47 32 31 5 .45 .66 91 E. Procedures Used in Selecting the Items for Test 2 and Test 3 Which Vary From Those Used for Test 1 Similarity of procedure. The major pattern used for ob- taining and refining the test items for Test 2 and Test 3 was similar to Test 1, but certain variations should be noted. This section of the study mentions only the differences; where differences are not mentioned, the same procedure as used for Test 1 is to be understood. Printed directions. Since the passage for Test 1 con- sisted of the directions for taking the test, this passage was revised and written for students to use in taking any fellowing tests. This was duplicated and labeled ”Instruc- tions”. Practice passagg. In view of the comments written by students after the laboratory trial of Test 1, asking for further explanation of the "Responses", a section of three practice items was written and recorded as the first part of the test. Number of items. Sixty items were used for the class- room trial, and fifty questions for the laboratory trial. Time between items. Since the time needed for students to respond to the orally presented items had been established during the trial periods of Test 1, the time studies were not 92 carried out with theseizwo tests. The time pauses arrived at for Test 1 were used throughout the classroom trials and the laboratory administration of Test 2 and Test 3. Analysis of incorrect responses. A hand tabulation was made of thirty high and low answer sheets for both tests. In addition to considering discrimination and difficulty, an at- tempt was also made to select those items whose "incorrect" responses were distributed rather evenly among the six re- maining foils. If an item had been marked incorrectly, but the incorrect responses tended to cluster in one or two of the foils, that item was discarded in favor of an item which tended to have the incorrect responses distributed among the ( various foils. Sappling the position of items. The possibility that the position.of an item in relation to its distance in time from the hearing of the passage might influence its discrim- ination and difficulty was sampled in four instances: Two from each test. Two items placed near the beginning of the test were repeated near the end of the test. The hand tabu- lation, mentioned in the preceding paragraph, revealed that for three of the four items sampled the difficulty of the item was very slightly increased by placing it nearer the end of the test. The discrimination, and the distribution of the incorrect responses was remarkably similar whether it came near the beginning or near the end of the test. One 93 item revealed a considerable increase in difficulty in its late position when compared with its early presentation. This tendency for some sampled items to become slightly more difficult, but to retain approximately the same dis- crimination when presented further in time from the passage emphasized the need for a constancy obtained by recording the material and presenting it in the same manner and same order to each group of students. Time of administration. Whereas Test 1 had been given classroom trials in the fall quarter and the laboratory trial during the winter quarter, Tests 2 and 3 were given classroom trials during the winter quarter and laboratory trials during the spring quarter of 1952. Administration. Conditions for administering the last two tests were free from the interference of "banging" steam pipes. The length of time required for each test was written on the blackboard: 27 minutes and 25 seconds for Test 2, and 26 minutes and 50 seconds for Test 3. Some attempt was made to motivate students by telling them that although the lis- tening test-exercise would not influence their present grades, they would have some type of listening test on the final comprehensive examination; and that the present practice would be valuable. Also, scores students received on the 94 first test were not given to them until they had completed the retest. After students had taken the retest, the instructor read the correct response for each item, and each student totaled his own score. It was found that there was suffi- cient time to have students take the.test, and also score it within one regular fifty minute period. Table II gives the difficulty and discrimination in- dices for Test 2, and Table III does the same for Test 3. 95 TABLE II ITEM.ANALYSIS OF TEST 2 Number of Number of Item Correct Correct Diffi- Discrim- Number Answers in Answers in cultyb inationc Highest 27%a Lowest 27% 1 18 6 .60 .63 2 17 9 .65 .45 3 17 5 .55 .60 4 17 9 .65 .45 5 16 4 .50 .60 6 16 11 .68 .28 7 14 4 .45 .51 8 20 13 .83 .66 9 11 6 .43 .26 10 12 7 .48 .26 11 19 , 12 .78 .50 12 11 5 _ .40 .32 13 ll 1 .30 .61 g a There were 20 answer sheets in the highest 27%, and 20 answer sheets in the lowest 27%; a total of 40. b The lower the number the more difficult the item. c A discrimination index of .31 is significant at the 5% level, and a discrimination index of .40 is significant at the 1% level. 96 TABLE II (Continued) ITEM.ANALYSIS OF TEST 2 Number of Number of Item Correct Correct Diffi- Discrim- Number Answers in Answers in culty ination Highest 27% Lowest 27% 14 7 2 .23 .35 15 9 2 .28 .44 16 12 3 .38 .48 17 12 3 .38 .48 18 4 1 .13 .32 19 13 3 .40 .52 20 10 5 .38 .27 21 16 6 .55 .51 22 13 9 .55 .21 23 10 2 .30 .48 24 16 7 .58 .46 25 11 3 .35 .44 26 7 3 .25 .26 27 19 9 .70 .61 28 19 6 .63 .70 29 14 5 .48 .45 3O 12 4 .40 .42 31 12 3 .38 .48 32 12 3 .38 .48 97 TABLE III ITEM.ANALYSIS OF TEST 3 Number of Number of Item Correct Correct Diffi- Discrim- Number Answers in Answers in cultyb inationc Highest 27%a Lowest 27% 1 19 10 .73 .59 2 16 8 .60 .42 3 18 13 .78 .35 4 16 3 .48 .64 5 17 5 .55 .60 6 19 11 .75 .55 7 19 10 .73 .59 8 18 8 .65 .56 9 14 7 .53 .36 10 19 8 .68 .64 11 17 6 .58 .56 12 14 9 .58 .26 13 8 O .20 .68 a There were 20 answer sheets in the highest 27%, and 20 answer sheets in the lowest 27%; a total of 40. b The lower the number, the more difficult the item. c A discrimination of .31 is significant at the 5% level; and a discrimination of .40 is significant at the 1% level. 98 TABLE III (Continued) ITEM.ANALYSIS OF TEST 3 Number of Number of Item Correct Correct Diffi- Discrim- Number Answers in Answers in culty ination Highest 27% Lowest 27% 14 17 8 .63 .48 15 8 4 .30 .24 16 14 4 .45 .51 17 9 3 .30 .35 18 8 2 .25 .40 19 13 4 .43 .46 20 11 4 .38 .38 21 16 2 .45 .70 22 8 10 .45 -.10* 23 15 4 .48 .55 24 15 5 .50 .50 25 18 9 .68 .52 26 13 2 .38 .59 27 19 3 .55 .79 28 15 3 .45 .60 29 ll 9 .50 .10 30 17 10 .68 .41 31 17 6 .58 .56 32 14 6 .50 .40 * This item gave a positive discrimination in classroom trials, and a negative discrimination in the laboratory trials. 99 Table IV shows the degree of difficulty obtained by averaging all items in each of the three tests. An obser- vation of the data reported in this table reveals that Test 1 is much more difficult than.the "ideal" of fifty per cent. Tests 2 and 3 are sufficiently close to the "ideal" to be used. TABLE IV AVERAGE DIFFICULTY OF THE THREE TESTS Test 1 Test 2 Test 3 38.90 46.87 52.43 F. Summary of Chapter This chapter has discussed the selection of items in- cluded in the three tests: (a) the method used to obtain the trial items, (b) the development and refinement of the items in Test 1, (c) the listening laboratory trial of Test 1, (d) the final selection of items for Test 1, and (e) the proce- dures used in selecting the items for Test 2 and Test 3. Item analyses of difficulty and discrimination were presented for each of the items included in the final test forms. The individual items for each test are found in Chapter VIII. CHAPTER VI CHAPTER VI THE RELIABILITY OF THE TESTS A. Method Used Although it requires additional time for the adminis- tration of the same test to the same students at a later date, the method used to check the reliability of the three tests was the test-retest type. As reported in Chapter V, approximately one-half of each group of students in the listening laboratories took the same test a second time one week after the first adminis- tration. The chief criticisms directed at the test-retest method for determining reliability are stated by Adkinsl to be the possibilities of memory if the test is administered too soon, the possibility of learning new materials that would influ- ence the score on the second taking of the test, and the difficulty of maintaining comparable testing situations. Although these criticisms seem significant, the possi- bilities of using the Spearman-Brown split-half method, or the Kuder-Richardson formula, were equally unsuited to the particular tests. The split-half Spearman-Brown method was 1 Dorothy C. Adkins, and others, 0p. cit., 149-150. 102 discarded because of the seven-fold nature of the "standard- ized" foils. In thirty-two questions, or items, there are only two No. l foils, and five each of the foils numbered 2 through 7. The Kuder-Richardson formula, according to Adkins,2 assumes that the test measures only one factor, that all inter-correlations among the items are equal, and that all the items have the same difficulty. The present tests were assumed to measure not one but several factors, and the items were known to have varying degrees of diffi- culty. B. Test-Retest Correlation The test-retest correlations for the three tests have been given in Table V:3 TABLE V TEST-RETEST CORRELATIONS Test 1 Test 2 Test 3 Correlationa .70 .77 .71. Number 69 35 39 a All correlations are significant at the 1% level. 2 Ibid., 153-154. 3 Individual test scores for all students used in the labora- tory trial of the three tests are given in Appendix B. 103 Since two of the major criticisms of the test-retest method were considered the possibility of memory and the possibility of learning, the mean and standard deviation for test and retest groups were computed. These data are re- ported in Table VI: TABLE VI MEAN AND STANDARD DEVIATION FOR TEST AND RETEST Test 1 Test 2 Test 3 Mean S.D. Mean S.D. Mean S.D. Number 69 35 39 Original Test 11.97 5.67 14.49 5.12 17.05 5.62 Retest 14.62 5.95 15.60 '4.54 19.95 5.20 Difference 2.65 1.11 2.90 t 4.918 1.99b 4.388 ‘ Significant at the 1% level. b Not significant at the 5% level. There appears to have been some memory, or learning from taking the test or from outside sources, operating in Test 1 since the difference between the mean of the test situation and the retest situation is significant at the one per cent level. The most probable explanation is that of learning from taking the test itself. In both the classroom trials 104 and the laboratory trial, students reported that they had difficulty understanding that the instructions for taking the test constituted the material over which they were to be tested. Once students had heard the complete test, they understood much more clearly what they were to do, and they were able to obtain a higher score the second time they took the test. There is no significant difference between the means of the test and retest for Test 2. Apparently little learning took place either from taking the test or from outside sources. The test material, a definition, was probably not easily remembered. The classroom trials of both Test 2 and Test 3, tend to confirm the opinion that students under- stood these tests from the beginning. No objections, similar to those raised about understanding the passage of Test 1, were raised for either Test 2 or Test 3. The difference between the test and retest mean for Test 3 is also significant at the one per cent level. Some of this learning is to be explained from\memory. At the close of the laboratory retest period, some of the students men- tioned that they "got" some of the questions this time that they didn't "get” the first time, and that they "remembered" after the test that they had answered some items incorrectly. The passage in Test 3 is also composed of meaningful mater- ial probably more easily remembered than the definition of Test 2. 105 The writer questions whether, in the present instance, a difference between the means of test and retest is im- portant. The mean scores, as a whole, tended to increase, but the standard deviations remained very similar. If there tends to be a slight but positive increase in the test scores, then a common "practice" effect appears to be oper- ating. An examination of the shifts in scores between test and retest shows the general tendency for some increase in scores. In Test 1, five students received exactly the same score on the test and the retest, forty-four received higher scores, and twenty received lower scores. In Test 2, five students received the same score on the test and retest, twenty received higher scores, and ten received lower scores. In Test 3, one student received the same score on the test and retest, twenty-six received higher scores, and twelve received lower scores. Thus, on the average twice as many students received higher scores as those who received lower scores: An indication that something other than chance is operating to increase the scores. This influence is prob- ably practice. 0. The Reliability of the Test in Relation to the Reliability of Other Listening Tests Five recent studies, involving the construction of a test of listening comprehension as a major part, have re- ported the following test reliabilities: 106 Nichols4 reported the reliability for the combined six parts of his test as .802 by use of the Kuder-Richardson formula. The reliabilities of three of the most reliable parts of this test are .48 for Biology, .49 for Chemistry, and .66 for Economics. Brown5 used the Spearman-Brown formula and reported the reliability of the two parts of his test as: Part I, .810; Part II, .647. It should be pointed out that Part II of the Brown test is similar to the present test, whereas Part I is not. And it should be observed that Brown obtained a higher reliability for Part II of this test (.874) when it was pre- sented to high school students. The .647 reliability was obtained from college students. Blewett6 reported reliabilities of .76 for Part I of his test, and .75 for Part II, and .81 for the combined test by use of the Kuder-Richardson formula. Kramar and Lewis7 do not report the formula used, but indicate a reliability of .71. 4 The reliabilities on the separate tests were sent to the present writer by Professor Ralph Nichols. 5 James I. Brown, op. cit., 99-100. 6 Thomas T. Blewett, op. cit., 52. 7 Edward J. J. Kramar and Thomas R. Lewis, op. cit., l7. 107 Heilman8 correlated two forms of two of his sub-tests, and obtained a reliability of .80. The third sub—test showed a reliability of .84 obtained by the Spearman-Brown Prophecy formula. D. Summary of Chapter Although the reliability formulas used in several other studies vary, the retest reliabilities of .70, .77, and .71 obtained for the present tests appear similar to that for other tests of listening comprehension. In view of the fact that many of the other tests take from one to three hours to administer, the reliabilities of these tests, which run less than thirty minutes, and permit easy administration within a fifty minute class period, appear at least the equal of other listening tests. __ 8 Arthur W. Heilman, op. cit., 304. "-‘1'1 .4... ' “~ CHAPTER VII CHAPTER VII THE VALIDITY OF THE TESTS . . . The validity of the test depends upon the effec- tiveness with which it measures that which it is in- tended to measure, or, otherwise stated, upon the-3?- Tectiveness with which it accomplishes the purpose it is intended to accomplish. . . . . . . . . . . . . . . If a test is "valid", it is valid for a given purpose, with a given group of pupils, and it is valid only'to the degree that it accomplishes that specific purpose for that specific group. . . . Purpose and group. The specific purpose and specific group for which the present tests were designed can be re- stated: to determine the amount or degree of listening comprehension skill possessed by freshmen students in the Communication Skills program of Michigan State College. A. Validity Studied by Comparison with the Nichols Test The first measure of validity applied to the present tests was the determination of their relationship with an al- ready developed test, namely two parts of the Nichols Test.2 This test had been developed and used with 200 college 1 Herbert E. Hawkes, E. F. Lindquist, C. R. Mann, op. cit., 21. 2 The Nichols Test is described in Appendix A. 110 freshmen drawn from the communications courses on the St. Paul Campus of the University of Minnesota. Copies of the records for this test and questions on the various recorded lecture excerpts had been supplied by Professor Nichols for purposes of other listening studies being carried on in the Michigan State College course. To obtain the test scores for the correlations, all students took one of the present tests during the first ad- ministration; and then during the second week, approximately one-half of the students took two parts of the Nichols Test.3 The correlations between scores on each of the tests and the scores of the same students on the Nichols Test are shown in Table VII. TABLE VII CORRELATIONS BETWEEN TEST AND NICHOLS TEST Test 1 Test 2 Test 3 Correlationa . .50 .51 .81 Number 60 30 22 a All correlations are significant at the 1% level. 3 The administration of these tests was explained in Chapter V. 3K 111 The mean and standard deviation of the scores of stu- dents who took the test and later took the Nichols Test are reported in Table VIII. TABLE VIII MEAN AND STANDARD DEVIATION OF SAME STUDENTS ON TEST AND NICHOLS TEST Test 1 Test 2 Test 3 Mean S.D. Mean S.D. Mean S.D. Number 60 3O 22 Test 12.27 5.20 14.00 5.56 17.23 6.25 Nichols Test 14.50 3.61 15.30 4.06 15.86 3.83 The positive correlations between the three tests and L the Nichols Test indicate that the tests are measuring some- thing which is similar. The medium correlations indicate that some other factors are being measured. The limits be- yond which the obtained correlations will not vary ninety- nine times out of a hundred (Test 1: .21 to .71; Test 2: .07 to .78; and Test 3: .49 to .94) warrant the conclusions that to the extent the Nichols Test is a valid measure of some h~ factors of listening comprehension, the present tests are also valid measures of similar factors. 112 B. Validity Studied by Expert Inspection The second measure of validity used was that of in- spectional validity by a group of experts. A letter was addressed to three authorities in the field of listening, asking them if they would be willing to read the test ma- terials, express an opinion concerning the validity of the test, and to pass any other comment that they wished on the test materials. Professor Ralph Nichols of the University of Minnesota, Chairman of the Listening Committee of the National Society for the Study of Communication, was sent the following ma- terials for Test 1: Responses4 Explanation of Responses (Developed after the test had been administered to students in the listening laboratories) Special Emphasis Directions for Test 1 (Developed for the retest) Passage Items (The thirty-seven chosen after item analysis). His reply of May 26, 1952, read as follows:5 I have carefully examined your test #1 measuring listening comprehension. It seems to me that it should 9Th6 complete text of material for Test 1 is given in Chap- ter VIII. 'Personal letter. 113 Iarove to be a valid instrument. In fact, I believe you laave hit upon a rather novel and highly effective method <>f testing this skill. Your keying of your test items appears to me to be centirely accurate save for one or two very small details wwhich I have taken the liberty of indicating in pencil. (Ralph Nichols) Professor Harold Anderson of the University of Chicago, Chairman of the Vertical Committee on Listening of the Natzional Council of Teachers of English, was sent the fol- lovving material for Test 2: Responses6 Directions for Taking Listening Comprehension Test Practice Session Passage Items (The fifty items chosen for the laboratory trial) His reply of June 22, 1952, written under adverse con- ditions, reads:7 On the whole, I think you have developed a good test. I have only a few comments. 1. I wonder whether it is sound to test listening comprehension on only one kind of material? Instead of asking many questions on one short passage would it not be better to ask fewer questions and to use the rest of the time to listen to and respond to another type of passage -- say a narrative passage? You.will be testing only listening comprehension in the case of expository material. ——¥ 6 The complete text of material for Test 2 is given in Chap- ter VIII. 7 Personal letter. 114 2. I'm a bit uncertain about the wisdom of using 'the exact quotation items. To remember whether a group «of words is an exact quotation or not is in large part a test of memory. For example, item 19 [Discarded after item analysis] is an exact quotation, but item 2 [No. 2 on.the final form] is not. But they both ask the same question. How important is it for a listener to know ‘which is the exact quotation? May this not be measuring memory alone? - 3. I'm not sure that item 14 is unrelated. . . . [This item was discarded after item analysis.) 4. As I reread the test, I see that some of my com- ments above may not be valid. You call this passage on listening "Passage B". ELabeled "Passage for Test 2” in the final form.] Does that mean that there are to be a number of passages? 5. As I tried to imagine myself really taking the test, I had the feeling that the directions for taking the test or the method of responding was a bit too com- plicated - that keeping in mind the distinctions among the ways of responding was a distration. But I'm not able to suggest an improvement. I have some fear that the test may be testing too largely a person's ability to say whether an idea is re- lated to a main idea or to a subordinate idea. But I suppose that's an important ingredient in both listening and reading. Like so many tests, this one no doubt will largely measure general intelligence. Have you correlated the results with intelligence scores? These are only minor matters. On the whole, the test looks good to me. (Harold) Harold A. Anderson Professor James I. Brown, also of the University of Minnesota, and a member of both the Listening Committee of the National Society for the Study of Communication, and the 115 Listening Committee of the National Council of Teachers of English, was sent the following materials for Test 3: Responses8 1 Directions for Taking Listening Comprehension Test Practice Session Passage Items (The fifty items chosen for the laboratory trial) His reply of April 29, 1952, reads:9 After looking over your listening test I am wonder- ing whether my reactions are going to be of much use. I find it impossible to be as objective as I should, pri- marily because I went over some of the same problems in connection with my own test. Actually my two most important questions are ones that can be checked rather objectively. For example, your complex seven-point classification scheme may in- troduce complexities that increase the correlation be- tween your test and I.Q. or your test and.memory. Again, it would seem to me even 30 questions over a short pas- sage of about 22 lines is overly analytical. Further- more, after eight or ten minutes of attention to ques- tions, some being misstatements or containing wrong de- tails, would there not be a strong tendency for unde- sirable interferences, tending to invalidate your test? Both of these matters, however, can be checked rather objectively, by using a suitable I.Q. test and, say, the Wechsler sub-test on number repetition as a check on the memory factor. The Thurston test of Primary Mental Abilities might give you a similar check. . . Certainly you have every reason to set up the hypothesis that your test does measure listening comprehension. The remainder of your research would test whether or not that hypothesis holds up. . . . (Jim) James I. Brown 8 The complete text of material for Test 3 is given in Chap- ter VIII. Personal letter. 116 Since two of the authorities have raised a question about the memory factor involved in the identification of ’ ”exact" materials, the writer urges a consideration of those college lecture-listening situations, particularly in courses where students must remember exact names, exact numbers, and exact relationships; not approximations or similarities. The memory relationship was not studied, as suggested by Professor Brown, since the laboratory testing period had dIO been completed. However, a statement made by Strou may prove of some value in connection with the problem of memory. In discussing the problem of reading rate and reading com- prehension Stroud says: . . . . This procedure has been criticized on the ground that the obtained score is a memory score rather than a comprehension score. This criticism does not seem to be crucial. The association between learning and retention is much closer than the chapter headings in textbooks are likely to suggest. Any measure of learning is to some extent a measure of retention; andeaxcept for re- tention learning would be impossible. Moreover, it is difficult to attach any value to comprehension in reading except as it eventuates in the ability to make a better response afterward. There is probably no reason to doubt that the pupil who has the best understanding of what he reads, hears, or otherwise apprehends can give the most intelligent account of it afterward. In any situation that involves understanding, that understanding can be measured a moment later. There are no inherent diffi- culties in the measuring of understanding by a learning test. Those elements, such as places and dates, that do not provide good tests of understanding in a learning test likewise do not provide good tests of understanding in the typical reading test. . . . __k 10 James B. Stroud, op. cit., 130. 117 Nevertheless, the suggestion made by the authorities has impressed the writer to the extent that in the future he plans to construct a test with only five foils (omitting exact statements of controlling idea and details) and at- tempt to discover what, if any, differences are revealed about listening comprehension. The suggestion by Professor Brown that eight or ten minutes of attention to questions, including some misstate- ments and incorrect details, might invalidate the tests had also occurred to the writer, and this possibility was ex- plored in two ways: First by repeating, near the end of Test 2 and Test 5, two items that had previously been given near the beginning of the tests. As reported in Chapter V, little change occurred because of the position of an item. Second, items near the end of the tests were examined to see if there was any consistent tendency for these items to lack discrimination or to increase greatly in difficulty. Sta- tistical analyses of items near the end of the tests revealed no consistent tendency to differ from items in other parts of the tests. Students who tended to do well or poorly at the beginning of a test tended to perform at the same level toward the end of the test. Although statistical evidence indicated that students were making equally ”good" choices later in the test, Pro- fessor Brown's suggestion, plus observation of students in the laboratory test situation suggested that time might have 118 other influences. Students appeared to become "restless" after hearing and.responding to approximately twenty-five items. Therefore the tentatively projected thirty-eight items for each test was reduced to thirty-two items for each of the final tests. Summaryfof authoritative opinions. If the present writer interpreted the letters from the three authorities correctly, he inferred from Professor Nichols and Professor Anderson that the present tests measured listening compre- hension. He inferred from Professor Brown that the present test had only the hypothesis of measuring listening compre- hension, and that the hypothesis needed to be substantiated. Such substantiation has been further undertaken in the re- maining parts of this chapter. C. Validity Studied by Relation to Psychological Scores Most research studies involving the construction of a measure of listening comprehension have correlated the re- sults of scores on the constructed test with scores made by the same students on psychological or scholastic aptitude tests. One of the main reasons for determining the rela- tionship between listening comprehension tests and psycho- logical tests has been the generalized belief that if high correlation existed between the two, then the listening test was probably measuring "intelligence" rather than listening 119 ability. Two of the authorities who examined the listening tests suggested that they might be measuring intelligence. As a further check on the validity of the three tests, total raw scores were obtained for each student on the American Council of Education Psychological Examination. All incoming students at Michigan State College take these tests during their first weeks at the school. The scores of a sampling of 235 students who entered during the winter and Spring quarters of 1952 reveal a mean of 102.86 and a standard de- viation of 21.06.11 The correlations obtained between psychological scores and test scores for each of the three tests are given in Table IX. Nichols12 stated the relation between his total battery of tests and intelligence (A.C.E.) as .54. The correlations between scores on the two parts of his test used in the 11 Although this is a representative sample of students en- tering in the_winter and spring quarters of 1952, it is significantly different from the results obtained by sampling 1,465 first term freshmen students who entered in the fall quarter of 1951. Their mean score was 106.12, standard deviation 20.92. The mean difference is 3.26, which gives a E.of 2.21; and which is significant at the 5% level of confidence. The average score of 3,083 stu- dents (including transfer students) who entered in the fall of 1949 was 105.11, and the standard deviation was 22.03. 12 Ralph G. Nichols, "Factors in Listening Comprehension", op. cit., 158. 120 TABLE IX CORRELATIONS BETWEEN TEST AND PSYCHOLOGICAL SCORES Test 1 Test 2 Test 3 Correlation3 .44 .51 .52 Number 113 63 59 a All correlations are significant at the 1% level. present study and the scores on the A.C.E. test obtained by the population used in this study were as follows: Nichols Test and 52 students taking Test 1: .69 Nichols Test and 29 students taking Test 2: .64 Nichols Test and 22 students taking Test 3: .56 Among the other studies primarily concerned with devel- oping a measure of listening comprehension, the following correlations have been stated: Blewett13 stated the relationship between his test and the A.C.E. to be .51 1.04. Brown14 reported the relation- ship between intelligence (A.C.E.) for college students and Part I of his test to be .369, and for Part II,.287. Prince15 reported the relation (rho) between A.C.E. scores 13 Thomas T. Blewett, o . cit., 54. 14 James I. Brown, "Construction of a Diagnostic Test of Listening Comprehension", op. cit., 87. 15 Bernice Prince, op. cit., 61. 121 and her measure of ”listening effectiveness" as .631. Heilman16 found the relationship between total scores on the A.C.E. and listening to be .56. When the total literature on listening - regardless of whether test construction was a major or a minor element - is considered, the highest cor- 17 who found a cor- relation reported was that of Goldstein, relation of .72 between the Otis test and listening; and the lowest was that reported by Knower18 of .27, between the Iowa Qualifying Examination and listening to informative speeches. Intelligence and the present tests. In view of the varying relationships that have been stated between intelli- gence and listening, what can be said about the present tests? The correlations of .44, .51, and .52, appear to be approximately between the extremes reported: they agree with those of Blewett and Heilman; they are higher than those of Brown and Knower; and they are lower than those of Gold- stein and Prince. Until more definite information between intelligence and listening is available, it would seem that b the only reasonable conclusion to be drawn is that the pres- ent tests, having a medium correlation with intelligence, 16 Arthur W. Heilman, op. cit., 306. 17 Harry Goldstein, op. cit., 51. 18 Franklin H. Knower, and others, op. cit., 87. .- .. - “a.-. 9de 122 are neither more nor less than other listening tests, a measure of intelligence. D. Validity Studied by Relation to Reading Ability Another factor commonly studied in validating listening tests has been the relation of listening ability to reading ability. Much of the motivation for the study of this re- lationship probably grew out of the desire of many experi- menters to discover how listening compared with reading as a means of obtaining information. Some of it derived from the desire to determine whether auditory presentation was more effective than printed visual presentation in the sale of advertised merchandise. Since there was a large amount of material reported concerning the relationship of listen- ing ability to reading ability, a similar relation was determined for the present tests to see how they compared with other findings. Total raw scores on the reading parts of the American Council of Education Psychological Examination were corre- lated with students' scores on the present tests to arrive at the data in Table X. Nichols'19 coefficient of correlation between his bat- tery of tests and reading comprehension (Iowa Silent Reading 9 . l lialph G. Nichols, "Factors in Listening Comprehension", OE. cit., 158. 123 TABLE X CORRELATIONS BETWEEN TEST AND READING ABILITY Test 1 Test 2 Test 3 Correlation8 .53 .39 .61 Number 115 55 59‘ a All correlations are significant at the 1% level. Examination) was .46. The two parts of his test used in this study by the present population correlated with reading (A.C.E.) as follows: Nichols Test and 52 students taking Test 1: .71 Nichols Test and 29 students taking Test 2: .77 Nichols Test and 22 students taking Test 3: .69 BlewettzO found a correlation of .35 between the first part of his test and reading comprehension, and a correlation of .52 between the second part of his test and reading com- prehension. Brown21 found the relationships between reading ability (Nelson-Denny Silent Reading Test) and Part I of his test to be .311; and Part II, .356. Heilman22 reported a correlation of .66 between listening and the COOperative ‘1 20 Thomas T. Blewett, op. cit., 54. 21 James I. Brown, "The Construction of a Diagnostic Test of Listening Comprehension", op. cit., 89. 22 Arthur W. Heilman, op. cit., 307. 124 Test of Reading Comprehension. Princezs found a rho of .679 between listening effectiveness and the Survey Section of the Triggs Diagnostic Test of Reading. Rankin24 correlated the mean of four reading tests with the corresponding lis- tening test for his fifth and seventh grade pupils and ob- tained a figure of .48. Goldstein's‘a5 correlation of .78 between the same printed tests on the same materials pre- sented on film and.record is among the high reports. Al- ; though in vocabulary studies Anderson and Fairbanks‘g6 re- ported a corrected correlation of .95. Reading and theppresent tests. The present tests, With correlations of .53, .39, and .61 between listening and reading, suggest a number of implications. The most obvious appear to be the following: The relation of reading ability to listening ability is highly variable. The differences between Test 2 and Test 3, are apparently not due to the in- troduction of a "reading" factor, because both of these tests were idential in the amount of reading material: The same duplicated materials (Instructions for Taking Listening Com- 23 Bernice Prince, op. cit., 61. 24 Paul T. Rankin, "Listening Ability, II", op. cit., 420. 25 Harry Goldstein, op. cit., 51. 26 Irving H. Anderson and Grant Fairbanks, op. cit., 319. 125 prehension Test, and Responses) were distributed to all stu- dents. That the inclusion of reading materials for a listening test may influence the results of the test was specifically noted by Blewett. He found that two parts of his test, one of which contained more reading than the other, correlated .35 and .52 respectively with reading ability. He points out that, This might suggest, but certainly does not establish, that the reading factor, when introduced into a listen- ing test, does have a contaminating influence upon the listening scores. . . Further research is needed to clarify this issue.27 The results of the present study are conflicting: Test 1, which had the least amount of reading (Responses Sheet only) had a correlation nearer to Test 3 than did Test 2, which had exactly the same amount of reading ma- terial as Test 3. The difference does not appear to be ex- plained by either the difficulty of the tests, nor by the type of material used in the passages. Although there may be some similarities between the "Instructions" passage used for Test 1 and the "Definition" passage used for Test 2, these similarities do not seem sufficient to account for the differences between the first two tests and the last. The variability in the relationship between each test and read- 7 Thomas T. Blewett, op. cit., 57. 126 ing ability is nearly as great as the variability expressed by some other studies. In view of the variations these conclusions seem per- tinent concerning the relation of listening ability to read- ing ability: (1) The present tests show a considerable vari- ation. (2) Other studies show considerable variation. (3) There is a need for further study using different types of listening tests; some with and some without reading; and us- ing different kinds of material. (4) The variability of the present tests is well within the extremes found by other studies, and the present tests may be considered "normal". E. Validity Studied in Relation to Sex Differences Because Nicholsz8 and some others29 had reported a dif- ference in comprehension favoring male listeners, a study was made of the scores on the respective tests for each of the two sexes. The mean and standard deviation of male and fe- male scores, the mean and standard deviation of the combined male and female scores, and the differences between the mean scores of males and females have been reported in Table XI. 28 Ralph G. Nichols, "Factors in Listening Compreh nsion", op. cit., 159. 9 Mary Burton, op. cit., 49. also J._Edwin Loder, op. cit., 56. 127 TABLE XI DIFFERENCES BETWEEN MALES AND FEMALES ON THE TESTS Test 1 Test 2 Test 3 Mean S.D. Mean S.D. Mean S.D. Males 11.84 5.79 14.42 5.63 17.41 5.63 Females 12.76 4.50 13.95 4.68 16.50 6.26 Combined 12.11 5.46 14.26 5.33 17.11 5.86 Differencea .93 .46 .91 t scoresa .96 .30 .55 a None significant 3O Kramer and Lewis report that, In addition to the original purpose of this study the listening ability of men and women was investigated but there was no conclusive evidence to indicate that there is a difference in the scores of men and women listeners. Even though there was a difference in the scores it was possible for such a difference because of the chance or sampling error alone. The results on the present tests agree with Kramar and Lewis. However, the‘writer would speculate that in view of the slight difference in favor of the girls for Test 1 (The only one in which the girls surpassed the boys) that in tests develOped by actual trial and subsequent item analysis the make-up of the trial group is an important influence in 30 Edward J. J. Kramar and Thomas R. Lewis, 0p. cit., 20. 128 determining whether the final test will tend to show a superiority for either sex. The conclusions to be drawn from this study are that differences between the sexes are small, probably due to chance; and that such differences do not appear to offer an index or guide to validity. F. Validity Studied by Reference to Course Content The instructional materials for the listening labora- tory are presented in duplicated form, not yet available for print. A detailed report of instructional material for the first term was presented by Irvin.31 The major aspects of the listening instruction are presented in short (approxi- mately ten minute) talks at the beginning of each labora- tory hour. The students then hear a thirty-five minute lec- ture. The last ten minutes of the period are devoted to re- view, and checking with students to see how they were able to apply the listening instruction to the specific lecture. Following are the titles of each unit, and a very brief statement of the content of the unit: 1. Listening as a Fourth Skill. This unit emphasized the amount of time students and others spend in listening. 31 Charles Irvin, 0p. cit., 116-155. I... -. .- i. A- .l. 129 2. The Kinds of Listening. This unit introduces the belief that listening can be improved and defines the kinds of listening. 3. Preparation for Listening. Here students are told how to come physically, emotionally and intellectually pre- pared to listen to lectures. 4. Exercising Emotional Control in Listening. This deals primarily with the undesirable influences of identi- f1 cation and projection upon the content of heard materials. 5. Structuralizing in Listening. This discusses the organization of lectures in the form of introduction, pur- pose, body, and conclusion. 6. Listening for Main Points. At this time the stu- dents are told how to use main points to aid them as lis- teners. 7. Listening for Comprehension. The last unit tells the students to detect the purpose; identify and relate main points; and to discriminate statements of fact from state- ments of principle, and ideas from examples illustrating those ideas. The material in the first four units is pertinent to Skill in listening, but it is a means to better listening, not; the content of listening itself. Units 5, 6, and 7 begin the actual content of listening material. This content 541:.I...‘ atl..Vn 1V V I «Is I IJ . .. m ..., 130 material is further developed in the second and third quar- ters of the course. An examination of the seven foils (Chapter II) and the particular items for the tests (Chapter V) indicates that the students are asked to find the purpose (or controlling idea) of a lecture (or passage) whether it be stated at the beginning, middle or end of the oral presentation; and that students are asked to find the major idea or main points; and to discriminate clearly between purpose and development of material. Furthermore, the test seeks to do this in vari- ous degrees by asking the student to distinguish related from unrelated materials, and still further to distinguish between exact and similar materials. In view of the factors emphasized in the course con- tent, and the nature of the test materials, it seems that the tests measure those factors considered significant in the c ourse itself . G. Validity Studied by Reference to Materials in the Speech Text and in Reading Manuals Used in the Course The implications for measuring listening comprehension derived from studying the speech text and the reading manuals used in conjunction with the course were reported in detail in Chapter III. . an. 1.. 31. i».n " . I: .a .. v. 131 Those implications which are important to comprehension as derived from the study of the speech text were summarized as follows: 1. The listener should comprehend clearly the central idea of the talk. The listener should clearly identify the main divi- sions of the talk. The listener should remember the main divisions of the talk. The listener should separate the essential from the non-essential; including the selection of the major aspects of information and various uses of develop- mental material. The listener shouldlvemember such developmental ma- terials (details) as are significant to the compre- hension of the ideas. Those implications which are important to comprehension as derived from the study of four reading manuals were sum- marized as follows: 1. 2. 3. The listener's ability to remember and/or identify a central idea. The listener's ability to remember and/or identify a detail (or details). The listener's ability to differentiate between central idea and detail. 132 4. The listener's ability to correctly identify a cen- tral idea derived from stated details. 5. The listener's ability to determine relationship or lack of relationship. _ 6. The listener's ability to select the best summary statement. 7. The listener's ability to identify the correctness, and/or relatedness of conclusions of a factual na- ture derived from total material. Those factors which appear to be emphasized in the speech text, and those factors whichsappear to be emphasized in the reading improvement manuals have been included in the present tests: first by the develOpment of the seven foils which contain the basic divisions of controlling idea, de- tails, and relatedness or lack of relatedness; and second, in the items which contain specific statements about mean- ingful material presented in the passages. To the extent that factors of comprehension derived from these sources represent the desirable content of lis- tening comprehension, the present tests are valid. H. Summary of Chapter This chapter has examined the validity of the present tests in relation to: (a) Two parts of the Nichols Test, and indicated that the correlations were positive and signifi- cant at the one per cent level of confidence; and although 133 not high, the correlations indicated that to the extent that the Nichols Test was a valid measure of listening comprehen- sion, the present tests were also valid measures of listen- ing comprehension. (b) Inspectional validity by three ex- perts, with the conclusions that two of the three authorities considered the tests valid. (c) "Intelligence", and indi- cated that the present tests, with correlations about midway between the extremes of the correlations that have been found to exist between listening and "intelligence", were as valid as the "average" of listening tests. (d) Reading ability, and concluded that the wide variation found between the dif- ferent tests and the wide variation reported by other studies indicated the need for continued investigation; and that the present tests are well within the extremes found by other studies, and may be considered "normal". (9) Differences in comprehension between the sexes, and concluded that such differences were small, probably caused by chance, and do not provide an index of validity. (f) The course units in listening instruction, and indicated that the factors em- phasized in the content of listening were included in the test. (g) Those factors emphasized as significant to com- prehension in the speech text and reading improvement manuals used in the course; and concluded that those factors were in- cluded in the test; and to the extent that such factors are valid indices of the content of listening tests, the present tests were valid. CHAPTER VIII CHAPTER VIII SUMMARY - THE THREE TESTS The purpose of this study was to develop a test, capa- ble of being administered in the listening ”laboratory" of the Michigan State College course in Communication Skills, that would measure the amount or degree of listening compre- hension skill possessed by freshmen. The preceding chapters have reported the background leading to the need for a listening test; the specific fac- tors to be emphasized; the conditions influencing the type and length of the'test; the deve10pment of the seven foils; the selection of the passages; the development and refine- ment of the items; and theistudy of the tests for reliabil- ity and validity. This chapter reports the complete material for each of the three tests developed. It should be noted that Test 1, in its present form is too difficult for use without further revision. Test 2 and Test 3 are now suitable for use in the listening "labora- tory" of.the freshman Communication Skills course at Michi- gan State College. 136 Test 1 RESPONSESI Read carefully before beginning test. Please return with answer sheet. Keep before you for reference. 1. This is a guotation (exact repetition) of the central .1222 Of the passage. It was actually stated in these same words in the passage. ~ Note: Other terms used for "central idea” are: Controlling idea, thesis, thesis sentence, purpose, purpose sentence, topic sentence, etc. Most peOple use many words to convey just one idea. They explain with more words, give examples, etc. But all of these words attempt to present one idea clearly. This one idea they are present- ing we call a central idea. 2. This statement implies or suggests the same central idea as the passage. It "truly" represents the central idea of the passage, but it is stated in different words from those used in the passage. 3. This is a mis-statement or mis-representation ("false", twisted, deceiving, partially true, partially false) of the central idea of the passage. 1 To be printed, and distributed to each student. 4. 5. 137 This is a qgotation (exact repetition) of one of the details in the passage. It was actually stated in these same words in the passage. Note: Other terms used for ”detail" are: ”Sub-point, sub- idea, supporting material, developmental material, etc. The materials (words) that a speaker uses to clarify and make vivid any central idea are called details. A speaker may use as many details as he wishes. The most commonly used details are explan- ation, description, definition, illustration, ex- ample, statistics, comparison, and authoritative statement. Details may be said to be any material used to clarify or make meaningful a central idea. ,This statement implies or suggests one of the details, or a similar detail, stated in the passage. It "truly" represents a detail in the passage, or a similar detail, but it is stated in different words from those used in the passage. This is a mis-statement or mis-representation ("false', twisted, deceiving, partially true, partially false) of a detail in the passage. This statement is unrelated to either the central idea or a detail of the passage. It is neither a "true" nor a ”false" quotation or implication of the central idea or a detail; but is rather entirely unrelated to either the 138 central idea or a detail of the passage. The information is neither given in nor implied by the passage. SHORTER STATEMENT OF RESPONSES (A) Those concerned.with the central idea. 1. Quote. Exact repetition of central idea. 2. Same central idea, but in different words. 3. False, twisted, or wrong central idea. (B) Those concerned with the details. 4. Quote. Exact repetition of a detail. 5. Same, or similar detail, but in different words. 6. False, twisted, or wrong detail. (0) The one concerned with both central idea and details. 7. Unrelated. Not given or implied. 139 Test 1: Explanation of Responses2 Your attention please! Please take up the paper that has RESPONSES at the tOp. (Pause) ‘You will need this Responses sheet when you take the listening test. Look first at number 1. Note that number 1 refers to a guoted statement of the central or con- trolling idea of anything you may hear or read. For ex- ample, my purpose in talking about the Responses sheet is to have you understand it clearly. To state it another way: my major aim or thesis is to explain the Responses sheet you are now examining. , Ybu have many different backgrounds and to try to use one word that expresses the same idea to all of you is dif- ficult. Therefore, you will note after the explanation of response 1 that there are a series of words. (Pause) Any one of those words is intended to show you what we are call- ing a "controlling idea", or "central idea”. Some of you have been taught to call it a thesis, thesis sentence, pur- pose sentence, or topic sentence, etc. But whatever you have been taught to call it, the term central idea represents the major, over-all,sipgle idea conveyed by a speaker when he talks, or a writer when he writes. Look quickly at 2, and note that this refers to a cen- tral idea you have heard or read, but this time instead of 2 To be recorded or read by examiner: Preferably recorded. 140 being stated in the actual words of the speaker or writer, it presents the same idea but in different words. Response 3 indicates a false or twisted statement of a central idea. For example, if I should say that, ”My pur- pose in talking to you now is to tell you how these 7 re- sponses were deve10ped", I would be making a misstatement of the central idea, for you have already been told that the aim is to explain these responses to you. Next look at number 4. We have departed from the cen- tral idea and are now talking about the separate pieces that a speaker or a writer uses to communicate a complete idea to you. Here again, the terminology may not be familiar to all of you. So, several different words have been indicated to suggest what we are calling a "detail”. Observe that response 4 is an exact repetition of something you have heard or read. Now look quickly at 5. This response indicates a true statement of a detail that you may have heard or read, but put in different words from the way it was originally pre- sented to you. Response 6 is a false or twisted statement of a detail that you may have heard or read. For example, if I should say, "Response 6 is a correct statement about a detail", I would certainly be making a false statement about response 6 as you have it before you. The last response, number 7, indicates a statement that is unrelated to what you hear or read. It may also indicate 141 a statement on which you have no information to make any judgment. Place this responses sheet where you can easily refer to the Shorter Statement of Responses. (Pause) It is at the bottom of the page. (Pause) ‘You will want to refer to those later. Now, pay no more attention to the responses sheet for the next few minutes. (Pause) 142 Test 1: Emphasizing that Test is to be on Instructions.3 Your attention please! This is very important. This passage you are about to hear gives the instruc- tions for taking this listening test. 'You will be tested on these instructions, and only on these instructions to see how well you understand them. For example, let us say that some one is giving you in- structions on how to drive a car. When.he completes his ex- planation he says, "Now, before you try driving, tell me what I said.” In the same way, you.will receive instructions for tak- ing this test. When the passage on instructions has been completed you will be asked, ”What did they say?” The next material you hear will be the passage for Test 1, titled "Instructions". (Pause) 3 To be recorded. All materials to be read at the rate of 100 words per minute. 143 Test 1: Passage4 INSTRUCTIONS This is a test of listening comprehension. Just as you take reading comprehension tests to learn how much you under- stand when you read, this test is designed to tell you how much you understand when you listen. This passage tells you how to take the test. First, do not make any noise or you may prevent your- self and others from hearing the necessary material. Second, do not take any notes; just listen attentively. Third, listen carefully for you will need the information you are now hearing to do this test. After you hear this passage on instructions you will hear a number of statements. Any statement pgy'or ppy_ppp be related to these instructions you are now hearing. ‘You are to assign each statement to one of seven possible re- sponses. As soon as you hear a statement you should decide which of the seven possible responses that statement belongs in, and mark the appropriate space on your answer sheet i2- mediately, For example, if you should hear the statement, ”How to listen to a classroom lecture", you.would.mark space 7, for that statement is entirely unrelated to this passage on instructions for taking the test. 4 To be recorded. 144 Each statement will be given only once; it will not be repeated. Throughout the test you should mark only ppp_space for each statement. If you don't know, guess. “Your score will be computed on the number of correct answers. This completes the instructions for taking the test. No additional information will be given. Do not ask any questions. See how well you can classify some statements about the instructions you have just heard. End of Passage for Test 1. 145 Test 1: Items5 Number 1 on your answer sheet. ". . . Do not take any notes. . .” (4)6 (20 second pause) Number 2. ‘You should mark 7 answers for each question. (6) (19 second pause) Number 3. Instruction for taking the test is far from the main idea of the first passage. (3) (18 second pause) Number 4. It is better to answer every question on this test than to leave one blank. (5) (17 second pause) Number 5. "Each statement will be given only once." (4) (16 second pause) Number 6. Directions for the test. (2) (15 second pause) Number 7. The first passage will tell you nothing about the test. (3) (14 second pause) Number 8. Hearing and listening are sometimes considered the same. (7) (13 second pause) 5 To be recorded. The correct response is indicated in parentheses after each item. 146 Number 9. YOur score will be computed on the number right minus the number wrong. (6) (12 second pause) Number 10. The directions for the test are in the first passage. (2) (11 second pause) 11. The main ideas are the most important. (7) (10 second pause. All following items have a 10 second pause after each item.) 12. ”. . . the instructions for taking the test.” (1) 13. You.have several choices for each question. (5) 14. Instructions for taking a listening comprehension test. (2) 15. ". . . mark the appropriate space on your answer sheet immediately.” (4) 16. Listening is a valuable skill. (7) 17. How to take any listening test. (3) 18. Each statement you hear will be related to the passage. (6) 19. The example for response 7 was listening to a class- room lecture. (5) 20. The passage tells you how to select the correct answer for each statement. (3) 21. ‘You should give careful attention to the information. (5) 22. ”This passage tells you how to take the test." (1) 23. 24. 25. 26. 27. 28. 29. 30. 31. 32. 147 "Ybur score will be computed on the number of correct answers.” (4) we should try to improve our listening skill. (7) Instructions on how to take the test. (2) "You should wait a few minutes before deciding which response a statement belongs in. (6) Improved listening should be a part of education. (7) Answer every question whether you know the answer or not. (5) The instructions for the test are not a part of the test. (3) The directions for the test. (2) "If you don't know, guess.” (4) Mark only those answers that you are sure of. (6) (10 second pause) This is the end of Listening Test 1. 148 Test 2: /? INSTRUCTIONS FOR TAKING LISTENING COMPREHENSION TEST7 Read carefully before taking test. Return with answer sheet. This is a test of listening comprehension. Just as you take reading comprehension tests to learn how much you under- stand when you read, this test is designed to tell you how much you understand.when you listen. Do not make any noise during the test or you may pre- vent yourself and others from hearing the necessary material. You will first hear a short passage about three minutes long. Do p33 take any notes: just listen attentively to the passage. After you have heard the passage you will hear a number of statements. Any statement 231 or mg: p23 be related to the passage you have justlleard. You are to assign each statement to one of 7 possible choices. As soon as you hear a statement you should decide which of the 7 possible responses that statement belongs in; and you should mark the appropriate space on your answer sheet immediategy. Each statement will be given only once; It will not be repeated. ‘You.will, however, have adequate time to mark your answer sheet before you hear the next statement. 7 To be printed, and distributed to each student. 149 Throughout the test mark only one space (response) for each statement. Do not mark more than one space for any statement, or your answer for that question will be dropped from your score. If you don't know, guess. Your score will be computed on the number of correct answers. Now, re-read the 7 choices shown on the Responses page. Note that responses 1, 2 and 3 are different ways of talking about the central idea of the passage you will have heard; 4, 5, and 6 are different ways of talking about a detail in the passage; and response 7 indicates that a statement is unrelated to either the central idea or a detail. Fill in the top of your answer sheet according to the directions given by your instructor. Place the Responses page where you can refer to it while you are hearing the statements and marking your answer sheet. Then be prepared to listen carefully to the practice exercise, the passage, and then to the statements you will hear. You should have: 1. This page: Instructions for Taking Listening Comprehension Test. 2. Responses. 3. Answer sheet. 4. Electrographic pencil. 150 Test 2 RESPONSESB Read carefully before beginning test. Please return with answer sheet. Keep before you for reference. 1. This is a Quotation (exact repetition) of the central gggg of the passage. It was actually stated in these same words in the passage. Note: Other terms used for "central idea" are: Control- ling idea, thesis, thesis sentence, purpose, pur- pose sentence, topic sentence, etc. Most people use many words to convey just one idea. They explain with more words, give examples, etc. But all of these words attempt to present one idea clearly. This one idea they are present- ing we call a central idea. 2. This statement implies or suggests the same central idea as the passage. It "truly" represents the central idea of the passage, but it is stated in different words from those used in the passage. 3. This is a mis-statement or mis—representation ("false”, twisted, deceiving, partially true, partially false) of the central idea of the passage. 8 To be printed, and distributed to each student. 4. 151 This is a quotation (exact repetition) of one of the g3: ' tails in the passage. It was actually stated in these 5. same words in the passage. Note: Other terms used for "detail” are: ”sub-point, sub- idea, supporting material, develOpmental material, etc. The materials (words) that a speaker uses to clarify and make vivid any central idea are called details. A speaker may use as many details as he wishes. The most commonly used details are ex- planation, description, definition, illustration, example, statistics, comparison, and authoritative statement. Details may be said to be any material used to clarify or make meaningful a central idea. This statement implies or suggests one of the details, or a similar detail, stated in the passage. It "truly" represents a detail in the passage, or a similar detail, but it is stated in different words from those used in the passage. This is a mis-statement or mis-representation (“false”, twisted, deceiving, partially true, partially false) or a detail in the passage. This statement is unrelated to either the central idea or a detail of the passage. It is neither a "true" nor a “false" quotation or implication of the central idea or a detail; but is rather entirely unrelated to either the 152 central idea or a detail of the passage. The information is neither given in nor implied by the passage. SHORTER STATEMENT OF RESPONSES (A) Those concerned.with the central idea. 1. Quote. Exact repetition of central idea. 2. Same central idea, but in different words. 3. False, twisted, or wrong central idea. (B) Those concerned with the,details. 4. Quote. Exact repetition of a detail. 5. Same, or similar detail, but in different words. 6. False, twisted, or wrong detail. (C) The one concerned with both central idea and details. 7. Unrelated. Not given or implied. 153 Test 2: Practice Session9 ‘Your attention please: This is a practice session. You should do this part of the test, but you will not be scored on it. we will use numbers 41, 42, and 43 on your answer sheets. (Pause) Here is a very short passage. This is practice passage There are a great many situations in which we listen. We go to church or synagogue and listen to the minister, priest or rabbi; we go to the movies and look and listen; we turn on the radio and listen; we participate in a conversation and we talk and listen; we sit in class and listen to lectures, etc., etc. End of practice passage X. (Pause) Put your ”Responses Sheet" where you can easily refer to the "Shorter Statement of Responses”. It is at the bottom of the page. (Pause) Practice statement number 41. The main idea of the passage is how to listen. Is this statement related to the passage you heard? Vaguely, yes. So the correct response cannot be number 7. (Pause) Is the state- ment about the ppplp_passage or is it about one of the de- tails in the passage? The statement said, "The main idea of the passage is how to listen.” Therefore, there are two stated cues telling you that the statement refers to the 9 To be recorded. 154 central, controlling idea of the whole passage. These cues are the words ”main idea", which is just another way of say- ing "central idea"; and the word "passage", which is the ppplg.thing you heard. So, the statement is about the cen- tral idea, and not a detail. Therefore, the answer must be in response 1, or 2, or 3. (Pause) Is it a direct quotation of the central idea? No. So, response 1 is out. Is it a statement representing the correct idea of the entire pas- sage? No, because the central idea of the passage was about the mgpy situations in which we listen. Nothing was said about pg! to listen. So, response 2 is out. Is the state- ment a mis-representation of the central idea? Yes. So, we mark space 3. (Pause) Practice statement number 42. "If we go to a pep- rally we listen to speakers". Ask yourself, "Is this state- ment related to the passage?“. In a way, yes. It is another example, or detail, of one of the many situations in which we listen. So, response 7 is wrong. Is the state- ment about the whole passage? No, we already reached that decision when we decided that the statement was just another example or detail; so we know that our answer must lie in response 4, 5, or 6. Is it a quote of one of the details? No, pep rallies were not mentioned in the original passage. Therefore, the answer can't be response 4. But, does the statement suggest or imply a detail "truly" similar to those given in the passage? Yes, it is another similar detail. 155 So, we mark response 5, and.wait for the next statement. (Pause) Practice statement 43. "There are a great many situa- tions in which we listen". This is an exact repetition of a sentence in the practice passage. Does this quotation express the one, over-all controlling idea of the passage; or does it express one of the details? The quotation doesn't mention any details whatsoever; it is just a gen- eralized statement of the main idea of the whole passage. Therefore you mark space 1 on your answer sheet. This is the end of the practice session. (Pause) Be prepared to listen to the passage for Test 2 titled, ”Definition”. 156 Test 2: Passage10 DEFINITION As human beings we are constantly bombarded with all kinds of stimuli from the world around us. we see things with our eyes, taste with our tongues, smell with our noses, and feel with our senses of touch. For the next few minutes we will confine our attention to those stimuli we receive through the ear - those called sound - in an attempt to answer the question, What do we mean by the term listening? The first important factor in our definition of listen- ing is the reception of sound. When listening to television, sound-movies, or a speaker before us, we receive both audi— tory and visual impressions. When.we listen to the tele- phone, radio, recordings, or a person talking even though we cannot see him, we depend exclusively on the ear. The visual element may or may not be present. If present it is a further aid to understanding; but visual impressions may be excluded, and we can still listen. Second, we should note that we receive a great many different kinds of impressions through our ears. A broad classification of sound ranges from noise to music. we shall limit these impressions to word sounds or language symbols. So, in this definition we include only a person 10 To be recorded at approximately 100 words per minute. 157 talking; and we exclude all other auditory impressions such as train whistles, instrumental music, telegraphic code, tom-toms, etc. Third, we must have an attentive receiver. The most significant factor of listening is not that it involves sound, nor language symbols, but that these symbols are interpreted by a receiver to have meaning. If a recording of a speech is playing, but you are not present to hear it, we say you are neither hearing nor listening. If you are present, but looking out the window day-dreaming you may be hearing, but you are not listening. we can summarize our definition in this way: Listen- ing is the act of an attentive person interpreting word symbols received primarily through the auditory sense. End of Passage for Test 2. 158 Test 2: Items11 Number 1 on your answer sheet. "When listening to tele- .vision, sound-movies, or a speaker before us, we receive both auditory and visual impressions." (4)12 (20 second pause) Number 2. Listening is often improved by day-dreaming be- cause the latter brings new ideas to mind. (6) (19 second pause) Number 3. The definition, as a whole, included all the sound we can hear. (3) (18 second pause) Number 4. Day-dreaming may prevent listening. (5) (17 second pause) Number 5. '. . . Visual impressions may be excluded. . ." (4) (16 second pause) Number 6. The definition had three main parts: sound, type of sound, and an attentive interpretation. (2) (15 second pause) Number 7. This was mainly an attempt to clarify our under- standing of how we hear. (5) 11 To be recorded. 12 The correct response is indicated in parentheses after each item. 159 (14 second pause) Number 8. Deaf people often read the lips of a speaker to get his meaning. (7) (15 second pause) Number 9. If you are present, but looking out the window, you may be listening, but you are not hearing. (6) (12 second pause) Number 10. Listening is the act of hearing and understanding what another person says. (2) (ll second pause) 11. Blind persons are usually the best listeners. (7) (10 second pause after this item, and all remaining items.) 12. "Listening is the act of an attentive person interpret- ing word symbols received primarily through the auditory sense.” (1) 13. Sound may be classified along a scale having music at one end, and mere noise at the other. (5) 14. What does the word listening mean? (2) 15. "The first important factor in our definition of listen- ing is the reception of sound." (4) 16. Effective listening is one of the earmarks of a good student. (7) 17. The passage was concerned mainly with explaining the process by which we hear sound. (5) 18. 19. 20. 21. 22. 23. 24. 25. 26. 27. 28. 29. 30. 31. 160 The three significant factors in listening are: atten- tion, sight, and language symbols. (6) Visual cues or impressions may help us listen, but such impressions are not essential. (5) Listening is any means of conveying ideas between two or more people. (5) Some listening situations also involve sight. (5) ”What do we mean by the term listening?" (1) ". . . we exclude all other auditory impressions. . ." (4) More education is received through the eye than through the ear. (7) The passage is concerned primarily with limiting the term listening. (2) The third part of the definition states that we receive a great many impressions through our ears. (6) Most authorities state that seeing is more important to life than is hearing. (7) we use the ear to give us all meanings when we cannot see the speaker. (5) The complete definition consisted of fourxnajor points or parts. (3) 'You heard a definition of the word listening. (2) "A broad classification of sound ranges from noise to znusic.” (4) 161 32. It is very obvious that we hear only a few types of sound. (6) (10 second pause) This is the end of Listening Test 2. 162 Test 5: INSTRUCTIONS FOR TAKING LISTENING COMPREHENSION TEST15 Read carefully before taking test. Return with answer sheet. This is a test of listening comprehension. Just as you take reading comprehension tests to learn how much you under- stand when you read, this test is designed to tell you how much you understand when you listen. Do notrnake any noise during the test or you may prevent yourself and others from hearing the necessary material. You will first hear a short passage about three minutes long. Do not take any notes: just listen attentively to the passage. After you have heard the passage you will hear a number of statements. Any statement E2E_°r.EEI.EEE be related to the passage you have just heard. ' You are to assign each statement to one of 7 possible choices. As soon as you hear a statement you should decide Which of the 7 possible responses that statement belongs in; and you should mark the appropriate space on your answer sheet immediately. Each statement will be given only once: It will not. be repeated- You will, however, have adequate time to mark Your answer sheet before you hear the next statement. __ 13 To be printed, and distributed to each student. 163 Throughout the test mark only one space (response) for each statement. Do not mark more than one space for any statement, or your answer for that question will be dropped from your score. If you don't know, guess. Your score will be computed on the number of correct answers. Now, re-read the 7 choices shown on the Responses page. Note that responses 1, 2 and 5 are different ways of talking about the central idea of the passage you will have heard; 4, 5, and 6 are different ways of talking about a detail in the passage; and response 7 indicates that a statement is unrelated to either the central idea or a detail. Fill in the top of your answer sheet according to the directions given by your instructor. Place the Responses page where you can refer to it while you are hearing the statements and marking your answer sheet. Then be prepared to listen carefully to the practice exercise, the passage, and then to the statements you will hear. 'You should have: 1. This page: Instructions for Taking Listening Comprehension Test. 2. Responses. 3. Answer sheet. 4. Electrographic pencil. 164 Test 3 RESPONSESM‘ Read carefully before beginning test. Please return with answer sheet. Keep before you for reference. 1. This is a Quotation (exact repetition) of the central ‘iggg of the passage. It was actually stated in these same words in the passage. ' Note: Other terms used for "central idea" are: Control- ling idea, thesis, thesis sentence, purpose, pur- pose sentence, topic sentence, etc. Most people use many words to convey just one idea. They explain with more words, give examples, etc. But all of these words attempt to present one idea clearly. This one idea they are present- ing we call a central idea. 2. This statement implies or suggests the same central idea as the passage. It "truly" represents the central idea of the passage, but it is stated in different words from those used in the passage. 3. This is a mis-statement or mis-representation (”false", twisted, deceiving, partially true, partially false) of the central idea of the passage. 14 To be printed, and distributed to each student. 165 4. This is a Quotation (exact repetition) of one of the def tailg in the passage. It was actually stated in these same words in the passage. Note: Other terms used for "detail" are: "Sub-point, sub- idea, supporting material, deve10pmenta1 material, etc. The materials (words) that a speaker uses to clarify and make vivid any central idea are called details. A speaker may use as many details as he wishes. The most commonly used details are ex- planation, description, definition, illustration, example, statistics, comparison, and authoritative statement. Details may be said to be any material used to clarify or make meaningful a central idea. 5. This statement implies or suggests one of the details, or a similar detail, stated in the passage. It "truly" represents a detail in the passage, or a similar detail, but it is stated in different words from those used in the passage. 6. This is a miséstatement or mis-representation (”false", twisted, deceiving, partially true, partially false) of a detail in the passage. 7. This statement is unrelated to either the central idea or a detail of the passage. It is neither a "true" nor ”false" quotation or implication of the central idea or a detail; but is rather entirely unrelated to either the 166 central idea or a detail of the passage. The information is neither given in nor implied by the passage. SHORTER STATEMENT OF RESPONSES (A) Those concerned with the central idea. 1. Quote. Exact repetition of central idea. 2. Same central idea, but in different words. 3. False, twisted, or wrong central idea. (B) Those concerned with the details. 4. Quote. Exact repetition of a detail. 5. Same, or similar detail, but in different words. 6. False, twisted, or wrong detail. (<3) The one concerned with both central idea and details. 7. Unrelated. Not given or implied. 167 Test 3: Practice Session15 Your attention please: This is a practice session. You should do this part of the test, but you will not be scored on it. we will use numbers 41, 42, and 43 on your answer sheets. (Pause) Here is a very short passage. This is practice passage There are a great many situations in which we listen. We go to church or synagogue and listen to the minister, priest or rabbi; we go to the movies and look and listen; we turn on the radio and listen; we participate in a conversation and we talk and listen; we sit in class and listen to lectures, etc., etc. End of practice passage X. (Pause) Put your ”Responses Sheet" where you can easily refer to the "Shorter Statement of Responses". It is at the bottom of the page. (Pause) Practice statement number 41. The main idea of the passage is how to listen. Is this state- ment related to the passage you heard? Vaguely, yes. So the correct response cannot be number 7. (Pause) Is the statement about the Epplg_passage or is it about one of the details in the passage? The statement said, "The main idea of the passage is how to listen." Therefore, there are two stated cues telling you that the statement refers to the 15 To be recorded. 168 central, controlling idea of the whole passage. These cues are the words "main idea", which is just another way of say- ing "central idea"; and the word "passage", which is the Epplp_thing you heard. So, the statement is about the cen- tral idea, and not a detail. Therefore, the answer must be in response 1, or 2, or 3. (Pause) Is it a direct quotation of the central idea? No. So, response 1 is out. Is it a statement representing the correct idea of the entire pas- sage? No, because the central idea of the passage was about the 2521 situations in which we listen. Nothing was said about pg! to listen. So, response 2 is out. Is the state- ment a mis-representation of the central idea? Yes. So, we mark space 3. (Pause) Practice statement number 42. "If we go to a pep— rally we listen to speakers". Ask yourself, "Is this state- ment related to the passage?" In a way, yes. It is an- other example, or detail, of one of the many situations in which we listen. So, response 7 is wrong. Is the state- 'ment about the whole passage? No, we already reached that decision when we decided that the statement was just another example or detail; so we know that our answer must lie in response 4, 5, or 6. Is it a quote of one of the details? No, pep rallies were not mentioned in the original passage. Therefore, the answer can't be response 4. But, does the statement suggest or imply a detail "truly" similar to those given in the passage? Yes, it is another similar detail. 169 So, we mark reSponse 5, and wait for the next statement. (Pause) Practice statement 43. "There are a great many situa- tions in which we listen". This is an exact repetition of a sentence in the practice passage. Does this quotation express the one, over-all controlling idea of the passage; or does it express one of the details? The quotation doesn't mention any details whatsoever; it is just a gen- eralized statement of the main idea of the whole passage. Therefore you mark space 1 on your answer sheet. This is the end of the practice session. (Pause) Be prepared to listen to the passage for Test 3 titled, ”Listening Can Be Taught”. 170 Test 3: Passage16 LISTENING CAN BE TAUGHT One of the basic assumptions about the skill of listen- ing is that listening can be taught. Following are brief summaries of two research studies testing that assumption: In 1949, Professor Arthur Heilman did a study to measure the improvement of listening brought about by di- rect instruction. Heilman met students in the usual small classroom situation. 'Thestudents received direct instruc- tion in listening for six separate periods of about ten minutes each. Tests were given these 450 students before and after instruction. Results showed that the listening ability of college freshmen could be significantly improved through a program of training. Such training appeared to be particularly effective with those students whose listen- ing ability was originally lower than average. In the fall of 1950, Professor Charles Irvin conducted a study involving large groups of students at Michigan State College. Seven ten-minute units of listening instruction were given to about half the freshmen students. The instruction was given in the regular large lecture situation. Students were tested prior to training and immediately afterward. 16 To be recorded at approximately 100 words per minute. 171 The 1400 students who received listening training did from 9 per cent to 12 per cent better than the 1400 who did not get training. However, the major gains were made by those students who were classified as "poor" listeners at the beginning of the term. Over 50 per cent of the poor listeners who re- ceived training raised themselves from the category of "poor” up to the category of "above average"; while only 11 per cent of the poor listeners who did not receive train- ing raised themselves to "above average". From these two studies it appears that the assumption "listening can be taught", is a safe assumption to make. End of Passage for Test 3. 172 Test 3: Items17 Number 1 on your answer sheet. "Heilman met students in the usual small classroom situation.” (4)18 (20 second pause) Number 2. The poorest listeners at the beginning of the term were the best listeners at the end of the term. (6) (19 second pause) NUmber 3. The assumption that listening can be taught is not substantiated by experiment. (3) (18 second pause) Number 4. Students who received training in listening im- proved more than those who did not receive train- ing. (5) (17 second pause) Number 5. ”. . . The major gains were made by those stu- dents who were classified as 'poor' listeners. . (4) (16 second pause) Number 6. Training apparently increases listening ability. (2) (15 second pause) 17 To be recorded. 18 The correct response is indicated in parentheses after each item. 173 Number 7. It is now assumed that listening does not improve (14 with training. (3) second pause) Number 8. More studies should be made of listeners and (13 listening. (7) second pause) Number 9. Professor Heilman used about 1400 students. (6) (12 second pause) Number 10. Instruction can be given that will improve a (11 11. (IO 12. 13. 14. 15. 16. 17. person's ability to listen. (2) second pause) Listening is a valuable activity in many learning situ- ations. (7) ~ second pause after this item, and all remaining items) ". . . The assumption 'listening can be taught', is a safe assumption to make." (1) Professor Charles Irvin directed a study at Michigan State College. (5) It is possible to teach listening. (2) "The students received direct instruction in listening for six separate periods of about 10 minutes each." (4) It is universally known that most students are poor lis- teners. (7) The purpose of the passage was to improve your listen- ing. (3) 174 18. Professor Irvin tested his 450 students before and after training. (6) 19. The students meeting in the regular classroom received a total of about 60 minutes of instruction. (5) 20. Listening ability is caught rather than taught. (3) 21. Some students improve in listening skill without in- struction. (5) 22. "One of the basic assumptions about the skill of lis- tening is that listening can be taught." (l)19 23. "Following are two research studies testing that as- sumption.” (4) 24. There are many different ways and techniques of teaching listening. (7) 25. We can probably learn to listen more effectively. (2) 26. Both large and small groups of students received the same amount of listening instruction. (6) 27. We should acquire the habit of good listening. (7) 28. Irvin's students received a total of about 70 minutes of instruction. (5) 29. The only way we can improve listening is by instruc- tion. (3) 30. Listening can be learned through competent instruction. (2) 19 This item gave a positive discrimination in the classroom trial, but a negative discrimination in the laboratory trial. See Chapter V. 175 31. "The instruction was given in the regular, large lec- ture situation.” (4) 32. In one experiment the students received direct instruc- tion; in the other they received indirect instruction. (6) (10 second pause) This is the end of Listening Test 3. BIBLIOGRAPHY 177 BIBLIOGRAPHY Adams, Harlan M., ”Listening”, aparterly Journal of Speech, XXIV (April, 1938), 209-11. Adams, Harlan H., "Teaching the Art of Listening”, The Nations Schools, XXXIV (November, 1944), 51-54. Adams, Harlan M., ”Learning to Listen; An English and Social Studies Plan”, The Clearing House, XX (March, 1946), 401 “'03 e Adams, Harlan M., "Learning to be Discriminating Listeners”, English Journal, XXXVI (January, 1947), 11-15. Adkins, Dorothy C., and others, Construction and Analysis of Achievement Tests, U. S. Governmenthrinting 0 co, Wishington,fi: CT, 1947, 291 pp. Anderson, Harold.A., "Teaching the Art of Listening", School Review, LVII (February, 1949), 63-67. Anderson, Irving H., and Fairbanks, Grant, ”Common and.Dif- ferential Factors in Reading Vocabulary and Hearing Vocabulary", Journal of Educational Research, XXX (January, 1937), 317¥24T Armentrout, W. D., ”Discriminate Listening Basic”, Jburnal of the Association for Education by Radio, V (FeBruary, 191377'87. Arndté7g.7o., ”Listen”, English Journal, XXIX (May, 1940), '80 Arndt, C. 0. "Listen" Ps chological Review' XLVIII (September, 1941): 33%31637" ' Beery, Althea, "Listening Activities in the Elementary School", Elementary English Review, XXIII (February, 1946), 69479. Bennett, W. E., "Speech in a Technological Society", Journal of Communication, I, No. 1 (May, 1951), 16-20. Blair, Herbert, "The Art of Listening", Education, LII (May, 1932), 563-64. 178 Blewett, Thomas T., ”An.Experiment in the Measurement of Listeni at the College Level”, Journal of Communica- tion, I May, 1951), 50-57. Brigance, William N., Speech Communication, F. S. Crofts, N. Y}, 1947, 220 pp. Brown, James I., ”A Comparison of Listening and Reading Ability", College English, X (November, 1948), 105-107. Brown, James I., ”The Construction of a Diagnostic Test of Listening Comprehension”, Unpublished Ph.D. thesis, University of Colorado, 1949, 184 pp. Brown, James I., "Why Not Teach Listening?", School and Society, LXIX (February, 1949), 141-153. Brown, James I., "The Construction of a Diagnostic Test of Listening Comprehension, Journal of Experimental Edu- cation, XVIII (December, 1949), 139-146. Bucklew, John, "An.Exploratory Study in the Psychology of Speech Reception Journal of Experimental Psychology, XXXII (June, 194s$,'373-94. Burrt, H. E., "The Curve of Forgetting for Advertising Material“, Journal of Applied Psychology, IX (March, 1925), 5-21. Burton, Mary, ”The Hearing and Reading Comprehension of Vocabulary.Among High School Seniors“, School Review, 52 (January, 1944), 47-50. Caffrey, John, "The Establishment of.AudingaAge Norms", Sghool and Society, 70 (November 12, 1949), 310-312. Cantril, Hadley, and Allport, Gordon.W., A Stud of Radio Listenipg, Columbia Broadcasting System, N. Y., 1934, PP- Carlson, Evelyn F., ”Effective Listening", Chicago Schools JOurnal, XIX (March, 1949), 187-91. Carver, merton.E., ”A Study of Conditions Influencing the Relative Effectiveness of Visual and Auditory Presenta- tions”, Unpublished Ph.D. thesis, Harvard University, 1934, 534 pp. Carver, Merton E., ”Listening Versus Reading”, Chapter IX, 159-180, in Hadley Cantril and Gordon Allport, The Psychology of Radio, Harper and Brothers, N. Y., 1935, PP- 179 Chall, Jeanne S., and Dial, Harold E., ”Predicting Listener Understanding and Interest in Newscasts", Educational Research Bulletin, XXVII (September, 1948), 141-53. Chapman, Lucy H., and Cauley, Thomas, ”Becoming An Efficient Listener“, Chapter X, 221-240. In La ua 9 Skills, Grade Ten, Harcourt, Brace, N. Y., 1927, §15 pp. Clark,‘Weston R., "Radio Listening Habits of Children”, Journal of Experimental Education, VIII (September, 1939), 44-48. Cole, Luella, Note Taking, Rinehart and Co., N. Y., 1946, 136 pp. Corey, Stephen M., ”Learning from Lectures vs Learni from Reading", Journal of Educational Psych316gy, 25 Septem- Cortright, Rupert L., ”A Technique for Measuring Perception Differences for Radio and Direct Audience Speaking,” Unpublished Ph.D. thesis, University of Michigan, 1935, 324 pp. Cotton, Jack C., "Do We Hear With Our Eyes?", Scientific American, CLIII (November, 1935), 2559 Coulter, V. C., ”Reading and Listening", Education, LXV (February, 1945), 375-82. Curtis, Alberta, "The Reliability of a Report on Listening Habits“, Journal of Applied'Psychology, XXIII (Febru- ary, 1931), 127-30. Darrow, Ben M., ”Can the School Teach Discrimination in Radio Listening?", Educational Method, XIV (March, 1935), 311-15. . ‘1 Dewick, Henry N., ”The Relative Recall Effectiveness of Visual and.Auditory Presentation of Advertising Ma- terial", Journal of Applied Psychology, XIX.(June, 1935),345-64. ‘Dias, Earl J., "Three Levels of Listening", English Journal, XXXVI (May, 1947), 252-54. Dietze, Alfred 0., "Some Sex Differences in Factual Memory”, American Journal of ngcholpgy, XLIV (May, 1932), 319- Dolman, John, Jr., "From the Listener's Point of View", Quarterly Journal of Speech, XX (April, 1934), 203-06. 180 Elliot, Frank R., ”Eye vs Ear in Advertising", Sales Manage- ment, XXXVI (March:_1935), 196-199. Elliot, Frank R., "Memory for Visual, Auditory, and Visual- Auditory Materials", Archives of Psychology, No. 199 (1936), 29-36. Erickson, C. I., and King, I., ”A Comparison of Visual and Oral Presentation of Lessons in the Case of Pupils from the Third Grade to the Ninth Grades", School and 8001931, VI (August, 1917), 146-148. Ewing, William H., "Finding a Speaker-Listening Index", guarterly Journal of Speech, XXXI (October, 1945), 368- Fessenden, Seth A., "Listening”, 25-37. In S eech and the Teacher, Longmans, Green and Co., N. Y., 1946, 243 pp. Finch, Mildred H., ”Just Listen! The Neglected 4th Phase of Communication”, Clearing House, XX (May, 1946), 534-36. Finkenbinder, E. 0., ”The Curve of Forgetting”, American Journal of Psychology, XXIV (October, 1913), 8-11. Flanagan, Jehn C., "General Considerations in the Selection of Test Items and A Short Method of Estimating the Product-Moment Coefficient from the Tails of the Dis- tribution", Journal of Educational Psychology, 30 (1939), 574-353. Freed, Conrad.W}, "Silent Conditioning in the Schools“, gggrterly Journal of Speech, XXVII (April, 1941), “19‘. Frohardt, Kenneth A., "A Study of the Teaching of Listening", Unpublished M.A. thesis, University of Nebraska, 1949,. 196 pp. Garrett, H. E., Statistics in Ps cholo and Education, Longmans, Green andfiCSZ, N. 1., 1956,1388 pp. Gates, A. I., "Experiments on the Relative Efficiency of Men and'WOmen in Memory and Reasoning", Psychological Re- view, XXIV (May, 1917), 193-197. Glaze, J. E., ”The Association Value of Nonsense Syllables", Journal of Genetic Psycholqu, xxxv (February, 1929), 255 '259. 181 Goldstein, Harry, Readi and Listening Comprehension at Various Contr011ed gétes, Contributions to Education, No. 621,Bureau of PuBlIcations, Teachers College, Columbia University, New Yerk, 1940, 69. Greene, Edward B., "The Relative Effectiveness of Lecture and Individual Reading as Methods of College Teaching”, ggpggégPsychology Monographs, IV (December, 1928), Harwood, Kenneth A., ”An Experimental Comparison on Listening Comprehensibility with Reading Comprehensibility”, Un- published Ph.D. thesis, University of Southern Califor- nia, 1950, 157 pp. Hawkes, Herbert E., Lindquist, E. F., and Mann, G. R. (Editors) The Construction and Use of Achievement Examinations, Boughton—Mifflin Co., Boston,l936, 496 pp. Heilman,.Arthur‘W., "Measuring and Improving Listening Ability", Speech Monographs, XVIII (No. 4, 1951), 302- 308. Hendrickson‘ Amanda E., ”How to Teach'Yo People to Listen , North Dakota Teacher, XXVII February, 1948), 17-18. Henmon, V. A. 0., "Modes of Presentation and Retention”, Psychological Review, XIX (January, 1912), 79-83. Heron, W, T., and Ziebarth, E. N., ”A Preliminary Experimental Comparison of Radio and Classroom Lectures“, Speech Menoggapha, XIII, NO. 1 (1946), 206-208. Heye, Helen, “A Study of the Effectiveness of Selected Auditory Presentations at the Adult Level”, Unpublished Ph.D. thesis, State University of Iowa, 1941, 216 pp. Hill, F. E., "What Do They Learn", 185-186. In.£isten and Learn, George Grady Press, N. Y., 1937, 300 pp. Hollingsworth, H. L., Psychology of the Audience, American Book Company, N. ., , pp. Hood, Leon C., "Radio Listening Interests as Classroom Tools”, New Jersey English Leaflet, (May, 1947). Hook, J. N., ”Developing Good Listeners”, Journal of Educa- tion, CXXXII (April, 1949), 110, 112-111. 182 Hunter, W, 8., "Experimental Studies of Learning”, 13-19. In Handbook of ggperimental Ps cholo , Edited by Carl MurEEison, ar niversity Press, Worcester, Mass., 1934. Hurrell, Thomas N., "Factors Influencing Preference and Memory for Auditory Rhythm", Unpublished Ph.D. thesis, State University of Iowa, 1936, 216 pp. Irvin, Charles E., "Lend Me Ybur Ears", Chicago Schools Journal, XXXI (May, 1950), 265-266. Irvin, Charles E., ”An.Ana1ysis of Certain.Aspects of a Listening Training Program Conducted Among College Freshmen at Michigan State College", Unpublished.Ed,D. thesis, Michigan State College, 1952, 160 pp. Jayne, Clarence D., "A Study of the Learning and Retention of Materials Presented by Lecture and by Silent Film”, Journal of Educational Research, XXXVIII (September, , -80 Johnson, Kenneth 0., ”Effect of Classroom Training Upon Listening Comprehension”, Jeurnal of Communication, I (May, 1951), 57-62. Jehnson,‘lendell, People in Qpandaries, Harper and Brothers, N.‘Y., 1946,'532 pp. Jehnson,‘wende11, “The Spoken/Word and the Great Unsaid" §uarterly Journal of Speech, XXXVII (December, 1951 , Judson, L. 8., “Objective Studies of the Influence of the Speaker on the Listener", Journal of Expression, IV (May, 1929), 1-11. Kaulfers,'walter V., "wartime DeveloPments in Modern.Language Achievement Testing", Modern Language Journal, XVIII (1944), 139-150. Kennedy, A. H., ”A Study of Children's Hearing as It Relates to Reading", Unpublished Ph.D. thesis, Department of Education, University of Chicago, 1941, 247 pp. Kncwer, Franklin; Phillips, David; and Koeppel, Fern, ”Studies in Listening to Informative S eaking", JOurnal of Abnormal and Social Ps cholo , 40 February, 1915), 55‘88 e 183 Koch, H. L., ”Some Factors Affecting the Relative Efficiency of Certain Modes of Presenting Material for Memorizing , American Journal of Psychology, XLII (June, 1930), Kramer, Edward J. J., and Lewis, Thomas R., ”Comparison of Visual and Nonvisual Listening", Journal of Communication, I (November, 1951), 16-20. Krawiec, Theophile, "A Comparison of Learning and Retention of Materials Presented Visually and Auditorially”, Un- published Ph.D. thesis, New York University, 1945, 380 pp. Krueger, David H., "A Study of the Results of Teaching Factors of Listening Comprehension to College Freshmen in the Basic Communications II Course”, Unpublished M,A. thesis, Whittier College, 1950, 176 pp. Larsen, Robert P., and Fedar, D. D., ”Common and Differential “’ Factors in Reading and Hearing Comprehension”, Journal of Educational Peypbology, XXXI (April, 1940), 22I2252: Loder, J. Edwin, "A Study of Aural Learning with and without the Speaker Present , Jaurnal of Egperimental Education, VI (September, 1937), 46:60} Lehnas, Lucile, "Listening vs Talking", English Journal, xxvx (June, 1937), 4792190. Lewdermilk, Ronald R., "Attitude Shifts from Reading and from Radio-Program Listening", Unpublished Ph.D. thesis, Ohio State University, 1939, 286 pp. Luh, C. V., ”The Conditions of Retention”, Psychological Monographs, XXXI (1922), 1-7. 7 Lyman, R. L., The Mind at'Work in Studyigg, Thinkigg5 and 3636.125, §C°EE“Foreaman aha 0e, e Ye, , ppe Natthews, Jack, "The Effect of Loaded Language on Audience Comprehension of Speeches! S eech Monogrgphs XIV (1947), 176-186. ' ‘2? ’ McGeogh, J. A., The Ps cholo of Human Learning, Longmans, Green and Co., N. 1., 1342, 419 pp. Mercer, Jessie, "Listening in the Speech Class", The Bulletin of the National Association of Seconaar - 305001 PrincipaIs,XXIII-(Jinuary, 1948), - . 184 Micheels, William J., and Barnes, M. Ray, Measuri Educa- tional Achievement, McGraw-Hill Book Co., N. 1., 1950, PP- Miller, Nathan, ”Listening to Learn", See and Hear, I (February, 1946), 71-75. Mirrielees, Lucia B., "How Do Y0ur Pupils Listen?', Montana Education, XXIII (January, 1947), 9, 30. . Morkovin, Boris V., ”Growth Through Speaking and Listening”, Elementary English, XXVI (March, 1949), 129-131, 141. Morris, D.‘¥., and Huckleberry, A. N., ”The Stud?nt Teacher's Speech uarterl Journal of Speech, XXIX. December, 1944), 49%79‘9‘J- . Munsterberg, H o, and Bigham, J., ”Memory”, Psychological Review, IufiFebruary, 1894), 34-45. Murphy, George, "Art of Listening”, Catholic W0r1d, CXLIII (April, 1936), 104-106. Murphy, George, "We Also Learn by Listening“, Elementary Egglis , XXVI (march, 1949), 127-128, 157. Nashville Public Schools, Listeni Abilities of Pu ils in the Nashville Public Schols, Study madeby Puglic SchoI System, Nashv111e, Tennessee, September, 1950, 43 pp. Nelson, Harold.E., ”The Effect of Variation of Rats on the Recall of Radio Listeners of 'Straight' Newscasts', Unpublished Ph.D. thesis, State University of Iowa, 1947, 313 pp. Nelson, Harold E., and Moll, Karl R., ”Comparison of the Audio and Video Elements of Instructional Films", Journal of Communication, I (May, 1951), 62-66. Nichols, Ralph G., ”Listening: Questions and Problems”, guarteply Journal of Speech, XXXIII (February, 1947), Nichols, Ralph G., "Factors Accounting for Differences in Comprehension of Materials Presented Orally in the Classroom", Unpublished Ph.D. thesis, State University of Iowa, 1948, 298 pp. Nichols, Ralph G., "Factors in Listeni Comprehension”, Speech Monographs, XV (No. 2, 1942?, 154-163. 185 Nichols, Ralph G., ”Teaching of Listening”, Chicago Schools Jaurnal, XXX (June, 1949), 273-78. Nichols, Ralph G., ”Needed Research in Listening Communica- tion”, Journal of Communication, I (May, 1951), 48-50. Noel, Elizabeth Goudy, "Good Listening: New Proficiency in the Language Arts", See and Hear, II (April, 1947 , 33, 46. Numbers, M. E., "Learning to Hear”, Volta Review, XVI (October, 1942), 557-558. O'Brien, F. J., ”A Qualitative Investigation of the Effect of Mode of Presentation Upon the Process of Learning”, .American Journal of Psychology, XXXII (October, 1921), O'Reilly, Francis J., "Listening", English Journal, XVI Overstreet, Bonaro W}, “After This Manner, Therefore, Listen", Wilson LibraxlBulletin, xx (April, 1946), 597-598. Packard, Frederick 0., Jr., "Learning to Listen", Harvard V Educational Review, XIV (May, 1944), 197-201. Postman, Leo J., "The Time-Error in.Auditory Perception”, Unpublished Ph.D. thesis, Harvard University, 1946, 235 pp. Potter, R., ”How to Listen to 8 Lecture", Good Housekeeping, CXX (February, 1945), 25. Prince, Bernice L., ”A Study of Classroom Listening Effec- tiveness in Basic Communication and Its Relationship to Certain Other Personal Factors“, Unpublished M.A. thesis, University of Denver, 1948, 93 pp. Rankin, Paul T., ”The Importance of Listening Ability", English JOurnal, XVII (October, 1928), 623-630. Rankin, Paul T., ”Listening Ability: Its Importance, Measure- ment, and‘Development , Chicagp Schools JOurnal, XII Rankin, Paul T., ”Listening Ability II: Its Importance, Measurement, and DeEeIOpment", Chicago Schools Journal/‘1V XII (June, 1930), 417-420. Rankin, Paul T., "Listening Ability“, 172-183, Proceedi s of the Ohio State Educational Conference, 0510 Stage Uhiveriity, 001umbus, 0510, 1939, 219 pp. fl, -‘\. 186 Rider, M. R., ”Is Listening a Lost Art?', New‘York State Education, XXXVI (June, 1949), 704-7032 Rockwell, Leo L., "The Fourth R Is An L”, College Engligp, I (October, 1939), 61-67. Rulon, Phillip J., and Others, "A Comparison of Phonographic Recordings with Printed Material in Terms of Knowledge Gained Through Their Use Alone", Harvard Educational Review, XIII (January, 1943), 63-76. Rulon, Phillip J. and Others, ”A Comparison of Phonographic Recordings with Printed Material in Terms of Knowledge Gained Through Their Use in a Teachi Unit”, Harvard Educational Review, XIII (March, 1943 , 163-17 . ~ Rulon, Phillip J., and Others, ”A Comparison of Phonographic Recordings with Printed Material in Terms of Motivation to Further Study", Harvard Educational Review, XIII (may, 1943), 246-255. Rulon, Phillip J., and Others, "The Effect of Phonographic Recordings Upon.Attitudes", Harvard Educational Review, XIV (January, 1944), 20-37. Russell, R. D., "A Comparison of Two Methods of Learning”, Journal of Educational Research, XVIII (October, 1928), 235-238 e Schneider, Wallace A., ”A Review of Some of the Literature Related to Listening”, Unpublished M.A. thesis, Uni- versity of Denver, 1950, 86 pp. Selover, Robert B., and Porter, James P., ”Prediction of 3 the Scholarship of Freshmen Men by Tests of Listening and Learning Ability", Journal of Applied Psycholggy, XXI (October, 1937), 583:588. Sims, V. M., and Knox, L. B., ”The Reliability and Validity . of Multiple-Response Tests When Presented Orally," J0urnal of Educational Psychology, XXIII (December, 9 Skaggs, E. 3., ”Further Studies in Retroactive Inhibition", Psychology Monographs, XXXIV, No. 8 (1925), 523-529. Snedecor, George‘W., Statistical Methods, Iowa State College Press, Ames, Iowa, , pp. 187 Solheim, A. K., ”School and Good Radio Listening Habits", J0urnal of the Association for Education by Radio, V TJanuary, 1947), 07-68. Sondel, Bess, 'Everybody's Listening”, National Parent- Teacher, 46 (January, 1951), 14-16. Spache, George, ”The Construction and Validation of a Werk- Type Auditory Comprehension Reading Test", Educational and Psychological Measurement, X (1950), 24 - . Stanton, Frank N., “Mempry of Advertising Copy Presented Visually vs Orally , Journal of Applied Psychology, XVIII (FeB'Fuary, 19347153, - 4. Stephens College Test of Listeplng Abilit Form A, (Mimeo- graphed), Prepared‘By the Division 0 CommufiIcation, Stephens College, Columbia, Missouri, 1951, 20 pp. Sterner, Alice; Sanders, Katharine Monagham; and Kaplan, Milton A., Skill in Listeni , NOTE Pamphlets on Communication, National ouncil of Teachers of English, Chicago, 1944, 62 pp. \ Steward, Ann, ”Creative Listening", The Writer, LVII (June, 1944), 163-167. Strang, Ruth, Stud gype Reading Exercises, Bureau of Pub- lications, see are COIlege, Cqumb E University, N. ‘Y., 1935, 111 pp. Stratton, Ollie, ”Techniques for Literate Listening", English Journal, XXXVII (December, 1948), 542-544. Stromer,‘Walter F., “Strength of Opinion, Auditory Memory _Span, and the Ability to Gather Meaning from Contextual Clues as Factors in Listening", Unpublished M.A. thesis, University of Denver, 1950, 44 pp. Stroud, James B., Ps cholo in Education, Longmans, Green and Co., Inc., 0. 1., 1946, 664 pp. Stroud, James B., and Ammons, Robert B., In rovin Readi Ability, Appleton-Century-Crofts, N. T., 1949, 200 pp. Stump, N. F., ”Oral Versus Printed Method in the Presenta- tion of True-False Examination" Journal of Educational Research, XVIII (December, 1928),-423-424. 188 Sullivan, G. W., "Listening Behavior in the Secondary Schools", American Teacher, XXXI (December, 1946), 12-13. Sullivan, Helen Blair, "Construction and Evaluation of a Measure of Auditory Comprehension”, Unpublished M.A. thesis, Boston University, 1937, 44 pp. Tallman, Marion L., "Teaching Discriminating Radio Listen- ing”, English Journal, XXXVII (October, 1948), 408-412. Thornhill, P., ”Home Listening Club for‘Youth", Adult Educa- tion, XVII (September, 1944), 17-22. Tucker, W. S., ”Science of Listening", 19th Centugy, XCVII Tyler, I. Keith, ”The Listening Habits of Oakland (Califor- nia) Pupils", "English Journal, XXV (March, 1936), 206-215. Tyler, Tracy F., "Listening--The Neglected Phase of Com- munication", Journal of the Association for Education by Radio, VI (June, 1947), 71 pp. Villarreal, Jesse J., "A Test of Aural Comprehension of English for Native Speakers of Spanish", _peech Mono- Watts, H. S., "Do'You Listen7", Senior Scholastic, XLVIII (April, 1946), 17-18. . Weaver,.Andrew T., and Gorden, Gladys L., ”Creative Listen- ing", Chapter VI, 163-175. In Speech, Harcourt, Brace and Company, N. Y., 1946, 287 pp; webber, Christian 0., Readipggand Vocabular Developpent, Prentice-Hall, Inc., N. YT, 1951, 159 pp. weissman, Sidney M., "The Effect of Quality of Speech on the Listening Comprehension of Male College Students”, Unpublished seminar report, No. 3036, School of Educa- tion, College of the City of New'York, 1947, 23 pp. When, Forest L., "Training in Listening and in Voice and Diction for the Airplane Pilot”, guarterly Journal of Speech, XXX (October, 1944), 262- . ‘Whitehead, L. G., "A Study of Visual and Aural Memory Proc- esses”, Psychological Review, III (April, 1896), 258-263. 189 Widner, Ralph W., Jr., "A Preliminary Study of the Effects of Training in Listening", Unpublished M... thesis, University of Oklahoma, 1950, 152 pp. Wiksell, W. A., ”Problem of Listening", Quarterly Journal of Speech, XXXII (December, 1946), 505:508. Wilking, V., and‘Webster, Robert G., A College Developmental Reading Manual, Houghton-Mifflin Co., Boston, 1943, 330‘pp. Willon, Albert, "Comparison of Reading and Listening Compre- hension of Retarded Pupilsfi Unpublished seminar report, No. 2536, School of Education, College of the City of New York, 1938, 21 pp. Wilt, Miriam E., "What is the Listeni Ratio in Your Class- “ room?", Elementary English, XXVI May, 1949), 259-264. Wilt, Miriam.E., "A Study of Teacher Awareness of Listening as a Factor in Elementary Education”, Unpublished Ph.D. thesis, Pennsylvania State College, 1949, 236 pp. Worcester, D. A., "Memory by Visual and by Auditory Presenta- tion", Journal of Educational Psychology, XVI (January, 1925) , jig-27 e Yates, Paul S., "The Relations Between Reading Recognition Vocabulary and Their Respective Correlations with In- telligence", Unpublished M.A. thesis, Washington Uni- versity, 1937, 181 pp. ‘Young, William.E., ”The Relation of Reading Comprehension and Retention to Hearing Comprehension and Retention” Journal of Expgrimental Education, V (September, 1936 , Zeligs, Rose, ”Learning to Listen", Chapter XI. In Glim ses Into Child Life, William Morrow and Co., N. Y., 1942, 303rpp. APPENDICES APPENDIX A A DESCRIPTION OF TESTS USED TO MEASURE LISTENING COMPREHENSION 192 Because no previous study has brought together accurate detailed descriptions of these many differing instruments used to measure listening comprehension, the writer has at- tempted as complete a record as he was able, both for his own knowledge and for students doing research in the future. The tests described here have been classified accord- ing to two major divisions (A) Studies in which the develop- ment of a test of listening comprehension has served as a relatively minor partof the total research; and (B) Studies in which the development of a test of listening comprehen- sion has been of relatively important or major concern. In both sections of the literature surveyed, a des- scription of the test has been given, together with such information as was available concerning the method of ad- ministration, and analysis of the test. Whenever the literature reviewed reported analyses to determine relia- bility or to establish validity, the results have been included in this summary. This summary omits studies concerned primarily with persuasive material, shift-of-opinion, note-taking yp. no note-taking, and those whose objective was aesthetic lis- ing as being largely outside the limits of the present study of measuring the listening comprehension of expository lec- ture material. 193 A. Studies in Which the Deve10pment of A Listening Comprehension Test has been a Relatively Minor Part 1. Tests Based on Adaptations of Silent Reading and Vocabulary‘Tests: Anderson and Fairbanks1 give a report of their listen- ing test in the following manner:2 An unselected sample of 50 items from Form B of the Inglis Tests of English Vocabulary was recorded phonographically and used to test hearing vocabulary. The test was recorded as follows: A trained speaker read the standard sentences of expression and after an interval of three seconds repeated this reading, substituting the first comparison word for the itali- cized word; then, after an interval of five seconds, the original expression was read again, followed in three seconds by the same expression with the second comparison word substituted; after another interval of five seconds this procedure was repeated for a third comparison word and so on for all five words. Fifty items, separated by five second intervals, were re- corded in this way. High grade phonograph records were secured. After hearing each pair of sentences or expres- sions in each item, the subjects were instructed to record S if they believed the statements to be the same in meaning, or D if they believed them to differ in meaning. Special recording blanks were used. The subjects were further told that only one pair of the five sentences of expressions in each item was correct. 2 Irving H. Anderson and Grant Fairbanks, "Common Differ- ential Factors in Reading Vocabulary and Hearing Vocabu- lary,” Journal of Educational Research, 30 (January, 1937), 317-324. Ibid., 318. 194 The reported odd-even reliability of the test was .64; this coefficient was raised to .78 by the Spearman-Brown "prophecy" formula. Burton3 also used a standardized test: Forms X and Z of: . . . the Survey Test of Vocabulary. . . by L. J. O'Rourke. Each of the hundred items in this multiple choice test is presented in the form of a sentence or expression, after which there are five words. . . The subject marks,the number of the word similar to the underlined or emphasized word. . . . The test of vocabulary was presented orally. A sentence or expression was read, the word emphasized, then the word repeated. Five choices were giveg. Students marked the number of the correct word. Goldstein,6 although he did some very carefully con- trolled work on rates, nevertheless used an adaptation of The McCall-Crabbs Standard Test Lessons in Reading. Gold- stein had general narrative and descriptive passages from these tests recorded on discs by a trained speaker. Printed questions on the passages were used. 3 Mary Burton, "The Hearing and Reading Comprehension of Vocabulary Among High School Seniors", School Review, 52 (January, 1944), 47-50. 4 Ibid., 48. 5 Ibid., 48. 6 Harry Goldstein, Reading agd Listening Comprehension at Various Controlled Rates, Contributions to Education, No.—82l, Bureau of Publications, Teachers College, Columbia University, N. Y., 1950, 69 pp. 195 . . . Each passage is followed by approximately ten multiple-choice questions. The questions are so worded that careful reading of the passage is necessary. How- ever, some of the questions cannot be answered correct- ly by detailed reading only. It is necessary to under- stand the selection as a whole, and be able to recog- nize its parts, and eve to draw inferences from facts not overtly stated. . . Concerning the validity of the test, Goldstein says: If standardized measures are valid criteria, the test scores for films and records are satisfactgry indices of reading and listening comprehension. The.reliability of the tests to measure listening ability was .80 (corrected to .89). 9 Johnson used a previously developed reading test. He says: The listening tests were recorded abridged forms of the American Council on Education Cooperative English Test on Reading Comprehension (C2). Only a part of the comprehensiBn sections of each of the forms T, , and Q were used. Larsen and Feder11 also used a standardized reading test. Passages from the Nelson—Denney Silent Reading Test were presented orally to students. 7 Ibid., 11. 8 Ibid., 55. 9 Kenneth 0. Johnson, "The Effect of Classroom Training Upon Listening Comprehension," Journal of Communication, I (May, 1951), 57-62. 10 Ibid., 59. 11 Robert P. Larsen and D. D. Feder, "Common and Differen- tial Factors in Reading and Hearing Comprehension", Journal of Educational Psychology, XXXI (April, 1940), 2:47. '2—52 e 196 The odd-even reliability of the test was .79. 12 also used a standardized test: Four Sims and Knox forms of the Thorndike Test of Word Knowledge. This 100 item five-response test was administered to grades nine through twelve. Students were given a sheet of paper with numbers. Students were also given oral instructions as follows: Listen carefully to the first word I read, then listen to the other three (four, five) words used. One of the last three (four, five) means the same or about the same as the first word. Each list of words was read twice; then students wrote the word on the numbered paper they had received. Thevvriters indicate that the reliability was tested by the split-half method and corrected by the Spearman- Brown formula. Validity was determined by correlating with the visual test.14 2. Test Based on Classroom Lectures: Corey15 gave a carefully delivered lecture on the sub- 12 v. M. Sims and L. B. Knox, "The Reliability and Validity of Multiple-Response Tests When Presented Orally", Journal of Educational Psychology, XXIII (December, 1932), 650-662. 13 Ibid., 657. 14 Ibid., 659, 660. 15 Stephen M. Corey, "Learning from Lectures vs. Learning from Readings", Journal of Educational PsyEHology, 25 (September, 193477'459-470. 197 ject of outlining. The college students were told that they were not to take notes and that they would have an immediate recall test following the lecture. "The test used was a semi-objective, true-false, com- pletion and short answer type. . ."16 Reliability of the test according to the Spearman—Brown prophecy formula was reported as .779.17 Greene18 used a typical college lecture on the subject matter common to elementary psychology classes. Following the lecture students answered printed questions of a com- pletion or true-false type. Although tests were reported as "speeded", Greene indicates that they were so arranged as to let practically everyone finish.19 Reliability coefficients, using the Brown-Spearman formula averaged about .50 for the completion tests, and .41 for the true-false tests.20 To determine the validity of the tests, Greene states that he submitted the tests to two or three instructors who 16 Ibid., 459. 17 Ibid., 465. 18 Edward B. Greene, "The Relative Effectiveness of Lecture and Individual Reading as Methods of College Training,” Genetic Psychology Monographs, IV (December, 1928), 457- 503. 19 Ibid., 474. 20 Ibid., 475. 198 were familiar with short answer test methods.21 Later he asked students to "eXpress confidence ratings" of their answers.22 Loder25 with his eighth grade pupils used a series of talks on the subject of narcotics, and built 100 question true-false printed tests. The test analysis was carried out in the following man- ner: In order to eliminate errors all tests and their directions were carefully checked by several persons in the field of tests and measurements. Russell25 prepared about 1000 words on the subject of the mongoose adapted to the level of children in the fifth, seventh, and ninth grades. In the listening phase of the study, the teacher read the material to the pupils. The knowledge gained was measured on two tests: one of an essay type, and the other true-false. 21 Ibid., 475. 22 Ibid., 505. 25 Edwin J. Loder, "A Study of Aural Learning With and With- out the Speaker Present', Journal of Experimental Educa- tion, VI (September, 1937), 46-60. 24 Ibid., 47. 25 R. G. Russell, "A Comparison of Two Methods of Learning": Journal of Educational Research, XVIII (October, 1928), 55-258 0 199 Stump26 was interested in checking whether oral true- false tests were as satisfactory as printed true-false tests. The tests covered subject matter included in several regular college classes in Elementary Educational Psychology and High School Tests and Measurement. In presenting the oral test the questions were read slowly and distinctly. Each statement was read twice. Stump says that "No attempt is made in this study to prove the validity of the true-false examination as a test of school achievement."27 Young28 developed a series of 350-800 word passages suitable for grades four, five and six. The materials con- sisted of descriptions of industrial processes, nature science, hero stories, and narrative poems. In the listen- ing part of the study, the teacher read the selection to the students. Immediately after presentation of each se- lection, the teacher tested the pupils. One test of thirty items was used. The firSt twenty or so items were questions calling for short definite answers. These items formed a recall test. 26 N. F. Stump, "Oral Versus Printed Method in the Presen- tation of True-False Examination", Journal of Educa- tional Research, XVIII (December, 1928), 423-424. 27 Ibid., 428. 28 William E. Young, "The Relation of Reading Comprehension and Retention to Hearing Comprehension and Retention", Journal of Expprimental Education, V (September, 1936), 36 '59 e 200 The last ten or so items were multiple-choice units of three responses each. Whereas the recall items usually dealt with specific bits of knowledge, the multiple- choice units were concagned with general impressions, deductions, etc. . . . The reliability of the tests is reported as ranging from .87 t .018 to .92 1 .012. Validity, as determined by correlation with reading tests is reported to range from .54 to .67.30 3. Tests Based on Special Types of Material or Utilizipg Special Situations: (a) Phonograph records and films: Nelson and Moll31 were primarily interested in deter- mining the "relative contributions to learning of the audio and video elements in.some typical instructional films." Two sound-films were used on the subject of aerodynamics, and one on survival under desert conditions. In both studies, tests containing multiple-choice items with four alternatives were used. . . . Some of the items in each test were based on mater- ial in the visuals, others were based on the commentary, and items in a third group were based on information to 29 Ibid., 31. 50 Ibid., 32. 51 Harold E. Nelson and Karl R. Moll, "Comparison of the Audio and Video Elements of Instructional Films", Journal of Communication, I (Mary, 1951), 62-66. 201 be fougg in both the video and audio elements of the films. The authors report that the items were pro-tested, analyzed, andeaxamined by subject-matter experts. Roulonss and his associates designed tests to measure "factual and relational knowledge" of the content material covered in the recordings and similar reading material pub- lished for high school students.34 The tests consisted of twenty-five true-false items, and two 5-item sequence exer- cises. (b) Informative speeches: Knower, Phillips, and Koeppel:55 were concerned primarily with improving speaking ability, but their evaluation lead them into the development of a listening test of specific subject matter. 52 Ibid., 63. 53 Phillip J. Roulon and others, "A Comparison of Phono- graphic Recordings with Printed Material in Terms of Knowledge Gained Through Their Use Alone”, Harvard Edu- cational Review, XIII (January, 1943), 63-76. 34 Ibid., 68. 55 Franklin H. Knower, David Phillips, and Fern Koeppel, "Studies in Listening to Informative Speaking", Journal of Abnormal and Social Psychology, 40 (February, 1945), 82-88 e 202 The informative Speeches were based on three aspects of a beginning college physics course. "A 40-item multiple- choice test was formulated for each speech.”36 "The three tests had odd-even reliabilities of .92, .88, .91 respectively.”57 The validity of the tests was determined by giving them to students who had not taken the physics course, and those who had just completed the course. A significant difference was found between those groups. (c) Trade names: 38 was interested only in checking whether his Stanton subjects could remember the fictitious trade names they had heard and.read. He used printed tests composed of lined sheets of paper. The three types of tests were pure recall, which asked students to list the trade names; aided recall, whichnwas a type of completion test; and recognition, which was a sort of check list of trade names presented for identi— rication.‘59 56 Ibid., 82. 57 Ibid., 82. 58 Frank N. Stanton, "Memory of Advertising Copy Presented Visually vs. Orally”, Journal of Applied Psychology, XVIII (FeBFuary, 1934), 45-64. 39 Ibid., 51-52. 203 (d) Material from recent books: Selover and Porter40 developed a very simple type of listening test which had as its purpose the measurement of how well a subject could recall material that had been read to him. . . . The test is composed of 100 multiple choice items which are answered by indicating es, no, or did not sa , after the firsylsix pages of a reEEht boEE—HE?3 een read [to him]. (8) Lists and columns of words: Kreuger42 used two types of tests. The first which he refers to as the "Minnesota Test” is an unpublished test developed by Ralph Nichols of the University of Minnesota. The test consists primarily in directions: The subject has lists of words, etc., before him. He then receives oral directions to locate a specific word in much the same way that Heilman's subjects located geometric figures. 40 Robert B. Selover and James P. Porter, "Prediction of the Scholarship of Freshman Men by Tests of Listening and Learning Ability", Journal of Applied Psychology, XXI (October, 1937), 583-588. 41 Ibid., 583. 42 David H. Kreuger, "A Study of the Results of Teaching Factors of Listening Comprehension to College Freshmen in the Basic Communication II Course”, Unpublished M.A. Thesis, Whittier College, 1950. 176 pp. 204 Kreuger also developed a seven minute lecture on the subject of pure-tone audiometers; and followed this with ten true-false mimeographed questions.43 (f) Specially selected words, phrases, sentences: 44 Carver carried out a series of experiments, four of which involved the measurement of listening. Experiment I: Recall of Disconnected Materials Pre- sented Visually and Aurally. Six series of nonsense syllables, numbers, and words, containing ten numbers each were used in this experiment. . . . Additional memory material was em- ployed in the form of sentences, including five short and five long _pecific statements, five short and five long general statements. All of the sentences used were easy and typical of everyday conversation. All instructions were given orally by the writer. The subjects were always told in advance what the mode of presentation would be. When the auditory presenta- tion was concluded the announcer always said, "The end. The subjects were instructed in advance to write down immediately the items they were able to recall or otherwise to igact to the material in some specified manner. . . Experiment II: Recognition of Material Presented to the Eye and to the Ear. The material for this experiment consisted of two series of sentences. The first was designated as the original series, the second as the test series. The original seFies contained a list of 1700mm0nplace 43 Ibid., 33-36. 44 Merton E. Carver, "Listening Versus Reading, Chapter IX, 159- 180, in Hadley Cantril and Gordon‘W. Allport, The Psychology of Radio, Harper and Brothers, N. Y., 1935. 45 Ibid., 162. 205 sentences of varying lengths. The test series contained 15 sentences, always presented to the subjects in printed form. "In the test series certain sentences were repeated exactly as they had been given in the original series, certain sentences were omitted altogether, some new sentences were added, and in some sentences one single important word was altered. . . . Illustration (Original Series) My friend was waiting on the corner. A growing child is constantly forming more bone, more muscle, and more blood. (Test Series) Do you think each of the sentences below is similar to any sentence you have just read (or heard)? Check your Opinion. If only one word in the sentence has been changed UNDERLINE that WORD. A growing child is constantly in need of more food, more exercise, and more fresh air. exactly the same same idea worded differently not read before My friend was waiting on the corner. exactly the same same idea worded differently not read before . . . The auditory presentation was given over the radio. The sentences were spoken at a uniform rate with an even emphasis. A short pause occurred between sen- tences. When the series was concluded, the announcer said, "the end", and the subjects turned immediately to the test series.4 Experiment III: Comprehension of Connected Meaningful Material. Paired samples of connected meaningful passages together with sets of directions were selected with the help of judges. This collection included passages of a narrative, descriptive, abstract, andeaxplanatory type 0 O O 0 To test the subjects' com rehension, an objective test (true-false or completion was employed in many 45 Ibid., 169. 206 instances. 9t other times short essay tests were used. . . .4 Experiment IV: (Carver's No. VII) 'Comprehension' of Meaningful and Fictitious Words. Two lists of words were prepared. Each list was composed of twenty meaningful words and five non- meaningful or fictitious words (jokers) which Hare made to appear as plausible as possible. . . . Subjects were asked to make one check mark if they had seen or heard the word before; two check marks if they thought they could define the word; and three if they were sure they could define the word. (g) Material not given: . Erickson and King49 merely indicate that "Questions were immediately furnished each pupil on mimeographed sheets."50 51 Spache gives no report on the nature of his test. He does mention that it tests for main ideas and details, 47 Ibid., 171. 48 Ibid., 176. 49 c. I. Erickson and Irving King, "A Comparison of Visual and Oral Presentation of Lessons in the Case of Pupils from the Third to the Ninth Grades", School and Society, VI (August, 1917), 146-148. 50 Ibid., 146. 51 George Spache, "The Construction and Validation of a Work-Type Auditory Comprehension Reading Test", Educa- tional and Psychological Measurement, X.(1950), 2493253. 207 and that the reliability of the test as determined by the Spearman-Brown formula is .788.52 B. Studies in Which the DevelOpment of A Listening Test was A Significant or Major Part Rankin53 (1930) appears to be the first person who was interested in measuring listening itself rather than for purposes of some other related problem. He first used the Detroit Reading Test, but discarded it because he felt that it was neither designed primarily to measure listening nor was it a real life listening situation.54 He later devel- oped what he considered a more satisfactory test to measure listening ability. After some preliminary statements, designed to make the pupil conscious of the importance of ability to understand what he hears, the examiner says: "A number of you have taken part in meetings, like class meetings, where a decision had to be made. Of course, the reason for meeting together is to listen to the idea of everyone in the group for the purpose of picking out the best idea. Each person needs to under- stand just what everyone else contributes to the dis- cussion. Otherwise one can not vote intelligently. "Today the examiner is going to tell you what each boy and girl in a certain sixth grade said at his class meeting. Pretend you are a member of that class. Lis- 52 Ibid., 251. 53 Paul T. Rankin, "Listening Ability, II, Its Importance, Measurement, and Development", Chicago Schools Journal, XII (June, 1930), 417-420. 54 Ibid., 418. 208 ten to what is said, so that you can vote wisely on the best scheme. Listen to every speech, trying to hear just what idea is being proposed by the speaker. In order to check on how well you understand each Speaker, the examiner will ask a question after each speech, and will suggest four possible answers, one of which is right. You are to choose the right answer from these four, and write it down. Finally you are to vote for what you think is the very best plan suggested." The introductory remarks of the chairman are used as the practice test. ”The other day Miss Jones told us that this was to be our room and that we could decorate it any way we pleased. This is a new building and so the room has wonderful possibilities. We are having this meeting today to decide what we shall do first. Each one of you may tell what he would prefer to have for decora- tion. Emily will write them on the board so that we can easily remember all of the suggestions. When you give your suggestion, please give your reason for it. "The question; 'What was the reason for the meeting?‘ 1. To complain about the bare room 2. To ask for a more cheerful room 3. To decide how to decorate the room55 4. To have Emily write on the board." Rankin calls particular attention to three points about the test: (1) The presence of a fairly real purpose on the part of pupils for the entire listening period; (2) the fact that the material was designed to be spoken, not written; . . . (3) the fact that the question on each speech relates directly to the contribution of that speech to the general argument, and thus tests whag a good listener should understand in that situation. 6 55 Ibid., 419. 55 Ibid., 419. 209 Although Rankin observed the relationship of some thirty-two items of the Stanford Revision of the Benet Ex- amination to listening, and made some correlations of lis- tening with the Detroit Alpha Intelligence Test, and some correlations with reading and vocabulary tests, he makes no mention of an item analysis of this first listening test. Sullivan57 (1937) was primarily interested in reading, but this interest lead her to develop a test of auditory comprehension for children. She expressed the opinion that auditory comprehension would serve as a better criterion of reading ability than group tests of intelligence. . . . It is the purpose of this thesis to construct a reliable measure of auditory comprehension adapted to the group testing of cgéldren in the age ranges of seven to twelve. . . . In the present test it was decided to measure the child's hearing comprehension by allowing him to iden- tify in pictures the words spoken by the examiner. . .59 There are several parts to Sullivan's test. Those pertinent to the present study are: l. The Auditory Vocabulary Test Seventy-five items. . . were assembled. . . . The groups of eight pictures with the alternate choices and 57 Helen Blair Sullivan, "The Construction and Evaluation of a Measure of Auditory Comprehension", Unpublished M.A. Thesis, Boston University, 1937. 44 pp. 58 Ibid., 5. 59 Ibid., 7. 210 five stimulus pictures were organized on two successive pages of tests. The child's record of answers was made by placing the number of the picture beside an appro- priate letter in the margin of the test. . . . 2. The Auditory Comprehension of Paragraphs The groups of fifteen pictures each, serving as the multiple choice answers to the questions on the para- graph test were arranged four to a page, and covered three pages. . . . The child's response was again in- dicated by putting the appropgaate number in brackets designated for each question. The reliability of the first test (corrected by the Spearman-Brown formula) was forecast as .966; and the cor- rected reliability of the second test was reported as .938.61 Nichols62 (1948) appears to be the first person to make a careful and controlled attempt to build a test of listening ability. Under the heading "Test Construction and Refinement"63 he says: Two hundred college freshmen, drawn from the fresh- man communications courses on the St. Paul Campus of the University of Minnesota, were chosen as the population to be tested. . . . EThey] were chosen in a fashion cal- culated to insure the use of a typical cross-section of the enrollments in those courses. Ten-minute excerpts were taken from full-period lectures normally given to various freshmen classes at the University of Minnesota. Six different subject- matter areas were selected in order to neutralize the 50 Ibid., 41. 51 Ibid., 43-44. 62 Ralph G. Nichols, "Factors in Listening Comprehension", Speech Monographs, XV (No. 2, 1948), 154-163. 55 Ibid., 155-156. 211 advantage which would accrue to listeners who had al- ready had college training in one or two of them. The chosen areas were literature, economics, biology, sociology, psychology, and chemistry. The faculty members contributing the lectures from which the ex- cerpts were drawn are regular instructions at Minnesota. To test the listener's comprehension of the material presented, twenty multiple-choice questions were con- structed to cover the content of each of the lecture excerpts. After the test battery had been administered to the selected test p0pulation of 200, a complete item analysis was made. To avoid weighting any of the six parts of the battery, seventy-two of the original 120 items were retained, or twelve of the twenty items originally composing each of the sub-tests. The dis- criminating power of the seventy-two retained items was determined through the use of the Flanagan table of the values of the Pearson product-moment coefficient of cor- relation in a normal bivariate population corresponding to given pr0portions of successes. The median coeffi- cient of correlation of the seventy-two retained items, indicative of discriminating power, was found to be .36. Only one item of the seventy-two retained had a coeffi- cient of correlation of less than .20. Ten of the seventy-two retained items had coefficients of correla- tion of .50 or above. The test was slightly easier than would be the ideal examination, the items tending to cluster about the 60 per cent difficulty level more than about the 50 per cent difficulty level. The listening test for all further analysis and consideration of comparative performances was composed of the seventy-two retained items. Its reliability was computed through the use of the Kuder-Richardson formula and was found to be .802. The original examination papers were re-scored upon the basis of the seventy-two retained items. The scores thus computed were there- after used as indicative of the listening comprehension of the 200 subjects in the test population. 212 Prince64 (1948) conducted what she called an "eXplora- tory study". The first purpose of the study was to devise means of measuring classroom listening effectiveness. The measurements of listening were achieved in the fol- lowing manner: 1. Test on Ability to Understand Assignments The instructor gave a series of four assignments at one time. The assignments covered periods of time from one day to three weeks. Students were told that perhaps they should take notes on the assignments. No other instructions about understanding the assignment process itself was given. The next day theassistant announced that in order to make sure that everyone in the class had understood the assignments that had been given the day before, each person was requested to write briefly what he understood the assignments to be. Students were free to consult their notes. No indication was given that the papers were to be used as a listening‘ check. From the shorthand transcription of the in- structor's talk giving the assignments, the essential points gor each assignment were listed by the investi- gator.6 The student papers were scored according to the number of essential points agreeing with the shorthand transcrip- tion. 2. Tests on Student Speeches Four times during the quarter students were tested on three aspects of a talk given by one of the student speakers. 64 Bernice L. Prince, "A Study of Classroom Listening Effec- tiveness in Basic Communication and its Relationship to Certain Other Personal Factors", Unpublished M.A. Thesis, University of Denver, 1948. 93 pp. 65 Ibid., 31. 213 These aspects included (a) content retention, described as similar to "comprehension" in reading tests; (b) inferences, the ability to draw conclusions; and (c) main or whole idea. Prince states that regular use of recording of speeches, and making out of tests on various speakers did not permit the students in the class to know that the material was be- ing gathered as a measure of listening effectiveness. Scores on these listening tests were determined in various ways. 3. Goldstein Test The recorded Goldstein test at the rate of 100 words per minute was administered and scored. 4. Subjective Judgment of Overt Listening Behavior as Indicated by Use of a Listening Profile Sheet A listening profile sheet consisting of eight general descriptive types of listeners was developed. Labeled "participator, subdued, bored, distracted, intense, irre- pressible, polite, and bless."66 Subdued is described as: Gives impression of a very quiet personality, facial expression does not change very much. Seems to be some tension, almost as though ready to defend him- self against attack at any time. In humorous situa- tions smiles slightly but not wholeheartedly -- maybe a little cynically. Does not ordinarily7look at other members of the class while listening. 66 Ibid., 38, 39. 67.22323: unnumbered pages of Appendix. 214 Prince carried on a great many other studies, but they are not related to the present study. Brown68 (1949) says of his study: . . . Based on the hypothesis that listening, as reading, is a composite of relatively discrete skills, ten specific skills or factors were chosen for measurement. Both receptive and reflective listening skills were in- cluded18nd’both natural and artificial listening test situations explored. Listening accuracy was directly explored in terms of (IT getting lecture details, (2) following a sequence of details in the form of oral di- rections, and (3) keeping a series of details in mind until questioned. Thoughtful or critical listenigg was directly explored in terms of (1) getting the central ideas, (2) drawing inferences, (3) distinguishing relevant from irrelevant material, (4) using contextual clues to determine word meanings, (5) using transitional elements as aids to understanding the organization and relationship between main and subordinate points, and in terms of (6) an impressionability and (7) a ration- alizing index. These particular factors were selected on the basis of (l) the most recent pronouncements of the Commission on the English Curriculum of the National Council of Teachers of English, (2) the conclusions reaches by Nichole in his study of factors in listening comprehen- sion, (3) diagnostic tests in the field of silent read- ing and reading comprehension, and (4) the Opinion of eleven experts, chosen largely from among the members of the Vertical Committee on Liggening of the National Council of Teachers of English. The following pages give both a quoted description of the several aspects of the test, and one selected question as an example of that aspect of the test. 68 James I. Brown, "The Construction of A Diagnostic Test of Listening Comprehension", Unpublished Ph.D. Thesis, Uni- versity of Colorado, 1949. 184 pp. 69 From an abstract of his study sent to the present writer. 215 Section I of the test has six parts. Part 1 serves both as orientation and as an attempt to get a "Rationalization Index". A sample advises students as follows: . . . Sometimes you will be asked to rate your agreement or disagreement to something as in the next items. Enter the number that indicates your opinion, a 1 if you agree strongly, 2 if you agree somewhat, 3 if you are unde- cided, 4 if you disagree, and 5 if you disagree strongly. Rely on your first impressions and work as rapidly as possible. Here's the first statement to rate: 6. All men are created equal. Enter the number indi- cating your degree of agreement or disagreement after number 6 on your answer sheet. . . . Part 2. Ability to follow details in the commonly met form of oral directions: Listen carefully to the following directions which concern the row of eight numbers at the left of your answer sheets under Part 2. Be sure to answer every item. work rapidly. 22. Subtract the smallest even number from the largest number in the row of numbers mentioned, and enter'hal; that result after number 22 in the answer column. . . 1 Part 3. Ability to keep a series of details in mind until questioned: Now let's try Part 3, Thinking Back. Often in the process of listening it is necessary to think back to something just said or to keep certain things in mind as a speaker goes on to make comparisons. This part will measure how well you do that. 70 James I. Brown, "The Construction of A Diagnostic Test of Listening Comprehension", 0p. cit., 144. 71 Ibid., 145. 216 43. In the series of numbers, 5 8 4 l 6, what is the second number? Enter past number in the answer column after number 43. . . . Part 4. Ability to make intelligent use of contextual cues in getting word meanings: Did you ever stop to think how many different meanings the same word may have? For example. . . . A good listener is one who makes full use of contextual clues to discover the meaning of strange words or familiar words used in a strange sense. . . . The first five words along the left of your answer sheet under Part 4 are followed by five correct dic- tionary definitions of that word. The context governs which of the meanings is intended. I shall read1you a sentence and you are to select the meaning which best defines the word in that sentence context. Enter the number of the word chosen, in the answer column. 65. In the sentence: Because Tom Sawyer didn't get along very well in school, people thought he was dull, which choice best defines the word dull -- stupid, un- feeling, listless, blunt, or tedious; Put the number of that choice in the answer column. 5 Part 5. Ability to use transitional elements as aids 0 to a speaker's organization: For the next items you are to listen to each sen- tence for a word that marks the organizational pattern, a listening road sign, so to speak. There are certain words that can either point ahead to what is to be said or back to what has already been said. Often the same word points in both directions. For example, when you turn on the radio and hear someone say, "The final reason. . . .” the word final lets you know that other reasons have already been given and that one last reason is about to be discussed. . . . Enter the one word in each of the following sentences which appears to have 72 73 Ibid., 147. Ibid., 149. 217 that for its chief function. If there is no such word, enter the word none in the answer space. . . . 98. To summarize what has beenvaaid: procrastination is at the root of the difficulty. Part 6. An attempt to get an impressionability index. Emotionally loaded words were added to the statements used in Part 1. Strength of opinion. It is thought by some that this may relate to general listening efficiency. Check your degree of agreement or disagreement with the fol- lowing statements, using the same rating scale you used for Part 1. . . . 114. What do you think of this silly idea: 7ghess play- ing develops one's powers of concentration? Section II of the test was based on a fifteen minute lecture on the subject of "How to Increase Your Vocabulary", adapted from the Reader's Digest. Part 1. Getting the Details. First, we'll see how well you got the details. The choices on your answer sheet, often in shortened form, are to serve as memory aids as you listen carefully to the reading of questions and are not intended to be read apart from the listening situation. Try to answer every item. 1. The Lincoln story mentioned a calf, dog, steer, cow, or horse. Indicate the correct answer by entering the number of the correct response. . . . Do nag write the word, only the number identifying the word. Part 2. Getting the Central Idea. 74 75 76 Ibid., 153. Ibid., 154. Ibid., 164. 218 Sometimes we can't see the forest for the trees. And sometimes we miss the main ideas because of the de- tails. The next part, Part 2, will measure how effec- tively you get the central ideas. 27. Which main division was most7important - the first, second, third, fourth, or fifth? Part 3. Depth of Understanding. Depth of understanding enables us to transcend the exact limits of what was actually said and see meaning- ful new relationships and the implications 53d infer- ences that grow out of that material. . . . The next items are to mark True or False by circling the T or F in the answer column. 59. Paderewski would probably consider desirable vocab- ulary growth as: being natural and effortless - true or false 60. requiring special effort 61. being hastened by special effort 62. requiring above average intelligence79 Part 4. Relevancy. In most lectures and articles there are things both directly and indirectly related to the central idea. In the following groupings you are to decide which item in the group is most directly and closely related to the central idea of the lecture, How to Increase Your Vo- cabulary. Circle the one letter in each group which identifies the most relevant item. 102. A. Snake story B. Uncle G§8rge story - which is more relevant, A or B? 77 Ibid., 166. 78 Ibid., 167. 79 Ibid., 168. 80 Ibid., 170. 219 Brown gave careful consideration to the construction of his test and constructed a preliminary form, tried it out with numerous groups and made item analyses and revisions in the trial form before running his final test. Attempts were made to insure validity through corre- lations with various other factors commonly used. Reliability, determined by the use of split-half and the Pearson-Brown formula was .810 for Section I, and .647 for Section II for the college population. Harwood81 (1950) investigating the "relationships be- tween written and spoken language of various levels of pre- dicted silent reading difficulty", developed a test of lis- tening comprehensibility.82 Seven sample "stories" adapted from radio, press and other sources were recorded by a trained speaker. Prior to the final recording, nineteen judges selected one speaker from a trial group of four. This one speaker seemed ”to read most like the reading of an average trained speaker."83 Each story was exactly three hundred words in length; and was timed to be presented in exactly two minutes. One hun- 81'Kenneth Albert Harwood, "An Experimental Comparison of Listening Comprehensibility with Reading Comprehensibil- ity", Unpublished Ph.D. Thesis, University of Southern California, 1950, 157 pp. 82 Ibid., 2. 83 Ibid., 44. 220 dred and five printed questions (fifteen for each story) with five-choice answers were distributed to students in answer booklets. After hearing, the recorded "story", the subjects saw and answered the questions. Subjects were given two minutes to answer the questions on each of the seven sections of the test. Total test time was planned at thirty minutes so that it might easily fit a fifty minute class period. Although questions are stated to have a narrow range of difficulty, no mention appears to be made of an item analy- sis of the questions nor of the reliability of the tests.84 The Nashville Public Schools85 (1950) in building a twelve year program in the language arts, wished to devote to listening the emphasis they felt its importance in life today required. They conducted a survey of listening abil- ities of their pupils as a preliminary part of the curricu- lum study. In the absence of any standardized tests appro- priate for the purposes, the materials were developed by committees of teachers who worked at the levels at which they teach. There were committees for the primary, the elementary, the junior high, and senior high levels. 84 Ibid., 91. 85 ”Listening Abilities of Pupils in the Nashville Public Schools, A Survey: Grades 1-12", Nashville Public Schools, Nashville, Tennessee, September, 1950. The material re- ported herein was received from Miss Lalla A. Walker in either mimeographed form or personal letter form. 221 The committees attempted to see that each test met the standards of interest and proper grade- -level diffi- culty for which it was produced. After the tests were develOped, they were tried out with a representative cross section of pupils. In the light of the results, they were revised and tried out again before being processed for general use. The listening tests and companion reading tests were developed on three types of material: narration, exposition, and directions. Each test was composed of a short selection86 to be read or heard, followed by ten questions to be an- swered True, False, or No Information. These questions were designed to test the following abilities: I. To get the main idea (question 1) 2. To understand a context clue (question 2) 3. To catch general significance (question 3) 4. To understand time sequence (question 4) 5. To remember details (questions 5-8) 6. To dgaw inferences or conclusions (questions 9-10 The listening materials were recorded and played for pupils. When the recording was completed the teacher said, "Listen carefully while I read some sentences about the article to you. Some of the sentences are true. Some are false. The article gives no information about some of them. 86 The sample of Exposition for Senior High School Pupils was on "Aluminum," and was about 300 words in length. 222 "Write your answer to each sentence on the small piece of paper that your teacher has given you. "After you hear a sentence, put a check mark after its number on the line which tells whether this sentence is true or false or has no information about it in the story." Stromer87 (1950), using a rating sheet as an "outside criterion" of listening, developed three tests, which might be aspects of listening.88 The rating sheet has four main areas: 1. Ability to follow oral directions. 2. Relevance of student's comments on class discussion. 3. Relevance of student's questions to the subject under discussion. 4. Relevance of student's answers to questions asked of him. Under each of the areas are statements ranging from the extremes of "very best" to "worst". Test 1. Ability to Gather Meaning from Contextual Clues. The test used in this study consists of twenty-five sentences, each built around a particular vocabulary word which the sentence helps to define. The vocabulary words are those appearing in the Cooperative English Test. Some of the sentences are designed to give a 87 Walter F. Stromer, "Strength of Opinion, Auditory Memory Span, and the Ability to Gather Meaning from Contextual Clues as Factors in Listening", Unpublished M.A. thesis, University of Denver, 1950, 44 pp. 88 Ibid., 9, 10. 223 fairly exact meaning to the vocabulary word, while others give no more than a general idea of what the word might mean. In administering the test, the actual vocabulary word is replaced by an artificial or nonsense word. These nonsense words are designed to sound like real words. . . . « Each sentence in this test was read to the students just once, after which the students were asked to write down all words or phrases they could which might be sub- stituted for the nonsense word. . . . In scoring, two points were allowed for each word or phrase that was considered exactly right as a substi- tute for the nonsense word; and one point was allowed for each word or phrase which might be accepted as a substitute for the nonsense word, but which seemed to ignore some 9f the finer shades of meaning in the con- text. . . .8 Test 2. Strength of Opinion. This test is based on the assumption that people hear what they want to hear, and that if they have strong opinions on particular subjects that they will listen "for that which will strengthen those beliefs, and partly or completely ig- nore that which disagrees with our beliefs."90 Test consisted of thirty statements to be rated from plus to minus ten. Since the test was BEE administered orally, this writer does not consider it a test of listening ability. 89 Ibid., 13-15. 90 Ibid., 15. 224 Test 3. Test of Memory Span. Several methods were tried, but it was discovered that subjects were writing down the numbers or sounds, so that the part of the test actually used consisted of: . . . seven items in which the series were meaningful words, such as names of cities, states, colors, foods, etc. In this group the students were asked to write down as many of the words in each series as they could remember, and to put the words in proper order if pos- sible. . . . In scoring one point was allowed for each word that was recorded correctéy whether or not it was in proper order in the series. The reliability of the rating scale was determined by comparing instructor and assistant instructor ratings on the same students. It was .58. The Contextual Clues test was analyzed, and defective items deleted. The corrected test had a Spearman-Brown co— efficient of .94. Validity of the test as determined with the rating scale was -.27 Reliability of the Strength of Opinion test was .75. The validity coefficient was .50. Validity coefficient of the Test of Memory Span is re- ported as -.10. Reliability is not reported. Memory Span is apparently not related to listening as measured in this study. 91 Ibid., 17. 225 Widnerg2 (1950) developed two comparable forms of a listening test. Both were based on material found in two debates between Lincoln and Douglas. Each test passage con- tains about five typewritten pages of material consisting of lecture material using many quotations from the two debates. Each "lecture" was followed by the administration of fifty printed questions with four foils for each question.95 The tests were constructed by actual trial, item anal- ysis and revisions. The reliability is reported as .836 and .839 for each test respectively.94 The validity of the items was checked by submitting them to five members of the speech and history departments.95 Blewett96 (1951) had two objectives, the one pertinent to the present study reads: To construct a listening comprehension test sufficiently reliable and vaéid to measure the learning resulting from listening. 92 Ralph William Widner, Jr., "A Preliminary Study of the Effects of Training in Listening", Unpublished M.A. thesis, University of Oklahoma, 1950. 152 pp. 95 Ibid., 89-119. 94 Ibid., 11. 95 Ibid., 10. 96 Thomas T. Blewett, "An Experiment in the Measurement of Listening at the College Level", Journal of Communication, I (May, 1951), 50-57. 97 Ibid., 50. 226 He describes his two part test as follows: The test on Content Retention attempted to appraise listening on what might be termed the "factual level"; that is, ability to comprehend and almost immediately recall factual matter, such as names of people, places, . . . etc. . . . This subtest was composed of five sections, each section containing a passage of information, followed immediately by a series of eight questions designed to test the extent to which the subject was able to recall the facts presented in the passage. Both the informa- tive passages and the questions were presented by means of a paper tape recording machine. . . . The questions were of the simple recall type, approximately sixty per cent of which could be answered in one word. . . . The subjects were allowed between eight and nine seconds in which to write responses to each question on a numbered line on a prepared answer sheet, each line having been keyed to the question presented by the recorded voice. The informative passages averaged 200 words in length, the shortest containing 198 words, the longest containing 206 words. Rate of delivery for both infor- mative passages and questions was recorded at between 120 and 140 words per minute. The time required forgghe administration of Subtest I was twenty-five minutes. The Test on Drawing Conclusions was designed to measure listening on a "higher level“ than that of more factual retention. It was concerned with the ability to draw conclusions from a series of related ideas pre- sented orally, to make inferences, to identify speaker attitudes. This test was composed of six sections. Each sec- tion contained a passage of information carefully se- lected from outstanding Speeches and essays. Each pas- sage of information was followed immediately by a series of questions. The passages of information and the ques- tions were presented orally. The questions were of the multiple-response type, the subjects choosing their re- sponses from among five possible answers appearing in typewritten form on a prepared answer sheet. The re- 98 Ibid., 51. 227 sponse-time for the questions varied from ten seconds for items involving the least reading to twenty-nine seconds for those involving the most reading. The informative passages in Subtest II averaged 270 words. . . . Rate of delivery. . . was recorded at be- tween 120 and 140 words per minute. The time requireg9 for administration of this subtest was fifty minutes. A tentative draft of the test was recorded and played for two consulting sections of students. They took and scored their tests, and then with the aid of the printed script carefully analyzed, evaluated, and revised questions and passages. The test was again recorded and tried out and the time intervals needed for answering questions to be read was determined. The revised test was tried out with 150 students to determine the discriminating value of the ques- tions and the reliability of the test. Reliability as deter- mined by the Kuder-Richardson formula was .76 for Part I; .75 for Part II; and .81 for the entire test. The mean dif- ficulty of the entire test was 50.50. Heilman100 (1951) described his test as follows: Two comparable forms of a test of listening compre- hension were built. Each form consisted of three sub- tests. Two sub-tests on each form consisted of items based upon sustained listening material as follows: Form A. "The Illinois Plan", a 1,875 word abstract from a pamphlet concerned with education of exceptional 99 Ibide , 51'52e 100 Arthur W. Heilman, "Measuring and Improving Listening Ability", Speech Monographs, XVIII (No. 4, 1951), 302- 308. 228 children in the state of Illinois, and "The American Scholar", a 1,604 word passage abstracted from a speech by Howard Mumford Jones, dealing with Emerson's views on American education. Form B. "Food Preservation", a 1,997 word passage abstracted from a pamphlet prepared by H. G. Heinz Com- pany, dealing with man's search for improved methods of food preservation, and "Education for Freedom", 8 1,679 word passage abstracted from a speech by Robert M. Hutchins, dealing with education in America. The third sub-test which was common to both forms of the test consisted of two parts, one of which in- volved critical listening to aurally presented problems dealing with directions and identification of figures. The material was contained in a four page 5%" x 8%" booklet provided each subject, although directions and other statements about things to be done with the mater- ial were presented aurally. The second part of this sub-test consisted of a series of stimulus statements each followed by four other statements called test statements. The subject, after hearing the stimulus statement judged each of the other four statements as either carrying the same meaning as the stimulus statement or a meaning different from it. The test of sustained listening material, sub-tests l and 2, forms A and B, was recorded. Records were cut by professional engineers. . . . The same narrator was used for all the material so that voice quality could not favor one form or the other. Rate of delivery was not held rigidly constant sentence by sentence, but the rate on all records of sustained material fell within the limits of 120 to 140 words per minute. A total of 41 multiple choice questions was prepared to test the listener's comprehension. These were printed and pre- sented to the subjects in test booklets. The directions, problems, and statements of sub- test 3 were likewise recorded for presentation to the listeners. They worked their responses on a regular IBM answer sheet during short intervals of silence which followed each item. This sub-test also contained 41 items. Thus each form of the test was comprised of 82 items. 229 Preliminary form of these tests, consisting of 112 and 117 items were administered to a sampling of approx- imately 200 college freshmen for the purpose of securing data for an item analysis, determining reliability and equating forms A and B. The Discrimination index of each item was found by the Flanagan Method. Items not having a discrimination index of .20 and falling within the difficulty limits of 20-80 per cent passing, were rejected. . . . Reliability of the tests was determined using only the retained items in the computation. First, the reli- ability of the sub-tests of sustained material was com- puted by determining the correlation coefficient between forms A and B. The obtained coefficient was .80. The reliability of the sub-test 3, problems, state- ments, etc., was computed by determining the correlation between odd and even items and using the Spearman-Brown Pr0phecy Formula to estimate the reliability of thelapll- length sub-test. The reliability obtained was .84. Kramar and Lewis102 (1951) were concerned with measuring listening with and without visual cues. They used freshman college students and built an eXplanatory lecture. The lecture was expository in nature and contained factual data concerning the country of New Zealand, its geographical, historical, political, and economic fac- tors. This subject was chosen because it was believed that it would limit the factor of previous knowledge as a variable. The test was composed of forty multiple- choice questions. The questions offered four responses so that the chance factor of success was minimized. The lecture and the test were administered to fifty students prior to the experimental situation. An item analysis revealed that there were several questions which failed to discriminate. . . . These questions 101 102 Ibid., 303-304. Edward J. J. Kramer and Thomas R. Lewis, "Comparison of Visual and Nonvisual Listening", Journal 9f Communication, I (November, 1951), 16-20. 230 were analyzed and changes made. . . . After the test was administered in the experimental situation an item analysis revealed that there were seven items which were still ineffective. These seven questions were eliminated and the papers were rescored on the basis of the thirty- three retaisgd items. The reliability of the test was .71. . . . The Stephens College Test of Listening Comprehension104 (1951) is described as follows: The Stephens College Test of Listening Comprehen- sion is an untimed’test of‘the student's ability to Tie- ten for comprehension of a speaker's meanings in eXposi- tory oral communication. This test is based on the as- sumptions that textual materials composed of extensive but complete excerpts from talks present a realistic functional situation common in the student's daily ex- perience; that two passages, one primarily explanatory, the other primarily argumentative, represent two very common listening situations; that testing a student's recognition of main idea and details and logical infer- ences and relationships of idea will give a valid measure of a student's general listening ability; and, finally, that pro-testing and post-testing of such general abil- ity, with intervening instruction in listening, will yield a measure of growth in this ability. Since the above processes occur more or less simultaneously in the listening situation, items testing them are interspersed throughout the test. Directions for the test are stated as follows: Two short talks will be read to you. Each talk is an excerpt from a longer speech. The first talk, or Part I of the test, is an excerpt from a lecture by Dr. Henry Bowman based on his book Marriage For Moderns; the second talk, or Part II of the test, 16 an excerpt—from a baccalaureate address given by Dr. Homer P. Rainey. 103 104 Ibid e , 16'17 e Stephens College Test of Listening Ability, Form A, (Mimeographed) Prepared by the Division of Communication, Stephens College, Columbia, Missouri, 1951, 20 pp. 231 You are to listen without taking notes. At the end of each part you will be asked to answer multiple-choice questions about what you have heard. The questions will be read to you and you will choose the one response which in your judgment is the best an- swer. In answering the questions, blacken the one num- bered space on your answer sheet that corresponds to the number in your test booklet. I will now read the first of the two parts. This is an excerpt from a talk on marriage. (Four double-spaced, elite typed pages on marriage are given.) Open your Test Booklet to Part I. I will state the question. You will choose the one response which in your judgement is the best answer. Blacken the one num- bered space on your answer sheet that corresponds to the number in your test booklet. I will read each question only once. Listen carefully. Ready now for question 1. 1. What point did the Browning story illustrate? The student has a test booklet with the following choices: 1. That marriage is like poetry 2. That peOple can be happy though married 3. That successful marriage begins with a good choice 4. That not all marriages are as happy as theirs. There are twenty-five questions. Part II of the test is carried out in a similar manner, and has a total of twenty questions. No information concerning test analysis is given in the mimeographed material, but a personal letter from Professor Donald Bird105 states that "the test has been analyzed care- 105 Personal letter from Prof. Donald Bird, of Stephens Col- lege, June 17, 1952. 232 fully by the Research Service and has gone through two re- visions." APPENDIX B INDIVIDUAL TEST SCORES TEST 1 -- MALE SCORES -- TEST AND RETEST fit Total Total Test Retest Psychological Reading Student Score Score Score Score 1 12 No Test No Test 2 8 12 113 79 3 17 22 84 52 4 6 10 63 13 5 5 8 110 47 6 8 7 80 31 7 21 22 100 78 8 13 18 95 51 9 2 9 No Test so Test 10 14 13 108 74 11 7 - 20 103 52 12 12 18 99 49 13 12 12 108 76 14 10 16 88 68 15 5 11 78 42 16 9 18 104 57 17 10 4 59 15 18 17 16 83 42 19 5 9 106 34 20 9 18 89 58 21 6 12 7O 21 235 Total Total Test Retest Psychological Reading Student Score Score Score Score 22 13 9 105 35 23 15 12 120 97 24 24 25 146 113 25 5 12 63 22 26 11 21 123 100 27 11 8 107 77 28 8 11 No Test No Test 29 6 5 114 82 3O 10 8 101 69 31 21 21 141 121 32 2O 16 114 95 33 10 4 78 64 34 6 15 91 63 35 28 26 144 107 36 16 15 90 32 37 5 11 74 44 38 11 12 No Test N0 Test 39 13 16 no Test No Test 40 7 5 No Test Ne Test 41 11 18 No Test Ne Test 42 10 6 79 33 43 3 12 101 35 44 11 14 86 29 45 16 16 93 62 236 Total Total Test Retest Psychological Reading Student Score Score Score Score 46 26 29 Ne Test No Test 47 11 18 95 57 48 10 9 73 29 49 19 25 - 78 41 50 5 10 70 34 51 15 20 106 60 52 17 16 96 64 237 TEST 1 -- FEMILE SCORES -- TEST AND RETEST Total Total Test Retest Psychological Reading Student Score Score Score Score 53 8 11 108 52 54 8 ll 96 23 55 15 12 129 93 56 10 11 97 81 57 15 12 79 53 58 18 23 101 75 59 12 14 79 29 6O 6 10 103 82 61 20 25 107 43 62 2O 23 121 92 63 19 18 106 91 64 16 16 119 78 65 6 6 94 58 66 14 25 120 80 67 16 18 109 54 68 18 19 114 80 69 7 23 99 81 TEST 1 -- MALE SCORES -- TEST AND NICHOLS TEST 238 Total Total Test Nichols Psychological Reading Student Score Score Score Score 70 16 14 103 66 71 l3 15 104 74 72 8 16 111 93 73 20 19 123 100 74 9 17 No Score No Score 75 24 18 122 103 76 8 12 77 5O 77 10 13 93 69 78 14 16 104 109 79 15 17 92 54 180 25 17 88 52 81 8 14 114 53 82 22 16 118 89 83 15 16 97 69 84 2 13 No Score No Score 85 14 15 135 85 86 l2 14 103 38 87 l 8 56 8 88 10 22 102 97 89 11 15 96 42 9O 12 10 95 53 239 Total Total Test Nichols Psychological Reading Student Score Score Score Score 91 9 ll 62 33 92 18 18 No Test No Test 93 3 10 72 17 94 20 14 101 81 95 6 12 109 37 96 8 15 127 70 97 12 19 138 100 98 19 22 123 79 99 9 ll 74 19 100 8 18 101 70 101 17 16 119 132 102 19 22 114 86 103 ll 20 No Test No Test 104 9 14 102 68 105 10 13 No Score No Score 106 6 13 106 82 107 16 20 125 79 108 10 20 No Score Ho Score 240 TEST 1 -- FEMALE SCORES -- TEST AND NICHOLS TEST Total Total Test Nichols Psychological Reading Student Score Score Score Score 109 13 12 84 51 110 12 12 80 27 111 12 8 6O 37 112 5 9 79 40 113 10 14 68 57 114 14 14 91 45 115 18 15 89 69 116 13 15 111 53 117 16 18 129 107 118 16 19 81 72 119 11 9 7O 22 120 14 8 59 39 121 l4 14 85 58 122 7 10 No Test no Test 123 8 14 104 79 124 11 15 124 67 125 18 16 No Score No Score 126 10 12 96 36 127 5 10 86 41 128 21 13 104 84 129 9 8 86 54 TEST 2 -- MALE SCORES -- TEST AND RETEST 241 Total Total Test Retest Psychological Reading Student Score Score Score Score 130 10 12 102 43 131 15 13 85 38 132 25 23 150 . 120 133 14 18 112 63 134 23 21 108 56 135 13 14 136 109 136 20 19 132 91 137 14 21 112 62 138 9 9 73 28 139 17 17 99 61 140 8 15 85 60 141 21 21 120 61 142 19 21 109 42 143 8 9 115 69 144 10 12 9O 65 145 24 28 111 83 146 7 10 110 25 147 11 11 59 39 148 12 9 115 70 242 TEST 2 -- FEMALE SCORES -- TEST AND RETEST Total Total Test Retest Psychological Reading Student Score Score Score Score 149 14 17 150 124 150 18 19 122 128 151 13 16 107 87 152 20 18 76 46 153 13 16 74 35 154 24 17 . 94 55 155 16 18 114 63 156 ll 10 110 70 157 13 10 97 72 158 7 8 92 50 159 20 16 127 93 160 7 18 68 54 161 13 16 128 123 162 12 15 No Test No Test 163 14 14 73 15 164 12 15 127 100 243 TEST 2 -- MALE SCORES -- TEST AND NICHOLS TEST Total Total Test Nichols Psychological Reading Student Score Score Score Score 165 22 17 108 67 166 17 16 138 82 167 16 12 119 69 168 10 12 89 41 169 l3 17 90 61 170 14 17 100 63 171 6 13 No Test No Test 172 25 15 136 65 173 13 19 91 62 174 8 17 106 70 175 6 6 47 15 176 l4 16 117 58 177 10 12 114 46 178 19 18 123 73 179 24 19 124 100 180 19 20 173 97 181 5 17 91 57 182 14 10 72 38 183 19 18 122 80 184 9 18 91 81 185 17 20 115 107 244 Total Total Test Nichols Psychological Reading Student Score Score Score Score 186 20 17 108 62 187 11 18 112 68 188 9 12 111 67 Fran-r .gpr'qg-» M'- 245 TEST 2 -- FEMALE SCORES -- TEST AND NICHOLS TEST Total Total Test Nichols Psychological Reading Student Score Score Score Score 189 22 23 147 102 190 14 ll 95 66 191 18 16 130 97 192 10 9 101 75 193 9 18 108 81 194 7 6 7O 16 246 TEST 3 -- MALE SCORES -- TEST AND RETEST Total Total Test Retest Psychological Reading Student Score Score Score Score 195 10 16 78 51 196 17 19 105 91 197 15 15 97 35 198 26 25 94 89 199 11 16 No Test No Test 200 17 . 15 107 59 201 17 2O 88 68 202 19 18 118 45 203 25 28. 119 102 204 16 22 121 94 205 16 26 94 70 206 15 23 89 57 207 13 10 91 26 208 22 29 103 69 209 15 22 105 86 210 22 24 125 79 211 23 27 131 103 212 20 24 128 103 213 11 16 98 72 214 22 24 108 72 215 23 18 No Test No Test 216 10 14 119 79 247 TEST 3 -- FEMALE SCORES -- TEST AND RETEST , Total Total Test Retest Psychological Reading Student Score Score Score Score 217 13 16 99 72 218 11 21 113 67 219 21 24 138 114 220 13 24 100 79 221 24 23 140 117 222 10 16 99 74 223 19 24 112 87 224 27 25 132 122 225 15 14 71 70 226 8 4 124 96 227 9 14 77 22 228 7 17 104 51 229 25 21 149 131 230 26 ' 25 151 97 231 14 17 98 59 232 23 22 94 63 233 15 2O 74 34 248 TEST 3 -- MALE SCORES -- TEST AND NICHOLS TEST . Total Total Test ’ Nichols Psychological Reading Student Score Score Score Score 234 15 15 110 77 235 19 18 124 100 236 10 10 121 48 237 14 11 9O 45 238 3 8 69 29 239 24 17 104 113 240 25 20 131 110 241 16 21 115 84 242 26 19 134 86 243 17 16 9O 76 244 21 19 94 63 245 15 19 97 55 246 17 17 87 37 247 19 13 100 64 248 20 20 130 92 249 25 19 129 94 250 16 12 75 25 251 3 9 98 53 252 24 21 121 98 249 TEST 3 -- FEMALE SCORES -- TEST AND NICHOLS TEST Total Total Test Nichols Psychological Reading Student Score Score Score Score 253 22 18 134 89 254 11 13 125 86 255 17 14 90 55 BRET-1.15mi” LOAN Ar: ‘7 53 INTER Lifififgd Lust APR 7 ’55‘ EV 26 '55 lrnzg'i’t BRARY LEAN SEP 1 7 '55 mm uanm LOAN OCT16'55 INTER UBRARY LOAN NW 6 ,55 ”mass Im'li'sc MICHIGAN smTE UNIV. LIBRARIES m1|WI”HIIWIWII"MINIMUMWIWIWI 31293102521972