AN INQUIRY INTO POSSIBLE APPLICATION OF “CLOZE-PROCEDURE” AS A DIAGNOSTIC AND THERAPEU'I'IC TECHNIQUE IN CASES OF ADULT DYSPHASIA Thesis for degree of Master of Arts Michigan State University WILLIAM HOWARD HAAS I 960 LIBRARY Michigan Stan University AN INQUIRY INTO POSSIBLE APPLICATION OF "CLOZE-PROCEDURE" AS A DIAGNOSTIC AND THERAPEUTIC TECHNIQUE IN CASES OF ADULT DYSPHASIA by WILLIAM HOWARD HAAS A.THESIS Submitted to the College of Communication Arts Michigan State University of Agriculture and Applied Science in partial Fulfillment of the requirements for the degree of MASTER OF ARTS Department of Speech 1960. A FPIEWED KZ Jyfiw ABSTRACT (William H. Haas) "Cloze-Procedure" originated and validated by Wilson L. Taylor at the University of Illinois, has been reported to indicate working intelligence, attention span, knowledge and learning of a subject, perception, language skill, listening comprehension, and reading comprehension. The purpose of this investigation was to ex- plore possible application of this concept to cases of adult dysphasia. The specific area of measuring learning was chosen for experimentation. The following hypothesis was stated: Comparison of the results of pre-indoctrination and post-indoctrination scores obtained from a given text will indicate post scores are higher than pre scores; therefore, measuring the degree of learning that must have taken place. The collected data consisted of the responses of 15 subjects to 8h word deletions on five reading tasks. Experimentation was initiated through three trials at "blosure." Trial One was instigated to measure existing knowledge for the topics in the investigation. Trial Two was accomplished to test the ability of this technique to quantitatively measure a degree of learning that took (William H. Haas) place following indoctrination sessions. A third attempt was provided in order to ascertain the instrument's sensiti- vity to measure recall after the delay of a one week time factor. An analysis of the results indicated a substantial increase in the raw scores and.mean gain between Trials One and Two. However, a comparison between the scores of Trials Two and Three, was not as obvious; therefore, a statistical procedure was applied in the form of a one tail simple‘g test. The results were highly significant beyond the .01 level of confidence. It was concluded that ”close-procedure” was successful in measuring learning, within the conditions of the study, and that the technique was sensitive to the degree of measuring delayed reaction to recall. The multiplicity of applications this technique seemingly possesses was suggested. The concept of "closure" was discussed as a therapeutic aid, an evaluation instrument, and a potential resource for future research. 'c‘haries Pedrey, P‘fiffif Major Professor ii II. III. IV. VI. VII. VIII. IX. X. XI. TABLE OF CONTENTS ACKNOWLEDGMENTS............................. LIST OF TABLES.............................. INTRODUCTION................................ PURPOSES OF STUDY........................... THE CLOZE PROCEDURE......................... HYPOTHESIS AND EXPERIMENTAL DESIGN.......... RESULTS..................................... CONCLUSIONS AND DISCUSSION.................. SUMMARY..................................... APPENDIX.................................... BIBLIOGRAPHY................................ iii iv < \I-lrl-J l2 19 214 29 31 1&2 ACKNOWLEDGMENTS Foremost, I would like to express my sincere appreciation to Dr. Charles Pedrey, Director of the Speech Clinic at Michigan State University, for his encouragement, guidance, and kindness in planning and developing this paper. Special thanks are extended to Dr. Max Nelson, Director of the Hearing Clinic at Michigan State University, and Mrs. David Hayes, Psychologist in Ft. Worth, Texas, for their critical comments and suggestions. A special acknowledgment is extended to Mr. William J. Haywood, Consultant in Clinical Psychology at the Society for Crippled Children and Adults' Rehabilitation Center in Ft. Worth, Texas, for his invaluable contribution to the analysis of the statistical data for this study. iv LIST OF TABLES Table Page 1. Order of Subject Participation in IndiV1dual TGStingooeoooooooooeeoooooo 18 2.. Mean Number of Correct Responses on Three Attempts at Closure.......... 20 3. Mean Difference, Standard Deviations, and Standard Error Revealed Through Three Trials at Closureoooooeoeoeooooo 22 h. Standard Error and E Score Between Trials TWO and Threeooooeooeeooeooeeoo 23 5. Raw Scores of 15 Subjects in Three Attempts at "Closurenooeoo000000000000 31 INTRODUCTION Hospitals and rehabilitation centers throughout the United States report that the referral incidence of adults with aphasia is constantly increasing. This higher incidence of referral is due, partially, (l) to more exacting medical diagnosis of aphasia, (2) increases in strokes and accidents caused by the pressure of fast- paced living, and (3) progress of medical science in de- creasing the death rate immediately following the cortical insult.l With this subsequent influx of aphasic patients into Speech centers throughout the country, it becomes concurrently imperative that the speech therapist re-evaluate, closely define, and improve approaches to be used in therapy. Information regarding therapy techniques for the initial stages of building a basic language function is much more readily found than information for measur- ing and s0phisticating a language function that already exists. Many Speech therapists working in the "vocation- ally oriented" rehabilitation center often find themselves in need of this type of information. For our discussion here, therefore, we might classify aphasic patients into 1Nancy E. Wood, "Helping the Aphasic Adult," Journal of Rehabilitation, 22 7, (January, 1950). p. 7. 2 two categories: (1) those with little or no basic language function, and (2) those with a basic language function in need of sophistication. It is with the letter that we will have primary interest. Linguists and psycho-linguists have urged our profession to utilize more of the linguistic concepts when dealing with this type of language behavior.2 They tell us that they have a wealth of information which seemingly would have great potential for application to aphasia problems. A survey of the literature brings forth the gestalt notion of "clozure." In fact, a method to measure this in regard to language behavior has been introduced and validated by William L. Taylor and associates. It is called the "cloze-procedure."3 The term "cloze" is derived from the idea of closure, or the tendency to fill in a missing link in a well-structured whole. Given the sequence, "dogs bark and meow," most readers would immediately supply "cats." As the actual procedure has been the-missing worked out for application, the eXperimenter deletes every nth word in a text, leaving the equal sized blanks in their places and the subjects read through the passage filling in 2M. H. Sargill, "Modern Linguistics: RecoVery from Aphasia," Journal of Speech and Hearing Disorders, 19 (December, 1934), p. 507. 3Wilson L. Taylor, "Application of 'Cloze' and Entropy Measures to the Study of Contextual Constraint in Sample of Continuous Prose," (unpublished Ph.D. dissertation, University of Illinois, 195A). 3 the missing words. The more closely the totality of sequential cues in the passage elicits at each test point the same word thought as intended, the higher will be the subject's "cloze" score, or communication success.Ll In the above sequence, the reader could not have guessed "cats" unless he: (1) had functional native ability, (2) attended to the message, (3) was familiar with the words and grammar of the message, (4) associated the message with feline sounds, (5) understood the whole meaning of the mutilated sentence, and (6) had language habits which made him use the same word the message called for. "The 'guess' depended on his intelligence, attention, knowledge, perception, language habits, and comprehension."5 The significant advantage of Taylor's "cloze- procedure" is that it taps simultaneously all of the complex determinants affecting word choice, both at the various levels of organization and through stretches in context. "Cloze" scores seem to depend on most, and per- haps all, of the many factors which have significant influence on communication; receptive, expressive, and intellectual abilities. uIbid., p. 68. 5Wilson L. Taylor, "Cloze Procedure," Agrisearch, 2:2 (February, 1956), p. 3. PURPOSES OF STUDY As pointed out above, the procedure has been reported to indicate working intelligence, attention span, knowledge and learning of a subject, perception, language skill, listening comprehension, and reading comprehension. If "cloze-procedure" is successful in measuring these areas with normal adults, what application can be made with cases of adult dysphasia? Obviously, in a paper of this SCOpB, it is impossible to investigate all areas with integrity. Consequently, this writer has chosen to study the "procedure's" usefulness in measuring the amount of learning exhibited by a given patient or group of patients for a particular text. HYPOTHESIS. Comparison of the results of pre and post "cloze" scores obtained from a given text will indicate post-indoctrination scores are higher than pre— indoctrination scores; therefore, measuring the amount of learning that must have taken place in cases of adult aphasia. A secondary purpose is to discuss other possible application of the concept, specifically: its potential as a therapy method, an evaluation instrument, and its implication for future research. To facilitate the discussion of these objectives this investigation is arranged into the following areas: 5 (l) definitions and delimitations, (2) historical discussion of the methodology, research, and previous application of "cloze-procedure," (3) discussion of experimental design, hypothesis, and results, (4) other potential uses of the technique, and (5) conclusion. Definitions and delimitations..There is great variance in the use of the term aphasia among the pro- fessionals in the field, and by the laymen. There is agreement that the term should be limited to language disorders which derive from organic impairments. However, a language disturbance caused by deafness, blindness, or by various psychological disorders should not be referred to as aphasia. Dysarthria, although neurological in nature, is not included because faulty symbolizing is not characteristic.6 Therefore, for the needs of this writing, aphasia is defined as the loss of symbolic language as the result of brain damage. For the purposes here, there is no need for classification definitions. Dysphasia is a term which will be used to designate a milder degree of such disturbance. Under this category will be the patient who exhibits a useful language function. 6Helmer R. Myklebust, ”Aphasia in Children: Language Development and Language Pathology," Handbook of Speech Pathology, ed. Lee E. Travis (New York: Appleton-Century— Crofts, Inc. 1957) p. 507. ' 6 A basic language function will be said to have been fulfilled when the patient: (1) obtains a vocabulary of approximately 100 words which he can both understand and verbally eXpress, (2) uses short phrases or sentences to express his needs, (3) can read sentences and short paragraphs with comprehension, and (A) can eXpress him- self in writing using at least single words. Absolute accuracy, of course, should not be a factor in all cases. "Cloze—procedure" is a technique that provides a measure of the degree of correspondence between the language habits used by a human transmitter while ex- pressing a message and those employed by a human receiver while interpreting this message. It does so by inter- cepting and mutilating a message between transmitter and 7 receiver, and asking the receiver to make it whole again. A cloze score is simply the number of times deleted elements of a text are successfully anticipated and replaced in context.8 7Taylor, :Application of 'Cloze' and Entropy Measures...," p. 08. 81bid. THE CLOZE PROCEDURE Historical review. The "Close-procedure" was first instroduced in 1953 at the University of Illinois. Initially, it was regarded mainly as an improved way of measuring the readability, or ease of reading by an in- dividual or group of individuals, of given texts covering specific subject matter. The great bulk of studies were devoted to the procedure's readability uses.10 Taylor demonstrated the feasability of his technique for this purpose. In fact, it "produced" much more satisfactorily than either the Flesch or Dale- Chall formulas, traditionally used as demonstrations by the authors of these formulas in the same way, but on specialized texts it alone yielded sensible results. For example, both Flesch and Dale-Chall indicate a passage from Gertrude Stein as being easy. Taylor's procedure shows Stein, more appropriately, as n23 easy! In other words, this technique takes into account the unpredictable expressive style characteristic of Stein.11 9Wilson L. Taylor, "Cloze-Procedure: A New Tool for Measuring Readability," Journalism Quartly, XXX (Fall. 1953) h15'533. lOIbid. 11Ibid. p. h25. 8 As a measure of degree of comprehension, the "procedure" also has enjoyed success. In a very care- fully designed experiment using Air Force personnel for which comprehension tests were already available, it was found that "cloze" scores correlated very highly with initial comprehension scores (pre-indoctrination) and also predicted final comprehension (post-indoctrination). Approximately 150 Sampson Air Base trainees were used for this study. The experiment was administered in two parts separated by a week's time. As a first step, the subjects were given the "clozé'form, then a carefully constructed "before" comprehension test. This was done to measure their existing knowledge of the article's contents. Secondly, the subjects were given the article to study. Then, an "after" comprehension test was given. And lastly, another "cloze" form was administered. The results showed the "cloze" and traditional score differ— ences were statistically significant to above the .001 level of confidence. Further, scores using "closure" were positively correlated with general intelligence at .74. Intelligence quotients were obtained from the Armed Forces Qualification Test.12 The usefulness of the technique has not been limited to the English language. The procedure was found l2Wilson L. Taylor, "The Cloze Procedure: How it predicts Comprehension and Intelligence of Military Personnel," Technical Memorandum No. 13, Urbana, Illinois: Division of Communications, University of Illinois, (1953). 9 to be successful in Korean, 3 language written in a mixture of phonetic symbols and Chinese characters.13 Other studies have shown the "cloze-procedure's" effectiveness in measuring the degree of listening comprehension, as well as reading comprehension. Kerrick, at the University of California, has recently directed research which has shown that there is a significant correlation between how well readers say they like an article and their "cloze" scores on them.15 Taylor reports that studies have been undertaken which use still broader application of the "procedure" in other specialized areas. For example, individual differences and aptitudes have been studied in relation to reading comprehension.16 I To this writer's knowledge, there has been no application of this technique to the language behavior of the dysphasic patient. Methodology of administration and scoring; Method- ology for application of the "procedure" has been for- - l mulated by Taylor through a series of pilot studies. l3Wilson L. Taylor, "Recent Developments in the Use of 'Cloze-Procedure'," Journalism Quarterly, XXXIII (Winter, 1956), p. 40. 1”Ibid., p. 47. 15Ibid. 16Taylor, "Cloze Readability Scores as Indices of Individual Differences in Comprehension and Aptitude," Journal of Applied Psychology, XLI (February, 1957), 19—26. l7Taylor, "Cloze-Procedure: A New Tool...," p. 415. 10 After a particular text has been chosen for study, the experimenter systematically deletes a random number of words. Taylor has found that deletion of every fifth word puts the blanks as far apart as they need be for most investigations of normal subjects. However, blanks may be farther apart if the experimenter feels it is necessary. The mutilated texts are then reproduced with each missing word replaced by a standardized blank. Copies are distributed to every member of a test group. All sub- jects are asked to replace the deleted word. Each person is informed that all blanks are of uniform size and in no way indicate the length of the missing words.18 Every time a test-subject prOperly supplies the correct word, he scores one point. His "cloze" score for any text is the total number or percentage of deleted words he guesses correctly. For normal subjects, Taylor found nothing to be gained by accepting synonyms for the fill-ins. The problem of what an acceptable replacement for an intended word should be is obvious. He has found that only those words which exactly match the original should be counted.19 However, in cases of dysphasia this might be dramatically impractical. For the purposes of most studies it was found that the mutilation process need take no account of the kind or importance of specific words to delete. The results 181bid., p. 416. 19Ibid. 11 based on the deletion of any word which gets systematically counted out appears to satisfy the needs for most in- vestigations. In fact, Taylor feels this is a superior method. The limiting of missing words to nouns, verbs, adjectives, or prepositions is both unnecessary and awkward.20 However, we might point out that here is an apparent potential for contextual study of ability of dysphasics to handle the various parts of speech. Pilot studies also have shown that for most research the eXperimenter should have the text lengthy enough so as to permit a minimum of 50 word deletions. This is said to be necessary to represent a sensitive account- ing of the difficulty of a passage. Enough words must be deleted to permit simple and more difficult ones to cancel each other in proportion to the frequency of their occurrence?1 The "Cloze-procedure", as outlined above, has been, with slight modification, common to most previous appli- cations of the concept. Only with some variation of these techniques could we foresee adaptation for formal testing of adult dysphasic patients. These variations are dis- cussed below. 2OIbid., p. 420. 211hid., p. 419. HYPOTHESIS AND EXPERIMENTAL DESIGN In our attempt to pioneer an application of the "Cloze-procedure” to our subjects at hand with one of the many parameters of language behavior, we have randomly selected one specific area of possibility. Our question is: can we apply Taylor's concept to adults with dysphasia? More specifically; will "Cloze" scores indicate existing knowledge of a given subject, and subsequently evaluate learning that has occurred after study? Re-statement of hypothesis. Comparison of the results of pre-indoctrination and post-indoctrination scores obtained from a given text will indicate post scores are higher than pre scores; therefore, measuring the degree of learning that must have taken place. Sample. "Cloze" tests were given to 15 adult dysphasic patients enrolled and/or discharged from the Crippled Children and Adults' Rehabilitation Center in Ft. Worth, Texas. It was determined beforehand that all test-subjects exhibited a basic language function. They ranged in age from 28 to 61 years with a median age of 45 years. Median time in school was 13 years, with 5 subjects having completed four years of college and one subject completing eight years of higher education. Three subjects did not go beyond the eighth grade. 12 13 Subjects were not screened for dysphasia sub— classifications. Similarly, Specific areas of disability such as anomia, agraphia, and alexia were not measured quantitatively. However, all patients did possess a basic language function as defined earlier. Test materials. The stimulus material for this investigation consisted of five short texts on the same number of topics for each of the 15 test-subjects. (See appendix). Each of the five topics were graded at the fifth grade level of expectancy as to performance.22 All materials were typed in pica upper case letters on white 8%" by 11" paper. No instructions were necessary as to procedure for taking the tests as all subjects had identical orientation sessions as to its mechanics by practicing other tests on different topics. "Cloze" deletions were obtained by counting out five words and making a fifteen space blank for the sixth word. Five more words were typed, the sixth deleted, and so on, throughout all five passages. There was a ; total of 84 missing words for all passages combined. I There were four identical copies for each of the five texts. However, three were mutilated and one was not. The first copy was used for measuring each patient's pre-indoctrination aptitude. The second copy 22My Weekly Reader, January 18—22, 1960, pp. 1-u. lb. was used to measure post-indoctrination aptitude. The third c0py was to measure the effect of a week's delay regarding recall of the subject matter. The unmutilated copy was used for indoctrination purposes. Procedure. (Step One) On the first day of testing the text beginning -"On a stormy day..." (See appendix) was given individually to all test subjects in the same room by the same experimenter. An alphabetical order of subject participation was used. That is, the person's last name falling near- est to the beginning of the alphabet was first. The patient was issued one mutilated copy of the text. The only instruction given was as follows: "We will read the story together. I will stop when we come to a blank. Try to guess a word to fill each blank." As each attempt at "closure" was made, the investigator recorded the patient's response in the appropriate blank. A fifteen second time limit was imposed on the patient for each attempt at closure. If the subject failed to respond within this limit, the experimenter continued, leaving the space allocated for the missing word empty. Upon completion, the test was retained by the investigator and the subject was dismissed to the "waiting room". This alphabetical order of individual partici- pation was continued until all test subjects had com- pleted their attempt at pre-indoctrination testing for this tOpic. 15 (Step Two) Immediately following the individual testing, the entire group of test subjects assembled and 8 A5 minute indoctrination period started: (Each indoc- trination session for all five test days was identical in structure). The unmutilated texts were distributed. The experimenter read aloud the text. The patients read the passage in unison. Each member of the group was asked to paraphrase the contents of the article as briefly as possible. The group re-read the article in unison. The unmutilated forms were collected. UOEDD *1:th (Step Three) After the indoctrination session was concluded Step One was repeated. That is, each patient was tested individually using another mutilated copy of the text to measure post-indoctrination performance. The same order of subject participation was followed. The same time limits were imposed. Upon completion all test copies were retained by the experimenter and the subjects were dismissed for the day. On the second day of testing the text beginning - " was used. The order of subject "The quarter horse... participation was reversed, so that, the person's last name falling nearest to the end of the alphabet was first. The remaining procedure was identical to that of the three steps described above for the first day of testing. On the third day of testing the text beginning - "Food, as you know,... was used. The person's last name which fell most nearly to the middle of the alphabet was 16 first. Subsequent alphabetical order of participation was from this midpoint forward to the beginning of the alpha- bet and then from the end of the alphabet back to the midpoint. The remaining procedure was identical to the two preceding test days. On the fourth test day the text beginning - "To a " was used. The order of subject participation scientist... and remaining procedure was identical to that used on the first day of testing. On the fifth test day the text beginning - "Shrimp are found... was the tepic for the day. The order of sub- ject participation and remaining procedure was the same as that followed on the second day of testing. Obviously, immediate recall and learning were being measured through this "one day" pre and post indoc- trination testing for each of the five texts. How would this same group of dysphasic patients perform with delayed recall? One week later Step One, discussed above, was re-initiated alone. That is, each day the same group took the same tests they had taken one week before. Steps Two and Three were omitted. At no time during this experiment were any of the test subjects allowed to take cepies of the tests from the testing room. Scoring. All testing sessions and scoring were supervised by the experimenter. Scoring was accomplished in the same manner as the previous application described above. One important modification was necessary, however. 17 Due to the nature of the dysphasic's expressive incon- sistencies, it was necessary to allow for appropriate synonym substitutions in lieu of the exact word replace- ment. To determine if the synonym was an apprOpriate one, a panel of four Speech therapists was gathered to agree as to each word choice when it differed from the word originally intended. Only those substitutions which were unanimously confirmed as appropriate were scored. TABLE 1 ORDER OF SUBJECT PARTICIPATION IN INDIVIDUAL TESTING (Day) First Day Second Day Third Day Fourth Day Fifth Day Text "On a "The "Food, "To a sci- "Shrimp stormy quarter as you entist..." are day..." horse..." know..." found..." (Sub- 1 15 8 1 15 jects) 2 1h 7 2 1h 3 13 6 3 13 h 12 5 h 12 5 11 h 5 ll 6 10 3 6 10 7 9 2 7 9 8 8 l 8 8 9 7 15 9 7 10 6 1h 10 6 11 S 13 11 5 12 h 12 12 h 13 3 11 13 3 1h 2 10 1h 2 15 1 9 15 1 aThe above numbers assume an alphabetical arrangement. bFor the second trial at closure each day, the order was the same. For the third trial, to measure delayed recall one week later, the subject participation and se- quencing of the tests were identical. 18 19 RESULTS Observation and generalization are extremely difficult when attempting to measure any parameter of linguistic ability in cases of dysphasic individuals. Additional influencing factors to this difficulty arise from the lack of selectivity of subjects and the "sheer numbers" of test subjects. The reader of this report must be aware that any discussion of results is limited by all these factors. Group success with "closure." The data consisted of the responses of the 15 subjects to the 8A word de- letions on five reading tasks. If all subjects had an- swered correctly all deletions there would have been a sum total of 1260 correct responses, or if each individ- ual subject had answered correctly all deletions there would have been a sum total of 8A correct responses for each subject. All test scores combined yielded 435 correct responses or 34% of the deletions answered correctly for Trial One. Trial Two yielded 813 correct responses or 6A% of the deletions answered correctly. Trial Three yielded 747 correct responses or 59% of the deletions answered correctly. In Table 2 are presented the mean number of correct responses made by the subjects on the three trials. 20 TABLE 2 MEAN NUMBER OF CORRECT RESPONSES ON THREE ATTEMPTS AT CLOSURE Trial 1 Trial 2 Trial 3 Mean score (N315) 29.1 57.5 “9.8 A glance at the data indicating the differences between Trial One and Trial Two reveals a more than doubling of the individual scores, for this reason no test of sig- nificance was initiated. However, a mean gain was cal- culated and a standard deviation of the gains to indicate the variability of the scores was accomplished. Also, a standard error of the mean gain was computed to give an idea of the reliability. (See Table 3) The above scores should be taken only as tentative indications since the sample size was small and a normal curve had to be assumed without proof. However, the significance of the results of Trial 3 was not as obvious. Therefore, a test of significance was accomplished. In the selection of an appropriate statistical method the following points are of importance: 21 1. We were not dealing with a study determining whether the difference between two means is significant when these means represent the performance of different groups - boys and girls, high school graduates and college graduates, or aphasics and non—aphasics. 2. However, we were concerned with a statistical technique that is designed to predict the significance of the difference between two means obtained from the same test administered to the same group upon different occasions. 3. The study incorporated a sample size that was small and a normal curve had to be assumed without proof. Specifically, our statistical approach needed to be adaptable to evaluate the effect of the intervening seven day delay upon the final scores; or to estimate the effect of no indoctrination or study interpolated between test and retest. Garrett has outlined the procedure which meets our requirements most precisely. This method has been specifically designed for small groups which range in members from 12 to 25.23 The method is basically 8'2 test (the quantity 3,13 the distance from the mean ex- pressed in terms of standard error of the mean)24 for small groups where the means are calculated. Regarding the utilization of this technique, Garrett has offered this defense: "In problems like this, 23Henry E. Garrett, Statistics in Psychology and Education (New York: Longmans, Green aha Company, 1947) pp. 210-211. 2"Ibid. p. 191. 22 establishing and dealing with a mean gain or difference involves less calculation and is, also, to be preferred to the method of calculating standard errors (SE'S) for each mean, an SE of the difference, and the correlation between the scores."29 (1) Following this method, the mean difference between the subject's scores of Trial 2 and Trial 3 was established and a standard deviation or SD was calculated on these scores. This is given in the chart below. TABLE 3 MEAN DIFFERENCES, STANDARD DEVIATIONS, AND STANDARD ERROR REVEALED THROUGH THREE TRIALS AT CLOSURE W Mean Difference SD SE Between trial 1 & 2 28.4 10 2.56 Between trial 2 & 3 7.73 6.40 1.65 (2) The next step, according to this technique, was to calculate the standard error or SE of the mean difference. The formula is: SE . This gave a standard error of the mean difference which was 1.65. (3) How significant was our mean difference of 7.73? This significance was expressed by the critical ratio or E which became 7.73/l.65 - 4.68. 251bid., p. 211. 23 (4) Each subject gave two scores, thus the degrees of freedom were (N-l) or 14. For 14 degrees of freedom using a table of p_of 2.62, we will be exceeded in the positive direction in 1% of the trials. Since the critical ratio or 3 calculated above exceeds this number, the conclusion is that the difference of 7.73 is highly significant beyond the .01 level of confidence. Entries were taken from the .02 column of the 3 table for significant levels of .01 because the interest was in the probability of a positive difference as large as 6 or larger than 7.73.2 TABLE 4 STANDARD ERROR AND t SCORE BETWEEN TRIALS TWO AND THREE m L— M SE 3, df P (N-l) Between trials 2 and 3 1.65 4.68 143 . 2:2.01 8With 14 degrees of freedom a 2.62 is needed for confidence level of .01 (one tail test). Therefore, a p 4.68 = P \\.01. 26Ibid. CONCLUSIONS AND DISCUSSION One's conclusions are always limited to the factors studied, and it would be impractical to draw generalizations which involve factors and conditions which have only been assumed. It would appear that the "Cloze-procedure", within the conditions of this experiment, did Significantly measure a degree of learning that must have taken place in the 15 cases studied. We might conclude, also, that this technique significantly predicted with sensitivity the effects of a one week delayed recall factor. However, the potential efficiency of this instrument is still un- determined; in that, this study did not incorporate a comparison with another standardized measure of learning. Undoubtedly, this is a possibility for future research. Regardless of the results of a projected study of this nature, this investigator questions any implication that there is a measure of learning which is inherently superior, and that certain experimental conditions will permit this superiority to Show itself, while other conditions might hide this superiority. However, we would like to suggest that this technique does have Significant possibilities regarding the. 2k 25 measurement of learning with cases of adult dysphasia. Moreover, we would like to point out again that there is a significant advantage in Taylor's concept; in that, it seemingly taps simultaneously all of the complex determinants affecting the success of the communication act. Therefore, the multiplicity of applications this instrument seemingly possesses appears to be substantial. "Closure" as a therapeutic technique. AS re- ported above, success with "closure" seems to depend on most of the many factors which have significant in- fluence on communication. This, therefore, is a "power" test of verbal performance. More appropriately, we might say that this is a technique which possesses a powerful stimulation potential in that it, also, stimulates Simultaneously all of the complexities affecting word choice, both at the various levels of abstraction and through stretches in context. The following application can be made: To take advantage of both visual and auditory stimulation in a "word finding" task, a topic of interest is chosen by the patient. A text of the topic is originated, and subsequently mutilated. The patient is asked to read aloud from the mutilated copy. The therapist reads in unison with the patient, stopping at each deleted word. The patient attempts to fill the blank. The patient has 26 the advantage of being stimulated in a more normal manner by the sequential cueing of contextual material, has a topic of interest to him, and has both auditory and visual stimulation. With some imagination various modifications of this application can be formulated. For example, the filling in of the deletions could become a writing ex- ercise. By removing the visual stimulation of the printed text, a listening exercise is readily accomplished. "Closure" as an evaluation instrument. Many of the areas of application utilized in therapeutics suggested above seemingly have almost equal potentials in the evaluation process. If the materials for measurement could be graded as to difficulty regarding levels of concreteness or abstraction, academic levels of achieve- ment, and length; a rather comprehensive instrument for . evaluation could be devised. As we have reported earlier in the paper, the principle of "closure" has enjoyed success in indicating working intelligence, attention span, knowledge and learning of a subject, perception, language skill, listening comprehension, and reading comprehension with normal subjects. With the exception of measuring in- telligence per se, we might hypothesize that "cloze- procedure" could become a useful tool in ascertaining 27 the other parameters of linguistic ability with dysphasic patients as listed. Implications for future research. Obviously, any of the areas of study with normal persons discussed in this report could likewise be studied with dysphasic subjects. However, it appears to this writer that there are two problems reported in the literature with which this concept could be useful. There are conflicting studies reporting the dysphasic's ability at "naming" tasks. Schuell suggests that small words are more difficult to elicit in con- textual speech as they have a tendency to be overlooked and have slight auditory prominence. She has further implied that with increasing length of the stimulus unit more errors are found because of increased diffi- culty with reauditorization.27 With stimulation words presented in isolation, Siegel found a predominance of errors on long words.28 Another contradictory area has been reported by Siegel. Closely allied to the disagreement over the success or failure of the dysphasic to use "long" or 27Hildred Schuell, "Auditory Im airment in Aphasia: Significance and Retraining Techniques,‘ Journal of Speech and HearinngisorderS, 18:1 (March, 1953),I4-2l. 28Gerald M. Siegel, "Dysphasic Speech Responses to Visual Word Stimuli, Journal of Speech and Hearipg Disorders, 2:2 (June, 1959), pp. 152-160. 28 "short" words, is the dysphasic's ability to display equal performance with the various parts of speech in context.29 Regardless of these contrary reports, the "Cloze—procedure" would appear to be an extremely useful instrument to facilitate further analyses of these problems. In order to accomplish this task, the following basic plan is offered: (1) Originate a text of suffi- cient length so as to allow a gradual transition of the stimulus material from rather concrete to highly abstract. (2) Systematically delete every nth word as prescribed in previous application reported in this study. (3) However, repeat the deletion process often enough by rotating the order of word deletions so that eventually every word in the text has been withdrawn. For example, if every fifth word were chosen, the experiment would require five tests to complete the process. (4) A procedure for administering the test could be closely paralleled to the one we have utilized in this investigation. 29Ibid. SUMMARY It was hypothesized that Wilson L. Taylor's "Cloze-procedure" would measure learning exhibited through the performance of dysphasic adults. The data consisted of the responses of 15 subjects to 84 word deletions on five reading tasks. EXperi- mentation was initiated through three trials at "closurey Trial One was instigated to measure existing knowledge for the topics in the investigation. Trial Two was accomplished to test the ability of this technique to quantitatively measure a degree of learning that took place following indoctrination sessions. A third attempt was provided in order to ascertain the instru- ment's sensitivity to measure recall after the delay of a one week time factor. An analysis of the results indicated a substantial increase in the raw scores and mean gain between Trials One and Two. However, a comparison between the scores of Trials Two and Three, was not as obvious; therefore, a statistical procedure was applied in the form of a one tail simple p_test. The results were highly signi- ficant beyond the .01 level of confidence. It was con- cluded that "Cloze-procedure" was successful in measuring learning within the conditions of this study, and that 29 30 the technique was sensitive to the degree of measuring delayed recall. The multiplicity of applications this technique seemingly possesses was suggested. The concept of "closure" was discussed as a therapeutic aid, an evaluation instru- ment, and a potential resource for future research. APPENDIX A TABLE 5 COMPARISON OF RAW SCORES OF 15 SUBJECTS IN THREE ATTEMPTS AT "CLOSURE" WITH NUMBER OF YEARS OF FORMAL EDUCATION SUBJECTS Years of Trial 1 Trial 2 Trial 3 Education 1 8 13 34 28 2 20 61 7h 63 3 8 A0 58 37 h 12 18 6o 44 S 12 22 56 52 6 16 28 66 62 7 8 16 39 21 8 12 19 45 45 9 16 25 S9 63 10 16 31 71 58 11 12 18 ha 38 12 12 26 60 56 13 12 69 77 77 14 16 24 SS 50 15 16 27 61 53 31 APPENDIX B ON A STORMYIDAY IN PHILADELPHIA, MORE THAN 200 YEARS AGO, LIGHTNING CRACKLED THROUGH THE SKY. A SMALL SILK KITE TOSSED ABOUT IN THE WIND. WITH THAT KITE AS "BAIT", A MAN WAS "FISHING" FOR LIGHTNING. AS YOU KNOW, BEN FRANKLIN MADE A GOOD CATCH THAT DAY. ATTRACTED BY'A WIRE ON THE KITE, LIGHTNING RAN DOWN THE WET KITE STRING TO.A KEY. THEN FRANKLIN TRAPPED IT IN A JAR. HE PROVED THAT LIGHTNING WAS A FORM OF STATIC ELECTRICITY. THUS, ONE OF NATURE'S GREAT MYSTERIES WAS SOLVED. FOR CENTURIES, MAN HAD FEARED THE GREAT FLASHES OF LIGHT IN THE SKY. FRANKLIN USED HIS COMMON SENSE TO FIND OUT WHAT THESE FLASHES REALLY WERE. 32 33 THE QUARTER HORSE IS A HORSE THAT CAN RUN VERY FAST FOR A QUARTER MILE. BUT RACING IS NOT THE ONLY REASON FOR THE QUARTER HORSE'S POPULARITY. THE.QUARTER HORSE IS SEEN AT RODEOS AND HORSE SHOWS. IT IS THE FAVORITE MOUNT IN MANY RIDING CLUBS ALL OVER THE WEST. THE QUARTER HORSE HAS PROVED TO BE A GOOD "CUTTING" HORSE. CUTTING MEANS SEPARATING A CALF FROM THE HERD. THE CUTTING CONTEST IS BECOMING MORE AND MORE POPULAR. IN THE CUTTING CONTEST, A HIGHLY TRAINED HORSE, RIDDEN WITH A LOOSE REIN, CUTS A CALF AWAY AND HOLDS IT AWAY. 34 FOOD, AS YOU KNOW, IS A SOURCE OF ENERGY. THE SMALLER A WARM-BLOODED ANIMAL IS, THE FASTER IT BURNS FOOD, OR FUEL. HUMMINGBIRDS BURN FOOD SO RAPIDLY THAT THEY MUST EAT ALMOST CONSTANTLY. AS TWILIGHT COME, THE BIRDS STUFF THEMSELVES WITH INSECTS AND FLOWER NECTAR. THEN THEY SETTLE DOWN FOR THE NIGHT. THEIR BREATHING SLOWS AND THEIR BODY TEMPERATURE DROPS. BY THE MIDDLE OF THE NIGHT, THEY ARE HARDLY ABLE TO MOVE. THESE ARE SIGNS OF HIBERNATION. BY HIBERNATING FOR THE NIGHT, HUMMINGBIRDS ARE ABLE TO GO WITHOUT EATING FROM DUSK TO DAWN. 35 TO A SCIENTIST, A CRANBERRY MAY NOT BE CALLED A CRANBERRY. IT IS A PLANT WITH SEVERAL SCIENTIFIC NAMES. A SCIENTIST DIVIDES PLANTS AND ANIMALS INTO DIFFERENT FAMILIES AND GIVES EACH MEMBER OF THE FAMILY A NAME. A SCIENTIST KNOWS THAT THE CRANBERRY IS A COUSIN TO THE BLUEBERRY. BOTH OF THESE BELONG TO THE HEATH FAMILY. HEATH, OR HEATHER, IS A LOW SHRUB THAT GROWS ON THE MOORS (WASTELAND) IN GREAT BRITAIN. REAL HEATHER IS NOT COMMON IN AMERICA. MANY PLANTS IN AMERICA, HOWEVER, BELONG TO THE HEATH FAMILY. THE BEAUTIFUL MOUNTAIN LAUREL AND TRAILING ARBUTUS ARE TWO MEMBERS OF THE HEATH FAMILY. 36 SHRIMP ARE FOUND IN THE GULF OF MEXICO. MOST SHRIMP BOATS HAVE A CAPTAIN AND A CREW OF THREE. THE MEN MAY EAT AND SLEEP ON THE BOATS. SOME BOATS STAY OUT A WEEK OR LONGER. MOST OF THE SPACE IN THE BOATS IS KEPT FOR "SHRIMP PASSENGERS". THEY ARE CAUGHT IN A BIG NET. ON BOARD, THE SHRIMP ARE BEHEADED. ONLY THE TAILS ARE KEPT. THEY ARE PACKED IN ICE. SHRIMP FISHING IS OFTEN DANGEROUS. STRANGE OCEAN CREATURES ARE SOMETIMES CAUGHT IN THE NETS. ONE CAPTAIN CAUGHT A SHARK LONGER THAN AN AUTO. THIS FISH HAS A LONG SNOUT WITH SAW-TOOTHED EDGES. 37 'ON A STORMY DAY PHILADELPHIA, MORE THAN 200 YEARS . LIGHTNING CRACKLED THROUGH THE SKY. SMALL SILK KITE TOSSED ABOUT THE WIND. WITH THAT KITE "BAIT", A MAN WAS "FISHING" LIGHTNING. AS YOU KNOW, BEN MADE A GOOD CATCH THAT . ATTRACTED BY A WIRE ON KITE, LIGHTNING RAN DOWN THE KITE STRING TO A KEY. FRANKLIN TRAPPED IT IN'A . HE PROVED THAT LIGHTNING WAS FORM OF STATIC ELECTRICITY. THUS, OF NATURE'S GREAT MYSTERIES WAS . FOR CENTURIES, MAN HAD FEARED GREAT FLASHES OF LIGHT IN SKY. FRANKLIN USED HIS COMMON TO FIND OUT WHAT THESE REALLY WERE. 38 A QUARTER HORSE IS A THAT CAN RUN VERY FAST A QUARTER MILE. BUT RACING NOT THE ONLY REASON FOR QUARTER HORSE'S POPULARITY. THE QUARTER IS SEEN AT RODEOS AND SHOWS. IT IS THE FAVORITE IN MANY RIDING CLUBS ALL THE WEST. THE QUARTER HORSE PROVED TO BE A GOOD " "HORSE. CUTTING MEANS SEPARATING A FROM THE HERD. THE CUTTING IS BECOMING MORE AND MORE . IN THE CUTTING CONTEST, A TRAINED HORSE, RIDDEN WITH A REIN, CUTS A CALF AWAY HOLDS IT AWAY. BIBLIOGRAPHY Books Garrett, Henry E., Statistics in Psychology and Education. New York: Longmans, Green and Company, l9fl7. Myklebust, Helmer R., Handbook of Speech Pathology. "Aphasia in ChIldren: Language Developmefif and Language Pathology, Edited by Lee E. Travis. New York: Appleton-CenturyéCrofts, Inc., 1957. Articles and Periodicals Argill, M. H. "Modern Linguistics: Recovery from Aphasia,m Journal of Sppech and Hearing Disorders, 19 TDecember, l95h),7507. Schuell, Hildred. "Auditory Impairment in Aphasia: Significance and Retraining Techniques," Journal of Speech and Hearinngisorders, 18:1 TMarch, 1953). lE-Zl. Siegel, Gerald M. "Dysphasic Speech Responses to Visual Word Stimuli," Journal of Speech and Hearing Research, 2:2 (June, 1959), 1523160. Taylor, Wilson L. "Cloze-Procedure: A New Tool for Measuring Readability," Journalism Quarterly, XXX (Falls 1953), MIS'h33; . "Recent DeveIOpments in the Use of 'Cloze- Procedure'," Journalism Quarterly, XXXIII (Winter, 1956), H6. . "Cloze Readability Scores as Indices of Individual Differences in Comprehension and Aptitude," Journal of Applied Psychology, XLI (February, 1957) 19-26 a Wood, Nancy E. "Helping the Aphasic Adult," Journal of Rehabilitation, 22:7, (January, 1956) p. 7. ha 43 Other Sources Taylor, Wilson L. "Application of 'Cloze' and Entropy Measures to the Study of Contextual Constraint in Samples of Continuous Prose," (Unpublished Ph.D. Dissertation, University of Illinois, 195h). . "Cloze Procedure," Agrisearch, 2:2 (February, 1956 ) , l'ho . "The Cloze Procedure: How it Predicts Comprehension and Intelligence of Military Personnel," Technical Mpmorandum No. 13, Urbana, Illinois: Division of Communicaflons, University of Illinois, (1953). My Weekly Reader, January 18-22, 1960, l-h. “Pr *M‘P‘ .- .‘1 I ROOM USE ON Ll" .- , ~ 7 , t“ “b . . . “1mg \J‘ut F ' W" 5 ~‘.—.‘ ‘-¢ @ ' .1. w.tfi'¥mké:f_-de_‘_ . MICHIGAN STATE UNIVERSITY LIB III! II III RIIR'ES 0 O 3062 054 3 1293