' '7 ~ A v - fl.mmm"—~“w——vww—uw“r—w——'iv- —-~— . - . v v — _ v -, I - -‘ o‘lu‘. | . THE BIASIING EFFECT OF VISUAL CUES ' IN LANGUAGE EVALUATIONS‘ I Thesis for the Degree of M; A. MICHIGAN STATE UNIVERSITY . KATHRYN ZIMMERMAN HARLTON 1972 , - o _. . .. , - . ‘. “ ‘ . .‘.— .‘O‘ Q I . - § . 5.. '9 . . . . «r ‘. . . . .-‘ 4 e‘. I . . .;_.. o. I ' Q - . , - “.1 .l-.' .‘L ("a d -. _- d‘. I.-" -‘O-O so ..a' C1. 3,-~. ,- Q ‘ y". ' g' C!- v O. «f.- L‘k .3- '.'. .:. ... .. ------------ I .F J. llllll lllllllll L 088 0841 llllllllllll 30 ‘lllllllllll 31 ll _._ é a a: ‘ “3 i If I 1 k 0:me ’ ~ ’. "V W - “’ ‘3' BINDING BY HUM: & SUNS BMW LIBRARY BINDERS “Illa-1 man-In “I 75" J MSU RETURNING MATERIALS: Place in book drop to ' {‘1' 131,59 LJBRARJES remove this checkout from “ your Y‘GCOY‘d. FINES W111 be charged if book is returned after the date stamped be1ow. Y5?1v?' ' ‘4’.’§‘: ABSTRACT THE BIASING EFFECT OF VISUAL CUES IN LANGUAGE EVALUATIONS BY Kathryn Zimmerman Harlton The phenomenon of experimenter bias has been recognized and studied by investigators in both the physical and social sciences. Experimenter bias has been defined as the influence pre-information has upon the eXperimenter's evaluation of a subject's present per- formance. This bias has been shown to exist in the fields of Educa- tion, Sociology and Psychology, in both laboratory and non-laboratory environments. However, there has been little investigation of the phenomenon of experimenter bias in the areas of speech pathology and audiology. The purpose of this study was to investigate the possible existence of experimenter bias in the evaluation of children's langu- age samples, utilizing visual and auditory pre-information. The evaluators used in this study were fifteen audiologists and fifteen speech pathologists, all of whom were master 's level graduate students. The evaluators were required to make subjective evaluations of a ten-minute pre-recorded language sample of a seven year old normal speaking male child. The subjective measures consisted of two types of rating scales. The first type required rating four language attributes (structural sophistication, grammatical accuracy, creativity. and content) on a seven-point continuum. The second type used five descriptive attributes (like-dislike, mature-immature, good-bad. pleasant-unpleasant, and intelligible-unintelligible) on a seven-point continuum. The thirty evaluators were assigned to three ten-member experi- mental conditions, (five audiologists and five Speech pathologists). These three conditions differed in the type of pre-information and the attached photograph (visual cue) which the evaluator was given prior to listening to the speaker. The types of pre-information were termed: negative information (NI), positive information (PI), and lack of informa— tion (LI). The pre-information consisted of a folder containing contrived descriptions and information on behavior and intellectual achievement similar to the type that is found in clinical files. The photograph of the child was attached to the folder. All of the evaluators performed their tasks individually under controlled experimental conditions. The results of the study revealed no strong biasing effects among the audiologists and speech pathologists. However, it was found that the audiologists tended to be influenced more tlmn speech pathologists, and that this influence was in the predicted direction for both scales. The audiologists tended to rate the speaker better under the positive infor- mation condition. The speech pathologists, on the other hand, tended to rate the speaker better under the negative information condition. If the situation were reversed however, and the speech pathologists were engaged in an audiological task, it can be hypothesized that the speech pathologists would tend to perform the task in a predicted direction. At this point in the investigation of experimenter bias with regard to speech pathology and audiology it can be speculated that case history information does not have an effect on the evaluator. However, sug- gestions are presented for future research studies which involve direct interaction between the evaluator and the child in order to discover possible subtle biases occurring in speech pathology and audiology. G //’7 754/ THE BIASING EFFECT OF VISUAL CUES IN LANGUAGE EVALUATIONS By Kathryn Zimmerman Harlton A THESIS Submitted to Michigan State University in partial fulfillment of the requirements for the degree of MASTER OF ARTS Department of Audiology and Speech Sciences 1972 Accepted by the faculty of the Department of Audiology and Speech Sciences, College of Communication Arts, Michigan State University, in partial fulfillment of the requirements for the Master of Arts degree. —_.,..—. . Q?Z\24é/%‘L all”.-. Director of Thesis pee . ommittee: ii ACKNOWLEDGMENTS I would like to thank Dr. Daniel S. Beasley, my thesis director, and Dr. William F. Rintelmann and Dr. Leo V. Deal, my committee members, for their kind assistance and helpful suggestions. ‘ In addition, I would like to extend my grateful appreciation to my husband, Bruce and my parents Mr. and Mrs. John L. Zimmer- man for their kind encouragement, thoughtfulness, and support. The interest and support of Mr. and Mrs. William F. Harlton, Jr. is also much appreciated. Finally, I would like to thank Kay Kiomento for all her time and effort, and a special thanks to Pat Larkins for her encouragement. thoughtfulness, and friendship. iii TABLE OF CONTENTS ACKNOWLEDGMENTS .............................. . ........ iii LIST OF TABLES .......................... . ..... . . . . . . ..... vi CHAPTER I. INTRODUCTION ..... . .................................. l Experimenter Bias . . . . ...... . .......... . ................ l Experimenter Bias in Speech Pathology and Audiology. . . . . . . 8 StatementoftheProblem................. .............. . 15 II. EXPERIMENTAL PROCEDURES. . . . . . . . .......... . . . . . . . . 1? Speaker . ........................... . . .................. l7 Evaluators ..... . . .......... . . ...... . . . . ................ 18 Experimental Conditions ......... . . . . . . . . . ........... . . . . 18 Procedures.............. ..... ......... 20 Analysis . . . ......... . . . .......... . . .................... 21 III. RESULTS . . . . .................. . . . . .................... 22 Language Scales ................... . . ................... 23 Overall Effect of Biasing Pre-Information ............ 23 Effect of Major and Pre-Information by Scale ......... 23 DescriptiveScales....... ....... ................. 27 Overall Effect of Biasing Pre-Information ............ 27 Effect of Major and Pro-Information by Scale . . . . .' . . . . 27 Evaluator Listening Time ....... . ........ . . .......... . . . . 28 iv IV. LIST OF REFERENCES. ............. . . . . . . . ..... . ...... . . . . Summary ............................................ . . DISCUSSION ........................................... Clinical Implications .......... . ..... . .................. Implications for Future Research . . . . . . ............ . ..... APPENDIX A. B. CASE REPORTS I. NEGATIVE INFORMATION ........................... II. POSITIVE INFORMATION. . . . . . ..... . ................ III. LACK OF INFORMATION . . . . . . ..... . ........... . . . . . LANGUAGE EVALUATION SCALE INCLUDING INSTRUC- TIONS00.000.00.00...OOOOOOOOOOOOOOOOOO. DESCRIPTIVESCALE ....... RAWDATA........ ........... 31 34 40 42 45 49 51 53 54 57 59 TABLE LIST OF TABLES Page Mean score data and standard deviations for each scale for the first Subjective class of measurement (structural sophistication, gram- matical accuracy, creativity, and content) for audiologists, Speech pathologists, and all sub- jects combined. . . ............. . . . . . ............ 25 Mean score data and standard deviations for each scale in the second Subjective class of measurement (like-dislike, mature-immature . good-bad. pleasant-unpleasant, intelligible- unintelligible) for audiologists. speech patho- logists and all subjects combined ......... . ...... 29 Mean time for listening to and evaluating taped speech and standard deviations for each condi- tion of pre-information (negative. positive. and lack of information) for audiologists. speech pathologists and all subjects combined ....... . . . . . 32 vi CHAPTER I INTRODUCTION - Rosenthal (1966) has defined experimenter bias as the extent to which experimenter effect or error is asymmetrically distributed about the "correct" or "true" value. It is a measure of net error, and gener— ally refers to the preconceived ideas the experimenter has about his subject and his subject's performance. For the past decade investiga- tors in the physical and social sciences have studied various aspects of the concept of experimenter bias. Johnson (1953) states that "Our assumptions define and limit what we see, i. e. , we tend to see things in such a way that they will fit in with our assumptions. even if this involves omissions or distortions. " (p. 79) Experimenter Bias When studying experimenter effect and experimenter bias, it is necessary to consider several distinguishing concepts. Rosenthal (1966) defined experimenter effect as the extent to which the data obtained by an experimenter deviate from the "correct" value. The measure of experi- menter effect or experimenter error is some function of the sum of the absolute deviations of that experimenter '8 data about the "correct" value. Thus, experimenter effect is a rsfection of "the degree to which the tester influences a person's test results." (Hipskind and Rintelmann. 1969, p. 298). This influence can be subdivided into active and passive effects. Active effects are those associated with unintended differences in the experimenter's behavior that can influence the behavior of the subject. Passive effects, on the other hand, are associated with the experimen- ter's appearance rather than his behavior. Rosenthal (1966) stated that distinguishing between active and passive effects is very difficult. and there have been no experiments reported to aid in making this distinc- tion. In addition to the above can be added the consideration of the "self- fulfilling prophesies" and "demand characteristics" associated with ex- perimenter effects. The concept of the "self-fulfilling prophecy" was developed by Merton (1948) and reflects upon a person's expectation of an event. This expectation changes the behavior of the person in such a way as to make the prophesied event more likely to occur. The reality of the self-fulfilling prophecy has been shown in education in the form of the effect of teacher expectancy of a child's intellectual performance on that child's actual performance. MacKinnon (1962), as reported by Rosenthal (1966), feels that if it is expected that a child of a certain in— telligence will not respond positively to a given task and if this expecta- tion is known to him. the probability that he will respond positively is greatly reduced. "Demand characteristics" refer to the evaluator's expectations or biases as conveyed to the subject by certain cues which shape the subject's behavior (Orne, 1962). Orne has demonstrated that a variety of experi- ments are performed by subjects in a manner that they believe they should perform. If a subject believes that catalepsy of the dominant hand will result when hypnotized, then such a reaction will occur when hypnosis is performed. If they are not led to believe that catalepsy is a part of hypnosis, they will not show this reaction when hypnotized (Orne, 1969). While self-fulfilling prOphesies deal with expectations of a subject's behavior by an experimenter, demand characteristics deal with the sub- ject's expectations of himself and the experimenter. Thus, the subject may feel obligated to be a good subject and please the experimenter. Orne (1962) suggested that the subject may feel he must "validate the experimental hypothesis" as revealed to him previously by the experi- menter. Beez (1968) suggested the possibility that "demand character- istics" may unintentionally reveal to the subject just what the experi- menter is investigating and what he expects from his subject. One area in which experimenter bias has been studied is the field of animal psychology. Rosenthal and Fode (1963) performed a study in which they labeled an experimenter's rat as "bright" or "dull. " The actual performance of the rat when tunneling a T-maze was affected when the rat was tagged with a particular label. The experimenters who believed their rats were "bright" found that their rats performed better and improved daily. The other group of experimenters who belier their rats to be "dull" did not see the same results. Cardaro and Ison (1963) studied the turning behavior of planaria. They asked the experimenters to watch for the turning. The experimenters who were told to expect more turning by their planaria recorded more turn- ing. Inadvertent recording errors by the observer (Rosenthal, 1970) could be one possible explanation for these observations. Another explanation might be unintentional conditioning of the animals by the experimenter. Several investigators have studied the presence of experimenter bias, in human behavior research, including how such bias would affect the results of the study, and how this bias could influence the reactions of the subjects. Rosenthal and Fode (1963) required subjects to rate photographs of a person's face on a ten point scale of success to failure. The experimenters were told that they would either obtain high "success" ratings or low "failure" ratings from their subjects. The authors reported that experimenters expecting success ratings from their subjects obtained higher mean ratings than evaluators who expected failure ratings. Associated with the bias presented by the experimenter's precon- ceptions of the results he will obtain are the factors of age, sex, anxiety level and need for approval, of both the experimenter and the subject. The closer the experimenter and the subject are in age, the more influ- ence the experimenter has over his subject's results (Rosenthal, 1966). Further, female subjects rated photographs in a person-perception task as less successful persons when the experimenter was a female rather than a male (Rosenthal, 1966). Experimenters lower in need for approval obtained ratings of the photos as successful peOple. In another study involving person-perception task it was found that persons with high anxiety obtained higher ratings of success of the photographs (Rosenthal. 1966), although these results have been questioned by other investigations (Rosenthal, Kohn, Greenfield, and Carota, 1965). The less anxious experimenters obtained higher ratings of the success of the photographs. Experimenter bias may be conveyed through both visual and audi— tory cues. Rosenthal (1970) reports a study by Adair and Epstein (1968) involving the person-perception task. The study was conducted in two stages. In the first part of the study the subjects both heard and saw the evaluator. In the second stage, the subjects simply heard a tape record- ing of the experimenter's instructions. It was anticipated by the authors that only the visual session would reveal any form of bias. However. even when the subject simply heard the tape recorded instructions, an experimenter effect was evident. Also, there was a greater biasing effect for the nonvisual condition when the experimenter expected lower ratings. In another study of the person-perception task (Rosenthal and Fode. 1963) three conditions were utilized: visual and auditory, visual only, and auditory only. For the first condition the subjects both heard and saw the experimenter. Four groups involving experimenter-subject interaction were formed in the second condition. Group I subjects saw and heard the experimenter. The experimenter had been led to expect low ratings for all groups. Group II subjects were separated from the experimenter by a screen. A nonverbal condition was employed in group three: the subjects read the instructions while the experimenter remained silent. In Group IV the experimenters were led to expect high ratings. This group used both visual and auditory cues. The results indicated that the nonvisual group ratings came between the ratings for Group I and Group IV. The nonverbal group obtained ratings almost the same as for Group I. The authors concluded that with visual cues removed the bias was diminished. The loss of verbal cues obtained an effect similar to the non-visual group in the study conducted by Adair and Epstein. Everyday situations have also prompted researchers to delve into the effects of evaluator bias and study how these biases are related to these life situations. For example, evaluator bias was investigated in the classroom. The teacher was considered the experimenter and the students were the subjects. A study was conducted to test the effect of teacher's self-fulfilling prophesies and to study how these prophesies affected the behavior of children from a low socioeconomic environment. (Rosenthal and Jacobson, 1968). At the beginning of the school year, the children were given The m '_I'£s_t o_f General Abilities, which con- sists of measures of verbal and reasoning skills. The teachers were told that this was a test of "intellectual blooming" and that some stu- dents could be expected to make high gains. Eight months later the same test was re-administered. The results showed the children who were expected to gain in intelligence did so, and they gained more than a control group of children. The control group of children were not expected to gain in intelligence. The most affected area seemed to be the lower grades. Evans and Rosenthal (1969) replicated this study, only this time they used middle-class children. The results in this study were similar to those in the previous study. In both studies it was found that the experimenter effect was more prevalent on the reasoning tasks than on the verbal tasks. Rosenthal and Evans felt that the expectations of the teacher might have served as an unintentional self-fulfilling pro- phecy. The effect of "demand characteristics" on Headstart children was studied by Beez (1968). Each experimenter was instructed to work indivi— dually with a child. They were told that the purpose of the study was to see how the child would perform on various experimental tasks. Before coming in contact with the child, the experimenter was given hypothetical data about the child. The data included background information, testing behav- ior, and results and interpretation of test data concerning the child. Re- sults showed that experimenters who had favorable expectations for their child tried to teach more symbols to the child. They also rated him higher on achievement, social competence, and intellectual ability. Further. mean number of symbols learned was higher for children who were expected to achieve . Thus. the reality of experimenter bias phenomena in psychology and education. in general, has been demonstrated. It is presumed that such effects likely exist in the areas of speech pathology and audiology and require investigative consideration. Experimenter Bias in Speech Pathology and Audiology The audiology profession has been concerned with the possible biasing effects associated with previous knowledge of audiograms and scoring errors. Hipskind and Rintelmann (1969) conducted a study in which they investigated tester bias in relation to pure tone and speech reception thresholds and speech discrimination scores. Efforts were made to induce tester bias. The examiners were given one of four types of pre-information. This pre-information contained an audiogram with the subject's actual threshold, his threshold increased 10 dB, his thres- hold decreased 10 dB, or no information pertaining to the subject's threshold. There was no effect associated with the type of pre-information. Nelson and Chaiklin (1970) compared two methods of scoring and scoring bias for speech discrimination testing. A write-down method involved the subject's writing down the word as he heard it. A talkback method involved the subject's repeating the word to the examiner. The examiner recorded the word as correct or incorrect. The authors felt that there might be an increased possibility of scoring bias in the talk- back method. It was felt that the examiner might be questionable. Re- sults showed that only inexperienced examiners revealed a significant difference between scores for the two scoring methods. There have been few studies conducted in the field of speech patho- logy relative to experimenter bias. One of the major concerns is the evaluation of speech, hearing and language behavior. In this situation the experimenter bias may be more appropriately termed evaluator bias. An example of a problem in speech pathology as related to evaluator bias is that of the language of the inner-city black child. The language spoken by the inner-city black child has been of recent concern to the speech pathologist. There has been controversy as to whether or not to teach "standard English" to these minority group children. There are two vieWpoints characteristic of the term "nonstandard English. " One view is that the child's speech is deficient relative to the norm. The child's language is termed a restricted code, thus restricting the child in cognitive skills. The other vieWpoint sees the child's language as simply different from "standard English. " The child's language would be considered appropriate for his particular culture and environment. Williams (1970) felt that these "disadvantaged" children are cate- gorized into stereotypic groups. That is, the listener stereotypes the child into a certain group from his own personal reactions to the child's speech. The language evaluator, as a listener, generalizes his reactions to the child's speech in a manner reflective of a negative "self-fulfilling prophecy. " The evaluator expects a certain pattern of speech to be used by the speaker, and any language patterns that do not coincide with the prophecy can be considered incorrect by the evaluator at the outset. 10 Siegel stated that: . . .ignorance of the areas of speech pathologies and language deviations may constitute an experimental safeguard against particular biases or expectations. The experimenter who is unaware that cleft palate speakers generally have a particular pattern of speech disturbances. or that certain sounds are usually mas- tered before others may be less likely to impose these expectations on the tests he administers. (1962. p. 34) Unfortunately, the nature of speech pathology is such that the speech pathologist. as an evaluator of behavior, enters the testing situation with several possible biases. The fact that clients are referred to the speech pathologist immediately implies a pathology on the part of the client and is a possible source of evaluator bias (Manning and Beasley. 1971). Back- ground information usually accompanies the referral of the client. The speech pathologist tries to become familiar with this information before seeing the client. The real danger of bias may appear when this back- ground information includes a label such as "cleft palate, " "mentally retarded," "brain damaged" (Manning and Beasley. 1971). Siegel (1963) conducted a study dealing with interpersonal inter- action of adults with retarded children. The adults were informed that the children either had high verbal skills or low verbal skills. The adults were assigned to teach a particular task to the children. The results indicated tint the adults did not respond differently to the child- ren with regard to the labels given the children. Siegel concluded that 11 the labels "did not significantly affect adult responses to these children" (p. 422). A study conducted by Meitus, Ringel, House, and Hotchkiss (1972) investigated the effect of clinician bias with regard to evaluating the articulation behavior of children. Thirty clinicians viewed case pre- sentations via videotape. Prior to the evaluation, the cliniCians were given fictional case histories containing positive—bias, negative-bias, or no information. The clinicians rated phonetic inventories, judged the pro- ficiency of articulation, and provided a prognosis and therapeutic judg— ment. The results indicated no significant differences in scores for all three groups. The authors feel that because the responses under the posi- tive bias and the negative bias agree strongly, case histories are used for other reasons than determining therapy and prognosis. Cowen, Weber, Hoddinott, and Klein (1967) conducted a study involv- ing Mean Length of Response (MLR) as a function of stimulus, experimen- ter and subject. They tested ninety-six school age children from both lower and upper socioeconomic levels. The MLR measures were obtained by two of the authors. It was found that certain stimuli elicited a greater length of response. The sex, socioeconomic level, and age of the sub- jects interacted with the experimenter effect and the experimenter. The differences between the two experimenters' scoring and recording methods also contributed to the experimenter effect. The authors concluded that the Mean Length of Response varied with the stimulus used. the age. sex. 12 and socioeconomic level of the subject and the experimenter doing the evaluation. A study involving evaluator bias and children's language perfor- mance examined objective as well as subjective measures (Manning and Beasley, 1971). The study was conducted in such a manner as to find out whether or not Speech pathologists are influenced by information they receive before beginning to test a child's language abilities. The study consisted of judgments of tape recordings of two speakers. Forty evaluators were randomly assigned to one of four conditions of biasing information: negative information, positive information, incomplete information, and lack of information. There were two classes of measurement: 1) objective, consisting of three forms, Mean Length of Response (MLR), Mean of Five Longest Responses (M5LR), and Type-Token Ratio (TTR); and 2) subjective, consisting of four seven- point rating scales representing a different aspect of language, struc- tural sophistication, grammatical accuracy, creativity, and content. These subjective scales were taken from Elliot, Hirsh and Simmons (1967). The speakers were two white male first grade children. They were considered to be of normal intelligence with no significant speech pathologies. The evaluators were instructed to listen to a tape record- ing of the two children and obtain a score for the several measures utilized. The results of the study did not indicate any significant biasing 13 effects for the four types of pre-information given to the evaluators. Also, no significant differences were revealed between the two speakers between the conditions of pre—information for the various classes of measurement (Manning and Beasley, 1971). The authors provided several explanations as to the reason the Speech pathologists were not biased. One was that masters' students were relatively resistent to evaluator bias. They are taught to form their own Opinions about a client and not to do so before that client's language had been assessed by them personally. Another possibility could have been that the children were not seen personally by the evaluators, thus providing the opportunity for the evaluators to inter- act on a personal level with the children. Further, evaluators had no visual cues. Visual cues or auditory cues are independently sufficient to establish evaluator bias. However. studies have shown that a bias is more likely to occur when both visual and auditory cues are used simultaneously (Rosenthal and Fode, 1963; Adair and Epstein, 1967). If visual cues were employed, the likelihood of the occurrence of a biasing effect would have been increased. If a subjective scale of measurement was employed that required the evaluator to indicate his own personal preference as to how he genu- inely liked the Speech of the child, there might have been a more accurate measure of how the evaluator truly felt about the child's language and speech. This rating scale could have been employed along with the l4 Elliot, Hirsh and Simmons (1967) scale. While the one set of scales could reflect upon the structural sophistication, grammatical accuracy, content, and creativity, the other scale could have been used for measur- ing the evaluators personal preference for the child's Speech and langu- age. The semantic differentiai type scales devised by Osgood, Suci, and Tannanbaun (1957) may satisfy the above problem, since they reflect upon the listener's subjective reaction to the child's Speech and language behav- ior. That is. they do not measure the stimulus per se, as do the Elliot et a1. scales, but rather people's reactions to that stimulus. Subjective measures in the case of the study by Manning and Beasley (1971) may be defined as descriptions of a Speaker's language. The scale usually consists of five to seven points between two opposite terms. This type of scale was used by Elliot, Hirsh and Simmons (1967) in their inves- tigation of the language of deaf children. These rating scales can be con- sidered to coincide with the person-perception tasks mentioned on the preceeding pages. Rating scales can be considered to be subjective measures because they depend heavily on the rater's perceptions of the Speaker's language (Manning and Beasley, 1971). The possible effects of evaluator bias may be reflected in the amount of time devoted to the listening of the language sample by the evaluator. That is. individuals who spend more time listening to the language sample may in fact be more cautious in their evaluations. If 15 so, then those individuals would be less likely to reveal biasing effects. Related to this may be the evaluator's major professional area of inter- est. Thus, audiologists, who are trained to evaluate a different upset of language behavior from speech pathologists, may very well reveal different results on an evaluator bias task than the speech pathologists. Statement of the Problem The physical and behavioral sciences have acknowledged and stud- ied the various aspects of experimenter bias and experimenter effect. Investigators in audiology and speech pathology (Hipskind and Rintelmann. 1969; Nelson and Chaiklin, 1970; Williams, 1970; Meitus et a1. , 1972; Seigel, 1963; Manning and Beasley, 1971) have conducted studies involving the possibility of evaluator bias with regard to their respective professions. However, further research needs to be conducted to determine what possible effects, if any, evaluator bias has in the evaluation of the language skills of children. The purpose of this study is to determine whether an element of evaluator bias does exist in reference to subjective measures of a child's language performance with the aid of pre—biasing information and visual cues. Specifically, the following questions will be investigated: 1. Would speech pathologists and/or audiologists be influenced by infor- mation obtained prior to a language evaluation ? Would the appearance of the child. as reflected by photographs, have any biasing effect on the evaluation of the child's language performance ? 2. If the evaluator bias occurs. will it be more likely to occur because 16 the evaluator had a prior look at the child's picture and had a history of background information about the child or would it occur more readily if the evaluator had no prior information about the child but simply had a taped language sample to evaluate ? 3. Would there be any major differences in the evaluations of the Speech pathologists as Opposed to the evaluations of the audiologists ? CHAPTER II EXPERIMENTAL PROCEDURES This study consisted of judgments of tape recordings of one Speaker by thirty experimenters, fifteen audiologists and fifteen Speech pathologists. who were randomly assigned to one of three conditions of pre—information, within each sub-group (speech pathologists and audio- logists). Each condition was comprised of four forms of subjective measures of language. Each condition was represented by one of three types of pre-information intending to bias the experimenter. Visual cues were employed in the form of photographs. Sgaker The Speaker, used in an earlier study (Manning and Beasley. 1971), was a white male first grade child, age seven years-zero months, from an elementary school in Lansing, Michigan. The child was of normal intelligence according to school records and teacher's reports. The public school speech pathologist reported that his Speech was not characterized by any significant misarticulations. The Speaker was asked to tell a story about each of the pictures in the Children's Ammeption 1e_s_t (Bellak, 1954). The story was recorded on a Panasonic Model RQ7OGS tape deck. The recording was obtained by a female Speech pathologist, who was uninformed as to the 17 18 purpose of the study. Her instructions were to obtain a ten minute langu— age sample from the Speaker. The Speaker and the Speech pathologist were the only ones present while the recording was being taped. Evaluators The language evaluators were fifteen master's students in audiology and fifteen master's students in speech pathology from a large midwestem university. Before the evaluators listened to the tape recording of the Speaker, they were asked to read information about the child contained in folders. Ten evaluators (five from each major) were given folders con- taining negative information, ten were given positive information folders to read, and ten were given a folder with no information. Experimental Conditions There were three conditions of pre-information used to bias the evaluators. The conditions were "negative information" (NI), "positive information" (PI), and "lack of information" (LI). The pre-information folders contained hypothetical "case reports" similar to ones found in a speech and hearing clinic or public school. The "lack of information" folder had only the child's age and a folder number. In the "negative information" and the "positive information" conditions, the folders included a folder number, age, and grade level. In addition to these factors, the negative and positive conditions also contained the social history of the child and his family, various measures of intelligence and achievement. and a photograph purported to be of the child in question. 19 The negative information folder contained a photograph of a child with gross malformations of the facial region. The positive information folder contained a photograph of a normal, "average looking" child. The lack of information folder contained neither picture nor information. The reports were placed in a manila folder and given to the evaluators just before they listened to a tape recording of the normal speaker. An example of the case report for the negative condition may be found in Appendix A-I, the positive condition Appendix A-II, and the lack of infor- mation condition Appendix A-III. There were two scale classes of subjective measurement. The first class, used by Manning and Beasley (1971), was comprised of four seven- point rating scales. Each scale represented a different aspect of language. These scales were taken from Elliot, Hirsh and Simmons (1967) and included "structural saphistication, " "grammatical accuracy, " "creativity, " and "content." (See Appendix B). The second class of subjective measurement was comprised of five seven-point semantic differential type rating scales: like-dislike; mature- immature; good-bad; pleasant-unpleasant; and intelligible-unintelligible. (See Appendix C). These scales were of a more general nature than the first set of scales. Directions for rating the speaker indicated that the evaluator should listen very closely to the speaker and rate him as scour- ately as possible according to the stipulations on the rating sheets. The 20 evaluators were reminded that this would be their own personal judgment as to how the speaker's language and speech appealed to him. The experiment was designed so that ten evaluators were randomly assigned to one of each of the three conditions of pre-information. The folders were coded by number as to the condition of pre-information they contained. The code was not familiar to the evaluator, thus minimizing the likelihood of the evaluator affecting the results. Procedures The evaluator was seated at a table in a small, quiet room. A tape recorder was placed on the table in front of the evaluator so that he or she could have easy access to it. The evaluator was instructed on how to use the tape recorder. The evaluator was then given the following printed instructions to read silently while the investigator read them orally: The purpose of this study is to obtain various ratings for each of four basic aSpects of langu- age and five measures as to the quality of langu- age. You will receive instructions for obtaining these ratings now. You will hear a tape record- ing of a child's Speech. Before listening to this tape recording you will be given some information to familiarize you with the child. Please read this carefully. After you have read the folder. you will listen to the tape recording for as long as you wish. After you have listened to the tape. please fill in the rating scales as accurately as possible. The purpose of the study in which you are participating requires that you do pgt discuss the folders or tape recording with anyone. Thank you for your cooperation. 21 The evaluator was allowed to ask further procedural questions if he so desired. Finally, a folder was given to the evaluator. The evalua— tor was allowed to read the folder for as long as he wished. After he read the folder, he listened to the tape. At this time, the evaluator was timed as to how long he listened to the tape recording. This time was recorded next to the evaluator's name and type of information given him to read. When the evaluator completed the rating scales, he was again reminded not to discuss the task with anyone. Analysis The investigator recorded all data (evaluator scores and time) by hand. These data are shown in Appendix D. The mean score for each type of pre-information in each scale were recorded and a standard deviation for each mean was also obtained. CHAPTER III RESULTS Fifteen audiologists and fifteen Speech pathologists were required to make subjective judgments of a tape recorded Speech and language sample of a single child. There were two scale classes of subjective measurement. The first class was comprised of four seven—point rating scales. Each scale represented a different aSpect of language. The scales were taken from Elliot et a1. (1967). The second class of subjec- tive measurement was comprised of five, seven-point rating scales. The evaluators were randomly assigned to one of three conditions of pre-information: negative information, positive information, and lack of information. Visual cues were employed in the form of photographs. intending to bias the evaluator. The purpose of this study was to determine whether eXperimenter bias could exist in evaluating subjective measures of a child's language performance with the aid of pre-biasing information and visual cues. In addition, an attempt was made to determine whether one group (audio— logists or speech pathologists) would be more influenced by biasing pre- information than the other. 22 23 Table 1 consists of mean scores and the standard deviations of these scores with regard to the Elliot et al. , (1967) subjective language scales. Table 2 consists of standard deviations and mean scores of the five seven-point subjective scales. Table 3 depicts a mean listening duration and standard deviation in relation to the evaluator 's major and condtion of pre-biasing information. am we: Overall effect of biasing pre-information. --Reference to Table 1 indi- cates that overall there were minimal differences between biasing con- ditions. However, there was a trend for the negative pre-information condition to be rated higher, i. e. , better (xr-5. 85), than the other two conditions. This is followed by the positive pre-information (;=5. 75) and lack of information (£5. 65) conditions. There is a trend that suggests that audiologists were more influenced than Speech pathologists. This influence is particularly evident for Creativity where audiologists rated the positive pre-information x=6. 40. Effect of major and pre-information by scale. -—Table 1 reveals the effect of pre-information according to the evaluator's major area of interest and type of pro-information. Overall, subjects tended to show minimal differences between classes of pre-information (mean scores range from 5.40 to 5. 50) for the structural sophistication scale. Speech pathologists tended to rate the Speaker slightly higher than did audiologists under all 24 three types of pre-information. However, between types of pre- information minimal differences existed. Relative to grammatical accuracy, audiologists tended to evaluate the Speaker highest with positive pre-information (x=6. 40), followed by negative pre-information (it-=5. 80) and lack of information (2E5. 20). respectively. The speech pathologists rated the Speaker for all pre- information conditions equally (;=6. 20). Creativity scores were rated similarly for speech pathologists who had negative pre-information and lack of information folders (St-=6. 20 and x=6. 00, reSpectively). The score for positive pre-information was lower (;=5. 20). Audiologists, however, tended to rate positive information higher than negative information, (;=6.40 and i=5. 60, reSpectively). Lack of information was rated slightly lower than negative information (SI-=5. 20). Speech pathologists obtained equal scores for the positive pre-information and lack of information conditions for the Content scale (32:5. 80), while negative pre-information was rated higher (it-=6. 00). The audiologists obtained the same mean score ('x=6. 00) for negative pre-information as the speech pathologists. However, lack of informa- tion was rated lower (;=5. 80), followed by positive information (32:5. 20) for the audiologists. The standard deviation scores obtained for both speech pathologists and audiologists tended to be small. However. Speech pathologists standard deviations were smaller than standard deviation scores for audiologists. These minimal differences in the mm $4 :4 2... a: as as... $5 84 «as 35 S... $5 8338 23:3 3.... SS 35 8.... 86 is one 85 31m one 3.6 one :82 scanned? em... 84 $5 3... 3; $5 $5 $5 $5 $5 a: :5 accuses 233m em.» 86 2...... 36 owe cue Ste 3.5 $3 3;.“ 83 86 :82 393.5% :8on a: a: an... e: a; a; 2... $3 a: 84 $5 :2 Season Egan ES one Ste SS 8.» owe 33 SS .23 8.5 as 85 8oz 338863. 4.3. E E E 4.3. E E E .34 E E E 555.420 Shampoos. zoae