REDUNDANCY EN SIMULTANEOUSLY ‘ PRESENTED AUNO-VESUAL MESSAGE ELEMENTS AS A DETEKMENANT OF RECALL Than ‘or “10 Degree, of DH. D. WCHEGAN SEATE UMYERSITY Thomas F. Baldwin 1966 III IIIIII III III III IIII IIII II III IIII III IIII IIIIIIIII I/ _ If: :3 :I 1173 Universit A. Y A” THESIS This is to certify that the thesis entitled "‘ _,,“ Redundancy in Simultaneously Presented Audio-Visual Message I Elements as a Determinafit’ of Recall presented by Thomas F. Baldwin has been accepted towards fulfilhnent of the requirements for Ph .D . degree in Communication Major professor I 1/28/66‘ Date 0-169 w. -- 4.; .. .— 4...»...— -'~ -—-—-—r-....._. ABSTRACT REDUNDANCY IN SIMULTANEOUSLY PRESENTED AUDIO-VISUAL MESSAGE ELEMENTS AS A DETERMINANT OF RECALL by Thomas F. Baldwin The presumed communicative efficacy of words and pictures does not always hold in practice. Travers tenta— ‘ tively suggests that for almost all instructional circum— stances and purposes, single channel communication is most appropriate. Others have discovered evidence that an interference may develop between picture and word negating the desired response to the message. There is also the suggestion, by Broadbent, of a natural limitation on the cognitive capacity to receive information from two sensory channels simultaneously. Broadbent proposes an internal system for screening the mass input which arrives at the periphery of the organism so that information is transmitted selectively and in orderly fashion to the central nervous system. The interior "filtering" system accounts for the reduction of the mass input. The selected information then travels through the cognitive utilization channel successively; the information from only one channel at a time. However, if the utilization Thomas F. Baldwin -2- channel is clear at any time, other information may be drawn from a short term storage system. This is how information coming simultaneously from two sensory channels may enter the utilization system. It is hypothesized that, if message elements arriving simultaneously from audio and visual channels must enter the cognitive utilization system successively, the more the message can be "compressed," the greater the likelihood that the entire message will be processed. Sequential redundancy, as described by Shannon and Weaver, and synchronous redundancy, as described by Osgood and Sebeok, would represent an index of "compressibility." An attempt was made to measure redundancy in the audio elements of a motion picture film clip, in the visual elements and in the relationship between the two. Each of these measures was based on a separate group of subjects. A fourth group saw and heard the entire message. Immediate recall of both audio and visual elements was recorded. The audio ele- ment was defined as a simple sentence; the visual element as a single camera "shot." Audio and visual elements were pre- cisely opposed to form audio-visual units. Recall responses by element and by unit were correlated with the redundancy indices for the unit. The following specific relationships were hypothesized: Thomas F. Baldwin -3- 1) The recall of the audio element in a message where audio and visual elements are presented simultaneously is positively related to audio redundancy. 2) The recall of the visual element in a message where audio and visual elements are presented simultaneously is positively related to visual redundancy. 5) The recall of both audio and visual elements which are presented simultaneously in a message is positively re- lated to audio redundancy. 4) The recall of both audio and visual elements which are presented simultaneously in a message is positively re- lated to visual redundancy. 5) The recall of both audio and visual elements which are presented simultaneously in a message is positively re- lated to redundancy in the audio-visual relationship. 6) The recall of both audio and visual elements which are presented simultaneously in a message is best predicted by a multiple regression equation based on audio redundancy, visual redundancy and redundancy in the audio—visual relation- ship. All results were in the hypothesized direction. While significance levels were not predetermined, for the relation- ships hypothesized in two and five above, p < .05. REDUNDANCY IN SIMULTANEOUSLY PRESENTED AUDIO-VISUAL MESSAGE ELEMENTS AS A DETERMINANT OF RECALL BY ,s - \I'" 03‘ Thomas FT Baldwin A THESIS Submitted to Michigan State University in partial fulfillment of the requirements for the degree of DOCTOR OF PHILOSOPHY Department of Communication 1966 ACKNOWLEDGMENTS I am deeply indebted to my entire guidance committee for the design and conduct of a graduate program which nurtured some of the questions raised in this thesis and provided the necessary knowledge and encouragement to begin exploration: Dr. David K. Berlo, Dr. Charles A. Blackman, Dr. Randall P. Harrison, Dr. Colby Lewis, Dr. Charles F. Schuller and Dr. Verling C. Troldahl. Special thanks are due Dr. Harrison for his direction of the work. His great interdisciplinary depth in the literature of visual communication has been invaluable; his imaginative suggestions, helpful at every point; his patience, essential. To Janice, who sustained the effort throughout, and to Andrea, who virtually grew up with the project, I cannot hope to eXpress my gratitude. ii Chapter II. III. IV. THE THE TABLE OF CONTENTS PROBLEM AND RELATED RESEARCH . . . . . . Introduction . . . . . . . . . . . . . . The Problem. . . . . . . . . . . . . . Related Theory and Research. . . . . . . HYPOTHESES . . . . . . . . . . . . . . . Redundancy . . . . . . . . . . . . . . Redundancy in the Single Channel Perceptual System. . . . . . . . . . . Hypotheses . . . . . . . . . . . . . . . METHODOLOGY. . . . . . . . . . . . . . . . Materials and Subjects . . . . . . Recall . . . . . . . . . . . Audio Redundancy . . . . . . . Visual Redundancy. . . . . . . Audio-Visual Redundancy. . . . Analysis . . . . . . . . . . . FINDINGS . . . . . . . . . . . . . . . . . . The Recall Patterns. . . . . . . . . . . Audio Redundancy . . . . . . . . . . . . Visual Redundancy. . . . . . . . . . . . Audio-Visual Redundancy. . . . Relationship Between Redundancy and Recall The Prediction of Maximum Recall . . . . CONCLUSION . . . . . . . . . . . . . . . . . Hypothesis 1: Audio Recall and Audio Redundancy . . . . . . . . . . . . . . Hypothesis 2: Visual Recall and Visual Redundancy . . . . . . . . . . . . . . Hypothesis 3: Maximum Recall and Audio Redundancy . . . . . . . . . . . . . . Hypothesis 4: Maximum Recall and Visual Redundancy . . . . . . . . . . . . . . Hypothesis 5: Maximum Recall and Audio- Visual Redundancy. . . . . . . . . . . Hypothesis 6: The Prediction of Maximum Recall . . . . . . . . . . . . . . . . iii Page \INH 19 19 25 29 32 32 34 38 40 43- 44 45 45 48 50 50 51 55 56 S6 57 58 58 59 60 Chapter TABLE OF CONTENTS (cont.) The Findings and the Broadbent Construct Implications . Summary. APPENDICES . REFERENCES'. iv 118 10. ll. 12. 13. LIST OF TABLES Recall Patterns by Audio-visual Unit for all 69 subjects. . . . . . . . . . . . . . . Recall Patterns by Audio-visual Unit for 41 Subjects with the Highest Loading on the Principal Response Factor. . . . . . . . . . . Redundancy Index for Words and Sentences Constituting the Audio Elements. . . . . . . Visual Redundancy Index for Each of the 10 Visual Elements . . . . . . . . . . . . . . Redundancy in the Audio-visual Relationship for the 10 Audio-visual Units. . . . . . . . . Product Moment Correlations of Audio Redundancy with Recall Patterns for 69 Subjects . . . . . Product Moment Correlations of Audio Redundancy for 41 Subjects. . . . . . . . . . . . . . . . Product Moment Correlations of Visual Redun- dancy with Recall Patterns for 69 Subjects . Product Moment Correlations of Visual Redun- dancy with Recall Patterns for 41 Subjects . Product Moment Correlations of Audio-visual Redundancy with Recall Patterns for 69 Subjects . . . . . . . . . . . . . . . . . . . Product Moment Correlations of Audio-visual Redundancy with Recall Patterns for 41 subjeCtS O O O O O O O O O O O O O O O O O O 0 Multiple Correlation Coefficient for Audio Redundancy, Visual Redundancy and Audio-visual Redundancy with Recall of both Elements (high first factor subjects) . . . . . . . . . . . . Beta Weights and Partial Correlation Coeffi- cients for Audio Redundancy, Visual Redundancy and Audio-Visual Redundancy with Recall of Both Elements (high first factor subjects) . . 46 49 51 52 53 53 53 54 S4 54 55 55 Figure 1. LIST OF FIGURES Broadbent Model of the Information Trans- mission Process, as Modified by Travers. vi Appendix A. D. E. F. G. LIST OF APPENDICES Recall of Audio and Visual Elements in Newsfilm . . . . . . . . . . . . . . . Recall of Audio and Visual Elements in Educational Film . . . . . . . . . . . Correlations of Message Certainty with Recall . . . . . . . . . . . . . . . . The Experimental Film. . . . . . . . . Recall Instrument. . . . . . . . . . . Recall Score Sheet . . . . . . . . . . Audio Redundancy Instrument. . . . . . Visual Redundancy Instrument . . . . . Audio-Visual Redundancy Instrument . . Principal Factor Loadings. . . . . . . Recall Curves. . . . . . . . . . . . . Redundancy Curves. . . . . . . . . . . vii Page 66 71 73 76 78 83 85 97 101 108 110 115 Chapter I THE PROBLEM AND RELATED RESEARCH Introduction Modern electronic media such as television and sound motion picture films have the potential for transmitting complex audio and visual message elements simultaneously through multiple sensory channels. The audiovisualist, the television journalist, advertising art director, film maker, and others concerned with pictorial presentation in one form or another, are prone to believe that this word-picture complex communicates more effectively than words alone. But the presumed communicative efficacy of words and pictures does not always hold in practice. In some cases, under some conditions, a single channel is more effective. On the basis of a comprehensive sur- vey of the literature and initial exploratory studies of multiple channel communication, Travers tentatively sug- gests that for almost all instructional circumstances and purposes, single channel communication is most approPriate. (33) Others have discovered evidence that an interfer- ence may deve10p between picture and word negating the desired reSponse to the message. There is also the sug- gestion of a natural limitation on the cognitive capacity to receive information from two sensory channels simul- taneously. Seldom in audio-visual research, however, are the elements of audio-visual communication isolated to deter- mine their relationship to the general effect. It is the purpose of this study to examine the major components-- the audio and the visual elements--and the synchronous relationship between the components. More Specifically, to study a characteristic (redundancy) which may be deter- mined for each of the two components and for the synchronous relationship between the components, and relate this char- acteristic to one kind of effect (recall). The Problem There is a long tradition of intuitive, empirical and theoretical support for the superiority of word-picture communication. Some of this support will be presented here, briefly, as well as a more lengthy treatment of the exceptions. The latter are less commonly discussed. In Volume I, Number 1 of the Psychological Review (1894), Munsterberg concludes a series of experiments by saying: "A series of presentations offered to two senses at the same time is much more easily reproduced than if given only to sight or only to hearing.“ (23:37) Communication practitioners have also emphasized the power of the word-picture combination. In 1922, Walter Lippmann wrote: ". . . it is probably true that . . . no idea is lucid for practical decision until it has visual or tactile value." (17:162) Wilson Hicks says: “Pictures and words together perform a more effective function than either can perform alone.“ (1:122) An audio-visual text- book suggests that “. . . words with pictures are usually better than either words or pictures alone." (34:75). Two theories in the audio-visual literature support these conclusions. The first is the cue summation theory which suggests that the stimulus cues in each channel are additive in their effect. It is stated somewhat more formally by Miller, §£_al.: "When the cues elicit the same reSponses simultaneously, or different responses in proper succession, they should summate to yield increased effectiveness." (20:78) A second theoretical position attributes a reinforce- ment effect to related stimuli presented simultaneously in two channels. Gr0pper illustrates with this example: The demonstration of a solid object expand— ing as it is being heated can serve to cue or prompt the verbal response, "expands." A second use of the demonstration might in- volve asking the student, before the heat is actually applied, to predict what would occur if heat were applied to the object. The visually perceived expansion following the application of heat would serve to pro- vide reinforcement or confirmation for the student. It would indicate to him that his reSponse, "it will expand," was correct. Its function here is similar to that provided by verbally telling the student "it expands" or by verbally telling him that he is cor- rect. (11:77) When the audio and visual channels are closely related, each presents cues which elicit reSponses that are con- firmed (reinforced) almost immediately in the other chan- nel. The learning theories of Hull, Skinner and others consider reinforcement essential to learning. (15) Skinner, in particular, has demonstrated that immediate confirma- tory reinforcement enhances learning. (16) Phrased in another way by May and Lumsdaine the theory states: ". . . non-verbal stimuli . . . when combined with the verbal explanations [form a] total pattern [which] has more power than words alone to evoke the desired responses. The non-verbal stimuli serve as reinforcers in the Pavlovian sense of reinforcement.“ (18:166) Unfortunately, audio and visual cues do not always summate or reinforce each other to provide the desired effect in terms of the various learning criteria. Both the cue summation and reinforcement theories acknowledge that cues in each of the two channels may interfere with each other. Miller, §£_gl;, note that contrary to a summation effect: "When the cues elicit incompatible responses, they should produce conflict and interference." (20:78) Gr0pper suggests that under certain conditions reinforce- ment is not likely: "When more than one reSponse is under the control of a stimulus, the more likely is less effi- cient learning to occur as the result of reSponse inter- ference or competition." (11:80) Interference has been observed in some of the appli- cations of multiple channel communication in educational and informational media. Lumsdaine and Gladstone used two sound filmstrips to teach the old military phonetic alphabet. One presentation was "fancy"-—“cartoon illus- trations accompanied by musical or other sound effects and/or by humorous comments by the narrator." This ver- sion might be considered to have a high level of informa- tion compared to the "simple" production which included only the printed words on a neutral background as the narrator pronounced them. Learning occurred as a result of both productions, but the "fancy“ presentation was less effective. The attention gaining devices included in the "fancy" version did in fact draw attention, but this interfered with the efficiency of the learning. (18) In a film study, Neu developed four variations of a "clear, straight-forward" instructional film: 1) visual attention-getting devices, relevant, 2) visual devices, irrelevant, 3) auditory attention devices, relevant, and 4) auditory devices, irrelevant. The relevancy of an attention—getting device in either channel was determined by its relationship to the intended message. Neu found some evidence that the basic film was most effective. For some of his pOpulations the basic version, with no Special devices to attract attention, was accompanied by the highest scores. The relevant and irrelevant atten- tion devices added to the information level of the message but not to the effectiveness of the film. (25) In another film study, Zuckerman experimented with levels of information in the sound track of a film find- ing that ". . . verbalization may be increased to a point where it interferes with, and actually reduces, learning." (36) Deutschmann, Barrow and McMillan measured the learning of both relevant and irrelevant information from film and "natural laboratory" presentations. They assumed that the more confined film presentation would be most efficient. "Channel efficiency" scores were determined for each sub- ject by deducting the percentage of correct irrelevant items from the percentage of correct relevant items. Channel efficiency scores were higher for the film pre- sentation, as predicted, suggesting to the authors that there were in fact interferences in the natural labora- tory which could be screened out by the motion picture camera's frame. (9) Here again, is an indication that a limitation on the information level of the message enhances learning. In an attempt to estimate the relative influence of pictorial elements on interest and learning in television news, Hazard analyzed the three components of most news- casts: l) man—on—camera, 2) still photographs, and 3) newsfilm. A measure of information gain established no significant difference between the three types of presen- tation, but the data disclose the highest gains in in- formation for the man-on—camera unaided by still or motion pictures. (14) A series of three experiments was undertaken by Hart- man to assess the effect of multiple channel communication (audio, print, pictorial). Over—all, the presentations including the pictorial channel were least effective. In discussion of the first experiment, which provided the best Opportunity to check interference, Hartman states: "If the relation between the pictorial and verbal chan- nels had been one of non-interference, the addition of the pictorial channels to the presentation would neither add nor detract, and the results should be essentially similar to those obtained with the verbal presentations." (13:33) Instead, learning was at the lowest level when the picture channel was added. Why do we find evidence of interference? Related Theory and Research A plausible explanation of interference, or the dif- ficulty sometimes experienced in attempting to simultaneously use multiple channels for communication, is offered in an interesting historical account by Sumstine, writing in 1918: Several years ago, it was customary to have lecturers describe the motion pictures in the theaters; but at present they have dis- continued. Several managers of motion pic- ture theaters were asked the reasons for such discontinuance. Each one gave the same answer. The patrons objected to the combination, saying that they could not look and listen at the same time. (29:237) There is evidence in psychological research that sup- ports the finding of the movie theater owners. Audio- visual information is received in separate sensory chan— nels and must be processed by the organism. Whether two sensory channels, activated simultaneously, have access to the central cognitive system at the same time, is an essentially relevant question which is not usually asked. It is generally assumed that both audio and visual elements of film or television messages are received and utilized. Recent studies, however, have cast some doubt on this assumption. Reviewing the Mowbray and Broadbent experiments to be discussed below, Berlyne notes one of the reasons for interference. He calls it occlusion: . . . an organism can do only a limited num- ber of things at once, even of things no two of which are mutually exclusive. The capa- city of the brain, therefore, must be setting limits to the efficiency with which informa- tion from two simultaneously active sources is utilized. (2:13) -9- A tradition of research has focused on the possibil- ity of the division, or Splitting, of attention between simultaneous perceptual tasks. Division of attention might more appropriately be called division of attention in cognition, to indicate that attention is measured by some cognitive act related to the exposure to particular stimuli. The general conclusion is that the division of attention seems to be nearly impossible. (35) Where ex- perimental subjects appear able to divide attention, it is assumed that an essentially unitary attention has been alternated between the two perceptual tasks. In this tradition, Mowbray's research is most recent and most relevant to the Simultaneous audio-visual pre- sentation of information. By presenting different prose passages simultaneously in auditory and visual channels, Mowbray found that reSponses to subsequent questions on the passages did not rise above a chance level for one of the channels, except for the condition of easy passages in both channels. He concludes that: . . . as alternation of attention becomes increasingly more difficult, the possibility of performing adequately both of two simul- taneous perceptual tasks becomes more remote. This evidence tends to support the thesis that attention is unitary, at least with reSpect to complex, cognitive acts . . .(21:37l) Because of the timing of the visual presentation, sub- jects reported being aware of attending to the audio pas- sage after assimilating the exposed visual passage. -10- Apparently there was enough time to attempt to shift at- tention prior to exposure of the next visual element. An important point is that the subjects attempted to per- ceive the material successively rather than Simultaneously. Another eXperiment by Mowbray further reduced the possibility of alternation of attention. (22) This time subjects were presented two different pairs of map coor- dinates simultaneously, one pair in the auditory mode and one pair in the visual mode. They were to locate details on maps using the pairs of coordinates. The Simultane- ously presented pairs of different coordinates were ran- domly imbedded within irrelevant words which were the same in both channels. All audio elements were paced to match the visual elements. Half of the subjects were instructed that each set of relevant coordinates would be presented simultaneously withanother set of relevant coordinates. The other subjects were "uninstructed." Subjects were encouraged to guess at the locations if they had any rea- sonable basis. The analysis indicated that in a few cases both sets of coordinates were used correctly. These, however, were below the theoretical frequencies expected by chance using the most conservative estimate of chance levels. The "instructed" group made no more successful completions of both simultaneously presented locations than the "uninstructed" group. Among the "uninstructed," no subject was able to tell the experimenter the Special -11- simultaneous condition of the relevant items. Almost all denied being aware of it, and many were vehement in their disbelief. Again, Mowbray concludes that Simultaneous multiple-channel perception is not possible: "All Of the results from this experiment suggest that complex per— ceptions involving language cannot be affected by differ- ent sensory modes at the same time." (22:92) Two demonstrations of this result also appear in pilot studies by the present author. In the first study, each Of a series of four television news stories on film were projected for subjects. Immediately after viewing, subjects were asked to write down as much Of the visual and audio information as they could recall. TO analyze these reSponseS, the stories were broken down into the simultaneously Opposed audio and visual elementS--short statements of fact in the sound track and scenes in the visual portion. For all four stories, when the percent- age Of subjects recalling an audio element was high the percentage recalling its Opposing visual element was very low. When subjects recalled visual elements, audio recall was low. There seemed to be an alternation of attention between the audio and visual elements. This held for individual subjects as well as the group. (See Appendix A.) The same experiment was attempted with a short film clip adapted from an instructional film produced by walt -12- Disney. (30) Again, there was a marked alternation in recall between Opposing audio and visual elements. (See Appendix B.) Recall of one was high at the expense Of the other, in most cases, suggesting Mowbray's conclusion that complex perceptions cannot be affected by different sensory modes simultaneously utilized. These findings indicate that, at least for complex materials, attention seems to be unitary. Broadbent, however, has demonstrated the human capacity tO utilize two sets Of information delivered simultaneously to two sensory channels. At the same time he agrees that atten— tion (in cognition) is unitary. Presenting two different sets of digits to each ear simultaneously, Broadbent found subjects able to repeat all Of the digits. One ear would receive digits such as 4-2-7, and the other 8-6-3. The "natural reSponse“ was to report 4-2-7, 8-6-3 (or 8-6-3, 4—2-7). Subjects never reported the simultaneous pairs in temporal sequence as they were transmitted, i.e., 8-4, 6-2, 3-7. In fact, when subjects were told to reSpond in this latter order, they had great difficulty, some finding it impossible. TO eXplain these findings Broadbent proposes that some mechanism must Operate to receive items simultaneously, with still another mechanism Operating which will only handle items successively. The former he calls the S system and the latter, the P system. The S system receives -13- all the sensory input to the organism, but the ultimate perceptual processing system (P), which is unitary, admits information only from a single channel at a time. Other information may be held in storage in the S system, how- ever, for brief periods. The dual system accounts for the ability Of subjects to reSpond to Simultaneous inputs of different information and also the successive manner of that reSponse. (7) The duration of the storage period was tested in another experiment. (4) In this instance, the P system was kept occupied for various periods Of time after the arrival Of information in another channel. A stream Of six digits would be presented to one ear. Two different digits would be presented simultaneously to the other ear with the first two of the six. For another set of Six digits, the second ear would receive different digits simul— taneously with the third and fourth. In the final condi- tion, the two simultaneous digits would be paired with the last two digits in the stream Of six. Subjects were told to repeat first the six, then the two. This could be done quite well for the last condition, but much less well for the conditions where the two were presented ear- lier. Additionally, the latter Of the two digits was more likely to be recalled than the first--it had been in storage for a shorter time. Broadbent concludes that ". . . if the two digits were in the S system for more than a second or -14- so before their admission to the P system they were less efficiently recalled." (5:227) TO be sure that the capacity for storage was not pe— culiar to the auditory channels, Broadbent designed another eXperiment using the audio and visual channels. (6) This, of course, is of particular significance in the present context. In one condition, two different digits were presented simultaneously, one to the right ear and the other to the eyes. After the third such pair of digits, the subject was to write them down in any order. In another condi- tion, subjects received three digits successively in the right ear as before, but the visual digits were exposed all at once, appearing as soon as the first aural digit and disappearing after the third. Subjects were able to correctly recall the digits from both channels fairly consistently: as well or better than control Situations where the digits were presented binaurally, successively in the visual channel and suc- cessively in the auditory channel. This establishes that the capacity for reception of simultaneous information is not Specific tO the binaural condition. And, it was again found to hold that all information from one channel was reproduced before the other. This was the case in 157 Of 160 correct reSponseS. Again we have the suggestion by Broadbent that "adequate reSponse to simultaneous stimuli -15- was achieved by delaying all the information on one chan- nel." (6:147) It is interesting to note that while sub— jects showed some tendency to consistently report from one sensory channel first, they divided equally in this preference with half reporting visual information first, the other half favoring auditory information. To review, Broadbent proposes an internal system for screening the mass input which arrives at the periphery Of the organism so that information is transmitted selec- tively and in orderly fashion to the central nervous system. The interior "filtering" system, as Broadbent describes the P and S systems working together, accounts for the reduction Of the mass input. The selected information travels through the final system single file, one might say, or successively: the information from only one chan— nel at a time. However, if the single channel is clear at any time, information may be drawn from the Short term storage system. This is how information coming Simultan— eously from the two sensory channels appears to enter the P system. According tO Broadbent, the storage system is only Operative when there is an overload of information. Storage is not necessary when information arrives success- ively. On the other hand, if the main channel is occupied with the input Of one sensory channel beyond the maximum storage time Of two or three seconds, the information in storage from another channel is lost. -l6- Travers has used the Broadbent constructs to schema- ticize the audio-visual information transmission process and to more explicitly indicate what he calls information "compression." (See Figure 1.) "Compression" refers to any process within the ner- vous system which reduces redundancy. (33:523) At the outset, naturally occurring events and some "precompressed," or precoded, stimuli are received at the receptor level whereupon it is compressed by the sense organs and the peripheral nervous system. For instance, visual material is compressed to the determining boundaries which suffi- ciently carry the message. At another level, compression takes place through categorization. This process is de— fined as the "rendering of discriminably different stimuli as equivalent." (33:408) One would assume that the system generalizes stimuli where certain discriminations do not seem to be important or necessary. Travers also uses the term "categorizing" interchangeably with "coding." The latter is considered in information theory terms as a "reduction in the number of Signals needed to relay a given amount of information." (33:409) In other terms, the process might be called "recoding" or "chunking." Miller defines the process as organizing or grouping input into familiar units or chunks. The input is given in a code that contains many chunks with few bits per chunk. The Operator recodes the input into another code -17- ANo.mummv .MHO>MH.H. >9 OOHMHUOE mm .mmwooum Gowmmflfimcmuu GOHUMEH0msH map mo HOUOE usmncmoumlla .mflm mmmuOum Ammmuoum msfluasmmu SDAB HMHHODME Omnoum £DH3 musmsfl mo SOHDMNAHOOODMO smsonnu SOHmmOHmEOO a .SOflyomumusH _ usmuso " Emumhm Emflcmnomz \fi coaumNHHHu: mama mcapumamm Hmccmnu mamcflm was t mSHHOuHGOS mcflnoumz monoumm mum mo mmomuu msHsEflum muss nuns Ummmmumfiou Emummm mso>nms Hmumnmflnmm can mammuo mmsmm on» an coflmmmumfioo A coaum8u0msfl pmmmmnmeoomnm AI AI musw>m mcflunsouo \ SHHMHSDMZ AIHO>OH noummo [on um usmcH —18- that contains fewer chunks with more bits per chunk. There are many ways to do this recoding, but probably the Simplest is to group the input events, apply a new name to the group, and then remember the new name rather than the original input events. (33:109) Returning to the Travers schematic, the compressed "stimulus traces," or chunks, then enter a monitoring or selecting mechanism in the schematic. This mechanism prevents the utilization channel from.becoming overloaded. Otherwise the utilization system would become jammed with information. The function is tO limit the "input Of inform— ation SO that only manageable quantities have access to the higher centers." (33:9.08) This corresponds to Broad- bent's filtering concept. Information may be stored at this point for later selection, providing the stimulus "trace" is selected within two or three seconds. The schematic suggests an answer to one Of the ques- tions we have raised—-why does interference occur. In the process, we also answer another question which is raised by the answer to the first--why does interference not always occur. In other words, we may infer, from the schematic, conditions under which audio-visual communi— cation may sometimes be cognitively received and other times not received. Chapter II THE HYPOTHESES The model of the single channel perceptual system seems to suggest that redundancy is a major factor in the capacity of the system to receive information simul- taneously from the audio and visual elements of a multiple channel message. Redundancy may be in the audio element, in the visual element and in the relationship between the audio and visual elements. ,In this chapter we will: 1) discuss the forms of redundancy in motion pictures and television, 2) relate redundancy to the Broadbent-Travers concept Of the single channel perceptual system, and 3) put forth a set of hypotheses based on the apparent func- tion Of redundancy in the perceptual system. Redundancy In their paper on synchronic psycholinguistics, Osgood and Sebeok discuss redundancy within audio and visual channels, and between the channels. (27) While their terms relate Specifically to inter-personal communication, they seem apprOpriate, with some adaptation, to other forms.of communication. Osgood and Sebeok identify bands over which Imessages travel synchronously; the vocal-auditory band, -19- -20- the visual-gestural band and the manipulational-Situational band. In audio—visual communication, certain elements Of the interpersonal vocal-auditory band may be missing be— cause Of the technical limits of the recording and repro- ducing equipment. Distortion, or missing frequencies, however, if detectable, are not often critical. The vocal- auditory band, then, is comparable to the audio element of the audio-visual message. The visual-gestural band is sometimes present in audio-visual communication, as in man-on—camera presenta- tions. Again, the fidelity may suffer somewhat in the reproduction processes, but the result still has most of the characteristics Of the interpersonal form. The manipulational-Situational band is construed by Osgood and Sebeok to include graphic communication, which, one could assume, would include photographic or electronic representation. This band is comparable to the most common type of visual element in audio-visual communication. Osgood and Sebeok refer to the relationship between the vocal-auditory, visual-gestural and manipulational- Situational bands as between band organization. The authors acknowledge the evidence for seqpential redundancy for both audio and visual communication. This is related in the literature Of information theory and lsinesics. But they also propose a synchronic redundancy, -21- within and between bands. The notion of sequential redundancy between parts Of a message as serially unreeled is now a fairly common one, particularly as a result of the work of Shannon, Miller, and others. The notion that there can also be synchronic redundancy among simultaneous events within the same band or between bands is less familiar but equally reasonable. Both linguists and information theorists have taken cognizance of redundancies with- in the linguistic band per se, the former Observing that phonemes are for the most part overdetermined (in terms Of clusters Of correlated features) and the latter re- porting that one can experimentally cut out 50 percent or more of the total information in the auditory channel without seriously hampering intelligibility. There is also redundancy between discretely and continu- ously coded Signals in the vocal—auditory band--witness how stress is typically ac- companied by lengthening Of vowels, how stress and raised pitch tend to go together, and so forth. (27:88) These authors do not discuss redundancy in the mani- pulational—Situational band, but Osgood, in a later article, (26:28) notes the identification by Harrison and MacLean of synchronic redundancy in graphic facial representation. The latter used cartoon faces with Simple features as stimuli and asked subjects tO relate descriptive adjectives to the faces. Such features as medially upturned brows and down—curved mouth, by themselves, tended to elicit the same responses. Combining the features does not bring significant new reSponses. "When combined, the elements tend to elicit the same responses again, only more strongly.“ (12:15) -22- This is an example of synchronic redundancy within the manipulational—situational band. It deals with static visual content which is present in audio-visual communica- tion even if our unit of analysis is the "shot" in motion pictures or television. Another form of redundancy in the static visual pat- tern is described bypgarner: Redundancy means that fewer patterns exist than could exist, and when we say that a particular finite sequence is redundant we I are simply saying that this sequence is one I Of many which could occur, but that the I number which could occur is far less than the total number of such sequences which could occur if there were no redundancy. Thus a unique pattern is judged to be re— dundant because it seems to be one of a I smaller class of patterns than does the I figure judged non-redundant, irregular, com— I plex, etc. . . . What I am suggesting, in i summary, is that a judgment of the redun- dancy of a unique figure (or judgment of other characteristics presumably related to redundancy) is not really a judgment of the redundancy of the figure itself (which is a meaningless judgment) but is, rather, a judgment of the redundancy Of the particular . subset of figures from which the unique \ figure is presumed to have come. (10:203-205) Garner uses the example of a vertical line which would be judged more redundant than a slanting line, because there are a smaller number of vertical lines possible than slanting lines. He would also consider the Square more redundant than the trapezoid because there are more ways to construct a trapezoid than a square. Redundancy in this sense may be related to the in- finitely more complex static picture. If the picture -23- represents a Specific instance of a small subset of ex- periences, it is more redundant than the picture which represents a more general set of visual eXperiences. This is another way of viewing its level of abstraction. In addition to synchronous redundancy in a static Situation within a motion picture or television shot, we must also contend with sequential redundancy. Usually some action occurs. The probabilities Of subsequent ele- ments in an action, once it has begun, are contingent upon the character of the action. Once the bow has been drawn, the Shot Of the arrow would likely be redundant. In the foregoing, we have discussed sequential and synchronic redundancy in the audio and visual elements (or bands) of the audio-visual message, but not redundancy in the audio and visual relationship--the between band organization. Osgood and Sebeok say: "Redundancy between bands, e.g., between vocal—auditory and gestural-visual bands, has been for the most part neglected. . ." (27:88) They identify, however, from "informal Observation," the extremes in redundancy between bands: (1) synchronic complementation, the usual situation in which gestural signals have the same significance as vocal signals and hence complement one another; (2) synchronic contrast, the more informa- tional situation in which gestural and vocal Signals have different (usually Op- posed) significance and hence change each other in some fashion. (27:88) -24- The redundant case, synchronic complementation, is illustrated by the vocal “No" and the Shaking of the head. Closer to our interest in audio-visual film or television communication is the example of synchronic redundancy between the manipulational-situational and the vocal- auditory bands, e.g., diagramming or doodling on a pad to facilitate communication. The Simplest case of non-redundant synchronic con- trast might be illustrated by the vocal "No" and the head nodding up and down. More commonly synchronic contrast is found in sarcasm, where the gestural-visual band denies the vocal-auditory band. Osgood and Sebeok suggest that at another level, synchronic contrast may be represented by an irrelevant relationship between two bands. Birdwhistell calls this form Of contrast "meta- incongruence," which occurs “when an utterance has one contextual meaning while the accompanying body motion conveys a different meaning (for example, the actors in television commercials, while verbally praising a product to the skies, can sometimes be seen rejecting it in their body motions)." (3:15) While an irrelevant or incongruent relationship is not common and even difficult to achieve in the interper- sonal communication situation, it is a very likely occur- rence in audio—visual communication where the relation- Efllip is constructed rather than Spontaneous. -25- In this section we have attempted to describe the various forms of redundancy in audio-visual communication, in particular, motion picture films and television--redun- dancy in the audio channel, in the visual channel and redundancy in the relationship between the two. The task remains to relate redundancy to the model of the single channel perception system. Redundancy and the Single Channel Perception System As Travers uses the concept Of compression in the model of the information transmission process, the more redundant the message the greater the compressability Of the message elements. The more the message may be compressed, the more it is likely to stay within the capa- city Of a single channel utilization system. Compressed information from one channel will pass through the utili- zation channel more rapidly. Other information, simul- taneously received from another channel which has been stored briefly, will then more likely gain access to the utilization channel before it has dissipated. It will be recalled that according to Broadbent's experimental findings, the storage life is very brief, a matter of one or two seconds. This application of the single channel model, with .its storage system, was tested in the previously-mentioned -26- pilot study by the present author using an adaptation of a Short film clip from a Walt Disney educational pro- duction (30). The visual elements in this analysis were scenes or "shots." The audio elements were Opposing state- ments in the sound track. One group of subjects was asked to judge each of the visual elements in terms Of the sub- jective degree of certainty about the content of the elements. A five-point scale with the polar terms "cer- tain” and "uncertain" was used for this purpose. The same scale was used with another group of subjects to determine the relative "certainty" of each of the audio elements. On the same scale, a third group judged the relationship between the two channels. It was assumed that degree Of perceived "certainty" would be a function of the degree of redundancy. The scale should reflect degrees on a continuum between cer— tainty and entrOpy. The measures Of redundancy were subsequently cor- related with measures Of immediate recall for both audio and visual elements taken on still another group. In terms of the Broadbent and Travers models of the perceptual system, it was predicted that in the case of high visual redundancy, the system would be more likely to accommodate the visual element, since the element could be compressed or coded. It would also be more likely for lxoth simultaneous audio and visual elements to be recalled .yv- tp! 5.1 I. “ll id§ F1 F ”fin u 0" -27- since the visual compression would allow more time for the momentarily stored audio information to be brought into the utilization channel (P), if this were the order of entry. If the audio elements were to enter the utili- zation channel first, the compressed visual information would be more likely to fit into any remaining time. The same rationale could be presented for the pre- diction that high audio redundancy would be related to high recall and to the recall of both elements in a si- multaneously presented audio-visual unit. Finally, it was predicted that the cross-channel relationship would be positively related to the recall of both simultaneously presented elements. If the ele- ment in storage interacts with the element which has already passed into the utilization channel, then it is more likely to, itself, enter the channel. In other words, the stored element may become redundant or more certain by its rela— tionship to the other element which has already entered the cognitive system. An analysis Of the data from this pilot study indi- cates the strongest relationships are with visual redundancy and recall. Visual redundancy seems to be re- lated positively to the recall of both elements and to a lesser degree to the recall of visual elements. There is nc: clear relationship between audio redundancy and recall. -One reason for the failure to Obtain the predicted result. -28- here lies in the incapacity Of the instrument for discrim- ination between audio elements. Cross-channel relation- ships seem to be slightly related to the recall Of both channels as predicted. The relationship may have been stronger if the film had been presented to the subjects rather than a verbal description of the visual elements. (See Appendix C.) There are three difficulties in interpreting the data above. Two have already been mentioned. First, either the elements Of the audio channel did not vary enough in redundancy, or the instrument used to measure redundancy was not sensitive enough. Secondly, redundancy in the relationship between channels was not measured by using the original materials, but rather a printed verbal des- cription of the visual channel and a printed audio script. A better estimate of the relationship would have been obtained by using the same sensory channels as the ori- ginal presentation. Finally, it should be noted that there was some Op- portunity for alternation of attention between channels. For all visual elements, the Opposing audio element was slightly Shorter, by a second or two. Nevertheless, the empirical data from the pilot study lend some support to the contention that redundancy in tide visual channel and redundancy in the relationship between channels is related to recall. And, our theory -29- would predict the same effect for redundancy in the audio channel. Hypotheses This would lead us to hypothesize the following rela- tionships: l) The recall of the audio element in a message where audio and visual elements are presented simultaneously is positively related to audio redundancy. 2) The recall of the visual element in a message where audio and visual elements are presented simultan- eously is positively related to visual redundancy. These two hypotheses are derived from the theoretic model. If audio redundancy is high, the recall Of the audio element Should also be high, since the redundant element is more easily compressed. The same would hold for visual redundancy with visual recall. There is also a kind Of repetition inherent in redundancy which Should have the effect Of cue summation. In some studies, with unfamiliar forms and words, recall appears to be a func- tion Of frequency of exposure (8:252). 3) The recall of both audio and visual elements .which are_presented Simultaneously in a message is posi- tively related tO audio redundancy. -30- 4) The recall Of both audio and visual elements which are presented Simultaneously in a message is positively related to visual redundancy. According to the Broadbent theory as interpreted here, compression potential in one channel should increase the likelihood Of both elements in an audio-visual unit being recalled. The element may pass through the utilization channel more rapidly, thus permitting the second element to be called up from storage before it has dissipated. Or, if the element is selected for storage, it is, in its compressed form, more likely to fit into any remaining time after the Opposite element has passed the utiliza- tion channel. In this case the effect of repetition in gag element does not explain recall Of both elements. 5) The recall of both audio and visual elements which are presented Simultaneously in a message is positively related to redundancy in the audio-visual relationship. If the elements of a simultaneously presented audio— visual unit are related, or redundant, the element in storage may become compressed after the element which has passed into cognition since it is repetitious and only its unique characteristics need enter the utilization chan- nel. This latter process would be Shorter in duration, with greater Opportunity for any non-redundant information ' V . . A: 9. _31- in the element to enter the utilization channel prior to dissipation or the next input. 6) The recall Of both audio and visual elements which are presented Simultaneously in a message is best predicted by a multiple regression equation based on audio redundancy, visual redundancy and redundancy in the audio-visual rela- tionship. It is assumed that audio redundancy, visual redundancy, and audio-visual redundancy are independent and therefore not correlated by unit. Together, the three Should each contribute independently to the prediction of the recall of both elements in a given unit. The terms of these hypotheses will be defined Opera- tionally in the chapter that follows. Chapter III METHODOLOGY TO provide evidence for the validity Of the hypothe— sized relationships, it is first necessary to identify meaningful audio and visual elements. These must be pre- cisely Opposed temporally to form the audio-visual units. Recall Of the elements Of the message must be recorded. Indices Of visual redundancy, audio redundancy and redun- dancy in the audio-visual relationship for each of the elements and units must be obtained. These indices may then be correlated with recall of the elements to indicate the degree Of relationship. Materials and Subjects An existent film was used as the basis for the audio- visual message. It was the same film clip as used in pilot studies mentioned in Chapters I and II. The clip was taken from a Walt Disney color production, Switzerland, designed for social studies units at the upper elementary and se- condary levels. The particular segment used described the cheesemaking procedures of a village cheesemaker in the Swiss mountains. Some rewriting and editing was necessary to make each -32- » ‘¢ 6.‘ (1' vi T -33- of the audio and visual elements match precisely. The final production was 40 seconds long with 10 scenes and 10 sentences, each about four seconds in length. (See verbal description Of scenes and the script, Appendix D.) Each visual element was a Single scene in the film. More technically, as defined by the University Film Pro- ducers Association, the visual element would be considered a "Shot"--the piece Of film resulting from a single run of the camera. (24:27) The Shots from the original film were edited precisely to the desired length by frame counts. Some Shots in the original sequence were not used. This did not disrupt the continuity, however. There were no titles or closing credits. The experimental content was preceded and followed by Opaque leader. The audio elements were Simple sentences, some with modifying phrases. The tape was narration-only with no background music or natural sound. The new sound track was recorded on magnetic tape in the Department of Tele- vision and Radio studios at Michigan State University. Technically, it would meet high standards. The narrator is the instructor of television and radio announcing at Michigan State who has had a number of years professional eXperience as an announcer. To project the film with sound in synchronization, the tape was cued to its starting point on a recorder. The prOper volume was preset. The recorder was started -34- at the moment the picture appeared on the screen. In all administrations, the synchronization was perfect. Since the film was designed for upper elementary and secondary levels, subjects were selected from within this range. They were high school students, mostly juniors, attending workshOpS in communication arts and in music at Michigan State University in the summer of 1966. Recall Seventy—four subjects were exposed to the film in synchronization with its narration. They were in separ- ate groups Of 30, 12 and 32. Subjects were told, as were all others participating in this project, that they were taking part in an ongoing research project of the College Of Communication Arts, Michigan State University. All materials were so labeled. NO other instructions were given at this time except to note that the film was only 40 seconds long and therefore it was necessary to pay close attention from the start. This was done in an attempt to have subjects immediately alert. All three groups viewed the film under approximately the same conditions, although at different times over a two-week period. The rooms were the same Size and darkened to about the same light level. With the projector and recorder located in the same place in each room, the image Size and sound level were nearly -35- identical in all rooms. There were no apparent distractions or disturbances during any showing, although in one room the screen had a small blemish in the lower right hand cor- ner which appeared as a black mark within the picture. Immediately after the film, subjects were given a booklet which included the instructions and three pages for the required reSponse. (See Appendix E.) The instruc- tions were read aloud as subjects read Silently. They were asked to write as much as they could recall from the narration and from the visual portion. The reSponse pages were divided down the middle and lined. The Space at the left was labeled for the visual, the right hand side was labeled for the narration. Subjects were told to write items down as they came to mind without regard to order or to the relationship between the visual and narration columns. Five subjects did not reSpond in the manner re- quested. The forms for these subjects were voided, leav- ing a total of 69. The responses to each audio and visual element for each subject were coded by the author in a binary system; l-recall, O-nO recall. (See recall score sheet, Appendix F.) If the "essence" of an audio or visual element was mentioned it was scored as recalled. The mention had to be Specific enough to be assigned to a particular element. Occasionally, a subject would be confused as to whether an item originated in the visual or the narration. In -36— this case the item was scored for its proper channel. In the great majority of cases, it was not difficult to code the recall reSponseS. The responses could be easily identified with their apprOpriate elements. In some instances, however, it was necessary to make a judg- ment as to whether enough information was included in a response to warrant coding it as recall. All reSponses were coded again by the author a week after the first coding to provide an indication Of the reliability Of the coding over time. There were 105 disagreements between the first coding and the second out of a possible 1380 (92.39 percent agreement). Each of the points Of dis- agreement was then reconciled. A substantial prOportion of these were errors in coding which simply required cor- rection on the final code Sheet. Others were the type where a somewhat arbitrary judgment was made. These were resolved after careful study of the reSponse in question. No index Of interjudge reliability was Obtained. While this would have been desirable, the labor involved for additional coders would have exceeded the limits of what one might ask. There was, however, no Opportunity for intentional or unintentional bias in favor of the hypo- theses sinceIQLL recall coding was done prior to the coding Of redundancy indices. It was assumed that there would be a Single predom- inant pattern of reSponse—-that subjects would tend to -37- uniformly recall certain audio and visual elements and not others. This assumption is indeed critical. If sub- jects responded randomly in entirely individual patterns, there would be no point in the analysis. A factor analysis of persons was therefore performed to determine a reSponse pattern. If a single or dominant pattern emerged, those subjects with the highest loadings on that factor would be the best indicators Of a general effect (of the message on recall). The deviants from the pattern, if there were not tOO many, could be eliminated as "bad" items would in an item analysis. Subjects might deviate from a general pattern for a number of reasons such as a peculiar attrac— tion to a particular element, momentary distractions, low level Of attention, etc. The following scores were Obtained from these data for all 69 subjects and also for the group with the highest factor loading on the dominant reSponse pattern: a. the total number of subjects for each of the 10 audio-visual units recalling bggh audio and visual elementS—-RECALL BOTH (audio=l, visual=l). b. the total number of subjects for each of the 10 audio-visual units recalling the audio element--AUDIO RECALL (audio=l, visual=l or 0). c. the total number of subjects for each of -38— the 10 audio—visual units recalling the visual element--VISUAL RECALL (visual=l, audio=l or 0). Audio Redundancy A device Similar to Taylor's cloze procedure was used to measure sequential redundancy in the audio ele— ments. The procedure involves systematic or random dele- tion or words in printed passages. Blank lines replace the deleted words. Subjects are asked to guess the miss- ing words. If a passage is redundant, the context may provide cues to the missing words. The more redundant a passage, the higher the probability of correct responses. (32) Since our purpose was to determine the relative re- dundancy Of audio elements--sentences--the usual cloze procedure could not be employed. Instead Of a fixed number of random deletions or a fixed ratio Of systematic dele- tions such as every fifth word, different subjects re- Sponded to each word Of the sentence, except the first. Another change in the usual form of cloze procedure was made to better accommodate the measure to audio pre— sentation. Only the words prior to the missing word were supplied to the subject. In cloze procedure, words that come after the blank also provide contextual cues. This -39- seemed more apprOpriate for self—paced reading material where the reader may scan the whole sentence and return to ponder the blank. The listener is not able to make as good use of after-the-blank contextual cues. Since the listener is forced to attend to an ongoing, externally— paced message, he is not as likely to be able to relate the after-the-blank context to the missing word. The pre- sent procedure of supplying only the words before the blank would also seem to better fit the concept of sequential probability or predictability which would be most crucial in a Single channel utilization system. With this procedure no measure was possible for the first word Of each sentence. At least the first word was necessary to provide a context. The longest sentence Of the 10 had 13 words which meant 12 deletions. It was arbitrarily decided to Obtain at least three responses for each word. This required a total Of 36 subjects. Each subject reSponded to one deletion in each of the 10 sentences. After all the required deletions had been made for a single sentence, they were randomly shuffled. All of the sentences were shuffled SO that each subject received deletions which were randomly located within the sentences. The word deleted in the first sentence for a given subject might be the seventh; the deletion in the second sentence for the same subject might be the third, etc. The deleted sentences appeared in the same order as -40_ in the film script, each printed on a separate page. In oral instructions, subjects were told that they were to guess at the next word in the portion of a sentence on each page. After a few seconds to make the guess, the complete sentence was read by the administrator. This was done so that subjects could take advantage of any contextual cues from previous sentences which might contribute tO the redundancy Of subsequent material. (See instructions and sample instrument, Appendix G.) The instruments were administered to two groups, one of 24 students and the other of 12, at different times in the same week. "Correct" reSponseS were the precise word, or a derivative thereof, used in the film script. Because some words in the Shorter sentences had more than three deletions, a fraction was computed for each word based on the number of correct reSponseS in relation to the number of possible correct reSponses. The fractions were added over all the deleted words in a sentence and divided by the number of words in the sentence less one. The resultant figure was used as an index Of redundancy for that sentence. Visual Redundancy Rating scales similar to those described for "visual certainty" in Chapter I were used to measure visual redun— dancy. It was established in the pilot study that subjects -41- could discriminate between film scenes on the basis of a five-point scale with poles labeled "certain“ and "un- certain." Subjects were instructed to rate a scene on the basis of the degree of certainty in their comprehension of the scene: CERTAIN - : : : UNCERTAIN. The first point was described as "certain, quite clear;" the second as "fairly certain, fairly clear;" the middle point as "not sure;" the fourth as "fairly uncertain;" and the fifth as "completely uncertain, unclear, confus- ing." (See Appendix H.) Each scene was projected on the screen followed by Opaque leader. They were Shown in the original order for the same reason that the audio elements were kept in the jproper order, SO that prior elements could contribute to the redundancy measures as the film progressed as would Ibe the case in a natural situation. The projector was stopped for 10 seconds after each scene. Subjects were to rate the scene during this period. Thirty—three sub- ;jects participated. The scales were coded 4, 3, 2, l, 0 from left to right, from certain to uncertain. The total Score over all subjects for each scene served as the index ‘tca its visual redundancy--the higher the score the higher ‘tlle redundancy. AS discussed in Chapter II, the "certainty“ scale I .~ n a a c has -42- is taken to represent a range between relative certainty and relative entrOpy. If one might think of the process of viewing a picture in two steps, the first would be an orientation response where the picture is scanned for familiar cues or points of reference. If there are enough of these, we might expect a second step, an identifying or recognition reSponse Of fairly high certainty. If the cues or referents are insufficient, unfamiliar or unclear, we might expect a longer orientation period and less cer- tainty in the identification or recognition reSponse. From the initial moment of the orienting reSponse, to the time the scene content is satisfactorily identified, uncertainty or entrOpy which originates in the orienting reSponse is being reduced. If the elements or actions in a scene are not successfully identified, the entrOpy originating in the orientation reSponse continues, or may even increase. The latter would be the case if incompat- ible or incongruent cues are discovered. For our purposes, it was assumed that if enough non- conflicting visual cues were presented in a shot to prO- ‘Vide some certainty in the identifying response, subjects “nauld be able to indicate this on a scale representing Cuertainty Of comprehension. Response to fewer, inadequate, Cir conflicting visual cues would register on the negative Stide of the same scale. Both sequential and synchronic IViadundancy would be reflected in the reSponse. -43- Audio-Visual Redundancy Redundancy in the audio—visual relationship for each audio-visual unit was determined by asking subjects to match the elements. The scenes were randomly ordered and separated by Opaque leader. A different random order was established for the audio elements. These were printed on a single sheet of paper. Each sentence was preceded by a blank line. SO that subjects would be thoroughly familiar with the sentences, they were read orally twice by the administrator while the subjects read along Silently. After a scene was projected it was numbered orally. The number was to be placed on the blank next to the sen- tence which most closely corresponded to the scene. Sub— jects were permitted to use a sentence more than once. They were told to guess if there were no Obvious choice. Thirty subjects participated. A slight revision was made for a second administration. The instructions were changed to permit subjects to use more than one sentence for a scene if others also were appropriate. This was to avoid the possibility Of some sentences being excluded because others fit about as well. The instruction read: ". . . write the [scene] number Opposite the sentence that most closely corresponds with that scene and any other sentence that might also fit the scene. You therefore might have a particular scene number -44- written in front of more than one sentence." Nineteen subjects participated in this administration. In both versions, subjects had 40 seconds to make their choices for each scene. At the end of this period they were warned to finish and look up at the screen for the next scene. (See Appendix I.) The correct matches, according to the original script, were totaled over all subjects for each audio-visual unit. The higher the score for a unit, the greater the redun- dancy in the relationship between the audio and visual elements. If subjects were able to see something in one element that correSponded to another, and these were the elements which were actually matched in the film, then there is some redundancy in the relationship. It was assumed that the less agreement (the fewer correct matches) among subjects, the less Obvious the relationship and therefore the less redundancy. Analysis The degree of relationship for each hypothesis was determined by computing Pearson product moment correlation coefficients. The predictability Of the recall of both Opposing elements waS determined by multiple correlation techniques using the three redundancy measures. Chapter IV FINDINGS The Recall Patterns The Object Of the factor analysis of recall responses was to find a single factor which represented the general over-all reSponse pattern of subjects. The principal axis analysis indicates a substantial first factor which accounts for 39 percent of the variance (Eigenvalue, 27.14; 69 variables). None Of the other factors accounts for more than eight percent. Most of the subjects had a high loading on the prin- cipal factor. An arbitrary point Of .60 was established. All those subjects (N=41) with a factor loading above .60 were grouped together for separate analysis. (See Appendix J.) The recall reSponses were computed for this group as well as for all 69 subjects. The computation of the various recall patterns for each audio-visual unit was Simply a matter of totaling the number Of subjects reSponding in the particular pat— tern. These are presented in Table l for all 69 subjects and in Table 2 for the factored group. AS would be expected, there is less deviation from the modal response pattern for each of the audio-visual -45- oo.ha hN.NH Hm mm MN ma m cN h m H ca Hamomu OHU5< Oh.HN N¢.ha He mm. m mm m mm m NH 0 Av HHMOOH Hmsmfl> OO.NH oo.ma Hm Nm N NH o NN N o 0 ca QUOQ Hamomm 2mm: .c.m OH m m h m m v m N H peas HmSmH>IOHOs< HODOMM uncommon Hmmflocflum on» so mcflcmoH ummsmflz may QDH3 muomflQSm He MOM pass Hmsmfl>IOHOsm an msumuumm aamomm .N manna -46- o¢.mN m¢.oa ms 00 Nm Hm 5H 0v ma Ha m mN HHMOOH OHUSd o¢.mm mm.vN N0 am m 00 HA Om ca MN 0 mo Hamomu HMSmH> oo.ON w¢.ma me me c mN H mm m m w mN anon Hamomm Cmmz. .U.m OH 0 m h o m w m N H Dans HMSmH>IOHUs¢ muomnflsm mo Ham How DHCS HmsmH>IOHUsm xn mahmuumm Hamowm .H magma -47- units in the second table. The deviants have been screened out by the factor analysis. If we look across each of the reSponse categories, we can see that there is substantial variation among the audio—visual units within each category. In Table 2, with a maximum score Of 41, the standard deviation of RECALL BOTH is 13.00; VISUAL RECALL, 17.427 and AUDIO RECALL, 12.27. There is some danger that the recall patterns might be determined by a serial position, primacy-recency, effect. (8) The learning of a list Of nonsense syllables, for instance, is usually most rapid for the beginning and the end of the list. A similar effect has also been Observed with other materials, e.g., greater recall Of the first and last stories Of a radio newscast. (31) Curves for RECALL BOTH, VISUAL RECALL and AUDIO RECALL over all ele— ments for both the 69 subject and 41 subject groups, are presented in Appendix K. The familiar U-shaped curve indicating the serial position effect does not appear, ~but there is some evidence of a primacy-recency effect. It can be seen in Table 1 that the recall of both elements of the first, ninth and last audio-visual units is higher than the mean for all units. The recall of the fifth and seventh units is also higher than the mean, however. Also in Table 1, on visual recall, the first, fifth, seventh, eighth and ninth elements are higher than the mean recall. -48- Table 2 discloses an identical result. There seems to be some possibility of the serial position effect, but it cannot account entirely for recall patterns as irregular as those reported. Coincidentally, however, all three redundancy indices are high at the end points. (See Appendix L.) It would be expected that the redundancy measures for each element or unit would be distributed randomly, except perhaps for the last few elements which might become redundant because of information gained from prior material. If there is a primacy—recency effect, given the coincidence Of high redundancy in the first and last elements, it would work in favor Of the hypotheses. This possibility requires a more circumSpect interpretation Of the results. Audio Redundancy The results of the measure Of redundancy in the audio elements is reported in Table 3. The index for each word is given immediately above the word. It is a ratio Of the number Of subjects with the correct response to the total number Of subjects reSponding to the word. The audio redundancy index was a summation Of the word indices divided by the number of words deleted from each sentence. These appear in Table 3. It should be noted again that subjects were given only the words preceding -49- mac. .mm>am£m so OOMODm mum mommoso OLD OmHQOH cam Umuma 00.H 00. 00. 00. 00.H mm. 00. 00. has. .mmmmso mmfl3m mm 00.H 00.H 50. .OH s3ocx NHSOEEOU Oahu UmumHHusm> mnu Hos m.uH >m3 mnu mm .0 mm. 00. 00. 00. mm. 00. mm. mm. mm. 00¢. .mpHoE smcoo3 canon oucH commend ma ommmno was cone .0 mm. 00. 06. 0e. 00. 00. 00. 00.- mwo. .Om>oEmH mum mcuso mnu Hm>umucH Homoum m umumm 3oz .5 00. 00. 00. 00. 00. 00. 00. 00. mNN. .mum> ammo mmHmH CH Omumo: ms musuxHE one .0 0c. 00. 0c. 00. 00. 0m. 00. 00-. .MHHE on» 0» qumm msflapuso m mpcm HOMME mmmmno use .m mu. 0m. mm. 00. 00. 00. 00. 00. ON. mmo. .cofluuomoum SH umuusn 0cm mmmoso O>Hoomu HHHB HOCBO muH .c 0N. 00. mm. 00. 00. 00. 00. 00. 00m. .UOHSmmmE ma 300 some SQ co>flm MHHE mo USSOEM OQB .m 00. mN.0m. mm. 06. mm. 50. 00. 00. mHH. .monm ms mumm umxmfi mmmmso HMHOHMMO wsu an mmoHO .N mN. mN.00. 00. 00. 00. 06.00. mNm. .mousuwmm panama CH mumum mBOO msucoa HOEESm on» nmsouza .H mm. 00. 00. mm. 00. 00. 00. 00.H mecH mosmc pas: Icscmm Oflcsfi HmsmH>IOHUs< mucofimam oflcsm onu OSHDSDHumsoo mmocoucom cam mpuoz now xmcsfl mocmpcscmm .m magma -50- the deleted word, not the entire sentence. It is likely that this cut the probability Of correct reSponse by about half. Given this limitation, the variation among the 10 sentences is quite good, ranging from .056 for element seven to as high as .419 for element 10. Visual Redundancy The results Of the visual redundancy measure are in Table 4. Subjects apparently were able to discriminate between the scenes on the five-point "certain-uncertain" scale. The lowest scoring element was the eighth with 38, an average of one point from the most extreme reSponse at the “uncertain" pole. The highest scene totaled 134 which is just Short Of the maximum certainty score it could have attained (140). The majority Of subjects used the entire scale; only 15 failed to do so. Audio-Visual Redundancy Redundancy in the audio-visual relationship is reported below in Table 5. The results from both versions are re- ported in the same table. Subjects, of course, were better able to make correct matches, given the Opportunity to indicate more than one possibility. Again, there is sub— stantial variation among the unitS--from no correct re- Sponses to unit three to 100 percent on the tenth unit. -51- Table 4. Visual redundancy index for each of the 10 visual elements Visual element Visual redundancy index 1 133 2 113 3 103 4 . 83 5 100 6 90 7 111 8 38 9 121 10 134 There is a high correlation (r=.95) between the two versions of the measure. Since the second version, un- limited matching procedure, is more appropriate for our purposes, as indicated in Chapter III, it will serve as the index of audio-visual redundancy in subsequent tables. The Relationship Between Redundancy and Recall The product moment correlations between the various recall patterns and the measures of redundancy are reported in Tables 6-13. Tables 6 and 7 report the recall correla— tions with audio redundancy for the entire group in the -52_ Table 5. Redundancy in the audio-visual relationship for the 10 audio-visual units _—A— Audio—visual A-V redundancy, A—V redundancy, unit Single match, N=30 unlimited match, N=19 1 28 18 2 2 6 3 1 0 4 4 1 5 25 18 6 3 5 7 30 18 8 4 7 9 5 8 10 30 19 recall measure (N=69) and for the subjects remaining after the factor analysis (N=4l). Tables 8 and 9 report the same information for visual redundancy. Tables 10 and 11 report the correlation of the recall measures for both groups Of subjects with the audio-visual redundancy mea- sure. Certain correlations, other than those bearing dir— ectly on the hypotheses, are also reported for purposes of later discussion. Significance levels, and the t values. (two-tailed), are reported for all coefficients. Although several hypotheses were deduced from the theoretical -53- rationale, the study was considered to be exploratory and no Significance levels were established at the out- set. It was decided to report all findings. Furthermore, it was recognized that the N of only 10 worked against the probability Of significance at conventional levels. Table 6. Product moment correlations Of audio redundancy with recall patterns for 69 subjects _r_ d.f. _p p Recall both .35 8 1.07 (.40 Audio recall .53 8 1.76 (.20 a m Table 7. Product moment correlations Of audio redundancy with recall patterns for 41 subjects _r_ _d-f- s 2 Recall both .39 8 1.19 (.30 Audio recall .59 8 2.08 (.10 Table 8. Product moment correlations of visual redundancy with recall patterns for 69 subjects _r_'_ d.f. p p Recall both .63 8 2.28 (.10 Visual recall .70 8 2.75 (.05 -54- Table 9. Product moment correlations Of visual redundancy with recall patterns for 41 subjects g d.f. p p Recall both .58 8 2.00 (.10 Visual recall .66 8 2.47 (.05 =_ fl Table 10. Product moment correlations of audio-visual redundancy with recall patterns for 69 subjects .r_ M; 1:. 2 Recall both .76 8 3.34 (.02 Visual recall .85 8 4.54 (.01 Audio recall .55 8 1.87 (.10 ==: a========================= Table 11. Product moment correlations Of audio-visual redundancy with recall patterns for 41 subjects 3 51-3“- £ 2 Recall both .70 8 2.76 (.05 Visual recall .83 8 4.27 (.01 Audio recall .51 8 1.68 (.20 -55- The Prediction of Maximum Recall If audio redundancy, visual redundancy and audio— visual redundancy are all used together for the prediction of the recall Of both elements in the audio-visual unit, we have the result reported in Table 12. In this table, the multiple correlation coefficient is reported with its P value and probability. In Table 13, the Beta weights, their F values and probabilities, and the partial correlation coefficients are reported for the three variables. Table 12. Multiple correlation coefficient for audio re- dundancy, visual redundancy and audio-visual redundancy with recall Of both elements (high first factor subjects) Multiple r Source S.S. ‘gpfp m.s. §_ ‘p .81 Regression 1009.49 3 336.50 3.94 .07 Error 512.51 6 85.42 Total 1522.00 9 I Table 13. Beta weights and partial correlation coeffi- cients for audio redundancy, visual redundancy and audio-visual redundancy with recall Of both elements (high first factor subjects) Beta Standard Weight Error E p Partial r Audio Redundancy .3054 .2708 1.27 .30 .42 Visual redundancy .3172 .2381 1.78 .23 .48 Audio-visual redundancy .5200 .2716 3.65 .11 .62 Chapter V CONCLUSION At this point it may be said that there is an appar- ent relationship between redundancy as we have defined it, in simultaneously presented audio-visual message units and the recall of the elements in the message. There is some support for all Of the hypotheses advanced in Chapter II. All correlations are in the hypothesized direction, and almost all Show a high degree Of relationship. As eXpected, not all correlations were significant at conven- tional levels (under the conditions of this study, a cor— relation of .632 is necessary for Significance at the .05 level). Hypotheses 1: Audio Recall and Audio Redundancy The recall of the audio element in a message where audio and visual elements are presented Simultaneously is ppsitively related to audio redundancy; The predicted relationship here is strong, although not quite high enough for great confidence in the result. There is some likelihood that a correlation as high as the one Obtained could occur by chance. For all subjects in the recall measurement, r=.53, p(.20; for the subjects -56- -57- remaining after the factor analysis, r=.59, p(.lO. The measure may have been improved by a revision Of the scoring method for audio redundancy. It might have been more consistent with the rationale for the measuring device, to weight the correct response by the number Of syllables in the response. The present system would give the correct response "the" exactly the same weight as the correct response "ventilated." Hypothesis 2: Visual Recall and Visual Redundancy The recall of the visual element in a message where audio and visual elements are presented simultaneously is positively related to visual redundancy. More confidence may be placed in this result. For all subjects in the recall measure, r=.70, p(.OS; for the subjects remaining after factor analysis, r=.66, p(.05. It is interesting to note the effect of the factor analysis on the results. The correlation of audio redun- dancy with audio recall is improved by the increased homo- geneity in the over-all pattern Of recall responses, but 22p the correlation Of visual redundancy with visual recall. From Observation Of the raw data, not included here, it might be concluded that audio recall was much more hetero- geneous than visual recall. To force out a general pattern over both audio and visual recall, improved the homogeneity of the audio pattern at the eXpense of the visual pattern. -58- Separate factor analyses of audio and visual recall bear this out. The first visual recall factor accounts for 53 percent of the variance while the first audio recall factor accounts for only 30 percent. The over-all first factor, as reported in Chapter III, is in between at 39 percent. Hypothesis 3: Maximum Recall and Audio Redundancy The recall of both audio and visual elements which are presented simultaneously in a message is positively related to audio redundancy. The results here are not conclusive. For the entire group on the recall measure, r=.35, p(.40; for the group after factor analysis, r=.39, p(.30. Although a fairly high degree of relationship Obtains, it could be a chance occurrence. The previously mentioned heterogeneity of the audio recall reSponses undoubtedly worked against higher correlation. Hypothesis 4: Maximum Recall and Visual Redundancy The recall Of both audio and visual elements which are presented Simultaneously in a message is positively related to visual redundancy. The relationship of the recall Of both elements in the audio-visual unit to visual redundancy is quite high -59- and not very likely to have occurred by chance. For the full group Of subjects on the recall measure, r=.63, p(.10; for the smaller group, r=.58, p(.10. Hypothesis 5: Maximum Recall and Audio-Visual Redundancy The recall Of both audio and visual elements which are presented simultaneously in a message is positively related to redundancy in the audio—visual relationship. The correlation is very high. For the full group on the recall measure, r=.76, p(.02; for the group after factor analysis, r=.70, p(.05. It might be suggested that the strong relationship is an artifact of the procedure. If audio and visual elements are, in fact, redundant, the recall Of one ele— ment would be the same as recalling the other. The re- sponses to one or the other would be indistinguishable. Redundancy in the relationship, however, as measured by the matching procedure, assumes only an identifiable rela— tionship between the two channels. In the recall scoring, reSponses were attributed to the proper channel without difficulty. For instance, one Of the highest audio-visual units in redundancy was number five. The audio element was: "The cheese maker adds a curdling agent to the milk." The visual element was a close-up Of a man's left hand pouring a clear liquid into a large vat while the other hand stirred with a paddle. —60— A common recall response would be to mention something like "a curdling agent is added" in the narration column, using the same words as the actual narration. The reSponse in the visual column would be mention of a liquid being poured into the vat and being stirred. While the unit measures high in redundancy, the responses to the two elements are readily distinguishable. In fact, it can be argued that while the degree Of redundancy between audio and visual elements may vary, there can never be "perfect redundancy" as some writers suggest (28). The visual and audio elements, by their nature, must elicit different associative reSponses. Hypothesis 6: The Prediction of Maximum Recall The recall of both audio and visual elements which are presented Simultaneously in a message is best predicted by a multiple regression equation based on audio redundancy, visual redundancy and redundancy in the audio-visual rela- tionship. This is indeed true in the present case; R=.81, p=.07 (using the recall measure based on the subjects remaining after factor analysis and the audio-visual redundancy mea- sure based On the unlimited matching procedure). The mul- tiple correlation is higher than any of the correlations of the individual redundancy measures with the criterion, recall of both elements. The partial correlations indicate -61— that each redundancy measure contributes: audio redundancy, .42; visual redundancy, .48; and audio-visual redundancy, .62. However, there is some question about the signifi— cance of the partial correlations and the weights of the contributing variables in the multiple regression equation. None is significant at conventional levels. The Findings and the Broadbent Construct These findings tentatively testify to the value of the Broadbent construct of the Single channel utilization system with its short term storage capacity. AS much as possible, the Opportunity for alternation of attention was limited. Certain of the elements in our film were ppp registered by immediate recall. These ele- ments were most likely to occur at points where the message was most heavily loaded with information; where the message was least redundant. When the message becomes most re- dundant, we are likely to have maximum reca11--reca11 Of both elements. This conforms to what would be expected from a limited capacity utilization system and the Opera- tion Of a storage system, where one element may be stored while the other is in process through the utilization channel as discussed in Chapters I and II. —62— Implications The findings would probably not surprise the sophis- ticated producer of informational audio—visual materials. Redundancy in the audio and visual channels is undoubtedly related to what might be intuitively judged as simplicity. Many producers of informational audio-visual materials would agree that content and structure ought to be simple and direct. Most producers of such material would also agree that there ought to be an interacting relationship between the audio and visual materials presented simultaneously. Nonetheless, in practice, the principles are not always followed. The materials used in this study are cases in point. The newsfilm used in the pilot studies contained a substantial proportion of only indirectly related audio and visual elements. The Switzerland film clip, while changed somewhat, was quite close to the orig- inal and, Of course, it represented a rather complete range of variation on all three redundancy measures. The substantial number of audio and visual elements which never reach the utilization channel of the receiver would nO doubt disturb the communicator, if his intention is to maximize information received. For all Of the materi- als used in this study, there was a low ratio Of elements received in cognition to elements transmitted. Yet, the costs of preparing the transmitting audio-visual materials -53- are high in terms of human resources. There are some obvious needs for further research. The necessary limitation on the number of elements which can be measured by immediate recall forces extremely high correlations before we can be relatively confident that they are not chance occurrences. We might be willing to accept lower correlations, however, if a number of repli- cations were performed. If these replications were over a variety of materials, another purpose would be served. We would then have some confidence that the findings were not Specific to the par- ticular content Of the material in this study. Mode of presentation should also vary. It would seem that the results would generalize from classroom film pre— sentation, to other means of audio-visual communication such as television and to other settings, but this assump- tion was not tested. One might also experiment with types of reSponse. Recognition is probably also meaningful in this context. Subjects might be eXposed to a long run of synchronous audio and visual elements and asked to note those to which they had been previously exposed. These reSponses would not be difficult to quantify. It might also be possible to devise both verbal and visual tests to indicate learn- ing from the various elements. The main obstacle would be to arrive at items of equal difficulty for all elements. -64- It would be nearly impossible, to determine whether the difficulty Of an item was inherent in the item or whether it reflected inadequate reception Of the element. If the findings could stand the test Of replication with different content, in different audio-visual media and under different reception conditions, and for differ- ent reSponse criteria, the instruments for measuring audio redundancy and visual redundancy could be very useful in a number Of Situations where the evaluation Of informa- tional material is important. They might be used to check educational films or short, "single concept" films much as programed instructional materials are validated. Tele- vision commercials might undergo the same treatment. Television news also; as in the old readability formulas that used tO be periodically imposed on neWSpaper and maga- zine content. The instrument for measuring redundancy in the audio-visual relationship would need to be adapted to conventional audio-visual presentation where elements do not match as perfectly as they were forced to in the exper— imental film. Audio and visual elements frequently overlap. Another problem is to account for pauses in the audio chan- nel which would serve the same function as redundancy. An additional step of great practical potential, would be an attempt tO establish an Optimum level Of redundancy for various desired reSponses. Or, stated in the converse, what is the maximum information load for a multiple channel —65- message. Summary We have assumed the Broadbent model Of a Single chan- nel utilization system which has a momentary storage capa— city. From this model, it was hypothesized that redundancy in and between the audio and visual channels Of an audio- visual message would enhance the possibility of storage where alternation of attention is not possible. Redun- dancy was measured in the audio portion and in the visual portion of a film message. An index Of redundancy was also taken for the relationship between the simultaneously matched audio and visual elements. Correlations Of immediate recall measures with the indices of redundancy supported the hy— potheses; some, however, at a level of confidence which would not be tenable without replication. APPENDIX A RECALL OF AUDIO AND VISUAL ELEMENTS IN NEWSFILM -66- Peeeem'hee OF sumac-rs RECA LL I A) 6— ELEmnemrr .u 43 -Lo .51 .54 -5\ «III A: 41 .3q .8!» .33 .50 4'7 .41 .1] J? .IS .17. ‘0? -.oc. n03 SEQODDS SCENES VI$U.RL.. RECALL. —— -—- sumo REGAL—l— N=I6 0 ‘lezltanidflazat‘lo‘ldflszaw 112.1 a 4 5I6I'1IXI‘7I/quI/zl FIILDI 0 DJ) ITS I I IILI 3 I4I5I6I '1 ISI 9IIquI I?— [MINI/r]- Figure Al. -- Recall of Opposing audio and visual elements in a newsfilm story about business and industry. From film and script used in the WJIM-TV year-end report covering 1964. -57- I’Eficemaee OF suBIEUFS RECALLUOG— ELémEDT' .5/ I II ——— sumo RECALL #8 .96 ““5 .42. I .5‘] .54. I .33 I .30 \ .1'1 .24, -2I .18 .Is I .n. I .0‘1 .05 .03 VISRAL RECALL- Seeoubs o 4 8 11162011283231.9041 Sgeoes I I IzlaI¢I516I718l9IwInI Hume wows! I ILI3I4I5IbI'II8I 9| MINI Figure A2. -- Recall of Opposing audio and visual elements in a newsfilm story about state apportion- ment. From film and script used in the WJIM—TV year-end report covering 1964. -68— .coma mGHHO>OO phonon osmiummm >BIZH03 map SH poms Daemon cam anm scum .musmcwoom hmsnmfls usonm NHOum EHHmmBOS m CH mucoEmHm HMSmH> cam Oflcsm mnemommo mo Hamomm II .md musmflm I he: _m;.§_ : - 1.:- e- n- Ire-.PwF... 714-. .63-942- fisxq Ms -..\_m-_d-_:_2_mfim_efi J SIN 4:..- dfi- gm” 2.: assetsSSS-ads.3w-.md8= essssaasfinsnds.3. tea-m.- -.a 0 223mm m- u .z 3.. o I 1 III Jul-Own- .adt 6.3m. da-jcawd )4ng I-xdfiwzl :Il waxnfldw “”0 ‘53an -69- mamas OF S uea’eeTS REQR L LNG— ELEMENT .94I .9] 7? J§ J; .L‘) .bL .63 .60 .57 I ~54 I .5! .H I .45 I Hi .34 I I .36 I 33 I .30 .20 I .74 I oil .17 I AS J1 I .09 I as l 63 __ Lzufib ALL. A .-_- Aucbto /\ /\ Remu- / \\ 1/ I Mus” \ l I \ / ‘ v ‘ seasons 0 z 4 scenes I t HubW>UAHTSI l I z 68loIZI¢ILIIZALzzl26UJO lzlsl 4 I51“ Isl 4 I slelfllsl‘Il Figure A4. -- Recall Of Opposing audio and visual elements in a newsfilm story about picketing at the state capitol. From film and script used in the WJIM—TV year-end report covering 1964. -70- APPENDIX B RECALL OF AUDIO AND VISUAL ELEMENTS IN AN EDUCATIONAL FILM -71- Numsea OF SLLSIEQTS RECALL IL) e ELE me NT w .. / ‘N In 40 32 a. 31 31 3o 2.! 24 M u. 20 M It H I1 m X I. 4 2 o I z..8 4 .5 L '1 3 q to u )1 ELEMEIOTS VISILPtL- KEARN— —-- Ruble RECALL. N -Sb Figure B1. -- Recall of audio and visual elements in a film clip from a Walt Disney produc- tion entitled Switzerland. -72- APPENDIX C CORRELATIONS OF CERTAINTY WITH RECALL MEASURES _73- TABLE C1 CORRELATION OF VISUAL CERTAINTY WITH RECALL II Recall Patterns r Recall, both Opposing audio and visual elements .65 NO recall, either opposing element -.66 Recall, audio element only .00 Recall, visual element only .35 Total recall, visual (recall both plus visual only) .59 Total recall, audio (recall both plus audio only) .52 TABLE C2 CORRELATION OF AUDIO CERTAINTY WITH RECALL Recall Patterns r Recall, both Opposing audio and visual elements -.22 NO recall, either Opposing element .29 Recall, audio element only .18 Recall, visual element only -.32 Total recall, visual (recall both plus visual only) -.10 Total recall, audio (recall both plus audio only) -.37 -'74- -75- TABLE C3 CORRELATION OF CROSS-CHANNEL RELATIONSHIP WITH RECALL .[-, -3========= =================== Recall Patterns r Recall, both Opposing audio and visual elements .38 NO recall, either Opposing element -.20 Recall, audio element only -.05 Recall, visual element only -.11 Total recall, visual (recall both plus visual only) .15 Total recall, audio (recall both plus audio only) .19 APPENDIX D THE EXPERIMENTAL FILM -76- .mm>H0£m CO UUHOHW 0H0 mommmno Hmnuo aufl3 macaw mommmno can .OmeQmH 0cm cmumn m.mu on so mmmmsu mmomHm cmE m2 0H .omomno mmw3m mm ssocx NHGOEEOO omen Omum mummno Eoum nuoHo afluco> may no: m.uH .Nm3 wnu mm m.m« NMH cam OHOE msfl>ofimu cmE m: m .mcaoE cmcoo3 Ussou Ammmum mucosa- Oucfl cmmmwum mH mmmmno on» cone o.¢u om um>mH msoH mcflxuoz Sum m2 0 maamun canvas .cm>oEmu mum mcuso any .mpuso nua3 pmaafim nuoHu .Hm>umusfl Homonm m Hmumm 302 m.v« mo- uso mcflasmn moans DOS 5 .mum> some xHflE mo um> once wmumH SH poumms we unauxfle one e.m“ mm nuoHO msfluummcfl moss: DOS 0 .MHHE Osu ou usmmm muflum can: moum .MHHE mo mafiacuso m mops memEommono one w.mu Hm um> ODSH cmusom OHSUHH DOS m .cofluuomoum SH Hmuusn cam SOHO mmmwso m>wmumu HHH3 Mezzo muH m.mu cm mmumH mcflsHSu cmE 03» m2 c .OOHSmmmE ma 300 some xHHE mo Hmsflmusoo ma cm>fim xHflE mo uSSOEm one m.mu me CH mcquo3 HOOD can: DU m .mosm as mumm umxmfi muflm um>o um> mmumH one Immmmno HMAOHmmo may .NQ mmoHO 0.0” 00 Imsfl3m mcflcaflsn SH :mE m: N mousummm panama CH mnmum mGHMDGsOE .mmceoaflsn m3ou .mnucoa umEEsm mnu nmsouze 0.¢u 00 Show .musummm SH mBOO ma H ezmfimqm OHQDG mQZOU-mm mum-Emma Beams-mam AfiDmH> .02 _77- APPENDIX E RECALL INSTRUMENT _73- COLLEGE OF COMMUNICATION ARTS Michigan State University INSTRUCTIONS You have just seen a sound film about cheese-making in Switzerland. On the following pages you will find a form for writing as much as you can remember about the narration and the visual. Write all the details you can recall from the narration in the right hand column. Write all the details you can recall from the visual in the left hand column. DO not be concerned about the order of the items in either column; put them down as they come to you. It is not necessary to match the narration with the visual. The two columns, as you fill them in, do not have to have any relationship to each other. Please try your best to write down all of the elements that you can recall from both visual and narration. -'79- VI SUAL NARRAT ION -80- VISUAL NARRAT ION -82- APPENDIX F RECALL SCORE SHEET -33- Elements .b U1 0‘ \l (I) \O 1" O Subject f Chan. 1' 2 3 fi TOTAL APPENDIX G AUDIO REDUNDANCY INSTRUMENT -35- INSTRUCTIONS y AUDIO REDUNDANCY In a moment I will distribute a small booklet. Please do not Open it until I tell you. PASS OUT BOOKLETS. Please write your name in the upper right hand corner of the first page. On each Of the pages inside this booklet is a part of a sentence. You are to read the word or words quickly and try to guess what the next word would be. Guess quickly and write the word on the blank provided. The word you supply is not necessarily the last word Of a sentence, in fact in most cases it will not be. The length of the blank has nothing to do with the length Of the word that is miss- ing. Each blank is the same length no matter how long the next word is. StOp writing when I say “stOp.“ I will then read the complete sentence so you will know what the miss- ing word was—-you are not to write it down at this point, however. Once I have finished reading, go immediately to the next page and try to guess the word in that sequence. Make sure you turn only one page at a time.’ We will go through ten sentences in this manner. I will repeat the instructions. REREAD ALL AFTER “PASS OUT BOOKLETS." Are there any questions? Please turn over the first page and start. READ THE SENTENCE SILENTLY, ALLOW TIME TO FILL IN A WORD, THEN SAY . . . StOp: READ THE SENTENCE ALOUD. Now turn over to the next page and try to guess the word. -86- Through -87- Close by, the Official cheese maker -88- The amount of -89- Its owner will receive cheese -90- The cheese maker adds a curdling agent to the -91- The mixture is _92- Now after a prOper interval, -93- Then the -94- By the way, it's not the ventilated -95- Dated and labeled, the cheeses -96- APPENDIX H VISUAL REDUNDANCY INSTRUMENT -97- COLLEGE OF COMMUNICATION ARTS Michigan State University -98- -99- INSTRUCT IONS In a moment you will see silent scenes from a Short film. We are interested in finding out how well you under— stand the scenes. You will be asked to rate the scenes in accordance with the degree Of certainty you have in your understanding of the scene. Again, you are to rate the scenes in terms of the degree of certainty about your comprehension of what you saw. If the scene is quite clear to you--if you are certain that you comprehend what you saw, check the blank next to the word CERTAIN: CERTAIN Z : : : : UNCERTAIN If the scene is fairly clear--if you are fairly cer- tain you comprehend what you saw, check the second blank: CERTAIN : V: : : UNCERTAIN For "not sure," check the middle blank: CERTAIN : : Z : : UNCERTAIN For fairly uncertain, check the fourth blank: CERTAIN : : : \/: UNCERTAIN If you are completely uncertain about what you have seen--if the scene is very unclear or confusing to you, check the blank next to the word UNCERTAIN: CERTAIN : : : : p: UNCERTAIN The screen will be blank for ten seconds after each scene. Make your rating of the scene during this period. lO. CERTAIN CERTAIN CERTAIN CERTAIN CERTAIN CERTAIN CERTAIN CERTAIN CERTAIN CERTAIN -lOO- UNCERTAIN UNCERTAIN UNCERTAIN UNCERTAIN UNCERTAIN UNCERTAIN UNCERTAIN UNCERTAIN UNCERTAIN UNCERTAIN APPENDIX I AUDIO—VISUAL REDUNDANCY INSTRUMENT -lOl— [SECOND ADMINISTRATION] COLLEGE OF COMMUNICATION ARTS Michigan State University INSTRUCTIONS This will take just a few minutes. You will be asked to match motion picture scenes with sentences which might be in a script for the film. The scenes and the sentences have been mixed up. So that you are familiar with the sentences, we will read them over twice. Please turn the page and read along silently as they are read aloud. In a moment we will look at the scenes you are to match with the sentences. You will be shown a scene only once. The screen will go blank after each scene. As soon as it does, you will be reminded of what number it is. You are to write the number Opposite the sentence that most closely corresponds with that scene and any other sentence that might also fit the scene. Again, write the number Opposite the sentence that seems most closely related to the scene and any other sentence that might possibly match the scene. You therefore, might have a particular scene number written in front of more than one sentence. Guess, if you see no obvious choice. You may also use a sentence more than once. In other words, some sentences may have more than one scene number on the blank next to it. Other sentences may not be used at all. You will have about 40 seconds to make your choices. At the end of this -102- -103— time, you will be warned so that you can finish and look up at the screen for the next scene. Do not change your final choices for a scene after the next one starts. Since the scenes last only about four seconds, you will have to pay close attention. —lO4- Its owner will receive cheese and butter in proportion. Now after a proper interval, the curds are removed. The amount of milk given by each cow is measured. Close by, the official cheese maker sets up shop. Then the cheese is pressed into round wooden molds. The mixture is heated in large open vats. Dated and labeled, the cheeses are stored on shelves. Through the summer months, cows graze in upland pastures. By the way, it's not the ventilated type commonly known as Swiss cheese. The cheese maker adds a curdling agent to the milk. [FIRST ADMINISTRATION] COLLEGE OF COMMUNICATION ARTS Michigan State University INSTRUCTIONS This will take just a few minutes. You will be asked to match motion picture scenes with sentences which might be in a script for the film. The scenes and the sentences have been mixed up. So that you are familiar with the sentences, we will read them over twice. Please turn the page and read along silently as they are read aloud. In a moment we will look at the scenes you are to match with the sentences. You will be shown a scene only once. The screen will 90 blank after each scene. As soon as it does, you will be reminded of what number it is. You are to write the number opposite the sentence that most closely corresponds with that scene. Again, write the number Opposite the sentence that seems most closely related to the scene. Guess, if you see no obvious choice. YOu may use a sentence more than once. In other words, some sentences may have more than one scene number on the blank next to it. Other sentences may not be used at all. You will have about 60 seconds to make your choice. At the end of this time, you will be warned so that you can finish and look up at the screen for the next scene. Do not change your final choice for a scene after the next -105- -106- one comes up. Since the scenes only last about four seconds, you will have to pay close attention. -lO7- Its owner will receive cheese and butter in proportion. Now after a prOper interval, the curds are removed. The amount of milk given by each cow is measured. Close by, the official cheese maker sets up shOp. Then the cheese is pressed into round wooden molds. The mixture is heated in large open vats. Dated and labeled, the cheeses are stored on shelves. Through the summer months, cows graze in upland pastures. By the way, it's not the ventilated type commonly known as Swiss cheese. The cheese maker adds a curdling agent to the milk. APPENDIX J , PRINCIPAL FACTOR LOADINGS ~108- Table J1. Factor loadings on the principal factor in the analysis of recall reSponses (asterisk indicates subjects with loading of .60 or over). Subject Factor Subject Factor Subject Factor number loading number loading number loading '1 .230 24* .600 47* .906 .279 25 .444 48 .561 3* .642 26 .525 49* .805 4* .661 27* .625 50* .641 5* .602 28* .640 51* .791 6* .636 29 .399 52* .685 7* .677 30 .577 53* .821 8* .771 31* .692 54 .544 9 .567 32* .775 55* .656 10* .619 33 .430 56* .633 11 .476 34* .721 57 .585 12* .630 35 .557 58* .804 13* .712 36* .706 59 .569 14 .596 37* .669 60* .639 15 .597 38 .491 61* .716 16* .625 39 .426 62* .828 17* .711 40* .687 63 .432 18* .638 41* .816 64 .523 19 .117 42 .069 65 .323 20 .362 43 .437 66* .798 21* .806 44 .584 67 .442 22 .444 45* .659 68* .815 23* .717 46* .644 69* .896 -109- APPENDIX K RECALL CURVES ~110- Numeefi. OF sumac“: RECHLLMM} ELEMEUT' 48 45 94 J9 SL 53 30 41 4‘ &|” 18 I5 ll 7 ‘ 6 3 O NlMNaL z a ‘1 5 6. 7 £3 9 lo ELEJAEUTS OF SWECTS Keen Lu 0c.- augmenT' 33 3o 47 a! a; la O NFQF‘R z s '+ 5' L 1 ELEJAEUTS 6 9'/0 -111- Figure K1. -- Recall curve for the subjects recalling both ele- ments of each of the 10 audio—visual units (N=69). Figure K2. -- Recall curve for subjects recalling both audio and visual elements in each of the 10 audio-visual units (N=4l). Numsee. OF mesa-rs RaMHIJNG- ELEmAexn' 42 3? 56 33 3o .27 41 all [8 AS )1 9 Q 3 O l 2: 3 4 £5 4 7 8 9'10 ELEMEMT‘S NKLMSEQa OF surecrs chmLLmO ELemao-r 42 39 BL 33 50 21 2! LI [8 6’ It q (a 3 011345579910 ELEMEDTS ~112— Figure K3. —- Recall curve for subjects recalling the visual element in each of the 10 audio-visual units (N=4l). Figure K4. -- Recall curve for subjects recalling the audio element in each of the 10 audio-visual units (N=4l). 0 | z. a 4 .5 6» 1 8 9 lb EELEJAGQOTTS Figure K5. —— Recall curve for the subjects recalling the visual elements in each of the 10 audio—visual units (N=69). -113- thumsee. 0F sweats flea». L! DC} éLemen—r La 5: 56 54 52 50 49 46 44 42 40 38 36 J! 31 30 1.! 2L 1" Z7. 20 /8 IL /4 11 IO ovsvm l 2 £3 4 .5 6 ‘7 8 9 In E LE MEN‘TS Figure K6. -- Recall curve for the subjects recalling the audio element in each of the 10 audio-visual units (N=69). —114- APPENDIX L REDUNDANCY CURVES -115- I23156789/o mole—v nsum. um‘r ILJ‘ISG'IGQIO Hubto—V\sum_ UMrT —116- Figure L1. -- Visual redundancy by audio- visual unit. Figure L2. -— Audio redundancy by audio— visual unit. PM» o -v \sum. Eewmncy :3 . ’ n ‘ A [4 {Mb I5 q ‘1 /4 I3 II B ( ‘ z a 4 s a 7 a 9 Jo m10-V\SLLM_WI‘I’ 0 Figure L3. -- Audio—visual redundancy by audio- visual unit. -117- REFERENCES -118- 10. 11. 12. REFERENCES Ball, John and Byrnes, Francis C. (Eds.). Researchy Principles, and Practices in Visual Communica- tion. Washington, D.C.: Department of Audio- visual Instruction of the National Education Association, 1960. ' Berlyne, D. E. Conflict! Arousal and Curiosity. New York: McGraw-Hill, 1960. Birdwhistell, Ray L. "Background to Kinesics," Etc., 1955, Vol. 13, 10-18. Broadbent, D. E. “Immediate Memory and Simultaneous Stimuli," Qparterly Journal of Psychology. Vol. 9 (1957). 1-11. Broadbent, D. E. Perception and Communication. New York: Pergamon, 1958. Broadbent, D. E. "Successive Responses to Simultaneous Stimuli," Quarterly Journal of Experimental Psychology. Vol. 8 (1956), 145-152. Broadbent, D. E. "The Role of Auditory Localization in Attention and Memory Span," Journal of Ex- perimental Psychology. Vol. 47 (1954), 191-196. Deese, James. The Psychology of Learning. New York: McGraw-Hill, 1958. Deutschmann, Paul J., Barrow, Lionel C,, Jr. and McMillan, Anita. "The Efficiency of Different Modes of Communication," AV Communication Review, Vol. 9 (1961), 263-270. Garner, Wendell R. Uncertainty and Structure as Psy- chological Concepts. New York: Wiley and Sons, 1962. Gr0pper, George L. "Why‘gg a Picture Worth a Thousand Words," AV Communication Review, Vol. 11 (1963), 75-95. Harrison, R. and MacLean, M. S. Facets of Facial Communication. Michigan State University (mimeo), 1965. -ll9- 13. 14. 15. 16. 17. 18. 19. 20. 21. 22. 23. 24. -120— Hartman, Frank R. “Investigations of Recognition Learning Under Multiple Channel Presentation and Retention," AV Communication Review, Vol. 9 (1961), 24-43. Hazard, William R. "On the Impact of Television's Pictured News," Journal of Broadcasting, Vol. 7 (1962-63), 43-51. Hilgard, Ernest R. Theories of Learning, 2nd edition. New York: Appleton-Century-Crofts, 1956. Holland, J. G., and Skinner, B. F. The Analysis of Behavior: A Program for Self Instruction. New York: McGraw-Hill, 1961. Lippmann, Walter. Public Qpinion. New York: Mac- Millan, 1922. May, M. A. and Lumsdaine, A. A. Learning from Films. New Haven: Yale University Press, 1958. Miller, George A., The Magical Number Seven, Plus or Minus Two: Some Limits on our Capacity for Processing Information. In David C. Beardslee and Michael Wertheimer (editors), Readings in Perception. Princeton, N.J.: Van Norstrand, 1958. Miller, Neal E. 2; a1., "Graphic Communication and the Crisis in Education," AV Communication Review, Vol. 5 (1957). Mowbray, G. H. "Simultaneous Vision and Audition: The Comprehension of Prose Passages with Vary- ing Levels of Difficulty," Journal of Experimen- tal Psychology. Vol. 46 (1953), 365-372. Mowbray, G. H. "The Perception of Short Phrases Presented Simultaneously for Visual and Auditory Reception," Qparterly Journal of Experimental Psychology, Vol. 6 (1954), 86-92. Munsterberg, Hugo. "Studies from the Harvard Psy- chological Laboratory: A. Memory," Psychologi— cal Review, Vol. 1 (1894), 34-38. Nelson, Stanley (Chairman of the Nomenclature Committee, University Film Producers Association), Journal of the University Film Producers Association, Vol. 12 (1960). -'~ A,_. p I ‘— 'an—m- ..._ 25. 26. 27. 28. 29. 30. 31. 32. 33. 34. 35. —121— Neu, D. Morgan, "The Effect of Attention Gaining Devices on Film—Mediated Learning,“ Technical Report SDC 269-7-9, Instructional Film Research Reports, Port Washington: U. S. Navy Special Devices Center, 1950. Osgood, Charles E. "Dimensionality of Semantic Space for Communication Via Facial Expressions," Scandinavian Journal of Psychology. Vol. 7 (1966), 1-30. Osgood, C. E., and Sebeok, T. A. “Psycholinguistics: a Survey of Theory and Research Problems," Journal of Abnormal and Social Psychology, Vol. 49, Part 2, (1954), Morton Prince Memorial Supplement. Severin, Werner J. Cue Summation in Multiple Channel Communication. (Paper prepared for the Associa- tion for Education in Journalism National Con- vention, Iowa City, Iowa, August, 1966.) Sumstine, David R. "A Comparative Study of Visual Instruction in High School," School and Society, Vol. 7 (February 23, 1918), 235-238. Switzerland, Educational Film Division, Walt Disney Productions, Burbank, California. Tannenbaum, Percy H. "Effect of Serial Position on Recall of Radio News Stories," Journalism Quarterly, Vol. 31 (1954), 319-23. Taylor, Wilson L. "Cloze Procedure: A New Tool for Measuring Readability," Journalism Quarterly, ~Vol. 30 (1955), 415-33. Travers, Robert M. W. Research and Theory Related to Audiovisual Information Transmission. Uni- versity of Utah, Bureau of Educational Research, 1964. Wittich, Walter A. and Schuller, Charles F. Audio- visual Materials: Their Nature and Use. New York: Harper & Bros., 1962. Woodworth, R. S. Experimental Psychology. New YOrk: Henry Holt, 1938, 712. J..- {1" ') ti -122- 36. Zuckerman, John V. "Commentary Variations: Level of Verbalization, Personal Reference, and Phase Relations in Instructional Films on Perceptual- Motor Tasks," Instructional Film Research Re- ports, Port Washington: U. S. Navy Special Devices Center Technical Report SDC 269-7-4, 1949. 9R ICHIGnN STATE UNIV. LIBR 1H1|""1111"I111111111111111111 3129310200117 155 "1| In 13