Date 0-7639 // aa—r ,4’ .— I lllll’lmllllllll 2, This is to certify that the thesis entitled THE EFFECTS OF INDIRECT AND INDUCTIVE TEACHING STYLE ON COGNITIVE, AFFECTIVE, AND BEHAVIORAL LEARNING IN THE DEBRIEFING OF A COMMUNICATION GAME presented by ERIC M. EISENBERG has been accepted towards fulfillment of the requirements for M.A. Communications degree in MASTER OF ARTS W01 Ptofm February 14, 1980 I. I B R A R Y Michigan State University OVERDUE FINES: .u 25¢ perduporitem 1. 1mm“ .‘ ‘54" ‘ [gum .. RETURNING LIBRARY MATERIALS: .‘ ‘3, u,” ' 1 Place in book ret rem ‘ “fl” “ urn to . charge from circulation recozs THE EFFECTS OF INDIRECT AND INDUCTIVE TEACHING STYLE ON COGNITIVE, AFFECTIVE, AND BEHAVIORAL LEARNING IN THE DEBRIEFING OF A COMMUNICATION GAME By Eric M. Eisenberg A THESIS Submitted to Michigan State University in partial fulfillment of the requirements for the degree of MASTER OF ARTS Department of Communication 1980 ABSTRACT THE EFFECTS OF INDIRECT AND INDUCTIVE TEACHING STYLE ON COGNITIVE, AFFECTIVE, AND BEHAVIORAL LEARNING IN THE DEBRIEFING OF A COMMUNICATION GAME By Eric M. Eisenberg This investigation examined indirect and inductive teaching style as potential predictors of student learning in the debriefing of a communication game. Indirectness was defined as the teaching style which solicits students' ideas as the focus for class discussion. An inductive teaching style is organized from the simple to the complex, from the concrete to the abstract. Student learning was reflected in cognitive, affective, and behavioral gains due to the educational experience. The hypothesized relationships were organized in a structural equation model. An overall test of the goodness-of-fit yielded an insignificant chi-square value of 37.7627, p = .0016 with l6 degrees of freedom. Six hypotheses concerning specific parameters were tested. Indirectness and inductive presentation were significant predictors of affective learning. Behavioral learning was significantly predicted by inductive presentation, but not by indirectness. Cognitive learning operationalized by test score was not significantly pre- dicted by either indirectness or inductive presentation. DEDICATION To my mother, for her unconditional faith in me. Your unspoken vision has been clearly understood. ii ACKNOWLEDGMENTS I would like to extend special thanks to Dr. Cassandra L. Book and Dr. Peter R. Monge for their continued support of my research. Their tough questions and critical comments aided my thinking in both theoretical and methodological areas. Dr. Book's continued positive attitude towards my work in general has been invaluable. Harm thanks to Dr. Edward L. Fink, who was always available for help and support although he was not a member of my guidance committee. More importantly, Dr. Fink transformed a college graduate petrified of statistics into a doctoral student with a strong interest in research methodology. I am indebted to my undergraduate assistants, William Morgan, Susan Boyer, Dan Kozlowski, David Gilland, Cindy Gaugush, and Carol Brys for their long hours of work and dedication which as much as any- thing else helped make this research flow smoothly. Finally, I am truly grateful for the moral and intellectual support I have received from my good friends, Ronnie Kurchner-Hawkins and Steven Burch. They were there when it counted. iii TABLE OF CONTENTS LIST OF TABLES .......................... LIST OF FIGURES ......................... Chapter I. RELATED LITERATURE AND STATEMENT OF THE PROBLEM ....................... Literature Review .................. Indirectness ................... Inductive Presentation .............. Gaming Simulation ................. Gaming Simulation in the Social Sciences ..... Related research ................ Past Approaches .................. Debriefing ................... Invalid testing ................ Experiential teaching and experiential learning ................. Theoretical Model .................. Cognitive learning ............... Affective learning ............... Behavioral learning .............. Research Hypotheses ................ Chapter II. METHODOLOGY AND PROCEDURES ............. Procedures for Data Collection .......... Subjects .................... Detailed procedures .............. Pilot tests .................. Instrumentation .................. Indirectness .................. Coder training ................. Inductive presentation ............. Cognitive learning ............... Affective learning ............... Behavioral learning .............. Experimental Design ................ iv Page vi l4 TE 16 TABLE OF CONTENTS (Continued) Chapter III. RESULTS .................... Structural Equation Model ........... Description of Variables ............ Manipulation Checks ............. Overall Test of the Model .......... Specific Hypotheses ............. Chapter IV. DISCUSSION ................... Summary of Findings .............. Overall Model ................ Specific Hypotheses ............. Problems and Limitations ............ Model Evaluation and Revision ........ Implications for Practice ........... Suggestions for Future Research ........ REFERENCES .......................... APPENDICES .......................... Appendix A: Categories of Interaction Analysis (Flanders, I970) .............. Appendix B: Lesson Plan ................ Appendix C: Pilot Cognitive and Affective Instruments Appendix D: Cognitive Instrument ............ Appendix E: Affective Instrument ............ Appendix F: Instructions for Behavioral Task ...... vAppendix G: Script for Behavioral Task ......... 78 79 83 88 90 92 93 LIST OF TABLES 3191?- 1399. l Interrater reliabilities for coders using the FIAC system ......................... 37 2 Descriptive statistics for untransformed variables . . . 48 3 Descriptive statistics for transformed variables: affective and behavioral indicators ........... 49 4 Correlations among multiple indicators ......... 50 5 Means of all endogenous variables for all groups (including control) ................... 52 6 I/D ratios for experimental groups and averages across three coders ................... 54 7 Variance accounted for in the endogenous variables . . . 55 8 R2, E2, and E2 - R2 for all bivariate relationships: test for non-linear association ............. 65 vi LIST OF FIGURES E19933 Page 1 Theoretical model of learning effects .......... 24 2 Full model of learning effects .............. 27 3 Parameter estimates and standard errors for a model of the effects of indirectness and inductivity on learning ......................... 57 vii CHAPTER I RELATED LITERATURE AND STATEMENT OF THE PROBLEM The task of designing educational experiences which consistently promote learning is a complex one. A review of the literature in educational methods and classroom interaction reveals many varied approaches to the study of effective teaching and to the study of education in general. Few of these approaches have yielded useful generalizations for teachers or students, and those generalizations which do exist are not commonly put into practice. Communication researchers have on occasion chosen to study the classroom environment. They have typically taken two approaches. The first, sometimes called "instructional communication," is the study of communication variables, strategies, processes and systems as they relate to formal instruction. This approach can be applied to the study of any type of classroom, regardless of the nature of the content under study. “Communication education" on the other hand, is an area of research which has focused mainly on the effective teaching of communi- cation concepts, principles, and skills. Traditional educational conceptualizations of classroom variables are used to study the teaching of communication related content. In practice, the distinction has not been clear. Researchers often do one and call it the other. The two general orientations l 2 are discussed briefly here to point out that communication researchers studying classroom interaction can and should benefit from both approaches. This study attempts such a combination: communication variables are examined in the teaching of communication concepts, principles, and skills. The following broad research question was posed: "What effect does teacher verbal behavior in the classroom have on student learning?" Three general types of student learning have been assessed: (1) cognitive, (2) affective, and (3) behavioral. Taken as a set, these three describe all of the ways a student may change as a result of an educational experience. Two general patterns of teacher verbal behavior are of interest in this study; indirectness and inductive presentation. Indirectness is a structural variable reflecting the degree to which a teacher verbally encourages student input and builds on students' ideas, regardless of the content of those ideas. Inductive presentation deals more directly with content; it is the degree to which simple or concrete ideas occur prior to complex or abstract ideas in the presentation of a concept. The ideas may originate with either the teacher or the student. This study is a test of the effects of indirectness and inductive presentation on cognitive, affective, and behavioral learning. LITERATURE REVIEW An enormous amount of research was done in the first half of the Twentieth Century which attempts to relate teacher verbal behavior to 3 student achievement. Countless observations of classroom interaction were conducted, with little consistency in their results. Marsh and Wilder (1954, p. 4) in their review of published research on teacher effectiveness conducted from 1900 to 1952 conclude: No single, specific, observable teacher act has yet been found whose frequency or percent of occurrence is invariably (and) significantly correlated with student achievement. One plausible explanation for this finding is inadequate measure- ment. Whatever the "true" predictor variables of student achievement are, the classroom observational techniques may not have been sensitive enough to record them. More precise and reliable methods of measurement were called for. Classroom interaction analysis category systems were developed, in the hopes that they could better reflect what was happening in the classroom. Simon and Boyer (1967) list 100 separate interaction analysis category systems. Friedrich and Brooks (1970) review several systems which have particular relevance for speech communication classes. Most notable among there instruments have been those developed by Flanders (1965) and Amidon and Hough (1967). Better instrumentation advanced the inquiry somewhat. In his review of classroom research utilizing classroom communication variables as predictors of student achievement, Flanders concluded: It can now be stated with fairly high confidence that the percentage of teacher statements that make use of ideas and opinions previously expressed by pupils is directly related to average class scores on attitude scales of teacher attractive- ness, liking of the class, etc., as well as to average achievement scores adjusted for initial ability. (1969) 4 This dimension of classroom communication (using students' ideas and opinions) is called indirectness and is readily measured by Flanders Interaction Analysis Category System (FIAC: see Appendix A). A teacher who uses ideas and opinions previously expressed by students is considered to be primarily indirect, while a teacher who expresses known facts or his or her own opinions most of the time is classified as primarily direct. Using these terms, Flanders concluded that the only consistent predictor of student achievement available in classroom communication at the time of his review was teacher indirectness. Indirectness Indirectness is conceptually defined as the teaching style which incorporates students' ideas and opinions as the central focus of class discussion. Ideas are solicited through probing, open-ended questions, used both to bring out initial observations and to build to more general principles. The indirect teacher clearly accepts the responsibility of coordinating student ideas, and of insuring that the line of thought progresses. Indirect teaching is comparable to inquiry teaching (Massialas, 1975), the major difference being that while the inquiry teacher asks good questions, the indirect teacher uses students' responses in forming a cohesive pattern of thought. The major difference between a generally direct and a 7 generally indirect teacher is that the focus of the interaction for the latter centers on ideas and opinions predominantly generated by students. The philosophical groundwork for education which takes into account student experience is rich. American pragmatists Dewey, 5 James, and Pierce are champions of the classroom setting where "education is of, by, and for experience" (Dewey, 1938, p. 29). This way of thinking about education offers implicit support for indirect teaching, an approach which capitalizes on student experience and facilitates learning by making new information meaningful in the context of the students' pre-existing cognitive structure. The indirect approach is also consistent with arguments made by many educational psychologists. They contend (Ausubel, 1968, p. 127, 133-142; DeCecco & Crawfbrd, 1974, Ch. 3) that the knowledge already existing in an individual's cognitive structure is the single most important factor in determining whether or not new knowledge will be learned and understood. If students are expected to apply to the future what they have learned, they must have the opportunity to use what they have learned in the classroom. The transfer of knowledge from one situation to the next is aided by mental or actual practice (Yelon, 1974). Beginning with students' ideas and opinions and helping them to develop them in the classroom, the indirect teacher provides an opportunity for students to use what they are learning. Alterna- tively, direct teachers may address students as a passive audience who are never called upon to check their understanding through use. The above discussion of indirectness focuses only on one struc- ~ tural dimension of classroom interaction. A teacher may use students' ideas and opinions but still not provide a cohesive structure for the material to be learned. Ideas should be presented in such a way that they are best understood and retained. One dimension of presentation is inductivity, or the degree to which simple and concrete ideas pre- ceed complex and abstract ideas in the presentation of a concept. Inductive Presentation While the definition of indirectness specifies that an indirect teacher will solicit ideas from students, there is no restriction placed on the kinds of ideas or their sequencing. It has been sug- gested that the order in which the statements are presented (by teacher or student) makes a difference in effective teaching (Scott & Young, 1976; Lashbrook & Wheeless, 1978). Inductive presentation is conceptually defined as the sequencing of ideas from the simple to the complex, and/or from the concrete to the symbolic. Deductive presentation is the opposite. This se- quencing should be noted not over the entirety of a lesson, but rather in reference to each concept presented. For each main point, an inductive presentation involves the statement of specific examples of a particular phenomenon prior to generalizations or abstract statements of principles. A classroom can be organized inductively or deduc- tively with a teaching style that is direct or indirect; while the source of the ideas will differ, their organization will remain the same. Possible combinations of these two variables are further described in the section on experimental design. Lashbrook and Wheeless (1978) provided the following proposition: Instructional communication activities which progress from concrete to symbolic operations are more effective (in facilitating learning) than ones which work from symbolic to concrete operations. (1978, p. 444) Gagne's heirarchy of learning (1968) suggests that people are first able to recall some fact, then subsequently learn to apply it in similar, then different situations. It follows that classroom messages should be ordered in a way which is isomorphic with the way 7 in which people typically process information, in this case induc- tively. The next section deals with a specific pedagogical technique where indirectness and inductiVe presentation are particularly salient: gaming simulation. Gaming Simulation Gaming simulation is a pedagogical strategy which attempts to incorporate indirectness and an inductive presentation in facilitating learning. It uses students' ideas and opinions in a structure which begins with the concrete experience (the game) and advances to more symbolic representations (the post-game debriefing). Gaming simulation provides a referent experience for students where none may exist. Although there is widespread use of gaming simulation in the communication classroom, it is often unclear whether its use is consistent with either indirect or inductive presentation. While it seems clear that the debriefing of a game would be inductive (generalizing from the concrete experience to abstract principles), this is not always the case in practice. Games are often undertaken with some predetermined outcome in mind, and debriefing may end up ignoring the students' experiences altogether. Instead, the debrief- ing session may serve as a forum for the teacher to relate his or her own conclusions about how the game "should have gone." In theory, gaming simulation implies a follow-up strategy which is both indirect and inductive. In practice, premature generalizations may follow a game, and students' ideas may be ignored or played down in deference to the teacher's ideas. Given this variability in ways to debrief a game, the implemen- tation of gaming simulation provides a specific practical context for testing the effects of inductive presentation and indirectness on student learning. Gaming Simulation in the Social Sciences The term gaming simulation refers to any educational experience which has the qualities of a game (actors, roles, rules, etc.) and also attempts to teach something about a large, relatively unacces— sible system through a smaller, accessible one. While Raser (1969) argues that games are as a rule not well developed enough to be called simulations, most researchers and practicioners are comfortable with the definition of gaming simulation as the performance of game-like activities in simulated contexts. Teachers of traditional speech courses have for a long while understood the distinction between process and content orientations to teaching communication. If the terminal objective of a class is to get students to communicate more effectively, then an educational strategy should be employed which allows for the actual practice of communication in the classroom. This way of thinking is usually offered as justification for mock interviews, group decision-making exercises, and public speeches in the classroom. An interpersonal laboratory component has been added to many courses professing to teach communication skills in higher education (Bochner, 1974, p. 279). Ruben and Budd point out that: 9 Becoming truly competent in communicating-- in communicating and being communicated with-- requires a unique combination of theoretical understanding and practical skill. Neither alone is sufficient. (1975) Related research. Research on the effectiveness of gaming simulation in facilitating learning has left many hypotheses uncon- firmed. In his classic study of the effectiveness of gaming simu- lation as a teaching tool, Cherryholmes supported only one hypothesis about game effects--that students found them fun to play (1966). There is some support for an increase in cognitive learning through the use of games. Baker (1968), Emery and Enger (1972), Lindblad (1973), and Wing (1968) all treated the recall of a body of content on a paper and pencil test as their dependent measure of cognitive learning. A11 report significant results. Pierfy (1977) reports that "students who participate in simulation games will retain information longer than if they learned the information through more conventional approaches" (1977, p. 259). Some studies have reported significant affective achievement through gaming simulation (Chartier, 1972; Boocock, 1963; Clark, 1970) while others found no significant affective differences (Livingston, 1972). Despite these contradictions in research findings, the use of gaming simulation in the classroom has continued to increase. House, in an editorial on the state of the art in gaming, pessimis- tically states: There are large numbers of studies that purport to show that games teach something but few that prove scientifically that games either teach better or teach something that cannot (or is not) taught by other means. (1973, p. 451) 10 House uses this finding to support the argument that people involved in the use of gaming simulation should redirect their interests away from education and training. Further attempts to empirically discover what has made intuitive sense to many researchers have been frustrated. Rosenfeld reports that "variations in the simulation environment can produce variations in the effectiveness of the game as a teaching device," and that "A great deal of variation in teaching/learning effectiveness has been found from game to game" (1975). This suggests that the argument which states “The same game was run in repeated trials" is inaccurate; games vary drastically with their style of presentation as would any pedagogical strategy, and it is precisely this variability which may have a differential effect on learning. The teaching strategies involved in presentation, play, and debriefing combine with the personalities and learning characteristics of the students in deter- mining how a game will turn out. One factor which appears to be crucial in the presentation of a gaming simulation is the post-game debriefing session, sometimes called "processing.“ Walling (1977) supports the hypothesis that "full processing" (pre- and post-game discussions) facilitates learning better than one without the other, or neither. Ruben (1977) underscores the critical role of debriefing in the gaming experience: In many respects, the manner in which an experiential activity is implemented and debriefing is conducted is more crucial to the outcome than the properties inherent in the activity. This argument is consistent with those arguments made earlier in favor of inductive presentation, which pointed to the advisability ll of incorporating new ideas into the student's pre-existing cognitive framework. Debriefing may serve as the major opportunity for the teacher to establish the link between the concrete gaming experience and more general principles applicable outside of the classroom. In other words, debriefing may serve to make a body of content more relevant to students. Debriefing may differ in games with imposed rules from those in which students help develop the rules. Ruben describes the debriefing process for games with well-defined rules: . . . debriefing format and style should be designed to maximize the transfer of specified information to participants. The more predict- ably participants come to shared understandings of their learnings . . . the more effective the outcome. Discussion should strive to augment these ends. (1977, p. 229) Careful attention should be paid not only to the game itself, but to the nature of the accompanying discussions. Researchers have made convincing arguments that a game viewed in isolation is limited in its learning effects and needs to be linked to other instructional tools such as group discussion as a means of "completing" the experience (Boocock & Schild, 1968). One argument for including a post-game discussion states that since games are interactive in nature, they tend not to lead players to symbolic representations. A post-game debriefing session might help accomplish this latter aim (Bruner, 1966; Dale, 1969). Should this be true, it would underscore the uselessness of assessing cognitive learning in the game alone condition. Others state that post-game discussion is a necessary condition for maximizing learning from a game (Abt, 1968; Boocock, 1970). 12 This claim lacks empirical support. Chartier reviews the‘ literature which assesses the effect of combining gaming simulation with post-game discussion, and reports: There is no reported research to support the claim that linking a simulation game with a discussion period will maximize learning out— comes. (1972) He further notes that "It appears that the highly participative character of games militates against cognitive learning," and tests the hypothesis that gaming simulation in combination with discussion will maximize learning over simulation without discussion or dis- cussion without simulation. Chartier concludes that: (1) Subjects who participated in a simulation game with a discussion did demonstrate higher learning outcomes at the affective level than did (subjects in the other con- ditions). (1972, p. 213) (2) While game theorists (Abt, 1968; Boocock & Schild, 1968) have claimed that cognitive gains can be maximized through combining discussion with simulation, cognitive learning outcomes for the combined discussion simulation condition were not significantly different from the other conditions. (3) The research thus far with simulation games would indicate that there are no measurable differences between it and other teaching techniques with respect to cognitive learning. (Chartier, 1972) Other research has produced conflicting results. Walling (1977) concludes that "the full processing teaching method was more effective in producing learning at all higher cognitive levels than either the lecture alone or the game alone methods." In support of the notion that one must be concerned with varieties of game presentation as opposed to examining games in isolation, he reports, "the game alone 13 teaching method was not an effective strategy for producing cognitive knowledge when compared with conventional teaching methods" (Walling, 1977, p. 24). Walling's explanation of his results emphasizes the relationship between indirectness and debriefing: First, it may be assured that processing (de- briefing) is in fact practicing. The processing components provide, but do not guarantee, mental practice of higher order concepts for students. When used in conjunction with a lecture and/or a discussion, simulation games force the students to overtly "practice" concepts which are implicit and/or explicit in the conventional teaching methods. (1977, p. 24-25) Abt (1972) also describes post-game discussion as a kind of mental practice in which the student can reflect more objectively and abstractly on the learning outcomes of the game. Winne describes the . debriefing session as "memory support" (1977, p. 391). Other researchers (Bilek, 1972; Fletcher, 1971; Livingston, 1973) have encouraged the use of a debriefing session as a way to provide students with a more complete, symbolic, and shared understanding of game objectives. Walling concludes that gaming simulation must be viewed as having the same limitations as other forms of instruction, constrained by the educational philosophy and ability of the teacher. Further, "few studies empirically examine how games might be inte- grated into the total teaching process. Since teaching is basically a communication activity in itself, this oversight seems significant" (Walling, 1977, p. 1). Chartier also recognizes the need for a finer conceptualization of the ways in which_games are administered: Further investigations might include . . . investigations relating simulation to discussion formats other than instrumentation -- e.g., leader centered or permissive, student centered formats . . . studies measuring the effects of l4 simulation with discussion upon other cognitive behaviors than those used in this study . . ." (1972, p. 217) Studies which attempt to determine the "best" combination of simulation and discussion are overly simplistic in that they ignore the complexity of classroom interaction processes. What needs to be recognized is the enormous variability inherent in an activity like "debriefing," such that one could measure the differential effects of different approaches. Types of interaction patterns could be examined, as is done with more traditional teaching techniques. Rather than searching for "an effective game," one would look instead for those factors which make for the effective use of a game. This study is a beginning attempt to make some of these more microscopic distinctions. The next section outlines the major problems in inter- preting previous research. Past Approaches Teachers of communication and related disciplines dealing with human interaction have turned to gaming simulation in an attempt to involve students more directly in the learning process (Bochner & Kelly, 1974; Ruben & Budd, 1975). Research on the effectiveness of this technique has been shown above to be inconclu- sive; the notion that gaming simulation might in some way be situa- tionally superior to traditional pedagogical techniques has not been supported, and where partial support has been found, conflicting findings abound. Three major problem areas exist in the research which attempts to assess the learning effectiveness of gaming simulation. They are 15 as follows: (1) Debriefing has been treated as an all-or- none occurrence. (2) Invalid tests have been used to assess learning. (3) Experiential teaching has been confused with experiential learning. Debriefing. The term simulation implies that the practiced activity is presumed to show some connection to the world beyond the classroom. Debriefing is employed to make explicit the link between the two. Most research has treated debriefing as an all-or—none occurrence. The linking of the educational simulation to the outside world is the critical justification for doing the activity. An all-or-none conceptualization of this linkage is overly simplistic. While gaming simulation is one way of treating concrete experiences for students in the classroom, these experiences may not be meaningful unless somehow incorporated into the student's ongoing body of experience. For example, if the debriefing were conducted by an “indirect" teacher with an inductive presentation, one could expect to see students' ideas drawn out through probing questions and developed by students into more abstract generalizations. If debriefing were conducted by a direct teacher with a deductive presentation, one would expect to see the teacher giving his or her perception of what generalizations are apparent from the game, followed by examples. Other combinations of these variables are possible, and there are, of course, numerous other variables which might be considered in a more precise conceptualization of debriefing. These should suffice in making the point that 16 debriefing can vary considerably depending on classroom interaction patterns, and these variations might have an effect on what students learn from a gaming simulation. These differences in debriefing style are not far-fetched but happen for some very specific reasons. The gaming environment is similar to any communication environment which is highly ambiguous, where information concerning "correct" approaches is in demand. Acting in a way consistent with past educational experiences, students will often request information from the teacher to aid in making key decisions. Students often become especially sensitive to both explicit and implicit advice given by the teacher in this environment. Depending on how the teacher responds to this demand by students, debriefing may become an occasion to provide personal opinions (of the teacher) and a summary of the more "correct solu- tions" (also from the teacher's perspective). A teacher who resists the temptation to give advice allows for a larger range of novel ideas and opinions to be voiced and developed. Invalid testing, A valid test is one in which the evaluation clearly mirrors the objective (Yelon, 1977). There should be iso- morphism between the objectives of a game and the evaluative tool used to assess the learning of those objectives. Cherryholmes (1966) reasoned that one should first specify what a game is attempting to teach, and then develop an instrument suitable fbr measuring the accomplishment of this end. Invalid testing is particularly a problem given the emphasis on behavioral outcomes as objectives of many courses in communication. Many researchers who have tried to assess the learning effects of l7 gaming simulations which teach behaviors have collected measures of behavioral intention or commitment as indicators of learning (Elliot, 1978). In measuring the attainment of specific goals such as "to become an effective speaker," or "to perform a better interview,“ paper and pencil measures of intended behavior may be misleading. Numerous researchers have discussed the so-called "attitude-behavior inconsistency," which states that there are many ways in which a person's behavior may differ from their stated attitude. While this caveat may not be universally applicable, it points to the questionable validity of measuring behavioral competency via paper and pencil tests (Campbell, 1963; Seibold, 1975). Pierfy (1977) reports that: 1 Virtually every study (reviewed) collected data on learning usually by means of paper and pencil tests usually administered immediately after treatment. For the most part, the tests measured the knowledge of facts and principles. Invalid testing may result from two factors. It occurs when (a) behaviors are measured using paper and pencil instruments, and (b) measures of facts and comprehension are used to assess learning from a game which purports to teach behaviors. Invalid testing is so widespread a problem that it could be the major reason why most studies have failed to find gaming simulation as a viable alternative to more traditional teaching tools. Green- blat (1975) insightfully asks, "Do the data reflect poor outcomes or poor measurement?" In an example of invalid testing, Ruben (1976) studied those factors which would best predict success of Canadian sojourners who were to go to Kenya and found that many sojourners who performed well 18 on paper and pencil tests of cross-cultural effectiveness found great difficulty once they arrived in Kenya; they were eventually sent home. It is probable that the wrong abilities were being assessed with the cognitive test, and that the correspondence between verbal-cognitive competencies and behavioral competencies was low. Verbal-cognitive competency -- the capacity to conceptualize and articulate variables, dimen- sions and issues that need to be taken into account to explain or predict effective func- tioning in a situation -- may be a necessary condition for behavioral or social competence. Behavior competence -- the capacity to display behaviors that are defined as appropriate or functional by others -- is a sufficient condi- tion for effective social functioning and is probably at least a minimum condition for success in many task-oriented situations." (Ruben, 1976) Effort should be made to develop evaluative instruments which clearly measure the game objectives, since inferences from measures of cognitive learning to behavioral learning may be tenuous or inappropriate. If verbal-cognitive knowledge is an expected outcome of the game, paper and pencil tests are appropriate. If behavior change is also a desired outcome, then some fbrm of behavior should be observed and assessed. Experiential teachinggand experiential learning, The use of a gaming simulation does not guarantee that experiential learning will take place, although this is often the expressed goal. Numerous barriers to students' developing personal generalizations from the game experience exist; some have been discussed above. In general experiential learning may not occur when the teacher fails to solicit and/or fully develop students' experiences in the debriefing session and instead centers the session around his or her own perceptions of 19 the game experience. While allowing students to play a game is experiential teaching in the limited sense that a "hands-on" experience has been provided in the classroom, following up with a direct or deductive debriefing session would not then encourage experiential learning. This observation again points to the inseparability of gaming simulation and post-game debriefing. Pierfy (1977) argues that incorporating debriefing into each of the treatment groups of experiments testing the effectiveness of games corrupts the findings by not allowing us to see the effect of the game alone. He contends that debriefing allows the students who learned best during the game to give the information away to the others such that it is no longer possible to assess the true effect of the game on individuals. While this argument appears to make sense, it ignores a basic conceptual assumption. Experiential learning assumes a model of intellectual growth which moves from the concrete to the symbolic (Bruner, 1966). Knowledge can be regarded in two important ways: it is shared, and it is symbolic. All meaningful knowledge can be expressed in the form of verbal propositions (Ebel, 1974). It is meaningless to exclude the debriefing session when testing for "game effects:" it is precisely this debriefing that completes the gaming experience by allowing the shared symbolic conclusions. It would be difficult to know in any theoretically meaningful way what effects an experiment comparing games without post-game debriefing would be testing. With these considerations in mind, the next seetion lays out the theoretical model to be tested. 20 THEORETICAL MODEL The first section of this paper described how educational researchers attempted to classify the dimensions of classroom inter- action to make it more manageable for study. An ideal model of the learning which results from gaming simulation could specify many more variables than would be practically measurable in any one study. Variables such as nonverbal behaviors, personality characteristics, class size, tolerance for ambiguity, credibility, information pro- cessing capabilities, and others all might be included in an ideal model. The literature review in the first part of this paper has described several fundamental variables of interest and identified problems in past research which will be dealt with in this study. Different types of debriefing sessions will be considered in assessing learning from games. Since the game used in this study purports to teach a skill, skills will be assessed by means other than paper and pencil tests. Debriefing is seen as inseparable from the gaming experience, as it is the place where concrete experiences are generalized into more abstract principles. Hence, all assessment of game effects includes post-game debriefing as a part of the treatment. The exogenous variables for this study are indirectness and inductivity as defined above. The endogenous variables are cognitive, affective, and behavioral learning. Cognitive learning. Cognitive learning is conceptually defined as the recall of previously learned facts. It is operationalized via 21 a paper and pencil test containing questions over the material taught in the game and debriefing. Affective learning. Affective learning is conceptually defined as the change in the composite of feelings the student has towards the teacher, subject matter, and the structure of the classroom experience. Students are asked to report their level of satisfaction with the game/debriefing in terms of the teacher, the topic, and the class structure. Each of these three evaluations are used as multiple indicators (of affective learning) in the model. Behavioral learning. Behavioral learning is conceptually defined as the observable application of learned principles in a context different from the one in which they were learned. This definition implies a change in ability to demonstrate principles. Principles are predictable interactions between people and events which may be causative or correlative (Yelon, 1978, p. 204). The student who demonstrates a knowledge of the principle through the corresponding behavior is said to have transferred or behaviorally learned the principle. Transfer and behavioral learning are used here synony- mously. Behavioral learning should not be confused with skill learning, which most often suggests the correct performance of a physical action involving the use of the muscles directly.l A behavioral measure which reflects the principles taught in the gaming simulation was administered. More detailed description of the instrumentation of the above variables appears in Chapter 2. The next section specifies the hypotheses to be tested. 22 Research Hypotheses In the previous section, some suggestions have been made con- cerning which classroom communication variables might impact on student learning. Each proposed causal relationship or hypothesis can be expressed in the form of a structural equation. Taken to- gether, all of the hypotheses form a system of structural equations which best represents the process under study. The typical way of testing these relationships is to examine each individually, with each supported bivariate relationship pro- viding support for the model as a whole. An alternative approach is to test the entire system as a whole, and to develop the overall model which best fits the data. The major advantage of this approach is that all of the proposed relationships can be assessed simul- taneously, using all of the available information in making parameter estimates. Parameter estimates for each of the bivariate relations may still be obtained, but they are influenced by all relations both specified and unspecified in the model. A statistical test is performed to see how well the specified system of equations fits the data (Monge, 1979; Duncan, 1975). In addition to evaluating all of the theoretical relations simultaneously, structural equation modeling has the added advantage of allowing one to test a measurement model for each unobserved variable. Rather than computing indices and entering them as endo- genous variables into a multivariate analysis of variance (for example), structural equation modeling allows us to propose multiple indicators of a given variable, which can later be evaluated for the 23 degree to which they load on the unobserved variable. Loadings are also calculated utilizing all of the information available from all of the relationships specified and unspecified in the model. The theoretical model of learning effects is presented in Figure 1. The overall hypothesis to be tested is that the full set of relationships specified by the model will be obtained in the data. H : B 0 l “’f " This hypothesis says that the matrix of regression coefficients is not equal to the null matrix. It is the overall test of goodness of fit of the model. Indirectness and inductive presentation are hypothesized to be significant predictors of cognitive, affective, and behavioral learning. The x2 test for the overall model is used since it is restrictive and minimizes type I error. The specific relationships which comprise the model are presented below: H : B > O 2 11 The more indirect the teaching style, the more cognitive learning will result. This hypothesis has received conflicting treatment in the literature. This direct relationship is suggested by Flanders' review relating indirectness to increased achievement in general (1969) but has not been supported by studies done in the context of gaming simulation. This hypothesis is consistent with theories of cog- nitive psychology outlined earlier, which argued that increased cognitive learning will result when new material is integrated into a person's preexisting cognitive framework. 24 £1 T ”i ‘— Cl g2 ‘5 ”2"‘T ‘2 ”3‘ ‘33 Figure 1. Theoretical model of learning effects. Legend for theoretical model: 5] = indirectness £2 = inductive presentation n1 = cognitive learning n2 = affective learning n3 = behavioral learning :1 C2 C3 errors of prediction 25 H3: 82-l > O The more indirect the teaching style, the more affective learning will result. This hypothesis makes strong intuitive sense and is supported by all of the early research on the effectiveness of gaming simulation (Cherryholmes, 1966). Games which explicitly take into account the students' point of view are likely to foster a feeling that the teacher is concerned, and positive affect should result. H4: 83] > O The more indirect the teaching style, the more behavioral learning will result. Many educational psychologists have shown that for a person to transfer a learned skill from one situation to the next, they should have some opportunity to initially use the skill. Indirect teaching allows students to "mentally practice" what they are learning rather than to passively receive a body of information that may have little relation to them. > 0 The more inductive the presentation, the more cognitive learning will result. There is strong support fbr the proposition that an inductive presentation should facilitate cognitive learning. It has been argued that when ideas are ordered from the simple to the complex, or the contrete to the abstract, it is easier for the individual to learn and to recall them (Gagne, 1966). > O The more inductive the presentation, the more affective learning will result. This proposition has little support, and is primarily 26 an extrapolation of hypothesis five. Gagfie (1966) argues that an inductive organization is consistent with the way in which people typically assimilate information. Hence, this type of presentation should prove less frustrating for students and lead to somewhat more positive affect. The more inductive the presentation, the more behavioral learning will result. In a way similar to indirectness, inductive presentation may allow students to manipulate specific examples of a phenomena when attempting to learn it. Indirectness and inductivity both appear to be necessary but not sufficient conditions for behavioral transfer (demonstration of the skill in a new situation). The indirect teacher gives the student the opportunity to practice the concepts of interest, but they may be abstract or hard to apply; the inductive presentation insures that concrete and manageable examples are dealt with at first, but they may originate from the teacher and students may not refect on them. Taken together, inductive presentation and indirectness seem to provide for mental practice for students, which facilitates behavioral learning. The model as represented in Figure l specifies only one indicator for each endogenous variable. In addition to one measure of cogni- tive learning, three measures of affective learning and two measures of behavioral learning were collected. These additional operationali- zations were added as multiple indicators and are displayed in the full model as represented in Figure 2. Once the model has been specified (which is functionally the same as writing the structural 27 Figure 2. Full model of learning effects. Legend for the full model. indirectness inductive presentation cognitive learning affective learning behavioral learning multiple indicator of affective learning multiple indicator of affective learning multiple indicator of affective learning multiple indicator of behavioral learning multiple indicator of behavioral learning errors of prediction errors of measurement 28 equations of interest), it is important to determine whether or not an algebraically unique solution exists for each of the parameters in the system. If each of the equations (there are three, one for each endogenous variable) has a unique solution and one or more of the equations have more than one unique solution, this is a desirable position and the model is said to be overidentified. A model must necessarily be just-identified for parameter estimation; it must be overidentified to be testable (and hence, evaluated). The model specified in Figure 2 is overidentified with 16 degrees of freedom, using the necessary but not sufficient condition of the counting rule (Nambooderi, Carter, & Blalock, 1975). Again, this means that (a) each of the parameters in the model may be estimated, and (b) there is enough information available to test the model's overall goodness- of-fit with the data. The combined measurement/theoretical model is called the full model. There are two kinds of relationships in the full model which need further explanation. First, errors of prediction (in equation) are constrained not to covary. This is the standard assumption employ- ed in ordinary least squares regression, which states that there is no unspecified variable which is correlated both with the exogenous and endogenous variable in the equation. This assumption is tested in a later chapter. Second, the covariance between the exogenous variables is fixed. This is so because in an experimental study, such as the present one, the covariations among the exogenous (manipulated) variables is either zero or known. In this case it is known before the analysis is performed, and an additional degree of freedom can be added to the model by fixing the covariance at this known value 29 (Costner, 1971; Alwin & Tessler, 1973). The next section details the procedures of the study, and the specific methods of data analysis. CHAPTER II METHODOLOGY AND PROCEDURES This chapter describes the research methods and experimental procedures employed to test the seven hypotheses specified in Chapter One. Treatments were in the form of one hour classroom sessions where subjects were involved in a gaming simulation designed to teach the appropriate use of feedback when receiving directions in a face-to- face interaction (Leavitt & Mueller, 1951). Following the adminis- tration of the game, which was identical for all treatment groups (classrooms), subjects participated in post-game debriefing sessions designed according to four combinations of indirectness and inductive presentation described in detail later in the chapter in the section on experimental design. A fifth group received no treatment (game or debriefing), participated in two of the three posttests, and served as a control group for the study. Procedures for Data Collection Subjects. One hundred students enrolled in an introductory course in communication were recruited fer the study. Subjects were in the fifth week of a first course in communication concepts, principles, and skills. Prior to recruiting subjects, the researcher conferred with instructors of this course to see whether the concepts they taught in the course were covered in the game. Students from 30 31 two sections in which material similar to that utilized in the research was taught were excluded from the research. None of the other sections had covered these topics. Subjects were recruited through class visitations and requests for volunteers. Following recruitment, subjects were assigned to one of the five groups using a random number table and assigned a date and time to arrive at the experimental site. Detailed procedures. Subject turnout for each experimental session was 14 (control group), 19, 18, 18, and 17. The four experi- mental sessions each began with a gaming simulation and a debriefing session. The topic of the gaming simulation was the importance of giving feedback in a dyadic interaction in which instructions were being given. The game used to demonstrate these concepts was "one- way, two-way feedback," originally designed for an experiment done by Leavitt and Mueller (1951). To conduct the game, the researcher selected a volunteer from the group to describe a series of geometric patterns. No one but the volunteer was permitted to see the patterns. The remainder of the group's members were asked to reproduce the patterns as accurately as possible from the volunteer's instructions. This procedure was repeated three times with different sets of figures by the same volunteer. The main difference between trials was that in the first trial, group members were not permitted to give any verbal, nonverbal, or vocal feedback. In the second trial, only vocal and nonverbal feedback was permitted. In the third trial, verbal feedback was allowed. Following each trial, the actual patterns were posted on the board by the researcher and students were asked to 32 raise their hands if they reproduced the diagram correctly. The researcher recorded the number correctly completing the diagram as well as the total amount of time taken to give the instructions in each trial. Results of the game in all cases demonstrated that while complete verbal, vocal, and nonverbal feedback was more time consuming, it led to more accurate results. The absence of feedback took the least time and resulted in the least accurate reproductions. Once the table of results had been posted, the volunteer rejoined the group and the debriefing began. The structure of each debriefing session followed the four experimental groups described later in this chapter: indirect inductive, indirect deductive, direct deductive, and direct inductive. Following the debriefing, subjects were sent to testing rooms for the behavioral posttest. In the case of the control group, subjects began the study with this test. Subjects were greeted in the testing room by a research assistant trained in the administration of the behavioral task. Subjects were asked to sit back-to-back with the assistant so they could not see a constructed LEGO model on the assistant's desk. Each subject was provided with LEGO pieces and an instruction sheet (see Appendix). Assistants were provided with scripted instructions on how to construct the LEGO model. Assistants were trained prior to the study in how to read the instructions at an even pace. Assistants were told that the instructions they were to read were purposefully vague to encourage questioning from subjects. An attempt was made through training to minimize variability among the assistants in the completeness with which they responded to questions. 33 Assistants were told to remain as close to the scripted instructions as possible in answering subjects' questions, and to repeat sections verbatim first if possible. When all subjects had completed the behavioral test, a cognitive test was administered. Finally, the affective test was distributed, read aloud, and an opportunity was provided for questions regarding the method of measurement to be raised. Additional examples of sample responses to affective questions were provided by the researcher. Students were free to leave upon completion of the affective test. Subjects in the control group did not complete an affective test, as there would be no stimuli for them to respond to. Subjects recorded the last feur digits of their social security number on all forms to facilitate cross-referencing. When all of the posttests were turned in and completed, the researcher gave a five- minute presentation on the rationale and hypotheses for the study. Subjects were cautioned not to discuss this information with anyone until the following week, when the experiment would be completed. Pilot tests. The cognitive test was piloted twice to assess the reliability of the measure. Each pilot test was done with 40 subjects. The first pilot had an extremely low reliability (.04) which was most probably due to random guessing, since subjects were completely ignorant of the subject matter (Ebel, 1974). The scores on the test were very low, and the mean was close to what would be expected by chance. This is an indication that the experimental treatments could have an effect on learning but indicates little about the internal consistency of the test. In an effort to get a 34 reliability estimate that was more interpretable, the second pilot test was preceded by a brief lecture on the main points to be covered in the test. Reliability of this test was .76, which was deemed sufficiently high to proceed with the experiment. The behavioral test was administered to five people to assess the ambiguity of the task. The task was made progressively more difficult to ensure that people would need to ask questions and that they could not complete the task in the assigned time period. Instrumentation Indirectness. Indirectness has been conceptually defined as the teaching style which incorporates students' ideas and opinions as the central focus of class discussion. Miller and Nicholson (1976) detail four criteria for operational definitions: they should (1) tap as much of the richness of the conceptual definition as possible, (2) allow for the standard usage of terms, (3) facilitate replication, and (4) allow calibration of the phenomena of interest on to a good scale. Operationalization of indirectness is done with these four criteria in mind. Indirectness is operationally defined using Flanders' Interaction Analysis Category System (FIAC) (see Appendix). The FIAC is a valid Operationalization of indirectness in that it emphasizes the importance of teacher talk. The indirect teacher takes the clear responsibility for the coordination of student ideas through probing questions used to elicit student input. The Operationalization would be more complete if an assessment of non- verbal behaviors was included. Some nonverbal category schema are 35 available, but they are as yet too complicated for easy administration. The FIAC is a valid Operationalization of indirectness because it attempts to reflect utterances in the context they were made. While a teacher may say certain things which would appear to be indrect if considered in isolation, he may be using these statements in a dramatically different way. What would appear alone as a legitimate question typically eliciting a student response may in fact be rhe- torical. Training in the use of the FIAC makes explicit rules which take into account the context surrounding what is said and the probable intent of the interactants. The FIAC seems particularly justifiable in light of the other three criteria for assessing operationalizations. The I/D ratio (indirect to direct statements) is standard usage in most studies of indirectness, and alternative measurements are not that different. Replication using a well-established system and formula would not be difficult. Coding interaction in this way and subsequently computing the I/D ratio yields a precise and understandable scale which clearly reflects teacher indirectness. The FIAC consists of ten categories, all reflecting types of verbal communication by teachers and students. The first seven categories are different types of teacher talk, the last three student talk, and silence and confusion (Flanders, 1966). .The indicator of indirectness that has been used most frequently in research on teaching is the I/D ratio, or the ratio of indirect to direct statements (Flanders, 1965; Furst, 1967; Gunnison, 1968.) Operationally, it is the ratio of statements in cate- gories one through four (teacher accepts feelings; teacher praises or 36 encourages students; teacher accepts or uses students' ideas; teacher asks questions) to the number of statements coded in categories five to seven (teacher lectures; teacher gives directions; teacher criti- cizes or justifies authority). While there are other ways of opera- tionalizing indirectness in the literature, this one is sufficiently simple and precise, and has been shown to have predictive validity. A presentation which receives a high ratio is highly indirect; the teacher spends more time questioning and encouraging students than he does lecturing. A presentation which receives a low ratio is highly direct; the teacher spends more time lecturing than questioning or encouraging students. Coder training. Three undergraduate students were trained to use the category system.. Coders met weekly for ten weeks and were given extensive experience in applying the system to both live and videotaped interactions. Lack of reliability among coders and lack of clarity in the categories was corrected through repeated training sessions. In addition, specific guidelines for using the system were provided for coders. Reliabilities were assessed seven times in the course of the ten weeks. Interrater reliabilities at the time of the experiment were .94, .96, and .93. Table 1 presents the interrater reliabilities for all seven training sessions. Indirectness was experimentally manipulated to be dichotomous: Two treatment groups were highly indirect and two were highly direct. Each debriefing session was coded by three assistants, and I/D ratios were calculated. These ratios are reported in Chapter 3. 37 Table l. Interrater reliabilities for coders using the FIAC system.1 Coder Reliability of Each Trial Combination l 2 3 4 5 6 7 Experiment I/II .69 .81 .73 .77 .94 .96 .97 .94 I/III .72 .77 .79 .76 .81 .94 .89 .96 11/111 .66 .94 .76 .87 .78 .86 .93 .93 1Trial number two was coding a live lesson. Trials took place approxi- mately every two weeks. Inductive presentation. Inductive presentation is operationally defined with the aid of a modified EDIT system, originally developed by Myers and Myers (1975) as a guide for the debriefing of gaming simulations. As originally conceived, the EDIT system has four major steps: 1) Experience. The running of the gaming simulation. 2) Describe. Immediately following the game experience, students confine their comments to particular details about the game. If a student attempts to make a more general statement at this time, he or she is prevented from doing so and is redirected to describing the game experience as completely as possible befOre making any inferences. 3) Infer. Students infer general principles from the specific observations they have made in the previous step. Application to situations beyond the game context are still prohibited at this stage. Concrete gaming experiences are rephrased in more abstract terms. 38 4) Transfer. Finally students apply the generalizations which they have made in the previous step to specific instances outside of the classroom. Inductive presentation has been conceptually defined as the sequencing of ideas from the simple to the complex, and/or from the concrete to the symbolic. While an excellent guideline fer game debriefing, the EDIT system is not exactly parallel to this conceptual- ization. It is different in three important ways. First, the EDIT system is designed primarily for use by the indirect teacher. Induc- tive presentation as defined above can be either teacher-centered or student-centered. Second, while the progression from "experience" to "describe" to "infer“ is clearly inductive, the progression from "infer" to "transfer“ is deductive. Third, the EDIT system is applied globally to a set of concepts and principles. This means that descriptions of concrete experience covering a variety of game topics are offered prior to generalizations about any of these topics. Inductive or deductive presentation is measured not as an overall progression from the specific to the general (or vice-versa), but rather as the organization of ideas in this fashion within each main point. This division by main point better captures the sense of inductive presentation as it is isomorphic with the theories of information acquisition discussed briefly in Chapter One (see, e.g. Gagne, 1966). Experiences which would come under the "transfer" heading of the EDIT system were combined with those from the "describe" step, such that both of these activities followed generalization in the deductive conditions, and preceded it in the inductive conditions. 39 Using the criteria for operational definitions described earlier, it appears that this operationalization suffers from different short- comings than that of indirectness. The EDIT system has been reora ganized to yield a valid representation of the variable. In principle, this operationalization should reflect the conceptualization com- pletely. In practice, the terminology used is somewhat ambiguous and confusing. No better system for evaluating inductive presentation was found (this issue would be more critical if measuring an attri- bute variable, rather than an active one with only two levels). The major difficulty with this system is that it is not standard usage, and although it yields values on a precise continuous scale, measure- ment is by fiat (Torgerson, 1957), and not derived from other relation- ships or taken from other research. Further, replication would be necessary but somewhat more difficult due to the newness of the terms. This operationalization seemed the best for measuring this variable, but should be carefully evaluated and improved with continued use. Two experts in communication education and teaching strategies were consulted as a manipulation check for inductive presentation. Each expert was asked to rate each of the four debriefing sessions on a 1-7 scale ranging from very deductive to very inductive. Experts were provided with criteria for each type of presentation as presented above. Results of this manipulation check appear in the next chapter. Cognitive learning, Cognitive learning has been conceptually defined as the recall of previously learned facts. A valid test would be one which clearly reflects the material taught and does not include 4o irrelevant material or test irrelevant attributes. A valid test must also be reliable. The five-part lesson plan was written first, and the cognitive test written from it. A true-false format was used to minimize the effects of any factors other than the recall of learned propositions. Items were written covering each main point in the lesson, and nothing else. Cognitive learning was assessed with a 22 item true/false test developed by the researcher to test the knowledge of the five main points included in the lesson plan (see Appendix). The 22 questions were designed to cover the five topics in the lesson and no others. A copy of the cognitive test is included in the Appendix. The cognitive test was pilot tested twice before administration. This was done to assess reliability. The true-false format with this few questions (22) made for simple administration but weak reliability (.76). Administration to groups without any knowledge of the lecture topic yielded scores near chance level, and a con- commitant low reliability, due to random guessing (.04). The com- parison of these pilot administrations provides evidence that the test was in fact measuring the material taught, if somewhat unreliably. Affective learning, Affective learning was assessed by the students' reported satisfaction with the teacher, the topic, and the structure of the debriefing and game. Direct magnitude estimation‘ was used in an attempt to generate precise results which are amenable to mathematical manipulation (Stevens, 1956; Hamblin, 1974; 41 Shinn, 1974; Fink & Huber, 1977). This technique begins with the development of a standard for comparison with which students are familiar. Students were asked to reflect on their educational careers and to think of an average teacher they had had, one with whom they were satisfied an "average amount." They were then asked to assign this amount of satisfaction an arbitrary value of 100 units. This 100 units was to serve as their yardstick in assessing satis- faction in other related areas. They were also told that total dissatisfaction would yield an estimate of O, and that twice the average amount would be 200. All positive numbers were allowed as estimates. Students were asked to compare their satisfaction with the teacher, topic, and structure of the gaming simulation with the arbitrary standard described above. They were then asked to estimate their satisfaction separately for each of these three dimensions of the experience. Estimates were used as multiple indicators of the unobserved variable affective learning. This operationalization of affective learning directly reflects the conceptualization presented in Chapter One. Since multiple indicators are used, an additional concern is the degree to which each of these is related to the proposed unobserved variable (affective learning). Affective tests were administered to assess whether (a) students could make magnitude estimates which fell along a reasonable scale, and (b) additional instructions were needed. Additional oral instructions and examples were added. 42 Behavioral learning. A transfer task was designed to measure how well students had learned to give feedback from the gaming simulation. Five assistants were trained in the administration of the task, which consisted of providing subjects with purposefully ambiguous instructions on how to construct a model from small raised blocks (called LEGO blocks). Behavioral learning was operationalized in two ways: (a) the rate at which subjects asked questions during the construction period, and (b) the percent of statements (feedback) given by subjects which were concurrent with the instructions given by assistants. Each subject was provided with the same written instruc- tions on how to complete the transfer task (see Appendix). These instructions were carefully worded to indicate that (a) one could interrupt the script at any time to ask questions, and (b) that there was some motivation for completing the task. No subject completed the task in the permitted ten minutes. Instructions were vague enough to encourage questioning. For each of the indicators described above, utterances were first divided into feedback and nonfeedback utterances. Utterances which were not related to the task were not counted as feedback. Rate of feedback was calculated by dividing the total number of feedback (task-related) utterances by the total time in minutes which it took to "complete" the task. This formula standardized the number of utterances a person made for the overall amount of time they took to finish the task. The procedure adjusts for the overly talkative or the overly quiet person. For example, using this formula, a person making two utterances in four minutes would get the same rate score as a person making four utterances in eight minutes. 43 The percentage of concurrent feedback was calculated by dividing the total number of feedback utterances made prior to the completion of the first reading of the instructions by the total number of utterances a person made overall. This measure is meant to reflect the degree to which subjects interrupted the detailed instructions to ask for clarification and/or report on their progress and under- standing. The instructions took approximately two minutes to read. Following the completion of the instructions, subjects continued to ask questions and requested that parts of the script be reread. Both measures reflect the main thrust of the lesson, that feed- back should be short and frequent (reflected in the rate), topic- related (reflected in the selection of feedback statements), and concurrent with the message received (reflected in the percent of concurrency). Even so, reflection of the main thrust of the lesson does not imply that all of the principles of the lesson were reflected in the transfer task. Students who learned some of the principles might not perform well on the task, or equally well as the students who have learned nothing. An attempt was made to get at an overall assessment of the principles involved that would be relatively easy to administer. Much attention was paid to training the research assistants in the administration of the transfer task. This was done to insure that no one administrator systematically affected the subjects' responses. The task was pilot tested to determine the appropriate difficulty level to insure that subjects asked questions. There was 44 no past usage of either the rate or concurrency measure, hence reliability was evaluated post hoc through the examination of multiple indicators. Experimental Design A 2x2 (two-factorial) posttest only control group design was used in the study. Criteria for selection of this design were that (a) since it is a true experimental design, it controls fOr common threats to internal and external validity (Campbell & Stanley, 1966), (b) other researchers have made strong arguments for increased adoption of this design (Kiestler, gt_a1,, 1969), and (c) since a posttest only design can be used with random assignment to groups to reflect change in knowledge or behavior over time, or learning. A pretest was not employed since it would have introduced reactivity into the design. Subjects were randomly assigned to each of the five groups using a random number table. The following treatment groups were specified to test the effects of indirectness and inductive presentation on the three kinds of learning: I. Indirect, inductive presentation. II. Indirect, deductive presentation. III. Direct, inductive presentation. IV. Direct, deductive presentation. V. Control (no game or debriefing). In condition I, the teacher builds on students' ideas in developing general principles. For each main point covered, students were first asked probing questions about the game experience and were 45 then encouraged to draw conclusions about the experience. Emphasis was on the use of students' ideas first in relating specific experiences and then in making generalizations. In condition II, students were encouraged at the outset to make generalizations and inferences before reporting specific experiences for each major point. As each concept was developed, students were encouraged through probing questions to first make inferences about the general significance of the game and then to cite specific examples from the game or their own experience. Condition III combined a lecture (direct) format with the inductive presentation of ideas. For each major point, the teacher first gave specific examples from the game or outside experiences, and then moved toward the development of a more general principle. The emphasis was entirely on the instructor's input with an inductive organization. In condition IV, the teacher also lectured, but introduced generalizations prior to specific examples for each major point. In condition V, there was no game or debriefing. The students moved directly to the posttest battery. The next section provides the results of this investigation. CHAPTER III RESULTS In this chapter, the procedures outlined in Chapter Two are used to test the seven hypotheses specified in Chapter One. Speci- fically, this chapter includes (a) a general discussion of the appli- cation of a full information, maximum likelihood technique of para- meter estimation to a system of structural equations, (b) descriptive statistics for each variable before and after transformations to meet the assumptions of analysis, (c) comparisons with the control group for each relevant variable, (d) results of the manipulation checks, (e) an overall evaluation of the goodness-of-fit of the model to the data, and (f) tests of each of specific hypotheses of interest. STRUCTURAL EQUATION MODEL A system of simultaneous structural equations was specified to reflect the relationships among the variables described in Chapter One. Although especially useful with non-recursive models, structural equation modeling has some advantages over more traditional methodo- logies even when used in conjunction with a recursive system. First, an overall test of the goodness-of-fit of the model can be performed. If the fit is not satisfactory, the model can be respecified, improved, or rejected. Second, in addition to the estimation of theoretical 46 47 relationships, a measurement model may be included such that multiple indicators of true variables may be evaluated simultaneously with all other relationships in the model. This is preferable to the common practice of losing information through the creation of indices. Structural equation modeling can be applied to both experimental and non-experimental designs (Costner, 1971; Bagazzi, 1977). In practice, parameter estimates may be generated using maximum likelihood estimation, with a program such as LISREL (Linear struc- tural relationships; Joreskog & Sorbom, 1978). These estimates are used to reconstruct a variance-covariance matrix of the original variables. This reconstructed matrix is then subtracted from the input covariance matrix to yield a residual matrix, the examination of which is one indicator of where the model is working well, and where respecification is necessary to fit the data. DESCRIPTION OF VARIABLES Descriptive statistics for all of the variables in the model are provided in Table 2. The indicators of affective learning (measured with magnitude estimation techniques) were positively skewed. This was corrected for in the usual fashion by a natural log transformation. which made all of the distributions more normal (Hamblin, 1975). The transformed statistics for these variables are presented in Table 3. All further discussion and analyses involving affective indicators use these transformed values. Two criteria can be employed to assess the value of multiple indicators. First, they should be normally distributed to satisfy the 48 zowumucmmmca -- 000.0 000._ 000.0 000._ 000.0 000.0 00 00000000000 -- 000.0- 000._ 000 0 000.0 000 0 000.0 000000020000 0F0.0 000._ 000.0 000.0 000._ 000.0 000.0 -HWM0WM0W0WMMMMMW0 _00.0 000.0 000.0 000.0 000.0 0_0.F 000.0 00HM0MMW0 000.0 000.0 000.000 000.0 000.000 000.00 000.00_ 000wwwmww< 0__.00 000.0 000.000 000.00 000.000 000.00_ _00.000 "wwmmw< 000.00 000.0 000.000 000.00 000.000 000.00. 000.000 awuwwww< 00_.0- 000.0- 000.00 000.0_ 000.0, 000.0 000.00 wnwmnmmw0 mvmoucsx mmmczmxm Eaewxmz 5:20:02 magma :mwwwhwmm cam: mpamwe0> .mwpnmwem> vmecoemcmcpcz eom 000u0000u0 m>0000eumoo .N mFth .00000> 002000000 uFo>0 op AF. + mpamwe0>v one; Ego» on» 0:00: swampsupmu men 0m000000> Pp .0eoumuwucw chow>0gmn 0:0 m>vpume$0 00m0000e0> cmecoymcmgu gee muwumwpmum m>0000000mo .m «Fame 50 assumptions of the analysis. Second, they should be linearly re- lated to one another, since they purport to be measuring the same underlying variable. Once the distributions of the affective indi- cators were transformed to be less skewed, the correlations among them were high and positive (see Table 4). Table 4. Correlations among multiple indicators. Teacher Satisfaction WITH Topic Satisfaction r = .4469 p = .0002 Teacher Satisfaction WITH Structure Satis- r = .8657 p = .0000 faction Topic Satisfaction WITH Structure Satis- r = .5031 p = .0000 faction Rate of Feedback WITH Percent Concurrence r = .3049 p = .0089 Behavioral learning was measured in terms of (l) the rate of feedback, and (2) the percentage of utterances concurrent with the initial instructions given by the assistant. The same two criteria for assessing multiple indicators stated above were applied here. While rate of feedback had a relatively normal distribution, percentage of concurrent utterances was positively skewed. Various transformations were tried (Mosteller & Tukey, 1979), and ultimately the natural log transformation was found to be appropriate. A small constant was added to each ratio previous to taking the logarithm to avoid undefined values of the transformed variable. Intercorrelation between these variables were positive but not as strong as for the affective indicators. 51 Table 5 displays the means for each of the endogenous variables by cell (treatment group). Since no treatment was provided for subjects in the control group, it was not meaningful to assess their affective learning. In examining the data, one can perform a variety of tests of decreasing generality, many of which give similar results. First, comparisons between the experimental and the control group were made for each relevant variable. Other comparisons specified by the hypo- theses were tested via parameter estimation of the specified struc- tural equation model, which gives information nearly identical to the analysis of variance solution (Burke & Schuessler, 1973). The advantage is that the parameter estimates for the whole model take into account more information in their calculation. Comparisons of treatment to control groups are displayed in Table 5. The critical value for difference between groups was set at p_< .05. for cognitive learning, the control group scored signi- ficantly less than all other groups. This indicates that all of the treatments had some kind of effect. A significant difference (F (4,75) = 4.396, p_< .005) among all of the groups is due mainly to the low score of the control group. The control group scored significantly less than all experi- mental groups for the first but not the second indicator of behavioral learning. A significant overall difference (E_(4,69) = 3.645, p_< .01) is due mainly to the control group and also to the main effect of inductive presentation on behavioral learning which is discussed later in this chapter. 52 .mo. v.m mazoem pcosummcu cacao seem 000000000 00pc00000000m0 0000.0- 0000.00 0000.000 0000000 0000.0- __00.0 0000.0 0000.0 0000.0 0000.0, 00>»wuwmmmmwwwmwmw 00_0.0- _000.0 0000.0 0000.0 0000.0 __F0.00 A000wwn0MMMwwunwumw 0000.0- 0000.00 0000.00 0000.0 0000.0 0000.00 A0>Ww0nwmflwywwmwmw 0000.0- 0000.0 0000.0 0000.0 0000.00 0000.00 A00000u0flwwwwunwwuww 00000000 00000000 000000000 000: 00000 0003 0000000 0003 00000000 0000000000 mo 0000 coppummmwumm :o0uu0wmwpmm cowuommmwumm m>0p0cmou mo mmmucmuema .Apocpcoo mcwuzpucwv 000000 000 000 00000000> maocmmovcm 000 we 0:00: .m m—nMH 53 The assumption of uncorrelated errors of prediction ordinarily made in regression analysis was tested for this model. The model which fixed the correlations among these errors to be zero fit slightly better than the one which allowed them to be correlated (x2/df = 2.36; ledf = 2.67). This, plus the fact that none of the estimated correlations among the errors of prediction were significant at the p.< .05 level, indicates that one is justified in assuming uncorrelated errors of prediction in this model. Manipulation Checks Both manipulation checks provided unequivocal support for the effectiveness of the manipulation. Coders using Flanders' system found a large difference between indirect and direct treatments (see Table 6). Inductive presentation was less easy to assess. Two experts in classroom communication and education were asked to rate each group presentation on an inductive to deductive seven-point scale. These raters were blind to which group received which manipu- lation. Experts were provided with conceptual definitions of each type of presentation. In every case, they marked the extreme end of the scale at the point where the manipulation was intended to be. Overall Test of the Model The first test performed was for the overall goodness-of-fit of the model to the data, also called H This hypothesis says that 1. the matrix of regression coefficients is not equal to the null matrix. The x2 value with 16 degrees of freedom is 37.7627 with a corresponding probability of .0016, which is the probability of the 54 Table 6. I/D ratios for experimental groups and averages across three coders. Coder l Coder 2 Coder 3 Mean Experimental I (Indirect/Inductive) 3-3 4.1 3.3 3.6 Experimental II (Direct/Inductive) 0 0 0 0 Experimental III (Indirect/Deductive) 3.5 4.4 7.5 5.1 Experimental IV (Direct/Deductive) 0 0 0 0 research hypothesis occurring. Although this probability is clearly insignificant, the x2 statistic depends a great deal on sample size, and in this case (in the context of an overall test) it is extremely sensitive to misspecification. Joreskog (1978) suggests that the most sensible criterion for assessing a model's fit is the x2 to degrees of freedom ratio. If the ratio is less than five (Monge, 1979), this means that for that number of degrees of freedom there is a good possibility that the model could be respecified to fit the data. The x2/df ratio for this model is 2.36. Attempts at respe- cifying the model for a better fit are reported in Chapter Four. The next section deals with the specific research hypotheses in the study. Specific Hypotheses Six hypotheses concerning specific parameters were advanced and tested; three were supported. Parameter estimates and standard 55 error for all relationships in the model appear in Figure 3. Para- meters which are statistically significant are marked with an asterisk and their corresponding probability values are listed. The amount of variance explained by the predictor variables in each endogenous variable is reported in terms of R2 values for each equation (Table 7). The exogenous variables explain the most variance in affective learning. Table 7. Variance accounted for in the endogenous variables. EQUATION ONE: n] = 8115] + 81252 + c] (R2=.OO8) cognitive learning EQUATION TWO: 8 E + :2 (R2=.245) affective learning 4. 1 B22€2 2= . _ . 315] + 83252 + :3 (R .124) behav1ora1 learning n2 EQUATION THREE: n3 = e 21 This hypothesis states that indirect teaching will lead to increased cognitive learning. This hypothesis was not supported at the p_< .05 level. H3: 82] > 0 This hypothesis proposed that indirect teaching will lead to greater affective learning. This was supported by the data at the E.< .025 level. H4: 83] > 0 The causal relationship proposed from indirectness to increased behavioral learning was not supported in this study. Figure 3. 56 Parameter estimates and standard errors for a model of the effects of indirectness and inductivity on learning. 51 152 indirectness inductive presentation cognitive learning affective learning (teacher) affective learning (topic) affective learning (structure) behavioral learning (rate of feedback) behavioral learning (concurrency of feedback) error error, error error error error error error error of prediction in n1 of prediction in n2 of prediction in n3 of measurement in l of measurement in of measurement in Y Y of measurement in Y Y of measurement in Y Y 0301-th of measurement in 57 mm m mczmwm .mmmmcpcmgma cw umpxogmg mew mmme:Wmm mo mcocgm vgmuchmkk .8. v m em 28:22... 33..» 8:365 8:22? am: rm :3. T1 . 25.3 m 2%.. am: e *m..m.~ *mmm. as; $132.» EN. - Tl «.mzl I Aom_.v mop.r 58 “5‘ 812 > This hypothesis states that inductive presentation should bring 0 about an increase in cognitive learning. This hypothesis was also not supported. Both parameters estimating the effects of the exo- genous variables on cognitive learning were insignificant and had large standard errors associated with them indicating extremely unstable estimates. Most of the variance in cognitive learning was left unexplained by the model. This hypothesis states that there is a causal relationship from inductive teaching and affective learning, such that an increase in inductive teaching leads to an increase in affective learning. This hypothesis received support at the p < .005 level. This hypothesis states that inductive teaching will lead to increased behavioral learning. This hypothesis was supported at the p < .025 level. CHAPTER IV DISCUSSION ' The purpose of this chapter is to interpret the findings of this investigation, discuss the problems and implications of the research, evaluate the procedures employed, and suggest directions for future research. SUMMARY OF FINDINGS Overall Model The ledf ratio was 2.36, which falls well below the rule 0f thumb criterion of 5.0. This indicates that the model might be 2 respecified to fit the data. A large drop in x , compared to the change in degrees of freedom, would indicate that a real improvement 2 and had been made. Reductions of similar proportion in both x degrees of freedom indicate that the apparent improvement in fit is in fact capitalizing on chance (Joreskog & Sorbom, l978). While the probability for this model occurring is very low (p_< .00l6), even a minor misspecification can disturb the fit since the omnibus x2 test is very sensitive to (a) relationships hypothesized to be significant which turn out not to be, and (b) relationships hypothesized not to be significant which turn out to be. 59 60 §pecific Hypotheses Causal relationships were found to exist from both exogenous variables to affective learning. The relationship from indirect teaching to an increase in positive affect was consistent with findings from studies assessing learning from gaming simulation discussed in Chapter One (Cherryholmes, 1966; Chartier, l972). The relationship from inductive presentation to affective learning is less well supported in the literature, but it follows that if ideas are made relevant to the student, this should be appreciated through increased satisfaction with the course, teacher, or class topic. Additional confidence in the relationships among indirectness, inductive presentation, and overall satisfaction can be obtained from the significant* positive loadings of each of the indicators of affective learning on the true variable, as well as from the overall 3? of .245. The strongest conclusion to be reached here is that teacher message strategies which either encourage student input (and use that input constructively) or organize ideas inductively promote more positive affect than those which do not. Inductive presentation was found to be a good predictor of behavioral learning. Teachers who organize the presentation of each concept in a debriefing session from the specific to the general (simple to complex, concrete to abstract) are more likely to have students retain and apply the desired behaviors (on a short-term basis) than those who teach deductively. While this finding is not applicable to educational situations where cognitive abilities are fp_< .05 61 the sole objective, it is critical to much communication education which emphasizes behavioral outcomes. In this research, indirectness and inductive presentation were both significant predictors of affective learning in the context of the gaming simulation. Indirectness and inductive presentation made no significant difference in facilitating cognitive learning as operationalized in this study. Inductive presentation was found to be preferable to deductive teaching in the teaching of behaviors which would be manifested at a later point in time. PROBLEMS AND LIMITATIONS This investigation differs from others in instructional communi- cation research in that (a) behavioral learning was measured through simulated transfer tasks, and while not a perfect operationalization of real-world tasks, this is considerably better than measures of behavioral intention or commitment, (b) multiple indicators of affective and behavioral learning were tried and evaluated as poten- tial operationalizations of the true variables of interest, (c) magni- tude estimation was used to measure affective learning, yielding continuous, ratio-level data, and (d) an attempt was made to pilot test and develop a short, reliable true/false examination to measure cognitive learning. Problems were encountered in assessing the reliability of the true/false test. When administered to students with no previous knowledge of the topic being tested, it appeared that random guessing ensued since the reliability was almost zero (.04). This problem was 62 ultimately handled through the administration of a short lecture on the topic before testing the pilot group. No one participating in the pilot studies also participated in the experimental study. In addition, pilot data was not used in any later analysis. While reliability improved following the lecture (.76), it was still not particularly high. It would seem that 22 items is too few for a true/false test if a high reliability (greater than .80) is desirable. The transfer task used to assess behavioral learning had measure- ment problems. First, the task only tested a subset of what was taught in the gaming simulation and debriefing. Students who learned only tangential issues would not perform well on the task. Second, variables such as spatial ability may have had an effect on the frequency and amount of feedback necessary to complete the transfer task. An attempt was made to control for this a_prjgri via an extremely ambiguous task, but spatial skills still may have made a difference in the number of questions subjects needed to ask or the timing of those questions. Due to numerous cancellations by volunteer subjects, the pre- servation of random assignment to groups may be questioned. There was, however, no apparent reason to expect that "dropouts" from groups were systematic occurrences. The interactive nature of the manipulation (students' input was used in two of the treatments) made it impossible to ensure perfect standardization of the debriefing across all experimental conditions. A serious attempt was made (via a scripted lecture outline) to keep each debriefing identical in content. Anecdotal observation by research assistants and experts indicated that the manipulation was 63 standard from experiment to experiment. Teacher-generated examples differed from student-generated examples, however, and this could conceivably affect comprehension. A The judgment was made to reduce reactivity by administering the behavioral measure previous to the cognitive measure. This judgment could have been evaluated better if half of the tests were adminis- tered in this order and half in the reverse order. Examination of the path diagram above (Figure 3) provides insight into where the model is working well, and where revisions could be made. These coefficients and corresponding standard errors reveal two major problems with the model. The first is in the pre- diction of cognitive learning. 0f the l6.2l0 units of variance in cognitive learning, l6.084 remains unexplained by the exogenous variables in the study (t_= 5.788, p_< .00l). The model needs to be revised in some way to better account for the variance in cogni- tive learning. Different operationalizations of cognitive learning could be tried and evaluated. Alternatively, additional exogenous variables could be added to the model which might lead to better prediction. The functional form of the hypothesized relationships could be examined and respecified as something other than linear associations. Lastly, the null hypothesis that inductive debriefing makes no difference in cognitive learning from gaming simulation may be accepted, and the model revised accordingly. The second major problem is evident from the covariance matrix of errors of measurement in the endogenous variables. Significant amounts of variance (t_= 5.446, p_< .025; t_= 5.559, p_< .025; t_= 5.170, p.< .025; all two-tailed tests) are left unexplained in 64 the affective measures of topic and structure, and the behavioral measure of percent of concurrent feedback. This finding suggests that (a) the assumption of uncorrelated errors of measurement may be unfounded and/or (b) some relationship exists (affective to behavioral learning, for example) whose specification would explain this variance. Assorted revisions could be tried to better fit the data to the model. Major ones would include respecifying relationships involving cognitive learning, or those which would affect the co- variances among the errors of measurement. Revised models would be assessed by comparing the new ledf ratio to the old one. A smaller ratio indicates an improvement in the overall fit of the model to the data. Model Evaluation and Revision The functional form of each bivariate relationship in the model was examined for non-linear association. One-way analysis of 2 for each relationship. Values variance was performed to yield E of E2 (combination of linear and non-linear association), R2 (linear association alone) and their difference are reported in Table 8. Degree of non-linear association was in each case trivial; hence the hypothesized relationships were not transformed before attempting a better fit of the model to the data. In an effort to respecify the model to best fit the available data (not a test of hypotheses but an attempt to provide the "best" model for this data set) various strategies were tried. First, three paths originally specified were deleted from the model. This 65 Table 8. R2, E2, and E2 - R2 for all bivariate relationships: test for non-linear association. Relationship R2 E2 E2 - R2 X1 with Y] .0144 .0148 .0004 X] with Y2 .0625 .0656 .0031 X] with Y3 .0196 .0198 .0002 X1 with Y4 .0784 .0934 .0150 X] with Y5 .0576 .0578 .0002 X] with Y6 .0576 .0585 .0009 X2 with Y1 .0100 .0106 .0005 X2 with Y2 .1849 .1872 .0023 X2 with Y3 .0094 .0249 .0155 X2 with Y4 .0135 .0187 .0052 X2 with Y5 .0484 .0499 .0015 X2 with Y6 .0144 .0153 .0009 66 was done because indirectness had little impact on behavioral learning, and because neither of the exogenous variables seemed to make a difference in cognitive learning as operationalized in this study. In addition to the absence of linear relationships, the lack of significant non-linear association among these variables lent further support to the decision to drop these paths from the model. This revised model fit less well than the original one (x2 = 41.195, p_= .0002, ledf = 2.94). The major problem was still in the signi- ficant correlations among errors of measurement in the indicators of the endogenous variables. Significant and correlated errors of measurement might result from (a) poor instrumentation (unreliable), (b) an unspecified relationship among the true or observed variables in the model, or (c) an unspecified variable which might effect the endogenous variables in the model in a similar way. It was proposed that behavioral and affective learning might not be independent outcomes in this study, but rather that one influences the other. Affective learning may precede and impact upon the potential for behavioral learning, or vice versa. Both of these revisions were tried. In each case, estimated regression coefficients were not in the direction expected (e.g., positive affect should lead to increased behavioral learning), and the resultant XZ/df ratio was not an im- provement over the original one (xz/df = 2.88 for affective to behavioral; XZ/df = 3.01 for behavioral to affective). It would appear that with this small number of variables, the theroetical limits of what can be meaningfully respecified have nearly been reached. Post hoc explorations did not yield a model with better 67 fit than that originally specified. Better measurement (especially of the behavioral variable) and the addition of exogenous variables which might effect both endogenous variables in the model might lead to improved prediction and fit. The next section describes some possible implications of the findings of this study. IMPLICATIONS FOR PRACTICE The use of communication gaming simulations is far from an automatic guarantee that students will learn well; rather it appears from this study that the type of debriefing style will make a major difference in learning effects. Inductive and indirect debriefing styles lead to higher positive affect from students than do deductive or direct. In addition, the teacher who desires behavioral learning should organize his presentation inductively. Although this com- parison was not made explicitly, this research suggests that it is not important who (teacher or students) comes up with the ideas, as long as they are inductively arranged. Further work with this inter- action should be done before generalizations can be made for practice. Finally, it is still unclear, contrary to convincing arguments presented at the beginning of this paper, whether or not communi- cation games debriefed in any fashion make a difference in facili- tating cognitive learning. These results can be applied to educational practice in communi- cation differently depending on one's educational goals and philo- sophy. What is clear from this research is that the widespread use of gaming simulation as a teaching tool may be inappropriate in many 68 situations, or at least needs to be examined to determine the correspondence between types of objectives and types of debriefing. Previous research has demonstrated consistently that games without any accompanying discussion teach no better or worse than tradi- tional methods, and many teachers still conduct games without follow-up. While this approach does not maximize learning from games, this study has demonstrated that the addition of a debriefing session does not guarantee maximization either. Debriefing style must be matched against the potential goals of the activity. The teacher must decide whether cognitive, affective, or behavioral learning is of prime importance, and then choose a type of debrief- ing (if a game is chosen at all) which best suits his or her goal. These adjustments in teaching style are relatively simple to accom- plish with some practice. Inductive presentation is mainly a matter of outlining a presentation differently; many studies have shown the ease of training teachers to be indirect, and to monitor themselves in practice. Finally, this study has implications for models of communication effectiveness in general. Many training programs exist which attempt to teach students or employees to be more effective communicators and most of these programs use gaming simulation as a teaching tool. The finding that inductive debriefing leads to increased behavioral transfer has two implications for these programs: first, an induc- tive debriefing style should be used if the objective is to train more effective communicators, and second, that any kind of paper and pencil test of verbal-cognitive abilities administered as evaluation (of the program or the individuals) may be misleading in its results. 69 The potential for discrepancy between verbal-cognitive and behavioral competencies was discussed earlier in reference to the Kenyan sojourners who performed well on paper but failed overseas (Ruben, 1976). The predictions of affective learning received strongest support in this study and appear strongest in past research on learning effect from gaming simulation. Some classroom situations require more positive affect than others, and some teachers (and students) have greater need for high positive affect from students. In a relatively long-term interaction like the classroom, consistent use of deductive, direct debriefing following a stimulating game may lead to long- standing frustration and resentment. Thisimplication can be criti- cally important in situations where negative affect may inhibit other kinds of learning (via mental blocks or emotional reactions). When affective learning plays this central a role, indirect and inductive debriefing should be used to better enable students to learn cogni- tively and behaviorally. The strategies of indirectness and inductive presentation are well known in forensics and debate. The popular expression "let him talk until he hangs himself" may seem at first at odds with the goals of education, but the effects are similar. The persuadee provides the input, makes the necessary connections, and eventually teaches himself. The persuader guides the discussion skillfully to ensure that certain areas are explored. Similarly, the student makes relevant his or her experiences and the teacher provides a context for the desired connections to be made. Inductive presentation in debate is a way of getting the opponent to agree with a series of 7O premises such that he must logically accept the main argument; in education the process is the same but the goal is increased under- standing through building a logical framework. It should be clear that with some practice, teachers who use gaming simulation can make changes in their debriefing style and evaluation procedures to be consistent with the findings of this study. Further, if these findings could be generalized beyond the classroom or gaming context, it would be interesting to see in which contexts indirect or inductive presentation might be inappro- priate or facilitate negative affect. Since satisfaction and behavioral outcomes are critical issues in organizations, or in the case of mass public appeals (or more intimate persuasion settings), studies which attempt to generalize these findings to those settings might uncover some useful generalizations. SUGGESTIONS FOR FUTURE RESEARCH The major limitation of this study is its cross-sectional design. The classroom situation is a dynamic one--specific teaching styles may lead to changes in affect, which then may effect learning; consequently, the degree to which students have learned may affect teaching style. It seems that educational models should deal with a process as it evolves over time, and as the endogenous variables affect one another. Further emphasizing the processual nature of classroom interaction, it appears that students and teachers alike have rigid expectations of what classroom interaction is like; this is evidenced by the regularities in participation and seating styles 71 found in nearly every schoolroom. A longitudinal study needs to be done which incorporates the expectations teachers and students have, measures effects of particular communication strategies at an initial point in time, and then follows the interwoven effects of strategies on learning and learning on strategies as they change over time. Does learning increase over time, or become extinguished, given some amount of repetition? Time lags for assessing changes in affect and other learning need to be established. Structure of presentation rather than origin of messages seemed to make the key difference in facilitating behavioral learning from communication games. Interaction between these and other conceptually similar variables should be studied in an effort to develop an ideal profile of the effective facilitator of a communication game. Magnitude estimation seems a viable way to measure satisfaction. Subjects expressed no difficulties in making magnitude estimates when given written instructions and some oral clarification. Future studies should experiment with not bounding the scale at the bottom (0) and should continue to allow subjects time to ask questions concerning the logistics of the measurement technique. It is not clear whether written instructions alone would have been sufficient in administrating the technique. There is no further need to measure the effects of gaming simu- lation and indirect or inductive teaching styles on affective learning. Hypotheses of this form have been confirmed numerous times in the literature and have been once again in this study. It is clear that gaming simulation will result in positive affect. 72 Different theoretical perspectives might have aided this research in the generation of propositions and the interpretation of findings. Since the order in which ideas are presented is commonly manipulated in persuasion research (primacy recency, for example), as well as the degree to which the persuadee is actively encoding (counter attitudinal advocacy; Miller & Burgoon, 1975), future research in this area might tie more directly into this literature. This study should be replicated with better measurement. Multiple choice questions are recommended for the cognitive test, since they are maximally efficient (less cumbersome than essays) and highly reliable. Additional variables which might affect both affective, cognitive, and behavioral learning should be specified. Behavioral tasks should move in the direction of real-world tasks whenever feasible, since behavioral learning should be a serious con- viction of the educator who purports to prepare a student to be a better communicator in the world outside of the classroom. REFERENCES REFERENCES Abt, C. C. Serious games. New York: The Viking Press, 1970. Alwin, D. F., and Tessler, R. C. Causal models, unobserved variables, and experimental data. American Journal of Sociology, l974, 89; 58-86. Amidon, E. J., and Hough, J. B. Interaction anaLysis: Research, theory, and application. Reading, Mass.: Addison-Wesley, 1967. Ausubel, D. P. Cognitive structure and the facilitation of meaningful verbal learning. Journal of Teacher Education, 1963, 14, 217-230. Bagozzi, R. P. Structural equation models in experimental research. Journal of Marketinngesearch, 1977, 14, 209-226. Baker, E. H. A comparative study using textbook and simulation approaches in teaching junior high school American history. UnpubTished doctoral dissertation, NorthWestern University, 1966. Bellack, A. A., et al. The language of the classroom: meanings com- municated in high school teaching. USOE Cooperative Research Project No. 2023, Institute of Psychological Research, Teachers College, Columbia University, 1965. Bochner, A., and Kelly, C. W. Interpersonal competence: rationale, philosophy, and implementation of a conceptual framework. Speech Teacher, 1974, 23, 1-23. Boocock, S. 5. Effects of election campaign game in four high school classes. Research program on the effects of games and simulated environments in secondary education. Report No. 1. Baltimore, MD.: Johns Hopkins University, Department of Social Relations. Boocock, S. S. and Schild, E. 0. Simulation games in learning. Beverly Hills, Calif.: Sage Publications, 1968. Bruner, J. S. Toward a theory of instruction. New York: Norton, 1966. Burke, P., and Schuessler, K. Alternative approaches to analysis of variance tables. Sociological Methodology, H. L. Costner (ed.), 1973-1974. 73 74 Campbell, 0. T., and Stanley, J. C. Experimental and quasi-experi- mental designs for research. Chicago: Rand McNally, 1966. Chartier, M. R. Learning effect. Simulation and Games, 1972, 3, Cherryholmes, C. H. Some current research on the effectiveness of educational simulations: implications for alternative strategies. American Behavioral Scientist, 1966, lQ_(2). Clark, W. A research note on simulation in the social sciences. Simulation and Games, 1970, 1, 203-210. A Costner, H. L. Utilizing causal models to discover flaws in experi- ments. Sociometry, 1971, 34, 398-410. Dale, E. Audio-visual methods in teaching. New York: Dryden Press, 1969. DeCecco, J. P., and Crawford, W. R. The psychology of learning and instruction. Englewood Cliffs, N.J.: Prentice-Hall, 1974. Dewey, J. Experience and education. London: Collier-MacMillan Co., 1938. Duncan, 0. 0. Introduction to structural equation models. New York: Academic Press, 1975. Ebel, R. Essentials of educational measurement. Englewood Cliffs, N.J.: Prentice-Hall, 1972. Elliot, T. C. The effects of an intercultural communication unit in the basic speech course. Paper presented at the meeting of the International Communication Association, Chicago, 111., May 1978. Emery, E., and Enger, T. P. Computer gaming and learning in an intro- ductory economics course. Journal of Economic Education, 1972, 3. 77-85. Fink, E. L., and Huber, K. C. Self-awareness and definitional processes within dyadic interaction: Simulation of a dynamic system. In X. J. R. Avula (Ed.), Proceedings of the First International Conference on Mathematical Modeling, V61. 5. Rolla, Missouri: University of Missouri-Rolla, 1977. Flanders, N. A. Analyzing teaching behavior. Massachusetts: Addison- Wesley, 1970. Flanders, N. A., and Simon, A. Teacher effectiveness. In R. Ebel (Ed.), Encyclopedia of Educational Research. New York: Mac- millan Co., 1969, 1423-1431. 75 Fletcher, J. L. The effectiveness of simulation games as learning environments. Simulation and Games, 1971, 425-454. Friedrich, G., and Brooks, W. The use of systematic observation instruments for the supervision of teaching. Speech Teacher, 1970, 283-288. Furst, N. F. The multiple languages of the classroom. Paper pre- sented at the meeting of the American Educational Research Association, Chicago, I11., February 1967. Gagne, R. M. The conditions of learning. New York: Holt, Rinehart, and Winston, 1966. Greenblat, C. Gaming-simulation. New York: Sage, 1975. Gunnison, J. P. An experiment to determine the effects of changing teacher classroom behavior through training of student teachers in the use of Flanders' interaction analysis category system. Unpublished doctoral dissertation. Tempe, Arizona: Arizona State University, 1968. Hamblin, R. L. Social attitudes: magnitude measurement and theory. In H. M. Blalock, Jr. (Ed.), Measurement in the Social Sciences. Chicago: Aldine Publishing Co., 1974. House, P. Gaming at equilibrium--some observations. Simulation and Games, 1973, 4, 454-458. Joreskog, K. 0., and Sorbom, D. LISREL: Analysis of linear struc- tural relationships by the method of’maximum likelihood. Chicago: National Educational Resources, Inc., 1978. Kiestler, C. A., Collins, B. E., and Miller N. Attitude change. New York: John Wiley and Sons, 1969. Lashbrook, V. J., and Wheeless, L. R. Instructional communication theory and research: an overview of the relationship between learning theory and instructional communication. In B. D. Ruben (Ed.), Communication Yearbook 2. New Brunswick, N.J.: Transaction Books, 1978. Leavitt, H. J., and Mueller, R. Some effects of feedback on communi- cation. Human Relations, 1951, 4, 401-410. Lindblad, S. Simulation and guidance: teaching career decision- making skills in the Swedish compulsory school. Simulation and Games, 1973, 4, 429-439. Livingston, S., and Stoll, C. Simulation games: An introduction for the social studies teacher. New York: ’Free Press,71973. 76 Massialas, E., Sprague, N., and Hurst, J. Social issues through inquiry: Coping in an age of crisis. Englewood Cliffs, N.J.: Prentice-Hall, Inc., 1975. Miller, G. R., and Burgoon, M. New techniques in persuasion. New York: Harper and Row, 1973. Miller, G. R., and Nicholson, H. E. Communication inquiry, Massa- chusetts: Addison-Wesley, 1976. Monge, P. R. The problem of correspondence between systems theory and systems research in the study of organizational communi- cation. Paper presented at the annual meeting of the Speech Communication Association, San Antonio, Texas, November 1979. Morsh, J. E., and Wilder, E. W. Identifying the effective instructor: a review of quantitative studies, 1900-1952. Research Bulletin, Air Force Personnel and Training Center, San Antonio, Texas, 1954. Mosteller, F., and Tukey, J. W. Data analysis and regression. Massa- chusetts: Addison-Wesley, 1977. Myers, G. E., and Myers, M. T. Instructor's Manual. To accompany Communicating: (people speak. New YOrkziiMcGraw-Hill, 1975. Nambooderi, N. K., Carter, L. F., and Blalock, H. M. Applied multi- variate analysis and experimental designs. New York: McGraw- Hill, 1975. Pierfy, D. A. Comparative simulation game research. Simulations and Games, 1977, 254-268. Raser, J. R. Simulation and society. Boston: Allyn and Bacon, Inc., 1969. Rosenfeld, F. H. The educational effectiveness of simulation-games: a synthesis of recent findings. In C. Greenblat (Ed.), Gaming- Simulation. New York: John Wiley and Sons, 1975. Ruben, B. 0., and Budd, R. W. Human communication handbook: simula- tion and games. New Jersey: Hayden, 1975. Ruben, B. 0. Assessing communication competency for intercultural adaptation. Group and Organization Studies,_l976, 3, 334-354. Ruben, B. 0., Askling, L. R., and Kealey, D. J. Cross-cultural effectiveness: an overview. In David S. Hoopes (Ed.), Inter- cultural Communication: State-of-the-Art Overview. Pittsburgh: Society for Intercultural Education, Training, and Research, 1977. Scott, M. D., and Young, T. J. Personalizing communication instruc- tion. Communication Education, 1976, 25, 211-221. 77 Seibold, D. R. Communication research and the attitude verbal report- overt behavior relationship: A critique and theoretic reformu- lation. Human Communication Research, 1975, 3, 3-32. Shinn, A. M., Jr. Relations between scales. In H. M. Blalock (Ed.), Measurement in the Social Sciences. Chicago: Aldine Publishing Co., 1974. Simon, A., and Boyer, E. 0. Mirrors for behavior: An anthology of classroom observation instruments. Philadelphia: Research for Better Schools, Inc., and the Center for the Study of Teaching, Temple University, 1967. Stevens, S. S. 0n the psychophysical law. Psychological Review, 1957, 34, 153-181. Torgerson, W. S. Theory and methods of scaling. New York: John Wiley and Sons, Inc., 1958. Walling, J. I. An experimental study of conditions which affect learning from simulation games in Speech communication instruc- tion. Unpublished doctoral dissertation, University of Illinois, 1976. - Wing, R. Two computer based economic games for sixth graders. In S. Boocock, and E. 0. Schild, (Eds.), Simulation and Games in Learning. Beverly Hills, Calif.: Sage, 1968. Yelon, S. L., Davis, R. H., and Alexander, L. T. Learning system design. New York: McGraw-Hill, 1974. APPENDICES APPENDIX A CATEGORIES OF INTERACTION ANALYSIS (FLANDERS, 1970) 1.* ACCEPTS FEELINGS PRAISES OR ENCOURAGES ACCEPTS OR USES IDEAS OF STUDENTS ASKS QUESTIONS LECTURING TEACHER TALK GIVING DIRECTIONS \l as 01 h (A! N o o o o o o CRITICIZING OR JUSTIFYING AUTHORITY 8. STUDENT-TALK--RESPONSE STUDENT TALK 9. STUDENT-TALK--INITIATION IO. SILENCE OR CONFUSION *There is no scale implied by these numbers. Each number is classi- ficatory; it designates a particular kind of communication event. To write these numbers down during observation is to enumerate, not to judge a position on a scale. 78 APPENDIX B LESSON PLAN 1. Explain the nature of the game. a. Giving effective directions. b. Explain that they should each try to draw the figure from person giving instructions as best they can without making any sounds or gestures. c. Repeat game this time with allowing students to ask questions. d. Time recorded for each trial, as well as number showing good accuracy. II. Debriefing. a. When feedback is introduced into an interaction where one individual is giving directions, there is a tradeoff in the effec- tiveness of communicating the message and the time it takes (effi- ciency) of finishing the task. Interactions with feedback tend to be more time-consuming but produce more accurate results (effective but inefficient communi- cation). Non-verbal as well as verbal feedback is important. Although verbal feedback in the form of asking for specific clarification is 79 80 usually most critical, non-verbal grunts or groans, as well as para- linguistic cues can help to direct the sender's explication of the message. In addition, feedback can be unintentional as well as intentional. In the absence of feedback, the individuals who do not have the opportunity to ask for questions are likely to become frustrated. Conversely, the more feedback given, the more confident the receivers become in their successful progress with the task. Senders may be more or less confident as feedback increases. A distinction can be made between free feedback and zero feed- back. Free feedback is unlimited response to a message. While it may be constructive for a while, it is likely to become hostile and aggressive in some systems. Prolonged exposure to free-feedback sometimes results in overstimulation. In the case of zero feedback, where there is no opportunity to respond, the person receiving the instructions is most likely to be frustrated. Free feedback is more time-consuming, but produces more accurate results than zero feedback. Feedback is not restricted to living systems. It is a general concept which applies to all transactional systems. It is theore- tically impossible for humans to know what they are like in the total absence of feedback. We use it to build our self-image. In giving feedback to others, there are a number of recommen- dations which can be made to do so most effectively. It is gener- ally more productive to focus feedback on specific behaviors than on people. Critique the action, not the person. Similarly, it is often unjustified to give feedback concerning inferences you have made from observations. It is more constructive to give feedback 81 on specific isolated observations, which can be most easily corrected. This is to say that feedback should be as much as possible objective description of an event, not subjective judgment. Also, think ahead and try to predict how the other individual will respond to your comments. While feedback is one way of improving the effectiveness and efficiency of a task, it is likely that the accuracy with which a task is done will increase over repeated trials regardless of the amount of feedback. The point in time at which the feedback is provided makes a difference. Concurrent feedback helps to increase the rate and efficiency with which people translate their thoughts into speech. It is also more likely to give direction-givers confidence in the fact that their message is being received. Delayed feedback gener- ally works against speech fluency and confidence, and promotes a feeling of doubt. Whether feedback is positive or negatiVe effects whether it will increase desired behaviors. Speech fluency is adversely affect- ed by negative feedback. In a group discussion, negative feedback leads to a rapid shifting from one topic to another. Negative feed- back tends to work better in eliminating undesirable behaviors. A very common problem in youth counseling is that people tend to interpret even positive feedback as being negative. When people rise to positions of authority, there is a marked increase of aversion to negative feedback. I The kinds of feedback an individual gives in practice vary mainly due to the personality of the individual. Motivated people 82 are very likely to give feedback, where more apathetic or uncon- cerned people are not. APPENDIX C PILOT COGNITIVE AND AFFECTIVE INSTRUMENTS EFFECTIVE COMMUNICATION Imagine that you have just left a class where the topic of discussion was: "HOW CAN WE BEST UNDERSTAND PEOPLE WHEN THEY SPEAK TO US?" The major idea discussed was "FEEDBACK," and its influence on effective communication. 0n the next few pages, you will find true/false questions dealing with the notion of feedback. You have been provided with an answer sheet on which you should mark your responses. Please place your student number on both the answer sheet and this question sheet now. Some of the words used in the questions may be new to you. Try to figure them out and answer the questions as best you can. 00 not leave questions blank--make your best attempt to score the correct answer. IF A STATEMENT IS MAINLY TRUE, MARK 1. IF A STATEMENT IS MAINLY FALSE, MARK 2. Thank you very much for your help with this research. Eric M. Eisenberg 83 84 MARK 1 IF TRUE. MARK 2 IF FALSE. DO NOT LEAVE QUESTIONS BLANK. 10. 11. 12. 13. 14. 15. 16. Feedback can be unintentional. The more senses involved in giving feedback, the less likely that the message will be understood. A common result of detailed feedback is increased efficiency. An increase in the amount of feedback usually leads to a decrease in communication effectiveness. It is possible to give feedback that is neither verbal nor non- verbal, but vocal. Efficiency can be defined as the degree to which a receiver understands a task. It is generally more productive to focus feedback on people than on specific behaviors. Useful feedback does not usually dwell on isolated observations, but comes as a result of informed inferences. The use of detailed feedback is generally considered to be ineffective but more efficient. As the quantity of feedback given decreases, confidence in the task decreases. Feedback should be mainly objective description, not subjective judgments. The preferred way of giving feedback is cautious advice, so that it is clear who will take responsibility for the task. It is a generally good idea to attempt to predict how indivi- duals will respond to your feedback. When a person has been given an abundance of feedback, he or she often becomes less secure about his or her understanding of the message. It is probably impossible for us to know what we are like in the total absence of feedback. The accuracy with which a task is done will increase over repeated trials, regardless of the amount of feedback. 85 MARK 1 IF TRUE. MARK 2 IF FALSE. 17. 18. 19. 20. 21. 22. 23. 24. 25. 26. 27. 28. 29. 30. 31. 32. 33. A person giving directions to someone is more likely to be confident while doing so if there is an abundance of feedback. In an interaction involving a lot of verbal communication and near zero feedback, the person giving the instructions is most likely to be frustrated. Feedback is restricted to living systems. Free feedback is likely to be hostile and aggressive. Prolonged exposure to free-feedback rarely results in over- stimulation. Delayed feedback helps to increase the rate and efficiency with which people translate their thoughts into speech. Concurrent feedback is more likely to make direction-givers confident in their abilities than is delayed feedback. As feedback increases, understanding of the message increases. Negative feedback tends to enhance undesirable behaviors. In communication networks, positive feedback leads to the most modifications of verbal behavior. As feedback increases, the time it takes to complete the task decreases. In a group discussion, negative feedback leads to a rapid shift- ing from one topic to another. Increasing the amount of feedback given is one good way to cut down on the amount of time it takes to present a message. A common problem in some counseling situations is the client's interpreting negative feedback as positive. Aversion to negative feedback increases dramatically when people assume positions of authority. Speech fluency is adversely affected by both positive and nega- tive feedback. Receiving verbal directions without an opportunity for feedback is generally less frustrating than receiving directions with ample opportunity for feedback. 86 MARK 1 IF TRUE. MARK 2 IF FALSE. 34. Motivated individuals are more likely to listen closely to directions than they are to provide feedback. 35. The amount of feedback an individual gives is usually a function of their personality. 36. The effectiveness of a message is best defined as the time it takes to complete a task. 37. Free-feedback is more time-consuming, and produces more accurate results than does zero-feedback. 38. Non-verbal feedback is generally more important then verbal feedback in determining whether or not verbal directions will be understood. For the final six questions, mark your answers DIRECTLY ON THIS QUESTION SHEET. STUDENT # THINK OF THE LAST COM 100 CLASS YOU ATTENDED. RECALL THE TEACHER, THE TOPIC, AND THE STRUCTURE OF THE CLASS. FOR THE FOLLOWING THREE STATEMENTS, CIRCLE THE RESPONSE THAT BEST REFLECTS YOUR FEELINGS. SA - STRONGLY AGREE D - DISAGREE A - AGREE so - STRONGLY DISAGREE N - NEUTRAL 39. I AM SATISFIED WITH THIS TEACHER IN GENERAL. SA A N D SD 40. I AM SATISIFEO WITH THIS SA A N D so TOPIC (SUBJECT). 41. I AM SATISFIED WITH THE WAY THE CLASS IS STRUCTURED. SA A N D SD GO TO THE NEXT PAGE! 87 RECORD THESE RESPONSES DIRECTLY ON THIS SHEET. IMAGINE AN AVERAGE AMOUNT OF SATISFACTION TO BE 100 UNITS. THIS WILL BE YOUR MENTAL RULER. WHEN EVALUATING YOUR SATISFACTION, COMPARE IT WITH THIS STANDARD. TOTAL DISSATISFACTION WITH SOMETHING WOULD YIELD AN ESTIMATE OF 0. AVERAGE SATISFACTION WOULD BE INDICATED BY AN ESTIMATE OF 100. ALL POSITIVE NUMBERS ARE PERMISSIBLE AS ESTIMATES. THINK AGAIN OF THE SAME CLASS YOU DESCRIBED IN QUESTIONS 39-41. 42. IF 100 IS AN AVERAGE AMOUNT OF SATISFACTION WITH A TEACHER, HOW SATISFIED ARE YOU WITH THIS TEACHER? 43. IF 100 IS AN AVERAGE AMOUNT OF SATISFACTION WITH A TOPIC (SUBJECT), HOW SATISFIED ARE YOU WITH THE TOPIC OF THIS CLASS? 44. IF 100 IS AN AVERAGE AMOUNT OF SATISFACTION WITH CLASS STRUCTURE, HOW SATISFIED ARE YOU WITH THE STRUCTURE OF THIS CLASS? Thank you again for your cooperation. Please hand in both the question and answer sheet. APPENDIX D COGNITIVE INSTRUMENT Last four digits soc. security # EFFECTIVE COMMUNICATION PRETEST. MARK EACH STATEMENT AS TRUE OR FALSE. DO NOT LEAVE ANY QUESTIONS BLANK. 10. 11. Feedback is restricted to living systems. The more senses involved in giving feedback, the less likely it is that the message will be understood. An increase in the amount of feedback usually leads to a decrease in communication effectiveness. It is possible to give feedback that is neither verbal nor non- verbal but vocal. Efficiency can be defined as the degree to which a receiver understands a task. It is generally more productive to focus feedback on people than on specific behaviors. The use of detailed feedback is generally considered to be in- effective but more efficient. - As the quantity of feedback given decreases, confidence in the task decreases. A person giving directions to someone else is more likely to be confident while doing so if there is an abundance of feedback. In an interaction involving a lot of verbal communication and near zero feedback, the person giving the instructions is most likely to be frustrated. Free-feedback is likely to be hostile and aggressive. 88 12. 13. 14. 15. 16. 17. 18. 19. 20. 21. 22. 89 Prolonged exposure to free-feedback rarely results in over- stimulation. Delayed feedback helps to increase the rate and efficiency with which people translate their thoughts into speech. Concurrent feedback is more likely to make direction-givers confident in their abilities than is delayed feedback. As amount of feedback increases, understanding of the message increases. Negative feedback tends to enhance undesirable behaviors. In communication networks, positive feedback leads to the most modifications of verbal behavior. As feedback increases, the time it takes to complete the task decreases. Non-verbal feedback is generally more important than verbal feedback in determining whether or not verbal directions are understood. Receiving verbal directions with no opportunity for feedback is generally less frustrating than receiving directions with an extended amount of time for feedback. Increasing the amount of feedback given is one good way to cut down on the amount of time it takes to present a message. It is probably impossible for us to know what we are like in the absence of feedback. APPENDIX E AFFECTIVE INSTRUMENT Last four digits soc. security # _____ REFLECT ON THE CLASSROOM EXPERIENCE YOU RECENTLY HAD ON THE NATURE OF FEEDBACK. RECALL WHAT YOU THOUGHT OF THE TEACHER, THE TOPIC, AND THE STRUCTURE OF THE CLASS. FOR EACH OF THE FOLLOWING THREE STATEMENTS, CIRCLE THE RESPONSE WHICH best reflects your opinion. SA = STRONGLY AGREE D = DISAGREE A.= AGREE SD = STRONGLY DISAGREE N = NEUTRAL N0 = NO OPINION 23. I WAS SATISFIED WITH THIS TEACHER. SA A N D SD NO 24. I WAS SATISFIED WITH THIS TOPIC. SA A N D SD NO 25. I WAS SATISFIED WITH THE STRUCTURE SA A N D SD NO OF THIS CLASS. IT IS POSSIBLE TO BE SATISFIED WITH AN OBJECT OR AN EXPERIENCE TO A VARIETY OF DIFFERENT DEGREES. IF YOU HAVE A DAILY JOB, YOU MAY BE HIGHLY SATISFIED WITH IT, OR TOTALLY DISATISFIED. SOMEWHERE BETWEEN TOTAL SATISFACTION AND TOTAL DISSATISFACTION, THINK OF WHAT WOULD BE 90 91 AN "AVERAGE AMOUNT" OF SATISFACTION WITH YOUR JOB. IN EVALUATING ANY JOB YOU TAKE, IT WOULD BE POSSIBLE TO USE THE "AVERAGE AMOUNT“ AS A MENTAL RULER, A MEANS OF COMPARISON FOR AMOUNT OF SATISFACTION. IMAGINEJTHE AVERAGE AMOUNT OF SATISFACTION YOU EXPERIENCE WITH;; TEACHERS; CALL THIS 1OO UNITS. TOTAL DISSATISFACTION WITH A TEACHER WOULD BE INDICATED BY AN_ ESTIMATE OF 0. ALL POSSIBLE NUMBERS ARE PERMISSIBLE AS ESTIMATES. THINK AGAIN OF THE CLASSROOM EXPERIENCE YOU JUST COMPLETED. RESPOND TO THE FOLLOWING QUESTIONS WITH NUMBER ESTIMATES. 26. IF 100 IS THE AVERAGE AMOUNT OF SATISFACTION YOU EXPERIENCE WITH TEACHERS: HOW SATISFIED WERE YOU WITH THIS TEACHER? 27. IF 100 IS THE AVERAGE AMOUNT OF SATISFACTION YOU HAVE WITH A TOPIC: HOW SATISFIED WERE YOU WITH THIS TOPIC? 28. IF 100 IS THE AVERAGE AMOUNT OF SATISFACTION YOU EXPERIENCE WITH CLASS STRUCTURE: HOW SATISFIED WERE YOU WITH THIS CLASS STRUCTURE? THANK YOU AGAIN FOR YOUR PARTICIPATION. PLEASE BE SURE THAT YOU HAVE MARKED THE LAST FOUR DIGITS OF YOUR SOCIAL SECURITY NUMBER ON ALL SHEETS. APPENDIX F INSTRUCTIONS FOR BEHAVIORAL TASK In the envelope in front of you, you have some unassembled LEGO blocks. The person sitting with their back to you has an assembled model made from the same combination of blocks. The Object Of the game is for him or her to explain to you how to build the model without you actually looking at it. They will begin by giving directions from a prepared outline, but you may go beyond the outline if you wish. This is NOT the kind of exercise where you are required to remain silent. You will have ten minutes to attempt to build the model. An extra .05 credit will be provided for persons completing it accurately in this amount of time. When you have finished reading this, give the person you are working with the last four digits Of your social security number, and indicate that you are ready to begin. Good luck! 92 APPENDIX G SCRIPT FOR BEHAVIORAL TASK "THE MODEL LOOKS LIKE A SMALL MODERN BUILDING. IT IS MADE UP OF PIECES WHICH ARE BOTH RED AND WHITE. THE BOTTOM THREE PIECES, THE FOUNDATION, ARE BOTH RED AND WHITE. THE MODEL IS FIVE LEVELS HIGH, WITH THE TOP PIECE BEING A SMALL WHITE ONE. THIS TOP WHITE ONE IS SORT OF A BRIDGE BETWEEN A LARGE WHITE PIECE PLACED SIDEWAYS, AND THE SAME PIECE IN RED, ONLY IT DOES NOT ACTUALLY REACH THE PIECE, IT JUST HANGS IN THE AIR AND TOUCHES IT. STARTING FROM THE BOTTOM NOW, THE NEXT LAYER DIRECTLY ABOVE THE BASE HAS THREE PIECES AGAIN, ALSO TWO LARGE REDS AND ONE LARGE WHITE, AND THEY ARE CONNECTED TO THE BASE, BUT NONE OF THEM ARE ACTUALLY CONNECTED TO EACH OTHER. THE NEXT LAYER, NUMBER THREE, CONSISTS OF FOUR WHITE PIECES, TWO OF THE SMALL ONES, THE MEDIUM SIZED ONE, AND A LARGE ONE. ONE OF THE SMALL ONES SITS APART FROM THE OTHER THREE-~NONE OF THE PIECES ON LAYER THREE ARE ATTACHED TO THE WHITE PIECE ON LEVEL TWO. THE FOURTH LAYER IS JUST A LARGE WHITE AND A SMALL RED PIECE, PLACED PARALLEL TO EACH OTHER, WITH A SPACE IN-BETWEEN. THE TOP PIECE AGAIN IS A SMALL WHITE ONE THAT COMES FROM THE TOP OF THE LARGE WHITE ONE AND JUST TOUCHES THE RED ONE." 93