... Stag. THEE This is to certify that the dissertation entitled The Design, Development and Field Testing of a Technique To Measure the Effectiveness of Adult Education Instructors in Managing Their Verbal Communication of Intent when Establishing the Instructor/Learner Relationship presented by JOHN BROWN-PARKER has been accepted towards fulfillment of the requirements for Ph.D degreein Continuing Education Major professor Date May 281 1982 MSU is an Affirmative Actum/Equui Opportunity Institution 0-12771 IVISSI_J RETURNING MATERIALS: Place in book drop to LJBRAflJEs remove this checkout from ”- your record. FINES will be charged if book is returned after the date stamped below. SEEM USE 0"? E3 THE DESIGN, DEVELOPMENT AND FIELD TESTING OF A TECHNIQUE TO MEASURE THE EFFECTIVENESS OF ADULT EDUCATION INSTRUCTORS IN MANAGING THEIR VERBAL COMMUNICATION OF INTENT WHEN ESTABLISHING THE INSTRUCTOR/LEARNER RELATIONSHIP BY John Brown-Parker A DISSERTATION Submitted to Michigan State University in partial fulfillment of the requirements for the degree of DOCTOR OF PHILOSOPHY Department of Administration and Curriculum 1982 ABSTRACT THE DESIGN, DEVELOPMENT AND FIELD TESTING OF A TECHNIQUE TO MEASURE THE EFFECTIVENESS OF ADULT EDUCATION INSTRUCTORS IN MANAGING THEIR VERBAL COMMUNICATION OF INTENT WHEN ESTABLISHING THE INSTRUCTOR/LEARNER RELATIONSHIP by John Brown-Parker The aim of this study was to investigate the effectiveness of instructors in managing their verbal communication of intent when establishing the initial instructor/learner relationship. Six major phases were employed in the methodology: 1) The development of a conceptual framework explaining the role intent plays in the instructor/learner transaction and the design of a coding technique; 2) Pilot testing; 3) Content validation by a panel of judges; 4) Modification after panelists' feedback; 5) Training of naive coders to learn and apply the technique with reliability; and 6) Field testing of the coding technique. From the findings of the study, a major conclusion was that initial interaction between instructor and learners facilitates the instructor being perceived as helpful and well-intentioned. The instructors most effective in establishing rapport were distinguished from those who were least effective by consistently adopting patterns of communication in which they frequently: 1) checked for misunderstandings; 2) ensured learners had enough time to respond; 3) reinforced their learners' responses through John Brown-Parker the use of positive praise, regard or acknowledgement; and 4) indirectly explained their intent through the use of questions, suggestions or requests. The communication patterns used by those instructors who were perceived as least well-intentioned, were characterized by: l) a lack of interaction with learners; 2) a propensity to use long sequences of direct explanations or clarifications of intent; 3) the frequency with which they prepared their learners that upcoming explanations were not meant to be seen as malicious or arbitrary, and 4) made excuses for some behavior based on having no alternative course of action by reference to their ascribed power as instructor. To Judi, the "special woman" in my life ii ACKNOWLEDGMENTS A sincere appreciation is expressed to Dr. Joe Levine, academic advisor and chairperson for this study. Joe extended his trust and allowed the freedom for self directed inquiry that made this research an exciting and meaningful growth experience. On reflection, the rigor he demanded is appreciated as well as the support and encouragement he offered to overcome the dissonance that accompanies any serious learning endeavour. To Dr. Mel Buschman, Dr. Larry Sarbaugh and Dr. Ted ward, appreciation is expressed for their wise counsel and guidance in the preparation, design and completion of this study. Acknowledgement is made of Larry Sarbaugh for intro- ducing the central concept used in this study, that of 'per- ceived intent'. To Ted Ward, an appreciation for his help in better understanding the 'what', the 'why' and the 'how' of research. To Mel Buschman for his humanism as well as his pragmatism when skies became a little grey and cloudy. An acknowledgement is made of those peOple who shared their time to give constructive criticism, feedback and assistance. To Dr. Judd Field, Dr. George Axinn, Dr. Bill Donahue, Dr. Bob Scrivens, Jim Lemaire, Hugh Kelly, Rex Kinder and Colin Thomas whose ideas and suggestions provided valuable new perspectives that strengthened this study. iii To Bob Heron, Brian Ross, Maureen Johnston, Fred Kaad, Bill Fillas, Frank Hayes, Kevin Halpin, Jim Derrick, Hugh Kelly and participating learners at the International Training Institute and the Milperra College of Advanced Education, thanks are extended for their cOOperation and assistance during the pilot test. To Wendy and Len Newman, Stella Miria, loane Iro, Mary Nyirongo and Doreen Palmah, an appreciation for their assistance in data collection for the field test. To June Bates, Toni Konning and Toni Skinner for their help in transcribing and typing phases of the study. To participating administrators, instructors and learners in the field test who without their voluntary assistance and cooperation, the study could not have been conducted. To the coding team - David Brice, Jim.McGrath, Bob McKenzie, Ross Smith and Bruce Teackle - members of the Quadriplegic Association of New South Wales, a special thanks for their loyalty and determination to show that being phys- ically disabled does not infer an intellectual handicap. To Robert Sage, President of the Sage Foundation for his generous grant towards the costs of the study. To Ken and Lyn MCMillan, old and loyal friends, a special thanks for allowing unconditional access to the computer and other office facilities of the McMillan Financial Services organization. iv To Dick Johns and David Matters for their help in understanding programing techniques and the operation of micro-computers. It was a pleasure to learn with and from these people during various phases of the study. To our families and friends, a sincere appreciation for having patiently tolerated our year of isolation and retreat from their companionship and activities. Finally, to Judi, my wife, colleague and assistant who has worked side by side with me to help collect data, train the coding team, transcribe the tapes, type the trans- cripts and to critically discuss, edit and type the final copy of this dissertation; my thanks for her love, support and patience. TABLE OF CONTENTS LIST OF TABLES . . . . . . . . . . . . . . . LIST OF FIGURES . . . . . . . . . . . . . . Chapter 1. IDENTIFYING THE PROBLEM . . . . . . PURPOSE . . . . . . . . . . . . . Research Objectives . . . . . . ASSUMPTIONS AND BIASES . . . . . . IMPORTANCE . . . . . . . . . . . . GENERALIZABILITY . . . . . . . . . LIMITATIONS . . . . . . . . . . . DEFINITION OF TERMS . . . . . . . OUTLINE OF CHAPTER CONTENTS . . . 2. PRECEDENTS IN LITERATURE . . . . . . PHILOSOPHY AND INTENT . . . . . . LAW AND INTENT . . . . . . . . . . SOCIAL PSYCHOLOGY AND INTENT . . . COMMUNICATION AND INTENT . . . . . PERCEIVED RELATIONSHIP AND INTENT EDUCATION AND INTENT . . . . . . . METHODOLOGICAL PRECEDENTS . . . . SUMMARY . . . . . . . . . . . . . vi Page xiv 20 Chapter Page 3. DESIGN OF THE CONCEPTUAL FRAMEWORK . . . . . . . THEORETICAL ORIENTATION . . . . . . . . . . . Identifying the Parts of the System . . . . DEVELOPMENT OF A'CLASSIFICATION SCHEME FOR MEASURING AN INSTRUCTOR'S VERBAL MANAGEMENT OF INTENT . . . . . . . . . . . . A Schema for Managing Impressions of Intent Modifications to the Thomas and Pondy Schema DESIGN OF A CODING TECHNIQUE . . . . . . . . . Design for Selecting Groups of the 'Most Effective' and 'Least Effective' Instructors . . . . . . . . . . . . . . . Design for Recording and Transcribing . . . Design for Coding Categories and Combination Categories . . . . . . . . . Design for Analysis and Display of Coded Data . . . . . . . . . . . . . . . . Design for Reliability and Validity of the Coding Technique . . . . . . . . . SUMMARY . . . . . . . . . . . . . . . . . . . 4. METHODOLOGY OF THE STUDY . . . . . . . . . . . . PHASE 1: DEVELOPMENT OF AN INITIAL DRAFT OF THE CONCEPTUAL FRAMEWORK, CLASSIFICATION SCHEME AND CODING TECHNIQUE . . . . . . . . PHASE 2: PILOT TESTING TO ESTABLISH THE ADEQUACY OF THE CLASSIFICATION SCHEME AND FEASIBILITY OF THE CODING TECHNIQUE . . . . PHASE 3: THE CONTENT VALIDATION OF A TENTATIVE CONCEPTUAL FRAMEWORK, CLASSIFICATION SCHEME AND CODING TECHNIQUE . PHASE 4: FINAL MODIFICATIONS TO THE CONCEPTUAL FRAMEWORK, CLASSIFICATION SCHEME AND CODING TECHNIQUE . . . . . . . . vii 46 46 46 47 49 51 59 60 61 62 64 69 71 73 73 74 75 76 Chapter 4. PHASE 5: TRAINING OF A CODING TEAM . . . PHASE 6: THE FIELD TESTING OF THE CODING TECHNIQUE . . . . . . . . . . . . Step 1: Selection of Subjects . . . . . Step 2: Data Gathering . . . . . . . . . Step 3: Identifying Groups of 'Most Effective' and 'Least Effective' Managers of Intent . . . . . . . . . . Step 4: Transcribing Tapes and Coding Transcripts . . . . . . . . . . . . . Step 5: Display of Coded Data . . . . Step 6: Analysis of Coded Data . . . . . SUMMARY . . . . . . . . . . . . . . . . . 5. FINDINGS OF THE FIELD TEST . . . . . . . . . COMPARISON OF THE VERBAL COMMUNICATION BEHAVIORS USED BY THE 'MOST EFFECTIVE' AND 'LEAST EFFECTIVE' GROUPS Findings in Response to Research Question 1.0 . . . . . . . . . . . . Findings in Response to Research Question 1.1 . . . . . . . . . . . . Findings in Response to Research Question 2.0 . . . . . . . . . . . . Findings in Response to Research Question 3.0 . . . . . . . . . . . . . SUMMARY . . . . . . . . . . . . . . . . 6. SUMMARY, CONCLUSIONS AND RECOMMENDATIONS . . SUMMARY . . . . . . . . . . . . . . . . . CONCLUSIONS . . . . . . . . . . . . . . RECOMMENDATIONS . . . . . . . . . . . . . Page 89 91 93 . 93 110 126 141 148 151 151 153 164 Chapter 6. Recommendations for Modifications to Classification Scheme. . . . . . . . . Recommendations for Gaining Access . . . Recommendations for Identifying Suitable Support Personnel . . . . . . Recommendations for Data Analysis . . . Recommendations for Further Research . . APPENDICES A. COMMUNICATION OF INTENT IN THE CLASSROOM . . B. INITIAL CLASSIFICATION SCHEME . . . . . . . C. RECOGNITION RULES FOR UNITIZING AND CATEGORIZING TRANSCRIPTS . . . . . . . . . D. ANDERSON AND ANDERSON (1962) INTERVIEW RATING SCALE . . . . . . . . . . E. PROCEDURES FOR AUDIO RECORDING . . . . . . . F. PROCEDURES FOR TRANSCRIBING TAPES . . . . . G. PROCEDURES FOR UNITIZING AND CATEGORIZING TRANSCRIPTS . . . . . . . H. SAMPLE TALLY SHEET . . . . . . . . . . . . . I. SAMPLE PRINTOUT SEQUENCE FOR ONE INSTRUCTOR . J. SAMPLE STATISTICAL REPORT FOR ONE INSTRUCTOR . K. SAMPLE REPORT OF SEQUENCES OF THREE UTTERANCES L. GUETZKOW'S PROCEDURES FOR ESTIMATING UNITIZING RELIABILITY . . . . . . . . . . M. GUETZKOW'S PROCEDURES FOR ESTIMATING CATEGORIZING RELIABILITY . . . . . . . . . N. PROCEDURES USED DURING THE PILOT STUDY . . . O. PROCEDURES FOR SELECTING PANELISTS . . . . . ix Page 165 166 167 167 167 172 182 186 188 197 198 199 200 201 202 204 205 206 212 216 Appendix P. CHANGES MADE TO THE CLASSIFICATION SCHEME ON THE RECOMMENDATION OF THE PANELISTS . . Q. PROCEDURES USED TO IDENTIFY AND TRAIN A CODING TEAM .,. . . . . . . . . . . . . . R. INVITATION TO PARTICIPATE IN THE FIELD TEST S. CORRESPONDENCE TO PARTICIPANTS IN THE FIELD TEST . . . . . . . . . . . . . . T. MEAN SCORES OF INSTRUCTOR EFFECTIVENESS IN ESTABLISHING INITIAL RAPPORT . . . . . . . U. UNITIZING AND CATEGORIZING RELIABILITY COEFFICIENTS OBTAINED BY CODING TEAM . . . BIBLIOGRAPHY . . . . . . . . . . . . . . . . . . . Page 224 225 232 236 241 243 245 Table 10 LIST OF TABLES A Comparison of the Proportional Use of Each Category by the 'Most Effective' Instructors A Comparison of the Proportional Use of Each Classification by the 'Most 'Effective' Instructors . . . . . . . . . . Differences in the Proportional Use of Categories and Combination Categories by the 'Most Effective' Group . . . . . . . . . Summary of Findings from Table 3 to Identify those 'Most Effective' Instructors Whose Use of Categories were Similar . . . . . . . Differences Between the Frequency of Categories Used in the First and Second Halves of the Lessons Presented by the 'Most Effective' Instructors . . . . . . . . A Comparison of the Ten Most Frequently Occurring Paired Utterances Used by the 'Most Effective' Group . . . . . . . . . . . Differences in the Use of the Selected Paired Utterances by the 'Most Effective' Group . . Summary of Findings from Table 7 to Identify those 'Most Effective' Instructors Whose Use of Paired Utterances were Similar to each other . . . . . . . . . . . . . . . Differences Between the Paired Categories Used in the First and Second Halves of the Lesson by the 'Most Effective' Instructors . Differences in Probability of One Category Following Another in Selected Paired Utterances for 'Most Effective' Group . . . xi Page . 99 100 102 103 104 105 106 Table 11 12 13 14 15 16 17 18 19 20 21 22 Page Summary of Findings from Table 10 to Identify the X Probability of One Category Following Another in Frequently Used Paired Utterances by the 'Most Effective' Instructors . . . . . 107 Differences in the Probability of One Category Following Another in Selected Paired Utterances for the First and Second Halves of the Lesson for Each of the Instructors in the 'Most Effective' Group . . . . . ... . 108 Ranking of Incidence of Sequences of Three Utterances Frequently Used by the 'Most Effective' Instructors . . . . . . . . . . . 109 Summary from Table 13 to Identify Patterns in the Sequences of Three Utterances Frequently Used by the 'Most Effective' Instructors . . 110 A Comparison of the Proportional Use of Each Category by the 'Least Effective' Instructors 111 A Comparison of the Proportional Use of Each Classification by the 'Least Effective' Instructors . . . . . . . . . . . . . . . . . 112 Differences in the Proportional Use of Categories and Combination Categories by the 'Least Effective' Group . . . . . . . . . 113 Summary of Findings from Table 17 to Identify those 'Least Effective' Instructors Whose Use of Categories were Similar to Each Other 114 Differences Between the Paired Categories Used in the First and Second Halves of the Lesson of Instructors in the 'Least Effective' Group 115 A Comparison of the Proportional Use of Selected Paired Utterances by the 'Least Effective' Group . . . . . . . . . . . 117 Differences in the Use of the Selected Paired Utterances by the 'Least Effective' Group . . 118 Summary of Findings from Table 21 to Identify those 'Least Effective' Instructors whose Use of Paired Utterances were Similar to Each Other . . . . . . . . . 119 xii Table 23 24 25 26 27 28 29 30 31 32 33 Differences Between the Paired Categories Used in the First and Second Halves of the Lesson by the 'Least Effective' Instructors Differences in Probability of One Category Following Another in Selected Paired Utterances for 'Least Effective' Group . . . Summary of Table 24 to Rank the X Probability of One Category Following Another in the Paired Utterances Used by the 'Least Effective' Instructors . . . . . . . . . . . Differences in the Probability of One Category Following Another in Paired Utterances Used by the 'Least Effective' Instructors During the First and Second Halves of the Lesson . . . . . . . . . . . . . . . . . . . Ranking of Incidence of Sequences of Three Utterances Frequently Used by the 'Least Effective' Instructors . . . . . . . . . . . Summary from Table 27 to Identify Patterns in the Sequences of Three Utterances Frequently Used by the 'Least Effective' Instructors . Differences in the Proportional Use of Categories by the 'Most Effective' and 'Least Effective' Subgroups . . . . . . . . Differences in the Use of Selected Paired Utterances by the 'Most Effective' and 'Least Effective' Subgroups . . . . . . . . Ratios Between 'Most Effective' and 'Least Effective' Subgroups in their Use of Paired Utterances . . . . . . . . . . . . . . . . . Differences in the Probability of One Category Following Another in Selected Paired Utterances Between Subgroups of 'Most Effective' and 'Least Effective' Instructors Comparison of the Probability of One Category Followed by Another . . . . . . . . . . . . xiii Page 120 122 123 124 125 126 131 134 136 138 140 Figure 10 11 12 13 14 15 16 LIST OF FIGURES Page Positive Cycle of Interpersonal Relations . . . Negative Cycle of Interpersonal Relations . . . MacKay's Model of Intentionality . . . . . . . . Tubb's Model of Intentional and Unintentional Communication . . . . . . . . . Stake's Model for Processing of Descriptive Data The Function of Intent in Educational Contexts . Model of Intent for Program Evaluation . . . . . Communication of Intent: An Open Systems Model . Managing Impressions of Own Intent . . . . . . . Modifications to Thomas and Pondy (1977) Schema Final Revision of the Classification Scheme for an Instructor's Verbal Management of Intent . Sample Cues to Assist in Identifying Verbal Categories . . . . . . . . . . . . . . . . . . Pairing Utterances in a Sequence . . . . . . . . Worksheet Used to Calculate Sources of Variance Between Proportions . . . . . . . . . Differences in the Proportional Use of Categories in the 'Repairing' Classification Differences in the Proportional Use of Combination Categories . . . . . . . . . . . xiv 30 30 31 31 37 38 39 48 49 51 53 54 66 86 128 129 Chapter 1 IDENTIFYING THE PROBLEM In an effort to establish an adequate instructor/ learner relationship, the well-intentioned adult education instructor has an incentive to avoid communication behaviors that could be interpreted by learners as contrary to their psychological needs or learning interests. To minimize inaccurate communication of intent when establishing the instructor/learner relationship, the adult education instructor functions in two roles. First, as observer, to discern verbal messages and nonverbal cues that indicate learners have understood or misunderstood the instructor's intent. Second, in response to this feedback the instructor functions as actor, consciously managing his verbal and nonverbal communication to ensure that the learners' perceptions are congruent with his desire to be seen as a helpful and well-intentioned instructor. During the first class meeting the adult learner has minimum information about the instructor's competence, integrity or intent. Thomas and Pondy (1977) and Schmuck and Schmuck (1971) suggest that each party has a need to know the other's intent in order to predict how they might interact appropriately in the instructor/learner relationship. 2 When confronted with feelings of anxiety and under some pressure to view oneself in positive terms, an adult learner unfamiliar with the "culture of the classroom", may tend to see others as the source of any discomfort or frustration. Seeing himself as fair and reasonable, and identifying with other learners in a similar situation, a learner often shifts the blame for any dissonance onto the most obvious source of discomfort -- the instructor. The learner is apt to selectively attend to and recall negative aspects of the instructor's verbal and nonverbal communication. This can culminate in an exaggerated impression that the instructor is being intentionally uncaring and uncooperative in the instructor/learner relationship. Within the context of the adult education classroom, this situation becomes an extremely difficult communication problem for the instructor to resolve. The instructor must attempt to accommodate the needs of each adult learner who brings into the initial meeting a bundle of diverse expectations, goals, experiences, assumptions, norms, beliefs, world views and even linguistic differences (Houle, 1961; McNeil, 1976; Knowles, 1977; and Sarbaugh, 1979). Given the authority and responsibility to facilitate learning, classroom instructors typically function in a predominantly oral communication situation. It is a situation in which a large group of learners are influenced, whether favorably or unfavorably, by the instructor's overt 3 attempt to control, manipulate or manage the learning environment. A great deal of the instructor's success is dependent upon his repertoire of verbal communication skills. The quality of this communication is suggestive of a particular type of instructor/learner relationship. That is, the perceived relationship and intent of the instructor is judged as positive with a desire to help and share; or as neutral, with a tendency to ignore or be indifferent; or as negative, with a tendency to dominate, frustrate or thwart the learner. No studies in adult education were found that offered a conceptual or operational framework providing adequate descriptive categories of verbal communication used by instructors to manage their intent in the instructor/ learner relationship. No empirical studies were found that attempted to measure the instructor's verbal management of intent. At present many well-trained and well-intentioned adult educators are dismayed and discouraged when their efforts to build adequate instructor/learner relationships are not reciprocated and instead, learners drop out or conflicts occur. Therefore it seemed worthwhile to learn more about adult education instructors who are 'most effective' or 'least effective' in managing their verbal communication of intent when establishing an instructor/ learner relationship. PURPOSE The purpose of this study was to develop and field test a technique for classifying and measuring the verbal communication used by adult education instructors for managing their learners' perceptions of instructor intent when establishing an instructor/learner relationship. Research Objectives In order to facilitate an orderly and systematic approach to the design, development and field testing of the technique, a number of research objectives were formulated. Objective #1 To develop a conceptual framework, a classification scheme and a measurement technique to allow objective data collection. This first step was based upon an approach advocated by Amidon and Hough: In the behavioral sciences, principles of human behavior are often derived as a result of a specific pattern of activities. An overall conceptual framework is first proposed from which hypotheses are formulated and tested. The development of this framework as a first step is important in that it gives both substance and direction to the process of formulating and testing. When hypotheses are accepted, the data from such research provide the formulation of theory. When principles of human behavior can be derived from theory, then theory gives direction to action. Specific instances can then take on generalizable meaning (Amidon and Rough, 1967, p. 2). Objective #2 To field test this technique for classifying and measuring the verbal communication used by adult education instructors to manage their learners' perceptions of the instructor so one is seen as helpful S and well-intentioned when establishing the instructor/ learner relationship. The following enabling objectives were formulated to give direction to various phases of the field test: Phase 1: To pilot test the feasibility of this technique which allows reliable coding of an instructor's verbal management of intent. PhaSe 2: To invite a reaction panel of expert judges to act as external criteria for content validation of the conceptual framework, classification scheme and coding technique. Phase 3: To modify the tentative conceptual framework, classification scheme and coding technique based upon the feedback provided by the panel of experts. Phase 4: To train a naive group of coders to learn the classification scheme and apply the coding technique with reliability. Phase 5: To establish if there were statistically significant differences between the verbal communication used by instructors who were 'most effective' and 'least effective' in their management of intent when establishing the instructor/learner relationship. The following research questions were asked during this final stage of the field test. They were as follows: 1.0 Are there differences or similarities among the verbal communication behaviors used by the five instructors who were identified by learners as 'most effective' managers of their intent? 1.1 Are there differences or similarities among the verbal communication behaviors used by the five instructors identified by learners as 'least effective' managers of their intent? 6 2.0 Do the verbal communication behaviors used by groups of the 'most effective' instructors differ from those verbal communciation behaviors used by groups of 'least effective' instructors? 3.0 Are there patterns of verbal communication which are used more consistently by the group of 'most effective' instructors than by the group of 'least effective' instructors? Conversely, are there patterns of verbal communication which are used more consistently by the group of 'least effective' instructors than those used by the group of 'most effective' instructors? ASSUMPTIONS AND BIASES A recognized bias and five assumptions underpin this study. First, the bias is held that the humanistic and democratic assumptions and practices of andragogy are an appropriate foundation for the management of adult education classroom environments. This approach adopts those mentalistic and psychological theories which stress learner needs, mutual trust and respect, mutual support and help, physical and emotional support, acceptance of differences, mutual responsibility for planning outcomes, freedom of expression and access to information, and encouragement of self-directed and learner-centered instruction. Educational thinkers influencing this bias are Abraham Maslow (1970), John Dewey (1938), Carl Rogers (1969), Julius Nyerere (1976). Paulo Freire (1970) and Malcolm Knowles (1977). By 7 identifying this bias at the onset of the study, it can be understood why the conceptual framework of this study emphasizes the positive dimension of an instructor's verbal management of intent; that is, those verbal communication behaviors used to help avoid misunderstandings, misinterpretations or potential conflict when establishing an instructor/learner relationship. A description of the characteristic uniqueness of the reality in which we move can help us to better understand some phenomenon. As Weber (1949, p. 78) argues, "order is brought into this chaos only on the condition that in every case only a part of concrete reality is interesting and significant to us, because only it is related to the cultural value with which we approach reality". In line with this thinking, the first of the five assumptions is that it is possible to develop an observational technique that minimizes distortion and provides a plausible representation of actual events or reality. This assumption is based on the view that a true representation of reality can be had by expressly and consciously selecting, analyzing and organizing specific observable phenomena. The second assumption is that the verbal communication classified by the researcher, and evaluated by a panel of expert judges, is a plausible representation of the major observable verbal behaviors used to manage the learners' perceptions of instructor intent. Based on the Thomas and Pondy (1977) 'Intent' model of conflict 8 management, it follows that verbal communication can be initiated by either instructor or learner to discern the other's intent. One can ask for or offer clarification of specific statements or actions; one can announce frustrations or reactions to another's behavior; one can offer explanations, excuses or reparations; or one can give or receive feedback that will avoid future misunderstandings. The third assumption is that instructors who are successful in establishing adequate rapport or 'ideal relationships', are also seen by their learners as well- intentioned. Instructors who are less successful in establishing adequate classroom rapport, will be seen by learners as less well-intentioned. The fourth assumption is that the greater the difference between people's normative beliefs, overt behaviors, role expectations, and world views, the more dissimilar will be the perceptions and intepretations of each other's intent. The learner's perception of the instructor's intent will then influence how the learner develops the subsequent relationship with the instructor. This assumption evolved from an intercultural communication principle postulated by Sarbaugh (1979) who contends that as "the perceived relationship and intent moves from most homogeneous to most heterogeneous, the probability of communication breakdown increases" (p. 71). 9 The fifth assumption is that the adult learner, unsure of the normative rules and expectations of the classroom will be particularly sensitive to his personal assessment of his adequacy in the face of academic demands and the congruency of his interests with those of his instructor, peers and institution (Boshier, 1972 and 1973; Clarke, 1980). If there are incongruencies or misunderstandings, the learner will tend to see themself as the cooperative and reasonable party in order to retain self respect and esteem (Thomas and Pondy, 1977; Lerner and Simmons, 1966). It seems plausible to assume that a great deal of the blame for incongruencies or frustrations will be shifted onto the teacher for allowing this to occur. Exaggerated attributions of intent will be ascribed to the instructor if the behavior: 1) seems to constrain the learner's behavioral alternatives or outcomes; 2) is perceived as intentionally detrimental to the learner's interest; and 3) is considered anti-normative or unnecessary (Tedeschi, 1973). IMPORTANCE There are four main elements of this study that underlie its importance to adult education. First, the study can benefit the adult learner. Successful facilitation of optimal conditions for learning requires instructors to be successful managers of their communication. If instructors become more aware of the 10 verbal behaviors that help to accurately communicate instructor intent, classroom failures could occur less frequently. Second, the study could be of particular interest to adult education instructors in understanding more about the verbal communication patterns they use to successfully manage their learner's perception of instructor intent. Generally, instructors have an incentive to manage their communication in an attempt to have learners perceive them in some specific way. In the learner-centered classrooms of adult education, it is important for the instructor not only to be trustworthy, helpful and willing to share, but also to be seen by learners as trustworthy, helpful and willing to share. In short, it is important for the instructor to be perceived by his learners as well-intentioned. Third, the notion that a person's intent provides a central organizing principle in making sense out of another person's behavior has been supported by philosophical heuristic argument. To be human is to form intentions towards our world and to assume that others have intentions toward us. In recent years there has been increasing interest in attempting to operationalize and utilize the seemingly ambiguous concept of intent (Anscombe, 1966; Maselli and Altrocchi, 1969; Schmuck and Schmuck, 1971; Crittenden, 1974; Thomas and Pondy, 1977; Sarbaugh, 1979; and Freyberg, 1980). Thus, it is important to build upon this growing body of existing knowledge and attempt to 11 further understand a concept that may help to better describe and explain human communication processes. Fourth, while many classroom climate assessment systems and teacher/learner interaction analysis systems indirectly acknowledge the role which intent plays in establishing adequate instructor/learner relationships (Pratt, 1979; Simon and Boyer, 1974; and Lake et al., 1973),. no studies were found that utilize intent as the central organizing concept. The methodological precedents of this study may provide a foundation for further inquiry into the role intent plays in the establishment of adequate interpersonal relationships between instructor and learners. GENERALI ZABILITY The subjects used in this study were a convenience sample of Australian instructors drawn from a metropolitan region of Australia. Because of the non-random selection of instructors in the sample and the small class sizes, the findings of the field study cannot be generalized to a wider population of instructors, nor to adult education instructors who work in similar educational settings in Australia. Similar studies will need to be carried out using this coding technique to establish the strength of its reliability and usefulness. 12 LIMITATIONS In this study there are five limitations to the methods and approach used to investigate an instructor's verbal management of intent. First, this study was limited to an examination of verbal communication used by adult education instructors. No attempt was made to observe or measure the nonverbal management of intent by the instructors. It is acknowledged that: l) the nonverbal communication channel carries over 65 percent of social meaning (Birdwhistell, 1970) and has an overwhelming influence upon interpersonal communication (Smith, 1979); 2) there is an intimate relationship between nonverbal and verbal communication (Knapp, 1978; Mehrabian, 1968); 3) often nonverbal behaviors conflict with or negate the verbal message (Mehrabian, 1971); and 4) the study of nonverbal communication in teaching has significant potential in helping to better understand the teaching process (Smith, 1979). However, as Banks et a1. (1978) suggest, because of the methodological complexities and problems in observing and recording nonverbal communication, it was decided to limit the study to a single variable, that of verbal communication. Research relative to tabulating, analyzing and interpreting nonverbal behavior in classroom settings is in its infancy. There is as yet no agreement on a unit of measurement, procedures, interpretative 13 rules or generalized stability of any conclusions reached (Banks et al., 1978. p. 14). Second, single variable verbal communication is part of an information system that is generally examined using either a structural or content approach (Monane, 1967). According to Donahue, Hawes and Mabee (1981), the structural approach focuses on the individual as the unit of analysis and assesses who talks to whom, about what and how frequently. In contrast, the content approach focuses on utterance patterns and assesses: 1) what is said (content). 2) how one feels about what is said (relationship), and 3) what one should do about what and how something is said (control). While a learner's attribution of intent might be based on all of the above three types of information, the measurement technique developed for this study is limited to the content or what is said in a verbal utterance. Third, the coding scheme used for systematic recording of an instructor's verbal management of intent is event-sequential and utilizes a verbal utterance as the unit of analysis. Like any communication construct, the unit of analysis and categories used in the coding scheme are a function of the research interest and a limited approximation of behavior. Their appropriateness is dependent upon their functional value and their plausibility. There is no reason to believe that to code according to a time interval is any more important than coding predefined functional units that ignore duration. For example, one category may take tens of seconds to l4 articulate while a simple 'we' or 'our' message may take less than a second. On the other hand, all coding systems using units of analysis that are discrete and exhaustive, whether based on a time interval or a sequence of events, assume that each behavior tallied is equivalent. This poses the problem of losing the intensity of a rare but important event by its ending up as a tally on a score sheet (Ellis, 1977). Given these limitations and the nature of this exploratory study, the use of an event-sequential coding scheme was adopted. Fourth, audio-recordings of instructors were limited to the initial meeting of each class in the third term rather than in the first term of the year. It was expected that most learners would be meeting their instructor for the first time. This was not the case as a large number of learners had joined their class in the previous term. As a result, some learners may have based their rating of the instructor's effectiveness in establishing the instructor/ learner relationship on previous meetings rather than the instructor's effectiveness during the particular class meeting that served as the source of data. Fifth, because the field test sample was selected from two sets of extreme scores, the internal validity of the findings could normally be effected by statistical regression. However, the instrument to identify high and low scores in establishing initial rapport was designed only for use at the first class meeting and was not intended to 15 be administered for a retest or post—test to assess any change in the relationship. Although the two groups of instructors were selected on the basis of the extreme scores, the instructors were subsequently evaluated on the unrelated dependent variable of instructor verbal communication behaviors. This would suggest the findings associated with the second variable should be reasonably free of the regression effect. DEFINITION OF TERMS The following terms and phrases are used in the description of this study. Definitions for each term and phrase are provided to form a common basis for understanding. Adult Education. The process whereby men and women, alone or in groups, attempt to improve themselves by increasing their skills or knowledge, developing insights and appreciating or changing their attitudes; or the process by which individuals or agencies attempt to change men and women in these ways (Houle, 1970). Andragogy. A model of assumptions about adult learners in contrast to traditional concepts of youthful learning subsumed under pedagogy. The model assumes that as individuals mature: 1) their concept moves from being a dependent personality toward being a self-directed human being 2) they accumulate a growing reservoir of experience that becomes an increasingly rich source for learning; 3) their readiness to learn becomes orientated increasingly to the developmental tasks 16 of their social roles; and 4) their time perspective changes from one of postponed application of knowledge to immediacy of application and accordingly their orientation toward learning shifts from one of subject-centeredness to one of performance-centeredness (Knowles, 1980). Attribution. The inference that an observer makes about the causes of behavior, the 'what' or the 'why' of either one's own or another person's behavior (Bar-Tal, 1978). Conflict. The process which begins when one party perceives another has frustrated or is about to frustrate some concern of theirs (Thomas, 1976). Evening College. School-based providers of non-credit leisure and vocational instruction in Australia for adults below the college level. Motive. The 'why' of human behavior. Motive is quite distinct from intent. Motive is the moving force, desire, wish, want, need or cause which induces action. It is the state of feeling that impels one towards an act. In general use, the meaning of motive often shades into the meaning of intent (Words and Phrases, 1958; Roget's Thesaurus, 1978). A person may have a good intent but a bad motive, or a person may have a good motive but a bad intent. Motives are described on a continuum from 'good' through 'arbitrary', 'willful', 'deliberate' to 'malicious'. Intent. The 'what' of human behavior. It has six main conceptual synonyms: aim, purpose, design, object, goal or objective (Webster, 1961; Words and Phrases, 1958). The 'zone of intent' which the study addresses is 17 confined within the establishment of interpersonal 'relationships between instructor and learners (the affective domain), rather than the learning goals or objectives set by the learner or instructor (the cognitive domain). Intent is an act or emotion of the mind that seldom is capable of direct or positive observation or proof. It is a mental or psychological state and can be evidenced only by words or conduct of the person who claimed to have entertained them. A declaration of intent, however, may be false. Intention is used as a synonym for intent in this study. Perceived Intent. A logical process of inference from cues perceived by an individual or individuals. From the perceiver's point of view, any effect of another person's actions, whether past, present or anticipated, is a potential reason to believe that this person has engaged in that action. To infer that the action occurred for certain reasons is to specify one's perception of that actor's intent (Jones and Davis, 1962). The perceived intent of another is usually expressed as an underlying disposition on a continuum of descriptors from 'helpful' through 'neutral' to 'harmful' (Sarbaugh, 1979). The term does not refer to the more intuitive process based on the personal knowledge or awareness of one's own intentions. Paralinguistic Cues. The voice qualities including pitch, pitch control, rhythm control, tempo, articulation, 18 resonance, glottis control and vocal lip control (Harrison, 1974). Verbal Communication. The exchange of information though oral or written linguistic symbols or signs. Nonverbal Communication. The exchange of information through nonlinguistic symbols or signs. Coding. A system for translating real events into quantifiable units. For this study it is the transformation of qualitative data obtained by audio-recordings of instuctors into a form which renders them open to quantitative treatment. This operation requires coders to separate qualitative materials into units and then classify the unitized material according to a category scheme devised for the study. OUTLINE OF CHAPTER CONTENTS In Chapter 2, precedents in literature are reviewed to provide an overview of the conceptual and operational approaches that have been used to examine the role 'intent' plays in human relationships. In Chapter 3, the design of the conceptual framework is explained. It includes a discussion of the theoretical orientation to communicating intent in the classroom, the development of a classification system for measuring an instructor's verbal management of intent and the design of the coding technique. 19 In Chapter 4, the methodology employed in each of the six major phases of the study are described and explained. The phases are: 1) development of an initial draft of the conceptual framework, classification scheme and coding technique; 2) a pilot test; 3) validation by a panel of judges; 4) final modifications and refinements based on the findings of the pilot test and feedback from panelists; 5) training of a naive group of coders to learn and apply the technique with reliability; and 6) field testing of the coding technique. In Chapter 5, the findings of the field test are reported. These are presented in three parts: 1) instructor effectiveness as rated by learners; 2) coding reliability; and 3) the analysis and comparison of a small group of 'most effective' and 'least effective' instructors in their verbal management of intent. Chapter 6 summarizes the conclusions drawn from the - findings of the field test and provides implications and recommendations for future research. Chapter 2 PRECEDENTS IN LITERATURE The review of precedents in literature provides an overview of the conceptual and operational approaches that have been used to examine the role intent plays in human relationships. Though very few studies have dealt with the management of intent, the studies that are reviewed form a logical theoretical basis for the conceptual framework developed for this study. PHILOSOPHY AND INTENT The concept of an instructor verbally managing his intent is based on the premise that reasons for acting in some way do exist in a person's mental state which can result in intentional actions toward somebody or something. Anscombe (1966) argues: Ancient and medieval philosophers -- or some of them at any rate -- regarded it as evident, demonstrable, that human beings must act with some end in view, and even with some one end in view. The argument for this strikes us as rather strange. Can't a man just do what he does, a great deal of the time? He may or may not have a reason or a purpose; and if he has a reason or a purpose, it in turn may be what he happens to want; why demand a reason or purpose for it? And why must we at last arrive at some one purpose that has an intrinsic finality about it? The old arguments were designed to show that the chain could not go on forever; they pass us by, because we are not inclined to think it must even begin; and it can surely stop where it stops, no need for it to stop at a purpose that 20 21 looks intrinsically final, one and the same for all actions. In fact there appears to be an illicit transition in Aristotle, from 'all chains must stop somewhere' to 'there is somewhere where all chains must stop' (Anscombe, 1966. p. 34). This explanation shows why some linkage between a person's intents or purposes, and their actions must at least begin. Given that intentional action does exist, useful approaches to describe and explain this phenomenon within human relationships can proceed. Attempts to interpret the phenomenon of intentionality, to operationalize a theory of intent, or even delineate the unique characteristics of how people articulate their intentions have been left to recurrent philosophic argument. Few philosophers have overcome a sea of conceptual and semantic ambiguity to satisfactorily describe, explain or define the slippery concept of intent. The most common philosophic approach taken to the study of intentionality is ontological or metaphysical in nature and views intentionality as a predominantly mental act. Arguments are mainly concerned with the conscious awareness of one's own intent and the subsequent relationship between intentions and actions (Aquila, 1977; Ross, 1978 and Griffin, 1978). Within this context, Meiland (1964) draws the useful distinction between purposive and nonpurposive intentions as well as conditional and non-conditional intention. Both categories of intent suggest that time, symmetry and congruence are integral components of intention. 22 Kramer (1978) utilizes the concept of control to explain which actions are intentional. In communication, people have a battery of sociokinetic verbal and nonverbal powers which they use to increase status, influence, persuasion, attraction and trustworthiness, and do intentionally or unintentionally control others. Whether such control is used to help or to injure is a moral issue. Lowe (1978) offers the proposition that in any relationship intentionality has moral dimensions. He suggests receivers tend to judge sender's actions as blameworthy and malicious, when they are perceived as intentional. Injustice is seen to occur not so much because some basic need is not being met, but rather the injustice lies in the perception that a person is intentionally exploiting a situation that prevents or deprives satisfaction of needs. In contrast to ontological and metaphysical approaches, a phenomenological approach adopted by Matejeko (1975) helps explain another perception of intent, this time from the observer's point of view. From the dialectical viewpoint, social life is first of all a process of becoming, not just of being. The human psyche is an active element and not only a passive receptor of an external world. ... Thought and activity penetrate one another. We ... impose our own order upon reality which in its true nature is chaotic and multidimensional. "The mind selects only certain characteristics of phenomena as significant; it finds cultural meaning in only a segment of reality; it falsifies the world at the very least by omission. Culture, then, turns upon the mind and perpetuates the illusion" (Matejeko, 1975. p. 11). 23 Two key assumptions are made by Matejeko that underpin the rationale for much of this study. First, people have only a limited capacity to process the reality around them. Second, because of this limitation, people must be selective about what messages they decide to process, and that tends to be a culturally learned perceptual behavior. This dialectical approach has its roots in Karl Mannheim's sociology of knowledge with its theme of social relationism (Berndt and Berndt, 1973). This idea of perceiving 'where one stands' in the social hierarchy is seen as a determining factor in social action and commitment, and supportive of the notion of perceived relationships and intent (Sarbaugh, 1979), a concept central to this study. The related theory of cultural relativism (Herkovits, 1949; Benedict, 1934; and Whorf, 1967) supports this study's conceptualization of the classroom as a mini- culture with its own set of rules and norms that influence how instructors and learners interact with one another. It is a Situation where conflicts can occur if a new learner is not made aware of the intent behind certain instructor behaviors which may be unfamiliar or puzzling to the learner. In summary, philosophic writings have typically approached the concept of intent by stressing its mentalistic dimension. There is a preoccupation with how individuals form intentions or control their intentions in 24 the role of actor. The arguments are eloquent, and the logics complex and sophisticated. Invariably, their arguments overlook the fundamental distinction of people being both an actor and observer (Thomas and Pondy, 1977). While there is a clear distinction between acting and observing, the distinction between thought and action or between mentalistic states and behavioral responses, is problematic. However, these unresolved issues do not refute the notion that a person as actor can be forming intentions while controlling the verbal and nonverbal behaviors to which others may attribute certain intents. In the role of observer, one also simultaneously processes vast amounts of feedback to make continual judgements about the intent others have toward them. LAW AND INTENT Legal precedents provide a useful guide to how the concept of intent can be successfully defined and utilized within context specific situations dealing with human relationships. Interpretations of intent are dependent upon the context of the case to which it is applied. The conceptual framework of this study also provides an organizational structure to help understand the role intent plays in the instructor/learner relationship within the context of an adult education classroom. An important operational distinction made in law, is that 'intent' and 'motive' are not one and the same thing. 25 'Intent', in its legal sense, is quite distinct from motive. It is defined as the purpose to use a particular means to effect a certain result. Motive is the reason which leads the mind to desire that result (Baker v. State, quoted in Words and Phrases, 1959, p. 14). In general, there is legal agreement that intent can only be implied or inferred from expressions or conduct or both, when considered in the light of the given circumstances. SOCIAL PSYCHOLOGY AND INTENT Maselli and Altrocchi (1969), in a summary of research findings dealing with the attribution of intent; suggest that: Attribution of intent, often observed in human interactions, is central to person perceptions and interpersonal relations and indirectly raises theoretical issues concerning the relevance of intentions. . . . Attribution of intent is alternatively described as a logical process of inference from cues or as a more intuitive process based on personal knowledge of one's own intentions. The internal-external dimension and individual coping techniques with motivational arousal are . promising individual variables. Attribution of intent often contributes to perception of the social world as more predictable and to socially appropriate behavior but can also lead to behavior which is destructive to self and others (Maselli and Altrocchi, 1969. p. 445). To make sense out of the complexities of human behaviors and make a person's environment more stable and predictable, people are generally held responsible for their behavior. Intentions are attributed to them in order to help explain the reasons for their behaviors. 26 Nadel and Altrocchi (1967), Heider (1958a, 1958b), Jones and Davis (1965) and Bronfenbrener (1964) suggest that the attribution of intent is the essential link between observable acts and non-observable inferences about a person's disposition towards another person. In relationships where power or intimacy play an important role, one often observes extreme attributions of intent related to either great trust or great mistrust, often resulting in extreme consequences. Further, Nadel and Altrocchi (1967) found that hostile intent tends to be attributed to persons who represent privilege, achievement or advantage. When applying these findings to the adult education classroom, it is plausible that when new learners are faced with a great amount of potentially confusing information or acts contradictory to their expectations, there might be a tendency to attribute blame to the most obvious cause of this discomfort or confusion, the instructor. The process and logic employed in this type of attribution of intent appears to be based on the concepts of covariation, configuration and inference (Kelley, 1973). The man in the street and the scientist share the same general approach to the interpretation of behavior. Both assume that B = f(P,E). Behavior is a function of the person and the environment. Thus behavior is assumed to convey information about both P and E. . . . An observer of a person's behavior can make judgments about several different (though interrelated) aspects of its meaning: (a) the positive or negative quality of the consequences of that behavior, (b) the specific nature of the motivation that underlies it (the P factor because 27 there is always some kind of P involvement), and (c) the main type of cause(s) involved in the behavior (the allocation between P and E, the stability of the causal factors) (Kelley and Thibaut, 1978 p. 214). This type of inferential logic, so often used by people in their everyday living and thinking (Schultz, 1951), is important in understanding how adults view the intent of their instructor in the early development of the instructor/learner relationship. Learners tend to make inferences about the intent of the instructor based on unfamilar situational or communication cues and on the false assumption that the instructor should think and act in the same way as the learner. Bar-Tal (1978) suggests that perceptions of intent in a relationship may well prove to be one of those causal dimensions that are used as excuses for success or failure in the learner's performance. However, studies drawn by DeCharms et al. (1965) and Maselli and Altrocchi (1969) make the distinction that people differ in the degree to which they attribute intent to others, and in the degree to which intention is attributed to them. Thomas and Pondy (1977) note the relative neglect of addressing the conflict management activities of the principal parties themselves. This appears to be a legacy of behaviorism and experimental gaming where intentions, feelings, attitudes or ideas simply accompany or follow behavior. They do not argue this theoretical orientation is invalid but for practical purposes it defines conflict management in such a way that it places the main parties in 28 a relatively primitive mode of interaction. The key to conflict management by principal parties is to understand the role of higher mental processes during an interaction. The role of attributed intent is central within any conflict episode: . . . attribution of other party's intent is a central activity in conflict episodes, and that these attributions play a crucial mediating role in shaping each party's reactions to the other's behavior -- specifically mediating hostility and retaliation (Thomas and Pondy, 1977. p. 1089). Their study of attribution data from executives, indicates that there are strong biases in the perception of intent. Individuals tend to see themselves as cooperative and reasonable while they tend to attribute competiveness or unreasonableness to the other party. Each party in this potential conflict situation acts as both observer to discern the other's intent, and as actor to manage the other's impressions of their own intent. The Thomas and Pondy (1977) model is based largely upon attribution theory applied to a management situation, but it does not address the dynamics of interaction that clarify the specific processes used to perceive or manage the communication of intent. COMMUNICATION AND INTENT Ekman and Friesen (1969) operationalize intent in terms of the deliberate use of nonverbal acts to communicate a message to another person. While their schema focuses on inferences about the sender's mentalistic state it does not 29 address the behavioral responses to feedback from the receiver. On the other hand, Schmuck and Schmuck (1971) conceptualize a reciprocal communication process based on personal intent. Communication is seen as emanating from individual needs, motives and desires and involves sending messages about personal intent, whether they are concerned with control, information, love or anger. Effective communication exists between two people when the receiver interprets the sender's message in the same way the sender intended it. Congruence of message is dependent upon the matching of intentions, behaviors and interpretations. Figures 1 and 2 illustate the crucial role intent plays in understanding miscommunications that occur during interpersonal transactions. It is a situation when the message received does not accurately reflect the intentions of the sender. For example, when persons, intentionally or unintentionally, attempt to impress, they set the stage for distrustful communication. While attempts to impress are not an unnatural human phenomenon, it tends to encourage further superficiality and concealment, defensiveness, justification or falsification to keep 'face'. MacKay (1972), has also developed a model of intentionality using the two extremes of goal-directed communication and non goal-directed communication, as illustrated in Figure 3. 30 I 2 W m o! A m o! A we at an: hub M! and 1mm MI}... “9.“- uupM. can. ole-om. w —# “WIM- ndm union. mum. m and“ m I I Positive We. at m m Out i m Iggigpzisonal 1 cm" 1" Relations Inn-u trio-nay. In” Adam M‘ : m m. N to M “M. sharing. I” V——-f M W. to " ho- no. can an. M a 00.: W M at OM . ‘ 3 FIGURE 1 POSITIVE CYCLE OF INTERPERSONAL RELATIONS (Schmuck and Schmuck, 1971. p. 52) I 2 Manolo‘s-l hm at A m M A m. I I : rat-loam a law to Wave. sanctum 000 “Ii ' I . I cannot or Manning m fl " mm “M" bounty or ammo. ' um". monuv , mm. m W E , Negat ive m d m Wot-I Onto-u Cycla Of I I Interpersonal WW! mac-sol m Relations 1 L um hum that ‘_ . . was Won. W N Mun. M Non-m" «doom 0! m or ham. " ~.' A‘I M. We ""- " son um mm. mm, W' 0! low mum. M 0! Oman Pow m.- a! 0mm 4 J FIGURE 2 NEGATIVE CYCLE OF INTERPERSONAL RELATIONS (Schmuck and Schmuck, 1971. p. 53) 31. Goal-directed Interpreted as goal directed Ndmmmhalsigufls . , \ . Non-goal directed NOt interpreted as goal directed FIGURE 3 MACKAY'S MODEL OF INTENTIONALITY (MacKay, 1972. p. 24) Tubbs (1978) uses a four cell matrix that illustrates the ease by which even the most unintentional message can be misinterpreted. Figure 4 clarifies Tubbs' model. ihuentuxml Ihfintamfixmal V lxfl. ‘Wabal VEflxd er innenckmal 0nENEmtflxufl Nmmmnbal lkmweflmd Nammmbal Inmamimmfl. Ikdnflafijonal FIGURE 4 TUBBS' MODEL OF INTENTIONAL AND UNINTENTIONAL COMMUNICATION (Tubbs, 1978. p. 3) 32 McMahan (1976) sees verbal and nonverbal communication performing different but complimentary functions in person perception during social interaction. In his study, subjects receiving incongruent communication cues made more extreme attributions concerning a speaker's intentions than when communication cues were consistent. Sarbaugh (1979) introduced the notion of 'perceived intent' as a more productive and manageable way to operationalize the concept of intent. Using a transactional analysis theoretical orientation, he approached the intent of interacting parties from introspection about his own intent in various situations. He identifies six categories of intent: 1. To share experience, beliefs, feelings and materials. 2. To help with a task, including dealing with feelings and questions. 3. To ignore or avoid the other person, including messages. 4. To disrupt a transaction, or the efforts to establish interdependent activity. 5. To dominate the relationship through 'put downs' manipulating power, status, etc. 6. To injure the other person or group physically, socially or psychologically. This would include attacks on status, integrity, self-concept, etc. (Sarbaugh, 1979. p. 33). When two parties have the same positive intent in engaging in a transaction, the intent is more likely to be realized. Conversely, if intents are not the same, or both are negative, there is a fairly high probability that the intent of one and often both participants will be frustrated. The intent of the participants is more likely to be known if participants share a homogeneity of code 33 system, world view, values, role expectations and other normative beliefs. In most transactions, intent is not explicitly stated. It generally is inferred from prior and present cues emitted by the other party. The meaning derived from those cues then forms the basis for the 'perceived intent' which in turn sets the tone for the communication (Sarbaugh, 1979. p. 34). Sarbaugh (1979) argues that there is an integral connection between perceived relationships and perceived intent. PERCEIVED RELATIONSHIP AND INTENT Sarbaugh (1979) suggests that one of the principle variables which determines degrees of commonality in any communication interaction, is the participants' perceptions of their relationship and intent. In his taxonomy, 'perceived relationship' is defined as including perceptions of feelings, goal orientation and the structural configuration of the relationship. 'Perceived intent' is operationally defined in three dimensions; intent to share or help, intent to ignore, or intent to disrupt, dominate or injure. Baron and Byne (1977) support the notion that perceived intent is an integral component of the perception of any Situation or relationship. They consider the process of social perception in three sections. First, nonverbal communication is presented as the main channel for understanding emotions and feelings, a position supported by Knapp (1978), Ekman, Friesen and Ellingworth (1972) and 34 Birdswhistell (1970). Second, the personality characteristics such as motives and intent are inferred from observed verbal and nonverbal behavior through the process of attribution. Third, impression formation occurs when diverse bits of information are combined and integrated to form a unified concept of a person. How one expresses impressions of this unified concept of self in a relationship is central to the Thomas and Pondy (1977) 'Intent' Model of Conflict Management, and the central concern of this study. Most literature precedents agree that the perceived quality of a relationship is dependent upon the quality of interpersonal communication (Reckman and Goethals, 1973; Newcomb, 1956; and Kelley and Thibaut, 1978). The perceived quality of relationships is a topic of emphasis in both counselling, education and other helping service literatures. Little attention has been given to utilizing or operationalizing the concept of 'intent' in the forming, maintaining or repairing of helping relationships. EDUCATION AND INTENT In the literature of adult education, little emphasis has been placed on the differences between the way children and adults attribute intent and responsibility. As Piaget (1965) found, until approximately age ten, many children take the seriousness of an act's consequences into account rather than the actor's intentions, when attributing 35 responsibility. This view receives support from studies by Shaw and Sulzer (1964) and Walster (1966) who would suggest that adults, rather than children, pay far more attention to, and are held responsible for, acts that are perceived as intentional. In a study examining the different ways adults and children perceive the power of negative emotional communications, Fernster et a1. (1977) found that: 1) both adults and children perceived and communicated negative emotions accurately, 2) there was little difference between adults and children in the ability to express emotions, 3) adults perceived fear more accurately while children communicated fear more accurately, 4) adults communicated sadness more accurately than children and 5) there are developmental trends in the ability to accurately perceive verbal communications. Givens (1978) suggests there is also a tendency for people to perceive more accurately and communicate negative nonverbal behaviors. Adult strangers, not linked by clear role relationships or expectations, tend to initially respond to one another with innate adversive rather than culturally learned affiliative signals: a situation not unlike the first meetings of adult education classes. From a more traditional viewpoint, Steele (1970) uses a definition of intent that typifies how the concept is currently confined to the cognitive domain and expressed as behavioral outcomes of instruction: 36 A teacher's intents are the ideals and objectives he holds along with the outcomes he expects from whatever unit of instruction he teaches. Practices are his actions in relation to his students, including assignments and tests (Steele, 1970. p. 3371-A). Fresburg (1980) points out that the study of intent in education has been confined to either educational philosophy or specification of behavioral goals. He shares a concern with Crittenden (1974) that although teaching is an intentional activity, under pressure from the experimentalists, purpose has tended to become a rather dirty word in psychology. By association, the notion of purpose also has been neglected in research on teaching (Fresburg, 1980. p. 39). Teachers too often intend something in their goal statements but do not necessarily attempt it. Fresburg (1980) sees the role of feedback in teaching as crucial in understanding whether a teacher's intentions have been understood. The teacher functions with two controls. The first is his instructional plan of action, his intent. The second is his insightful and self- conscious interpretation of his own and his students' behavior gained through feedback (Hough and Duncan, 1970. p. 16). Stake (1969) also considers 'goals', 'objectives' and 'intents' to be synonymous. His use of 'intents' includes the planned-for environmental conditions, the planned-for demonstrations, the planned-for content coverage, and the planned-for student behavior. His model illustrates the contingencies among intended antecedents, transactions and outcomes and identifies the congruence 37 between intended and observed antecedents, transactions and outcomes. As shown in Figure 5 this 'intent' model considers contingencies between past, present and future intents, as well as judging their congruency against actual observations of behavior. l Intended i lObserved i i Antecedents f CONGRUENCE I lAntecedents i l_ ‘ l LOGICAL COMEINGENCY EMPIRICA CONTINGENCY i l Intended iObserved ‘lTransactions I I CONGRUENCE I lTransactions LOGICAL COD:IINGENCY EMPIRICQ CONTINGENCY l l Intended I I CONGRUENCE I, Observed Outcomes , IOutcomes FIGURE 5 STAKE'S MODEL FOR PROCESSING OF DESCRIPTIVE DATA (Stake, 1969. p. 118) Axinn (1974) argues that any meaningful conceptualization of education must take into account the essential function of the intent of teachers and learners. He suggests that the most practical base for categorizing educational activities is the intent of learners and teachers. It is his view "... that both (intent of learners and teachers) are essential to the educational process and 38 that the importance of neither can be dismissed if the vital and central dynamic of that process is to be appreciated and preserved, let alone improved" (Axinn, 1974. p. 13). Figure 6 illustrates Axinn's intent paradigm that shows a clear division between types of educational contexts and the subsequent intent of outcomes from both the teacher's and learners' perspective. He identifies educational outcomes as either intended or unintended and characterizes these according to the various educational systems: formal, nonformal, informal and incidental. Slamms "hamhef' 8’31”an INTENDED UNINTENDED I '1 e mar. 0 Pennecthm: Formal (school) INTENDED Non- formal In“ formal (Out-of—school) Batic UNINTENDED In-formal (Incidental) FIGURE 6 THE FUNCTION OF INTENT IN EDUCATIONAL CONTEXTS (Axinn, 1974. p. 9) Figure 7 shows how Farmer, Voravarn and Vorapipatana (1974) have utilized the concept of intent within a time frame of immediate, intermediate and ultimate consequences. 39 Abouma. -HOHuamnby coofioufimzm -manonumav. emecmuewwa», A»: Am .a .qnoH.Hm um Hofiuwm Eouw wouamvm zfluwcwoo .o>Huoommm "was an moconvomcoo o>wumwoc u : coconcomcoo Hmuunwc n z oucOScomcoo o>flufimon n + Aem>uomnoo_xuwucmuch munmuno _ munmcH _ _ A _ ooocoucH a? _ wvv is» &v mannamawo> can oousmfiw mmUZMDUMmzoo onHon Emumoua mo ooummo mflOfiumEnmm< monam> asaomonnnm oszz (1980) —>§ (1981) Ckumif. Gmxgcqy Chxmif. (kmegmsr Scanning Assessing -Scanning Assessing -Scanning ~3mxmemfing [a] > .. . .. :3 Ekphmuung Ibmdahungdhemmdng Expkunnng-nggwtnn; U ~Stating. -Stating ' i§ -<flarrfinng -4flarifinng n. Ikewmfing Itepmfing-Jkemmfing Eqmflishgrfiuuing -Fegmxhng iEnxmimg-Unhuenthml Exmnfing -Unuumnt- Emmnfing -Unhuent— uxmd fixed E -Lqfitfimne Ahafitflnne 4antfimme g: -Nolfltenuu> -Nolfltenm- <3 ive scum: <1: 2‘2 Repahfingafixflogflns Repahfing‘dkxflcgy Repahfingidfixflogy ~Penance -Penance ~Penance FIGURE 10 MODIFICATIONS TO THE THOMAS AND PONDY (1977) SCHEMA 52 misinterpretations of the instructor's intent. Second, a Reactive set of verbal behaviors which are used by the instructor after an obvious communication breakdown has occurred, and the instructor attempts to excuse his action and repair the relationship. Figure 10 illustrates how the Thomas and Pondy (1977) schema was progressively modified and developed for this study. Initially the schema was divided into four main classifications: Assessing, Explaining, Excusing and Repairing as shown in Appendix B. However, after the pilot study and the assessment by a group of expert panelists, the classification scheme was modified to include the two additional classifications of Preparing and Equalizing. Categories in the final schema were numbered 1 - 13 for ease of coding and manipulation of data. A category 13 was added to provide for the coding of utterances made by the learner. A complete summary of the final classification scheme and examples of common cues used by instructors to verbally manage intent are presented in Figures 11 and 12. Assessing. The Thomas and Pondy (1977) classification Scanning was too inclusive for the classroom situation. An additional category Requesting was added to make up a new first order classification of Assessigg. This conditional form of questioning gives the learners the freedom to respond by affirming or disagreeing with what has been asked of them. There is some degree of learner control implied in this verbal message. 53 CLASSIFICATIGC our cam ESCRIPTIG‘ G Venn. BEI'WIG ASSESSING 1 ; Asks for feedback about the learner's mental state and react on to or understanding of the instructor's actions. 2 §§QH§§¥lflfii Asks for information, action or understanding in the orm o a request. Using this conditional form of questioning, the learner is free to disagree or reject the invitation or request. PREPARING EBE§ABIN§3 Attempts to convince the learners that upcoming communi- cat on is not based on malevolent or arbitrary motives. This oc- curs when the instructor anticipates that some future behavior may be easily misunderstood or misinterpreted. To avoid upsetting the learners. advanced ground work is given as a gesture of goodwill. This may include Statements of regret. time sequence indicants or tension release in the form of humor, anecdotes or quotes. EQUALISING iugglngg Indicates the relationship between the instructor and earner is symmetrical and based on equality - neither one up or one down. The power of the instructor is shared by the use of 'wefl 'us' and 'our' messages as well as self disclosure and empathizing. 'we' messages that include the instructor but exclude the learners are not part of this category. R : Gives positive acknowledgement, encouragement. praise. re nrorcement, agreement, acceptance or concurrence with the learn- ers, without rejection. Includes the use of common courtesies and politenesses to the learner. This may be a simple please or thanks EXPLAINING §g§§§§1{g§: Gives direction in the form of a suggestion or opinion an imp ies autonomy of the learners with freedom for alternatives. : Gives explanations or statements of the instructor's purposes, goals, strategies, outcomes. reasons, objectives. etc. §L93151+flgg Restates. repeats or clarifies the instructor's origi- ns exp anation that may be misunderstood. It occurs when the instructor perceives a communication failure or potential breakdown EXCUSING Eggn§gng - ”NQNTENTIONAI: Disclaims knowledge of a previous event or e avior t at as rustrated the learner. and makes the excuse that it occurred accidentally, through a misinterpretation or lack of awareness. The purpose of the excuse is to convince the learner that the behavior was not meant to be malevolent or arbitrary. Séiuglufi - EEQITIMATE: Defends past or present actions as being i erate ut legitimate in the role of instructor to rule out any malevolent or arbitrary motive and show the action was fair and reasonable. Includes excuses where there were no alternatives but to follow established norms or rules. REPAIRING Aggpggy: Directly admits personal blame or error for a misunder- stsn ng and seeks forgiveness. It is a psychological sacrifice to demonstrate the instructor's care and concern for the learners. AN : Makes up for a wrong done to a learner when an apolo y is not enough. Directly offers a concrete form of additional he p or assistance to repair the relationship. LEARNER W3: All learner communication with the instructor. This can e initiated by the learners or occur in response to the instruc- tor's communication. FIGURE 11 FINAL REVISION OF THE CLASSIFICATION SCHEME FOR AN INSTRUCTOR'S VERBAL MANAGEMENT OF INTENT S4 CDDE CAM SMLEOJES 1 Scanning Is that clear? Is wrong" Well Doyouimderstand? mat'symopinicm" Vhat'sya2reactionto..7 Ham 2 Requesting Wildyhiexplainthatforue? Couldwehave silence? Would you move forward? Could you tellueif...? 3 Preparing Ibfortunatelytinehasninoutso. First, Vhatymsaidwasmealternativehit. Nett week... BeforeIgomIshaildTEllym. 4 Sharing We... 02... ...withus. ('we"nessages) Vbeanenttocollege... (self disclosx2e) Ithinkllmowhowyoufeel. (enpathizing) 5 Regarding Good. Mann. Yes. Fine. O.I(. 6021. That's clear Couldbe. Maybe. 'lhatwaskind ofym Please 'mankyou. Goodmrning 6 Suggesting flsymmght Ithink couldusea orpen 'Ihefieldtripcouldinclude 7 Stating mepi2poseofthisch23eis. . 'l‘hisneans. 'Ihereasonthisisincludedis. . Takecnlysix. 8 Clarifying matlueanttosaywas... Ineedtogoover thatagain... Letaerepeatthat... That is, aiFriday... Inotherwords To rephrase .. 9 Excusing Itwasanaccident. Ihadnoidea... Iwasn't lbintentimal aware Really, it wasn't meant that way. 10 Excusing Ihavenochoicebutto... Itwasunavoidable legitimate as... Themiversityrequires... Itismy respcnsibilityto insist As instructorltnist. ll Apology Iwaswrmg. Imadeamstake. Forgiveue. Sorry. Iapologize. Itwasmyfault. 12 Penance letnemakethatuptoyuiby... WnatcanIdo to... Perhapslcaildrepairmyblunderby... 13 learner * (All learner cannmication with the instructor) FIGURE 12 SAMPLE CUES TO ASSIST IN IDENTIFYING VERBAL CATEGORIES 55 Preparing. At first, the Thomas and Pondy (1977) classification of Preparing was seen as one specific type of an explaining activity and seemed better oganized as a category within the Explaining classification. However, after pilot testing it was apparent that instructors used the preparing behaviors so frequently that it was necessary to create a separate classification. It was found that the instructors often used humor, jokes, anecdotes, poems, stories and sequencing indicants to forewarn the learners that upcoming communication would not be based on malevolent or arbitrary motives. These unique ways of laying the groundwork to avoid upsetting learners did not fall neatly within the classification of Explaining. Equalizing. After the pilot test, the classification of Equalizing was added to the scheme. Informal conversations with adult learners indicated their need for an empathetic instructor who would not talk down to them. This empathy aspect of a relationship is.described by Rogers (1969) in terms of unconditional positive regard. The simple acceptance of learner responses without rejection, the extending of basic courtesies and self- disclosure (Kossen, 1975) by the instructor was seen as an activity to equalize the power in a relationship. In the pilot test, it appeared that the 'most effective' instructors used identifiable communication behaviors in an effort to make the relationship appear more equal. That is, neither one up or one down. 56 Two categories make up the classification Equalizing. First, the Sharing category indicates the relationship between instructor and learner is symmetrical with the interaction based on equality. The power of the instructor is shared by the use of 'we', 'our', 'us' messages, as well as the use of self-disclosure. Second, the category Regarding incorporates the use of positive praise or the extending of courtesies or politenesses normally accepted in society such as a simple 'please' or 'thankyou'. It also includes the instructor's positive regard or acknowledgement of learners' responses through agreement, acceptance or concurrence so as to avoid rejecting their ideas as worthless. The addition of this classification and its two categories weakened the original classification scheme by reducing its previous simplicity. Both categories in the final Equalizing classification are not discrete. All other categories used in the scheme are discrete categories. Both the Sharing and Regarding categories can occur on their own, but more frequently were found to occur as an integrated part of the message of another category in the scheme. For example, the utterance: 'Could you please move to the front of the room?', combines the categories of Requesting and Regarding. While the Thomas and Pondy (1977) model of intent does not include the dimension of control as being important in a person's management of intent, it was found to play an 57 important and persuasive role in a learner's perceptions of the adult education instructor's management of intent. Thus these combination categories were incorporated in the new classification of Equalizing in the present scheme. Explaining. The Thomas and Pondy (1977) classification of Explaining was found to be far too inclusive to adequately distinguish the various types of explaining activities typically used by instructors in the classroom. This classification was broken down initially into two separate categories of Stating and Clarifying. The category of Stating accomodates the direct verbal explanation of an instructor's intent. This includes statements or explanations of educational purposes, goals, objectives, strategies or outcomes. The category of Clarifyipg was added to differentiate another type of explaining activity that is used to restate or clarify an original message that might have been misinterpreted or misunderstood. It occurs when an instructor perceives or anticipates a communication breakdown. An additional category of Suggesting was also added after the initial pilot study as the 'most effective' instructors used suggestions when communicating in a ratio of about 4:1 to those instructors identified as 'least effective'. Suggesting is conceptualized as giving a direction in the form of an opinion or recommendation. This indirect form of explaining implies autonomy for the 58 learners as the suggestion is phrased in a conditional form. Freedom for learner response is implied as an alternative response can be offered by the learners. The words 'perhaps', 'might', 'could', and 'would' identify Suggesting utterances. Excusing. The post hoc excusing of behavior which Thomas and Pondy (1977) describe in terms of Unintentional Excusing, No Alternative Excusing and Legitimate Excusing were more specifically operationalized, but without substantive changes in the concepts. After the pilot study, it was found that coders had difficulty in distinguishing between the No Alternative and Legitimate examples of the Excusing categories. As these categories were infrequently used in the classroom situation, the two categories were collapsed under the category Excusing - Legitimate. Repairing. The repairing of a relationship which has been damaged by the learner's misinterpretation of the instructor's intent is termed by Thomas and Pondy (1977) as Apologies and Penance. These categories were further operationalized for this study. The category of Apology includes verbal behaviors used to accept personal blame or error and ask for forgiveness. It is a psychological sacrifice to demonstrate the instructor's concern for the learner. The category of Penance includes verbal behaviors offering a more substantive form of instructor sacrifice when it is obvious an apology is not enough to repair the 59 relationship. These distinctions were inferred by Thomas. and Pondy (1977) but never specifically articulated. These two categories make up the final classification of Repairing in this study. Summary of Modifications. The final nomenclature for identifying an instructor's verbal management of intent incorporated six classifications and twelve categories. As displayed in Figures 10 and 11, it includes: Assessing - Scanning and Requesting; Preparing; Equalising - Sharing and Regarding; Explaining - Suggesting, Stating and Clarifying; Excusing - Excusing Unintentional and Excusing Legitimate; and Repairing - Apology and Penance. A further category Learners was added to allow examination of the interaction between instructor and learners. It includes communication with the instructor that is initiated by the learners as well as the learners' responses to the instructor's communication. To ignore learner communication would be violating the interactive nature of any communication process. DESIGN OF A CODING TECHNIQUE A coding technique was designed to record and analyze the frequency and sequence of utterances incorporated in an instructor's verbal management of intent. To field test this technique, audio-tapes were made of thirty two instructors' verbal communication during their initial lesson and the tapes of the five 'most effective' 60 and the five 'least effective' instructors were then transcribed and coded. This transformation of qualitative data into quantitative data involved two operations. First, the transcripts of each instructor's communication were separated into utterances. Second, a category code was assigned to each utterance. To assist coders in more accurately conceptualizing and recording their perceptions of the categories, a number of recognition rules were developed as criteria for breaking up transcripts into segments (unitizing) and then assigning a category or combinatiion category to each segment (categorizing). These rules are described in Appendix C. Design for Selecting Groups of the 'Most Effective' and 'Least Effective' Instructors A learner response form, based on the Anderson and Anderson rating scale of initial rapport (Anderson and Anderson, 1962) was adapted and refined during the pilot test. It served as a criterion measure to identify from thirty-two instructors sampled, the five instructors 'most effective' in managing their intent, and the five instructors who were 'least effective' in managing their intent. It was assumed that instructors who were successful in establishing adequate classroom rapport or 'ideal relationships', would also have been seen by their learners as well-intentioned. The converse would be true for those instructors who were 'least successful' in establishing rapport with their learners. 61 The Anderson and Anderson (1962) rating scale of initial rapport was the most reliable and valid instrument available to measure 'ideal relationships'. It reflects observable communication behaviors within relationships which are conceptually consistent with the management of another's perceptions of intent. It also is a more appropriate and simpler measurement tool than the Barrett- Lennard Relationship Inventory (1962) which is more reliable in measuring 'something of the adequacy of certain kinds of important relationships' (Lake, Miles and Earle, 1973). The Instructional Process/Environment Questionnaire (Pratt, 1979) which was developed especially for use in adult education classrooms was rejected for lack of sufficient validation or proven reliability. Details of the development of the student rating scale are provided in Appendix D. Design for Recording and Transcribing To record an instructor's verbal management of intent a microcassette audio tape was made of the first hour of the initial lesson for each instructor sampled. Since the coding technique was designed to analyze the content component of an instructor's verbal message, the use of an audio tape eliminated the nonverbal communication channel. Further, typed transcripts eliminated paralinguistic cues that might distract coders from the content of the message. Details of procedures used for recording are described in Appendix E. 62 Audio tapes were transcribed, the transcription independently checked for errors, typed and proof read. Special attention was paid to punctuation and appropriate paragraphing. Procedures for transcribing tapes are described in Appendix F. Design for Coding Categories and Combination Categories The first twelve categories in the classification scheme in Figure 11 define the functional behaviors that an instructor employs to verbally manage his intent. Category thirteen defines any learner verbal interaction with the instructor. For ease of coding, each of the thirteen categories were designated a corresponding arabic number from 1 to 13. All categories, except for the categories of Sharing (category 4) Regarding (category 5) are mutually exclusive and were defined so only one event could be coded at any one time. The two categories Sharing and Regarding can occur as separate events as well as concurrently with all other categories except themselves and category 13. For example, an instructor's utterance '"Joe, what's wrong?'", is first coded as Regarding (category 5) because the instructor acknowledged the learner by name. However, the utterance is a question that seeks information about the learner's mental state and is therefore also coded as Scanning (category 1). In this case, two categories occurred concurrently in the same utterance so the coder would combine the category numbers and form a new combination category of 51. 63 For uniformity of coding, when category 4 or category 5 are combined with any other categories, the 4 or 5 category always precedes the other category in the combination. This creates twenty-two possible combination categories. They are: 41, 42, 43, 45, 46, 47, 48, 49, 421, 411, 412 and 51, 52, 53, 54, 56, 57, 58, 59. 510, 511, 512. The total number of discrete categories or combination categories that can be derived from the final coding technique is thirty-five. That is, thirteen major categories and twenty-two combination categories. These thirty-five discrete categories and combination categories are exhaustive and inclusive of all the possible instructor verbal communication of intent that takes place between an instructor and his learners. Rules for Unitizing and Categorizing. To enable coders to identify the occurrence of a category or combination category, sample cues for each category were included as an addendum to the category definitions of the classification scheme. During the pilot test and with the assistance of the coding team, these rules were formulated and refined. A final set of recognition rules for unitizing and categorizing, along with procedures for unitizing and categorizing transcripts are described in Appendix G. Validity of the CodingfiTechnique. To enhance the validation of the conceptual framework, classification scheme and coding technique developed for this study, two 64 procedures were adopted. First, a reaction panel of six experts were invited to act as external judges and offer feedback on its sufficiency and appropriateness. Second, to gain a measure of predictive validity in a field test, the application of the coding technique should show that substantial differences exist between the frequency or sequences of utterances used by instructors who are the 'most effective' verbal managers of intent and those who are 'least effective'. Design for Analysis and Display of Coded Data The functional utility of the coding technique is dependent upon its potential to assist in the collection, tabulation and interpretation of utterances or utterance sequences which recur or combine in a variety of ways. The following procedures were designed to assist in the meaningful analysis of coded data. Tabulating and Recording. Transcripts were coded, the coding recorded on tally sheets and keyed into a microcomputer in sequences of 80 utterances. A display screen with a capacity for eighty characters allowed visual checking while entering code numbers. The entire sequence of coded utterances was printed by the computer and checked for errors. A suite of six microcomputer programs were used to assist in the tabulation, display and analysis of data. The first computer program was used to input and print the 65 entire sequence of coded utterances used by an instructor. A sample of a tally sheet and a printout are shown in Appendices H and I respectively. A second program was used to edit the sequence of coded utterances and allowed alterations, deletions and insertions to be made. The remaining four programs printed frequency reports of instructor utterances, paired utterances or sequences of three utterances. l. Prequencngeports The third program printed a statistical summary of utterances used by an instructor. The statistical summaries for the 'most effective' and the 'least effective' groups were also analyzed to calculate an aggregate summary for each group. ‘A statistical report for each instructor shows the frequency and totals for classifications, categories and combination categories as well as percentages for categories and combination categories. A sample Frequency Report is shown in Appendix J. 2. Sequence Reports of Paired Utterances The fourth program calculated and printed a report of all utterance pairs used by an instructor. Aggregated summaries for more than one instructor were also printed. This matrix provides a display of a possible 1925 pairs which the 35 category coding scheme can produce. The matrix consists of 35 rows and 35 columns. Rows are read horizontally on the matrix and columns are read 66 vertically. .Each of the 35 rows and columns correspond to one of the 35 major categories or combination categories of the coding technique. Each row represents a first utterance, and each column the succeeding utterance. The utility of pairing utterances in a sequence of dialogue is shown in Figure 13. Coded Transcript: "Goodsgirl./ Take that up slowly./ (Learnerliesponds)/ Don't go too fast,/ because if youego too fast it will break./ John howSAre you going?/ (Learnir responds)/ Soiiy,/ didn't mean t3 disturb you."/ Paired Utterances: lst pair 3rd pair 5th pair 7th pair ('“J‘wfi r—M F—‘g—w r-_J\—-—— 5 7 l3 7 8 51 13 ll 9 ‘JW ‘ffi—j LINN L——-..H.-. ' 2 d pair 4th pair 6th pair 8th pair FIGURE 13 PAIRING UTTERANCES IN A SEQUENCE As illustrated in Figure 13 the sequence in the coded transcript is represented by a series of code numbers that are read from left to right. To form a paired sequence, each code number is used twice, except for the first and last numbers which are only used once. In the above sequence, nine category codes or combination category codes were used to form eight pairs. As in the matrix design that 67 uses this method of pairing, there will always be 'n-l' pairs from 'n' major category or combination category codes. 3. Sequence Reports of Three Utterances The fifth and sixth programs identified and printed the frequency of predecessors and successors of selected utterance pairs. An utterance pair from a two-way summary may be chosen for analysis only if its frequency of occurrence is at least one percent of the table total. To find the frequency of predecessors or successors of just one utterance pair at a 1% threshold involves a search of a 35 x 35 x 35 matrix and its 42,875 possible combinations of a sequence of three utterances. A sample Sequence Report of Three Utterances is shown in Appendix K. Interpretation of Matrices. First, if two or more matrices have unequal total frequencies, selected tally totals for cells of the two-way matrices are converted into percentages. This allows comparisons between various categories used by instructors to verbally manage their intent. Composite matrices of a number of instructors can also be compared using this method. Second, inferences about often used sequences of utterances are made. By understanding the relationship between the rows and the columns, probability statements are made about what precedes or what follows a paired category of interest. Using procedures to analyze matrices suggested 68 by Flanders (1970) and Ellis (1977), flow diagrams of utterance sequences, based on matrix data, are identified. Third, the matrix is used to identify cells of interest in which there is a heavy concentration of tallies, as well as cells in which there are no tallies. Fourth, as the tallies in the matrices represent frequencies or proportions of nominal data, statistical tests for data analysis are limited to nonparametric techniques as suggested by Siegel (1956), Flanders (1970) and Ellis (1977). They suggest the chi square statistic is an appropriate procedure for use with matrices of frequency data which represent contiguous states. Chi square tests for goodness of fit or as a test of independence are employed to determine whether: 1) the frequencies of utterances and utterance sequences used in one part of an instructor's lesson differ significantly from another part of the same lesson; 2) the frequencies of utterances and utterance sequences used by instructors identified within a homogeneous group are similar to one another; and 3) the. frequencies of utterances and utterance sequences between instructors who are 'most effective' managers of their intent are different from those who are 'least effective' managers of their intent. Cochran (1954) and Mimball (1954) provide procedures for the correct partitioning of chi square contingency tables to help find specific sources of variance. -69 Design for Reliability and Validity of the Coding Technique In order to assess the reliability and validity of this newly developed coding technique, a number of strategies were adopted to establish if the coding technique: 1) could be learned and applied with consistency between independent coders; 2) could be judged by a panel of experts to have content validity; and 3) could show that significant differences exist between the frequency of sequences of utterances used by instructors who are 'most effective' or 'least effective' managers of intent. Coder Reliability. The issue of being able to use this coding technique with reliability is crucial to the study. Frick and Semmel (1978) suggest a number of practical means for minimizing intercoder error. Coders should reach nearly perfect agreement on unambiguous examples with the expert coder before actual data collection. Coders should also be expected to reach agreements on ambiguities that might arise in coding. They emphasize: Criterion-related and intraobserver agreement measures have been recommended for both before and during a study, but these measures should not be used as evidence of observer agreement in the actual classroom. Rather, these are measures to assist an investigator in documenting adequacy of observational skills. The purpose of such efforts is to minimize the possibility that observers are primarily responsible for potentially unreliable observational data. Ellis (1977), argues that the critical issue is that only a coding scheme that can be learned and applied with 70 consistency is reliable. He identifies four sources of poor reliability: The first three are research design such as inadequate data sampling; poor training of observers; and the coding scheme itself (e.g., too few categories, poor category definition). The fourth problem area is statistical tests for reliability. (Ellis, 1977. p. 12). Ellis (1977), Hirokawa (1980) and Donohue et a1. (1981), argue for the superiority and appropriateness of Guetzkow's estimate of reliability with sequential interaction data. Guetzkow's (1950) formulae are straight forward procedures for calculating unitizing and categorizing reliability coefficients. As well, they take into account the complexity of the coding scheme and the degree to which coders agree with each other. To ensure the coding technique was learned and applied with consistency between independent coders, Guetzkow's (1950) procedures for estimating unitizing and categorizing reliability were adopted. These estimates for unitizing reliability provide a measure to compare how often independent coders make errors when dividing up the same transcript into utterances. For example, a page of a transcript may be divided into utterances at different points such that the total utterances obtained for each coder are equal in number, but the utterances may not be coterminous. As well, two coders may end up with a different number of utterances if one coder regards a given sentence as containing two or more utterances while the other coder regards the same sentence as only one utterance. 71 An explanation of Guetzkow's (1950) formula for calculating the reliability of the unitizing process is included in Appendix L. To ensure agreement among coders when assigning category codes to utterances, Guetzkow's (1950) procedure for estimating categorizing reliability was used. This was based on the assumption that dividing a transcript into utterances is independent of the subsequent assigning of a category code to each utterance. The procedure for calculating the reliability of categorizing requires the use of two formulae as shown in Appendix M. Guetzkow (1950) argues that for practical purposes, "experimenters need not have more than 150 units of qualitative material classified by two coders to obtain stable estimates of the probability with which each unit is classified correctly" (Guetzkow, 1950. p. 54). He concurs with Frick and Semmel (1978) that periodic checks of masses of up to 150 utterances are needed to ensure coding standards are being maintained. For this study, acceptable intercoder coefficients between independent coders were set at the .95 level when computing unitizing reliability and .80 for categorizing reliability. SUMMARY The purpose of this chapter was to describe and explain the conceptual development and design of the 72 theoretical framework, classification scheme and coding technique. The direction of explanation moved from general theoretical issues to the specific operational definitiions of the categories used to measure an instructor's verbal management of intent in the instructor/learner relationship.. The three sections included the: 1) Theoretical Orientation; 2) Development of a Classification Scheme for Measuring an Instructor's Verbal Management of Intent; and 3) Design of a Coding Technique. In the following chapter, Methodology of the Study, the procedures employed during the six major phases of this study are described. These include the procedures followed from the development of the initial draft of the coding technique through to its final field testing. CHAPTER 4 METHODOLOGY OF THE STUDY In this Chapter the methodology employed in each of the six major phases of the study are described and explained. The phases were: 1) the development of an initial draft of the conceptual framework, classification scheme and coding technique; 2) a pilot test to establish the adequacy of the classification scheme and the feasibility of the tentative coding technique; 3) the content validation of a second draft by a panel of experts; 4) final modifications based upon the feedback provided by the panel of experts; 5) the training of a group of coders to learn the classification scheme and apply the coding technique with reliability; and 6) the field testing of this technique. PHASE 1 DEVELOPMENT OF AN INITIAL DRAFT OF THE CONCEPTUAL FRAMEWORK, CLASSIFICATION SCHEME AND CODING TECHNIQUE A tentative classification scheme and coding technique were developed and shared with a number of American adult education practitioners. The resulting reactions indicated the need for modification of the coding procedures and use of an event sequential approach. 73 74 4 PHASE 2 PILOT TESTING TO ESTABLISH THE ADEQUACY OF THE CLASSIFICATION SCHEME AND FEASIBILITY OF THE CODING TECHNIQUE On returning to Australia, a pilot study was conducted in order to: l) establish the adequacy of applying the classification scheme to a natural classroom setting; 2) establish if acceptable intercoder reliability coefficients could be reached between the researcher and a colleague in its use; and 3) refine the data gathering procedures and to test the adequacy of the instruments to be used in the field test. Seven procedural steps were followed during the pilot study. These were: 1) selecting the participants for the pilot study; 2) protecting the anonymity of participants; 3) testing the suitability of microcassettte recorders; 4) transcribing and typing of transcripts; 5) establishing the adequacy of a student response form; 6) establishing acceptable intercoder reliability coefficients using the coding technique; and 7) thanking participants in the pilot study (see Appendix N for a summary of these procedures). The pilot study achieved three objectives. First, it was established that with modifications, the classification scheme could be applied to a natural classroom setting. Second, acceptable levels of intercoder agreement were reached between two independent coders when using the tentative coding technique. 75 Third, the following data gathering procedures or instruments were tested and refined: l) a microcassette recorder that produced quality one hour audio-tapes was selected and recording procedures were established; 2) procedures for transcribing were developed; 3) the modified Anderson and Anderson (1962) learner response form was adapted for the Australian classroom setting and its use as an adequate criterion measure to distinguish instructors who are 'most effective' and 'least effective' in establishing rapport with learners was achieved; and 4) procedures for the selection of participants, protection of their anonymity and follow up correspondence were established. PHASE 3 THE CONTENT VALIDATION OF A TENTATIVE CONCEPTUAL FRAMEWORK, CLASSIFICATION SCHEME AND CODING TECHNIQUE Upon completion of Phases 1 and 2, a second draft of the conceptual framework, classification scheme and coding technique was constructed. To enhance the content validation of this second draft, a reaction panel of experts were selected and invited to provide feedback on its adequacy. A panel of six persons were invited to carry out three tasks: 1) to respond to specific questions about the clarity, relevance, format and examples provided in each of the classifications of the scheme; 2) to respond to a number of general questions regarding the overall strengths and weaknesses of the classification scheme and coding 76 technique; and 3) to make any general recommendations to improve the overall quality of the conceptual framework, classification scheme or coding technique. a Appendix 0 provides a summary of the criteria and procedures used to select the panelists, as well as a description of the tasks they were requested to carry out. *1 PHASE 4 FINAL MODIFICATIONS TO THE CONCEPTUAL FRAMEWORK, -:, CLASSIFICATION SCHEME AND CODING TECHNIQUE Feedback elicited from the panel of experts provided critical insights into the strengths and weaknesses of the second draft of the conceptual framework, classification ‘ scheme and coding technique. An attempt was made to achieve consensus for each question posed and for the general commemts made by the panelists. The major changes made to the second draft as a result of this feedback are described in Appendix P. PHASE 5 TRAINING OF A CODING TEAM f1 In this crucial phase of the study, acceptable levels of intercoder agreement had to be achieved before commencement of the field test. A number of people not initially aware of the research purposes had to learn how to break down sections of speech into units and code them according to the classification scheme. Each coder had to work independently and obtain intercoder agreement around the .80 level. 77 Five major tasks were undertaken in this phase of the field test: 1) identifying persons for the coding team; 2) preparing a training schedule; 3) training of coders; 4) coding of field test transcripts and remedial training and 5) bringing closure to this phase of the study. Detailed procedures used in this phase of the field test are provided in Appendix Q. PHASE 6 THE FIELD TESTING OF THE CODING TECHNIQUE The purpose of this final phase of the study was to determine the predictive validity of the coding technique when field tested in a natural classroom setting. If the coding technique is to be used with any confidence, it seems reasonable to expect that it distinguishes some consistent and meaningful differences between the verbal communication used by instructors who are 'most effective' or 'least effective' managers of intent. In order to achieve this purpose, the verbal communication behaviors used by a group of five instructors identified by their learners as 'most effective' and a group of five instructors identified as 'least effective' in managing their intent, were systematically analyzed and compared. Specifically, the following research questions guided this phase of the study: 1.0 Are there differences among the verbal communication behaviors used by the five instructors who were identified 78 by learners as 'most effective' managers of their intent? 1.1 Are there differences among the verbal communication behaviors used by the five instructors identified by learners as 'least effective' managers of their intent? 2.0 Do the verbal communication behaviors used by groups of the 'most effective' instructors differ from those verbal communciation behaviors used by groups of 'least effective' instructors? 3.0 Are there patterns of verbal communication which are used more consistently by the group of 'most effective' instructors than by the group of 'least effective' instructors? Conversely, are there patterns of verbal communication which are used more consistently by the group of 'least effective' instructors than those used by the group of 'most effective' instructors? In an attempt to answer these questions, the following procedural steps were followed in the field testing of the coding technique. Step 1: Selection of Subjects Subjects were drawn from a convenience sample of thirty-two adult education instructors attached to four evening colleges in a metropolitan region of Australia. None of the sixteen male or sixteen female instructors held formal qualifications in adult education and two-thirds of the instructors were not trained teachers. Class sizes of the sampled instructors ranged from five to nineteen participants. 79 The following sequence summarizes the procedure utilized in the selection of subjects: 1. A state coordinator and regional coordinator for adult education were contacted by phone and letter in order to gain permission to approach evening college Principals in their area. Permission was granted. 2. Five evening college Principals, including one College Board, were contacted by telephone, letter, and then by a personal visit. 3. Working in cooperation with four Principals, participation in the field test was presented to instructors as a mutually beneficial and reciprocal learning experience. A letter was written which invited instructors to help in the field test (see Appendix R). 4. Principals distributed the letter to their evening college instructors and over a period of two weeks noted the details of those willing to participate in the field test. 5. Thirty-eight interested instructors were phoned for the following purposes: 1) to explain details of the field test; 2) to gain approval to record their first or second class meeting using an unobtrusive micro-cassette recorder; and 3) to request that five minutes before the end of the lesson, the learners be given time to respond to a student response form. It was stressed that participation was voluntary and anonymity of all participants would be protected. 80 6. Dates, times and locations were finalised with the thirty-two adult education instructors and two hundred and ninety-five learners who voluntarily assisted in the field test. 7. None of the participants were informed of the exact nature of the investigation until data gathering was completed. To avoid the introduction of any unnecessary bias, the study was explained in general terms of examining different communication styles used in adult education classrooms. 8. Before the completion of the college term, a written invitation was extended to all the Principals and participating instructors of each evening college, to visit the researcher's home to discuss the study (see Appendix S). As a reciprocal gesture, the researcher presented a session on classroom assessment techniques at the inaugural adult education inservice training workshop for the region. 9. At the end of the study, a brief summary of the results were mailed to all participants in the field test. Step72: Data Gathering A data collection team was trained in the use of micro-cassette recorders as well as procedures for administering the learner response form at the end of each instructor's lesson. These procedures are described in Appendix E. Audio recordings were made of each two hour lesson and learner response forms were administered five minutes 81 before the end of each lesson. They were collected and placed in a large envelope clearly marked with the instructor's designated number. At the end of each evening, the response forms were scored and results recorded. Step 3: Identifying Groups of 'Most Effective' and 'Least Effective' Managers of Intent The learner response form, developed and refined during the pilot test, served as a criterion measure to identify from the thirty-two instructors sampled, the five instructors 'most effective' in managing their intent, and the five instructors who were 'least effective' in managing their intent. A summary of the mean scores of the learner response forms is provided in Appendix T. Step 4: Transcribing Tgpes and CodingfiTranscripts The first hour of the audio tapes of the 'most effective' and 'least effective' groups of instructors were transcribed, checked for accuracy, typed, and proof read. The completion of these tasks involved approximately twenty hours of working time. Procedures for these tasks are described in detail in Appendix F. The transcripts were then coded by the researcher and the coding team using procedures described in Appendix G. Acceptable intercoder coeffecients between independent coders were set at the .95 level when computing unitizing reliability, and at the .80 level for categorizing reliability. 82 Unitizing and categorizing reliability coefficients above .80 were established for the initial 400 utterances of - the first three transcripts coded. Acceptable levels of agreement above .95 were obtained for randomly selected segments of 100 utterances in the other seven transcripts coded. For further details of these coefficents, see Appendix U. Feedback was provided to each coder on the reliability coefficients obtained and review sessions were conducted to focus on commonly occurring coding errors. After completion of the study, letters of thanks and a summary of the research findings were sent to each member of the coding team. Step 5: Display of Coded Data Once acceptable levels of intercoder agreement were reached for unitizing and categorizing the coded transcripts, the coded data was tabulated and keyed into a microcomputer. This was analysed using a suite of six programs developed for the study and for use with a 48K desk computer. Step 6: Analysis of Coded Data In order to answer the research questions that were formulated to guide the systematic analysis and comparison of the verbal management of intent between the group of 'most effective' and 'least effective' instructors, the following analyses of coded data were carried out. 83 Research Question 1.0 Are there differences or similarities among the verbal communication behaviors used by the five instructors who were identified by learners as 'most effective' managers of their intent? To determine whether the verbal communication used by the five instructors in this group differed from each other, chi-square tests for goodness of fit were employed.. Significance levels were set at the .05 level. Additional partitioning of 2 X 5 contingency tables into 2 X 2 auxiliary tables was carried out following procedures suggested by Cochran (1954), Kimball (1954) and Ellis (1977). This allowed subdivision of chi-square into components to reveal specific sources of variance among the five instructors in the 'most effective' group. a. Conversion of Frequencies into Proportions. Because of the unequal numbers of utterances used by the five instructors in the first hour of their lessons, the frequency data were converted to proportions. This allowed meaningful camparisons to be made within this 'most effective' group of instructors. b. Differences Between Categories and Combination Categories For each instructor, a tally was made of the use of the major categories and combination categories. The total number of utterances made by each instructor was recorded. Then, the use of any one major category or combination category was calculated as a proportion of the total number of utterances used by that instructdr during the first hour 84 of the lesson (categories 1 to 12). The frequency of the utterances made by learners, (coded as category 13), were calculated separately as a proportion of all utterances used by both the instructor and the learners (categories 1 to 13). Using Brandt and Snedecor's formula for computing chi-square in a 2 X 5 contingency table (cited in Cochran, 1954), tests of significance for differences between proportions were calculated for each category or combination X2 = ijj - pTx. Pq A step by step guide to computation of this chi-square test, category. The formula used was: the use of auxiliary 2 X 2 tables, and the subdivision of X2 into components, is provided in Cochran (1954, pp. 430- 434). Possible sources of variance between proportions were then identified. 2 X 5 contingency tables were further partitioned into selected 2 X 2 auxiliary tables to subdivide chi-square into components and to reveal specific sources of variance among the 'most effective' group of instructors. c. Differences Between the Use of Selected Paired Utterances Using a matrix for each instructor, the most frequently occurring paired utterances were identified and rank ordered. The ten most frequently occurring paired utterances were then selected for analysis and comparison. 85 A prepared worksheet similar to that shown in Figure 14 was used to compute possible sources of variance between the proportional use of selected paired utterances for each instructor in the 'most effective' group. A 2 X 5 contingency table was partitioned into selected 2 X 2 auxiliary tables to subdivide chi-square into components and reveal specific sources of variance between the five instructors. d. Differences Between the Probability of One Category Following Another in Selected Pairs Two criteria were applied to select those paired utterances to be investigated and those instructors in the 'most effective' group who showed statistically significant similarities in their use of the particular paired utterance under investigation. First, the ten most frequently used paired utterances were selected for investigation. Second, only those instructors within the 'most effective' group whose use of the paired utterance did not differ significantly from one another were selected for comparison (a homogeneous subgroup). Third, using the matrices for each of the 'most effective' instructors of this homogeneous subgroup, the cell frequencies for the paired utterances under investigation were compared. 86 s g; 8 . g ' g ‘8 .3”: ,3, Calculation of X2 m g," g 39 3'75 §‘ within the "Effective" group .5 L (‘34 08 gig .3: CATEGORY l & 2 Subdivision of X2 into couponents 1 92 54.! 643 .1461676 ' 2 '33 (944(- 833- ‘1';qu tructor df 8.8. X2 p 3 77 1063 ”40 .06754-3 2.:3 I 12.070501! 90.73 .00; 4 ”q m5 954.142.6259 1:4:5 2 .39446/ 3.26 NS 5 )84 Q?! I165 .6745“) 2+32H-‘H-5 I . (3524c! ,,go NS 2 650 5964 4-4nl4-= .I409756 Z . 4 12.6IO3OI 104.161 .00! iAEE—GPE? 3 Subdivision of X2 into canponents 1. 45 5612 64-3 .oeqqiw 2 55 '7‘77 832 .oebnose Insmmto“ df 55- X2 P 3 us 1028 114-0 0182456 1:2. I .oosgisso .0: ,6 4 94. 750 334 .1007194 3:415 2 .0050360 ooz N5 5 120 104-5; ”(fit/030043 1+2 :st ’ 1.0945510 13.42 -0’ z 415 4.148 4614:.0010160 ;/./02843 CATEGORY 4 1 l 64-2, 64-3 14015552. tructor df 8.5. X2 p 2 '9 3‘3 831. “©9365 I: I .0070652. .7ZEHS 3 15 “25 114-0 .0131574 314 I .0061232. «2?... Ni 4 3 :24 93+..ooqsqz5 ”5:3”; I -07744/5 7.851'00 5 3 ”62, 1:65 $.0025751 1+5~r3+4 :3 I l' /6?0/ (H .‘ /'/.02. .00; I 4.4, 45683 #614 200996017 2 41258é747 26.21.0211 FIGURE 14 WORKSHEET USED TO CALCULATE SOURCES OF VARIANCE BETWEEN PROPORTIONS 87 Fourth, the probability of the succeeding category of an utternace following its preceding category in a selected paired utterance was calculated using the following formula: Cell frequency for succeeding utterance Row frequency for preceding utterance P: Fifth, using a prepared worksheet, similar to that shown in Figure 14, possible sources of variance between the probability of one category following another category in a selected paired utterance was computed. A 2 X 5 contingency table was partitioned into selected 2 X 2 auxiliary tables to subdivide specific sources of variance between homogeneous groups of instructors. e. Differences Between Selected Sequences of Three Utterances Sequences of three utterances were selected if the initial paired sequence appeared as 5% or more of the total utterances used by each instructor. The five most frequently occurring sequences of three utterances were rank ordered and compared. f. Differences Between Categories and Paired Categories Used in the First Half and the Second Half of a Lesson To establish if an instructor's verbal management of intent remained constant throughout the lesson, the total utterances used by that instructor were divided into the first and second halves of the lesson and then compared. Chi-square tests of association were employed to detect statistically significant differences between the utterances used in these two equal segments of the lesson. 88 Researchguestion 1.1. Are there differences or similarities among the verbal communication behaviors used by the five instructors who were identified by learners as 'least effective' managers of intent? To identify any differences, the same procedures were used as have been described for the 'most effective' group. Research Question 2. Do the verbal communication behaviors used by groups of 'most effective' instructors differ from those verbal communication behaviors used by groups of 'least effective' instructors? To answer this question adequately, it was necessary to select only those instructors within the 'most effective' group that showed homogeneity in their use of the category or paired utterance under examination. Similarly, it was necessary to select only those instructors in the 'least effective' group using the same criterion. Thus, only those subgroups of instructors already identified as homogeneous in response to Research Questions 1.1 and 1.2 were compared. These homogeneous subgroups identified in the 'most effective' group and in the 'least effective' group were then compared to assess if there were any statistically significant differences between: 1) the use of all categories and combination categories; 2) the use of selected paired utterances; and 3) the probability of one category following another in selected paired utterances. 89 Chi-square tests for independent samples were employed to establish if there were statistically significant differences. A significance level of .05 was set for all comparisons. To compare differences between the use of selected sequences of three utterances in each group, the five most frequently occurring sequences were identified, rank ordered and then compared. Research Question 3. Are there patterns of verbal communication which are used more consistently by the group of 'most effective' instructors than by the group of 'least effective' instructors? Conversely, are there patterns of verbal communication which are used more consistently by the group of 'least 'effective' instructors than those used by the group of 'most effective' instructors? Using the similarities and differences already identified within and between instructors in the 'most effective' and 'least effective' groups, consistently occurring patterns of verbal communication used to manage intent were compared. SUMMARY In this Chapter, the methodology employed in each of the six major phases of the study were described and explained. The phases were: 1) the development of an initial draft of the conceptual framework, classification scheme and coding technique; 2) a pilot test to establish 90 the adequacy of the classification scheme and the feasibility of the tentative coding technique; 3) the content validation of a second draft by a panel of experts; 4) final modifications based upon the feedback provided by the panel of experts; 5) the training of a naive group of coders to learn the classification scheme and apply the coding technique with reliability; and 6) the field testing of this technique. In the following chapter, the findings of the field test are reported. These are presented in three parts: 1) instructor effectiveness as rated by learners; 2) coding reliability; and 3) the analysis and comparison of the 'most effective' and 'least effective' groups of instructors in the verbal management of their intent. Chapter 5 FINDINGS OF THE FIELD TEST In this chapter, the findings of the field test are reported. The verbal communication behaviors used by the 'most effective' and 'least effective' groups of instructors were analyzed and compared. The following summary highlights the major similarities or differences found during the field test in the use of the categories, paired utterances and sequences of three utterances: l. Instructors who were perceived as most helpful and well-intentioned when establishing the instructor/ learner relationship tended to verbally interact more frequently with their learners throughout the lesson than those instructors who were 'least effective' managers of intent. 2. Instructors who were perceived as most helpful and well-intentioned when establishing the instructor/ learner relationship tended to interact more frequently with their learners throughout the lesson by the use of: 1) Scanning (questioning to assess if learners have misunderstood or misinterpreted their instructor's intent), and 2) Regarding (reinforcement of learner responses through the use of positive praise or acknowledgement of the learners' contributions or efforts). 91 92 3. Instructors who were perceived as most helpful and well-intentioned when establishing the instructor/ learner relationship tended to indirectly explain their intent by making a suggestion rather than always using a direct statement or giving an order to the learners. 4. Instructors who were perceived as most helpful and well-intentioned when establishing the instructor/ learner relationship were the only instructors to follow up an apology for an error or misunderstanding with an offer to do something concrete to repair the relationship. 5. Instructors who were perceived as the least helpful or least well-intentioned when establishing the instructor/learner relationship tended to make more excuses for their action in terms of having no alternative course of action, or that their action was legitimate in their . ascribed role of instructor than the 'most effective' managers of intent. 6. Instructors who were perceived as the least helpful or least well-intentioned' when establishing the instructor/learner relationship tended more frequently to prepare their learners that an upcoming explanation was not meant to be seen as malicious or arbitary than those instructors 'most effective' in managing their intent. 7. Similarities existed between the 'most effective' and 'least effective' instructors in that: l) the sequences of the most frequently paired utterances examined were not random and the coded utterances in a sequence could 93 be maximally predicted by knowing the immediately preceding coded utterance; 2) the proportional use of categories and sequences of utterances remained stationary or constant throughout the recorded session; and 3) the subgroups compared were made up of two to five instructors who shared a homegeneity in their use of a category, paired utterance or sequence of three utterances. COMPARISON OF THE VERBAL COMMUNICATION BEHAVIORS USED BY THE 'MOST EFFECTIVE' AND 'LEAST EFFECTIVE' GROUPS Three main research questions were formulated to guide the systematic analyses and comparison of the verbal communication behaviors used by the five instructors 'most effective' and five instructors 'least effective' in their management of intent. The findings are reported in response to each of these questions. Findings in Response to Research Question 1.0 Research Question 1.0 asked: Are there differences or similarities among the verbal communication behaviors used by the five instructors who were identified by learners as 'most effective' managers of their intent? Differences or Similarities in the Proportional Use of Categories by the 'Most Effective' Group Because of the unequal number of utterances used by the five instructors in the first hour of their lessons, the frequency data for each instructor were converted to proportions as shown in Table l. 94 TABLE 1 A COMPARISON OF THE PROPORTIONAL USE OF EACH CATEGORY BY THE 'MOST EFFECTIVE' INSTRUCTORS CM mm“ - , .» “True?" 5 11 RANK ORDERING or '12 scams 1 5127111111190 12.75 22.60 6.50 16.27 15.62 13.90 Stating 41% 2 REQUESTING - .12 .10 - .17 .11 Regarding 14% 3 PREPARING 7.00 6.61 9.02 10.07 10.10 9.00 Scanning 14% 7. 5117.11ch .16 2.20 1.32 .96 .26 1.00 Clarifying 13% 5 120711101110 16.02 11.06 13.07 11.15 19.66 14.43 Preparing 9% :76 succssnnc 2.33 5.09 7.01 7.31 9.10 6.94. Suggesting 7g 7 sumac 40.59 17.90 48.68 63.00 13.00 1.0.92 Sharing 1: a CLARIFYING 21.15 13.11. 11.67 11.75 10.39 12.90 Apology .39; 9 sxcusmc-ummsu-nomu. - - .u. .24 .09 .17 Excusing 3 10 EXCUSING-I.EGITIMATE - - .09 .12 - .02 (Unintentional).2% 11 APOLOGY - .12 .10 .2: .77 .10 Penance .15 g 12 PENANCE - .12 .10 - .17 .11 Requesting .l% ifutal N of utterances 1-1211 643 032 114.0 034. 1165 4616 Excusing 5 11 1.0/11:11:11 29.57 12.70 25.31. 32.20 27.19 29.09 (Legitimate) .025 f 16:61 N 61 utterances 1-13: 912 1237 1527 1210 1600 6507 ' The percentages in each instructor category (1-12) are calculated as percentages of (hr total utterances in the 1-12 categories used by each instructor ' The percentages for each instructor in the learner category (13) are calculated as ' percentages of the total utterances in all categories (1-13) An examination of obvious differences in the proportional use of categories in Table 1 indicates Instructor 2 verbally Scanned (category 1) for misunderstandings three times as much as Instructor 3. Referral to their coded transcripts revealed Instructor 2 used the Scanning category 22% while teaching an intricate skill and using a very conversational but Socratic style of questioning. On the other hand, Instructor 3 used the Scanning category only 7%, adopting a traditional demonstration technique to teach long sequences of skills and only scanning for understanding at the end of each 95 sequence. The other major discrepancy within the 'most effective' group was the use by Instructor 1 of category 8, Clarifying. This category was used 21% of the time, which was 7-10% higher than other instructors. Inspection of the coded transcript revealed Instructor 1 was giving a great deal of individualized instruction of a detailed and creative nature. There was a tendency to explain abstract ideas by clarifying a concept using a variety of concrete comparisons or analogies. Except for these differences, the proportional use of all categories by the five instructors or their learners in the 'most effective' group did not vary more than 10%. However chi-square tests of significance for differences between proportions using a 2 x 5 contingency table showed significant differences in the proportional use of all categories (p =.05). In an effort to establish some homogeneity between the verbal communication used by the five 'most effective' instructors, the twelve categories as shown in Table l were collapsed into the six classifications of Assessing, Preparing, Equalising, Explaining, Excusing and Repairing as displayed in Table 2. Again chi-square tests found significant differences among the proportional use of each classification by the instructors or learners in the 'most effective' group (p=.05). 96 TABLE 2 A COMPARISON OF THE PROPORTIONAL USE OF EACH CLASSIFICATION BY THE 'MOST EFFECTIVE' INSTRUCTORS Instructor 1 Cum-L CLASSIFICATION l 2 3 la 5 3(- RANK WRING a: 'x. SCORES 1 ASSESSING 12.75 22.60 6.75 16.27 15.79 16.09 2 PREPARING 7.00 6.61 9.02 10.07 10.10 9.02 Explaining 61% 0w . . 1 tMUMJSlmj 16.1713.3616.3912.1119.9115.61 5908118109 155 -—— I . 1. 13111171101110 66.07 57.21 60.16 62.95 52.96 60.06 Assessmg 14" ...... . 'n V 5 uxcusmc - - .51 .16 .09 .22 Prepari 9 9" Repairing .45 6 REPAIRING — .26 .35 .26 .96 .61 . , Excus1ng .2n L N/uiterances ”-6)" 6413 832 11110 830 1165 (16110 7 I LEARNER 29.57 12.76 25.16 12.20 27.19 29.09 07611610666. 11-71: 912 1217 1527 1210 1600 6507 * The pertentages in each instructor classification (1-6) are calculated as percentages of the total utterances in the I-6 classifications used by each instructor : The percentages for each instructor in the learner classification (7) are calculated as percentages of the total utterances in all classifications (1-7) Unable to establish that the five 'most effective' instructors shared any commonality in their proportional use of categories, the 2 x 5 contingency tables used for computing chi-square tests of significance for differences between proportions were further partitioned into selected 2 x 2 auxiliary tables. This subdivided chi-square into components to reveal specific sources of variance among the 'most effective' group (p =.05). In Table 3 it can be seen that some subgroups of instructors within the 'effective group' show no statistically significant differences in their use of a particular category. On the left hand side of the page, the 2 x 5 contingency tables show the category under inspection, 97 TABLE 3 DIFFERENCES IN THE PROPORTIONAL USE OF CATEGORIES AND COMBINATION CATEGORIES BY THE 'MOST EFFECTIVE' GROUP CATEGORIES 1 6 2 (Scanning) Iaamczoa at 3.5. x: p lastauctoa Coapaaaaoa 1 z 5 4 5 :1-5 1:015 2 .394461 3.26 NS 2:! l l2.07059l 99.73 .001 9 82 188 77 119 184 650 1.445:205 I .136249 1.20 NS *9 643 832 1140 834 1165 4614 Total: 4 12.610301 104.19 .001 CATEGORY 3 (Preparing) ‘““““°“ 6: s s x8 -—-————-- . . . p Insteactaa c°"““°" ' 7 2 5 4 5 al-S 7:2 1 .0054550 .01 NS 5:415 2 .0030360 .02 NS * 45 55 112 84 120 416 192:5'405 1 1.0943520 13.42 .01 ** 643 832 1140 834 1165 4614 Total: 4 1.1028430 13.45 .01 CATEGORY 4 - 1114211116260. Iaattucxoa 715 1 .0070682 .72 NS 1 2 5 4 5 11-5 114 1 .0061232 .62 115 1051304 1 .0774415 7.85 .01 * 1 I9 15 8 3 46 7050304:2 1 .1680141 17.02 .001 ** 643 832 1140 834 1165 4614 Total: 4 .2586747 26.21 .001 CATEGORY 5 - Iaatauctoa —— (Regarding) CWPM" d: s.s. 1' p Iaatauctoa 214 I .357403 2.89 NS 7 2 3 4 5 17-5 7:5 1 .000355 .00 NS 244:145 1 .790241 6.40 .02 * 103 92 I49 93 229 666 244.145:5 I 4.250461 34.42 .001 ** 643 832 1140 834 1165 4614 Total: 4 5.398460 43.71 .001 CATEGORY 6 (Suggesting) Inauuczoa d: s.s. x: P Iaatauctoa CO'P““°" 7 2 5 4 5 (1.5 215:4 2 .1827250 2.83 NS 115 1 1.8966654 29.39 .001 * 15 49 89 61 106 320 z.5.4:1.s I .0175446 .27 NS ** 643 832 1140 834 1165 4614 Total: 4 2.0969350 32.49 .001 CATEGORY 7 (Stating) 1114;406:011. d: s s x: p lastaucroe ca'p¢4“°“ 1 z 1 4 5 1.1-5 1:: I .24718 1.02 NS 3:415 2 13.83627 57.23 .001 * 261 316 555 366 390 I888 102:50405 I .70266 2.91 NS ** 643 832 1140 834 1165 4614 Total: 4 14.78611 61.16 .001 CATE.GORY,_8 (Clarifying) 1 u ‘ ' as at ca x, 1 2 Iaataucfoe 5 ‘1‘; Cospaaxaon df s.s. p 2:51415 3 .424873 3.76 NS * 136 111 133 98 121 599 2.5.4.5:1 1 4.985282 44.13 .001 *‘ 643 832 1140 834 1165 4614 Total: 4 5.410155 47.89 .001 CATEGORY 13 (Learners) 18‘0“!“ at s 5 x1 -—————————- - . . p Iaatauctoa C0.”““°“ 1 2 3 4 5 aI-S 71515 2 1.02848 4.97 NS 214 1 .01017 .09 115 * 270 405 387 396 435 I893 204:10505 I 4.53084 21.98 .OOI *** 913 1237 1527 1230 I600 6507 Total: 4 5.57769 27.04 .001 communion CATEGORY 6’ -,. 11102711162011 (Shari 9) Cos-penises <1: s.s. x: p lastauctoa 1:5 I .000312 .01 NS 1 2 5 4 5 aI-S 2:5 1 .030390 .04 (S ~ 105:205 1 1.029775 12.52 .001 * 75 73 132 42 93 415 145020514 I 1.595108 19.49 .001 ** 643 832 1140 834 1165 4614 Total: 4 2.649874 32.37 .001 constructor CATEGORY 55 (Regarding) 11162466264 0: s s x, p Instauctoa Coapaaisoa 1 z 1 4 5 :1~5 :14 1 .0109744 :15 11315 2 3304490 5 20 NS ' 55 34 105 40 80 314 2.4 :14505 I 1.456712 2.68 001 3‘ 643 832 1140 834 1165 4614 Total: 4 1.8081360 28.51 .001 ' Total utterances for the category examined *' Total utterances for all categories ’1-12) *** Total utterances for all categories (l-l3) 5 All utterances in combination with either a category 4 or a categorv 5 98 the designated numbers for each of the five 'most effective' instructors and the proportional use of that category. On the right hand side of the page, 2 x 2 auxiliary tables subdivide X2 into components to reveal specific sources of variance between selected subgroups of instructors. For example in category 8, Clarifying, Instructors 2, 3, 4 and 5 do not differ significantly in the use of that category. The source of the variance among the five instructors lies with Instructor 1 whose proportional use of category 8 reveals a chi-square value of 44.13 with p =.001. To assist easy identification of those subgroups of instructors whose use of categories were similar, the relevant findings of Table 3 are summarised in Table 4. This table highlights subgroups of instructors who showed no statistically significant difference in their use of the category under examination. For example, in the use of Preparing (category 3), the subgroup made up of Instructors 3, 4 and 5 showed no statistically significant difference to one another. This was also the case with Instructors 1 and 2. However, these subgroups are recorded separately to indicate there was a statistically significant difference between them in their use of that category. Having identified homogeneous subgroups of instructors in the 'most effective' group, the next step was to establish if their use of categories remained constant throughout the lesson. To do this, all coded utterances for each of these instructors were divided into two according to the two 99 TABLE 4 SUMMARY OF FINDINGS FROM TABLE 3 TO IDENTIFY THOSE 'MOST EFFECTIVE' INSTRUCTORS WHOSE USE OF CATEGORIES WERE SIMILAR HOMOGENEOUS GROUPS or INSTRUCTORS WHOSE USE or A CATEGORY HAS N01 SIGNIFICANTLY CATEGORY DIFFERENT FROM EACH OTHER Ih4,5 Samuumg (CatJ.& 2) 3,4,5 Preparing (Cat.3) 1,2 Preparing (Cat.3) 1,5 Sharing (Cat.4) 3,4 Sharing (Cat.4) 1,3 Sharing (Combinations with Cat.4) 2,5 Sharing (Ccmbinations with Cat.4) 1,3 Reyuflim; (CaUxxmyEH 2,4 Ikganfing (caUxmmyffl 2,4 Regarding (Canbinations with Cat.5) 1,3,5 Regarding (Cbmbinations with Cat.5) 2,3,4 Suggesting (Cat.6) 1,2 Stating (Cat.7) 2,3,4,5 Clarifying (Cat.8) 2,4 Learners (Cat.l3) 1,3,5 Learners (Cat.l3) halves of each lesson and compared. As shown in Table 5, chi-square tests of association were employed to detect statistically significant differences between the categories used in these equal segments of the lesson. Examination of Table 5 reveals that there were no significant differences (p=.05) during the two halves of the lesson by subgroups of 'most effective' instructors in their use of Scanning (categories 1 and 2), Preparing (category 3), Sharing (category 4), Regarding (category 5) and Stating (category 7). There were some marked differences in the use of the categories of Clarifying (category 8) and the Sharing combination category (category 4). 100 TABLE 5 DIFFERENCES BETWEEN THE FREQUENCY OF CATEGORIES USED IN THE FIRST AND SECOND HALVES OF THE LESSONS PRESENTED BY THE 'MOST EFFECTIVE' INSTRUCTORS CATEGORIES 1+2 £42§§Q§Z_é CATEGORY 13 (Scanning) (Suggesting) (Learners) '4 X z p H X 2 p S X 2 p 9 1 .00 NS 0 1 u 1 L95 NS E 2 g 2 -08 NS 3 2 .99 Ms E 3 u 3 -04 NS 3 3 15.32 .001 g 4 .50 NS :3 “ 3"33 -°0’ g 4 .26 Rs 5 5 .78 NS 5 5 H 5 .33 NS CATEGORY 3 Eél§§9§z_l. COMBINATION CATEGORY 4 (Preparing) (Stating) (Sharing) 3 1 .56 NS 3 1 1.86 NS ‘3 1 1.07 NS 3 2 1.78 NS 3 2 1.53 Rs 0 2 1.13 NS 4 3 .04 NS 3 3 B 3 11.83 .001 3 4 .19 NS m 4 g 4 5 5 .13 NS 5 5 a 5 5.39 .05 ...4 CATEGORY 4 CATEGORY 8 COMBINATION CATEGORY 5 (Sharing) (Clarifying) (Regarding) X ' p 16 X 2 P H X 2 p S 1 - - u 1 S 1 .17 NS 3 2 g 2 4.40 .02 g 2 .31 NS 3 3 - - 3 3 .L98.fl$ O 3 L32.01 O 4 .60 NS 3 4 g 2.00 NS 2 c1 .00 NS 2 5 .50 NS .4 5 6.02 .02 ,5 5 .04 NS H CAT CATEGORY 5 EGORIES 9: 10. 11 & 12 Low frequency prevented meaningful comparison (Regarding) 1. x: p Note: i)- =Low frequency prevented meaningful 3 1 .02 NS comparison with X' g 2 .04 NS ii) X’ figures only calculated for 'most ‘2 2 ix :1: effective' instructors whose use of 5 5 ' paired utterances were 99; signficantly different On the other hand, no more than one instructor varied over time in the use of the categories of Suggesting (category 6), the Regarding combination category (category 5), and Learners (category 13). Instructor 3 showed the 101 greatest inconsistency in the use of categories over time and accounted for variance in categories 8, 13, and combination categories 4 and 5. Referral to the transcript indicated a marked change by Instructor 3 from a group demonstration in the first half of the lesson to individualized instruction in the second half, where instructor/learner interaction increased along with a subsequent increase in learner responses (category 13), instructor clarification (category 8), sharing (category 4) and regarding (category 5) verbal communication activities. Differences or Similarities in the Use of Selected Paired Utterances by the 'Most Effective' Group Table 6 lists the proportional use by instructors of the ten most frequently occurring paired utterances. It can be seen from the table that the five most frequently occurring paired utterances used by the 'most effective' instructors all involved verbal interaction between learners (category 13) and their instructor. While there appears no striking discrepancies in the proportional use of the paired utterances among the five 'most effective' instructors in Table 6, these differences are statistically significant at the .05 level. However a closer inspection of the 2 x 2 auxiliary tables used to subdivide chi-square into components in Table 7 reveals that only one or two instructors are the source of variance. 102 TABLE 6 A COMPARISON OF THE TEN MOST FREQUENTLY OCCURRING PAIRED UTTERANCES USED BY THE 'MOST EFFECTIVE' GROUP . . t r - ”figuflgguugflw 1'19"“: ° __ RANK ORDERING or x SCORES PAIRED OrrERAuces 1 2 3 4 5 x 13 --7 8.33 9.79 6.42 12.53 7.50 8.75 13 - 7 (Learner followed by Stating) 99'. 7-13 5.48 10.03 6.82 12.21 5.88 8.03 7 -1} (Stating " Learner) 8% . 11-«5 8.66 4.61 6.68 5.29 8.94 6.86 13 - 5 (Learner “ Regarding) 7% 1 «13 5.04 8.98 2.62 6.02 6.75 5.83 1 '13 (Scanning " L-earner) 55 5-13 6.03 1.941L19 2.854.443.83 5 '13 (Regarding " Learner) 4% 7 7 3 18 2 99 7 86 4 31 3 13 4 44 7 - 7 (Stating " Stating) 4% ' ' ' ' ' ' 7 - 8 (Stating " Clarifying) 4% 7 ~11 5.92 2.83 4.00 4.48 3.69 4.06 3 _ 7 (Preparing ., Stating) 3% {<8 8—7 2.08 1.62 2.75 1.63 1.06 1.81 8 - 7 (Clarifying u Stating) 2% Lil” 3.51 1.70 3.74 4.64 2.44 3.17 8 _ 8 (Clarifying n Clarifying) 1% a —-8 1.64 .32 .39 .57 .56 .63 5252: i) 9 Criteria for selection was the five most frequently occurring paired utterances used by the 'most effective' instructors ii) ** Criteria for selection was the five most frequently occurring paired utterances used by the 'least effective' instructors iii) Occurrence of paired utterances expressed as a percentage of the total utterances uses by each instructor iv) A paired utterance such as 13 - 7 means that a learner utterance was followed by an instructor stating his/her intent Table 8 is extrapolated from Table 7 and summarizes those 'most effective' instructors whose use of paired utterances was similar. For example, in the use of paired utterance Learner followed by Stating (13-7), the subgroup made up of Instructors 1 and 2 showed no statistically significant difference to one another. This was also the case with Instructors 3 and 5. However, these subgroups are recorded separately to indicate that there was a statistically significant difference between them in their use of the paired utterance 13-7. 103 TABLE 7 DIFFERENCES IN THE USE OF THE SELECTED PAIRED UTTERANCES BY THE 'MOST EFFECTIVE' GROUP PRER 0 U1 ii '7 Ina£Auctaa H (Learner-u-Stating) to-puuon 9‘ 3's“ "' P Isatauctoa 5:5 1 .0915030 1.15 NS 1 2 5 4 tl-S 1:2 1 .1115995 1.39 NS 5.5.1.2 1 .6117280 7.67 .01 * 76 121 98 154 120 569 505910114 1 2.1624095 27.10 .001 ** 913 1237 1527 1230 1600 6507 Total: 4 2.9772400 37.31 .001 PAIRED 011 R 7- 1......92... u s s x, (Statingutearner) Coapauum ' ‘ P 7 2 5 4 5 tl-S 1:5 1 .0092334 .13 NS . (15:4 2 2.0312640 27.53 .001 * 50 124 104 150 94 522 [95:20504 1 2.1506286 29.14 .001 ** 913 1237 1527 1230 1600 6507 Total: 4 4.1911260 56.80 .001 PAIRED U AN 13-5 Iu‘uuqtgg d! s.s. X3 P (Learner--Regardin9) Coapaaiaon l 2 5 4 5 t1-5 1:5 1 .0047185 .07 NS 2:514 2 .3155060 4.94 NS * 79 57 102 65 143 446 795:20504 1 1.6013295 25.09 .001 ** 913 1237 1527 1230 1600 6507 Total: 4 1.9215540 30.10 .001 PAIR 0 U1" RAN -13 17182844414204. 0: 5 S X1 P (Scanning—-Learner) “'Pm‘" l 2 5 4 5 tl-S 114:5 2 .171340 3.12 NS 512 1 1.672910 30.50 .001 * 46 111 40 74 108 379 1040515'2 1 1.148756 20.94 .001 ** 913 1237 1527 1230 1600 6507 Total: 4 2.993006 54.56 .001 PAIREE UT TERANEE 5’13 1018134492011 (If S . s . X 2 P (Regarding--Learner) c""“‘"°“ l 2 5 4 5 37-5 514:5 2 .1960432 5.33 NS . 1:2 1 .8761064 23.81 .001 * 55 24 64 35 71 249 102:50495 1 .0071094 .19 NS ** 913 1237 1527 1230 1600 6507 Total: 4 1.0792590 29.33 .001 P_—_RAIRED 011E ANCE 7-7 1945499596 a: 5.5. x, p (Stating--Stating) “WWW 1 2 5 J 5 xl-S 1:2:5 2 .049075 1.16 NS . 5:4 1 .858358 20.22 .001 * 29 37 120 53 50 289 1495:3114 1 1.320086 31.10 .001 ** 913 1237 1527 1230 1600 6507 Total: 4 2.227519 52.48 .001 PAIRED UTIERANCE 7-8 . . . In4tauctoa (Stating-;Clar§fying‘) 5 “-5 “van" 6: s.s. x: p 213:4:5 3 .1737284 4.46 NS * 54 35 61 55 59 264 2.5.405:l 1 .3712866 9.54 .01 ** 913 1237 1527 1230 1600 6507 Total: 4 .5450150 14400 .01 (Clarifyingo-Stating) c°'P“"“°" 1 2 5 1 5 :1-5 1:2 1 .0113165 .64 NS 5:4:5 2 1.0835315 60.85 .001 5 19 20 42 20 17 118 162150405 1 .4266967 23.96 0011 ** 913 1237 1527 1230 1600 6507 Total: 4 1.5215447 85.45 .0011 PAiR D UTTERA 7'7 In‘xgunga df s s X3 P i (Preparing-Stating) Co-pmun 5 4 5 :7-5 2:5 1 .0381856 1.25 NS 1 125:4 2 .0825582 2.69 NS 1 * 32 21 57 57 39, 206 205:10594 1 .5555287 18.12 .001 ** 913 1237 1527 1230 1600 6507 Total: 4 .6762725 22.06 .001 1—-__ PAIR D UTT RA -8 - ° _ ' - Isatsuttoa (Clarifyirgg Clasrifyi‘ng) “-5 Cup . on df s.s. x2 p 2:3:4:5 3 .0061288 .98 NS * 15 4 6 7 9 41 2059405:l 1 .1089482’ 17.40 .001 ** 913 1237 1527 1230 1600 6507 Total: 4 .1150770 18.38 .001 * Total incidences for the paired categories examined ** Total utterances for all categories (1-13) 104 TABLE 8 SUMMARY OF FINDINGS FROM TABLE 7 TO IDENTIFY THOSE 'MOST EFFECTIVE' INSTRUCTORS WHOSE USE OF PAIRED UTTERANCES WERE SIMILAR‘TO EACH OTHER HOMOGENEOUS GROUPS OF INSTRUCTORS WHOSE USE OF A PAIRED UTTERANCE HAS NOT SIGNIFICANTLY DIFFERENT FROM EACH OTHER PAIRED CATEGORY 1,2 (Learner followed by Stating) (13- 7) 3,5 1,5 (Stating " Learner) (7-13) 1,5 (Learner " Regarding)(13- 5) 2,314 1,4,5 (Scanning " Learner) (1-13) 3,4,5 (Regarding " Learner) (5-13) 1,2,5 (Stating " Stating) (7- 7) 2,3,4,S (Stating, " Clarifying)(7- 8) 1,2 (Clarifying " Stating) (8- 7) 2,5 (Preparing " Stating) (3- 7) 1,3,4 2,3,4,5 (Clarifying " Clarifying)(8- 8) As shown in Table 9, it was further established that the use of six out of the ten paired utterances examined remained constant throughout the lesson. In the use of Learner followed by Stating (13-7) and Preparing followed by Stating (3-7) only one instructor showed a significant difference in the first and second halves of the lesson. However, in the case of Stating followed by Stating (7-7) and Regarding followed by Learner (5-13), most instructors in the 'most effective' group were not constant in their use of these paired utterances. 1(15 TABLE 9 DIFFERENCES BETWEEN THE PAIRED CATEGORIES USED IN THE FIRST AND SECOND HALVES OF THE LESSON BY THE 'MOST EFFECTIVE' INSTRUCTORS PAIRED UITERANCE 13-7 (Learner--Stating) Instructor PAIRED UTTERANCE 7-13 (Stating--Learner) Instructor UbUNI—l PAIRED UTTERANCE 13-5 (Learner--Regarding) Instructor Note: UbWNO—l x1 p H .00 us 31 3.97 .05 o 2 L31 NS 33 U 4 a .27 NS :5 H X1 p u .72 NS 31 0 2 83 :4 .17 NS 55 x2 .00 1.95 .93 X! .4.39 .24 4.07 PAIRED UTTERANCE 1-13 (Scanning-~Learner) p NS NS NS PAIRED UTTERANCE 5-13 (Regarding--Learner) p .05 NS .05 PAIRED UTTERANCE 7-7 (Stating--Stating) X' P Significant difference 1 1.03 NS at .05 level and so no 2 :07 "3 meaningful comparison 3 3.57 NS 4 .38 NS could be made. 5 71.50 NS i)- ==Low frequency prevented meaningful ii) comparison with X' X' figures only calculated for 'most instructors whose use of effective' paired utterances were not sigificantly different PAIRED UITERANCE 7—8 (Stating-~Clarifying) Instructor X‘ p l 2 .17 NS 3 1.33 NS 4 .16 NS 5 2.86 NS PAIRED UTIERANCE 3-7 (Clarifying--Stating) Instructor X2 p 1 .05 NS 2 3.20 NS 3 4 5 PAIRED UTTERANCE 3-7 (Preparing--Stating) Instructor X‘ p 1 2.00 NS 2 1.19 NS 3 4.57 .05 4 .36 NS 5 .64 NS PAIRED UTTERANCE 8-8 (Clarifying-Clarifying) Instructor x1 p l 2 .07 NS 3 .00 NS 4 00 NS 5 00 NS Differences or Similarities Between the Probability of One Category Following Another in Selected Paired Utterances Used by the 'Most Effective' Group Table 10 examines the differences in probability of one category following another in selected paired utterances. In order to find out the extent that a person 106 TABLE 10 DIFFERENCES IN PROBABILITY OF ONE CATEGORY FOLLOWING ANOTHER IN SELECTED PAIRED UTTERANCES FOR 'MOST EFFECTIVE' GROUP ‘ EEIEQ UTTERANCFl5—7 -_ In4£4uczoe , (Learnerz Statsing) ‘ “-5 Cowman d! 5.5. x p 1.2.3.514 1 3.89151 18.50 .001 * 76 121 98 154 120 569 1:2:3:5 3 1.09412 5.21 NS ** 269 405 387 396 435 1892 Tocal: 4 4.98563 23.71 .001 PA D 011 R 7-1 11130.net“ d, s s x3 p (Stating--Learner) “"‘P‘u‘m‘ 1 2 5 4 5 11-5 2:4 1 .032001 1.41 NS 113:5 2 1.300230 5.73 115 * 50 124 104 150 94 522 204:10305 1 11.390869 48.88 .001 ** 184 271 401 323 318 1497 Total: 4 12.723100 56.02 .001 ’5 luuuctaa - df 5.5. x: (LearneruRegarding) c“"“‘“" p 1 2 5 4 5 a1-5 2:4 1 .109639 .61 NS 1:315 2 .873686 4.85 NS * 79 57 102 65 143 446 204:10505 ..1 9.666455 53.65 .001 ** 269 405 387 396 435 1892 Total: 4 10.649780 59.11 .001 ‘ PAIRED—“Run: Aug 1-13 ”mm... f . Conpcaiaon d 5'5‘ x P (Scanning--Learner) 1 2 3 4 5 11-5 * 46 111 40 74 108 379 ** 71 173 63 108 154 569 Total: 4 .422531 1.90 NS ‘ PAIRED UTT RANCE - 3 (Re srdin --Learner) "“"““5‘°“ df s.s. x3 p (I 29 3 4 5 c1-5 “'PW‘" ' 2:3:4:5 3 1.443594 5.93 NS * 55 24 64 35 '71 249 2o3.4o5:1 1 3.905816 16.04 .001 ** 90 69 139 85 211 594 Total: 4 5.34941 21.97 .001 PA ANCET-‘I - _- - InAtAucton 2 (Stating: Stat’ing) ‘ s r “5 “"44““ d: s.s. x p 1:2:4:5 3 .121235 .78 NS * 29 37 120 53 50 289 1920405:5 1 6.177272 39.65 .001 ** 184 271 401 323 318 1497 Total: 4 6.298507 40.43 .001 ‘ PAIjRjommamc - ' (Statin «Clarif in ) ”‘n‘“.‘w‘ df s.s. x1 1 9 2 3 Y 9‘ 5 t1-5 Conpwon p 2:5:415 3 .171600 3.05 as * 54 35 61 55 59 264 2050405:1 1 18.197873 106.00 .001 ** 184 271 401 323 318 1497 Tocal: 4 18.369473 107.05 .001 PAIRED UTTERA!E§ 8'7 In‘mctgn d: S s X: p (Clarifying--Stating) Conpmson ' ' 1 2 5 4 5 t1-5 1:2:5 2 .1016233 .59 NS 10295:304 1 2.0531910 11.95 .001 1' 19 20 42 20 17 118 3:4 1 1.3904790 8.08 .01 ** 114 102 112 95 111 534 Total: 4 3.5499033 20.62 .001 PAIR U NE 7'7 lat/mecca df 8 S x, ~ - Calpaaison ' ' p (Prepar1ng--Stat1ng) 1 2 5 4 5 t1-5 1:514 2 1.378730 5.64 NS 2:5 1 .201869 .83 NS * 32 21 57 57 39 206 505048205 1 6.526511 26.68 .001 ** 40 45 95 78 101 359 Total: 4 8.107110 33.15 .001 P8 II '8 11140.net“ df S S x, p (Clarifying-Clarifying) Conpawon 1 3 S a: 1-5 * 15 4 6 7 9 41 ** 114 102 112 95 111 534 Total: 4 .2281281 3.22 NS * Total incidences for the paired categories examined 2* Total utterances for the preceding category in the pair 107 could confidently predict the probability of one category following another, the ten most frequently paired utterances were analysed. For example, the paired category 1-13 in Table 10 shows that on the right hand side of the 2 x 5 contingency table there were no significant differences (p=.05) among the five 'most effective' instructors in the probability of a learner responding when his instructor scanned for understanding. In Table 11, a summary of the findings from Table 10 are extrapolated to identify the mean probability of one category following another in frequently used paired utterances by the 'most effective' instructors. For example, it is shown that on average, the probability with which one could confidently predict if learners would respond to their instructor's scanning (1-13) is 68% of the time. TABLE 11 SUMMARY OF FINDINGS FROM TABLE 10 TO IDENTIFY THE R PROBABILITY OF ONE CATEGORY FOLLOWING ANOTHER IN FREQUENTLY USED PAIRED UTTERANCES BY THE 'MOST EFFECTIVE' INSTRUCTORS PAIRED UTTERANCES EXAMINED R Probability (1 - 13) Scanning followed by Learners 68% (3 - 7) Preparing " Stating 57% (5 - 13) Regarding " Learners 38% (13 - 7) Learner " Stating 28% (7 - 13) Stating " Learners 25% (13 - 5) Learner " Regarding 24% (8 - 7) Clarifying " Stating 18% (7 - 8) Stating " Clarifying 16% (7 - 7) Stating " Stating 15% (8 - 8) Clarifying " Clarifying 8% 1(18 From Table 12 it can been seen that in most cases the probability of one category following another in the ten most frequently used paired utterances did not differ TABLE 12 DIFFERENCES IN THE PROBABILITY OF ONE CATEGORY FOLLOWING ANOTHER IN SELECTED PAIRED UTTERANCES FOR THE FIRST AND SECOND HALVES OF THE LESSON FOR EACH OF THE INSTRUCTORS IN THE 'MOST EFFECTIVE' GROUP PAI D UTTER (Learner-~Stating) Instructor UIUNH PAIRED UTTERANQ: 7-13 (Stating--Learner) Instructor PAIRED UTTERANCE 13-5 (Learner--Regarding) Instructor Notg: UibUNt—l UL‘WNH X! .72 .29 .03 1.51 X! 2.96 .00 17.05 .08 .14 x2 .13 .21 .01 .23 .02 p NS NS NS NS p NS NS .001 NS NS p NS NS NS NS NS Instructor Instructor Instructor UbuNH MbUND—fi PAI D UTTER (Scanning--Learner) x2 .23 .84 .13 .65 .00 x2 x2 4 6. .52 .52 30 -13 p NS NS NS NS NS PAIRED UTTERANQ; 5—13 (Regarding--Learner) p NS NS NS '0’ PAIRED UTTERANCE,7-7 (Statingo-Stating) p NS .05 .02 11"==Low frequency prevented meaningful ii) comparison with x' X' figures only calculated for 'most effective' instructors whose use of paired utterances were not signficantly different PAIRED UTTERANQ; 7-8 (Stating--Clarifying) Instructor 1 2 3 a 5 x! 2.04 '.00 .64 6.38 p NS NS NS .02 PAI§§0 UTT§RANC§ 8-7 (C1arifying--Stating) Instructor U§UNH X! .31 9.64 .16 p NS .01 NS PAIRED UTTERANCE 3-7 (Preparing--Stating) Instructor bcfiwND-J x2 .00 .06 2.50 2.28 .61 p NS NS NS NS NS PAIRED UTTERANCE 8-8 (Clarifying-Clarifying) Instructor U'IS‘UNH x! p 109 significantly throughout the lesson. The notable exception was in the probability of Stating being followed by Stating (7-7) which showed variances for two out of the three instructors whose use of that paired utterance was similar. Sequences of Three Utterances Used by the 'Most Effective' Group For the instructors in the 'most effective' group, the five most frequently used sequences of three utterances were rank ordered and compared. The ranking of incidence is shown in Table 13. From examination of Table 13, the most frequently used sequences of three utterances are summarized TABLE 13 RANKING OF INCIDENCE OF SEQUENCES OF THREE UTTERANCES FREQUENTLY USED BY THE 'MOST EFFECTIVE' INSTRUCTORS RANK ORDER 1 2 3 4 5 139543 13.57.13 8.1397 5413-»5 7»8+13" 13-97013 13--1-13 7o13-e7 1-13-v7 7—»l~13 1 2 3 1 1395913 7-v7-97 13-e7-d3 7913-57 7.1395 4 7+13-7 13-97913 3-’7~13 13-97d~8 13d>1§l3 '5 1395,13 13—>7~13 1-13-v5 5+13-+5 7-13-v7 7-1~l3 'MOST EFFECTIVE' INSTRUCTORS Criteria for selection: Sequences of three utterances were identified if the initial paired sequence appeared 5% or more of the frequency total of each instructor. These sequences were ranked according to their frequency for each instructor. 110 in Table 14 to highlight the interactive pattern of verbal communication between learners and their instructor. From Table 14, it can be seen that within the 'most effective' group of instructors, a predominant pattern of interaction between instructors and their learners emerged in the use of sequences of three utterances. Apart from Stating (category 7), these interactive sequences used by the 'most effective' instructors are typified by Scanning for misunderstandings (category 1) or the giving of positive praise or acknowledgement to learners (category 5). TABLE 14 SUMMARY FROM TABLE 13 TO IDENTIFY PATTERNS IN THE SEQUENCES OF THREE UTTERANCES FREQUENTLY USED BY THE 'MOST EFFECTIVE' INSTRUCTORS 13- 7-13 Learner -- STATING -- Learner 13- 1-13 Learner -- SCANNING - Learner 13- 5-13 Learner -- REGARDING - Learner 5-13- 5 REGARDING -- Learner - REGARDING 7-13- 5 STATING -- Learner - REGARDING 1-13- 5 SCANNING -- Learner -- REGARDING 7-13- 7 STATING -- Learner -- STATING 7- 8-13 STATING - CLARIFYING -- Learner 13- 7- 8 Learner -- STATING - CLARIFYING 7- l-13 STATING - SCANNING - Learner Note: i) A sequence of three utterances such as 13 - 7 - 15 is read as: 'a learner utterance, followed by the instructor stating, followed by another learner utterance' ii) Sequences of three utterances have been arranged to highlight the interactive nature of verbal communication between the instructors and their learners. Findings in Response to Research:guestion 1.1 Research Question 1.1 asked: Are there differences or similarities among the verbal communication behaviors 111 used by the five instructors identified by learners as 'least effective' managers of their intent? Differences or Similarities in the Proportional Use of Categories by the 'Least Effective' Group Table 15 shows the differences and similarities in the proportional use of categories by the five instructors in the 'least effective' group. An inspection of the table shows an obvious difference in the proportional use of categories by Instructor 7. This instructor verbally scanned for misunderstandings (Category 1) twelve times as much as Instructor 9, and two to three times as much as the other instructors. Referral to the coded transcripts TABLE 15 A COMPARISON OF THE PROPORTIONAL USE OF EACH CATEGORY BY THE 'LEAST EFFECTIVE' INSTRUCTORS Instructor - 1 CODE CATEGORY 6 7 8 9 10 ...-i“ RANK ORMRING “- x SCORES 1 scauuxuc 3.60 12.16 5.28 1.10 4.86 5.87 2 REQUESTING .17 - .12 - .17 .09 . 3 manumc ' 17.67 9.52 13.71. 9.56 6.87 11.51 Stating. 48E 4 SHARING ' - .25 1.35 1.10 .50 .66 Clarifying 22: “s 1151;190:111: 9.78 3.62 1.29 :76 11.79 6.11 2:32:32: 1:; 6 SUGGESTING 6. 52 1.88 7 . 24 4 . 4| 3. 52 4 . 7D Scanning 6% ...7-...S"“Nc 38.08 55.1: 46.50 52.91. 45.56 47.98 Suggesting 5% 8 crwa1rv1nc 19.90 17.42 19.26 27.76 25.96 21.52 . a 9 EXClJSlNC-UNIN‘I'ENTIONAL 1. 54 - .61 - .34 .48—J :::;::9 1’. 10 EXCUSINii-IECITIHATE 1.89 - .61 - .84 ...-63“ (Legitimate) 1% ll AHNDGY .86 - .98 .17 - 4S Excusing ‘2 ”m‘mm ‘ ' ' ‘ ' ’ . (Unintentional) .5: Total N of utterances 1-12‘ 583 798 815 544 597 3317 Apology 05% 13 LEARNER 13.50 14.29 7.18 2.86 11.71 10.85 Requesting .l% Total N of utterances 1-13' 674 931 878 560 700 374.3 Penance 0% * The percentages in each instructor category (1-12) are calculated as percents“?I 0‘ 1“? total utterances in the 1-12 categories used by each instructor The percentages for each instructor in the learner category (111 are calculated as percentages of the total utterances in all categories (1-131 112 revealed Instructor 7 would: 1) make a statement, pause, and then finish the statement with a rhetorical form of question §;S: "Obvious, isn't it?", "It looks awful, doesn't it?" or 2) leave no time for a response e43, "Did you see me? Watch me again.". While three instructors used the Apology category (category 11), none of the 'least effective' group used the Penance category (category 12). The twelve categories were collapsed into six larger classifications as shown in Table 16. However Chi-square tests of significance for differences between proportions using 2 x 5 contingency tables found differences in the proportional use of all classifications at the .05 level. TABLE 1 6 A COMPARISON OF THE PROPORTIONAL USE OF EACH CLASSIFICATION BY THE 'LEAST EFFECTIVE' INSTRUCTORS Instructor C1101: 0.115511710111021 6 7 8 9 10 i 1 ASSESSING 3.77 12.16 5.60 1.10 5.03 5.96 .. z PREPARING 17.67 9.52 13.76 9.56 6.87 11.51 RANK ORDERING 0‘ x SCORES 1 EQUALISING 9.78 3.88 5.66 3.86 11.89 6.77 Explaining 7a: 6 EXPLAININC 66.69 76.66 73.00 85.11 75.01 76.20 Preparing 12% 5 sxcusmc 3.63 - 1.23 - .01 1.11 Equalising 7g 6 REPAIRING .86 - .98 .37 - .65 Assessing % . N/utterances (l—6)* 583 798 815 566 597 3337 Repairing A55 1 7 ILEARNER 13.5016.29 7.18 2.8616.71 10.85 Nlutterances (l-7)£ 674 931 878 560 700 3743 * The percentages in each instructor classification (l~6) are calculated as percentages of the total utterances in the 1-6 classifications used by each instructor I The percentages for each instructor in the learner classification (7) are calculated as percentages of the total utterances in all classifications (1-7) As displayed in Table 17, a further partitioning of the 2 x 5 tables into 2 x 2 auxiliary tables subdivided 113 TABLE 1 7 DIFFERENCES IN THE PROPORTIONAL USE OF CATEGORIES AND COMBINATION CATEGORIES BY THE 'LEAST EFFECTIVE' GROUP CATEGORIES la 2 (Scanning) luauuctu of S S x3 . . . p lutauctaa ca"“‘“" 6 7 I t6-70 6:81”! 2 .0936375 1.68 NS 7:9 1 3.9515383 70.80 .001 * 22 97 44 6 30 199 6'8'7017'9 1 1.1038632 19.77 .001 ** 583 798 815 544 597 3337 Total: 4 5.149039 92.25 .001 CATEGORY 3 (Preparing) luauuctaa <1: 8 S x, — , . p luuuctoa Coepmaon 6 7 9 9 70 t6-70 719 1 .0000390 .00 NS 618170 2 1.7659343 17.34 .001 * 103 76 112 52 41 384 6080701709 1 2.6589337 26.11 001 *9 583 798 815 544 597 3337 Total: 4 4.4249070 43.45 .001 CATEGORY 4 (Shari ) Inauuctoa —_ "9 c... . .. a 3.5 x1 , Inauuctea 7: 70 1 .0020876 . 32 NS 6 7 8 9 70 116-70 8:9 1 .0019876 .30 NS 70701909 1 .0548332 8.37 .01 * 0 2 11 6 3 22 70909070“ 1 .0292369 6. 7 .05 ** 583 798 815 544 597 3337 Total: 4 .0881433 13.46 .02 CATEGORY 5 (Regarding) 1.1324891“ u s s x: p lustauctoa Cospc41aoa 6 7 9 9 70 t6-70 7:9:9 2 .0771831 1.34 NS 6:70 1 .0767610 1.33 NS * 57 29 35 15 68 204 70909:6070 1 2.6680359 46.49 .001 n 583 798 815 544 597 3337 Total: 4 2.8219800 49.16 .001 CATEGORY. 6 S gstin ) Instnuctaa ———-— ( 099 9 Conan“ df s.s. x: p lunactoa 6:8 1 .0176816 .39 NS 6 7 8 9 70 I 6-70 9: 70 1 .0227580 .51 NS 60919070 1 .5633753 12.57 .001 * 38 15 59 24 21 157 60909070:7 1 .8384789 18.70 .001 ** 583 798 815 544 597 3337 Total: 4 1.4422938 32.17 .001 CATEGORY 7 (Statin Intimates —— 9) Cal-pm." df s.s. x1 p muuccu mo 1 .030578 .12 11s 1 6 7 I 9 70 116-70 7:9 1 .156100 .63 NS ’ . 90701709 1 6.561910 18.28 001! * 222 440 379 288 272 1601 8'70'709:6 1 6.921242 27.73 .001} ** 583 798 815 544 597 3337 Total: 4 11.669830 46.76 001 CATEGORY 8 (Clarifying) 111.281.6264 ‘1‘ s.s. x, p Inaccuctoa Componeaon 6 7 8 9 70 lib-70 6:7:9 2 .2396648 1.42 735 9:70 1 .0916291 .54 NS * 116 139 l57 151 155 718 60707:9070 1 4.8744961 28.37 .001 ** 583 798 815 544 597 3337 Total: 4 5.2057900 30.83 001 custom 13 (Learners) runner“ df s s x, p Inauuctca Conpwaen 6 7 I 9 70 £640 617:” 2 .0520300 .54 NS 33' 1 .6133068 6.33 .02 * 91 133 63 16 103 406 6070701709 1 1.1384622 11.75 .001 *** 674 931 878 560 700 3743 Total: 4 1.8037990 18.65 001 communion CATEGORY 67(Sharing) Int/metal: at s s x, p lnauuctoa Ceepmaou ' ' 6 7 8 9 70 86-70 7:9:70 2 .3420398 2.68 NS 617 l 11.7572520 92.15 .001 * 89 46 276 49 41 501 6081709'70 1 25.2046322 197.53 .001 “ 583 798 815 544 597 3337 Total: 4 37.303924 292.36 .001 communion cartoon? 57(Rggarding) Issuactoa a s s x: p Instauctea Coapaaucu 6 7 8 9 70 16-70 7:70 1 .0005916 .02 NS 619:9 2 1.4879164 55.50 .001 * 42 19 ll 5 15 92 707026409 1 .0244964 .92 SS ** 583 798 815 544 597 3337 Total: 4 1.5130044 56.44 .001 * Total utterances for the category examined '9 Total utterances for all categories (l-lZ) “‘9 Total utterances for all categories (1-13) 5 All utterances in conbination with either a category 4 or a category 5 114 chi-square into components to reveal the specific source of variance among the 'least effective' instructors. An inspection of Table 18 shows a summary of the findings extrapolated from Table 17 and identifies those 'least effective' instructors whose use of categories was similar to each other. For example, in the use of Clarifying (category 8), the subgroup made up of Instructors 9 and 10 showed no statistically significant difference to one another. This was also the case with the subgroup of Instructors 6, 7 and 8. However, these two subgroups are recorded separately to indicate that there was a statistically significant difference between them in their use of category 8. TABLE 18 SUMMARY OF FINDINGS FROM TABLE 17 TO IDENTIFY THOSE 'LEAST EFFECTIVE' INSTRUCTORS WHOSE USE OF CATEGORIES WERE SIMILAR TO EACH OTHER HOMOGENEOUS GROUPS OF INSTRUCTORS WHOSE CATEGORY USE OF A CATEGORY HAS NOT SIGNIFICANTLY DIFFERENT FRW EACH OTKR 6,8A) Samufimgi (Cat.].& 2) 7,9 Pnflxuing (Cat.ZH 8,9 gmufing (Cat.4) 7,10 Sharing 7,9,10 Sharing (Canbinations with Cat.4) 6J0 Mnmflnw Kan M 7,8,9 Regarding 7,10 Regarding (Combinations with Cat.S) 6,8 Suggesting (Cat. 6) 9;“) Smxfifitnx; - 7,9 Stating (Cat. 7) 8,10 Stating 9,10 Clarifying (Cat. 8) 6,7,8 Clarifying 6,7,10 Learners (Cat. 13) 115 The findings of the chi-square tests of association (p=.05) used to establish if the use of categories remained constant throughout the lesson are displayed in Table 19. TABLE 1 9 DIFFERENCES BETWEEN THE PAIRED CATEGORIES USED IN THE FIRST AND SECOND HALVES OF THE LESSON OF INSTRUCTORS IN THE 'LEAST EFFECTIVE' GROUP CATEGORIES 1+2 m5: CATEGORY 13 (Scanning) (Suggesting) (Learners) H X: p 7-6 X! p 8" X' p 3 6 6.55 .02 3 6 .00 us § 6 .89 N3 2 7 g 7 5 7 7.22 .07 1'1 8 7.45 773 1. 8 .83 NS t: 8 t; 9 z; 9 .67 773 a 9 g 10 .52 NS 510 2.33 773 5 10 70.57.07 CATEGORY 3 CATEGORY 7 COMBINATION CATEGORY 7. (Preparing) (Stating) (Sharing) .. X! p .. X‘ P .. x2 p 8 6 3 6 o 6 ° 7 2.79 NS ” S 7 .05 NS :1 o 7 4.68 .05 1.7 8 L- 8 .59 NS 3 8 3‘: 9 .37 us :1 9 4.00 .05 1‘; 9 2.9., NS .510 .5 1° 4-76 -05 ,5 10 76.07 .007 CATEGORY A CATEGORY 8 COMBINATION CATEGORY 5 (Sharing) (Clarifying) (Regarding) x: p n X7 p H x: p u o o 3 5 - - g 6 L17 us .. 6 u 7 - - g 7 .78 NS 3 7 .75 us a s - - u 8 L64 N5 5 8 f; 9 - - g 9 .32 NS 07 9 5 10 - - H 10 .78 us .5 10 .73 NS CATEGORY S CATEGORIES 9L10, 11 & 12 (Regarding) Low frequencies prevented meaningful comparison 15 X‘ P Ngtg: i)--Low frequency prevented meaningful g 6 comparison with x' 3 7 7.69 NS .13.) x: f. 3 8 7.40 NS igures only calculated for 'least' 2 9 ”.27 .001 effective' instructors whose use of _, 10 paired utterances were 293 significantly different 116 The table reveals that there were no significant differences between the two halves of the lesson in the use of Preparing (category 3), Suggesting (category 6), Clarifying (category 8) and Regarding (combination category 5). Only one instructor varied over time in the use of Scanning (categories 1 and 2), and Regarding (category 5). Over half the instructors showed a variance over time in their use of Stating (category 7), Learners (category 13) and Sharing (combination category 4). However, due to the low frequencies of Category 9 (Excusing, Unintentional), Category 10 (Excusing, Legitimate), Category 11 (Apology) and Category 12 (Penance), meaningful comparison using chi- square tests for statistically significant differences was not possible. Difference or Similarities in the Use of Selected Paired Utterances by the 'Least Effective' Group The proportional use by the 'least effective' instructors of the ten most frequently occurring paired utterances are displayed in Table 20. It can be seen from this table that the five most frequently occurring paired utterances used by the ‘least effective' instructors involved no verbal interaction between learners (category 13) and their instructor. 117 TABLE 20 A COMPARISON OF THE PROPORTIONAL USE OF SELECTED PAIRED UTTERANCES BY THE 'LEAST EFFECTIVE' GROUP ””fiiniifiiz‘luufié'm I"°"“°‘°’ _ RANK ORDERING or '72 stones PAIRED UTTERANCES 6 7 8 9 10 X 13 -7 2.23 6.36 1.82 .56 6.01 3.26 7"‘3 1-63 ‘-‘9‘~‘9 -35 3‘72 2-‘3 '7 - 7 (Stating followed by Stating) 13% t 18 -5 3.86 2.80 2.96 1.25 7.01 3.58 7 . 8 (Stating " Clarifying) 9% 1 -13 1.63 6.36 1.25 .56 2.29 2.68 l 8 - 7 (Clarifying " Stating) 7% 5-13 1p.53 .651.37 1.07 6.57 1.95 1 3 - 7 (Preparing " Stating) 4% -7 7-7 4.46 17.8547.” 21.67 13.65 12.87 8 ' 8 (Clarifying " Clarifying) a: 7 —8 6.86 10.11 5.93 15.76 9.59 9.28 13 ' 5 (Learner " Regarding) (1% 66 8-7 3.57 7.10 5.1316.13 8.30 7.28 7 l '13 (Scanning " Learner) 3% 3-7 6.39 6.95 2.05 6.11 3.15 6.07 13 - 7 (Learner " Stating) 3% P—— 7 -13 (Stating " Learner) 2% 8-8 3.27 2.15 .91 6.80 6.72 3.60 1 S -13 (Regarding ,. Learner) 2% 595g; 1) * Criteria for selection was the five most frequently occurring paired utterances used by the 'most effective' instructors ii) '* Criteria for selection was the five most frequently occurring paired utterances used by the 'least effective' instructors iii) Occurrence of paired utterances expressed as a percentage of the total utterances uses by each instructor iv) A paired utterance such as 13 - 7 means that a Learner utterance was followed by an instructor stating his/her intent Reference to Table 21 reveals that there were statistically significant differences at the .05 level in the use of paired utterances by the 'least effective' group and the 2 x 2 auxiliary tables identify the particular instructor who was the source of variance in the use of each paired utterance. Table 22 is extrapolated from the findings of Table 21 to identify those 'least effective' instructors whose use of paired utterances were similar. For example, in the use of paired utterance Stating followed by Learner (7-13), the subgroup made up of Instructors 6 and 8 showed no 11.8 TABLE 21 DIFFERENCES IN THE USE OF THE SELECTED PAIRED UTTERANCES BY THE 'LEAST EFFECTIVE' GROUP 'WWIRANCE 13-7 Instauctaa . at 5.5. X' p (Learner-—Stating) “'pm‘“ 6 7 8 9 10 16-70 * 15 59 16 3 28 121 ** 674 931 878 560 700 3743 Total: 4 1.5888928 5.79 NS PAIR TT RANC - —‘-—‘—§J°.” 7 12:12:18: .. .1 : (Stat1ng--Learner) Iaatauctoa 7:70 1 .00900164 .38 NS 6 7 8 9 10 8 6-10 6:: 1 .00087270 .06 NS 668:7.10 1 .67303900 19.96 .001 * 11 39 13 2 26 91 6o7o8o10:9 1 .28328240 11.96 001 8* 676 931 878 560 700 3763 Total: 4 .76620056 32.30 .001 PA ANC - .1 ’ 12:12:11.2: .. s.s. .. . (Learnero-Regarding) 6 7 8 9 70 t6-70 68718 2 .0487344 1.41 NS 9:10 1 1.0286136 29.80 .001 * 26 26 26 7 49 134 6070889970 1 .1419282 4.11 .05 ** 674 931 878 560 700 3743 Total: 4 1.2192762 35.32 .001 O UTTER 1-13 . 12:12:13: .. .. ,, (Scann1ng--Learner) 6 7 8 9 10 t 6.10 6:: 1 .0056827 .21 NS 789870 2 1.3439618 51.69 .001 * 11 59 11 3 16 100 6088709070 1 .4170187 16.03 .001 ** 674 931 878 560 700 3743 Total: 4 1.7664632 67.93 .001 PAIR D A 513 ~ 5”“ 12:11:18: .. .. : (Regarding—~Learner) 6 7 8 9 70 1 6-70 889 1 .0029826 .16 NS 687:70 2 .6179440 32.31 .001 * 17 6 12 6 32 73 8098607070 1 .1139529 5.96 .02 9* 674 931 878 560 700 3743 Total: 4 .7348795 38.43 .001 PAIREO UTIERANC; 7-7 _ _ 62::822f2: 02 5.5. X7 p (Stat1ng--Stating) 6 8 9 10 r 6.10 7:9 1 .452741 6.06 .05 618870 2 2.816636 25.13 .001 * 30 166 71 120 94 481 709:6'8'70 1 9.933138 88.70 .001 *9 674 931 878 580 700 3763 Total: 6 13.200515 117.87 .001 # P“ . 1:71:28: .. .. : (Stating--Clarify1ng) " 7n6£4uctoa 7:10 1 .0110231 .13 us 6 7 8 9 10 1 6-10 6:8 1 .0310683 .37 NS 60811070 1 1.0060760 11.96 .001 9 66 96 52 88 67 367 6.8.7.10:9 1 2.7342276 32.51 .001 ** 676 931 878 560 700 3763 Tocal: 6 3.7823750 66.97 .001 A PAIRED UTTERAN 8 7 17161711161804 df s s x3 p (Clarify1ng--Stating) Conpmaon ' ' laatauczaa 7770 1 .0572102 .85 NS 6 7 8 9 10 (6-70 6:8 1 .0933229 1.38 NS 6o8s7o10 1 .7925218 11.76 .001 9 26 66 65 79 58 272 6o8.7.10:9 1 3.0811421 65.73 .001 ** 674 931 878 560 700 3743 Total: 6 6.0261970 59.72 .001 PAIRED UTTERAEE‘3-7 3:322:37: d! 3.5. x! p (Prepar1ng--Stat1ng) 8 9 70 t6-70 719870 2 .0882722 2.27 NS 8:6 1 .7147924 18.35 .001 * 43 46 18 23 22 152 7099708806 1 .0455892 1.17 NS ** 674 931 878 560 700 3743 Total: 4 .8486538 21.79 .001 W Instinct“. . df 5.5. X3 p “(Clarifying-Clarifying) “'pw‘" leatauctoa 9:10 1 .0001587 .00 NS 6 7 8 9 70 x6-10 6:7 1 .0486810 1.40 NS 99702607 1 1.2035236 34.62 .001 * 22 20 8 38 47 135 6079987018 1 .8336781 23.98 .001 ** 674 931 878 560 700 3743 Total: 4 2.0858414 60.00 .001 * Total incidences for the paired categories examined ** Total utterances for all categories (1-13) 119 statistically significant difference to one another. This was also the case with Instructors 7 and 10. However, these two subgroups are recorded separately to indicate that there was a statistically significant difference between them in their use of the paired utterance 7-13. TABLE 22 SUMMARY OF FINDINGS FROM TABLE 21 TO IDENTIFY THOSE 'LEAST EFFECTIVE' INSTRUCTORS WHOSE USE OF PAIRED UTTERANCES WERE SIMILAR TO EACH OTHER HOMOGENEOUS GROUPS OF INSTRUCTORS WHOSE USE OF A PAIRED UTTERANCE WAS NOT PAIRED CATEGORY SIGNIFICANTLY DIFFERENT FROM EACH OTHER lxwmner (3 Sta Sta Lamnmm ‘ \7\7 (DH O Manner hfinmer hammer Sta Sta Sta (mar 0 ~ Sta Sta Cl O ‘ HQHQSml-‘ko O 8 9 6 7 8 9 6 9 As shown in Table 23, the use of five out of the ten paired utterances examined remained constant throughout the lesson. In the use of paired utterances 13-7, 7-13, 13-5 and 1-13, no more than one instructor showed a significant difference in the first and second halves of the lesson (p=.05). 1220 TABLE 23 DIFFERENCES BETWEEN THE PAIRED CATEGORIES USED IN THE FIRST AND SECOND HALVES OF THE LESSON BY THE 'LEAST EFFECTIVE' INSTRUCTORS PAIRED UTTERANCE 13-7 PAIRED UTTERANCE 1-13 PAIRED UTTERANCE 7-8 (Learner-—Stating) (Scanning--Learner) (Stating--Clarifying) 1. X' p X‘ p X‘ p 3 6 1.67 173 3- 6 4.50 .05 E 6 2.17 713 g 7 .62 NS ‘6’ 7 .03 NS 0 7 .04 NS :.- 8 6.25.02 :1 8 g 8 .70 113 t; 9 .00 us 73‘ 9 g 9 ,510 2.29 773 310 7: 10 .06 113 H H PAIR D UTTER 7- 3 PAI D UTTERANC 5- 3 PAIRED UTTERANCE 8-7 (Stating--Learner) (Regarding--Learner) (Clarifying-~Stating) H x! p 7-0 x! p p X! p 3 6 .87 us 3 6 3 6 .17 NS '5’ 7 11.37 .001 g 7 g 7 .00 173 3 8 ..70 NS 3 8 .33 NS ‘7: 8 .56 NS 00 9 m 9 .00 173 ... 9 .5 10 5 10 :3 10 1.10 113 PAIfiD UTTERANCE 13-5 PAIREJDLUTTERANCE 7-7 PAIRED UTTERANCE 3-7 (Learner-~Regarding) (Stating--Stating) (Preparing--Stating) p X1 P - Significant difference 3 X7 p 3 6 7.38 NS at .05 level and so no 3 6 U 7 3.85 .05 - :3 7 7.39 N3 3 8 .62 NS meaningful comparison 3 8 11 9 could be made. 2 9 .04 NS 210 ... 10 .36 113 H PAIREQfiUITERANCE,8-8 Note: i)-- Low frequency prevented meaningful (Clarifying-Clarifying) comparison with X' x, p ii) X' figures only calculated for 'least' '3 . . ..- 6 7.64 713 effective instructors whose use of g 7 .20 NS paired utterances were 39; 3 8 51 ificantl different "1 9 1-63 NS 9" ’ £10 .53 17s 121 Differences or Similarities Between the Probability of One Category Following Another in Selected Paired Utterances Used byfthe 'Least Effective' Group From Table 24 it can be seen that there were no significant differences between the five 'least effective' instructors in the probability of one category following another in the paired utterances of 1-13 and 5-13. It can be extrapolated from these figures that on average, the probability of Learners (category 13) responding after an instructor asks a question (category 1) was 60% of the time, and the probability of Learners (category 13) responding after an instructor acknowledges or gives a learner positive praise (category 5) was 40% of the time. Table 25 summarizes the average probability of one category following another in the paired utterances used by the 'least effective' group. For example, from these findings it can be expected that when a Preparing category (category 3) is used by a 'least effective' instructor, it will be followed 51% of the time by a Stating category (category 7). Similarly, when a Stating category (category 7) is used by a 'least effective' instructor, it is probable that 43% of the time it will be followed by a further Stating category, 28% of the time by a Clarifying category (category 8) and only 9% of the time by a Learner response (category 13). 122 TABLE 2 4 DIFFERENCES IN PROBABILITY OF ONE CATEGORY FOLLOWING ANOTHER IN SELECTED PAIRED UTTERANCES FOR 'LEAST EFFECTIVE' GROUP jIPA'I'R‘§——T10 1171; Am; 13,-7 (Learner--Statin ) 7“““9‘” d: s.s. x! 6 7 8 g 9 10 86-10 “'PW‘" p 6:819:70 3 .629582 3.01 NS * 15 59 16 3. 28 121 69899970:7 1 4.191938 20.04 .001 9* 91 133 63 16 103 406 Total: 4 4.821520 23.05 .001 W mum“ . d! 5.5. X7 p (Statingo-Learner) “'"W‘" lastsuctoa 618 1 .0214927 .32 NS 6 7 8 9 10 8:11.10 7:10 1 .0306709 .66 115 69887670 1 .4001849 5.97 .02 * 11 39 13 2 26 91 6979897079 1 1.3406654 20.01 .001 ** 152 394 228 257 229 1260 Total: 4 1.7930139 26.75 .001 WA :, 13-5 71160.net“. . d! 5.8. X' P (Learnero-Regarding) c°"“‘““ 6 7 8 9 10 t6-70 8:98” 2 .1582860 .72 115 7:6 1 .4398500 1.99 NS 9 26 26 26 7 49 136 7.618.9410 1 4.7898778 21.66 .001 99 91 133 63 16 103 406 Total: 6 5.3880118 26.37 .001 A A 1-15 é;‘2jf:: a: s.s. x: p ( Scanning--L earner ) p 6 7 8 9 10 x6-10 * 11 59 11 3 16 100 99 15 . 91 28 6 27 167 Total: 4 1.762086 7.25 115 A D A 5-13 78618.net“ df S S x, (R » Conpaaiaoa ‘ ' p egardinfgulearner) 8 9 10 s6-10 * 17 6 12 6 32 73 99 40 28 34 13 67 182 Total: 4 1.518598 6.27 Ns PAIRED 7’7 lnatauctoa df S S x, p (Stating--Stating) Conpwaon 6 7 8 9 10 16.70 7:9:10 2 .666660 1.97 NS 6:8 1 1.185971 5.02 .05 9 30 166 71 120 96 681 648:7.4410 1 7.315817 31.00 .001 99 152 396 228 257 229 1260 16:61. 6 8.966268 37.99 .001 PAIRED UTTERANCE 7-8 lutauctas d: S s x3 (51’. t' . Cal-107114126081 ' ' p a ing--Clarifying) 6 7 8 9 10 16-70 679110 2 .6506380 2.26 :85 7:8 1 .0159500 .08 NS 9 66 94 52 88 67 367 7981699970 1 2.6330318 13.19 .001 99 152 394 228 257 229 1260 Tocal: 6 3.0996198 15.53 .01 PAIED fl! EEEEE E-T . . 52:““9‘2: 8: 8.3. x: p (Clarifying--Stating) "M 6 7 8 9 70 16970 78889 2 1.223905 5.01 NS 6:70 1 1.533114 6.28 .01 * 24 66 45 79 58 272 7080,8‘070 1 3.876370 15.88 .001 ** 103 135 111 145 148 642 Total: 4 6.633389 27.17 .001 PAI D U71 RANC -7 178684689304 df s s X3 P (Preparing,«5tating) Conpmaon 6 8 9 10 (6970 619110 2 .1710060 .68 NS 778 1 4.7722100 19.12 .001 * 43 46 18 23 22 152 7688669670 1 .0799484 .32 NS ** 88 69 63 43 39 302 Total: 4 5.0301734 20.12 .001 P71 -8 ' . , g;““€‘:: 88 s.s. x: p (Clarifying-Clarifying) PM 6 8 9 10 16-10 6:9 1 .1615226 .85 :75 778370 2 4.1885630 25.22 .001 * 22 20 8 38 47 135 6698768670 1 .4049304 2.44 NS 99 103 135 111 165 148 642 Total: 6 6.7350160 28.51 .001 * Total inci‘dences for the paired categories examined ** Total utterances for the preceding category in the pair 123 TABLE 25 SUMMARY OF TABLE 24 TO RANK THE R PROBABILITY OF ONE CATEGORY FOLLOWING ANOTHER IN THE PAIRED UTTERANCES USED BY THE 'LEAST EFFECTIVE' INSTRUCTORS PAIRED UTTERANCE EXAMINED 2 Probability (1-13) Scanning followed by Learner 60% (3- 7) Preparing " Stating 51% (8- 7) Clarifying " Stating 49% (7- 7) Stating " Stating 43% (5-13) Regarding " Learner 40% (13- 5) Learner " Regarding 33% (7- 8) Stating " Clarifying 28% (8- 8) Clarifying " Clarifying 28% (13- 7) Learner " Stating 23% (7-13) Stating " Learner 9% An inspection of Table 26 reveals that in seven out of the ten paired categories examined, the probability of one category following another did not differ significantly (p=.05) during the lesson. While there was variance between instructors in the paired utterance 13-5, only one instructor showed variance throughout the lesson in the use of the paired utterances 7-13 and 7-7. 124 TABLE 26 DIFFERENCES IN THE PROBABILITY OF ONE CATEGORY FOLLOWING ANOTHER IN PAIRED UTTERANCES USED BY THE 'LEAST EFFECTIVE' INSTRUCTORS DURING THE FIRST AND SECOND HALVES OF THE LESSON PAIRED UTTERANCE 13-7 PAIRED U‘I’TERANCE 1-13 PAIRED UTTERANCE 7-8 (Learner-~Stating) (Scanning--Learner) (Stating--Clsrifying) 2 o U a S 6 z. 3 6 5.38 NS 0 7 .00 us 3 7 3; NS 8 3 ‘2 NS :3 :3 . NS 3 . 3 8 I. 20 NS 1.1 8 I. 26 NS 3 9 - 05 u 9 .00 NS 3 9 .22 Ms c 10 .04 ,5 10 1.28 us c 10 .21 NS H H PAIRED UTTERflE 7-13 PAIR D UTT RANCE 5-13 PAIRED UTTERANCE 8-7 (Stating-~Learner) (Regarding--Learner) (Clarifying--Stating) X‘ p n X‘ p X’ p § 5 -76 ”5 § 6 .27 83 3 6 3 g ’4-;: mg g 7 .02 NS 3 7 .58 NS 3 10 ‘-4' as 3 8 .08 NS 3 8 .14 NS '5 ~ - 2 lg .00 NS 3 9 1.81 NS ... .03 NS 5 10 PAIRED UTTERANCE 13-5 PAIRED UIIERME 7-7 PAIRED UIIERANCE 3-7 (Learner--Regarding) (Stating--Stating) (Preparing--Stating) X2 p 2 H X P X2 o “ p U 6 .63 NS 8 6 .29 NS 3 6 N 7 I. 90 NS 0 U ° 00 S 3 8 6 08 0 2 a 7 6.22 .02 u 7 u 9 . .- 5 8 L26 us 3 8 E 10 ”:15 '0’ 3 1(9) .83 Ms 6’ 9 .71 113 H .00 NS ,5 10 1.25 NS N . PAIRED UIIERANCE 8-8 ote. 1)-=-Low frequency prevented meaningful (Clarifying-Clarifying) comparison with X’ x! p ii) X’ figures only calculated for 'least' 3 5 33 NS effective' instructors whose use of 3 '7 . paired utterances were at E 8 NS significantly different 2 13 I 45 Differences or Similarities Between Selected Sequences of Three Utterances Used by the 'Least Effective' Group Table 27 identifies in rank order the five sequences of three utterances most frequently used by the 'least 125 effective' instructors. Table 27, indicates that the predominant pattern of three utterances is the use of the combinations of the Stating (7) and Clarifying (8) categories. Table 28 has been extrapolated from Table 27 and has been included to highlight the lack of interactive communication between learners and their instructor. TABLE 27 RANKING OF INCIDENCE OF SEQUENCES OF THREE UTTERANCES FREQUENTLY USED BY THE 'LEAST EFFECTIVE' INSTRUCTORS 5 Ex: 6 7-6-8-6-8 7+7-68 7+8+7 13-9-5613 3--7--8 > E g 7 7+7-t-7 7-6-8-9-7 7-9-7->8 8+7-6-7 7-61613 U 33 g 8 7+8+7 7+7--7 31.7-7 8»7->8 B 5 13-65913 U) E E3 9 7646—7 7-8--7 8-—7-7 7+7-8 8+7—n-8 14 - 10 7-67a-7 7--7-o-8 7¢8+7 13+5-13 8+7+7 Criteria for selection: Sequences of three utterances were identified if the initial paired sequence appeared 5% or more of the frequency total of each instructor. These sequences were ranked according to their frequency for each instructor. 126 TABLE 28 SUMMARY FROM TABLE 27 TO IDENTIFY PATTERNS IN THE SEQUENCES OF THREE UTTERANCES FREQUENTLY USED BY THE 'LEAST EFFECTIVE' INSTRUCTORS 7-7- 8 STATING -- STATING -- CLARIFYING 7-8- 7 STATING - ClARIFYING - STATING 7-7- 8 STATING - STATING -- CLARIFYING 7-1-13 STATING -- SCANNING -- Learner 8-7- 7 CLARIFYING - STATING -- STATING 8-7- 8 CLARIFYING - STATING -- CLARIFYING 7-8- 8 STATING - CLARIFYING -- CLARIFYING 3-7- 8 PREPARING - STATING - CLARIFYING 13-5-13 Learner -- REGARDING -- Learner 5955: i) A sequence of three utterances such as 7-7-8 is read as: 'instructor stating, followed by instructor stating, followed by further instructor clarification' ii) Sequences of three utterances have been arranged to highlight the predominant use of Stating (Cat. 7) and Clarifiying (Cat. 8) and the lack of verbal interaction between the instructors and their learners. Findings in Response to Researchguestion 2.0 Research question 2 asked: Do the verbal communication behaviors used by groups of the 'most effective' instructors differ from those verbal communciation behaviors used by groups of 'least effective' instructors. To answer this question adequately only those 'most effective' or 'least effective' instructors who showed no significant differences (p=.05) in their use of categories were chosen for comparison. Thus only those subgroups of instructors already identified as homogeneous in response to research Questions 1.0 and 1.1 were compared. 127 Because of the low frequencies of Category 9 (Excusing, Unintentional), Category 10 (Excusing, Legitimate), Category 11 (Apology) and Category 12 (Penance), meaningful comparison using chi-square tests for statistically significant differences was not possible. As Figure 15 shows, the 'least effective' group tended to use these categories more than the 'most effective' group of instructors. Category 10 (Excusing, Legitimate) was the category most frequently used by the 'least effective' instructors, while the 'most effective' group were the only instructors to use Category 12 (Penance). Figure 16 has been included to provide an overview of the proportional use of the combination of the various categories with Category 4 (Sharing) and Category 5 (Regarding). From this figure it can be seen that the 'most effective' group used the Regarding combination category far in excess of the 'least effective' instructors. On the other hand, the 'least effective' group used the Sharing combination category far in excess of the 'most effective' instructors. 128 SSDNVHHIID HOLDflHlSNI TVIOI 30 Z OZ: 92- DE: SE- 09- 97- 09- SS: 09- 99-0 KEY: - 'Most Effective' group .L. 'J‘ g 1 D 'Least Effective' group 3 Category 9 = Excusing (Unintentional) - 10 : Excusmg (Legitimate) 9 10 11 12 11 = Apology CATEGORY 12 = Penance FIGURE 15 DIFFERENCES IN THE PROPORTIONAL USE OF CATEGORIES IN THE 'REPAIRING' CLASSIFICATION 129 SSDNWZIIII'I HOIDHHISNI 'IVIOJ. :10 Z SHDNVXEIJJ] 801311818111 'IVIOJ. :10 Z 9 S 9 E Z a) COMBINATIONS WITH "SHARING" CATEGORY KEY: I 'Most Effective' group [::] 'Least Effective' group Combination category 41 = Sharing + Scanning 43 = Sharing + Preparing 45 = Sharing + Regarding 46 = Sharing + Suggesting 47 = Sharing + Stating . a 48 = Sharing + Clarifying 61 43 65 66 67 48 410 410 = Sharing + Excu51ng COMBINATION CATEGORY (Legitimate) 51 = Regarding + Scanning 52 = Regarding + Requesting b) COMBINATIONS WITH " 53 = Regarding + Preparing "REGARDING CATEGORY 56 = Regarding + Suggesting 57 = Regarding + Stating 58 : Regarding + Clarifying 59 = Regarding + Excu51ng (Unintentional) 51 52 53 56 57 58 59 COMBINATION CATEGORY FIGURE 16 DIFFERENCES IN THE PROPORTIONAL USE OF COMBINATION CATEGORIES 130 Differences in Proportional Use of Categories by the 'Most Effective' and 'Least Effective' Groups In Table 29, the proportional use of categories by subgroups of 'most effective' and 'least effective' instructors are compared. Chi-square tests for independent samples show the statistical significance of differences between each group in their proportional use of a category. In six out of the ten categories examined there were significant differences in their proportional use by the 'most effective' and 'least effective' subgroups at the .01 or .001 levels. In the use of the other four catgories examined, at least one of the subgroups compared showed no significant difference in the use of that category. A closer inspection of these figures reveal that the 'most effective' subgroups used categories in the following ratios to the 'least effective' subgroups: Scanning 5:1, Regarding 3:1 and 4:1, Suggesting 7:4, Sharing (combination category) 12:7, and regarding (combination category) 4:1 and 2:1. On the other hand, the ratio in the use of categories by the 'least effective' to the 'most effective' subgroups were Preparing 3:2 and Clarifying 3:2 and 2:1. With the use of both Sharing (category 3) and Stating (category 7) there were significant differences between the 'most effective' and 'least effective' subgroups but the ratios of each subgroup were not constant. 143, The ratio of the 'most effective' to the 'least effective' subgroups in the use of Sharing (category 4) was 1:6 and 3:1 and Stating (category 7) was 4:5 and 5:4. 131 TABLE 29 DIFFERENCES IN THE PROPORTIONAL USE OF CATEGORIES BY THE 'MOST EFFECTIVE' AND 'LEAST EFFECTIVE' SUBGROUPS CATEGORIES 1+2 (Scanning) Proportion of Instructor categories to total x1 p utterances (1-12) * . .. 6:311. W "we .0... CATEGORY 3 (Preparing) Proportion of Instructor category to total X2 p utterances (1-12) 3;, 3:3 ml“ f 1575 10.23 .01 * 3+4+5 316 : 3139 u 7+9 Ira—rm ~30 “3 " Sh ' CATEGORY 4 ( aring) Proportion of Instructor category to total X2 p utterances (1-12) * 1+5 4 : 1808 58 NS ** 7+10 5 : I395 ° * 3+4 23': 1974 6 45 02 ** 7+IO 5 : I395 ' ' * 3+4 23 : 1974 ,4 NS ** 8+9 I7 : I359 C . ATEGORY 5 (Regarding) Proportion of Instructor category to total X2 p utterances (1-12) L, .313” W185 E 1666 80.73 .001 :* $Ig+9 38%—é—%}§% 149.60 .001 _______CATEGORY 6 (suggesnng)Proportion of Instructor category to total X2 p utterances (1-12) :* §:g+4 199 : 2806 7.36 NS ** 9+10 132 TABLE 29 (cont.) CATEGORY 7 (Stating) Proportion of Instructor categories to total X2 p utterances (1-12) * 1+2 651 g 1612 9 30 o; ** 7+9 723 : I352 * 1+2 651 : 1612 14 41 001 ** 8+10 577 = 1575 ° ' 8 f ' ) CATEGORY (Clari ying Proportion of Instructor categories to total x! p utterances (1-12) * 2+3+4+5 463 8 3971 58 50 00] ** 6+7+8 ‘12 = 2195 ' . * 2+3+6+5 663 . 3971 .6 9.10 m ”9'” '0‘” CATEGORIES 9, 10, ll & 12 Low frequencies prevented meaningful comparison ATEGORY 13 C (Learners) Fraportion of Instructor categories to total X2 p utterances (1-13) * 1+3+s 1092 : 6060 ** 6+7+10 ‘527‘7-2305 ’49-’0 -00’ * 2+4 801 : 2467 ** 6+7+10 m ”“3 -°°' COMBINATION CATEGORY 4 Proportion of (Sharing) Instructor categories to total x! p utterances (1-12) * 1+3 207 . 1783 ** 7+9+10 : 24.11 .001 * 2+5 166 : I997 ** 7+9+10 77—1939 2.34 NS COMBINATION CATEGORY Sproportion Of (Regarding) Instructor categories to total x' P utterances (1-12) * 1+3+s 26o : 2968 .. 7+1o “m3 ”'03 '0‘” * 2+6 76 : 1666 s 96 01 ** 7+10 3‘ ‘ I395 . . Note: i) ' Those 'most effective' instructors with no significant differences in the use of the category under examination. Each instructor's proportional use of the category to his/her total utterances are added to make up proportions shown in column 2. ii) ** those 'least effective' instructors with no significant differences in the use of the category under examination. Each instructor's proportional use of the category to his/her total utterances are added to make up proportions shown in column 2. 133 Differences in the Use of Selected Paired Utterances by the 'Most Effective' and 'Least Effective' Subgroups Table 30 provides a comparison of selected paired categories used by the 'most effective' and 'least effective' subgroups. There were statistically significant differences between the 'most effective' and 'least effective' instructors in their use of eight of the nine paired utterances compared. The exception was in the use of the paired category 3-7 (Preparing followed by Stating). 134 TABLE 30 DIFFERENCES IN THE USE OF SELECTED PAIRED UTTERANCES BY THE 'MOST EFFECTIVE' AND 'LEAST EFFECTIVE' SUBGROUPS PAI D UTT 3-7 (Learner--Stating) Proportion of paired Instructor categories to total X2 p utterances (1-13) * 1+2 197 : 2150 ** 6+7+8+9+10 IZI 1 3763 ,94°06 '00' * 3+5 218 : 3127 *4 6+7+8+9+10 m ”-77 ~00’ EAHEQAEEEEMELLQI (5t8t1"9“L°°"‘°r) Proportion of paired Instructor categories to total X2 p utterances (1-13) EAEEQJEEEEEQLIEQ. (Learmr"“°9”di"9) Proportion of paired Instructor categories to total X2 p utterances (1-13) * . *9. a?” W222 3 2513 71.71 .001 * . ** 2:31; W224 ; 399“ 20.64 .001 .flflflglfllflflflfligtil (Scanning--Learner) Proportion of paired Instructor categories to total X2 p utterances (1-13) :* gig+5 228 f 3743 53.29 .00, .flflflfllfllflflflfliitll (Regarding-'Lflrmr) Fraportion of paired Instructor categories to total X1 p utterances (1-13) * 3+4+5 79 : 2150 *1: 8+9 m 79.23 .007 135 TABLE 30 (cont.) liflfifliflflflflflthl (Stating--Stating) The 7-7 paired categories within the 'effective' and 'less effective' groups were significantly different at .05 level and so no meaningful comparison could be made PAI D UII R 7-8 (Stating--C1°rif71"9) Proportion of paired Instructor categories to total X2 p utterances (1-13) * 2+3+4+5 210 : 5594 *4: 6+8 W ’9-3’ '0‘” * 2+3+4+5 210 : 5594 .1... 7+10 m “~00 -°0’ PM 0 1111 RA -7 “huffimffintmfi Proportion of paired Instructor categories to total X2 p utterances (1-13) * . ** éifi %%—+—%%§g 22.04 .001 * 1+2 39 : 21 0 ** 7+1o 126—T'1651 75'35 ~00’ Eflflgfliflflaflfilitl (Preparingnfiatim) Proportion of paired Instructor categories to total X2 p utterances (1-13) * 1+3+4 146 : 3670 ** 7+9+1o ‘9I‘7‘IT9I "0 ”3 * 2+5 60 : 2837 .... 7+9+1o m ’7'“ '0‘” PAI D UTT R -8 (Clarifying-Clarifying) Proportion of paired Instructor categories to total x2 p utterances (1-13) ** 6+7 57 = 1555 ' ' * . ** giggh+5 fig—e—gggg 254.75 .001 525g: 1) * Those 'most effective' instructors with no significant differences in the use of the category under examination. Each instructor's proportional use of the category to his/her total utterances are added to make up proportions shown in column 2. ii) 8* Those 'least effective' instructors with no significant differences in the use of the category under examination. Each instructor's proportional use of the category to his/her total utterances are added to make up proportions shown in column 2. 136 Table 31 provides a summary of the ratios between -the 'most effective' and 'least effective' subgroups in their use of paired utterances. TABLE 31 RATIOS BETWEEN 'MOST EFFECTIVE' AND 'LEAST EFFECTIVE' SUBGROUPS IN THEIR USE OF PAIRED UTTERANCES PAIRED UTTERANCE EXAMINED RATIO OF USE 'Most Effective' : 'Least Effective' 7 - 13 (STATING followed by Learner) 5:1 & 7:4 1 - 13 (SCANNING " Learner) 4:1 13 - 7 (Learner " STATING) 3:1 & 7:2 13 - 5 (Learner " REGARDING) 3:1 & 2:1 5 — 13 (REGARDING " Learner) 3:1 'Least Effective' : 'Most Effective' 8 - 8 (CLARIFYING followed by CLARIFYING) 9:1 7 - 8 (STATING CLARIFYING) 3:1 8 - 7 (CLARIFYING " STATING) 2:1 3 - 7 (PREPARING " STATING) 2:1 Note: The use of all paired utterances by the 'zmost effective' and 'least effective' subgroups compared were significantly different at the .001 level. As shown in Table 31, every paired utterance that included interaction between the learners and their instructor, was used by the 'most effective' subgroups from twice to five times as often as the 'least effective' subgroups. On the other hand, the 'least effective' subgroups used paired utterances that contained Preparing, Stating or Clarifying categories from twice to nine times as 137 much as the 'most effective' subgroups. Differences in the Probability of One Category Following Another in Selected Paired Utterances Used by the 'Most Effective' and 'Least Effective' Subgroups An examination of Table 32 reveals that there were similarities in the probability of an instructor using an explanation (Stating, category 7) after a Learner's response (category 13); and a Learner (category 13) responding to an instructor's acknowledgement (Regarding, category 5). There was a significant difference (p=.05) in the probabilities of one category following another in all other paired utterances examined. From the Comparison of these probabilites displayed in Table 33, major differences are apparent. For example in the 'least effective' subgroups, the probability of a Clarifying category following immediately after a previous Clarifying category (8-8) was 20% more than the 'most effective' subgroup of instructors. 138 TABLE 32 DIFFERENCES IN THE PROBABILITY OF ONE CATEGORY FOLLOWING ANOTHER IN SELECTED PAIRED UTTERANCES BETWEEN SUBGROUPS OF 'MOST EFFECTIVE' AND 'LEAST EFFECTIVE' INSTRUCTORS PAIRED UTTERANCE 12-7 (LearneruSiati ) Total Total 2 net ctors category catigory X p 7 * 1+2+3+5 415 : 1496 .. 6+8+9+10 177—273 2 - 96 ”3 PAIRED UTT R 7- (Stating--Learner) Total Total Instructors category catfipory X‘ p 1 * l+3+5 248 : 903 ** 6+8 ‘24‘7—_380 123.93 .007 * 1+3t5 248 : 903 ** 7+1o 65 = 623 ‘5'57 '00’ * 2+4 274 : 594 ** 6+8 2E : 380 172.97 .001 *, 2+4 274 : S94 ** 7+10 65.: 623 166.82 .001 PAIRED UTTERANCE 13-2 (Learner--Regarding) Total Total Instructors category category x2 p 13 * l+3+5 324 : 1091 *4: 8+9+1O "8'2‘7—‘1'8'2 ”-55 '0‘“ * 2+4 122-: 801 ** 6+7 52 : 225 7.64 .0, * 2+4 122 : 801 ** 8+9+10 -EZ—?——I82 80.20 .001 * I+3+5 324 : 1091 *‘A’ 6+7 52 : 225 3.84 .05 W (Scanning--Lesrner) Total Total Instructors category category X! p * 1+2+3+4+5 379 : 569 ** 6+7+8+9+10 : 4°39 '05 PAIR D UTT R -l (Regsrding--Learner) Total Total x , p Instructors category category 1 5 * 2+3+4+S 194 : 504 .15 NS ** 6+7+8+9+10 73 : I82 139 TABLE 32 (cont.) PAIRED UTTER 7-7 ( Total (Stating--Stating) Total Instructors category category X1 p 7 7 * l+2+4+5 169 : 1096 ** 6+8 = 23.52 .001 * '1+2+4+5 169 z 1096 ** 7+9+10 : 187.50 .00! PAIR D UTI RA 7-8 (Stating-—Clarifying) Total Total Instructors category category X1 p 7 :* 313+“15 21° 3 1313 15.71 .00: * 2+3+4+5 210 : 1313 ** 6+9+10 : 62.13 .001 W (Clarifying--Ststing) Total Total Instructors category category x2 p 7 * l+2+5 56 : 327 .... mm mm ”-3" '0“ W (Preparing--Stating) Total Total Instructors category category X2 p * 1+3+4 14 : ** 6+9+10 “3§‘?“%§% ”°’9 '00’ * 2+5 60 : 146 3.59 NS ** 6+9+10 38 : I70 PAI§§D UII§RA§§§ 8-8 Clarifying-Clarifying) Total Total ’ Instructors category category X2 p 8 8 * . l+2+3+4+5 41 ; 534 44.4, .00, Note: ** 6+9 i)* in the use of the category under examination. Those 'most effective' instructors with no significant differences Each instructor's proportional use of the category to his/her total utterances are added to make up proportions ahoun in column 2. ii) ** Those 'least effective' instructors with no significant differences in the use of the category under examination. Each instructor's proportional use of the category to his/her total utterances are added to make up proportions shown in column 2. 140 TABLE 33 COMPARISON OF THE PROBABILITY OF ONE CATEGORY FOLLOWED BY ANOTHER % Probability SELECTED PAIRED UTTERANCES EXAMINED MOST LEAST EFFECTIVE EFFECTIVE 1 - 13 Scanning followed by Learners 66% 60% 3 - 7 Preparing " Stating 68% 51% 7 - 13 Stating " Learners 25% 9% 13 - 5 Learners " Regarding 24% 33% 8 - 7 Clarifying " Stating 18% 49% 7 - 8 Stating " Clarifying 16% 28% 7 - 7 Stating " Stating 15% 43% 8 - 8 Clarifying " Clarifying 8% 28% Note: The probability of one category following another in the selected paired utterance used by the 'most effective' and 'least effective' subgroups were significantly different at the .05 level. Differences or Similarities in the Use of Sequences of Three Utterances by the 'Most Effective' and 'Least Effective' Groups An examination of Tables 13 and 27 reveals that for the ten most frequently used sequences of three utterances by the 'most effective' group, there was always interaction between the instructor and his learners. Over 80% of these sequences contained either the instructor scanning for misunderstandings (category 1) or giving learners positive praise or acknowledgement (category 5). In contrast, in the ten most frequently used sequences by the 'least effective' group, only two sequences show any interaction between instructors and learners, or any use of Scanning (category 1) or Regarding (category 5). 141 Findings in Response to Researchguestion 3.0 Research Question 3 asked: Are there patterns of verbal communication which are used more consistently by the group of 'most effective' instructors than by the group of 'least effective' instructors? Conversely, are there patterns of verbal communication which are used more consistently by the group of 'least effective' instructors than those used by the group of 'most effective' instructors? Patterns of Verbal Interaction Between Learners and their Instructors The most distinctive differences emerging from the the comparison of the 'most effective' and 'least effective' instructors in their management of intent were the patterns of verbal interaction between learners (category 13) and their instructors. During the first hour of lessons, learners communicated twice as much with the 'most effective' instructors than with the 'least effective' instructors. This pattern remained constant throughout the lesson for the 'most effective' instructors but this was not the case with most of the 'least effective' instructors. A characteristic pattern occurred in the use of paired utterances that included interaction between learners and their instructors. In all cases the 'most effective' instructors used these interactive paired utterances from twice to five times as much as the 'least effective' instructors. Except for the paired utterance 5-13 142 (instructor Regarding followed by Learner response), the use of these interactive patterns of paired utterances by the 'most effective' instructors remained constant throughout the lesson. On the other hand, and except for the use of the 5-13 category, the 'least effective' group used these interactive paired categories with less consistency throughout the lesson. There was no clear pattern in the probability of an instructor's utterance following a learners' response, or vice versa. There were similarities in the use of the paired utterances of 5-13 (instructor Regarding followed by a Learner response) and 13-7 (Learner utterance followed by instructor Explaining). There was a 9% greater probability of instructor Regarding (category 5) following a Learner's utterance (category 13) in the 'least effective' group than in the 'most effective' group. On the other hand there was a significantly greater probability that learners would respond to questions or explanations made by the 'most effective' instructors than those in the 'least effective' group. A dominant pattern of verbal interaction between learners and their instructor occurred in the sequences of three utterances most frequently used by the 'most effective' instructors. In all cases, the ten most frequently used sequences of three utterances included some interaction between the instructor and his learners. These patterns of interactive sequences of three utterances used 143 by the 'most effective' instructors were typified by scanning for misunderstandings (category 1) and the giving of positive praise or acknowledgement to learners (category 5). Only two interactive patterns 13-5-13 (Learner utterance followed by instructor Regarding followed by Learner utterance) and 7-1-13 (instructor Explaining followed by Scanning followed by a Learner response) were used with any frequency by the 'least effective' instructors. Patterns in Assessing The greatest difference between the 'most effective' and 'least effective' instructors' use of categories was in their pattern of assessing if they had been misunderstood by their learners. The 'most effective' group used the categories of Scanning and Requesting (categories 1 and 2) five times more often than the 'least effective' instructors. In both cases, this pattern of assessing intent was constant throughout the lesson in both groups. The pattern of a learner responding to an instructor's question (paired category 1-13) occurred twice as often with 'most effective' instructors than with the 'least effective' group. In fact, there was a probability that learners would respond to assessing of intent 66% of the time, which was significantly different to that of the 'least effective' group. The sequence of three utterances, 7-1-13 (Stating followed by Scanning followed by Learner response) was used 144 frequently by both groups. However, the patterns of l3-l-13 (Learner followed by instructor Scanning followed by Learner response) and 1-13-5 (Scanning followed by Learner response followed by Regarding) were utilized more frequently by 'most effective' instructors than the 'least effective' group. Patterns of Explaining Three major verbal patterns emerged in the way the 'most effective' and 'least effective' instructors explained their intent to their learners. First, the 'least effective' instructors used the Preparing category before Explaining (3-7) consistently throughout the lesson and twice as much as the 'most effective' group. Second, the 'most effective' instructors utilized the category of Suggesting (category 6) as an alternate form of explanation a little less than twice as often as the 'least effective' group. Third, there was no clear pattern in either group's proportional use of Stating (category 7) over time, although the 'least effective' instructors Clarified (category 8) their communication twice as much as the 'most effective' group. However, the use of explanation in the form of Stating (category 7) combined with Clarifying (category 8) in paired utterances and sequences of three utterances was the dominant pattern of verbal communication used by the 'least effective' instructors. The four most frequently 145 used paired utterances by the 'least effective' instructors were made up of various combinations of Stating (category 7) and Clarifying (category 8) and were used from twice to nine times as much as the 'most effective' group. The use of these combinations were so frequent that one could predict from the 'least effective' group's pattern of explanation that there is a 49% probability that Clarifying (category 8) would be followed by Stating (category 7), a 43% probability that Stating (category 7) would be followed by Stating (category 7) and a 28% probability that Stating (category 7) would be followed by Clarifying (category 8). This is in contrast to only a 9% probability that Stating (category 7) would be followed by a Learner response (category 13). With sequences of three utterances, nine out of the ten most frequently occurring sequences used by the 'least effective' instructors contained a Stating or Clarifying category. Seventy percent of these sequences of three utterances were made up solely of combinations of Stating and/or Clarifying categories. Patterns of Verbal Reinforcement A distinct pattern of positive reinforcement of learners' utterances was used by all instructors. However, the 'most effective' instructors consistently used throughout their lessons the Regarding category (category 5), or a combination of Regarding with other categories, three to four times as often as the 'least effective' instructors. 146 The 'most effective' instructors reinforced their learners' responses by acknowledgement or praise two to three times as much as the 'least effective' group. The 'most effective' group used this paired category (13-5) consistently throughout their lessons while there were significant differences in its consistent use by the 'least effective' group. While the 'most effective' instructors used this pattern of reinforcement most frequently, there was a greater probability of the use of Regarding (category 5) following a Learner's utterance (category 13) when a learner eventually did respond to a 'least effective' instructor. It should be noted that the learners in the 'most effective' group talked with their instructors twice as much as did the learners in the 'least effective' groups. This pattern of positive reinforcement occurred in forty percent of the sequences of three utterances used by the 'most effective' instructors while it occurred only once in the 'least effective' sequences. Patterns of Sharing There were no distinct patterns in the use of the Sharing category (category 4) by either the 'most effective' or 'least effective' instructors. One 'most effective' subgroup of instructors used the sharing category three times as much as the 'least effective' instructors, while another subgroup of 'least effective' instructors used the Sharing category six times as much as a 'most effective' 147 subgroup. However, the use of the Sharing category (category 4) by the 'most effective' group did not vary significantly throughout the lesson, unlike the significant variations found in the 'least effective' group. While the 'most effective' instructors used the combination category of Sharing (category 4 combined with any other category) almost twice as much as the 'least effective' instructors, neither group used this combination category with consistency throughout the first hour of the lesson. Patterns of Repairing the Relationship Although low frequencies in the use of the categories of Excusing, Unintentional (category 9); Excusing, Legitimate (category 10); Apology (category 11) and Penance (category 12) prevented meaningful statistical comparison, the 'least effective' group tended to use these "repairing" behaviors far more than the 'most effective' instructors. This trend was most apparent in the use of the Excusing, Legitimate category (category 10). The exception was in the use of the category of Penance (category 12) which three 'most effective' instructors utilized, while no instructor in the 'least effective' group used this category. 148 SUMMARY In Chapter 5 the findings of the field test were reported. The verbal communication used by the 'most effective' and 'least effective' instructors was analyzed and compared. The following summary highlights the major similarities and differences found during the field test in the use of the categories, paired utterances and sequences of three utterances: l. Instructors who were perceived as most helpful and well-intentioned when establishing the instructor/ learner relationship tended to verbally interact more frequently with their learners throughout the lesson than those instructors who were 'least effective' managers of intent. 2. Instructors who were perceived as most helpful and well-intentioned when establishing the instructor/ learner relationship tended to interact more frequently with their learners throughout the lesson by the use of: 1) Scanning (questioning to assess if learners have misunderstood or misinterpreted their instructor's intent), and 2) Regarding (reinforcement of learner responses through the use of positive praise or acknowledgement of the learners' contributions or efforts). 3. Instructors who were perceived as most helpful and well-intentioned when establishing the instructor/ learner relationship tended to indirectly explain their 149 intent in terms of making a suggestion rather than always using a direct statement or giving an order to the learners. 4. Instructors who were perceived as most helpful and well-intentioned when establishing the instructor/ learner relationship were the only instructors to follow up an apology for an error or misunderstanding with an offer to do something concrete to repair the relationship. 5. Instructors who were perceived as the least helpful or least well-intentioned when establishing the instructor/learner relationship tended to make more excuses for their action in terms of having no alternative course of action, or that their action was legitimate in their ascribed role of instructor than the 'most effective' managers of intent. 6. Instructors who were perceived as the least helpful or least well-intentioned when establishing the instructor/learner relationship tended more frequently to prepare their learners that an upcoming explanation was not meant to be seen as malicious or arbitary than those instructors 'most effective' in managing their intent. 7. Similarities existed between the 'most effective' and 'least effective' instructors in that: 1) the sequences of the most frequently paired utterances examined were not random and the coded utterances in a sequence could be maximally predicted by knowing the immediately preceding coded utterance; 2) the proportional use of categories and 150 sequences of utterances remained stationary or constant throughout the recorded session; and 3) the subgroups compared were made up of two to five instructors who shared a homegeneity in their use of a category, paired utterance or sequence of three utterances. In the next and final chapter, a summary of all phases of the study are presented and conclusions drawn from the findings of the field test. From these conclusions, implications and recommendations are made for further research in the area of the management of intent in the instructor/learner relationship. Chapter 6 SUMMARY, CONCLUSIONS AND RECOMMENDATIONS The purpose of this study was to develop and field test a technique for classifying and measuring the verbal communication used by adult education instructors for managing their learners' perceptions of instructor intent when establishing an instructor/learner relationship. This section presents a summary of the major findings of the field test and the conclusions that were drawn from the research are then discussed. Implications will be drawn from the conclusions and recommendations made regarding further research in the area of the verbal management of intent. SUMMARY In conducting this study, nine major distinctions were found between those instructors rated as 'most effective' and those 'least effective' in their verbal communication of intent when attempting to establish an adequate instructor/learner relationship. For those instructors who were most successful in establishing initial rapport with their learners, their verbal communication was characterised by: l) a high frequency of verbal interaction between learners and their 151 152 instructor throughout the lesson; 2) the constant use of questioning to assess if the learners had misunderstood or misinterpreted their instructor's intent; 3) the constant reinforcement of all learner responses through the use of positive praise, reference to the learners by name, or the verbal acknowledgement of a learner's contribution or efforts; 4) the instructor explaining intent in terms of making a suggestion rather than using a direct statement of intent; and S) a tendency not only to apologize for an error or misunderstanding but also to make an offer to do something concrete to repair a relationship. For those instructors 'least effective' in establishing rapport, their verbal communication was characterised by: 6) a high frequency of explanations of intent by the contiguous use of various combinations of statements or clarifications; 7) the preparation of learners that an upcoming explanation was not meant to be malicious or arbitary in any way; 8) a tendency to make excuses in terms of having no alternative course of action, or that some action was legitimate in the role of instructor; and 9) repairing relationships frequently through making apologies or excuses, but never offering to do something concrete to repair a misunderstanding. There were a number of similarities in other patterns of verbal communication used by both the 'most effective' and 'least effective' instructors. First, in most instances the sequences of paired utterances examined 153 were not random but formed specific patterns such that a knowledge of a preceding event increased the predictability of what was likely to follow. While the probability of one category following another in a selected paired utterance appeared closely related to the proportional use of that paired utterance, there were two notable exceptions. It was predictable that when either of the 'most effective' and 'least effective' instructors questioned to check that they had been understood, there would be at least a 60 percent probability that this verbal scanning (category 1) would be followed by a learner's response (category 13). When either group of instructors used acknowledgement or praise, there was around a 40 perecent probability that the succeeding utterance would have been a learner response Second, over half of all instructors, each teaching in different subject areas, showed that their use of the various categories or sequences of verbal communication remained constant throughout the first hour of the initial lesson of the term. CONCLUSIONS The following conclusions have been drawn from the findings of the study: 1. A naive group of persons can be trained to learn the classification scheme and apply the coding technique with reliability when unitizing and categorizing transcripts of verbal communication used by an instructor. 154 While the coding technique is not difficult to conceptualize, it is difficult to apply with accuracy and consistency without thorough training in its use. Coder errors were often made in distinguishing between Stating (category 7) and Clarifying (category 8) when unitizing and categorizing long sequences of utterances that were largely made up by these categories. It was also difficult at times to distinguish whether the use of the word "we" was meant in the context of sharing power in the relationship and coded as category 4, gpg, "we the people in this room", or if it referred to the accumulated knowledge of experts in a content area, §;g, "we the experts who know about such matters". This ambiguity in meaning was never satifactorily resolved. 2. The coding technique can be used for obtaining objective data but can only measure one dimension of communication. That is, the content of verbal messages used to manage intent when establishing an instructor/learner relationship. It is important to understand that application of this coding technique is limited. The quality of a relationship in a verbal message, the control implied in a verbal message, the impact of the intensity of a message, or the congruence between verbal and nonverbal messages cannot be measured by this technique. When coding, the influence of paralinguistic cues contained in an utterance were avoided through the use of 155 written transcripts to assist the coders in focusing only on the content of the message. However, this resulted in a confusion as to whether an utterance was made in a positive, neutral or negative way. This lack of knowledge of the relationship, control or intensity implied in a message prevented a more accurate interpretation of the intent of an instructor's verbal communication. For example, the present coding system does not distinguish whether the 'most effective' instructors verbally scan for misunderstandings in an encouraging and positive tone, or whether the 'least effective' instructors verbally scan for misunderstandings using indifferent or negative tones. The use of a learner's name could be a genuine acknowledgement and regard of that learner's identity as an individual, but a learner's name said in a sarcastic tone could well be the antithesis of positive acknowledgement and regard. 3. The categories of the classification scheme represent communication behaviors that occur in an adult education classroom situation. Generally, this was the case except for five categories that occured infrequently in some instructional situations. As would be expected in the first meeting of a class, it was not necessary to repair the new instructor/ learner relationship through the excessive use of Excusing - unintentional (category 9), Excusing - legitimate (category 10) Apology (category 11) or Penance (category 12). Making use of Requests (category 2) instead of using a more direct 156 form of communication was used so infrequently during the field test that this category was collapsed and incorporated into the Scanning category (category 1). However, the Requesting category (category 2) was used a great deal by the 'most effective' trained instructors in the pilot tests which were conducted in more formal institutional settings and with much larger classes. 4. The findings of the study strongly suggest that adult education instructors who wish to been seen as helpful and well-intentioned when establishing the initial instructor/learner relationship in their first class meeting should ensure maximum verbal interaction with their learners. It appears that the frequency of interaction between instructors and learners is related in some way to the learner's positive perception of the initial instructor/ learner relationship. Conventional practice in adult education stresses the use of highly interactive 'ice breakers' in the first meeting to help establish a nonanxious learning environment and a relationship that will encourage further participation in learning. While no instructor observed during the field test used these 'ice- breaker' activities, there was still a great deal of verbal interaction between the 'most effective' instructors and their learners. Conversely with the 'least effective' instructors, there was a lack of interaction with learners and a preoccupation with getting information across to the 157 class. The level of interaction facilitated by instructors in the first meeting may well be an initial indicant to learners of the instructor's respect for the learners' ideas and experience and that in future classes the instructor will be genuinely willing to share the ascribed expert power of his role of instructor. While the facilitation of interaction appears an important factor in achieving the desired goal of being perceived as well-intentioned, the findings also suggest that to be effective, instructors should consciously manage their verbal communication and subsequent interaction using the following strategies: Frequent Scanning to Check for Misunderstandings. Adult eduation instructors should frequently scan for misunderstandings or misinterpretations. Instructors who do this will be perceived by their learners as more well- intentioned than those instructors who less frequently use the Scanning category in their verbal communication. When either a 'most effective' or 'least effective' instructor scanned for a feedback about possible misunderstandings there was approximately a sixty percent probability that a learner would respond. The crucial difference was that the 'most effective' instructors scanned for feedback about misunderstandings figg_times as often as the 'least effective' instructors, used the paired utterance 1-13 (Scanning followed by Learner response) £92; times as often as the 'least effective' instructors and the learners 158 responded to verbal scanning twice as often as the 'least effective' instructors. Useful verbal indicants of the quality of the instructor/learner relationship appear to be the frequent use of the following sequences of three utterances: 13-1-13 (Learner, instructor Scanning, Learner) and 7-1-13 (instructor Stating, instructor Scanning, Learner); and to a lesser extent, the use of 1-13-5 (instructor Scanning, Learner, instructor Regarding) and 1-13-7 (instructor Scanning, Learner, instructor Stating). Frequent Regarding of Learners' Efforts.. Adult education instructors who verbally reinforce their learners' responses through the use of positive praise or simple I acknowledgement of their learners' contributions will be perceived as more helpful and well-intentioned than those instructors who use this type of communication less frequently. When either a 'most effective' or 'least effective' instructor used the Regarding category (category 5) there was approximately a forty percent probability that a learner would respond. The crucial difference was that the 'most effective' instructors used this type of regarding communication to positively reinforce or acknowledge learners' efforts more than twice as often as the 'least effective' instructors. They used the paired utterance 13-5 (Learner followed by instructor Regarding) three times as often as the 'least effective' instructors and the 159 probability that learners could expect some reinforcement or praise after an utterance was ten percent higher than that of the 'least effective' instructors. More effective verbal managers of their intent should attempt to consistently adopt the use of the sequence l-l3-5 (instructor Scanning followed by Learner response followed by instructor Regarding). This verbal sequence appears to be a key indicant that distinguishes those most successful from those least successful in their verbal management of intent. In essence, and as supported by the findings, the 'most effective' managers of intent appear more concerned about frequently checking for possible misunderstandings, ensuring their learners had enough confidence to respond, and then showing the courtesy and concern to regard this response by some form of acknowledgement. Use of Indirect Explanations of Intent. Adult education instructors should indirectly explain their intent in terms of making a suggestion rather than always using a direct statement or order to the learners. The predominant feature of the 'least effective' instructor's verbal communication in this study was their propensity to use long sequences of direct explanations or clarifications, often from twice to nine times as often as the 'most effective' instructors. In contrast, the 'most effective' managers of intent made greater use of indirect verbal interaction through the use of asking more questions, 160 giving praise and acknowledgement, and suggesting rather than telling. Similar results to these were found by Flanders (1969) in Minnesota and New Zealand using his classroom interaction analysis technique with school age students. Instructors who scored high on students liking the instructor, motivation, fair rewards and punishments, lack of anxiety and independence; all used indirect influence in their classroom management. Although these findings were related to school age students, they also are consistent with the concerns and anxieties of adult learners when they commence a new class. 5. The findings of the field test suggest that adult education instructors who wish to be seen as helpful and well-intentioned when establishing the initial instructor/learner relationship should attempt to repair any misunderstandings by offering a simple apology for a mistake and should avoid making excuses based on the the ascribed power given to them by their learners in their role as instructor. While insufficient frequencies in the use of Excusing and Repairing categories (categories 9, 10, 11 and 12) prevented meaningful comparison for statistically significant differences, there were clear trends in their proportional use. Unlike the 'most effective' managers of their intent, the 'least effective' instructors never followed up an apology with any further action to diffuse 161 any potential hostility as a result of their error or behavior. This suggests a lack of empathy, a lack of experience or a possible preoccupation with pursuing some preconceived role of how an instructor should behave. This explanation gains plausibility when the data also suggest that the 'least effective' managers of intent usually made excuses for their behavior in terms of having no alternative course of action, or legitimizing their action by referrence to their ascribed power role of instructor. Those instructors unsuccessful in establishing an initially adequate instructor/learner relationship fallaciously believed that an instructor has little need to be concerned about the feelings of his adult learners. This is in contrast to the instructors perceived as most helpful and well-intentioned who were the only ones to follow up an apology for a misunderstanding with an offer to do something concrete to repair the relationship. 6. Those adult education instructors who tend to be perceived as least helpful or least well-intentioned when establishing the instructor/learner relationship can be identified by the frequency with which they prepare their learners that an upcoming explanation is not meant to be seen as malicious or arbitary. Unexpectedly the instructors perceived as 'least well-intentioned' tended to use the Preparing category (category 3) significantly more than the 'most effective' group. As the most predominant pattern of communication of 162 the 'least effective' instructors was a straightforward Stating and Clarifying of intent, it is plausible to think that they were also very direct in preparing their learners that an upcoming statement was not to be interpreted as malicious or arbitrary in nature. This reinforces a previous argument that more effective managers of intent in the instructor/learner relationship should rely on a more indirect, varied and interactive verbal communication that includes Scanning, Regarding and Suggesting, as well as the use of Stating and Clarifying to diffuse possible misunderstandings. This would minimise the necessity to use the Preparing category. 7. Similarities exist between the 'most effective' and 'least effective' instructors in that: 1) the subgroups that were used for statistical analysis and comparison were homogeneous in the sense that there were no significant differences in their use of the category or sequence of utterances under examination; 2) the sequences of the most frequently paired utterances examined were not random and the coded utterances in a sequence could be maximally predicted by knowing the immediately preceding coded utterance; 3) the proportional use of categories and sequences of utterances remained stationary or constant throughout the first hour of the lesson. The direction of these findings provide a positive indication that the coded data collected and compared in this study met the criteria of order, stationarity and 163 homogeneity posited by Hawes and Foley, 1976 and Hewes 1977. This adds strength to the assumption that the data gathered by use of the coding technique is not so inconsistent or randomly arrayed that it can not be combined and used with confidence for matrix analysis. 8. From the methodology developed for the study, additional conclusions were drawn. First, the training of physically handicapped persons to perform transcribing and coding tasks utilizes a previously untapped source of effective assistance when patience, perserverence and intellectual skills are demanded. Second, the use of microcomputers to assist in the tabulating and analysis of coded sequential data is a convenient and cost effective alternative to the use of larger institutionally based computers. Lastly, in this developmental study, the encouragement of others to actively participate in its design and implementation allowed the reSearch objectives to be met while at the same time providing a useful learning opportunity for all participants. The nature of the methodology necessitated the identification and selection of a large number of support personnel to assist with data collection, transcribing and coding of data. Without access to a network of voluntary or low cost contract personnel, there are budgetary and logistical constraints to replicating a similar study. It was found that organisations and sheltered workshops for the 164 physically disabled provided a pool of determined and competent support personnel whose services had been previously neglected for short term contract work of this ‘nature. The use of a microcomputer in this study was a useful tool for the level of analysis required. However, when working with the sequential data and matrix designs, limitations were experienced in developing adequate programs within the capacity and range of the microcomputer. In particular, even a double density 48k data memory did not have the capacity to satisfactorily analyse patterns of coded verbal behavior beyond sequences of three utterances. Consistent with the ethical and philosophical assumptions of adult education, it was encouraged that participation in the study by learners, instructors, administrators and contracted personnel might also be a mutually beneficial and nonthreatening learning experience. Time invested in sharing a tangible form of reciprocity and feedback through face to face discussion, informal small group luncheons followed up by more formal inservice training sessions and written feedback maximized involvement and ensured access for future adult education researchers. RECOMMENDATIONS A review of the design of the study and the results of the field test indicate the need for further modification to the classification scheme, procedures for gaining access 165 to participants, identification and selection of support personnel, and data analysis. As well, implications for further research need to be considered. In view of this, the following recommendations are made. 3290mmendations for Modifications to Classification Scheme 1. Subject to further testing, the category of 'Requesting' (category 2) might be collapsed back into 'Scanning' (category 1). It was rarely used in the field test and caused some coder confusion in distinguishing between the categories of 'Requesting' and 'Suggesting'. 2. More critical redefinition of the rules for coding the categories of 'Explaining' (category 7) and 'Clarifying' (category 8) are needed. Most coder errors occurred unitizing the long sequences of utterances that were largely made up by these categories. 3. To avoid confusion as to whether an utterance was made in a positive, neutral or negative way there is a. need to include the added dimension of the relationship implied in the message to more realistically interpret an instructor's verbal management of intent. Future approaches should utilize both instructor and learner behaviors using the classification scheme of this study and then add a 'relationship' code to indicate the quality of the relationship inferred in the utterance. For example, the same sequence of three utterances 1-7-13 used by two different instructors might be more meaningful when 166 distinguished as l(+)-7(+)-13(+) for one instructor and l(-)-7(+)-13(-) for the other. Two available techniques for coding interaction from a relational perspective have been developed by Ellis (1976) and Donahue (1980). Each coding technique has established a high degree of reliability and validity in communication research studies. Recommendations for Gaining Access 1. Because of initial difficulties experienced in obtaining access to classrooms, it is recommended that special attention be given to approaching administrators, principals and instructors. It should be emphasized that their participation will be a mutually beneficial and nonthreatening experience. In the Australian climate of staff cutbacks, insecurity of tenure and a tradition of obtaining little or no benefit or feedback from studies previously conducted in the classroom, there seemed a suspicion and mistrust of research. It appeared to be often equated with some form of job evaluation and was initially seen as a threat to one's instructional competence or integrity. 2. Particular attention and time should be invested in face to face contacts, the writing of simple, courteous and straightforward correspondence explaining the research purpose and building into the research design some tangible form of reciprocity and feedback to all participants. 167 Recommendations for Identifying Suitable Support Personnel From the experience of this study, it is recommended that the resources of organizations or sheltered workshops for physically disabled persons be utilized in similar studies as they provide a pool of determined and competent support personnel. Recommendations for Data Analysis The existing computer programs used for data analysis of this study should be modified to include: 1) a conversion of frequency data to proportions; 2) display of selected computations into histograms; and 3) an additional program for computation of chi-square analyses and the partitioning of chi-square to allow identification of differences within and between groups of instructors. Recommendations for Further Research Differences observed between instructors 'most effective' and 'least effective' in their verbal management of intent suggest further inquiry is warranted. In terms of further research effort, it is suggested that the following recommendations be considered: 1. That the results of the field test be further substantiated by replicating the study with samples of instructors drawn from different adult education settings. 2. That both instructor and learner communication be coded using the classification scheme develdped for this study. It would be useful to add a relationship code to 168 help indicate the quality of the relationship implied in an utterance. 3. That the coding technique be utilized to examine how instructors verbally manage their intent during conflicts that occur at different times throughout the duration of a course. 4. That the conceptual framework and classification scheme be used as a starting point for further inquiry into other aspects of the communication of intent in adult education environments. For example, the examination-of: l) the nonverbal management of intent; 2) the congruence between an instructor's verbal and nonverbal management of intent; 3) the ability of instructors to recognize culturally different messages of intent or to simultaneously synthesize multiple messages of intent; 4) the learners' perceptions and management of intent; 5) the learners' perceptions of the intent of the verbal communication used on the screens of computer based learning systems or in written materials. 5. Further research is also suggested in response to the following hypotheses that have been generated by the field test: a) Instructors who are perceived as most helpful and well-intentioned when establishing the instructor/ learner relationship facilitate more verbal interaction throughout the lesson than those instructors who are 'least effective' managers of intent. 169 b) Instructors who are perceived to be more helpful and well-intentioned when establishing the instructor/ learner relationship verbally interact more frequently with their learners than those instructors who are 'least effective' managers of intent. c) The verbal sequences of three utterances made up of any combination with either Scanning (category 1) or Regarding (category 5) are used more frequently by 'most effective' managers of intent than those who are 'least effective'. This verbal pattern is a characteristic indicant of instructors 'most effective' in their management of intent. d) It is predictable that learners will respond more frequently to the assessing of intent by instructors who are perceived as more helpful and well-intentioned than those seen as 'least effective'. e) Instructors who are perceived as most helpful and well-intentioned when establishing the instructor/ learner relationship interact more frequently with their learners throughout the lesson by the use of: 1) Scanning (questioning to assess if learners have misunderstood or misinterpreted their instructor's intent), and 2) Regarding (reinforcement of learner responses through the use of positive praise or acknowledgement of the learners' contributions or efforts). f) Instructors who are perceived as most helpful and well-intentioned when establishing the instructor/ 170 learner relationship make more suggestions to their learners than those 'least effective' instructors who favour direct statements of intent. g) Instructors who are perceived as most helpful and well-intentioned when establishing the instructor/ learner relationship tend to follow up an apology with an offer to do something concrete to repair the relationship far more than those perceived as 'least effective' in their management of intent. h) Instructors who are perceived as the least helpful or well-intentioned when establishing the instructor/learner relationship tend to make more excuses for their action in terms of having no alternative course of action, or that their action was legitimate in their ascribed role of instructor than the 'most effective' managers of intent. i) Instructors who are perceived as the 'least helpful or well-intentioned' when establishing the instructor/learner realationship tend more frequently to prepare their learners that an upcoming explanation is not meant to be seen as malicious or arbitary in any way than those instructors 'most effective' in managing their intent. In its present stage of development, the conceptual framework, classification scheme and coding technique provide a means for empirical description to identify characteristic patterns of verbal communication used to 171 effectively manage intent in the instructor/learner relationship. It provides a conceptual screen through which the verbal communication used to manage intent may be viewed. APPENDIX A 172 APPENDIX A COMMUNICATION OF INTENT IN THE CLASSROOM The Environment. This is the setting in which the system operates. The environment for this study was the adult education evening college classrooms of a metropolitan region in Australia. The Boundaries of the System. The boundary of the system, the focus of this study, was the verbal communication that takes place between instructors and learners. The Suprasystem. The suprasystem is conceptualized as comprising all 0 the complex and interrelated systems, their subsystems and components that are concerned with the communication of intent in adult education classrooms. Together with the environment, the suprasystem can be thought of as an ecosystem. Each of the systems are dependent upon the other. If there is a change in one, there will be a compensatory change in another to restore the equilibrium of the larger ecosystem. If one system completely breaks down, it may result in the collapse of the ecology of the suprasystem. For example, if an instructor fails to verbally acknowledge the expertise of a trainee who clearly has superior knowledge in some subject under discussion, an irreparable conflict may occur or a learner may drop out of the course. The Suprasystem of all communication of intent in the classroom is made up of four smaller systems linked in a circular interpersonal process as shown in Figure 1. Within the Suprasystem, two of the systems deal with mental or psychological states of the instructor or learners, while the other two systems deal with the behavioral responses of the instructor or learners. In order to illustrate the link between these psychological states and behavioral responses when communicating intent, it was necessary to further develop the model shown in Figure 2. The System. The suprasystem for this study can be viewed as consisting of four separate yet interrelated systems. As shown in Figure 1, System 2, the instructor's management of intent, was the system under study. The Subsystem. System 2 consists of two subsystems. Subsystem A is the nonverbal communication subsystem and Subsystem B is the verbal communication subsystem. This study was limited to Subsystem B, the verbal communication used by an instructor to manage intent. The Components. Subsystem B is made up of three components. These components are the content of the verbal message (what is said), the quality of the relationship contained in the message (how one feels about what is said), and the control implied by the message (what one should do about what and how something is said). Only one component, the content of the verbal message in an instructor's communication was examined and measured in this study. Egyel of Analysis. The component 'content of message' of Subsystem BIIh System 2 was the focus of analysis. A classification system of thirteen categories and twelve combination categories was built for conducting the analysis. Unit of Analysis. A verbal utterance was the unit of analysis. Depending upon the content of the message, an utterance was either one word, a phrase or a complete sentence. 173 Tl-E EMIROMENI': Aduu education evening couege etuuwome 06 a memo- poutan aegton tn AMI/Laue BOUNDARIES TO THE SYSTEM: Communication in aduzt education etasenoome SUPRASYSTEMg w: venbat’. and nonvenbat commuteatéon 06 intent :— _ D . ‘ ”W“!- ’-- r “W SYSTEM :1 lemma 4 I ' SYSTEM 2: Inst/metal: A Penceptéon 06 Leanne/L Intent ; . Management 06 Intent . E V, 4 Instructor's intentions are " cannmicated by. . _ w SUBSYSTEM A (@538 E Instructor perceives . ' Nonverbal Verbal ‘ V ' 3 the learners' intent 5 Q Coummica- (5i. * J as either helpful, or r,“ tion {‘5 (T neutral , or disruptive/ .. “GWENTS E harmful * tent of Megg 3 'Quality of Relat- E . * icnship in Phssag ‘-. . ‘. 'Ccntrol implied b ,l . the Mess e {37‘ t ,I_ ; V ..EEE‘Q'CFK5 . ' SYSTEM 4; Leannw' ,3 _; SYSTEM 3: Leannw' : § Management 05 Intent ; Penceptéon 05 momma Intent‘ - Learners ' intentims are ‘ cannmicated by: .1 SUBSYSTEM A SUBSYSTEM B 3 Nonverbal Verbal » Camunica- Camunication tion learners perceive the instructor's intent as either helpful, or CW5 , -~ neutral, or disruptive/ oContent of Message ‘ harmful Quality of Relat- ,~ . - A ionship inMessagee ‘~"‘ . oControl implied by » l the lbseggg t a. 'g . a ' oc‘e , . ..- ...... a; to LEVEL OF ANALYSIS: System 2 and a component 06 Subcyetem B (i i) ml‘r 0F ANALYSIS: A venbat uttuance 06 component "Content 06 Message" FIGURE 1 COMMUNICATION OF INTENT: AN OPEN SYSTEMS MODEL . Inctnucton's SYSTEl'Il. M i a , INSTRUCTOR, S COIERT INTENT ‘Thought to Action to help, inpress or control .. we ‘ 33‘ ,.':23,- ...",; »..::x..'e.z_._. .22-u . 'eawzhgfii'i — . "LA‘."...... Evaluated ag inst motives intentional as Interpreted Congruency of H Processed aga' Matched agains ieeds, it own values ~~ngw is check interes —> STEM . Inst/weton'b SY 2' Behavtonat Ruponce INSTRUCTGR'S (NERT INTENT COVMJN I CATED unintentional * Processed aga M tched against, C ngruency of n ed t own values needs, interests nun is checked: 9 interpreted as intentional unintentional Evaluated ag inst motives VERBAL PESSAGES WESSAGES CWICATIW ._.. ;-- ~. .. -- ... ,. BY VERBAL NONVERBAL NESSAGES PESSAGES FIGURE 3 THE LINK BETWEEN FORMING INTENTIONS AND OVERTLY EXPRESSING INTENTIONS (INSTRUCTOR) Instructor's Mental State. An instructor's mental state is separated into: 1) perceptions of intent, and 2) covert intentions. The instructor acts simultaneously as observer to perceive messages of others' intent, and as actor to covertly plan the management of his own communication. The three categories describing the intent of learners is based on a nomenclature developed by Sarbaugh (1979, pp. 33 and 50). Instructor's Behavioral Response. From a mental state of intent, there is a praxis from thought to action. The instructor makes a behavioral response to his mental state and overtly communicates his intent using verbal or nonverbal messages. The lines that connect the instructor's mental state and behavioral response on the model are drawn to indicate the simultaneous process of praxis. As the distinction between thought and action is problematic, a broken line links the instructor's mental state and behavioral response. The Process of Integpreting Messages of Instructor Intent. Figure 4 illustrates the link between an instructor's verbal and nonverbal messages, and the learners' perceived intent of these messages. The distinction made between intentional and unintentional messages is important (MacKay, 1972) since even the most caring or competent instructor may unconsciously incorporate easily misinterpreted habits or idiosyncratic expressions into his communication (Ekman and Friesen, 1969). 176 INSTRUCTDR'S OVERT INTENT COMMUNICATED NDNVERBAL MESSAGES Processed aga n own values tched against n ds, interest ngruency of message is checked Interpreted intentional unintentional Evaluated ag inst motives THE PROCESS OF INTERPRETING MESSAGES 0F INSTRUCTOR INTENT Messages Processed Through Learner's Values. The learner processes all messages through his own code system, norms, beliefs and world view. If the instructor says something that is not congruent with the learner's beliefs and experience, there will be a lack of shared meaning and a likelihood of a misunderstanding occurring. The concept of processing or filtering ideas through a set of criteria based on experience and values is adapted from Knowles (1970) and Sarbaugh (1979). Messages Matched Against Needs and Interests. The message is then matched against the learner's own needs or interests and interpreted as intentional or unintentional. Congruency of Message is Checked. Different types of verbal and nonverbal messages are sent simultaneously by the instructor. Agreement between verbal and nonverbal messages is critical for verifying the credibility of a message (Graves and Robinson, 1974). The learner continually checks the congruency of what is said, how it is said, and the instructor's consistency in carrying out his intentions. 177 Messa es Interpreted and Evaluated Against Motivee. Having simultaneous y 1) processed the instructor's message against their own values, 2) decided if their interests are compatible with those of the instructor, and 3) continually checked for incongruence between the verbal and nonverbal messages, the learners then evaluate these pieces of information against the instructor's assumed motives. Judgments are made regarding the instructor's disposition towards the learners. That is, does he intend to help, ignore or dominate the learners. It is acknowledged that people differ in the degree to which they attribute intent (Maselli and Altrocchi, 1969). This interpretative process of the model highlights the close but distinct functional link between motives and intent. The Link Between Forming Intentions and nggunicating Intentions (Learners) The process describing the learner's mental state and behavioral response as shown in Figure 5, is identical to the one previously discussed for the instructor. In short, the learners' perceptions of the instructor's intent are matched with the learners' own covert intentions. There is a praxis from thought to action and overt messages of intent are communicated to the instructor. VERBAL NOWERBAL etohelariignore or' dominate NESSAGES WESSAGES - ' - 1; . . PERCEFVED INTENI OF INSTRUCTOR COMMUNICATION us‘Enfi ” i -' ~i. ( i <-——-—-— . 0F Thougm LEARNERS’ CCNERT INTENTS LEARNERS OVERT INTENTS A :9 to cooperate, impress, distract C Ion Ieannent' , LeannenA' SYSTEM u: Behavtonat Reeponc .::_____ SYSTEM 3' Mental State FIGURE 5 THE LINK BETWEEN FORMING INTENTIONS AND OVERTLY COMMUNICATING INTENTIONS (LEARNERS) The Process of Interpreting Messages of Learner Intent The final link from System 4 back to System 1 is an identical process to the one described from System 2 to System 3. That is, the instructor processes messages, selectively matches the data against his own needs, checks the congruency of the message as either intentional or unintentional and then evaluates this interpretation against the learners' assumed motives. This final link is illustrated in Figure 6. COWWJHCATHII OF LEARNERS' Ov/ERT lNTENTS rnnms 6 THE PROCESS OF INTERPRETING MESSAGES OF LEARNER INTENT The Completed Cycle The circular process that links the four systems in the model of communication of intent in the classroom is complete. In summary, an understanding of the interrelationships between the systems for communicating intent assists in visualizing that the instructor's verbal management of intent is only one system in a complex and dynamic process of communicating intent in the classroom. VERBAL COMMUNICATION OF INTENT IN THE CLASSROOM At the systems level, there are two distinct subsystems: the verbal subsystem and the nonverbal subsystem. As this study focuses on the verbal communication of intent, Figure 7 further illustrates how: 1) the instructor and learners verbally communicate their intent, and 2) the psychological processes they use to infer intent. Highlighted in this model is the instructor's need to perceive multiple verbal messages of intent from the learners while on the other hand, the learners need only to focus their attention on those verbal messages of intent sent by the instructor. That is, from the students' viewpoint, his relationship with the instructor is perceived as being on a one-to-one basis. However, from the instructor's viewpoint, he must process and interpret multiple messages of intent that may be sent simultaneously from a large number of learners. SYSTEM 1- WWW“ ____, SYSTEM 2: WWW”? ' Mentat State Behavtonat Response “ Thought INSTRUCTOR'S COVERT INTENT To INSTRUCTOR'S OVERT INTENT to help, impress or control ACf'on COMMUNICATED —_.> 1‘ , BY VERBAL MESSAGES Evaluated against motives Intentional or unintentional as Interpreted \ . Hoymcy of message’ig'liugliled * Procesflegamst own values/ atched agalns n eds, interests etched a ai eeds inIJ Jts , ’ Congruenc o messag checkec Plbcéggeg against ENE‘Va'Jéi l ‘ Interpreted as intentional or unintentional Evaluated against motives Fifi-19.942; ignore or: daninate’. VERBAL MESSAGES PERCEIVEDINTENI'OE INSTRUCTOR cowuNICATION 7- «I: I--~I --«'»;I. I I ' we/,.€’,3 . *fl/ ' 5 21 .’ 41 5 5: - £3 7 - 15 .13 / a 3 56 /2> 7 7 '7 1.7 M 8 I / // l3 /3 12> Ia // A: // 7 7 I l 7 z /3 5 5é 7 ll. 12. l3. X4. 15. 16. I7. 18. 19. 20. ”I‘vxoxc» DJ 6\ (Fwo‘ul w h) muummmb (hacked: j _2/ WM“ 13 Ié hwéumflm. 6:9 APPENDIX I 2(31 APPENDIX I SAMPLE PRINTOUT SEQUENCE FOR ONE INSTRUCTOR mocoAmAmA A0923: mega ~\A mocmumumu A0953: candou u ONIA commoa a CO Anon umAAA may cA AoAuauuch mcu An com: moquoooamo Ao mocmaomm one up Andeanua a mo Auwa mzozm oA:o«A on— .AoAuaAAch cm oA co>Am Awesoz u xAm A>ux An .nA An nA An nA mm c A n: o m n m A Aon AA m: 5A A: nA A nA A MA A mA A: m An nA m: nA A A A A nnAo A n A n m A m m m nA n nA A A n A n m m A A mnAm A: A n e A o n n n nA A nA 0 Am Am a: A nA m nA A AAAm n A: AA A o A Am An A nA A: A: a n A m: A: AA n c A sAAm m a A n: A: A m A n A n m A m m A m m A A: A oon o a n n m n m o m m A m m m on n m A n: 9A: A muAA A A n: A n n m A A m A n m A: m A: A: n m: m A AmAA m A: A o A n A: m A m n A A n m A: n A: n nA A omAA n m nA A nA n nA A nA o A m A A nA A nA A nA A A nmAA A: nA a m A A An nA m nA m m A. n Am An nA m: nA A A :AAo An m o A: nA m: A m nA n A A .A m A n m m A m A nmA: A n A A n m m A A A m nA m A m m A o m n A AAA: 3 A: m: A n n A: A n A n: n: A 9A 0A A A n n nA A .nAo A e a A AA A A A nA m m nA A mm nA m A A: 5A n: A suAm m A A m: n: m A m A A A A m m A n o m A A A uAAm n A: n: m o: n: n o o nA 0 AA 0 n m AA A sA n a A mAAa m m n A A AA m m A A: n m A n A n A A o A A AAAm n: A n A m A A m A A A n A o A A A m mm An A :AA: A m A AA 0 n m m A n A m A c: n: A m A A n A mAA: AA 0 c A n m m o n o m A Am a n A A A An A A :AA: a An Am An mA m nA Am m A o n A o A A n o A n A AAA: n A n: A n m A: A A: 9A A: n A: n A n A: n m A A AAAm A n: nA m Am 0: o n sA n A n n A m n: A o: n a A AAAm A A A A a n A n A n n A A: n m A n n An nA A sAAm Am nA on nA A A n A n A n A n n A: o: m nA n o A a Am J A nA A n A An m n A: A n An n n: sA n A 3A n A a An A m nA m m: A: 3A A A: nA n: nA c: m n nA :: on o: An A A An A n mm nA m nA Am nA m nA Am A nA Am n nA n: An nA n A a AA nA m m: m nA A: n n: nA A: An nA m m: nA n nA An nA A: A n An A: n n nA n: nA A n nA Am nA m nA m nA Am nA An nA m A : AA A A nA nA a An nA m m nA m m nA m m A n mA a: o A n AA eA n: nA A nA w m A A mA A nA A m m a. n nA a: n A A AA m A: m A: A m: m m m A m A: m: nA A: n: nA m A: nA A A AA .5... Iln"llulllll.ll-l"l-l"llIUI'II| ‘llllll' -IID|"IA'II'I"IIIIII' IIIII"|IIII-II-'IIIIII-'IIII' .III'I'IIIIIIlIIIIIl I .....II' -‘I-III"I|'III'I|-III|"I- -o IIIIII'"| II'l-l' S J 0 on. 90 rA u-t ~C' r. ..f‘ A? fl GI. H] " CI. 8 rd I.“ CD IN ~O n ~7 n '5 C" x 3 APPENDIX J 2()2 APPENDIX J SAMPLE STATISTICAL REPORT FOR ONE INSTRUCTOR and: cm cA cow: was Auomoumo Aacu moEAu Awuou one new Axuoooumo go on: mooucmoumm Auoooumo coAuchaeoo maqa Auooouao a go om: AacoAuuoaoAQ uca «ago» Auooouau Aocuoca cqu AacAuAaoozv m Ao AocAAozmv a mo om: AuocuaoAv ma .Aoucacmmv Nu .Axoonoacv HA .AouaEAuaomA u chwaoxuv ca .AaacoAacoucuc: - ocAmauxAA A .AocAAAAAaAuA o .AocAAaAmv A .AocAumooozmv w .Aocfluuaoomv n .Aocnuazmv c .AoCAuonuumv n .Aocfiuuoaaozv N .AocAccaomv A commo~ a mo Anon aaAAA ecu cA Acuoauumcd )9 com: moocououu: taboo up «much :oAuaoAuumaaqu xnm Acuoauumcu An moAuooouao coAuacAaeoo u A¢oouh¢ouu~¢oouh x A m: A om . new . :Ao 4cowmhmowmAmowmAmowmh<3 u 53» v: . m MA mm . . A n A WA ”Inn'hfllrl.'h""ph"".htol""ll"l"""“""uuuunulfluh . A . R . . A c A n n— . . . . . . . . . . wAm . . . . . . . . . . HA: . . . . . . . . . . HA . . . . . . . . . . AA“ . . . . . . . . . . AA: . . . A . . . . . . AA . . . . . . . . . . SAn . . A . . . . . . . EA. . . A A . . . . . A SA . . . . . . . . . . 0w . . . . . . . . . . 0c . . A . . . . . . . 0 . . . . . . . . . A mm . . . A . . . . . . me . A H OH . . . . . . m . . . N . . . . . A kn . . . o . . . . . . he . . m 5N . . . A . n A . . . . . . . . . . or . . . . . . . . . . a: . . . h . . . . . n c . . . . . . . . . . me . . . c . . . . . A m . . . . . . . . . . en . . . . . . . . . . e . . . A . . . . . . fin . . A F . . . . . . H: . . A GA . . . . . . n . . . . . . . . . . Hm . . . . . . . . . . Ho . . . . . . . . . . M . . . . . . . . . . Am . . . . . . . . . A Ac . . . A . . . . . . A .v—Amayc— .hfl. A: xAmA Agme:uo usyawowcmt AmAA¢> APPENDIX K 204 APPENDIX K mauuwum : : uncuaod u mama oeuvuaoom : = uocumua u mnn~ nocuooa : = oeuxufluaqu u nqam hocuwoa : : OCHqum u math ocfi>ufiuo~u = = ocdaaum u o-~ nocuaoa : : azauuaoom u Mann accuse; >5 uozo-ou oeuccuum n mama mocmuoua: ocfiuoouoam on» u m oocauouu: ocauoooum as» n m conned uo use: gnu“; ecu Cw me acuuauuocm so new: moccaumuu: banana vouooaom ocunomooam no ocwuoooua Auuoaom >03.oou£—v mouuomouao ecu mo xocoaoous oca meauuoomu uncquuq och u xmm «acosauoo "Hmm SAMPLE REPORT OF SEQUENCES OF THREE UTTERANCES m . . . . . u . . N . . N . . . u . . . . . . . . . q . . .m “ I I I I I d I I u I N c I I d I N I I I I I I I I I I n u h - nfl H~ . . . . . . . . N a u n . a . . n . . . . N . . . . . . um I I I I I I I I I e ' I ” I N I e c I I I I d I I I N I N “a M II ”a 4 I I I I I I ~ I a d I d I I I N ' I I I I I I I I I d d “m . . . . . . . . u m N a N . . . . N . . . . . . . . . . . .m n“ I a I I I I I I I I I I I d c I I I N n I I I I u I I I I I I ”a m I I I I I I I I I I I d I I I I d I I I I a I I I I I I ”a n" I. h N I I I I I I I I nu ‘ d N“ I I N I d I I d N o I I I I I I “m N I I I I I u I I o I n I n— I ~ H d u I I I n a“ I I I I I I “l o ' h I I I I I I I I I I m I N I I I d g I I u I d I I w N I I Mm «a . . . . . . . _ q . a a . . . a n . . . . . . . . . . . ”a n. u n I I I I I I I d I I I I n I I I N N I I I I d I I I I I N ”m N I I N I I I ~ I I ~ I n I I u I u I I I I I I I I I ~ I u n" l a mu can 8“ on 0 mm m hm h on o n t mm m Nm N am _ sue be me he 0: me en n¢ N¢ —¢ .Lomnouuvmngnuououwgmv x_m» ucw53uco ~m\N«\ON tn bzomwm ><3 I muzzh 3: ucwucu Ia uszmmmcm: _magw> APPENDIX L 205 APPENDIX L GUETZKOW'S PROCEDURES FOR ESTIMATING UNITIZING RELIABILITY Guetzkow's (1950) formula for calculating the reliability of the unitizing process is: U: 'U', the reliability of the unitizing process is expressed as the difference between coders by a percentage of the sum of the utterances obtained by each coder. '0' represents the number of units obtained by the first coder and '0%' is the number of units obtained by the second coder. '0' is always the larger number, regardless of the coder involved. As illustrated in the figure below, Guetzkow (1950) also provides a simple and accurate graphic procedure to calculate an index of coder accuracy at the .01 or .05 levels of significance ('o/n'), given that the percentage difference between coders ('U') is obtained from the segments of approximately one hundred utterances ('N'). fill-RELATIONSHIP OF CODER ACCURAC! .' '1'. rior MATFmAl cmwv)*¥-'*— .... ...... a .....- -..... "‘ -‘ -0! -————_L.‘ . .. - . 9. ' . . I . ... .- ._ ,.. --.—1-..._.I....._.T__...H-,.._-_.:i } .N much or masses . L f... .05. “rum moon... . ..1.—..:.. . ..—.'..-.—_, . .. . “...... i ...—..-”- -..1___-. .kmout o: emu. I , . --_,____. L. ..Accunacr - -:~-~+~w—+~—~~ N 'l ONIIRENCE i1 (‘h C) U IN A! n. and ‘ 5 o :o .oo :50 200 2 : 3 we N A? 5‘ LEVEL FIGURE RELATIONSHIP OF CODER ACCURACY TO AMOUNT 0? MATERIAL CODED (Guetzkow, 1950. p. 50) For example, a percentage difference between two coders is 10% in one segment of 100 utterances. Using the abscissa for N on the 1% level scale, the point on the graph is located for 'U' = .10, and N = 1 (one segment of 100 utterances). The point falls slightly above the .05 index line for coder accuracy. APPENDIX M 206 APPENDIX M GUETZKOW'S PROCEDURES FOR ESTIMATING CATEGORIZING RELIABILITY To ensure agreement among coders when assigning category codes to utterances, Guetzkow's (1950) procedure for estimating categorizing reliability was used. This was based on the assumption that dividing a transcript into utterances is independent of the subsequent assignment of a category code to each utterance. The procedure for calculating the reliability of categorizing requires the use of two formulae as shown in Figure l. t2 + 2n?1 + Jth + 2nP1)2 - 4(c2 + n)n(P1)2 EKULATIIHQ 1: I? = 2(t3 + n) -’ 2(t2 + n) . = K 2 _ 2 + l EQUATION 2. P K _ 1 p K _ 1 p K _ 1 p = the probability with which both coders correctly classify an utterance K = the number of categories in the classification scheme Pl= the proportion of agreement actually obtained between coders P = the theoretical proportion of 'n' utterances upon which two coders can agree a t-test (t = 2.58 or 1.96) used to estimate the lower limit of p at the 12 or 5 Z levels of significance n = the number of utterances - H II FIGURE 1 FORMULAE FOR ESTIMATING RELIABILITY OF CATEGORIZING (Guetzkow, l950. pp. 47-58) 207 A practical example of the use of Guetzkow's procedure is illustrated in Figure 2. EQUATION 1: (2.58)2 + (2 x 400)(.90) 2[(2.58)2 + 400] + 4(2.58)2 + <2x400)(.90)12 - 4[(2.58)= + 4001400(2.53T= " 2[<2.58)2 + 4001 P == .855 EQUATION 2 : The value P = .855 is substituted in equation 2. -_2.__—p+—1___ 35 - 1 35 - 1 35 .855 ~————— 2 35 - 1p '. p == .92 FIGURE 2 EXAMPLE OF THE USE OF FORMULAE FOR ESTIMATING RELIABILITY OF CATEGORIZING (Guetkow, 1950. pp. 47-58) The proportion of agreement between two coders (P) in this example was estimated as .90 on 400 utterances in a category set consisting of thity-five classifications. This proportion was calculated from an intercoder reliability worksheet used in the field study as shown in the Figure 3. To test the probability (p) with which the two coders correctly classified the segment of 400 utterances at the 1% level of significance, the values P =.90, t = 2.58, and n = 400 were substituted in equation 1. Finally, P = .855 was substituted in equation 2 to calculate the probability (p = .92) with which both coders correctly classify an utterance. Subject: 208 Dm-CDDER mmnm mm at!“ Inst. #255158, #: I ‘5' Week: 13 (baxAz‘:S;‘~y aura” :n;.!fl,4f_- Category: (:0ch CoderB Bl 'fZDisagree \OQVO‘UIbu-NH 585835555558 m I. -C) ~81; Proportion of coder agreement, P1 = 100 - 9.94 = 90.062 = 3 INTERCODER RELIABILITY WORKSHEET .90 209 Guetzkow (1950) also provides an accurate and simple graphic procedure for estimating categorizing reliability. Figures 4 and 5 can be used to obtain an estimate of the value of interaction reliability (p) used in the previous example. '52 O O .90 .80 .70 8‘ f—SF‘L ‘. W. '-T nonu- 50‘; // - . E555 _5/eer’”’””TT——_—Ii ‘1250 ' v 951090“? ‘ l. 90 Pioamweo Pmeoanon or AGREEMENT N'NUMBER or umrs com-:0 -3° ‘ ' Sb** ' 100' i i ‘ISO ‘5 ' 'zoo' 51565+ SCALE FOR N AT l7- LEVEL THL 0N. T ICAL O f ' ' ' 50 T ' Ibo : L ISOT SCALE FOR N AT 5% LEVEL FIGURE 4 RELATIONSHIP OF OBTAINED AGREEMENT TO THEORETICAL AGREEMENT (Guetzkow, 1950. p. 53) In this figure, the curve labelled P‘ = .90 is used. With n = 250+ on the abscissa for the 1% level scale, P is read on the ordinate as 22.85. In Figure 5 on the next page, the curve for K between 10 and 50 is approximated for thirty-five categories. with 'P' on the ordinate as .85, the value 'p' on the abscissa is found to be = .91. The graphical estimate of .91 is very close to the estimate of .92 computed from the two formulae and hence the graphic method appears adequate for training purposes. Guetzkow (1950, p.54) argues that for practical purposes, "experimenters need not have more than 150 units of qualitiative material classified by two coders to obtain stable estimates of the probability with which each unit is classified correctly." He concurs with Frick and Semmel (1978) that periodic checks of masses of up to 150 utterances are needed to ensure coding standards are maintained. 210 (Lona LIMIT) THEORETICAL PROPORTION OF AGRE I THEORETICAL CORRECTNESS OF CLASSIFICATION (Lawn LIMIT)» FIGURE 5 RELATIONSHIP OF THEORETICAL AGREEMENT TO THEORETICAL CORRECTNESS OF CLASSIFICATION (Guetzkow, 1950. p. 52) 211 Literaturegprecedents supporting Guetzkow's approach Frick and Semmel (1978) suggest a number of practical means for minimizing intercoder error. Coders should reach nearly perfect agreement on unambiguous examples with the expert coder before actual data collection. Coders should also be expected to reach agreement on ambiguities that might arise in coding. They emphasize: Criterion-related and intraobserver agreement measures have been recommended for both before and during a study, but these measures should not be used as evidence of observer agreement in the actual classroom. Rather, these are measures to assist an investigator in documenting adequacy of observational skills. The purpose of such efforts is to minimize the possibility that observers are primarily responsible for potentially unreliable observational data. Ellis (1977), argues that the critical issue is that only a coding scheme that can be learned and applied with consistency is reliable. He identifies four sources of poor reliability: The first three are research design such as inadequate data sampling; poor training of observers; and the coding scheme itself (e.g., too few categories, poor category definition). The fourth problem area is statistical tests for reliability. (Ellis, 1977. p. 12). Ellis (1977), Hirokawa (1980) and Donohue et a1. (1981), argue for the superiority and appropriateness of Guetzkow's estimate of reliability with sequential interaction data. Guetzkow's (1950) formulae are straight forward procedures for calculating unitizing and categorizing reliability coefficients. As well, they take into account the complexity of the coding scheme and the degree to which coders agree with each other. APPENDIX N 212 APPENDIX N PROCEDURES USED DURING THE PILOT STUDY Step 1: Selection of Participants for the Pilot Study Administrators, instructors and learners of two colleges were visited and permission was sought and granted to: 1) make an audio tape of the first class meeting in the term and 2) to allow ten to twelve minutes at the end of their lesson to administer a short learner response form. Times and dates were finalised for data collection. Step 2: Procedures to Protect the Anonymitygof Participants To ensure anonymity, names of institutions or participants were deleted from the recordings and assistants were requested not to discuss the contents of each tape. Step 3: Testing the Suitability of Various Types of Microcassette Recorders A microcassette recorder was selected that recorded sixty minutes per side of each tape, was small enough to fit snugly into an instructor's coat pocket, and was sensitive enough to pick up quality recordings of both the instructor and learners in a small group setting. Because of the unobtrusiveness of the microcassette recorder, it was found that instructors and learners were no longer aware of its presence after the first five minutes of the lesson. Procedures for audio-recording are described in Appendix E. Step 4: Transcribingand TypingTranscripts Professional transcribing services proved too expensive so an alternative procedure for handwriting original transcripts, independently checking them against the tape, typing and proof reading was developed. These procedures are described in Appendix F. Step 5: Establishing the Adequacy of a Student Response Form As shown in Appendix D, the fifty items of the Interview Rating Scale (Anderson and Anderson, 1962) are divided so that eighteen items refer to client behaviors and the remaining thirty-two items refer to counselor behavior. Clients are asked to indicate on a scale from 1 to 5, their perceptions of the counseling experience. Scores may range from 250 (ideal) to 50 (minimum). This rating scale reflects observable communication behaviors that are conceptually consistent with the verbal management of another's intent. That is, it is assumed that instructor's who are successful in establishing adequate rapport and 'ideal relationships' are also seen by their learners as well intentioned. Instructors who are less successful in establishing classroom rapport will be seen as being less well intentioned. It is further argued that the most highly rated factors for successful counseling of adults, the listening and communication skills, are not the sole domain of 'professional counselors'. As Hall (1977) and Riggs (1978) suggest, the personal side of giving counsel and 213 establishing rapport in a relationship is shared by instructors and counselors alike. This instrument has also been used successfully as a criterion measure of the quality of communication in a relationship by Correll (1955); Brams (1961) and by Riggs (1978). Given the conceptual consistency of this instrument, the rating scale was adapted for use in an adult education classroom by changing the nomenclature of 'counselor' to 'instructor'; 'interview' to 'class'; 'client' to 'learner' and the title 'Interview Rating Scale' to 'Communication Rating Scale'. Further minor modifications were made after discussions with small groups of college instructors and learners. In the pilot study, one hundred and fifty adult learners were asked to complete the modified Communication Rating Scale. Learners were encouraged to circle any word or phrase that appeared ambiguous and then invited to discuss any suggestions that might improve the instrument. The study revealed that: 1) adult learners were uncomfortable with 'rating' their instructor on their first meeting but agreed they could offer 'first impressions' that might change over time; 2) some older adult learners had difficulty with the small type on the scale and found the mass of numbered boxes and the fifty questions a little overwhelming late in the evening; 3) the average time to complete the rating scale was between ten to twelve minutes; and 4) the scores recorded for eight instructors clearly distinguished those ranked high by learners as effective in establishing rapport and those ranked low. For example, in a possible range of 50 to 250, scores ranged from 147 to 250, and the highest mean score was 239 compared to the lowest mean score of 192.96. Based on these findings, subsequent discussions with instructors representative of the sample used in the field test and review by a panel of judges, final modifications were made to the instrument. Care was taken to substitute language that was more easily understood by Australian learners and which did not detract from the questions' semantic accuracy. The major modifications included: 1) A-Change of the instrument's title from 'Communication Rating Scale' to 'First Impressions'; 2) Use of larger, bolder type face and a reduction in the number of questions from fifty items to forty items. The criterion used for omitting these items was based upon the item analysis of the scale undertaken by Correll (1955). He found eight of the fifty items were of little value in differentiating between 'good' and ‘poor' communicators. Finally, a test of the completed instrument with a small group of learners, similar to those who would use the instrument in the field test, showed the rating scale took no longer than five to six minutes to complete. All directions and questions were easily understood and acceptable to the learners. 214 Step 6: EstablishingAcceptable Intercoder Reliability Coefficients using the Technique Typed transcripts of the audio tapes of six instructors were coded and analyzed. A colleague was trained to use a draft revision of the coding technique. Selected sections of the transcripts were coded independently and compared for intercoder agreement with the researcher. ‘ Intercoder agreement was assessed by the use of a procedure developed by Emmer and Millet (1970). This formula is: Coder agreement = l - Coder A - Coder B Coder A + Coder B. The formula uses the total tabulation of all utterances identified by each coder. 'A' is always the larger total, regardless of the coder involved. Intercoder reliability coefficients falling between .50 and .80 were established. Frick and Semmel (1978) and Bradley et a1 (1978) suggest reliability coefficients falling around .80 are practical and adequate figures to aim for. A comparison of coded transcripts of the instructor rated as the most effective at establishing rapport was compared with that of the least effective instructor. Differences were found in the frequency of categories and utterance sequences used with particular categories. For example, the most effective instructor used the categories of 'requesting', 'suggesting' and 'Clarifying' in a ratio of 4:1 to those used by the least effective instructor. Step 7: ThankiggParticipants in the Pilot Study On completion of the pilot test, letters of thanks were sent to all administrators and instructors who participated in the pilot study. At the end of the study, a follow up letter of thanks, a copy of the final classification scheme and a summary of the study's findings were also sent to all major participants in the pilot study. Samples of the types of letters sent are contained in this Appendix. 215 CORRESPONDENCE (Add-Less I (Date) (Name and addaus) Dear (Name) I wish to thank you for participating as a member of the (pilot study, 6iz£d (CAI OA'aeac£ion panczl in the research to design, develop and field test a technique to measure adult education instructors' verbal management of intent. Your assistance and cooperation have made a substantial contribution to the success of the overall research endeavor. For your information, I have enclosed a brief summary of my research findings, the final revision of the Management of Intent Classification and Code System, the First Impressions Student Response Form. I trust they may be of some interest to you in understanding more about how instructors tend to verbally manage their intent, and how some students perceive their instructor's intent. Again, thank you for your help. If you wish further information, please feel free to contact me at any time. I will be happy to share other aspects of the research or respond to any specific inquiries. Regards, mal- John Brown-Parker APPENDIX 0 216 APPENDIX 0 PROCEDURES FOR SELECTING PANELISTS Step 1: ContactingProspective Panelists Prospective participants were contacted first by telephone and second by personal visit to provide details of the research project, the intended function of the panel members, a timeline for completing the task and a request for the candidate's participation. Step 2: Selectingthe Panel Members Ten candidates were approached and six participants were chosen. From the four panelists at Michigan State University, two panelists had previously conducted extensive research in classroom assessment techniques. The third panelist was selected for his broad background in both adult education, communication and familiarity with systems theory. The fourth panelist was selected for his extensive expertise in communication systems theory and the analysis of sequential interaction data. 0f the two Australian panelists, one was selected because of his interest in legal definitions of intent as well as his experience in teaching to diverse groups of adults. The other panelist was selected for his competence as an adult education practitioner, an expertise in systems analysis and a wide knowledge of adult education practices throughout Australia. Step 3: Tasks for the Panelists Each participant received by mail, copies of the second draft of the technique and was asked to complete the feedback forms and share comments on a cassette tape as outlined at the end of this appendix. Step 4: Collection of Feedback Forms and Followup The completed forms and tapes were collected and a letter of thanks was sent to each panelist. After completion of the study, a further letter of thanks was sent to each panelist along with copies of the final revision of the classification scheme and a summary of the general research findings. PURPOSE : DIRECTIONS: 217 FEEDBACK FORMS FOR REACTION PANEL FEEIBACK FOR“! Fm CONTENT VALIDATION OF TI-E SCI'IB‘IA'DEVELOPED T0 MEASIRE AN INSTRICTG? S VERBAL MANAGHENT OF INTENT ‘l‘hepmpoeeofthisformisto solicityourreactims to qmstimsregardingthescimathattnebemdeveloped toclaseifytheverbalbeluviorsusedbyminetrmtorto mnagea learners perceptions of mutate: intent. waillbeaekedtorespmdtospecificqmstimsabwt eaehofthefou-tem classificatiauueedintheeclm. meillbeaekedtorespmdtoamnberofgerural questims relative totheoverall straigthsandmaknees- esoftheschem. Aswell. youwill be invited tomakeanygmeral auggestims the plausibility of the cameptual framrk that underpins the logic of the sebum. YURCANDIDREPQISESAREW. YOIRGIITICIS'BAND mmmmcrvmcwsmmmsmnmmnm FDIALREVISICNG‘THESCHEI’A. SIEEn1"IflIflLJA8IZAIBILIflIH_IHE_EDJILJ!!1£§1AIEQ.JIEEEEIS l. Pleaeereadtheattached extract franSectimloftheresearch proposal. 'Tdaxtifying the Problen." 2. Please read the attached copy of Sectim 3. Part A "A Conceptual Wk for Exaninim Magmt of Intmt." 3. Pleaeereadtlnsctmformmstructor's erbalManagenent of Intuit locatedmthenexttmpagesofthisfonn. m-W 1. Please refer to the questime m the attaclud blue colored form titled ‘Claesificatim Analysis". 2. Please read each classificatim and respcrd to the questim/s aekedebwttlntparticularclassificatiminthespaceprovided cm tin blue'Cleeaificatim Analysie'form. 3. Pleaee feel free topnote any additimal'camlnts in the space providedmtheblmfoem . our: ifywprefer. mgimte cements against the classificatime on the next two pages. Twm to the back page 5011 gunfire/L Meet/Lona "" DIRECTIONS: (gunmen) 218 5313.— W Pleasereadthegeneral questimsmthe llowform titled " General Questions for Tape-Reno spmses". Please note dam you: tespmse to these qmstims. Using the cassette tape provided. youuny wish to expmd or further clarify your written omits or suggestims to these generalqmstims. Ifthisisthecase. muldyoufirst idmtifythequestim you are to snsuer by quoting its number. and restating the questim itself. Any additicnal writtm or oral calm-Its regarding the appropriatmess of the sober: or its apporting conceptual scrum would be welcane. Sisal-W 2. Please place all materials in the envelope provided and 131“! : Mrs. Judi Bram-Parker 1533-? Spartan Village mane: (517) 355-2913 East lansing MI 48823 Mrs. Brow-Parker will call and pick up the antelope. WMWSOfln-IECR mmnsmnus ***** PAMOFM‘IOWVAIMTHEW «unseen-ream MW'SVERBAL WOFINIINT PART A: PART B: 221.9 GENERAL “EST IONS F0? TAPE'RECORDED RESPONSES THESCHB‘IA meteredreestrmgthsofthisschena? What are done weaknesses of this schem ? htadditimalchmgescmymrecmxdtoisprovetheoverall qmlityofthisschenn? TIECWEPTUAL WK that are three strmgths of the emcepmal fimrk ? mutatetlreewealmessesofthecmceptmlfi’mk? headditimalchangescanyourecmudtoiuprovetheoverall qmlityofthecmceptualfrmworkofthisstudy? "‘“ A cassette tape has been Included in «this package should "’“ you «ash «to expand 06 6%“ may you Men comments on Augguuons ’ 220 (SampKeI oz mo> w owcmsu o ome 30% odoDS 02 no» wanton ome cofiumouwanmmao 3.: no 8:365 at 88 ~ mcuocmunueoco acumen oz nor moon panama ecu on p oumuucouncm Henna oz no» coaumouwgmnmao may mH oz 8» c. :53 Boss 8523 so no 8:383. so a mnymvenxv oz as» p awcmzo m oxme_ons wasp: 02 an» «enema axes opauoouwunumdo each mo counoaocu as» soon s. 95953.85 has: 02 no» menu madame ecu on s 3%.? 33 oz 8» Encompass so A oz 8» a €83 Boss .6523 ecu wo cowucwuumwo use on wunuénxv hmmzm w_m>44u_¢