THE EFFECTS OF SYSTEMATIC MANIPULATION 0F VARIABLES IN THE LECTURE FORMAT 0N COLLEGE STUDENTS' COGNITIVE ACHIEVEMENT Dissertation for the Degree of Ph. D. MICHIGAN STATE UNIVERSITY MICHAEL LINTON BUDD 1976 This is to certify that the thesis entitled The Effects of Systemmatic Manipulation of Selected Variables in the Lecture Format on College Students' Cognitive Recall presented by Michael Linton Budd has been accepted towards fulfillment of the requirements for Ph.D. degree in Higher Education Major professor ABSTRACT THE EFFECTS OF SYSTEMATIC MANIPULATION OF VARIABLES IN THE LECTURE FORMAT ON COLLEGE STUDENTS' COGNITIVE ACHIEVEMENT By Michael Linton Budd The purpose of this study was to investigate whether lecture effectiveness could be enhanced by manipulating variables within the lecture format of instruction. The seventy-four students, basically college freshmen, re- cruited from a number of introductory psychology classes at a large midwestern university were divided into five groups: nominal lecture (nominal control), N = 2T; paced lecture (paced control), N = 6; maxi-paced lecture, N = 18; maxi-paced lecture with practice, N = 17; maxi-paced lecture with practice and feedback, N = 12. The first two control groups were given a non-redundant, relatively monotonic lec- ture by a graduate assistant introduced as an experimenter. The latter three maxi groups received the same lecture content from a nationally recognized educator and lecturer, possessing a more ex- citing and redundant style of presentation. The "expert lecturer" was introduced as a MSU professor and expert in the field of Magic Squares to enhance his reputation in the eyes of the students. In addition, the "expert" gave an interesting history of the discovery Michael Linton Budd and mathematical development of magic squares to enhance student in- terest. In the paced lecture, as contrasted to the nominal lecture, the lecturer presented information, i.e., a concept or rule, followed by a thirty-second pause during which the student was directed to think about what he/she had just been given in the lecture. The maxi-paced lecture with practice group was given the same paced lec- ture and pauses, but during the pauses were told to practice, in a workbook provided, using the information given in the lecture. The maxi-paced lecture with practice and feedback group, again, received the paced lecture but with the inclusion of both practice and feed- back of correct answers during the thirty-second pause. Each paced lecture was thirty minutes in length. Each of the maxi-paced lec- tures were controlled for consistency in repetition of information, in the use of advanced organizers, and in summary statements. No note taking, talking or questions were allowed in any of the groups. The dependent variable was the score received by students on an instrument designed to measure rule learning. It was hypothesized that: l. Students exposed to any paced lecture would show better performance as measured by a cognitive test score than students in the nominal lecture group; 2. The maxi-paced lecture presented by a "recognized expert" would further improve student performance over the paced control presented by the graduate student; 3. Within the maxi groups, the lecture with practice and feedback would increase students' score more so than the lecture Michael Linton Budd with practice only, which in turn would enhance cognitive test score more so than the maxi-paced lecture. It was found with reference to the above hypotheses, that: 1. Student performance can be enhanced, through the variable pacing, over student performance from a nominal lecture. 2. The enhancement effect of an “expert" lecturer and his accompanying style on student performance is not significantly greater than that of a graduate student trained for the task. Appar- ently, pacing is able to overcome the effects of teaching style. 3. No significant differences were found between the maxi lectures of the "expert"--i.e., the maxi-paced, the maxi-paced with practice, and the maxi-paced with practice and feedback--as measured by cognitive test performance. Of import is the apparent lack of effect of either practice or practice with feedback, especially when both practice and feed- back are generally held to be of major benefit to learning. Since the error rate was only 9%, the effect of feedback on incorrect responses of these groups' scores was minimal. Subsequent analysis of correct and incorrect responses of the "maxi with practice only" and the "maxi with practice and feedback" groups, for the effects of feedback, suggests that feedback to an incorrect response is more important than to a correct response. In the final analysis, the implications of this study for the practitioner stresses the import of pacing. Pacing forces the in- structor to stop, giving both himself and his students time to think (discuss, ask questions, practice, take notes, etc.)--time for Michael Linton Budd cognitive processing to occur. Pacing does enhance cognitive out- come; nevertheless, it must not be construed as a cure-all for the lecture. Though it seems at least a start in the proper direction. THE EFFECTS OF SYSTEMATIC MANIPULATION 0F VARIABLES IN THE LECTURE FORMAT ON COLLEGE STUDENTS' COGNITIVE ACHIEVEMENT By Michael Linton Budd A DISSERTATION Submitted to Michigan State University in partial fulfillment of the requirements for the degree of DOCTOR OF PHILOSOPHY Department of Administration and Higher Education 1976 G) Copyright by MICHAEL L INTON BUDD I976 DEDICATED to my wife, Lynn, and to my mother and father. ii ACKNOWLEDGEMENTS I wish to express my sincere appreciation to the following, without whose assistance, encouragement and inspiration this study would have been all but impossible: Jack Maatsch, Ph.D., for offering an opportunity for in- volvement in V.I.M.; for seeing to it that I followed through on commitments, i.e., for administering swift kicks at the proper time and place. Thomas Holmes, all-but Ph.D., for his invaluable assistance in the early-going; for computer analysis and peer-instruction in the truest sense. Dennis Hoban, Ed.D., for continued encouragement, at low- tides, throughout; for watching out for me; for being the other committee member. Howard Teitelbaum, Ph.D., director of the thesis, for making sure the study was Eight; for aid and direction at points of hyster- ical confusion--but for whom the analysis would not be complete, nor the program of study. Shirley Ballantine, secretary extraordinaire, for voluntarily typing and editing through the rough drafts, without much complaint; for taking as well as dishing it out. These individuals are the hard-core behind the study; they have given freely of themselves and their time, that it might be completed. Significant others for the duration: Ann Olmsted, Ph.D., for her wisdom, guidance, and occasional swift kick. William Sweet- land, Ph.D., committee director, for the wisdom and willingness to not direct too narrowly. All are without exception truly uncommon people; the best of teachers. I count myself fortunate to have been able to share their time. iv Page_ ACKNOWLEDGEMENTS ........................ iii LIST OF FIGURES ......................... viii LIST OF TABLES ......................... ix Chapter I. AN INTRODUCTION ..................... l History/Background of VIM ............... 3 Purpose of Study ................... 6 Research Questions and Hypotheses ........... 7 Methodology ...................... 8 Study Definitions ................... 9 Study Limitations ................... l0 Overview of Study ................... ll II. REVIEW OF LITERATURE ................. 12 Introduction ..................... l2 Enthusiasm of Lecturer ................ 12 Organization of the Lecture .............. 14 Note Taking ...................... lS Pacing ........................ 17 Practice and Feedback ................. 2T Summary ........................ 24 III. STUDY METHODOLOGY AND PROCEDURE ............. 26 Introduction ..................... 26 Population ...................... 26 Population and Sample Selection ........... 26 Procedures ...................... 27 Experimental Treatments ............... 27 Treatment Group Description ............. 28 Experimental Hypotheses ................ 3l Instrumentation .................... 32 Measures ...................... 32 TABLE OF CONTENTS Analysis of Instructional Task ........... 32 Test Format ..................... 36 Reliability ..................... 38 Data Analysis Techniques ............... 39 Summary ........................ 42 IV. ANALYSIS OF DATA .................... 44 Introduction ..................... 44 Experimental Hypotheses ................ 44 Research Hypothesis l ................ 45 Research Hypothesis 2 ................ 45 Research Hypothesis 3 ................ 46 Formulation of the Dependent Variables ........ 47 Formulation of Groups ................. 50 Analysis ....................... 50 Summary ........................ 59 V. SUMMARY; CONCLUSION AND DISCUSSION; IMPLICATIONS . . . . 60 Summary ........................ 60 Study Design .................... 60 Study Results .................... 62 Conclusion and Discussion ............... 63 Introduction .................... 63 Implications ..................... 68 For Research .................... 68 For Practitioners .................. 7l REFERENCES CITED ........................ 74 APPENDICES ........................... 79 Appendix A: Lecture Script ................. 80 Appendix B: A History of Magic Squares ............ 85 Appendix C: Test Instrument ................. 89 vi LIST OF FIGURES Figure Page 3.1 Group Membership ..................... 39 3.2 Group Membership ..................... 40 3.3 Group Membership ..................... 4l 4.l Effects of practice with and without feedback in jggggrggt_responses .................. 57 4.2 Effects of practice with and without feedback on jflggrrggt_responses .................. 57 4.3 Effects of practice with and without feedback on correct responses ................... 58 4.4 Effects of practice with and without feedback on correct responses ................... 58 vii Table 3.l 3-2 khan-b (II-DOOM LIST OF TABLES Experimental Lecture Treatments ............. Reliability coefficients for subbateries ......... Correlation of all dependent variables .......... Correlation of RULE variables .............. Correlation of CONCEPT variables ............. Correlation between RULE variables and CONCEPT variables. Treatment groups, X RULE score, standard deviations, minimum and maximum RULE scores ............ Analysis of variance of treatment groups on the vari- able RULES ...................... Scheffe post-hoe analysis contrasting the nominal con- trol with the paced lecture groups using total RULE score as the dependent variable ............ Scheffe post-hoc analysis contrasting the "expert" lecturer's groups with the graduate student's group using total RULE score as the dependent variable. . . . Scheffe post-hoc analysis contrasting the "expert's" lecture groups using the total RULE score as the dependent variable .................. Treatment groups, Y'CONCEPT score, standard devia- tions, minimum and maximum CONCEPT scores ....... Analysis of variance of treatment groups on the variable CONCEPTS ................... viii Page 28 38 48 ' 48 48 49 50 52 55 70 7O Cave, Normanne, ne stertas...nodus tuus novus quam tu ipse clarior est. R. Stanley Nadel ix CHAPTER I AN INTRODUCTION Lecture, an instructional method in which an actively par- ticipating instructor makes a presentation to a passive learner, accounts for slightly more than 55% of college and university facul- ties' instructional time (Hale Report, 1964). Instructors select this method of instruction despite a long history of research indi- cating its limited effectiveness in comparison to other methods.1 It has been suggested and demonstrated that lecture is an inappro— priate technique when: - the instructional objective involves the application of psychomotor skills; - the information acquired must be available through long- term retention; - the content material is fairly complex and detailed; 1A number of authors have reviewed and summarized the lit- erature on lecture versus other instructional methods. The follow4 ing are suggested as the more inclusive and are listed for the bene- fit of the reader, should he/she wish to pursue the matter: Bligh (T972), Costin (1972), McKeachie (T963) and McLeish (l968). Lecture, in some cases, is found to do as well as but not better than other methods of instruction. However, because of the questionable methodology and paucity of clarity in presentation of studies, most studies to date leave much to be desired and much to be imagined (McLeish, T968). - learner participation in the learning activity is crucial to the achievement of the objective (Verner and Dickenson, 1967).2 Recently, Maatsch et al. (1975), while showing methods of instruc- tion do in fact make a difference in student learning, with content of instruction held constant, has also shown lecture to be one of the least effective methods of instruction. Nevertheless, one is struck by the persistence in the use of lecture for those very purposes for which it has been found to be inappropriate. Possibly, this persistence is attributable to the fact that lecture is a too readily available method of instruction; possibly, as Holcomb (T975) suggests, to the fact that teachers tend to teach in the manner in which they themselves were taught or to a human vanity which somehow impels instructors to impress students by showing them how much they have learned. Because of this persistence, for whatever reason, this study proposes to investigate ways of im- proving lecture as an instructional method. 2Typically it is claimed that lecture is capable of achiev- ing three kinds of objectives (Paulsen, 1906; McLeish, 1968; Bligh, 1972; Holcomb, 1973): (l) the acquisition of information; (2) the promotion of thought; (3) changes in attitude; but certainly not those mentioned by Verner and Dickenson (1967). However, Bligh (1972), in an exhaustive compilation and subsequent and accessible tabling of information found that: (l) with the possible exception of programmed learning, the lecture is as effective as any other method of trans- mitting information, but no more effective; (2) most lectures are not as effective as more active methods of instruction for the promotion of thought; (3) changing student attitudes should not normally be the major objective of lecture. Though a given lecturer may be capable of accomplishing a vast number of different kinds of objectives, such effectiveness is understood ggt_to be generalizable to all lecturers. It is important to remain cognizant, too, that though lectures are criticized for poor preparation, or as repetition of standard textbooks History/Background of VIM This study is part of ongoing research in Variables in In- structional Methods (VIM), out of the Office of Medical Education Research and Development (OMERAD), at Michigan State University (MSU). As such, background on the origin and purpose of VIM is pertinent. In the summer of 1973, Maatsch, Holmes, and Hoban formed the nucleus of an ongoing symposia (in the truest, classical sense).3 From these meetings emerged a recognized, mutual frustration with the literature on the psychology of instruction. The literature did not provide any clear agreement on variables effecting instruction as measured by performance outcomes. Seven empirical questions evolved: 1. With content of instruction held constant, do methods of instruction make a difference in student learning? 2. Do methods of instruction differentially effect perfor- mance on various test formats? In other words, will a lecture en- hance performance on multiple choice questions but produce poorer scores on problem solving relative to other methods? 3. Do methods differentially affect long-term retention of material learned? 4. If methods make a difference, which independent variables inherent in those methods produce the difference? perhaps it is the fault of the lecturers who conmit the errors rather than the lecture method, itself. 3J.D. Hoban, Ed.D., Assistant Professor, OMERAD; T.F. Holmes, doctoral candidate, OMERAD; J.L. Maatsch, Ph.D., Professor, OMERAD. 5. Can we increase the effectiveness or the efficiency of any method by manipulating the key variables inherent in that method? In short, can we design more cost-effective methods? 6. Which individual difference variables affect learning outcomes and how do they interact with methods? In other words, are there aptitude-treatment interactions? 7. Finally, how important are-method variables and individ- ual difference variables relative to each other (Maatsch, 1975)? In order to test these questions and others, a mathematical puzzle, Magic Squares, was chosen as the cognitive task. This par- ticular task was selected, as it fulfilled a number of important re- quirements: 1. It was possible to control for entry level knowledge; 2. The complete task could be taught in ten to thirty min- utes; 3. The task lent itself to all of the different common in- structional methods (i.e., simulation, observation, seminar, lecture, programmed instruction, and reading); 4. Comprehension and retention of the (three) concepts and (six) §g1g§_involved in construction of a magic square could be directly tested; 5. Student ability to apply_these concepts and rules in prob- lem solving test formats (Maatsch, 1975). Since that summer of '73 over four hundred subjects, predom- inantly college sophomores from several classes of introductory psy- chology classes at Michigan State University, have been involved in VIM. As mentioned earlier, Maatsch found instructional methods do make a difference in student leaning. Of the different, common in- structional methods, simulation and lecture were found to be, respec- tively, the best and poorest methods. This difference may be ex- plained by variables inherent to the simulation method of instruction but not to lecture: Simulation...(is) paced by the student participant... He (has) time to observe, process, and respond before in- struction (continues). Furthermore, if he (responds) in- correctly, he (receives) feedback from the instructor and (is) coached until he (responds) correctly. (Maatsch, 1975). It is suggested that pacing of instructional input, allowing time for processing, either overtly or covertly, and feedback, with subsequent correction, for incorrect responses, are the major vari- ables which in fact cause the two methods to produce differences in outcome. If such is the case, inclusion of one or more of these vari- ables in the lecture method might enhance its overall effectiveness. This study proposes to investigate ways of improving lecture as an instructional method. If one considers the essential justifi- cation for the investment of 55% of faculties' instructional time (if not more), that is, the argument that lecture is the most econ— omical of instructional methods in terms of staff time, than this study can indeed make a contribution in seeking ways of improving the effectiveness of lecture.4 4Other justifications for the use of lecture are want to in- clude the following considerations: (a) students are too immature to learn effectively by reading; (b) lecture opens up the subject for students; (c) complex materials can be explained orally and economi- cally; the most recent materials or ideas, not yet available in Purpose of Study The purpose of this study is to test whether, by manipulating key variables within the lecture format of instruction, to more ac- tively involve students, the effectiveness of the lecture method can be improved. If, in fact, the effectiveness of lecture can be en- hanced through the manipulation of key variables, there is no reason to believe that this improved effectiveness of the lecture method should be applicable only for magic squares, the cognitive task of VIM.5 Rather an enhanced lecture method may be appropriately appli- cable to other cognitive tasks, even to those tasks for which lecture has been found inappropriate. textbooks, can be presented; (d) the lecture gives a framework, an outline, a critical point of view; it provides aesthetic pleasure and communicates enthusiasm; (e) lecture is better prepared, more pro- found and better thought out than inpromptu answers given during dis- cussions; (f) it can reach large numbers and bring the student into contact with many points of view (Hale Report, 1964); (9) lecture is flexible-~easily adapted to the audience, subject matter, available time and equipment; (h) lecture allows for spontaneity and is easily adaptable to the teacher's schedule--he/she cannot alwa 5 plan ahead sufficiently to have materials reproduced; (i soc1a facilitation" may occur: lectures make many people simultaneously concern themselves with the same ideas, and each member of the audience may be stimu- lated by an awareness that many others are responding to the lecturer at the same moment (Gage, 1969); (j) (and possibly the most compell- ing) at least the professor learns from having to prepare and deliver his courses, year after year--it clears his mind and is a half-way house to publication (Jones, 1923; Hale Report, 1964). It is in- teresting to note the emphasis of the above justifications: essen- tially other than student or outcome directed; basically instructor or administratively based. 5Through the cognitive task magic squares is basically the learning of rules, nevertheless, as Gagne (1970) readily points out, rules do in fact constitute the greater bulk of what is learned in school--elementary through higher education, including professional education. Admittedly, the rules may differ significantly in degree of complexity. Research Questions and Hypotheses In order to carry out the study's purpose, as noted above, the following research questions were considered: 1. Is it possible through the manipulation of variables, within the lecture format, to enhance the effectiveness of the lec- ture? 2. More specifically, will the inclusion of such variables as pacing, practice, feedback (the major variables affecting differ- ences in outcome between simulation and lecture) enhance effective- ness? 3. Will variations in lecturer style--i.e., in apparent en- thusiasm, humor, expressiveness, personality, in the delivery itself --make a difference in lecture effectiveness? 4. If so, will these variations in the lecture format order themselves with respect to degrees of effectiveness? More specifically, the study's hypotheses are as follows: 1. Students exposed to a pppgg_1ecture will perform better than students exposed to a nominal lecture (VIM) on a cognitive test. 2. Lecturer §3y1g_will make a difference in student perfor- mance. That is, a "recognized expert," who incorporates the varia- tions in three above, presenting the paced lecture will improve stu- dent performance over a paced lecture presented by a graduate student (VIM). 3. A paced lecture with practice will further improve student performance over the paced lecture on a cognitive test. 4. A paced lecture with practice and feedback will further improve student performance over the paced lecture with practice only on a cognitive test. Methodology This research study became possible through participation in the present and ongoing research project, being conducted in OMERAD at MSU: Variables in Instructional Methods (VIM). This research study was dependent upon methodology used in VIM. Previous findings, specifically with respect to the lecture method of instruction, serves as a point of departure. As such, the hypotheses listed above were tested within the experimental confines of the larger study, VIM. The sample for this study was forty-seven introductory psy- chology class students, predominantly sophomores, at Michigan State University. Because of a desire to present an omnibus picture of lec- ture, two previous groups of students (N = 27) from earlier VIM ex- periments were included for a combined total of 74. Chapter III will expand upon this inclusion. Assignment to treatment was essentially random as subjects signed up for experiments without knowledge of the treatment to be given. The hypotheses listed above concerning pacing, practice, feed- back and lecturer style were tested and the data analyzed using a one-way analysis of variance, and a Scheffé post-hoc analysis of the significance of obtained differences in treatments. Study Definitions The following definitions are essential to insure a common understanding in the concept feedback lecture pacing practice rule style following discussions. the term used to include a definition or rela- tion which needs to exist between components for a square to be recpgnized as magical. instructor's response which affirms a correct response or disconfirms or disrupts belief in an incorrect response by supplying a correct response, during the instructional process. an instructional method which incorporates a one-way flow of information from an actively participating instructor to a passive learner. a pause or wait-time, interspersed in the in- structional format, for the purpose of, either overt or covert, cognitive processing of just presented information. rehearsal or use of information presented during instruction. the term used to indicate instructions for the correct placing of a series of numbers in the cells of a square so as to form a magic square. observable differences in the manner or method of lecturing; e.g., a basically monotonic de- livery versus a delivery which is seen to in- corporate much vocal inflection. IO Study Limitations An empirical study of any sort suffers the difficulty of not being able to be or do all things. The following are suggested as limitations in the scope of this study: 1. Though the cognitive task, magic squares, was selected in order to fulfill a number of requirements (p. 3), nevertheless, transferability to other cognitive tasks, i.e., other content areas, either of a more or less complex structure, is not empirically tested in this study. 2. The present study does not account for over-time effects of instructor-student relation existing in a typical classroom environ- ment because it is a "one-shot" learning encounter. For example, the effects of motivation (over time) on intervening student activities, or of rapport, positive or negative, are not allowed. The larger study, VIM (Maatsch, 1975), does include control for intervening activity where testing for delayed recall. 3. An optimum length or frequency for pacing is not deter- mined, though this would seem more easily researchable than the effects of motivation or rapport, noted above. This length and fre- quency, coupled with the degree of complexity of the cognitive task is dealt with in greater length in Chapter IV, p. 56 and again in Chapter V, p. 65. 4. Because pacing appears to require additional time for the presentation of a given amount of infonnation the question of cost- effectiveness arises. A partial answer is presented in Chapter II, p. 21, which seems to include both input and output, factors essen- tial to a cost-effectiveness equation. ll Overview of Study Chapter I deals with the identification and background of the problem, statement of purpose, and research questions and hypotheses. Study definitions and limitations are also included in this chapter. Chapter II reviews the literature on lecture. The intent, as it necessarily should be, is to focus on the choice of variables--pacing, practice, feedback and instructor style--to be manipulated in this study. Chapter III discusses the research design, procedure, and instrumentation. Chapter IV presents an analysis and discussion of data. Chapter V presents a summary of results, a conclusion and dis- cussion to the hypotheses, and implications. CHAPTER II A REVIEW OF LITERATURE Introduction The review of related literature is organized under five major headings: (l) enthusiasm or style of the lectures; (2) organ- ization of the lecture; (3) notetaking in lecture; (4) pacing; (5) practice and feedback. Research findings in these areas provide the basis for the research questions, hypotheses and methodology of this study.1 Enthusiasm of Lecturer Coats and Smidchens (1966) found students to remember much more from a dynamic lecture than from a static one. The dynamic lectures were delivered from memory, with much vocal inflection, ges- turing, eye contact, and animation on the part of the lecturer; the static lecture was read entirely from manuscript, with no gestures 1The reader is directed to the following articles which offer reviews, summaries and bibliographies relevant to variables, other than the five mentioned above, which have, at one time or another, been included in studies interested in improving lecture: (Petric, 1963); McLeish, 1967; Verner and Dickenson, 1967. These are, of a number, the more complete; they are generally included in articles other than those which concern lecture versus other methods of in- struction. The variables indicated above were selected with the in- tention of ready availability and usefulness within the context of a number of constraints: instructor needs; justifications for use of lecture; etc. 12 13 or eye contact and vocal inflection held to a minimum. Mastin (1963), too, found mean achievement higher when lecture was presented en— thusiastically. Sherman and Blackburn (1975) seem to support the concept of enthusiasm when they suggest that improvement of "per- ceived" teaching effectiveness may depend more on changes related to personality factors than on those involving classroom procedures. Though enthusiasm seems to be important to comprehension and retention, Highlander (1954) and Paulson (1954) report source cred- ibility, source sincerity, and the audience's like or dislike for the lecturer to have no effect upon listener's comprehension. Two fairly recent studies, specifically Naftulin, Ware and Donnelly (1973) and Ware and Williams (1975), dealing with the Doctor Fox effect, serve to clarify to some extent the matter of enthusiasm or style. The earlier study (1973), "The Doctor Fox Lecture: A Paradigm of Educational Seduction," dealt with "student" ratings of teaching effectiveness. The authors hypothesized that experienced educators participating in a new learning experience could be seduced into feeling satisfied that they had learned despite irrelevant, con- flicting, and meaningless content conveyed by the lecturer. A pro- fessional actor, who looked distinguished and sounded authoritative, who was dressed with a fictitious but impressive curriculum vitae, was introduced to a group of highly trained educators, as one Dr. Myron L. Fox, "an authority on the application of mathematics to human behavior" (p. 631). The educators were indeed "seduced" into an illusion of having learned by the lecturer's gpylg; a style de- signed to reflect the highly seductive behaviors, enthusiasm, humor, Ill!!! l4 friendliness, expressiveness, charisma, and personality. The authors suggest that teaching effectiveness must be evaluated beyond student- perceived satisfaction (i.e., the "perceived effectiveness" of Sherman and Blackburn, 1975) for student satisfaction with learning mgy_rep- resent "little more than the illusion of having learned" (p. 630). In a subsequent study, Ware and Williams (1975), "The Doctor Fox Effect" appears to be much more than an illusion. Whereas Naftu- lin, Ware and Donnelly (1973) had found teaching style a major fac- tor in determining student ratings, Ware and Williams (1975) further suggest it to be a powerful influence on enhancing student test per- formance. This latter study would seem to be in keeping with the suggestion of Jenkins (1974): "It was impressively clear that the most important determinant of recall was the nature of the pygpp_the subjects experienced..." (p. 788). Possibly the most important issue involved in these studies is that raised by Naftulin, Ware and Donnelly (1973): "...what mix of style and substance in the lecture method is optimal for not just integrating information in a meaningful way but for providing learning motivation as well?" (p. 634). The Doctor Fox Effect is evidence that style alone is sufficient, but the reality of lecturers at large is such as not to support style as p:actica11y_sufficient; Naftulin, Ware and Donnelly (1973) do well to call for an assessment of the relative value of content and style. Qgganization of the Lecture With communication as a goal of lecturing, it is generally believed that a lecture will be more effective if there is an 15 evident order or sequence. Yet the importance of organization is questionable. Thompson (1967) reports, having summarized a number of studies: "Disorganization appears to affect comprehension in written communication, but effects upon comprehension and effectiveness in oral communication are doubtful." Beighley (1954) in a typical study introduced disorganization by shifting paragraphs in a controlled random method; comprehension was not significantly reduced. Gage ‘ (1969) suggests that the lack of organization in oral communication may be compensated by cues provided in delivery: "In any case, re- search on organization in lectures fails to support common-sense expectations" (p. 1454). However, several studies suggest that gen- eralizations on major ideas are better comprehended and retained than are details or specifics and that the better developed the gen- eralizations are, the better retained (Blewett, 1951; Trenaman, 1951). Too, Cofer (1961) and Hovland (1951) suggest that meaningful messages are learned better than are presentations with less meaning. Note Taking McClendon (1958) discovered that comprehending a lecture is not affected either by taking notes or not taking notes; where stu- dents only record the main points in contrast to making detailed notes, their comprehension of the lecture is not impaired. Freyberg (1965) found students who did not take notes at all, but merely lis- tened to a lecture, did better on a recall test taken immediately afterwards than did groups who took detailed notes and others who took outline notes. Hartly and Cameron (1967) compared the notes taken by students during a lecture with what the lecturer had actually 16 said. The material in a transcript of the lecture was divided into 'informational units;' the students' notes were then checked for the number of units recorded. Approximately one-third of what the lec- turer said was noted down by the students. The content agreement with a set of "ideal notes" prepared by the lecturer varied from about seventy percent during the first ten minute period to twenty percent during the final period. This would seem to coincide with the exper- iments of Trenaman which McLeish (1968) reports: in a forty-five minute broadcast on astronomy, assimilation fell off after fifteen minutes, and in thirty minutes the listeners were approaching the point where the information conveyed remained stationary or even de- creased. However, DiNesta and Gray (1972) found notetaking to facil- itate learning by causing the student to process the lecture content. Further, in a recent study, Aiken et al. (1975) found that separating the note-recording activity from listening to the lecture produced learning that was superior to both conventional concurrent note-taking and a no notes control. In this study, note taking, which was separate from listening to the lecture, occurred between lecture segments and corresponded in duration to the time of the directly preceding lecture segment. For example, lecture segment durations for normal speech rate condition were four minutes, five seconds; three minutes, twenty-five seconds; four minutes, thirty-five seconds; and four minutes. The note taking interval, as reported, would equal the durations noted, respectively. This method of taking notes im- mediately after lecture segments was termed "spaced" note taking. On a test for recall on the lecture, forty-eight hours later, it was found that: 17 1. It was twice as likely that lecture material noted would be recalled than if it was not noted; 2. This effect is considerably stronger with the spaced note taking than with the parallel note taking condition. However, a weakness or two should be noted as existing in this study: 1. There is no mention as to control of intervening activity during the forty-eight hours prior to testing. 2. "To equate for total time, subjects in the parallel note taking and no notes control conditions filled the time between lecture segments with a letter cancellation task" (Aiken et al., 1975, p. 440). Both conditions would seem to be detrimental to the strength of the findings: forty-eight hours allows for a great degree of intervening activity which is not accounted for; the letter cancellation task between lecture segments would seem to offer a degree of interference not found in the spaced note taking. Consequently, as Aiken et al. (1975) suggests, the spaced lecture format is a "promising instruc- tional strategy worth further investigation" (p. 443). mile An extension of the Aiken et al. study (1975) involving the effects of spacing, actually pauses, is found in other studies.2 A theoretical perspective was offered by Simon (1971) when, by combining 2A number of different terms are used in the literature to refer to pauses, allowing for cognitive processing, during instruc- tion: pacing, spacing, wait-time. These appear to be interchangeable terms; for purposes of this study, the term "pacing" is used. 18 data from a number of experiments, he was able to identify and measure a basic unit of human memory: a "chunk." He found the chunk capacity of short-term memory to be in the range of five to seven seconds. Moreover, Simon demonstrated that the transfer of information from short-term to long-term memory takes about five to ten seconds per chunk. M. Budd Rowe (1973) demonstrated the importance of pauses in a study involving the training of elementary teachers. She found that by increasing the length a teacher pauses after asking a ques- tion from a natural average of about one second to an average of three seconds, eight distinct benefits were accrued: l. The length of responses (number of words) increases; 2. The number of unsolicited but appropriate responses in- creases; 3. Failures to respond decreases; 4. Incidences of speculative responses increase; 5. Incidences of student-student comparisons of data in- creases; 6. Incidences of evidence-inference statements increases; 7. Frequency of student-initiated questions increases; 8. The variety in type of verbal moves made by students in- creases (p. 203). A side benefit noted by M. Budd Rowe is also of interest: "When the duration of the pauses is increased, the number of overt disciplinary moves teachers make actually declines, so’if the function of short pauses is control, the habit may be self defeating" (p. 223). Rowe 19 suggests, as had Taylor (1969), that in the lengthened pauses (or "wait-time") "time is bought for more cognitive procesSing on the part of students and for better use of feedback on the part of the teacher." Apparently, the pauses serve a cognitive function in that "the effect of increasing the duration of pauses...(had) been to shift the emphasis away from (classroom) control toward cognitive processing" (p. 223). Thus, as Rippey said in reaction to the VIM presentations, AAMC, 1975: After generations of inconclusive research as to what the teachers should say and how he should say it, the payoff really came from studying what happened when the teacher kept his mouth shut. I have often advised teachers that the moment of silence in the classroom may be the ONLY time when everyone is thinking...including the teacher, to quote Confuscious, "Man engage brain before putting mouth in gear." Coldevin (1975) compared the effects of three types of review segments--spaced, massed, and summary formats--in an attempt to: 1. Assess the comparative effects of insertion of review segments within a television program on strengthening re- tention of information derived from the program; 2. Isolate differential effects of spaced, massed, and summary review treatments on cognitive acquisition; 3. Identify possible relationships between production review strategies and attitude shifts toward the central themes in the program (p. 292). The "spaced" review was added as a variable identical to the place- ment of massed or distributed review "with the addition of a stressed hiatus between normal presentation of review statements" (p. 292). 20 The "space" was a five-second pause or rest break between individual review statement presentations. "This...variable was added in an attempt to provide a logical variation to previous research (in ad- dition to media format) since the strategy employed in review presen- tation may be more critical than placement position within a particu- lar program" (p. 282). Both cognitive acquisition and attitude shift were assessed to determine differential effects, if any, of the re- view treatments. An analysis of variance revealed a significant superiority of the spaced treatment over both the massed and summary treatments in cognitive acquisition. With no significant difference found between massed versus summary treatments Coldevin suggests that though "the inclusion of a review strategy gitpjp_(massed) a program may not be more effective than providing a total summary at the con- clusion when the presentation formats of the two strategies are iden- tical," nevertheless, "the internal temporal distribution of review presentation (spaced) appears to be a critical factor" (p. 298). The spaced treatment was also found, upon analysis, to be superior to other review treatments with respect to attitudinal shift. As suggested by Rippey (1975), the inherent nature of a for- mat allowing for pauses enables greater internalization and covert practice between repetitions: a rehearsal buffer for coding and re- trieval (Gagne, 1971). As a result, "according to this theory, in- formation coding is facilitated toward more effective retrieval at a later time. The greater retrieval of information in turn allows for more effective assignment of attributes to objects in the process of attitude information... (Again) placement within the program appears 21 to be less critical than the temporal distribution of review segments" (Coldevin, 1975, p. 300). A number of concerns should be apparent with the concept of pacing: (1) What is the optimum length of the pace to be? (2) What is the cost in terms of time and amount of material presented or not presented? and (3) What is the optimum frequency of the pacing to be? Questions 1 and 2 are empirical questions which can be researched, question 3 seems to have been dealt with by a number of people, par- ticularly Rippey (1975) when speaking of pauses: ...my comments do not suggest that the pace of presentation be slowed down. If anything, material is usually presented too slowly. Research with speech compressors has shown that the rate of comprehension of spoken material can at least double simply by speeding up the presentation rate. Unfortunately, increased pausing prolongs the time it takes to cover a given amount of material. However, the material in between the pauses can be speeded up. It is mind boggling to realize that in research or speech compressor intelligibility was completely destroyed by removing relevant pauses, while in research using the Cloze technique, comprehension was not reduced appreciably by re- moving up to 30% of the words in between. The most succinct and effective editorial advice is "Cut" (p. 4). Practice and Feedback It is generally understood and accepted that learning is facilitated when the student participates "responsibly" in the learn- ing process. As Such, Bligh (1972) suggests that rehearsal (or prac- tice), requiring an activity by the student, during lecture, would make that instructional method more effective. Holcomb (1973) also suggests student involvement, either overt or covert, during lec— ture, to enhance effectiveness by positively_affecting performance: The instructor could pause after brief presentations and require students to state the material in either oral or written form. This 22 restatement of the material could be either instructor or student checked. Rippey (1969) echoes these suggestions when applying a framework for analysis of educational research--originally proposed by John Ginther. He states that student involvement, overt or covert, is of significant importance when student performance is a consider- ation. Rippey (1969) found overt responses useful when material was unfamiliar or difficult, or when students were young, apathetic or had lower intelligence quotients. Covert responses were found to save 31mg_when conditions were reversed or when responses had low informa- tion value. Again, time--input and output--is a factor, seemingly as in justifications positioned for lecture. If the extra time it (takes) a student to make an overt response (is) to be of value, overt responses (have) to be relevant and the responses called for (have) to be of high information value (Rippey, 1969, p. 219). Maatsch (1975) in comparing the best and the poorest instruc- tional methods, e.g., simulation and lecture, respectively, suggests that two differences emerge as variables affecting performance out- COITIES : 1. Lecture (as a) continuous, non-redundant learning experi- ence...(does) not allow students an opportunity to stop and process new information or (to) receive feedback for processing new information incorrectly: Simulation... (is) aced by the student participant in the simulation. He (has) time to observe, process and respond before in- struction (continues). 2. ...if (the student responds) incorrectly, he (receives) feedback from the instructor and (is) coached until he (responds) correctly. (pp.6-7). It is apparent from the literature that information feedback is particularly crucial in cognitive learning. Gagne (1970) states 23 that "some means or other must be provided for (the student) to per- ceive the results of his activity, to receive from the learning en- vironment some feedback that enables him to realize that his perfor- mance is 'correct'" (p. 315). He concludes that feedback for the correct accomplishment will probably be "of considerable value for the efficiency of learning" (p. 316). McKeachie and Kulik (1975) support the importance of informational feedback in cognitive learn- ing and suggest four factors which contribute to positive changes in performance affected by feedback: 1. New information is provided. 2. The learner is positively motivated to change. 3. Anxiety does not disrupt attempts to change. 4. Learner knows what to do to achieve a better outcome. Typical programmed learning units are cited by McKeachie and Kulik in explanation: with the low probability of errors inherent to such units, "Knowledge of results conveys little infbrmation and is in a region of low motivation, according to the theory of achievement motivation" (p. 197). It is suggested that feedback would have a more positive effect when the task is more difficult and the motiva- tion and information provided are subsequently greater. Maatsch (1975) states more directly what McKeachie and Kulik (1975) seem to infer and what seems to be of a different bent than Gagne's emphasis on feedback for correct accomplishments: ("It (is) apparent that the role of feedback should be to disrupt incorrect responding and to coach the student to respond correctly if he is to retain the correct response" (p. 7). Though not at odds, it would 24 seem, with Gagne, Maatsch and McKeachie and Kulik do appear to repo- sition the import of feedback. Correct responses, as in fact Gagne (1970) hints, are not in need of reinforcement, i.e., repetition is not shown to be an important condition for retention (for either Gagne's concept or rule learning). It would appear that feedback, if it offers/provides nothing more than repetition, is not an essen- tial aspect of learning or retention. A student who is correct as he progresses through all points of instruction (as he may in a typical programmed instruction) would not necessarily benefit from feedback. He might, as McKeachie and Kulik (1975) suggest, suffer such negative results as inattentiveness or carelessness when fed something of little import or consequence. However, in order to avoid the reten- tion of incorrect information, as Maatsch points out, feedback must disrupt incorrect responding and, subsequently, supply a correct response. Summary This chapter has reviewed literature related to the present study. Research under five major headings indicate the following: 1. Enthusiasm or style of the lecturer is an important and effective variable with respect to student performance as well as student ratings of the instructor and the instruction. 2. Variations in organization of lecture content does not significantly reduce comprehension, but both major ideas and mean- ingful messages are retained better than details or specifics and meaningless messages. 25 3. Research seems to indicate that note taking itself does not necessarily facilitate retention, though it ppp_facilitate learn- ing by causing the student to process lecture content. Spaced note taking was found superior to both concurrent note taking and no notes in a study by Aiken et al. (1975). 4. Pacing, as research seems to indicate, is potentially a very significant variable for learning, in that it allows for an emphasizing of cognitive processing, i.e., the opportunity for either covert or overt practice. 5. Both practice and feedback were suggested to aide in the learning and subsequent retention of information. Maatsch (1975) indicates feedback must disrupt incorrect responding and then supply a correct response. The present study employs these findings in an attempt to look at a number of variables which might be used to enhance the effectiveness of lecture; variables which are both readily employ- able by lecturers and would seem to fit easily the continued and persistent use of lecture, allowing, also for the maintenance of justifications for the use of lecture, as mentioned in Chapter I. CHAPTER III STUDY METHODOLOGY AND PROCEDURE Introduction This chapter will present the research procedure by which variables were manipulated to enhance the effectiveness of the lee- ture method of instruction by more actively involving students in the instructional process. This presentation will encompass: l. A description of the population and sample selection pro- cedure; 2. A description of procedures including experimental treat- ments and a description of the experimenters; 3. A statement of study hypotheses; 4. A description of the study's instrumentation, including an analysis of the instructional instrument, test format and relia- bility; 5. A description of data analysis techniques. Eppulation ngulation and Sample Selection The subjects were predominantly college sophomores drawn from several introductory psychology courses at Michigan State University. 26 27 Students were recruited a week to ten days before the time of each of the five treatments. One and a half hours of time was allotted for each treatment. Students scheduled themselves by sign- ing a standard psychology form used for human research. A half page general description of the purpose of the research was available to students. The only clue as to which VIM instructional model might be used was that the sign-up sheets for this experiment contained twenty-five available name spaces while a concurrent experiment on simulation used a sign-up sheet with ten spaces. The sign-up system provided the student with a reminder card that contained the address, time and day of the experiment. When students arrived for the exper- iment they were told the purpose of the experiment, the method of instruction to be used and general rules, and introduced to the lec- turer. Students for the two control groups, the nominal lecture and the paced lecture, were recruited, instructed and tested during the winter term of 1974. The three maxi groups were processed during the spring term of 1975. Each experimental treatment was conducted in E-2 of Fee Hall East, an experimental classroom. Each session was videotaped for later viewing and subsequent confirming of consistency in presenta- tions. Procedures Experimental Treatments The treatments to be described here are variations on the lecture, as an instructional method. As will be apparent, (l) the 28 treatments attempt to mainipulate variables which might improve the effectiveness of the lecture by (2) more actively involving students in the lecture process. In order to present an omnibus picture of lecture, two groups --nominal lecture and paced lecture--from earlier VIM experiments were included as control groups for this study's design. There was a total of five groups by treatment: nominal lecture (nominal con- trol), paced lecture (paced control), maxi-paced lecture, maxi-paced lecture with practice, and the maxi-paced lecture with practice and feedback. TABLE 3.l--Experimenta1 Lecture Treatments. Treatment N 1. Nominal control 21 2. Paced control 6 3. Maxi-paced l8 4. Maxi with practice 17 5. Maxi with practice and feedback 12 Treatment Group Description All subjects in all groups were confronted with the same basic content and visual aids, except for the following differences in treatments . l. The nominal lecture group was given the instructional material verbally from a lecture manuscript with visuals projected 29 on a screen (Appendix A). The task was presented one element (i.e., concept or rule) at a time. Each element was of concrete form, ac- tual figures and numbers, displayed on an overhead transparency. Correct and incorrect examples of the rule were displayed concurrent- ly. The lecture was non-redundant, relatively monotonic and given by a graduate assistant, trained for and familiar with the task, Magic Squares. He was introduced as an experimenter. 2. The paced lecture group was given the same material and visuals in the same non-redundant and monotonic style by the same graduate assistant. The exception was the pppjpg_of the lecture. In the paced lecture, in contrast to the nominal lecture, the lec- turer presented the information, i.e., a concept or rule, followed by a thirty-second pause during which the students were directed to think about what he/she had just been given in lecture. The visuals used to illustrate concepts and rules were not available for refer- ence during the pause. The following three maxi groups received the same lecture content, but from a nationally recognized educator and lecturer with a more exciting style of presentation, in contrast to groups 1 and 2 above. In order to allow the "expert" the fullest realization and use of his style and personality, the only constraint imposed was the lecture manuscript (of group 1 and 2) as a point of reference. The expert lecturer was introduced as a Michigan State University professor and expert in the field of Magic Squares, to enhance his reputation in the eyes of the students in each of the maxi groups. In addition, the expert gave an interesting history of the discovery 30 and mathematical development of Magic Squares to enhance student in- terest in Magic Squares (Appendix B). 3. The maxi-paced lecture, then, was the paced lecture and pauses of group 2 with the exceptions and additions noted above. 4. The maxi-paced lecture with practice was given the same paced lecture and pauses by the expert lecturer. During the pauses, however, the students were told to practice, in a workbook provided, using the information, i.e., concept or rule, given in lecture. 5. The maxi-paced lecture with practice and feedback group, again, received the paced lecture from the expert, etc., but with the inclusion of pptp_practice and feedback, from the lecturer via visuals, for their answers during the thirty-second pause. In order to control for consistency in presentations across the three maxi groups post-hoc analysis of the videotape was made. The expert's style of presentation was found to include consistency in the following additional aspects: repetition of information (three statements explaining correct examples on each visual, one statement on the incorrect example); the use of advanced organization, by which the lecturer prepared the students for what he was to tell them by keeping them always aware of the plan of presentation; terminal sum- mary statements, i.e., a capsulizing of what had been presented. The expert lecturer's style, in contrast to the non-redundant, relatively monotonic style of the graduate student, of groups 1 and 2, trained for the task, was found to contain (1) climaxes and sub- climaxes in presentation of the instructional task. (2) The expert's speech varied in pitch and loudness, producing a much crisper speech 31 pattern, avoiding the patient monotone. (3) Too, he was much more animated in gestures and movement than the graduate student. All these aspects were seemingly employed as characteristics of the lec— turer's style and personality, to hold and focus student attention. He was, as it were, the more charismatic for it all. No note taking, talking or questions were allowed in any of the groups. Each group received the test instrument (described later) five minutes after the completion of the instructional task. During the five minutes the subjects were allowed to move about, satisfy their mutual needs, etc. but were instructed, when talking with each other, not to discuss other information given in the lecture. The instructional task was analyzed to individual elements (see Analysis of the Instructional Task). The order of presentation was determined on the basis of what was determined to facilitate re- call. That is, the assignment rules were presented in the order of descending frequency (Thiagarajan) as opposed to the order that they are normally used to make a magic square. Experimental Hypotheses The design and procedures described above were intended to test the following research hypotheses: 1. Students exposed to the paced lectures will show better performance on paper and pencil measures of cognitive performance than students given the nominal control (lecture). 2. Students exposed to the maxi-paced lecture presented by a "recognized expert" will show better performance than students 32 given the paced control (lecture) by the graduate student as measured by a cognitive test. 3. Further, students exposed to the maxi-paced lecture with practice and feedback will show better performance than students given the maxi-paced lecture with practice only, which will cause students to show better performance than students given the maxi-paced lec- ture. In other words, with respect to performance, the lecture methods will order themselves as follows: maxi-paced with > maxi-paced > maxi- > paced > nominal practice & feedback with practice paced control control Instrumentation Measures Measurements were taken to assess post treatment achievement. This section will deal with a description of what was tested and how it was tested. Analysis of Instructional Task The cognitive measures were adapted from the VIM research program by adding items. The original VIM test was designed to measure each task element of a complex game--Magic Squares--in a variety of conventional paperepenCil test formats. The elements of the learning task have been analyzed by Thiagarayan (1971) as involving different types of learning as de- fined by Gagne (1970). As used in this study, Magic Squares can be analyzed as follows: II. 33 Concepts A. The Defining Elements A Magic Square is a square with rows and columns of numbers. 1. The numbers in rows, columns and diagonals 2. Produce an identical sum 3. And no number can be used more than once in any one Magic Square 8. Number Series 1. Must be positive 2. Must ascend 3. Must maintain constant interval between adjacent numbers 4. Can start with any positive number C. Geometric Figure l. A square with an equal number of 2. Odd rows and columns Rules A. Rules for assigning numbers to a square 1. Name of rule: First number When is it used? When square is empty How is it applied? Place first number in top row, middle column Example: Example: 2. 34 Name of rule: Top to bottom When is it used? When last known number is in top row (exception, right corner) How is it applied? Place next number in the bottom row one column to the right of the last number ® Name of rule: Right to left When is it used? When last number is in right most column (exception, upper cell) How is it applied? Next number is placed up one row in the left most column Example: Example: 4. 35 @ Name of rule: Exception to the diagonal When is it used? When the last number has a cell one row above and one column to the right but that cell is already fillpg_with a number How is it used? Place the next number directly below the last number 3 @ 2 Name of rule: Diagonal When is it used? When the last number has a cell one row above and one column to the right and that cell is empty. How is it applied? 36 Place the next number in the empty cell one row up and one column to the right. game: 1 6 3 ® 4 2 6. Name of rule: Upper right corner When is it used? When the last number is in the upper right corner How is it applied? Place next number directly below last number E_xa_mple= 1 6 3 5 (D 4 2 Test Format All task elements of the original VIM first were tested in four, common, paper and pencil formats: recognition, recall, appli- cation, and problem solving. The recognition batteries were multi- ple choice with low alternatives. The concept elements were tested by verbal statements; the rules by graphic examples. In the latter 37 case, four mgjpp_squares, each with only enough numbers to illustrate one rule, were the stimulus material. The student was instructed to select the one of the four figures in which an assignment rule was not violated. The pggpll_batteries for concepts asked the student to list each of the elements that: (1) define what a Magic Square is, (2) describe a correct number series, and (3) indicate what geometric figure can be used. The rule subbattery required the generation of the name of each of the assignment rules and how they are applied. The gpplication of rules subbattery required placement of a given number in a magic square containing a niminum amount of stimulus ma- terial. This material was organized such that the given number could only be placed correctly by using a specific number assignment rule. The application of concept subbattery posed specific problems not seen in instruction that required comprehension of concept elements for solution. In the problem solving_subbattery students had to select and combine a number series and a geometric figure to form a correct magic square. This figure was of greater complexity than any given during the instructional treatment. In order to limit the instructive effects of test taking two features were incorporated in the cognitive test. First, the sub- batteries were presented in the order of least amount of stimulus material first. This resulted in a subbattery order of problem solving, application, recall and recognition. Second, students were instructed not to peruse the test. That is, they were not permitted 38 to look ahead or go back from the specific page that they were work- ing on at any given time. In order to encourage perservance, stu- dents were informed that the more difficult items were in the begin- ning of the test. The test instrument is reproduced in its entirety in Appendix C. Reliability Analysis of this instrument in previous experiments has iden- tified two task element factors: rules and concepts. These factors account for forty and twenty percent of the variance, respectively. (In order to increase the reliability of the concept batteries items were added to the original VIM test.) Table 3.2 lists the individual subbatteries of the cognitive measure and their respective reliabil~ ities. TABLE 3.2--Reliability coefficients for subbatteries. Subbatteries Cronbach Alpha Concepts Problem Solving * Application .921 Recall .679 Recognition .477 Rules Problem Solving .979 Application .974 Recall .644 Recognition .970 * Insufficient number of items to run calculations. 39 Data Analysis Techniques This study is essentially concerned with how different groups respond to a common piece of information. For purposes of analysis, each group is considered to be a separate unit. Since there are more than two groups involved, a technique which allows for the simultan- eous testing of all groups is required. The analysis of variance testing procedure is such a technique. The design for this study classifies each participant into mutually exclusive and jointly exhaustive categories based upon one characteristic: treatment group identification. Because of the need for only one characteristic, the design will be a one way classifica- tion scheme. Glass and Stanley (1970, section 15.12, pp. 362-368) refers to the combination, one-way design analysis of variance test- ing as the one-way anova with unequal n's. The following illustrates, by example, the development of the analysis for this study: Let us consider a one design variable called group membership. This can be illustrated in figure 3.1. Group Membership Figure 3.1 40 Let us assume there are five groups. Each group is consid- ered to be a level of the variable group membership. The groups can be called I, II, III, IV, V. This is illustrated in figure 3.2. Group Membership I II III IV V Figure 3.2 Further, assume that each participant in this study will take a post-test and have that test scored. The resultant score can be denoted by the letter X. Now, consider the case where there are several subjects in each group, although the number of subjects in each group need not be the same for all groups. It is necessary to be able to identify not only the group from which each respondent's score comes, but also 'who is who,‘ within each group. This is accomplished through the use of subscripts-~two for this illustration. The first subscript denotes the subject number within each group; the second subscript denotes the group to which the subject belongs. For example: x],2: the score (X) for participant l in group II x6,2: the score (X) for participant 6 in group II x5,5: the score (X) for participant 5 in group V X17’3: the score (x) for participant 17 in group III. 41 If these scores were to be placed in figure 3.2, above, the result would look like figure 3.3. Group Membership I II III IV V X X 1,2 17,3 5.5 X 6,2 Figure 3.3 The number of participants in each group is referred to as the frequency for each group and is denoted by the letter n. For this study the following names and frequencies are used: Ema Name. 11 I Nominal Control 21 II Paced Control 6 III Maxi-paced 18 IV Maxi with practice 17 V Maxi with practice and feedback 12 The analysis of variance procedure tests the hypothesis that all groups have the same mean score. If this procedure is rejected the testing procedure does not isolate which group or combination of groups might differ from the others. However, knowing which group or combination of groups differs from the others is critical to this study. Since the inquiry at this point is after the fact, i.e., differences have been detected somewhere, it is necessary to 42 proceed in a post-hoc fashion. A technique which allows this to occur was developed by Henry Scheffe and is referenced by Glass and Stanley (1970, section 16.14, pp. 388-393). This procedure allows one to contrast or compare two groups, or combinations of groups, with each other without violating necessary statistical assumptions. "The purpose of these procedures is the isolation of comparisons be- tween means that are responsible for or contributed to the rejection of Ho“ (Glass and Stanley, 1970, p. 382). Therefore, the logic of the analysis proceeds as follows: 1. classify each respondent score into the appropriate group; 2. test the omnibus hypothesis of no significant differences between any of the groups; 3. if 2 (above) can be rejected, perform Scheffe post-hoc multiple comparison procedures to isolate groups or com- binations of groups of interest which tend to cause or contribute to that rejection. Summary The sample for this study used 74 students, predominantly college sophomores, recruited from a number of introductory psychol- ogy courses at MSU. Five treatment groups received instruction on the construction of magic squares via lecture. Two groups were lec- tured by a graduate student trained for the task; three groups were lectured by a nationally recognized educator and lecturer. Treat- ments were videotaped for review at a later time to control for con- sistency in presentations. 43 Hypotheses concerning pacing, practice, feedback, and style were arrived at and tested. The resultant data was analyzed using a one-way analysis of variance, and a Scheffé post-hoc analysis of differences. CHAPTER IV ANALYSIS OF DATA Introduction The data collected and analyzed included a number of statis- tical and descriptive procedures. The computations performed in the study were done primarily on the CDC6500 computer. Programs used in the analysis were obtained from the Statistical Package for the Social Sciences (SPSS), version VI. Statistical procedures employed included analysis of variance and Scheffe post-hoc analysis of dif- ferences. This chapter will include a statement of experimental hypotheses, a rationale for the formulation of the dependent vari- ables and treatment groups, analysis and explanation of data. Experimental Hypotheses Since the research hypotheses are important only if the treatment groups differ from each other on the dependent variable of interest, a determination of an overall difference between the groups will be the first test performed. If the groups are found to differ, then the research hypotheses will be explored. The translation of the research hypotheses into testable experimental hypotheses are as follows: 44 45 Research Hypothesis 1 Students exposed to any of the paced lectures will show better performance as measured by a cognitive test score than stu- dents exposed to the nominal lecture. Statistical Hypothesis 1: H] u] ' uj where u] = mean of nominal lecture group p2 = mean of nominal paced p3 = mean of maxi-paced u4 = mean of maxi with practice “5 = mean of maxi with practice and feedback and l = l, . , 5 J = l, , 5 li‘J Research Hypothesis 2 Students who are exposed to an "expert" lecturer using pacing will do better than students exposed to a graduate student lecturer using pacing. 46 Statistical Hypothesis 2: H]: u] f uj where u] = mean of nominal paced lecture “2 = mean of maxi-paced u3 = mean of maxi with practice u4 = mean of maxi with practice and feedback and i = l, . , 4 J = l, , 4 iii Research Hypothesis 3 Among the paced lectures presented by the "expert,“ students exposed to the maxi-paced lecture with practice and feedback will show better performance than students given the maxi-paced lecture with practice only, which in turn would enhance performance more so than the maxi-paced lecture. Statistical Hypothesis 3: Ho: Umpf 5 ump S'um >11 >11 ”1‘ L’mpf mp m 47 where u = mean of maxi-paced with practice and feedback mpf “mp = mean of maxi-paced with practice “m = mean of maxi-paced lecture Formulation of the Dependent Variables The tests results and subsequent data were collected as des- cribed in Chapter III, Test Format. The subset of variables this study addresses are denoted as: RPS = rule problem solving RAP rule application RRL = rule recall RRN = rule recognition CPS = concept problem solving CAP = concept application CRL = concept recall CRN = concept recognition Table 4.1 shows the correlation between all variables. Upon inspec- tion of this table it can be seen that: (1) variables RPS to RRN do highly correlate with each other. (2) Further inspection shows vari- ables CPS and CAP to correlate highly with each other and nothing. (3) Finally, it is apparent that variables CRL and CRN correlate more highly with variables RPS to RRN than with CPS and CAP or with each other. The correlation suggested in (1), (2), and (3) may be more clearly demonstrated if Table 4.1 is broken into three tables 48 TABLE 4.l.--Correlation of all dependent variables. RPS 1.00 RAP .66 1.00 RRL .56 .62 1.00 RRN ..64 .80 .61 1.00 CPS .17 .15 .35 .29 1.00 CAP .23 .32 .28 .39 .83 1.00 CRL .45 .50 .52 .50 .26 .25 1.00 CRN .27 .52 .46 .43 .24 .41 .41 1.00 RPS RAP RRL RRN CPS CAP CRL CRN comparing variables RPS to RRN with each other (Table 4.2), variables CPS with CAP (Table 4.3), and RPS to RRN with CPA and CAP (Table 4.4). TABLE 4.2.--Corre1ation of RULE variables. RPS 1.00 RAP .66 1.00 RRL .56 .62 1.00 RRN .64 .80 .61 1.00 RPS RAP RRL RRN TABLE 4.3.--Correlation of CONCEPT variables. CPS 1.00 CAP .83 1.00 CPS CAP Table 4.2 shows a nearly uniform inter-item correlation pat- tern which suggests that the variables tend to measure the same 49 TABLE 4.4.--Correlation between RULE variables and CONCEPT variables. CPS .17 .15 .35 .29 CAP .23 .32 .28 .39 RPS RAP RRL RRN concepts. Table 4.3 shows a strong correlation between CPS and CAP, indicating a very strong tie between these variables. The low cor- relation pattern demonstrated by Table 4 suggests that these vari- ables are testing something different from RPS to RRN. This is as it should be as far as we have gone. However, the dependent variables, by a priori definition, were to have been rules (RPS to RRN) and con- cepts (CPS to CRN), and uni-dimensional. The two remaining variables CRL and CRN do not support this denotation of dependent variables. Rather, two things are suggested from Table 4.1: l. The dependent variables are not uni-dimensional but rather multi-dimensional (and are in need of more careful definition); or 2. The measurement instrument is in need of more development. In view of the correlations of variables and the reliability coefficients of Chapter III, the dependent variable for this study is limited to RULES, a simple, linear combination of Variables RPS, RAP, RRL, RRN. The dependent variable denoted CONCEPTS, a linear combina- tion of CPS, CAP, CRL, CRN, will not be used for analysis. A further eXplanation of this will occur in Chapter V. 50 Formulation of Groups The treatment groups for this study are defined as the: 1. Nominal Control Paced Control Maxi-Paced wa Maxi-with Practice 5. Maxi-with Practice and Feedback The treatment groups are fully described in Chapter III, p. 28. The groups, frequency (N) of each group, mean RULE score, RULE score standard deviation and minimum and maximum RULE scores are presented in Table 4.5. TABLE 4.5.--Treatment groups, X RULE score, standard deviations, minimum and maximum RULE scores. X' Minimum Maximum Treatment Group N Rules Rules Rules Rules Nominal Control 21 22.76 14.39 0 43 Paced Control 6 30.67 8.5 19 48 Maxi-Paced 18 31.17 13.05 25 . 46 Maxi-with Practice 17 34.47 7.62 2 49 Maxi-with Practice and Feedback 12 34.08 ' 6.99 21 47 Analysis The research design for subsequent statistical analysis is referred to as a one—factor analysis of variance-fixed effects model (Glass and Stanley). The results of the analysis using RULES as the dependent variable is shown in Table 4.6. 51 TABLE 4.6.--Analysis of variance of treatment groups on the variable RULES. Source df Squares Square F Sig Between 4 1667.15 416.79 3.24 .017 Within 69 886.79 128.48 TOTAL 73 10531.95 Because of the significant F test statistic a Scheffe post- hoc analysis of differences was performed to isolate the group or combination of groups which produced this overall difference. Since all post-hoc tables will be similar in format, a word of explanation concerning these tables is in order. Each table will have five columns. The first column is headed CONTRAST. This indicates which groups are being compared (or contrasted). Each group mean is iden- tified as following: The mean of the Nominal Control group on Rules = NC The mean of the Paced Control group on Rules = PC The mean of the Maxi Paced group on Rules = MP The mean of the Maxi-with Practice group on Rules = MwP The mean of the Maxi-with Practice and Feedback group on Rules = MwPF The next column is headed by the Greek letter V- The caret (A) is placed above the letter to indicate that it is an estimate of the theoretical value of the contrast specified in column 1. The formula for computing w is: w = CIX1 + szz + ...+Cij where 52 Cj are constants that must sum to zero and Xj are the sample means of the groups in the comparison on the dependent variable of interest. Column 3 is headed by the letter C. This column makes explicit the weights used to compare the groups of interest. Column 4 is entitled $66. This is the result of multiplying the Scheffé constant (Glass and Stanley, 1970, p. 393) by the standard error of O. The last column entitled SIG is to determine whether or not the interval formed by O: 580 contains the value g. If 9 lies within this inter- val then the result of the comparison is said to be nonsignificant (N.S.). If Q_does not lie within this interval then the results are significant. All comparisons are done at the .05 level of signifi- cance. The first statistical hypothesis can now be answered by con- trasting the nominal control against all paced lecture groups. The results are shown in Table 4.7. TABLE 4.7.--Scheffé post-hoc analysis contrasting the nominal control with the paced lecture groups using total RULE score as the dependent variable. Contrast 0 Values of C 586 Sig 4 NC-(PC+MP+MwP+MwPF) -39.338 4,-1,-1,-1,-1 38.266 <.05 Table 4.7 will be explained in detail for further clarifica- tion. It should serve as an example for an understanding of Tables 4.8 and 4.9. 53 The groups of interest are NC, PC, MP, MwP and MwPF. Since the hypotheses specifies that all groups shall be compared to Nominal Control, the Contrast column shows that the sum of all the group means will be subtracted from four times the group mean of the Nominal Con- trol group. The second column,¢3, has the value of -39.338. This is the result of: O = 4xNomina1(XPaced + xMaxi + xMaxi Paced + xMaxi Paced ) Control Control Paced w/Practice w/Practice and Feedback The sample means were computed from the RULES score of each partici- pant in the study. The values of C are the weights +4,-l,-1,-l,-l. The mathematical restriction is that these sum to O. This is the C359: 4+(-i)+(-i)+<-u+(-u=o. The Scheffé constant S is computed by the formula 5‘ “J" )l-oFJ-l ,N-J where J = number of groups in the contrast, in this case 5; N = num- ber of participants in the total contrast, in this case 74; F is the tabled value of the F distribution at the l-o level of significance, in this case 1-o==.05, with J-l and N-J degrees of freedom, in this case F, at the .05 level, with 4 and 69 degrees of freedom = 2.53. The standard error CO is computed by the formula 54 / 2 60 = MSw(2%j—) J where MSw = mean square within; Cj = constant or weight that multi- ples the appropriate group mean j; n5 = the number of participants in group j. In this case MSw = 128.48 (Table 4.6), Cj = 4,-1,-l,—l, -1 (Column 3) and hi is respectively, 21, 6, 18, 17, and 12. The result of (~39.338) (38.266) (-77.604),(-1.072) m i $60 Since Q_does not lie in this interval, this Scheffe post-hoc analysis shows that the nominal group differs significantly from the paced lecture groups (p < .05). Because the combined means of the paced lecture groups were subtracted from the mean of the nominal lecture group and the fact that the contrast is negative this implies that the paced lecture groups, as a whole, did better than the nominal lecture group (nominal control). This leads to the conclusion that the lecture can be enhanced in a positive fashion by using, at the very least, the variable pacing. The second statistical hypothesis can be answered by contrast- ing the paced lectures presented by the "expert" with the paced lec- ture presented by the graduate student. The results are shown in Table 4.8. The Scheffe post-hoc analysis shows that the paced con- trol does not differ significantly from the "expert" lecturer's paced lectures. This suggests that any advantage attributable to the style of "expert" lecturer is washed-out (countered) by the effects of pacing. 55 TABLE 4.8.--Scheffé post-hoc analysis contrasting the "expert" lecturer's groups with the graduate student's group using total RULE score as the dependent variable. v 1 A 0 Contrast O gfugs 50$ 519 (MP+MWP+MWPF) 37.718 1,1,1,-3 48.315 NS -3 (PC) The third statistical hypothesis can be answered by contrast- ing the "expert's" paced lecture groups amongst themselves. The re- sults are shown in Table 4.9. TABLE 4.9.--Scheffé post-hoc analysis contrasting the "expert's" lecture groups using the total RULE score as the depen- dent variable. Contrast O Vglugs 58$ 519 MP-MWP -3.30 1,-1 9.49 NS MP-MWPF ~2.91 1,-1 10.47 NS MWP-MWPF .39 1,-1 10.59 NS The Scheffé post-hoc analysis shows that the "expert's" paced lecture groups did not order themselves as expected (i.e., MwPF > MwP > MP). In fact, the analysis shows that the groups do not differ signifi- cantly from each other. This unexpected non-significant finding may be explained by inspecting the practice errors, or incorrect responses, made during the pauses for each of these two groups. An analysis of workbook practice responses for each group found that 91% of the subjects of 56 both groups responded correctly to a given item during the pauses. Since the error rate was only 9%, the effect of feedback on incorrect responses of that groups scores was minimal. Consequently, the contrast of mean group scores on TOTAL RULES produced no significant differences between the paced lecture with practice and feedback and the paced lecture with practice only. There are two explanations for the low error rate obtained, namely: (1) The complexity level of each concept and rule was too low, or (2) the pause after each concept or rule presentation made the practice task too easy. If the complexity level were raised, or the frequency of pacing lowered, the error rate could rise to a level great enough to have feedback affect the mean score of that group, to the extent that significant differences would occur between the paced lecture with practice and the paced lecture with practice and feedback. Figure 4.1 presents a test of this hypothesis. It is a X2 analysis of the shift of incorrect responses, made during the lecture, to correct responses on the corresponding items on a test five min- utes gfpg§_the lecture, among the practice only and the practice with feedback groups. The x2 value is 5.998 which is significant at the .05 level. This means that the correctness of the test answers is related to the group membership of the respondents. In other words, feedback does improve upon incorrect responding. Figure 4.2 shows the percentage shift produced by the two methods from workbook to test response. The percentage gain for the paced lecture with practice and feedback is much larger than the paced lecture with 57 practice suggesting that the feedback is a significant contributor to correct the initially incorrect responses. Workbook Without With Feedback Feedback Test "r°"9 right x2 = 5.998 p < .05 Figure 4.l.--Effects of practice with and without feedback in incorrect responses. Wrong on Wrong on % Group Workbook Test Gain With Feedback 9 l .89 Without Feedback 20 12 .40 Figure 4.2.--Effects of practice with and without feedback on incorrect responses. For completeness, an identical analysis of the correct re- sponses of both groups was made. Figure 4.3 shows differences in shifts from correct responses made during the lecture to similar items on the test is the ggpg_for both groups. The X2 value is .094 which is not significant. This means that the incorrectness of the test answers is not related to the group membership of the respond- ents. In other words, feedback does not improve correct responding. 58 Workbook Without With Feedback Feedback wrong 29 23 right 155 112 :N (All Figure 4.3.--Effects of practice with and without feedback on correct responses. Figure 4.4 shows the percentage shift from workbook to test response. The percentage loss of correctness of response from lecture pause to test is the same for both groups and reflects forgetting identified by Maatsch (1975). Right on Right on % Group Workbook Test Loss With Feedback 135 112 .17 Without Feedback 184 155 .16 Figure 4.4.--Effects of practice with and without feedback on correct responses. This analysis of correct and incorrect responses supports Maatsch (1975) when he indicates that feedback may be most effective when it disrupts an incorrect response, and subsequently supplying a correct response, rather than when it restates what is already known. 59 Summar The stated problem, the effectiveness of the lecture method of instruction can be enhanced through the manipulation of variables, within the lecture format, to more actively involve students, was investigated. Data for the purposes of this study were analyzed by using a one—way analysis of variance, and a Scheffe post-hoc analysis of differences. The results of the hypotheses tested are as follows: 1. Students exposed to any of the pgpgg_lectures will show better performance as measured by a cognitive test score than stu- dents exposed to the nominal lecture. Accepted. 2. Students who are exposed to an "expert" lecturer using pacing will show better performance as measured by a cognitive test score than students exposed to a graduate student lecturer using pacing. Rejected. 3. Among the paced lectures presented by the "expert," stu- dents exposed to the maxi-paced lecture with practice and feedback will show better performance than students given the maxi-paced lec- ture with practice only, which in turn would enhance performance more so than the maxi-paced lecture. Rejected. 4. A X2 test of independence between correct and incorrect responses in the practice only and practice with feedback groups, for the effects of feedback, suggests that feedback to an incorrect re- sponse is more important than to a correct response. CHAPTER V SUMMARY; CONCLUSION AND DISCUSSION; IMPLICATIONS This chapter will present: (1) A brief summary of the study to this point; (2) a section on conclusion and discussion; and (3) implications of the study for both researchers and practitioners. mi Study Design The purpose of this study was to investigate whether lecture effectiveness could be enhanced by manipulating variables within the lecture format of instruction to more actively involve students. The seventy-four students, basically college freshmen, re- cruited from a number of introductory psychology classes at MSU, were divided into five groups: nominal lecture (nominal control), N = 21; paced lecture (paced control), N = 6; maxi-paced lecture, N = 18; maxi-paced lecture with practice, N = 17; maxi-paced lecture with practice and feedback, N = 12. The first two control groups were given a non-redundant, relatively monotonic lecture by a graduate assistant introduced as an experimenter. The latter three maxi groups received the same lecture content from a nationally recognized educa- tor and lecturer, possessing a more exciting and redundant style of 60 61 presentation. The "expert lecturer" was introduced as a MSU professor and expert in the field of Magic Squares to enhance his reputation in the eyes of the students. In addition, the "expert" gave an in- teresting history of the discovery and mathematical development of magic squares to enhance student interest. In the paced lecture, as contrasted to the nominal lecture, the lecturer presented information, i.e., a concept or rule, followed by a thirty-second pause during which the student was directed to think about what he/she had just been given in lecture. The maxi-paced lecture with practice group was given the same paced lecture and pauses, but during the pauses were told to practice, in a workbook provided, using the information given in lecture. The maxi-paced lecture with practice and feedback -group, again, received the paced lecture but with the inclusion of both practice and feedback of correct answers during the thirty- second pause. Each paced lecture was thirty minutes in length. Each of the maxi-paced lectures were controlled for consistency in repe- tition of information, in the use of advanced organizers, and in summary statements. No note taking, talking or questions were allowed in any of the groups. It was hypothesized that: 1. Students exposed to any paced lecture would show better performance as measured by a cognitive test score than students in the nominal lecture group; 2. The maxi-paced lecture presented by a "recognized expert" would further improve student performance over the paced control presented by the graduate student; 62 3. Within the maxi groups, the lecture with practice and feedback would increase students' score more so than the lecture with practice only, which in turn would enhance cognitive test score more so than the maxi-paced lecture. Study Results Data was collected and analyzed using a one-way analysis of variance and a Scheffé post-hoc analysis of differences. It was found with reference to the above hypotheses, that: l. A paced lecture can improve significantly performance of a student over the nominal lecture, as defined in this study;' 2. A paced lecture enhanced in a number of ways and presented by a "recognized expert," did not improve student performance sig- nificantly more than a paced lecture taught by a graduate student; 3. The paced lectures delivered by the "recognized expert," i.e., the maxi-paced, the maxi with practice, and the maxi with practice and feedback, did not order themselves, as hypothesized. 4. Because of the unexpected finding in 3, ax2 test of in- dependence of differences on error rate and the effect of feedback on these errors, in both the maxi with practice and maxi with prac- tice and feedback groups, was performed. The results of the analysis indicates feedback to be more effective when used to disrupt an incorrect response, than when used essentially to restate what is already correct. 63 Conclusion and Discussion Introduction This section presents a conclusion and discussion for each of the three hypotheses in this study. An attempt is made, as it should be at this point, to incorporate the earlier findings, impli- cations or suggestions of the literature reviewed in Chapter II. Hypothesis 1: Students exposed to any of the paced lectures will show better performance, as measured by a cognitive test score, than students exposed to the nominal lecture. Based upon the findings of this study, student performance can be enhanced, through the variable pacing, over student performance from a nominal lecture. This finding would seem to reflect and to support the findings of Budd-Rowe (1973), Aiken et a1. (1975), Coldevin (1975), Rippey (1975), Simon (1974), Taylor (1969), Maatsch (l975)--all of whom emphasize the importance of a pause or wait-time to allow cognitive processing to occur. A concern with the Aiken et a1. (1975) study--the intervening activity of letter cancellation to control for the additional time incurred with pacing and its inter- ference capability--is answered with this study. Indeed, pacing is an effective variable for the enhancement of student performance from lecture. This study would seem to contribute to the existing body of literature on pacing in two areas: (1) The effectiveness of pacing is shown transferable to an additional subject area. That is, Aiken's sharks, Rowe's elementary education, Coldevin's forest preservation now include this study's magic squares. (2) The length of the pause 64 for pacing is shown to be, again, variable: from three seconds to five seconds to as long as two and a half minutes. However, the shorter pause, if effective, would obviously allow for the presenta- tion of more information in a given time period; or allow for, time wise, a shorter presentation. The latter would be more in keeping with the suggested ideal length, thirty minutes, for presentation. Hypothesis 2: Students who are exposed to an "expert" lecturer using pacing will do better than students exposed to a graduate student using pacing, as measured by a cognitive test score. Based upon the findings of this study the enhancement effect of an "expert" lecturer and his accompanying style on student per- formance is not significantly greater than that of the graduate stu- dent. This is not to say that the importance of enthusiasm, suggest- ed by a number of studies (Chapter II), nor that the seductive style of a "Doctor Fox" is totally negated. Rather, that pacing is appar- ently able to account for, i.e., overcome, the effects of teaching style. The effect of pacing is more significant when the components of the "expert's" style as identified in Chapter III, p. 30, and allowed for, are considered: repetition advanced organization summary statements non-monotonic speech highly animated in movement enthusiasm for the subject plus a history for relevance 65 Again, pacing, itself, allows the graduate assistant's presentation to account for the additional input provided by the repetitions, et al., of the "expert." Another way of viewing this, is to consider the findings of Jersild (1928) and Larsen (1940), as reported by Verner and Dickenson (1967, p. 88) who suggest that "repetition of the most important points presented in lecture increases its effectiveness; however, four or five repetitions of a point tend to induce a 'law of dimin- ishing returns.‘ Whereas the "expert's" style was such as to allow for four repetitions plus a pause, the graduate assistant's presen- tation did not repeat the information while allowing for the pause. As such, a "law of diminishing returns" was not achieved, nor the level of interference which seems to be suggested by such a law. Hypothesis 3: The paced lectures presented by the "expert" will order themselves such that students exposed to the maxi-paced lecture with practice and feed- back will show better performance than students given the maxi-paced lecture with practice only, which in turn would enhance performance more than the maxi-paced lecture. Based upon the findings of this study, no significant differ- ences were found between the maxi lectures of the "expert" as mea- sured by cognitive test performance. Of import is the apparent lack of effect of either practice or practice with feedback, especially when both practice and feedback are generally held to be of major benefit to learning. As suggested in explanation of this non-signif- icant finding, in Chapter IV, p. 55, an analysis of workbook practice responses found 91 percent of the subjects of both the maxi with 66 practice and the maxi with practice and feedback groups to have responded correctly to a given item during the pauses. Since the error rate was only 9 percent, the magnitude of the effect on incor- rect responses of that group was minimal. Consequently, the mean group score on TOTAL RULES produced no significant differences be- tween the paced lecture with practice and feedback and the paced lec- ture with practice.1 It was suggested (Chapter IV, p. 55) that if the complexity level of the task was raised, or the frequency of pacing lowered, the error rate would rise to a level great enough to have feedback affect the mean score of that group, to the extent that significant differences would occur between the paced lecture with practice and the paced lecture with practice and feedback. McKeachie and Kulik (1975), as cited in Chapter II, suggest this when speaking of typical programmed learning units and inherent low probability of error: "We might expect more positive effects of feedback in situa- tions in which the task is more difficult and the motivation and the information provided are thus greater" (p. 197). If a more difficult task is interpreted as one in which more errors are made, than the results of the X2 analysis (Chapter IV) support the fact that feed- back would indeed have a more positive effect. Feedback, in more difficult situations, would convey more information and, as a conse- quence, draw more interest and attention. Maatsch (1975) seems to 1Nor was there significant difference from the maxi-paced lecture. This latter point, though not verifiable with actual re- sponses, however, may be hypothesized to follow the same pattern in that: (1) error rate (of a covert response in this case) was so low that the magnitude of the effect of feedback on incorrect responses would have been minimal; and (2) the additional rehearsal provided in the maxi with practice lecture would offer no real advantage to an already correct response. 67 suggest this when he indicates feedback must disrupt incorrect re- sponding; correct responses, as Gagne (1970) hints, are not in need of reinforcement: one does not seem to pay so much attention to what he has answered correctly, as to that which he has answered incorrectly when feedback is involved. In fact, Anderson, Kulhavy, and Andre (1971) suggest that when knowledge of the correct result is immediately available, students become inattentive and careless in trying to answer a question. Diminishing the frequency of pacing would have the proposed effect and, too, parallel the suggestion of McKeachie and Kulik (1975): ...the importance of knowledge of results depends on the infor- mation provided. Delayed knowledge of results may be more in- formative than immediate knowledge and may stimulate rehearsal and attention. Delayed knowledge of results should work as well as (or better than) immediate feedback...(p. 197). In summary, the findings of this study suggest that: (l) the lecture can be enhanced, specifically through the variable of pacing, and (2) to maximally enhance the lecture one ought to con- sider the paced lecture with both practice and feedback, especially if the subject matter is complex, the lecture is longer and pacing is used less frequently than in the present experiment. (3) This study found pacing to wash-out or cancel the enhancement effects associated with lecturer seductiveness (i.e., dynamics, reputation, style, charisma), repetition or advanced organization. 68 Implications For Research Based upon the findings of this study and questions raised during the investigation of the stated problem, a number of implica- tions, though by no means exhaustive, for further research are sug- gested and discussed. Would a less frequent use of pacing result in (1) increased percentage of incorrect responses and, subsequently, (2) an increase in the impact of feedback? As is suggested earlier in this study, feedback is most effec- tive when disrupting incorrect responses; it seems of little value when used in conjunction with correct responses, for it provides a truly low level of information in such cases. Less frequent pacing would possibly allow for feedback of more import and consequence. Would a cognitive task of a greater complexity than magic squares show similar enhancement potential for pacing in lecture? Though Magic Squares, as mentioned earlier, is reflective of a vast share of all school instruction-~elementary through higher education, including professional schools, in that all are essentially "rule" oriented tasks, nevertheless, it seems advisable to consider empirical information. Would different activities during the pause provided by pacing do as well or better than the findings of the present study? Practice (overt or covert) with or without feedback are only two possibilities for incorporation with pacing. Note taking or discussion during the pauses, during the lecture, or after lecture 69 are additional possibilities. Aiken's et a1. (1975) study needs, as mentioned, to be replicated. How would variations on the length of time for the pause, itself, affect the effectiveness of pacing? How well would the "expert" lecturer of this study do without pacing? In order not to inhibit his "style," the "expert" was given only a basic script for order of presentaiton and visuals for illus- tration. One of the things not foreseen was the extent of "variables" which constituted the "expert's" style: repetition; advanced organ- ization; summaries; enthusiasm; non-monotonic speech; gesturings; etc. It would be interesting to investigate the effectiveness of the "expert" without pacing, but with all else held equal; both against pacing and the nominal lecture of the graduate student. Could a lecturer, if given "free reign,“ be as effective or more effective than the pacing of this study? The lectures for this study follow a basic script-—the grad- uate student adhered to it explicitly; the "expert's" style added much in the way of repetition, etc. What kinds of things might a lecturer do, not given the constraints imposed (order of presentation, pacing, etc.), which might be as effective or more? A challenge match (no pun intended), in the best sense of the term, could be ex- pected. Is it possible to identify the essential nature of that which is defined as CONCEPTS, for this study? By a priori definition there were to be two dependent vari- ables for the cognitive measure: RULES and CONCEPTS. As shown in 70 Chapter III and discussed in Chapter IV, CONCEPTS was not used for purposes of this study. The nature of the problem, i.e., whether the dependent variable is characteristically uni-dimensional or multi- dimensional; or whether the problem lies with the measurement instru- ment, i.e., insensitivity--is not clear. Especially, when the follow- ing, Tables 5.1 and 5.2, are considered: TABLE 5.1.--Treatment groups, X'CONCEPT score, standard deviations, minimum and maximum CONCEPT scores. X' 0 Minimum Maximum Treatment GrOUP N Concepts Concepts Concepts Concepts Nominal Control 21 15.00 3.24 5 l9 Paced Control 6 15.50 4.04 9 l9 Maxi Paced 18 14.28 3.92 4 19 Maxi-With Practice 17 16.59 1.87 14 20 Maxi-With Practice and Feedback 12 16.25 2.80 9 l9 TABLE 5.2.--Analysis of variance of treatment groups on the variable CONCEPTS. Sum of Mean . Source df Squares Square F 519 Between 4 58.68 14.67' 1.46 .225 Within 69 695.48 10.08 Total 73 754.16 As Table 5.2 with its non-significant F test statistic indicates, no significant differences occur between the treatment groups on the de- pendent variable CONCEPTS. 71 These implications for research are not exhaustive by any means, but are suggestions for variations on a theme, for upping the N, and for empirically testing "my-gut-feel.“ Magic Squares, for any number of reasons, is a ready vehicle to such ends. It allows for that which McKeachie and Kulik (1975) call, as they cite Snow: We are becoming more aware that laboratory findings cannot be easily generalized to educational settings and that we must follow two converging approaches: on the one hand, we must add interacting variables to the basic independent variables studied in controlled experiments; on the other hand, we must gradually decompose naturally occurring clusters of variables important for education to determine which variables are most important. The process is halting and frustrating to those who want clear, easy answers now. Unfortunately, simple gen- eralizations are suspect (p. 199). For Practitioners The study comes, as it ought, full circle, to the original problem: can the effectiveness of the lecture method be enhanced through the inclusion of variables which more actively involve stu- dents? This study indicates that it can, specifically through the use of pacing, and, maximally, with the inclusion of practice and feedback. The intent when identifying variables for this study, was to select those variables which would fill two requirements: (1) be readily accessible, i.e., employable by the greatest number of in- structors, adaptable without the necessity and accompanying trauma of "style" alteration--at end, usable; and (2) satisfy the needs of both the aforementioned administrative and professorial justifications for the continued and inordinately widespread use of lecture. Though initially pacing would appear to involve more expense, nevertheless, (as noted in Chapter II) Rippey (1975) suggests that 72 the material in between pauses can be speeded up. In fact, “the most succinct and effective editorial advice is 'cut'" (p. 4). As effective as pacing may appear to be within the confines and subsequent limitations of this study, there are at least two points which need to be emphasized. The first, suggested under Lim- itations in Chapter I, is the effect over time of instruction: moti- vational influence for intervening activities is not accounted for in this study. The enthusiasm or style of an "expert" lecturer may indeed do all that research indicates, or it may be just an illusion which facilitates activity. However, the effectiveness of pauses or wait-time, as reported in the literature, suggests that even the ex- pert may want to consider the use of pacing, if for no other reason than to check himself. Secondly, pacing is not a cure-all for what inherently ails the lecture method of instruction, though it does fit within the con- straints and needs espoused by the voided justifications for lecture. As this study indicates, pacing enhances the effectiveness of the lecture method of instruction even when this method is used inappro- priately. For, most certainly, Magic Squares is a task for which: - the information acquired must be available through long term retention; - the content material is fairly complex and detailed; - learner participation in the learning activity is crucial to the achievement of the objective. Nevertheless, pacing does not cover for a highly questional side effect of lecture: students, and to some extent their teachers, 73 learn to regard knowledge as a closed system. McLeish (1968) suggests that: Where the object of education is to develop conformism to declared truths, especially where the totality of knowledge is believed to be embodied in an accepted text, the lecture system is clearly the preferred instrument, as in the medieval univer- sities (p. 47). Pacing, in the final analysis, facilitates some highly desired changes in the lecture method of instruction. As Rowe (1973) found, pacing does indeed: 1. increase the length of responses; 2. increase the number of unsolicited but appropriate re- sponses; 3. decrease failures to respond; 4. increase the incidences of speculative responses; 5. increases incidences of student-student comparisons of data; 6. increase incidences of evidence-inference statements; 7. increase the frequency of student-initiated questions; 8. increase the variety in type of verbal moves made by stu- dent (p. 203). Apparently the real payoff comes when the teacher keeps his mouth shut: "the moment of silence in the classroom may be the ONLY time when everyone is thinking...including the teacher" (Rippey, 1975, p. 3). In the final analysis, the most significant contribution which this study makes to higher education is to provide additional empir- ical evidence supporting the value of this moment of silence, for both the lecturer and his students. REFERENCES CITED 74 REFERENCES CITED Aiken, E.G., Thomas, 6.5., and Shennum, W.A. "Memory for a Lecture: Effects of Notes, Lecture Rate, and Information Density." Journal of Educational Psycholpgy. 1975, 67:439-444. Anderson, R.C., Kulhavy, R.W., and Andre, T. "Feedback Procedures in Programmed Instruction." Journal of Educational Psychology. 1971. 62:148-156. Beighley, K.C. "An Experimental Study of the Effect of Three Speech Variables on Listener Comprehension." Speech Monographs. 1954. 21:248-253. Blewett, T.T. "An Experiment in the Measurement of Listening at the College Level." Journal of Educational Research. XLIV (April 1951), pp. 575-585. Bligh, Donald A. What's the Use of Lectures? Boston, Mass: Penquin Books. 1972. Coats, W.D. and Smidchens, U. "Audience Recall as a Function of Speaker Dynamism." Journal of Educational Psycholpgy. 1966. 57:189-191. Cofer, C.N. (ed.). Verbal Learning and Verbal Behavior. New York: McGraw-Hill Book Company, Inc. 1961. Coldevin, 6.0. "Spaced, Massed, and Summary Treatments as Review Strategies for ITV Production." AV Communication Review. Fall 1975. 23:289-303. Costin, Frank. "Lecturing Versus Other Methods of Teaching: A Review of Research." British Journal of Educational Technology, No. 1, Vol. 3. January 1972, pp. 4-31. Divesta, F.J. and Gray. S.G. "Listening and Note Taking: II. Imme- diate and Delayed Recall as Functions of Variations in The- matic Continuity, Note-Taking, and Length of Listening-Review Intervals." Journal of Educational Psychology, 1973. 64: 278-287. Freyberg. P.S. "The Effectiveness of Note Taking." Education for Teaching. 1965 (February). pp. 17-24. 75 76 Gage, N.L. "Teaching Methods.“ Encyclopedia of Educational Research. Fourth Edition. 1969. pp. 1446-1458. Gagne, R.M. The Conditions of Learnipg, New York: Holte, Rinehart, and Winston, Inc. 1970. Glass, G.V. and Stanley, J.C. Statistical Methods in Education and Psychology, Englewood Cliffs, New Jersey: Prentice Hall, Inc. 1970. Hartley, J. and Cameron, A. "Some Observations on the Efficiency of Lecturing." Educational Review. November 1967. 20:30-37. Highlander, J.P. "Audience Analyzer Measurements and Informational Effects of Speaker Variables in Radio Talks." Abstract in Speech Monoggaphs. XXI. August 1954. pp. 188-189. Holcomb, J.D. and Garver, A.E. Improving Teaching in Medical Schools (A Practical Handbook). Springfield, Illinois: Charles C. Thomas. 1973. Havland, C.I. "Human Learning and Retention." S.S. Stevens (ed.). Handbook of Experimental Psychology, New York: John Wiley and Sons, Inc. 1951. Chapter XVII. Jenkins, J.J. "Remember That Old Theory of Memory? Well, Forget It!" American Psychologist. November, 1974. pp. 785-795. Jones, H.E. "Experimental Studies of College Teaching: The Effect of Examination on Permanence of Learning." R.S. Woodworth (ed). Archives of Psychology, No. 68. November 1923. pp. 5-70. Maatsch, Jack L. "VIM I: The Effects of Instructional Models on Comprehension and Retention of Several Dependent Performance Variables." Paper presented at RIME, AAMC. 5 November 1975. Mastin, V.E. "Teacher Enthusiasm.“ Journal of Education Research. 1963. 56:385-386. McClendon, P.I. "An Experimental Study of the Relationship Between the Note-Taking Practices and Listening Comprehension of College Freshmen During Expository Lectures." Speech Mono- graphs. 1958. 25:222-228. McKeachie, W.J. "Research on Teaching at the College and University Level.“ N.L. Gage (ed.) Handbook of Research on Teaching. Chicago: Rand McNally. 1963. pp. 1118-1164. McKeachie, W.J. and Kulik, J.A. "Effective College Teaching." Review of Research in Education. 1975. 3:165-209. 77 McLeish, J. The Lecture Method. Cambridge Monographs on Teaching Methods. No. 1. Cambridge, England: Cambridge Institute of Education. 1968. Naftulin, D.H., Ware, J.E., and Donnelly, F.A. "The Doctor Fox Lecture: A Paradigm of Educational Seducation." Journal of Medical Education. Vol. 48. July 1973. pp. 630-635. Paulsen, R. The German Universities. London: Longmans. 1906. Paulson, S.F. "The Effects of the Prestige of the Speaker and Ack- nowledgement of Opposing Arguments on Audience Retention and Shift of Opinion." Speech Monographs. XXI. November 1954. pp. 267-271. Petrie, C.R. "Informative Speaking: A Summary and Bibliography of Related Research." Speech Monographs. 1963. 30:79-91. Rippey, R.M. "The Ginther Model: Four Dimensions of Research on Instruction." The Elementary School Journal. January 1969. pp. 215-223. Rippey, R.M. Reaction to VIM Presentations of Maatsch, Hoban, Holmes, Budd, RIME, AAMC. 5 November 1975. Rowe, M. Budd. "Pausing Phenomena: Influence on the Quality of Instruction." Journal of Psycholinguistic Research. 1974. pp. 203-224. Sherman, B.R. and Blackburn, R.T. "Personal Characteristics and Teaching Effectiveness of College Faculty." Journal of Educational Psyehology. 1975. 67:124-131. Simon, H.A. “How Big is a Chunk?" Science. February 1974. 183: 482-488. Taylor, I. "Content and Structure in Sentence Production." Journal of Verbal Learning and Verbal Behavior. 1969. 8:170-175. Thiagarajan, S. The Programming Process: A Practical Guide. Bel- mont, California: Wadsworth Publishing Company, Inc. 1971. Thompson, W.N. Quantitative Research in Public Address and Communica- tion. New York: Random. 71967. Trenaman, Joseph. "Understanding Radio Talks." Quarterly Journal of Speech. XXXVII. April 1951. pp. 173-178. University Grants Committee. University Teaching Methods_(Hale Report). HMSO. 1964. 78 Verner, Coolie and Dickinson, Gary. “The Lecture, an Analysis and Review of Research." Adult Education. Winter 1967. pp. 85- 100. Ware, J.E., and Williams, R.G. "The Dr. Fox Effect: A Study of Lecturer Effectiveness and Ratings of Instruction." Journal of Medical Education. Vol. 50. February 1975. pp. 149-156. APPENDICES 79 APPENDIX A Lecture Script 80 APPENDIX A INSTRUCTIONAL NARRATION FOR LECTURE (Modeling Thiagaragon Sequence) This is an instructional unit on what are magic squares and how to make them. Magic squares are squares that have been subdi- vided into rows and columns of smaller squares or cells. These cells are fitted with numbers and if the cell entries have a particular relationship the figure is called a magic square. Let us now see what the relationship must be. Fig. 1. This visual illustrates one rule of magic squares, that is the row totals must be equal. Therefore, the two figures on the left could not be magic squares. Fig. 2. This visual indicates another rule. The common totals must equal each other and also equal the row totals. Fig. 3. Also, as shown here, the diagonal totals must be equal to each other and to both the column and row totals. We now have essentially the rules for what a magic square is with one exception. There is a figure that will sum up in a similar way as magic square but is conventionally not regarded as one. Fig. 4. The rule is if a figure contains duplicate numbers, it can- not be called a magic square. 81 82 Fig. 5. Another clue that quickly eliminates some figures is illus- trated here. Notice that the figures on the left contain only even numbered rows and columns. Therefore, only odd numbered rows and columned figures like these containing 3x3 or 5x5 rows and columns can be magic squares. Additionally the number of rows must equal the number of columns. A general rule, then, for magic squares is: A magic square can contain no duplicate numbers and the totals for rows, columns, and diagonals must be equal. Now that you know what a magic square is you will learn to construct one. This task consists of two parts.- The first consists of knowing what kind of numbers can be used. Fig. 6. This visual illustrates four rules for numbers. Comparing both A series we find that only an ascending order can be used. Notice next that all the series on the right side maintain the same interval between adjacent numbers, in A the interval is l, in B it is 2, and in C it is 4. Looking at the first number of all positive examples, we can see that a number series may begin with any positive number. The final task we will deal with in this unit is how to assign a number series to cells to form a magic square. There are six rules to use here. These rules will be illustrated using a 1, 2, 3, 4, etc. number series. The first rule covers the most common case. More than any other situation one deals with a diagonal number assignment. The rule is: 83 Fig. 7. When an empty cell exists one space above and to the right Fig. Fig. of the last known number, the following number is placed in this empty cell. The figure on the right indicates the correct ap- plication of the diagonal rule. 8. The next rule specifies that if a cell exists one space above and to the right of the last placed number (in this case #5) but that cell is already occupied, (in this case by #l) the following number (i.e., #6) is placed directly below the last known number. This is the Exception to the Diagonal Rule. 9. In the process of moving diagonally upwards one will even- tually find himself in the top row. The general rule for this situation is illustrated here. If a known number is in top row, the following one is placed in the bottom row, one column to the right. This is called the Top to Bottom Rule. Of course, this rule cannot be applied when one reaches the upper right most cell because there does not exist a space to its right for the placement of the following number. Fig. Fig. l0. Another common situation is finding one's self in the right column as illustrated in this visual. The rule here is that the following number is placed one row above and in the left most column. The correct placement is again illustrated in the figure on the right. This is called the Right to Left Rule. ll. This cell has a rule all its own. When the last known entry is in the most upper right hand cell, the following number is placed in the cell directly below. This is called the gppg[_ Right Corner Rule. 84 Fig. 12. The last rule covers another special case--the entry of the first number. As indicated in this visual, the first entry is always placed in the upper most cell of the center column. This is called the First Number Rule. All the rules for constructing a magic square have now been illustrated. APPENDIX B A History of Magic Squares 85 APPENDIX B A History of Magic Squares Magic squares were first discovered in China about 2200 B.C. Legend has it that King Yu, ruler of all China, while on one of his daily walks along the Yellow River, chanced upon a turtle-~a turtle which but for its shell was much like any other turtle. This turtle's shell, upon examination, was found to contain nine (9) squares (3x3). In each square were dots ranging from one to nine and no two squares had the same number of dots. More amazing however was the discovery that when the dots in any column, or row, or diagonal were added, each was found to contain 15 dots. Further, any two symmetrical squares contained the same number of dots. The pattern of the dots, later called Lo-shu, was thought to be of mystical significance and began appearing on charms and magic stones. (The little turtle lived happily everafter.) News of the magic squares spread from China to India and Japan. Here a great number of studies and writings were produced prior to the first centruy A.D.; too, the pattern of dots appeared on charms and decorations. During this period of development a Japanese philosopher, Yokohoma, used the increasing knowledge of fifth (5th) order magic squares (5x5) to explain the intricate loop patterns so necessary to the development of silk looms. 86 87 Magic squares were not introduced into Western culture until the l400's--reputedly by one Euramel Muchopolus--centuries after they were discovered in the East. In the year l5l4, whether by intent or accident, Albrecht Durer constructed a magic square of the fourth (4th) order (i.e., 4x4) with the date l5l4 in the two bottom cells. Also during the century, a Greek, Aeneas Plinus, is credited with the writing of a little known study in which he applied the Pythagorean theorem and the limited understanding of the third (3rd) order magic squares to explain the structure of common vertices in polyhedrons (i.e., pyramids). Nevertheless, not much was known or understood about magic squares in Western culture until the l7th century. At that time mathematical construction of magic squares was begun in earnest by French mathematicians and philosophers. During this period 880 fourth order and over a half-million fifth order magic squares were dis- covered. Since the l7th century a great deal of work and many studies have been made of magic squares. The result has been productive in numbers as well as in kinds of magic squares. We now have: pure magic squares assoicate squares border squares composite squares composite border squares simple squares reversible squares 88 There are even: double and treble squares upside down squares diabolic squares semi-diabolic squares These have not been just academic exercises, either. An ex- tension of the knowledge of diabolic and semi-diabolic squares has been essential in the understanding and application of structural vectors and stress factors so necessary to the construction of in- creasingly taller buildings. Witness the Sears building in Chicago and the even taller Canadian National Railway Tower, recently com- pleted in Toronto, Ontario. All originating from the shell of a single, mystical turtle: amazing. APPENDIX C Test Instrument 89 VARIABLES IN INSTRUCTIONAL METHODS TEST FOR MAGIC SQUARES Name Telephone Number Age Major Sex 1. Before this instructional session, I had: (check one) constructed a Magic Square and knew the rules. constructed a Magic Square but forgot how. been shown how, but have never constructed one. seen one, but didn't know how to construct one. never seen anything like a Magic Square. U'Ith-J II. For me, the instructional session was: (check the place on the scale that best reflects your feeling) pleasant L J ,J J unpleasant li | 2* l3 {ITV ES clear . J» I J, confusing l 2' l3’ -l’4 fl? easy I J J J difficult l [2 l3 l’4 l5 exciting .J It | J; boring 1 l2 [3 I 4 l5 efficient J JJ J JJ inefficient 1 ’l27 l3 1T4 I5 III. I would like this type of instructional method: all the time | l . J never again 1 [2 J3 ]*4 5 STOP UNTIL INSTRUCTED TO CONTINUE 90 91 Try to construct a magic square. First select the correct number series from the alternatives listed below. Secondly, choose the correct empty magic square from the alternatives below. Finally, usinggthe correct number series,,fill out the empty magic square that you’have selected. If you have forgotten how to pTace any number, guess andicircle your guess. Then continue filling out the magic square the best you can. Choose the Correct Number Series A. l, 2, 4, 7 ... D. 100, 99, 98, 97 ... B. 2, 4, 4, 5, 6, 6, 7 ... E. -l, -2, -3, -4 ... C. 3, 5, 7, 9 ... F. -2, -l, O, l, 2 ... Choose the Correct Empty Magic Square 92 2. Draw a magic square without numbers that has between 20-30 cells. 3. Generate three completely different number series that could be used in magic squares. 4. A magic square that has between 70 and 100 cells must have number of columns and number of rows. 93 In questions 4 through 16 you will find a square and some numbers. Try to place the number appearing to the right of the square in its proper cell to form a magic square. (Assume that a l, 2, 3, 4 ...'series is being used). l9 20 22 23 94 l6 l5 l7 )6 IO. 95 20 21 ll. 12. 96 36 TO 37 ll l3. 14. 97 20 25 Zl 26 15. 16. 98 15 16 17. 99 Place the name of the number assignment rule in the first blank, indicate if this applies to the first or next number in the second blank and describe where the next number is placed. If you can't recall the name of the rule describe where the rule is applied and how the next number is placed. The rule involves placing first/next number (CircTe one)* (describe Where)‘ The rule involves placing first/next number (circle one)" (describe where) The rule involves placing first/next_ number (Eiréle one) (describe where) The rule involves placing first/next number (circle one) ‘"(describe where) r” The rule involves placing first/next number (circle one) ’(describe where) L. The rule involves placing first/next number (Circle one)’ (describe wheré) 18. 19. 20. 100 List the rules that determine whether an empty square (no numbers) could be used to form a complete magic square. List the rules used to generate a number series that could be used in a complete magic square. List the rules that are used to determine if a filled in square f” is a magic square. .n‘. 4 101 21. In questions 21 through 32 try to select the square that correctly places the largest number in each box. (Assume a l, 2, 3, 4 ... number series has been used.) Circle the letter for the figure you have chosen. A. B. 8 7 8 9 A C. D. J 8 7 9 6 A, B 22 8 5 15 7 .227 8 C D. 5 5 7 7 8 8 23. 24. 102 in) 25. 26. 103 11 12 11 12 11 12 11 12 L 3 D 27. 28. 104 10 11 11 1O A. 10 11 C. 11 10 26 27 27 26 27 26 27 105 29. 18 17 18 17 18 17 18 17 30. 1 1i Ilfgmmfvirm N 1.... 5‘ 31. A. 3 4 C. 4 3 32. A. 8 13 14 C. 8 13 4 3 g. 14 8 13 D! 8 13 14 14