III I”!!! W! U"(Ulflflclfllilfllfllllflllflilllllallflll 3 1293 72 35 ‘Qc‘ ‘A. -1,‘ F4 ‘1 E PU 1|“ ABSTRACT THE EFFECT OF STUDENT-CONTROLLED INSTRUCTION ON LEARNING BY John Pray Fry A number of areas of psychology have shown self- direction to be a motivating, satisfying, and an effective mode of interacting with one's environment. Despite the enthusiasm of educators for progressive methods (e.g., Summerhill), there is only limited and questionable evi- dence that self-direction is a motivating, satisfying, or an effective mode of interacting within a learning environ- ment. Moreover, the degree of effective learner self- direction probably depends on the individual learning style of the learner. The purpose of this investigation was to test the hypothesis that inquisitive individuals can control their learning or instructional strategies and thereby learn more and be more satisfied than by conventional means. In a manually simulated learner-computer environment, 192 55 were given degrees of control over the programming of John Pray Fry their own learning. The material learned was about "Com- puters and How They Work," and was recorded on 52 one- minute video tape segments. This information was presented on a TV monitor under the following conditions: a. "Student-Controlled Instruction" (SCI)--Ss were given the opportunity to select the sequence of presentation of the segments. After each seg- ment 83 could ask questions about the material. b. "Expert"--Ss viewed a fixed sequence of the segments prepared by content experts. 55 were not allowed to ask questions. Instead, each S heard the questions asked by one of the "SCI" Ss. c. "Random"--Ss viewed a randomly ordered sequence of the segments. Otherwise, 53 were treated the same as "Expert" 85. d. "Control"--Ss received no instruction. It was hypothesized that only 53 high in "inquisi— tiveness" would be able to take full advantage of the "SCI" treatment. Therefore, all Ss were given a battery of six tests to measure individual differences along this dimen- sion. Eighteen measures of aptitude, interest, and learner characteristics were also obtained for each S. The relative effectiveness of the treatments was determined by scores on a pretest, posttest, and retention .u -.-o An- .5 :-- a... y 1 v.1. O u .N‘ John Pray Fry test. In addition, ten measures-of attitude and motivation were obtained for each S. The predicted interaction between degree of "inquisitiveness" and degree of control was confirmed, but only for high aptitude Ss. High aptitude, high "inquiry" 85 learned more under the "Expert" treatment. Results for low aptitude Ss were inconclusive. Among the attitude measures, "SCI" 85 responded, overall, with a more favorable attitude toward the method of learning than other groups. Among the correlational analyses, the more questions "SCI" 55 asked, the more they learned and the more they liked the method of learning. I The results support the hypothesis that different types of students learn better (or worse) under different degrees of student control. Failure of some "SCI" 85 to perform better is attributed to their lack of familiarity with the environment. THE EFFECT OF STUDENT-CONTROLLED INSTRUCTION ON LEARNING BY John Pray Fry 3W: A THESIS Submitted to Michigan State University in partial fulfillment of the requirements for the degree of DOCTOR OF PHILOSOPHY Department of Psychology 1970 '1 (U .0. “ d 0'1 F1 ’h (I) t 7 In ACKNOWLEDGMENTS The author wishes to express his appreciation to Dr. Robert H. Davis for his inspiration and guidance in this research and to Drs. L. T. Alexander, T. W. Forbes, E. Jacobson, and F. R. Wickert whose suggestions were of great value. I am particularly indebted to Robert Goodman for his suggestions and assistance during the critical phases of data collection. Jackie Koehler and Traute Reimer deserve recognition for administering the "Attitude Survey" questionnaires and for typing the many forms required. I am also grateful to Stan Cohen, Gerald Gillmore, Stuart Thomas, Dr. R. D. Hart, Dr. F. E. LeCureaux, and Dr. F. B. Martin for their serving as judges. Special thanks go to Robert Lewis for his enthu- siastic performace as a computer science instructor, even though he did not understand what he was saying. I would also like to thank all of the students who gave of their time and energy so freely and who appreciated so much the goal of this research. Finally, I am so obligated to my wife, Pam, that she now feels free to do similar research. ii TABLE OF CONTENTS LIST OF TABLES . . . . . . . . . . . LIST OF FIGURES . . . . . . . . . . Chapter I. INTRODUCTION . . . . . . . . II. REVIEW OF THE LITERATURE . . III. Research on Student-Controlled Instruction . . . . . . . . The Research of Mager . . The Research of Grubb and Selfridge . The Research of Campbell . Research Findings and Theories from other Areas of Psychology . . . . Learning Theory . . . . . Organizational Psychology Engineering Psychology . . Humanistic Theory . . . . Individual Differences . . The Learner/Instructor Control Dimension . Instructional Functions . Individual Differences and Controlled Instruction . An Environment Where Student Can Function Effectively . . METHOD AND PROCEDURES . . . Experimental Design . . . . Factor One . . . . . . . . Factor Two . . . . . . . . Factor Three . . . . . . Student- Control Stratified Random Assignment . . . . . . Pilot Experiment . . . . . iii Page vii 10 ll 17 18 28 3O 31 32 43 44 48 49 53 53 54 59 59 61 62 m ”V 5.. Chapter Page Subjects . . . . . . . . . . . . . . . . . . 63 Pre-Experimental Measures (Independent Variables) . . . . . . . . . . . . . . . . . 64 Shulman's Six Tests . . . . . . . . . . . 65 General Inventory . . . . . . . . . . . . 67 "Attitudes Toward Learning" . . . . . . . 69 College Aptitude Tests . . . . . . . . . . 70 Experimental Procedures . . . . . . . . . . 71 First Learning Session . . . . . . . . . . 71 Second Learning Session . . . . . . . . . 72 Subject Logs . . . . . . . . . . . . . . . 73 Retention Test Session . . . . . . . . . . 74 Control Subjects . . . . . . . . . . 75 Development of the Subject Matter . . . . . 76 Development of the Segments . . . . . . . 77 Taping of the Segments . . . . . . . . . . 78 Ordering of the Segments . . . . . . . . . 79 Measurement of the Dependent Variables . . . 81 Development of the Tests . . . . . . . . . 81 "Post-Attitude Survey" . . . . . . . . . . 85 "Terminal Interview" . . . . . . . . . . . 86 "Post-Interest in Computers" . . . . . . . 86 Card Sequencing . . . . . . . . . . . . . 87 IV. RESULTS . . . . . . . . . . . . . . . . . . 88 Analysis of Variance . . . . . . . . . . 88 Gain 1 (Posttest Minus Pretest) and Gain 2 (Retention Minus Pretest) . . . . 91 Time Expended in Learning Sessions by Ss . . . . . . . . . . . . . . 96 Number of Pages of Notes Taken by 85 . . . 99 Time Expended on Notes by Ss . . . . . . . 101 Perceived Learning as a Function of the Environment . . . . . . . . . . 101 Attitude Toward Order of the Segments . . 104 Attitude Toward Method of Learning . . . . 106 Perceived Learning as a Function of Task Difficulty . . . . . . . . . . . 108 Reviewing or Relating Material to Past Experience . . . . . . . . . . . . . . . 110 Attitude Toward Computers . . . . . 112 Number of Questions Asked by "SCI" Ss . . 112 Number of Segments Repeated by "SCI" 53 . 114 Summary of AOV Results . . . . . . . . . . . ll4 iv v.3 'w Chapter Correlational Analysis . . . . . . . . . . Attitude Toward Learning . . . . . . . . Test Anxiety . . . . . . . . . . . . Number of Questions Asked . . . . . . . Time Expended in Learning Sessions . . . Number of Notes Taken . . . . . . . . . Time Expended on Notes . . . . . . . . . Pre-Interest in Computers . . . . . . . Miscellaneous Findings from the 3 x 4 Matrices . . . . . . . . . . . . . . . Summary of Correlational Analysis Results Comparison of Responses to the "Terminal IntervieW" Questionnaire . . . . . . . Agreement Among Rank- -Orderings of the Segments . . . . . . . . . . . . . . . . . DISCUSSION . . . . . . . . . . . . . . . Interaction Between Treatments and Subject-Types . . . . . . . . . . . . . Performance of Ss in the "SCI" Treatment . Performance of Ss in the "Random" Treatment . . . . . . . . . Performance of Low Aptitude/High Inquiry "SCI" Subjects . . . . . . . . . . . . . The Effect of Asking Questions . . . . . . Learning Material . . . . . . . . . . . . Concluding Remarks . . . . . . . . . . . . LIST OF REFERENCES 0 O O O O O O O O O C O O C O O APPENDICES . . . . . . . . . . . . . . . . . . . . A. CODING OF THE CARDS . . . . . . . . . B. "EXPERT" LIST OF QUESTIONS . . . . . . C. "RANDOM" LIST OF QUESTIONS . . . . . . D. INFORMATION SHEET . . . . . . . . . . E. MAP TO LABORATORY . . . . . . . . . F. ATTITUDE SURVEY TEST BATTERY . . . . . General Inventory (biographical data). Word Association . . . . . . . . . . Political Position . . . . . . . . . Attitudes Toward Learning. . . . . . Inventory of Beliefs . . . . . . . . Attitude Inventory (complexity scale) Attitudes Toward Tests . . . . . . . V Page 118 122 125 125 129 131 133 135 137 138 140 148 151 151 154 156 157 158 159 160 162 170 170 173 175 178 180 181 183 185 187 188 192 199 201 Chapter EXCERPT FROM MICHIGAN STATE UNIVERSITY READING TEST . . . . . . . . . . . . THREE EQUIVALENT FORM 60 ITEM TESTS. . INSTRUCTIONS . . . . . . . . . . . . . SUBJECT LOGS . . . . . . . . . . . . . E'S CODE CONVERSION LIST . . . . . . . SUBJECT MATTER SEGMENTS . . . . . . POST-ATTITUDE SURVEY . . . . . . . . . TERMINAL INTERVIEW . . . . . . . . . . TABLE OF INTERCORRELATIONS BETWEEN ALL VARIABLES . . . . . . . . . . . RAW DATA FOR ALL SS . . . . . . . . . vi Page 204 206 239 247 250 251 309 316 320 325 Table 1. 10. 11. LIST OF TABLES Number of $5 for Each Cell of Experimental Design . . . . . . . . . . . Subject Characteristics . . . . . . . . . . Proportion of Variance Accounted for by Factors in "Attitudes Toward Learning" Questionnaire . . . . . . . . . . . . . . Rank-Order Correlations Between Rankings Assigned to Segments by Pairs of Expert Judges I O O O O O O O O O O O O O O O O Coefficient of Concordance and Average Intercorrelations Among Rankings Assigned to Segements by Expert Judges . . . . . . Change in Test Reliability After Deletion Of Items 0 O O O O O O I O O I O O O O O PrOportions of Variance Accounted for by Factors in "Post-Attitude Survey" . . . . Intercorrelations Between Gain Scores for Instructional Treatment and Subject- Type cells I O O O O O O O O O O O O O 0 Means and Standard Deviations, 2 x 2 x 4 Factorial AOV, and Four One-Way AOV of Gain 1 Scores 0 O O O O O O O O I O O O I Means and Standard Deviations, 2 x 2 x 4 Factorial AOV, and Four One-Way AOV of Gain 2 Scores . . . . . . . . . . . . . . Means and Standard Deviations, 2 x 2 x 4 Factorial AOV, and Four One-Way AOV of Time Expended in Learning Sessions by Ss vii Page 55 64 70 80 80 84 85 90 93 94 97 - V In I‘ if. b- '74 Table 12. 13. 14. 15. 16. 17. 18. 19. 20. 21. 22. 23. Means and Standard Deviations, 2 x 2 x 4 Factorial AOV, and Four One-Way AOV of Number of Pages of Notes Taken by Ss . Means and Standard Deviations, 2 x 2 x 4 Factorial AOV, and Four One-Way AOV of Time Expended on Notes by Ss . . . . . Means and Standard Deviations, 2 x 2 x 4 Factorial AOV, and Four One-Way AOV of Perceived Learning as a Function of the Environment Scores . . . . . . . . . . Means and Standard Deviations, 2 x 2 x 4 Factorial AOV, and Four One-Way AOV of Attitude Toward Order of the Segments Scores . . . . . . . . . . . . . . . . Means and Standard Deviations, 2 x 2 x 4 Factorial AOV, and Four One-Way AOV of Attitude Toward Method of Learning Scores . . . . . . . . . . . . . . . . Means and Standard Deviations, 2 x 2 x 4 Factorial AOV, and Four One-Way AOV of Perceived Learning as a Function of Task Difficulty Scores . . . . . . . . . . Means and Standard Deviations, 2 x 2 x 4 Factorial AOV, and Four One-Way AOV of Reviewing or Relating Material to Past Experiences Scores . . . . . . . . . . Means and Standard Deviations, 2 x 2 x 4 Factorial AOV, and Four One-Way AOV of Attitude Toward Computers . . . . . . Means, Standard Deviations, and Range of the Number of Questions Asked by "SCI" 85 Within Each Subject-Type Group . . Analysis of Variance of Number of Questions Asked by "SCI" 55 . . . . . . . . . . Means, Standard Deviation, and Range of the Number of Segments Repeated by "SCI" Ss Within Each Subject-Type Group . . Analysis of Variance of Number of Segments Repeated by "SCI" Ss . . . . . . . . viii Page 100 102 103 105 107 109 111 113 115 115 116 116 Table 24. 25. 26. 27. 28. 29. 30. 31. 32. 33. Overall Correlations Between Individual Difference Measures and Criterion-Test Scores Within Each Treatment Group . . . . Correlations Between Attitudes Toward Learning and Criterion Scores Within Treatment and Subject-Type Groups and the Significance of Differences Between These Correlations . . . . . . . . . . . . Correlations Between Test Anxiety and Criterion Scores Within Treatment and Subject-Type Groups and the Significance of Differences Between These Correlations. Correlations Between Number of Questions Asked and Criterion Scores Within Treatment and Subject-Type Groups and the Significance of Differences Between These Correlations. Correlations Between Time Expended in Learning Sessions and Criterion Scores Within Treatment and Subject-Type Groups and the Significance of Differences Between These Correlations . . . . . . . . Correlations Between Number of Notes Taken and Criterion Scores Within Treatment and Subject-Type Groups and the Significance of Differences Between These Correlations . . . . . . . . . . . . . . . Correlations Between Time Expended on Notes and Criterion Scores Within Treatment and Subject-Type Groups and the Significance of Differences Between These Correlations . . . . . . . . . . . . . . . Correlations Between Pre-Interest in Computers and Criterion Scores Within Treatment and Subject-Type Groups and the Significance of Differences Between These Correlations . . . . . . . . . . . . Frequency and Percentage of 53' Response to Items of "Terminal Interview" Questionnaire Within Instructional Treatment Groups . . . . . . . . . . . . . Agreement Among Rank—Orderings of the Lecture Segments . . . . . . . . . . . . . ix Page 120 123 126 127 130 132 134 136 144 149 Figure 1. 10. 11. LIST OF FIGURES The Degree of Learner/Instructor Control for Selected Instructional Functions Over Selected Referent Researchers . . . Carrel Configuration: TV Monitor, Earphones, and Microphone . . . . . . . . . . . . . Control Room Configuration: Video Tape Recorders, TV Monitors, Patch Panels, TV Camera (Lens Only), Microphone, and Earphones . . . . . . . . . . . . . . . Scatterplot of Experimental Subjects in Terms of Their Aptitude and "Inquiry" scores 0 O O O O O O O O O O O O O O O 0 Mean Gain Scores for Instructional Treat- ments Within Each of Four Subject-Type Groups . . . . . . . . . . . . . . . . . Mean Time Expended by 85 for Instructional Treatments Over Four Subject-Type Groups Mean Attitude Toward Order of Segments Scores for Instructional Treatments Over Four Subject-Type Groups . . . . . . . . Mean Attitude Toward Method of Instruction Scores for Instructional Treatments Over Four Subject-Type Groups . . . . . . . . Mean Perceived Learning as a Function of Task Difficulty Scores for Instructional Treatments Over Four Subject-Type Groups Mean Gain 1 Scores for Subject-Types Over Three Treatment Groups . . . . . . . . Mean Gain 2 Scores for Subject-Types Over Three Treatment Groups . . . . . . . . X Page 45 56 56 62 98 99 106 108 110 152 152 Figure Page 12. Example of Typical Card . . . . . . . . . . . 172 13. Example of "Overview" Card . . . . . . . . . 172 xi .P.’ “'Vh Q- l); .w HA ‘V CHAPTER I INTRODUCTION A number of areas of psychology have shown self- direction to be a motivating, satisfying, and an effective mode of interacting with one's environment. There is only limited and questionable evidence that this is true for learning environments. The purpose of this investigation 'was to test the hypothesis that certain types of individuals can control their own learning or instructional strategies and thereby learn more and be more satisfied than by conventional means. Many psychologists and educators have pointed out the need to shape training programs to the wide range of individual differences among students. Crawford (1962), for example, has pointed out that because of limitations in available manpower and because of imperfect selection procedures, there is great heterogeneity of input into training programs designed with rigid output specifications. Along with many other specialists, he has emphasized that the adaptation of training to the interests, aptitudes, motivations, and background characteristics of trainees constitutes a major research problem. 1 Today, individualized computer-assisted instruction (CAI), adaptive to the individual needs of each learner, appears to be the goal of instructional system designers (Mitzel, 1970). However, in designing a CAI system, it is necessary to decide if the learner should control the programming of the system, if an instructor should control the system for the learner, or if some mix of the two is best. A few studies dealing with student-controlled instruction have been carried out. Mager (1961), Mager & McCann (1961), and Grubb & Selfridge (1964) have produced striking results and claim that student control increases learning effectiveness. However, they either failed to use «control groups or included other variables which may have «accounted for the observed results. On the other hand, ‘the only investigator in this area who has done controlled o mcowuocsm c h 0 HH X X X X X X HQ¢BOB .mm>«uommno mo unmeaflmuum HMCaEuwB .Acowume luwusoo no ucmEmOHomchuv mommoonm mcflsumma mm muasmwu mo mmcmasosx HmsowposuumsH cmuomawm How Houucoo HouosnumcH\Hmchmq mo mmummo m£BII.H madman HH .HH X >< X>< X >4 X>< X >< X>< X >< X >< X>< .xcaum Hmwpmamu\3mw>mm .cowumowammm\00fluomum .mmUMDOmmH wo mnwswsfi no Amcflsov maficowummno .ucmucoo mo mcflomm .usmucoo mo mcwocmsqmm .GOWumucwmwum mo mafim uwcs\m608\©osumz X X X X X X X X X X X X .umcumma mo Hw>ma muucm .maw>ma maumufiuo can mm>wuomnno mo cowuwadmma Amusu Iosuum Hamum>ov swamp can moan pumpsoo mo coauflcwmmo r. a 1m 1 n o ‘4. o J 1% m1 p h. .w q a .m 1 a. J 71 .1 b... .w w h A. mu 2 .m Houucoo HouosuumsH Esaaxmz Hmnsmsd Houucoo HouosuumcH mmmn Nmmmwhm spam mo onsuxaz mhwmom Houucoo Houucoo Hmsnmmq Hmsummq mmmq Essaxmz Hawnmemo Homo: mcowuocsm HmsowuosuumcH 46 self-explanatory. Function seven is "questioning (cuing) if by an instructor and "inquiry" if by the learner. The last two functions in Figure 1 refer to the evaluation of learning; the first as the learner learns and the second at some specified terminal point. Other instructional functions which are rather difficult to attribute to either the learner or the instruc- tor include: motivation, attention, and implementation of mediation of concepts. Magg£.--Research by Mager (1961; Mager & Clark, 1963; Mager & McCann, 1961) has been highlighted by arrange- ments under which the learner is given much responsibility and control over what is to be learned and how it is to be learned.” "Independent study programs" and "independent research" are familiar examples. The learner proceeds entirely on his individual initiative in selecting objectives, materials, study methods, and sequences of activity. If an instructor is present, he acts only as an additional source of information or confirmation. Campbell.--This researcher (Campbell, 1964) set the content, objectives and methods of presentation for the learner. However, the learning materials were in well- defined packages representing resources the learner could use in any sequence he wished. In addition to self-testing questions, an instructor was available for consultation. 47 Rogers.--Along the dimension of learner/instructor control Rogers (1969) occupies a rather unique position, mainly because he teaches a humanistic subject matter in a clinical way. Although he requires his students to read some books, he only suggests a wide range of possible titles. Otherwise, the class and he mutually determine the functions of instruction either as a group or as a dyad. Even ter- minal evaluation is mutually agreed upon. The inclusion of Rogers in the dimension is merely to point up the arbi- trary nature of assigning instructional functions to either the learner or the instructor. Pressey.--In recent years, Pressey (1963; Pressey & Kinzer, 1964), the father of programmed instruction, has advocated adjunct autoinstruction. This consists of multiple choice questions to be used by the learner, after he studies assigned text segments in a sequence of his choice. The questions are regarded not as examinations or criteria but as ways to induce the learner to respond, to receive feedback, and to undertake remedial study where he is weak. Ausubel.--A1though Ausubel (1963, 1968) is a leading cognitive theorist in educational psychology, he firmly represents those who contend that learning is not a process of self-instruction. He believes that pedagogi- cally sophisticated persons should select, organize, 48 interpret, and sequence learning materials. These are the functions historically assigned to the instructor. It should be noted that Skinner (1968) and pro- grammed instruction differ from Ausubel's position in only one function--pacing of the content. Even the latest development in this area, "tutorial" computer-assisted instruction, determines for the learner when to branch, repeat, or do remedial study (Atkinson, 1968). So called "individualized instruction" is still mostly under the instructor's control. Individual Differences and Student-Controlled Instruction It is possible to conceive of situations where maximum instructor control (Ausubel and Skinner) means no consideration of individual differences and where maximum learner control (Mager) means full consideration of indi- vidual differences. For example, at one end of the learner-instructor continuum is the plausible situation where the instructor arbitrarily lectures from an arbitrarily prepared text, insensitive to the learner's presence. At the other ex- treme, it is possible for the learner to relate to the instructor and material on an individual basis because he can demand from the instructor, at will, any information, examples, reviews or feedback in an order which he feels is necessary for his personal learning. 49 Since the latter extreme enlarges the role of the learner by making him more responsible for his learning, by demanding that he make decisions, and by engaging him in a dynamic, challenging, and interacting dialogue; it is quite possible that some learners will not thrive in such an environment. In other words, the learning environ- ment ought to match the learning style of the learner. The simplest test of any hypotheses about the learner/instructor control dimension would be to compare the effects on learning of the two extreme positions in Figure 1 over a range of learning styles. The extreme position associated with Ausubel and Skinner is well-known and needs no further explanation. On the other hand, the extreme position of Mager needs some modification, at least for experimental purposes. An Envirgnment Where Student Control Can Function Effectively Considering just the instructional functions listed in Figure 1, it can be seen that giving the learner com- plete control of the first two functions (definition of content area and definition of objectives), as Mager (1961) did is impractical. That is, if the experimenter wishes to compare the achievement of specified learning objectives with conventional instructional methods. Possibilities for complete learner control of the fourth function (method/ mode/unit size of presentation) will not be feasible 50 until far more sophisticated computer-learner systems are available. Therefore, learner control of at least four of the instructional functions must, for experimental purposes, be somewhat arbitrarily determined. The remaining instruc- tional functions, however, should be designed so that learner control is as close to that found in Mager's in- structional environment as is possible. Hopefully, such an environment will be far more responsive to the learner than that offered by Campbell (1964). Optimal prescriptions for each of the 11 instruc- tional functions follow: 1. Definition of content area and depth. To maintain intrinsic motivation, the learner ought to be able to satisfy his curiosity by learning beyond the defined content area. 2. Definition of objectives and criteria levels. The learner ought to be able to add his own learning ob- jectives and criteria level, as his interests and experi- ence demand. 3. Entry level of the learner. The learner ought to be able to Engin his learning at a point which is at his level of achievement. 4. Method/mode/unit size of presentation. To fit his style of learning, the learner ought to be able to demand additional methods or modes of presentation. For 51 example, if a learner has a need for diagrammatic or pic- torial representations, he ought to be able to receive them. So that the learner can integrate the units into patterns, unit size ought to be larger than that used with programmed instruction (Grubb & Selfridge, 1964; Ausubel, 1968; Smith & Smith, 1966; Pressey, 1963). 5. Sequencing of content. The learner ought to be able to actually sequence the content. This means: (a) manipulating and encountering the information in the most meaningful (to the learner) sequence, (b) selecting the route to the learning objectives, and (c) the freedom to generate mediation between new learning and old know- ledge within the learner's own cognitive structure. 6. Pacing of the content. The learner ought to be able to learn not only at his own pace but also he ought to be able to start and stop his learning at times convenient to him (i.e., stopping when he is satisfied). 7. Inquiry of resources. The learner ought to be able to: (a) demand additional information and receive it quickly, (b) seek help on and verification of hypotheses about unclear or discrepant information, (o) fill in the gaps in his knowledge or in the information supplied him, and (d) relate his past experiences to new information on demand. 8. Practice/application. The learner ought to be able to practice or try out his new knowledge in order to 52 solidify its meaning and to apply it to new situations as the new knowledge becomes familiar 9. Review/remedial study. The learner ought to be able to review any of the information presented and to receive remedial help for those skills or "learning sets" found to be less than adequate for continuing the main learning task. 10. Knowledge of results as learning proceeds. .The learner ought to be able to receive immediate knowledge of results (confirmation) as he learns. Reinforcement from external sources (motivation) ought to be available if the learner needs it. 11. Terminal attainment of objective. The learner ought to be able to receive immediate feedback of his ter- minal performance. Additional feedback ought to be avail- able for the learner's own objectives. The above prescriptions for each instructional function should, if properly implemented, result in a learning environment where the learner can effectively control his own learning. Only experimental investigation will tell us if this is true. CHAPTER III METHOD AND PROCEDURES In Chapter II an environment in which student con- trol could function was described. The research method and procedures used to investigate the efficacy of student control will now be described. Experimental Design A three-factor experimental design was employed to test the following hypotheses: l. The greater the student control, within limits, over the learning situation, the greater will be the amount of material learned as measured by gain and retention scores. The greater the student control, within limits, over the learning situation, the greater will be the satisfaction of the learner with the learning experience, as measured by attitude questionnaire. The above hypotheses will not hold for all learners. High "inquisitive" learners, as determined by a battery of personality measures, 53 54 will learn more and be more satisfied than the average learner. Low "inquisitive" learners will learn less and be less satisfied than the average learner. Factor One The first factor in the experimental design (Table 1) results from the following instructional treatments: Student-controlled instruction (SCI) treatment.-- Under this treatment Ss were given as much control over their own learning as possible given the constraints of time, experimental design, and the facilities available. 33 in this group were given the opportunity and freedom to select the sequence in which material to be learned was presented to them. The material consisted of 52 videotaped lecture segments about "Computers and How They Work" and was presented on a nine-inch TV monitor (Figure 2). S had a deck of freshly shuffled cards; each card contained one significant question about computers to which S might wish to know the answer (Appendix A). For each card there was a corresponding video tape segment which, in effect, answered the question on that card. Therefore, S could make use of previous experiences and learning to arrive at a decision as to which question he wanted answered. S indicated to E, by code, which question he wanted answered (for an explanation of coding, see 55 Table l.--Number of $3 for Each Cell of Experimental Design. Subject-Types Treatments Inquiry SCI Expert Random lContro = 1 High 12 12 12 12 High Low 12 12 12 12 High 12 12 12 12 Low LOW 12 12 12 12 56 Figure 2.--Carrel configuration: earphones, TV monitor, and microphone. Figure 3.--Control room configuration: TV camera (lens shown only), TV monitors, microphone, patchpanels, earphones, and video tape recorders. 57 Appendix A) and B would locate the corresponding video tape segment and play it to S (Figure 3). While this new information was being processed, another question might arise which would prompt S to ask for another segment. This process continued until all de- sired segments had been viewed by S. For purposes of ex- perimental control, 83 were limited to four or fewer repetitions and/or deletions of the total number of seg- ments. In addition, SCI Ss were also given the opportunity and freedom to ask questions about tape segments after they had been shown. If asked, E would appear on the monitor and answer each inquiry posed to him with a direct and suc- cinct answer. All questions and answers were recorded on video tape for later use with $5 in both treatments described below. Finally, 85 in this group were allowed to proceed at their own pace. That is, they could pause between seg- ments for as long as they wished, and they were allowed to take notes as needed. "Expert" treatment.--Ss in this treatment viewed video tape segments in a sequence predetermined by "experts" in the content area. (For definition of "experts", see below.) The sequence to be followed was presented to S as a list of questions (Appendix B), identical to those on the "SCI" cards. 58 Each S in this group was yoked with one of the Ss under the "SCI" treatment. Therefore, any repetition or deletion of segments by the "SCI" S was duplicated for his counterpart S in this group. 85 in the "Expert" treat— ment were not allowed to ask questions and had no recourse but to view the recorded questions (and answers) asked by his "SCI" counterpart. This auxiliary information was presented directly after the appropriate segment. Ss in this treatment were presented information in a completely determined and structured manner. A deliberate attempt was made to simulate a typical classroom in which the instructor lectures according to his conception of what is a "logical" sequence and where most students hear questions raised by classmates, whether they are meaningful and relevant questions or not. Unlike the typical class- room, but for reasons of experimental control, these Ss were allowed to proceed at their own pace and take as many notes as needed. ' "Random" treatment.--Again, each S in this treat- ment was yoked with one S from the "SCI" treatment. How- ever, "Random" Ss viewed the tape segments in a sequence determined by chance. (For method of randomization, see below.) The list of questions (Appendix C) were identical to those of the "SCI" cards. Except for order, "Random" Ss were presented the information in a manner identical to the "Expert" treatment. 59 "Control" treatment.--"Control" Ss received no information about computers. Instead, they were given a ten-minute talk about the experiment and their part in it. Otherwise, "Control" 85 received the same treatment as the other Ss. Factor Two The second factor in the experimental design (Table 1) results from a categorization of Ss in terms of college aptitude scores. Since this dimension had potential for interacting with instructional treatments, college aptitude test scores were obtained for all Ss and divided at the median (see details below). Factor Three The third factor in the experimental design (Table 1) results from a categorization of Ss in terms of "inquiry" scores. In a "SCI" type instructional system, as discussed above, where the learner can control his own instruction, there is an assumption that the learner will interact with the informational sources in a problem-solving and decision- making manner. Since the degree of control an individual may wish to have in such a learning situation is likely to be depen- dent on his initiative and since this degree of control may interact with instructional treatments, some means of 60 including this characteristic in the experimental design was desirable. Of the measures of personality which have inter- acted with instructional method, those delimited by Shulman et al. (1968) appeared to offer an excellent chance of separating learners who could take full advantage of the instructional system and those who could not. In addition to including many of the dimensions used in other studies, such as authoritarianism and test anxiety, Shulman's best six predictors of inquiry behavior, out of a battery of 17 tests, accounted for 50% of the measurable variance. More important, perhaps, is the fact that these measurements were obtained under conditions similar to the one offered to "SCI" 85 here. I Therefore, all Ss were given Shulman's battery of six tests and the dimension, "inquisitiveness", was treated as the third major independent variable in the experiment. S's scores on the six tests were standardized as T scores (i=50, 6=10) and sununed. High total scores reflected individuals high in cognitive complexity, pre- ferring the ambiguous, the assymetrical and the unexpected to the regular, articulated and predictable; liberal in political values; high in associational fluency; high in non-stereOpathy; high in verbal problem solving; and low in expressed test anxiety. This distribution of scores was then divided at the median. 61 Stratified Random Assignment Using these scores and the college aptitude scores, it was possible to establish a four cell matrix of Subject- Types who were, respectively, high on both aptitude and "inquiry" predictor scores, low on both of these dimensions, and two groups for which the dimensions were incongruent (Figure 4). However, due to the moderate correlation (y=.37) between the two dimensions, High, High and Low, Low groups were over-represented. Therefore, some of the Ss in these two groups who were, in fact, assigned to these groups until an equal number of Ss (48) were in each. From each of the four groups, twelve 85 were ran— domly assigned to each of the four treatments. This tech- nique insured that an equal number of each subject-type would be represented in each treatment category (Table 1). In practice, Ss were assigned to treatments, at random, after a set of four Ss was drawn, at random, from one of the groups. This technique helped guarantee that each set of four Ss would be approximately matched for aptitude and "inquiry". In addition, since "SCI" members of each set made inquiry of the subject matter while the other members could not, it was apprOpriate that that level of inquiry be consistent with the aptitude and "in— quiry" level of the other 35 of that set. "INQUIRY" standardized) (Sum of six tests, 62 400 - * * 380 - * * * * * * * m 360 - * * * a * ** * * * m 340 - ** ** ** * * ** * * ** * ** * ** 320 _ ** * ** * ** **:* k * * * 300 - *** * * * * * * * ** *** * * 280 ' * * * * * * ** ** * * * * 260 - * ** * * * * * * * * 3 * * * * * S 240 - * 3 * * ¢*** * * * 220 - * 5 ** * * 200 ' *** * * 4:,_ 0 10 20 30 40 50 60 70 80 90 100 LOW, HIGH College Aptitude (Percentile) Figure 4.--Scatterplot of Experimental Subjects in Terms of Their College Aptitude and "Inquiry" Scores. Pilot Experiment summer term, ing: A pilot experiment using 40 53 was run during the 1969. It provided information for the follow— Modification of instructions and operating procedures. Validation data for criteria test items. Color-coding of cards for "SCI" Ss. Revision of "Post-Attitude Survey” question- naire. 63 e. Feedback about the experiment for construction of "Terminal Interview" questions. The latter four of these changes are explained in detail below. The results of the pilot study were similar to those of the main experiment described below. Because of the changes mentioned above, the results of the pilot study are not reported here. Subjects Ss were 192 students enrolled in the introductory psychology course at Michigan State University during the fall term, 1969, and winter term, 1970. Due to the time required from each S (up to five hours), E went to all introductory psychology classes taught by Graduate Teaching Assistants where he solicited volunteers. The experiment was described as a learning experience which would give Ss the opportunity to become better acquainted with computers. Potential 58 were given an information sheet (Appendix D). Some 85 were disqualified because they had had prior courses in computer science. About 40% of the students contacted participated in the experiment. Of the Ss who began the experiment, four failed to complete it. A breakdown of 55 along the dimensions of age, sex, and class is shown in Table 2. 64 Table 2.--Subject Characteristics CLASS AGE Fr So Jr Sr Total 17-18 19-20 21+ Total 157 26 6 4 192 137 43 12 192 82 13 3 2 100 71 23 6 100 SEX Male Female Total N: 73 119 192 %: 38 62 100 Pre-Experimental Measures (Independent Variables) Each S began the experiment by taking Shulman's battery of six tests. Ss also completed two other in- struments: a "General Inventory" and a measure of their "Attitude Toward Learning." In addition, scores on some college entrance tests were available for all Ss. These tests were administered at the Learning Service office by one of the secretaries. After testing, Ss were given a map of the laboratory's location (Appendix E) and told 65 that they would be contacted by telephone for scheduling of their learning sessions. What follows is a brief description of the instru- ments used for the independent measurements: Shulman's Six Tests 1. "Word Association" (Appendix F) is a test of verbal flexibility used by Getzels and Jackson (1962). One point was scored for each different meaning and another point for each different word under each of the different meanings given. 2. "Political Position" (Appendix F) measures liberalism-conservatism. Names of political candidates used were up-dated from those used in Shulman's original study to reflect the times: i.e., from Goldwater, Johnson, R. Kennedy, and Romney to Reagan, Nixon, Humphrey, and T. Kennedy. The form used was actually Shulman's revised form. Scoring ranged from 4 (conservative) to 18 (liberal) as follows: a. Item #1 1-3 points b. Item #2 1-5 points c. Item #3a, b 1-5 points each: if Reagan and Nixon ranked 1, 2--l point if Humphrey and Kennedy ranked 1, 2--5 points if some other combination ranked 1, 2-- variable points 66 3. "Inventory of Beliefs" (Appendix F) was used in the studies by Stern, Stein and Bloom (1956) and closely parallels the scales developed by Adorno et a1. (1950). It correlates with the E (authoritarianism) scale (y=.67), the E (ethnocentrism) scale (y=.66), and the Egg (political-economic-conservatism) scale (y=.43). Stern et al. used the "Inventory of Beliefs" as a measure of college student "stereopathic" and "non-stereOpathic" personality. They found that non-stereOpathic under- graduate students had a much higher degree of success in the unstructured, relatively undisciplined programs at the University of Chicago. Stereopaths showed much greater difficulty coping with the generally undirected programs. Non-stereopaths tended to achieve much more successfully on comprehensive examinations than did the stereopaths, even when general aptitude was held constant. These tests emphasized problem solving and inference far more than memory for detail. One point was scored for each item answered in agreement with Shulman's key. 4. "Attitude Inventory" (Appendix F) measures cognitive complexity. That is, it measures an individual's reaction to the ambiguous, unpredictable and assymetrical. Barron (1953, 1967) used it to distinguish between creative and non-creative individuals in such fields as architecture. 67 In addition, this instrument also includes a six item measure of the tendency to "focus" or "scan" when reading school related materials. For purposes of analysis, the scores on the two measures were separated. One point was scored for each item in agreement with Shulman's key. 5. "Attitudes Toward Tests" (Appendix F) is a measure of test anxiety and is adapted from Alpert and Haber (1960). One point was scored for each item answered in agreement with Shulman's key. 6. Michigan State University "Reading Test." Excerpts of this instrument appear in Appendix G. It measures the ability to comprehend, analyze, interpret, and make apprOpriate inferences about selected passages on social science, literary, historical and natural science topics. It is therefore a test of general verbal problem— solving as much as a test of simple reading ability. This test is routinely administered and scored by MSU's Evalua- tion Service personnel for all entering MSU students as part of the orientation program. E obtained test scores on all Ss from Evaluation Service. General Inventory This instrument (Appendix F) was designed to deter- mine the following: 1. Sex: male or female. 2. Age: 17—18, 19-20, and 21 and over categories. 68 Class: Freshman, SOphomore, Junior or Senior categories. Socio-economic status: in order to grant a status to each S, E assigned a number from 0-9, based on the following criteria: a. Item #2 (father's occupation): if professional -------------- 4 points if businessman or farm owner ------------------- 3 points if white collar or skilled tradesman ------------ 2 points if industrial worker --------- 1 point if laborer ------------------- no points b. Items #3, 4, 5 (the highest level of education attained by father, mother, and older siblings--the points for the three items were summed and the average used in the total): if post-graduate or professional training --------------------- 5 points if college graduate ---------- 4 points if some college or trade school ----------------------- 3 points if high school graduate ------ 2 points if grades 9-11 completed ----- 1 point if grades 1-8 completed ------ 0 points "Pre-Interest in Computers": In order to grant a "Pre-Interest in Computers" score to each S, E assigned a number from 0-9 based on the following criteria: a. Item #11: if S had plans to take a com- puter science course--l point. (Other- wise, this item was used to insure that only inexperienced 85 were participating.) 69 b. Items #12, 13, l4, 16: If answered in the direction of showing interest in computers and: if strongly agree --------------- 2 points if agree ------------------------ 1 point all other ----------------------- 0 points EQEE‘ Items of the General Inventory instrument #9 and #10 were not included in any analysis because most Ss were uncertain how to respond to them. Likewise, items #1 and #15 were probably not relevant to the dimension under consideration and were therefore excluded. "Attitudes Toward Learning" This instrument (Appendix F) was developed specifi- cally to measure attitudes toward learning of the type in- volved in this research. Initially, a number of potentially useful items were obtained from a study by Davis, Marzocco, and Denny (1967). To this were added more specific items for a total of 40. This instrument was given to 72 general psychology students in the spring term of 1969. A reduc- tion in the number of items to 21 was made by means of factor analysis and inspection. The resultant instrument after being administered to the 192 Ss in the experiment was again factor analyzed. Table 3 presents the six factors which account for 53% of the variance after a principal axes solution followed by a varimax rotation. 70 Table 3.--Proportion of Variance Accounted for by Factors in "Attitudes Toward Learning" Questionnaire FACTOR PROP. VAR. ITEMS 1. Fear of asking quest1ons 1n .10 1,8,15 class. 2. Independence from instructor 07 7,14,16, while learning something new. ° 20 3. Frequency of review or self-test. '08 2’18 4. Control of instructional 10 4,5,6,9, functions. ° 11,19,20 5. Ease of learning. .09 3,12,17 6. Autonomy while learning. .09 10,19,21 College Aptitude Tests In addition to the MSU "Reading Test" above, Evaluation Service provided scores for Ss on the following dimensions: Percent of Participants 1. College Aptitude: a. "Scholastic Aptitude Test" 72% (SAT) b. "American College Testing 7% Program" (ACT) c. "College Qualification Test" 21% (CQT) 100% 71 Percent of Participants 2. MSU "Mathematics Test" 82% 3. MSU "Arithmetic Test" 92% Evaluation Service converted scores from the above tests into standard scores and then reported them as percentile scores. E reconverted the percentile scores into standard scores for use in analysis. Experimental Procedure Learning sessions were arranged for each 8 indi- vidually at his convenience. The only stipulation was that the S had to be able to attend two sessions of about an hour and one-half each on consecutive days. First Learning Session Each S began the first learning session with a 60 item pretest of his knowledge of the subject matter (Appendix H). This took approximately 30 minutes and was administered in the carrel room (Figure 2). After the pre- test was completed, E explained how the audio and video equipment worked and introduced Ss to either the cards ("SCI" Ss) or one of the lists ("Expert" or "Random" Ss). Instructions (Appendix I) were read by E over the closed- circuit TV system from the control room (Figure 3). This method helped insure that: (a) the presentation would be uniform for all 83; (b) S would become familiar with the 72 closed-circuit TV system (a warm-up for learning); and (c) E would be physically removed from S's presence. Thus, S could relax and pay attention to the content of the in- struction. Instructions for the different treatments were as similar as possible. Of necessity, however, there were basic differences among them. After one-half of the segments had been viewed, E informed S that the first session had been completed. For "SCI" Ss the stopping point was up to each S. Usually this point was also after one-half of the segments had been viewed. S was then given post-session instructions (Appen- dix I) which asked him not to learn more about the subject matter outside the laboratory until the conclusion of the experiment and not to communicate with other students about the experimental procedures employed. Finally, E took S's notes and kept them in the laboratory overnight. Second Learning Session When S returned to the laboratory on the following day, E returned S's notes, if any, and asked if there were any questions concerning the instructions. If not, S started and completed the segments and questions. After viewing the remaining segments, E informed S that he had as much time as he wished to go over his 73 notes and that when he was ready, a posttest would be given to him. When S was ready, E removed S's notes, then gave S a posttest followed by "Post-Attitude Survey" question- naire (Appendix M). When the posttest and questionnaire were completed, E informed S that in two weeks E would contact S for scheduling a retention test (Appendix H). Finally, S was again given the post-session instructions, as above. Subject Logs E kept a Subject Log (Appendix J) on S's perfor- mance. Initially, E recorded S's name, identification num- ber, his group and treatment, date, and test sequence. Then, E began recording, in minutes, the length of the learning sessions. For "SCI" 85, E recorded: (a) the number of each segment in the order chosen by S (E converted code words into numbers by means of the list in Appendix K), (b) a check (/) under the apprOpriate column and row for each segment repeated and/or inquiry made, if any, (c) the num- ber of each deleted segment, if any, at the end of the log, and (d) later, a verbatim transcription of every question asked by S. The total number of questions was also recorded. For the "Expert" and "Random" 83, E recorded ini— tially, a check (/) under the apprOpriate column and row to indicate when an inquiry on the "SCI" video tape or a 74 repeated segment was due. E wrote word "no" to indicate when a segment deletion was due. As the learning session progressed, E recorded a check (/) as each segment was viewed and an additional check (/) as each repeat, deletion, or inquiry was completed. The "Control" Ss required no log. E also recorded on the Subject Log the time, in minutes, spent by each S in reviewing his notes. Under "comments", E recorded any noteworthy behavior or comments made by S. Retention Test Session Two weeks after the second learning session, S was contacted and given a retention test, without review. Upon completion, S was given the "Terminal Interview" check- list (Appendix N). E then asked S to arrange the cards in what he considered to be the "best" sequence for learning. For the "SCI" 85 this request was mostly a repeat of previous instructions. Therefore, in order to motivate them, E told them to think of the task as if they were about to relearn the material or as if they were about to sequence the material for their roommate to learn. "Expert" and "Random" Ss had to be introduced to the cards and the color code before they could be asked to sequence the cards. In addition, these Ss were told that this was their chance to put the segments in a 75 sequence more "logical," perhaps, than the one they had viewed. E recorded the order chosen, by number, along side the 52 numbers on the Subject Logs. Then E explained the overall experimental design and hypotheses and showed S's test scores to him if de- sired. If there was time, E asked S to comment freely on the experimental environment. Finally, E signed S's Experimental Credit Card. By means of this act, S fulfilled one of the requirements of his psychology course--participation in psychological experiments. Control Subjects After completing the pretest, "Control" Ss were given a short ten-minute talk about the experiment and their part in it. Then they were given the posttest. Although this procedure saved both E and S time, carry- over of learning from the pretest to the posttest, if any, may have been amplified. In addition, the time between these tests and the retention test varied from one to two weeks rather than being exactly two weeks. When the "Control" 83 were given the "Post-Attitude Survey" questionnaire, they were instructed to mark "un- certain" (#3) for any item which they had no knowledge about one way or another. They were informed that such a task, even though it may appear to be meaningless, would serve as a base against which to measure experimental Ss. 76 Development of the Subject Matter E wrote and developed the subject matter (Appendix L) to be learned in the experiment. The subject matter learned by 85 ("Computers and How They Work") was chosen for the following reasons: 1. E was familiar with this area and in repli- cating Mager's (1961) study had used the same subject matter area. Therefore, development of the material could be con- structed around points that naive 85 would want to learn about. 2. The subject matter fulfilled the requirements of being: (a) meaningful, verbal knowledge, (b) somewhere between pure training and pure education extremes, (c) elaborated on in a number of sources, (d) intrinsically interesting to most college students, (e) sufficiently complex and interrelated so as to provide a challenge to the naive learner, and (f) not so new as to prevent the learner from coming to the experiment without a great deal of already acquired bits of information from a variety of sources. 3. For reasons suggested in the literature above, this subject matter fulfilled the further requirements of being capable of being taught and learned in a non- hierarchical manner. That is, it could be broken down into segments of an independent nature. Thus, the learner could enter the material at any point with impunity. 77 This entry could be strictly a matter of personal choice, not necessarily because one segment "naturally" followed another. 4. In addition, this subject matter could be con- structed of units that were larger than the "frames" of programmed instruction, but still be put into a "scrambled" or random order. 5. Finally, each segment could be constructed so that it answered one question about computers. Thus, the question (title) on each card (or list) served as an access or an information retrieval device. However, these ques- tions (titles) were arbitrarily determined by E. No evi- dence was found to show that this method of input to the 83 was a hinderance to them. Development of the Segments The material to be learned was gathered from a number of sources (Anderson, 1966; Desmonde, 1964; Flores, 1967; Laurie, 1963; Marchant & Pegg, 1967; Schmidt & Meyers, 1965; Shultz, 1963) designed to: a. Insure adequate coverage of the area. b. Be parsimonious. So that all of the material could be included in an hour's lecture time, the original 15,000 words were cut to a con- cise 10,000 words of written material. c. Be sufficiently difficult to prevent a ceiling effect in learning. 78 d. Include instructional objectives for each seg- ment. e. Encompass an adequate number of concepts (seg- ments) to make selection of the segments chal- lenging. In the end, 52 segments were produced, comprising eight different content areas (Appendices B and L). Taping of the Segments The subject matter was transferred to video tape segments by: (a) typing the content of each segment on acetate "cue cards", (b) projecting these sheets onto a screen which was placed directly beneath a television camera, and (c) focusing the camera on an instructor who read the projected "cue cards." The instructor's speaking rate was about 60 words a minute which is somewhat less than the average news- caster's rate (approximately 80 words per minute). In order to conserve tape, the order of taping was identical to the "Expert" sequence. If errors were detected during taping, the tape was rewound and the segment retaped. Two one-inch tapes, a total of one hour and 20 minutes, were required. Using an Ampex #7800 editor, these tapes were then used to electronically reproduce a duplicate set of "Ex- pert" sequenced tapes and two sets of "Random" sequenced tapes. 79 Ordering the Segments "Expert" order.--Of the six volunteer experts ob- tained to order the tapes, three were graduate students in psychology who were or had been consultants in computer science. The remaining experts were: (a) an instructor in MSU's Computer Science Department, (b) an Assistant Professor from the Computer Institute for Social Science Research (CISSR), and (c) the director of MSU's Data Processing Department. E scrambled (shuffled) the complete unindexed set of 52 segments and instructed each expert to order them in a sequence which they themselves would use if they were about to teach the material to naive students. Since the material for each segment could be put on one sheet of paper, the task of ordering the segments was readily accom- plished. Of the 15 possible rank-order correlations (Table 4) between the rankings assigned to the segments by any two experts, only three were significant (yhoz.274) and the average for all 15 was quite poor (yho= .005). Since agreement between experts over all 52 segments was essen- tially non-existant, E looked separately at the rank-order correlations for each of the content areas of the subject matter. This method of analysis yielded coefficients of concordance and average inter-correlations among rankings assigned by the experts that were consistently much 80 Table 4.--Rank-Order Correlations Between Rankings Assigned to Segments by Pairs of Expert Judges Judge 1 2 3 4 5 6 l 1.00 2 -.08 1.00 Judge 3 -.13 +.10 1.00 4 -.03 -.38 +.07 1.00 5 -.19 -.02 +.56 -.03 1.00 6 -.09 -.09 -.13 +.44 -.08 1.00 Table 5.--Coefficient of Concordance and Average Inter- correlations Among Rankings Assigned to Segments by Expert Judges 'Kendall Coefficient of Concordance (w) Average Content Area Intercorrelation (7) 1 0.44 0.33 2 0.77 0.72 3 0.56 0.47 4 0.52 0.42 5 0.21 0.05 6 0.44 0.33 7 0.19 0.03 _______8 916.9. 22.52 Overall 0.03 0.00 81 higher than the overall concordance and intercorrelation (Table 5). Therefore, E, by inspection, sequenced the segments first by overall content area and then within each content area according to the consensus of experts. The resultant "Expert" ordering of the segments is shown in Appendix B. "Random" order.--Two different random sequences were employed. The second sequence was employed after it was noted at the half-way point in the experiment that the "Random" treatment 85 were doing rather well and that it was possible that this might be due to the random sample somehow being sequenced "logically." In both cases, the random sequence was generated by numerous shufflings of the cards after they had been thrown across the floor and mixed up thoroughly. Both random sequences are shown in Appendix C. Measurement of the Dependent Variables Development of the Tests Initial effort.--In order to show differences be- tween treatments, adequate measures of learning must be obtained. Initially, three equivalent achievement tests were developed. Items for these tests were constructed in sets of three. Either a set asked different questions about a single concept or it asked a similar type of ques- tion, such as questions involving numerical memory. Then, 82 the items from each set were randomly assigned to each test form (Appendix H). Item analysis - I.--Following the pilot study, item analyses of the three test forms were performed. Items were inspected for low indices of discrimination, possibilities of ambiguity, being either too hard or too difficult, or not having all distractors equally distract- ing. The three forms were then revised. Because Form A had been given only as a pretest, its revision could not be as precise as forms B and C. Item analysis - II.--Upon completion of the experi- ment, the test forms were again item analyzed. Because pretest and "Control" Ss' scores were practically at chance levels, only scores from posttests and retention tests of "non-control" treatment 55 were used in these analyses. From these analyses it was observed that each form still contained some items which were contributing nothing to the measurement. A more valid and reliable test would result from the exclusion of these items. However, dele- tion of items after testing and then rescoring tests for those 83 has questionable merit. Normally, items on a test are deleted and then the revised form is retested on another population sample before the test is judged fit for use. Unfortunately, such a procedure could not be carried out in this case. 83 However, the Rasch method (Wright and Panchapakesan, 1969) of analysis, which is relatively new, has the follow- ing features, which are not true of conventional item analyses: (a) test calibrations are independent of the sample of persons used to estimate item parameters, (b) person measurements, the transformation of test scores into estimates of person ability, are independent of the selec- tion of items used to obtain test scores. This method identifies poor items during the test calibration phase. The retained items are then reanalyzed to obtain final estimates of S ability. 85 who get the same score are estimated to have the same ability. Therefore, the ten poorest items from each test form were deleted. Instead of a decrease in reliability (KR-20), as one would expect after a reduction in the num- ber of items, Table 6 shows a slight increase to occur. This offers evidence that deleting the items was, in fact, a sound procedure. Finally, all tests were re-scored. The raw scores were then standardized as T scores (x = 50,6= 10) prior to data analysis. This standardization served the purpose of removing all differences between the three test forms. 84 Table 6.--Change in Test Reliability After Deletion of Items Reliability Measures (Kuder-Richardson 20) Conventional Rasch Sample-Free Item Analysi Test Item Analysis . Form (60 items) Before Deletion After Delet1on (60 items) (50 items) A .699 .695 .743 B .805 .802 .824 C .794 .790 .808 Order of the tests.--The three forms were given to the Ss in a systematic fashion, as shown below: S Pretest Posttest Retention 1 A B C 2 A C B 3 B A C 4 B C A 5 C A B 6 C B A Since there were twelve 85 in each subject-type treatment cell, this series of alternations was repeated twice for each cell. Also, the four 55 in each set who were yoked across subject-type treatment cells always received the same sequence of test forms. There was a reason for alternating the sequence of test forms and not giving all Ss the same sequence. If Form A had always been the pretest, scores on it would 85 almost always have been associated with chance (i.e., Ss guessed at most items on the-pretest)». Since this test could not be evaluated on any other population, it could never be guaranteed to be a representative and equivalent test form from which to draw conclusions about gain scores. Even the Rasch sample-free item analysis demands an equally representative testing experience for the calibration of the test to be valid. "Post-Attitude" Survey E specifically developed this instrument (Appendix M) to measure Ss' attitudes toward this experiment, es- pecially the features of the learning environment. Although designed with four factors in mind, six emerged from a sub- sequent factor analysis. Table 7.--Pr0portion of Variance Accounted for by Factors in "Post-Attitude Survey" Factor Prop. Var. Items 1. Perceived learning as a .15 7,9,11,16,18, function of the environment. 19,20,21,22 2. Attitude toward order of the segments. '97 12,13 3. Attitude toward the method of instruction. ~13 112,315.15,20 4. Perceived learning as a function of task difficulty. ’08 2'3'6'11'18'21 5. ReV1ew1ng or relating material .06 10,14 to past experiences. 6. Attitude toward computers. .12 4,8,17 86 Table 7 presents the factors which account for 61% of the variance after a principal axes solution followed by a varimax rotation. All factors shown, will be used as criteria in subsequent analyses. "Terminal Interview" E developed this instrument (Appendix N) specifically to gather information from 55 which might help explain their performance and/or attitude toward the learning environment. In the pilot experiment Ss were given an open-ended form of this instrument (Appendix N). From the more fre- quently written answers there, E constructed a new form which insured that all Ss would consider the same item stems while still being free to add other information. Conse- quently, there was a consistent measure of all 55, unlike the original form. The first page of the final instrument was identical for all Ss. For obvious reasons, however, the second page contained questions specific to either the "SCI" or "Expert" and "Random" treatments. "Postelnterest in Computers" E devised a measure of "Post-Interest in Computers" from parts of the above two instruments. First, all three items from the sixth factor of the "Post-Attitude Survey" questionnaire were scored to give a range of from 3 to 15 points for each S. Then, the "Terminal Interview" 87 questionnaire was scored by counting the number of checks present in any category of item #3 and in the second and third category of item #4. The total range was from 3-21 points. This information will be compared to "Pre-Interest in Computers" scores and used as a noncomitant dependent variable below. Card Sequencing As explained above, the final task for the 83 was to sequence the cards. This task was designed to measure: (a) any change in segment sequencing by the "SCI" Ss, (b) any difference between "SCI" and "Expert" segment sequence, and (c) the degree of communality of segment sequences, after all 83 had become familiar with the material. (No "Control" 83 performed this task.) CHAPTER IV RESULTS This chapter consists of four sections. The first is concerned with analysis of variance of the dependent variables. The second section looks at correlational analysis of the data. Then, responses to the "Terminal Interview" will be compared. Finally, agreement among the rank-orderings of the lecture segments will be presented. Analysis of Variance Initially, analysis of the data was begun with posttest and retention test achievement scores as the ma- jor learning criteria. However, it was discovered that the average correlation between treatment groups on these scores was substantial (§=.47). This suggested that there was a dependency between treatment groups rather than independency as required by routine analysis of variance techniques. Upon calculating gain scores (e.g., the difference between pretest and posttest scores) observed correlations between treatment groups were reduced signifi- cantly. The average correlation for the cells shown in Table 8 is quite small (§;-.os). Therefore, instead of becoming involved in a complicated 2 x 2 x 4 x 12 factorial 88 89 repeated measures analysis of variance (AOV) design, inde- pendency between criteria scores was felt to be justified and a 2 x 2 x 4 factorial AOV was used on all analyses below. Finn's (1966) multivariate computer program was employed in the analysis. Of the 11 variables selected for use as covariates, only one, "Pre-Interest in Computers" had a significant overall relationship with the 11 depen- dent variables. A stepwise univariate multiple regression analysis was performed to determine the effects of indivi- dual independent variables on each dependent variable. Of the 11 variables 10 made no appreciabl£2contribution and were, therefore, eliminated from the analyses. Nonetheless, the covariate that remained (Pre- Interest in Computers) influenced the differences between means very little. It accounted for less than five per cent of the variance for the gain scores. Therefore, covariates are not included in the analyses tabled below. Tables 9 through 19 below present cell means and standard deviations, a 2 x 2 x 4 factorial analysis of variance, and four one-way analyses of variance tables for each of 11 dependent measures. Throughout the analyses, the number of Ss in each cell is 12 and probability levels less than .10 are not shown. Since the "SCI" instructional treatment was of primary interest, its mean has been contrasted with the 90 NHuz you mo.vm« NH. ma. ma. so. RH. «Hm. Hamum>o om.u v~.u mo.- NH. mm.u mo. 30a 30A m¢.- NH. mm.- mH.- 4m. mH.- roam me. so. 4m. se.- mm.- 446. sod roam mm. HH.- ~H.- mH.- Ho.- om.- nude ammumxm ammnHom mxmuHom ammumxm ammuHom mxmuHom muwsch .mwsuflud Aummuwum mDCHE ummu coausmummc N ZH