“04" 4...- “‘ ' We” _, 2 ‘, Mom .. _.. 2;: “l- e' ._ i l “55‘s ’3; :1 l MICHlGAN TATE UNlVERSITY LIBRARIES {j v“ v I : I '1 lllllll \\\\\\\\\\\\\\\\\l\\\ \\\\l -~ l 3 1293 10637 4022 B it“ U E bx. ”‘31? 7.1.” “" -19: :7 .‘ ' ‘;_;;-:".;‘;-3~_;“.m'z.m,jflfi ' l l l This is to certify that the ? dissertation entitled l v THE EFFECTS OF TNMFDTATE AND DELAYED FVVWRACK ON THE RETENTION OF FACTUAL AND RULE LEARNTNG presented by Arthur Samuel Tahachnpok has been accepted towards fulfillment of the requirements for Doctoral degreein flop llphprt‘mer‘t of Connselinfi, Educational Psycholory and shoeinl Education 4/- ably/é/ / Major profes r Dateflfajn /Z/72?77\ MSU is an Affirmative Action/Equal Opportunity Institution 042771 MSU LIBRARIES \. RETURNING MATERIALS: Place in book drop to remove this checkout from your record. FINES will be charged if book is returned after the date stamped below. l/ "CW R0 THE EFFECTS OF IMMEDIATE AND DELAYED FEEDBACK ON THE RETENTION OF FACTUAL AND RULE LEARNING By Arthur S. Tabachneck A DISSERTATION Submitted to Michigan State University in partial fulfillment of the requirements for the degree of DOCTOR OF PHILOSOPHY Department of Counseling, Educational Psychology and Special Education 1982 ABSTRACT THE EFFECTS OF IMMEDIATE AND DELAYED FEEDBACK ON THE RETENTION OF FACTUAL AND RULE LEARNING By Arthur S. Tabachneck The purpose of the present study was to compare the relative effects of immediate and delayed feedback on the retention of two categories of learning outcomes, the stat- ing of facts and the application of rules, in the context of an actual classroom setting. The study, specifically, was an attempt to correct for five limitations of previous stud- ies conducted on the same topic, namely: (l)the possible confounding of reinforcement and delayed feedback; (2)the use of dependent measures atypical to most classroom learn- ing situations; (3)the use of a potentially biased measure of retention; (4)the use of theoretically inconsistent defi- nitions of immediate and delayed feedback; and (5)the fail- ure to consider and/or control for the particular outcomes of learning desired. The entire populations from three sections of a freshman-level college chemistry course served as subjects in two experiments, one for each of the outcomes of learn- ing investigated. In each study the effects on retention of Arthur Samuel Tabachneck providing informative test feedback immediately (i.e., with- in two seconds) and delayed (i.e., after approximately 20 minutes) were compared. Actual planned course content and examinations were used as the stimuli and dependent varia- bles, respectively, and the manipulations were accomplished within the normal context of the course. The results clearly favored delaying the presenta- tion of informative feedback (as suggested by information processing theory), rather than presenting it immediately (as prescribed by the prOponents of reinforcement theory). Students who were given informative feedback approximately 20 minutes after they had reSponded to a practice quiz, scored significantly higher on examinations given one week later on the same content, than students who were given such feedback immediately. That is, regardless of the outcome being examined, or whether the test items were new or familiar, delayed feedback enhanced retention signifi- cantly more than immediate feedback. The results were discussed in terms of their limi- tations and implications for educational theory and prac- tice, and changes were suggested for improving future re- search on the topic. ACKNOWLEDGMENTS I gratefully acknowledge the following people: Dr. Walter Hapkiewicz, who chaired my committee during my tenure as a doctoral candidate, for his guidance, insight and perseverance. Drs. Lawrence Lezotte, William Frey, and John Schweitzer, for their insights, guidance, encouragement and willingness to serve on my committee. Mr. Philip Ansted, of the faculty at Mott Community College, for his insights and help in deve10ping the designs and instruments and, of course, for allowing me to utilize his students as subjects in the experiments. And last, though far from least, my wife and children for their help, patience, and the many years of freedom they gave up in allowing me to accomplish this endeavor. ii TABLE OF CONTENTS Page LIST OF TABLES . . . . . . . . . . . . . . . . . . . . v Chapter I. THE PROBLEM O O O O O O O O 0 O O O O O O O O 1 Introduction and Need . . . . . . . . 1 Purpose 0 O O O O O O O O I O O 0 O 0 10 Overview . . . . . . . . . . . . . . 11 II. REVIEW OF RELEVANT LITERATURE . . . . . . . . 13 Informative Feedback . . . . . . . 13 Why feedback works - behavioral explanation . . . . . . . . . . . 14 Why feedback works - information processing interpretation . . . . 18 Why feedback works - analysis . . . 21 Educational Taxonomies . . . . . . . 25 State Anxiety . . . . . . . . . . . . 28 III. METHODOLOGY . . . . . . . . . . . . . . . . . 3O Selection of the Sample . Instrumentation . . . . . Content Review Quizzes Examinations . . . . Procedure . . . . . Design . . . . . Hypotheses . . . Analysis of the Data Summary . . . . . . . o o o o o o o o o w J: IV. ANALYSIS OF RESULTS 0 O O O O O O O O O O O I ”3 Study I - Knowledge Stating . . . . . 43 Retention (H1) . . . . . . . . . . 43 State Anxiety . . . . . . . A8 Study II - Rule Applications . . . . N8 Retention (H2) . . . . . . . . . . 48 State Anxiety . . . . . . . . . . . 51 Summary of Results . . . . . . . . . 52 iii Chapter Page V. SUMMARY AND CONCLUSIONS . . . . . . . . . . . 53 Summary . . . . . . . . . . . . . . . 53 Problem . . . . . . . . . . . . . . 53 Purpose . . . . . . . . . . . . . . 54 Methodology . . . . . . . . . . . . 54 Results . . . . . . . . . . . . . . 55 Conclusions . . . . . . . . . . . . . 56 Limitations . . . . . . . . . . . . 57 Discussion . . . . . . . . . . . . 58 Recommendations for Future Research 61 BIBLIOGRAPHY O O O O O C O I O O O O O O O O O O O O O 63 APPENDIX A - PRETEST ON KNOWLEDGE STATING . . . . . . 72 APPENDIX B - PRETEST ON RULE APPLICATIONS . . . . . . 73 APPENDIX C - POSTTEST ON KNOWLEDGE STATING . . . . . . 7“ APPENDIX D - POSTTEST ON RULE APPLICATIONS . . . . . . 75 iv LIST OF TABLES Page Table 1.1 Studies Comparing the Effects of Various Delays of Informative Feedback on Verbal Learning . . . 5 3.1 Schematic Representation of the Experimental Design and Timeline . . . . . . . . . . . . . . 35 3.2 Experimental Design . . . . . . . . . . . . . . . 38 A.1 Means and Standard Deviations of Total Raw Score on the Measure of Retention of Factual Informa- tion 0 O O O O O O O O O O O O O O O I O O O O O “)4 A.2 Summary of Univariate Unweighted Means Analysis of Covariance on Students' Total Raw Scores for the Retention of Factual Information . . . . . . AA A.3 Means and Standard Deviations of Total Raw Score on the Measure of Retention of Familiar Factual Information I O O I I I O O I I O O O O O O O O ’46 A.A Summary of Univariate Unweighted Means Analysis of Covariance on Students' Total Raw Scores for Familiar Test Items . . . . . . . . . . . . . . A6 A.5 Means and Standard Deviations of Total Raw Score on the Measure of Retention of Unfamiliar Factu- al Information 0 O O O O O O O O O O O O O O O O ”7 A.6 Summary of Univariate Unweighted Means Analysis of Covariance on Students' Total Raw Scores for Unfamiliar Test Items . . . . . . . . . . . . . A7 A.7 Means and Standard Deviations of the Subjects' Responses to the question, "I felt that the questions on the quiz were arranged in order of difficulty" . . . . . . . . . . . . . . . . . A9 A.8 Summary of Univariate Unweighted Means Analysis of Variance on Students' Responses to the Ques- tion, "I felt that the questions on the quiz were arranged in order of difficulty" . . . . . A9 V Table Page A.9 Means and Standard Deviations of Total Raw Score on the Measure of Retention of Rule Applications 50 A.1O Summary of Univariate Unweighted Means Analysis of Covariance on Students' Total Raw Scores for Retention of Rule Applications . . . . . . . 51 vi CHAPTER I THE PROBLEM Introduction and Need A practice suggested by many of the currently popu- ular textbooks for classroom teacher training (Kulhavy, 1977), and which serves as the basis for most programs for computer assisted instruction (Sturges, 1978), is that of providing students with immediate feedback regarding the correct responses to any questions they attempt to answer. The practice, which stemmed from the theory of operant psy- chology (Skinner, 195A, 1968), is based on the assumption that providing students with immediate feedback can serve to reinforce such desired behaviors as answering questions cor- rectly and, thus, should increase the probability of those responses occurring again in the future (Carlson & Minke, 1975; Keller, 1968; Skinner, 195A, 1968). The notion has been supported by an abundance of research, has survived the lag that typically follows any discovery by science (Glass, 1971, 1972; Feyerabend, 1978) and is finally beginning to pervade most levels of educational practice (Glaser, 1979). The efficacy of the practice, however, is by no means clear and has recently been seriously challenged by some of the proponents of information processing theory, 1 2 the perspective that has since replaced the behavioral school as the predominant force behind most learning re- search today (Anderson, Kulhavy & Andre, 1971, 1972; English & Kinzer, 1966; Kulhavy, 1977; Kulhavy & Anderson, 1972; More, 1969; Phye & Baller, 1970; Sassenrath & Yonge, 1968; Sturges, 1969, 1972, 1978; Sturges & Crawford, 196A; Suber & Anderson, 1975). These researchers, assuming that human learning can most beneficially be viewed in terms of a pro- cess analagous to the structure and flow of the algorithms used to solve problems with computers, have argued that feedback: (1)merely serves as a reference to which learners can check the adequacy of that which they have stored in memory (Guthrie, 1970); (2)can only be effective if the in- dividuals have some base of knowledge to compare the infor- mation with; and (3)cannot be effective unless its presen- tation has been delayed until students have been given an Opportunity to cease "perseverating" over their original re- responses (e.g., Kulhavy, 1977). Their findings, known as the Delayed Retention Effect, have been quite consistent: On acquisition, or immediate retention, occasion- ally immediate feedback is better, but generally, there is no significant difference between immedi- ate and delayed informative feedback. Retention tests 1 to 7 days later show a different picture: Retention following delay of informative feedback is generally superior to that following immediate informative feedback. It is most important to note that in none of these studies has long-term retention following immediate informative feed- back been superior to that with delayed feedback! (Sturges, 1978; p. 378) 3 Such discrepancies between the findings of recent investigations and the prescriptions which emanated from the behavioral school, have already led some investigators to call for a re-evaluation of the effectiveness of providing immediate feedback (Pound & Bailey, 1975), as well as for the dismissal of operant notions from the study of school learning in general (Shulman, 197A). It can, however, be argued that the question of whether feedback should be given immediately or delayed has yet to be studied satisfactorily in the context of typical classroom situations or in rela- tion to many of the variables that now appear to be poten- tially relevant under such conditions. The problem has not been that researchers have failed to conduct comparisons of the relative effects be- tween immediate and delayed feedback on learning. Quite to the contrary, the literature has been richly filled with numerous such studies using both animal (Raymond, 195A; Renner, 1965) and human subjects (Ryan & Bilodeau, 1962; Sturges, 1978; Sturgis & Crawford, 196A), motor (Lorge & Thorndike, 1935; Noble & Alcock, 1958; Saltzman, Kanfer & Greenspoon, 1955) and verbal learning tasks (Crawford, 1966; Kulhavy & Anderson, 1972), and such dependent measures as running speeds, trials to acquisition, achievement and re- tention. The manipulations which have been attempted can be viewed as having been too far divorced from most classroom learning situations, and the resultant findings too u inconsistent, to allow the derivation of any unequivocal conclusions that might be relevant outside of the paradigms from which the studies were generated. A number of flaws, in this regard, can be identified. Regarding external va- lidity (Campbell & Stanley, 1963), for example, most of the studies on verbal learning used either computer assisted instruction or programed texts, situations quite well con- trolled, but nonetheless different from the quiz and testing environments usually found in the classroom (see Table 1.1). Since feedback from a computer, or the novelty of using a programmed text, could have been reinforcing in themselves, it is still not known whether reinforcement actually could have been discounted as being the causal variable, or if de- layed feedback would still be more effective if it were pro- vided in the form of a printed document as would more likely be the case in a typical classroom. A second limitation of the previous research is ob- vious when one considers the various outcomes that have been studied. Some researchers considered the time required for mastery most important, others were concerned with the num- ber of errors that subjects made before learning their task to perfection, while still others viewed either immediate or delayed achievement test scores as being the primary locus of any effects -- and the results varied dependent upon the specific tasks and measures used (cf., Crawford, 1966). Since most classroom learning situations eventually culmi- nate with tests of retention given a week or more after Table 1.1.--Studies Comparing the Effects of Various Delays of Informative Feedback on Verbal Learning Anderson, Kulhavy & Andre, 1971 Anderson, Kulhavy & Andre, 1972 Brackbill, Adams & Reaney, 1967 English & Kinzer, 1966 Hockman & Lipsitt, 1961 Kulhavy & Anderson, 1972 More, 1969 Noble & Alcock, 1958 Phye & Baller, 1970 Ryan & Bilodeau, 1962 Sassenrath & Yonge, 1968 Sturges, 1969 Sturges, 1972 Sturges, 1978 Suber & Anderson, 1975 Computer Computer Marbles Printed document Buzzer Printed document Trainer-tester card Teaching machine Read aloud Read aloud Slide projector Slide projector Slide projector Computer Printed document 6 informative feedback is provided, it seems only logical that the selection of that precise dependent variable would be necessary in order for the results to be generalizable to the classroom. Another problem emerges when one considers the be- havior that typically has been analyzed in studies of the Delayed Retention Effect, in relation to how people are cur- rently thought to process information. Most information processing theories distinguish between recall and recogni- tion, assuming that recall involves both processes, but that recognition entails virtually no retrieval activity (see, Gagne' & White, 1978; Mueller, 1980). Most studies of the Delayed Retention Effect utilized a design in which subjects were administered a limited number of recognition-type items and given either immediate or delayed feedback and, after a period of approximately one week had elapsed, were adminis- tered a posttest on the exact same items (Kulhavy, 1977). If feedback is presumed to serve merely as a confirming fac- tor of students' understanding of the topic being tested, and the mode of retrieval is irrelevant, then the above de- sign might be considered adequate in this regard. If recall and recognition are qualitatively different behaviors, or if the cues recognized in a familiar multiple choice question do not reflect one's ability to recall such information, then the repeated use of identical multiple choice questions might not at all reflect that which would be expected in the typical classroom testing situation. 7 One study, namely Sturges (1978), investigated whether the beneficial effects of delayed feedback would generalize to either unfamiliar items or familiar questions posed in a recall-type format, and his results cast doubts on the adequacy of the design in this regard. Giving imme- diate, delayed or no feedback to students answering recogni- tion-type items, he found little or no differential effect on how well they later answered the same questions measured with a recall-type format. The generalizability of the previous studies, thus, might well be limited to just those situations where students have to recognize the answers to questions they have previously seen in the same format -- a situation seemingly not typically confronted by most stu- dents in the classroom. A more reasonable alternative might be to use short answer-type items demanding recall which, for at least a portion of the posttest, are new but measur- ing the same material, at the same level, as the items on the pretest. In order to maintain adequate levels of relia- bility and validity, of course, a content area lending it- self to such a measure would have to be used (Wesman, 1971). A fourth problem exists in that the various periods of delay which have been compared have not always been those which would allow meaningful comparisons between the two theory-based prescriptions. Reinforcement theory suggests that any delay exceeding a span of only a few seconds should result in some decrement in learning. Information process- ing theory, on the other hand, suggests that delays of at 8 least twenty minutes are essential for enhancing retention (Sturges, 1978). Many studies have used theoretically con- flicting definitions of delay, however, with some consider- ing delays of up to twenty minutes to be immediate, and others labeling intervals of only ten seconds as delayed (Sassenrath & Yonge, 1968). Since the question of primary importance to the present study was whether immediate (ac- cording to reinforcement theory) or delayed feedback (ac- cording to information processing theory) is more effective at enhancing retention in the classroom, only delays of a few seconds and at least twenty minutes were considered relevant. Finally, assuming that the central hypothesis sup- ported by the above mentioned studies was not merely an ar- tifact of their designs, there exists the additional problem that the extent to which learning has been enhanced in pre- vious studies has varied considerably (Sturges, 1978), with no explanation given for why. One plausible hypothesis, as Gagne' (1977) points out, is that while researchers have been extremely lax in identifying or controlling the Spe- cific outcomes of learning they were studying, differences are likely to be found on precisely such variables: A serious consideration of practical knowledge of learning, I believe, must go beyond the most general principles of the learning process, such as contiguity and reinforcement. One must re- cognize that learning results in retained dispo- sitions which have different properties, differ- ent organizations, and which accordingly require that different conditions be established for their attainment. I call these learned 9 dispositions by the general name of capabilities. Their five main varieties are called intellectu- al skills, cognitive strategies, verbal informa- tion, motor skills, and attitudes. Learning in- vestigators and theorists often use these cate- gories (not always by these names) in their ac— counts of learning -- they are not entirely un- familiar. Yet these same investigators often do not choose to make these distinctions explicit. (p. iv) Given the corrective role attributed to feedback by the pro- ponents of information processing theory (e.g., Guthrie, 1970), along with the fact that learning and cognitive the- oreticians have long felt that initial and complex learning probably involve different memory structures and processes (e.g., Gagne', 1977; Piaget, 1952), one might expect to at- tain less variability and/or differential effects if the learning outcomes were actually controlled in the study. Since information processing theory does not account for situations where an informational base is unavailable for reference, an obvious hypothesis would be that immediate feedback might serve better at enhancing the retention of factual knowledge (where references for comparison might be minimal), and delayed feedback better at facilitating the retention of rule applications (which derive meaningfulness through reference to such facts; Gagne' & White, 1978). An additional argument in support of the need for further studies on the present topic is the fact that the notion of test anxiety (e.g., Spielberger, 1972), while cur- rently thought to be closely related to both learning and recall (Mueller, 1980), has been given only cursory 10 attention at best within the present debate. Although anxi- ety has been shown to have differential effects on recogni- tion versus recall (Eysenck, 1975), as well as short-term memory versus long-term retention (Levonian, 1972; Uehling, 1972), only one of the studies reviewed even considered it as a relevant variable and it failed at that to compare im- mediate versus delayed feedback conditions on that measure (Sturges, 1978). Purpose The purpose of the present study was to compare the relative effects of immediate and delayed feedback on the retention of two categories of learning outcomes, the stat- ing of facts and the application of rules, in the context of an actual classroom setting. The study, specifically, was an attempt to correct for five limitations of previous stud- ies conducted on the same topic, namely: (1)the possible confounding of reinforcement and delayed feedback; (2)the use of dependent measures atypical to most classroom learn- ing situations; (3)the use of a potentially biased measure of retention; (A)the use of theoretically inconsistent defi- nitions of immediate and delayed feedback; and (5)the fail- ure to consider and/or control for the particular outcomes of learning desired. Two studies were conducted, one for each of the out- comes of learning investigated. In each study the effects 11 on retention of providing informative feedback immediately (i.e., within two seconds) and delayed (i.e., after approx- imately 20 minutes) were compared. The questions to be answered were: Was there a dif- ferential performance on the two levels of questions across the groups? If so, did the evidence solely support a be- havioral interpretation, an information processing inter- pretation, or was there evidence of a functional interaction existing between the two theories? Were there corresponding differences existing between the groups on a measure of state anxiety? Overview The following chapters of this work describe, in greater detail, the study as outlined in this chapter. In Chapter II, a review of the literature relevant to the ef- fects of test item feedback is presented from the perspec- tives of behavioral and cognitive theories, as well as those views which appear to support the possibility of a function- al interaction between the two. The methodology that was used is described in Chapter III, including explications of the sample selections, procedures, instrumentation, design, statements of the hypotheses in their testable forms and the statistical model that was used for analyzing the data. Chapter IV consists of the actual data analyses, results and interpretations of the findings. Finally, a discussion of the results, the limitations of the study, recommendations 12 for future investigations on the topic, and conclusions are contained in Chapter V. CHAPTER II REVIEW OF RELEVANT LITERATURE Reviews of educational, experimental and instruc- tional psychologies reveal rather abundant bodies of litera- ture on the three individual topics of concern in the pres- ent study, namely the differential effects of immediate and delayed informational feedback on learning, and the explana- tory utilities of both learning hierarchies and anxiety. Research investigating the effects of those variables in typical classroom settings, or attempting to study the func- tional interactions which might exist among them, however, has been quite noticeable by its absence. The following re- view will focus on each topic separately, highlighting the literature most relevant to the present project. Informative Feedback There appears to be virtually universal agreement among those who study learning that informative feedback is the most fundamental condition in determining performance (Deese & Hulse, 1967). Some question has been raised over whether positive or negative feedback, or some combination of the two is actually most effective (Barringer & Gholson, 1979), but psychologists generally seem to agree that tell— ing students whether or not their answers to questions are 13 1A correct will increase the amount of information they will later remember (Kulhavy, 1977). Two related issues which psychologists have not been able to agree upon, though, are why feedback works and what effects should be expected as a result of delaying its presentation. Their disagreements have followed clearly defined paradigmatic lines. Why feedback works - behavioral explanation. One eXplanation for why informative feedback enhances learning is that it serves to reinforce the behavior of correctly re- calling the answers to questions and thus, by definition, increases the probability of such responses occurring again in the future (Kling & Schrier, 1971). That explanation, which developed from the behavioral school of psychology, was generalized from voluminous numbers of studies conducted primarily on animals and non-verbal or non-meaningful verbal stimuli. While prOponents of that school have often been criticized for making such generalizations without first having put the specific applications through rigid eXperi- mental tests (e.g., Kulhavy, 1977), the basic premises of behaviorism did warrant, even encourage, such practice. The behaviorist's philosophy was based on three primary tenets: (1)that the only way psychology could achieve the status of being an objective science was to limit their study to the analysis of observable behavior; (2)that the causes of be- havior lie in the environment and can therefore be manipu- lated and investigated; and (3)that all behavior is 15 comprised of simple elements that are universal to all ani- mals (Schwartz, 1978). After a period of refining their ideas, behaviorists (e.g., HUll, 1943, 1951, 1952; Skinner, 1938, 19A8, 195A, 1958; Spence, 1956, 1960; Thorndike, 1898, 1911; Watson, 1913) came to view a reinforcer as being any stimulus whose contingent presentation increased the likelihood of the oc- curence of the Operant behavior that preceded it, and their task as attempting to identify the relevant variables and parameters for which that notion held. Their studies were limited primarily to just one or a few animals, with inter- est concentrated on the observation of precise changes over time, and the relevant questions became: What types of stimuli were reinforcing?; How long of a delay could exist between the Operant and the reinforcer?; and What were the effects of various schedules of reinforcement? A few repre- sentative examples of their studies should help to portray how they came to deduce the effects they eventually pre- dicted of feedback. GreenSpoon (1955), in one of the earlier studies with humans, reinforced his subjects for using plural nouns in their speech by saying "hm-mmm" anytime they used plu- rals. The result was a large increase in their usage of plural nouns. Hidlum and Brown (1956), similarly, used the word "good" to reinforce people being interviewed on the telephone when they were responding in certain ways. Sub- jects who were reinforced when they endorsed the topics of 16 the interview shifted toward a more favorable attitude, while those reinforced for expressing a negative Opinion began to emit significantly greater numbers of those views. The implications of such studies were seemingly clear: the behavior of humans, like that of animals, could be modified by controlling their environment. Skinner in fact, as Merrill, Kowallis and Wilson (1981) point out, proclaimed feedback as a reinforcer on precisely such grounds: Skinner (195A) is credited with providing the im- petus needed in applying psychological principles to the design of instruction. The programmed in- instruction movement began on this note: "There is a simple job to be done. The task can be stated in concrete terms. The necessary tech— niques are known (Skinner, 195A, p.97)." Confi- dent that learning principles derived from S-R research could be applied directly to school learning, programmed instructional principles included the following: 1. Instruction should require overt responses from the student. 2. Instruction should provide immedi- ate reinforcement to student re- sponses. 3. Instruction should contain small steps. A. Instruction should elicit few or no errors in student response. 5. Instruction should be self-paced. (pp. 302—303) The debilitating effects of delaying reinforcement, similarly, have been studied primarily with animals. Grice (19A8), for example, showed that rats could not learn to 17 select a white rather than a black alley if the food they received at the end of the white alley was delayed for much longer than five seconds. Actually, they had to select be- tween a white or black alley, then run through a gray por- tion and, finally, into a gray goal box that contained food if the beginning of the alley was white. In order to elim- inate proprioceptive cues as an explanation, the white and black entrances were shifted from left to right randomly. The reason for the complexity of Grice's design was the same reason why it was believed that humans could not be used as subjects. If the study had been conducted keeping both al- leys entirely white or black, without gray sections in the middle or gray goal boxes, the debilitating effects of de- laying reinforcement would have decreased substantially, as the alleys themselves would have acquired conditioned rein- forcing properties (Deese & Hulse, 1967). As Kling and Schrier (1971) point out: Where language or other differential responses do not continue through the delay period, acquisi- tion is greatly impaired by even slight delays of reinforcement. As might be expected, young chil- dren can tolerate longer delays than completely nonverbal organisms, but children are signifi- cantly hindered by delay intervals (such as 30 seconds) which adults would probably find quite inconsequential (Hockman & Lipsitt, 1961). (p. 688) Other animals have shown similar abilities to persevere through relatively long delays between response and rein- forcement if conditioned reinforcers were present (e.g., Kelleher, 1956), but the results have been quite conclusive: 18 any delay, even with conditioned reinforcers present, will result in some decrement in learning. Why feedback works -information processing inter- pretation. While behaviorists did not study humans in investigating the effects of delaying reinforcement because they felt that language would serve as a secondary reinfor- cer and present a confounding to the results of their stud- ies, other investigators, interested primarily in human processing of language and information, did study that pre- cise phenomenon and their results were the opposite of what one would expect from the behavioral interpretation. Brackbill and her associates (Brackbill, Adams & Reaney, 1967), for example, had children learn concepts by selecting the correct one of two pictures in a series of paired pres- entations. Informative feedback was either given immediate- ly (i.e., within two seconds) or delayed by 30 seconds. In measuring the number of errors made during learning, no sig- nificant differences were found between the groups. In sub- sequent tests requiring the students to relearn the same in- formation, however, it was found that initially delaying the feedback had actually served to enhance later performance. While Brackbill has been criticized for not ade- quately controlling the amount of time that subjects were exposed to informative feedback (e.g., Crawford, 1966), re- searchers studying the effect in the area of computer as- sisted instruction have adequately controlled such 19 conditions and their results have remained quite consistent (Kulhavy & Anderson, 1972; Phye & Baller, 1970; Sturges, 1969, 1972, 1978; Surber & Anderson, 1975; Sturges & Crawford, 196A). Sturges (1978), in a fairly representative study, gave students a computer assisted test covering their regular class material, during which informative feedback was either not given, or was delayed for a period of two seconds, 20 minutes or 2A hours. No differences were found on the computer assisted test but, on a measure of retention given one week later, those students whose feedback was ini- tially delayed by either 20 minutes or 2A hours scored sig- nificantly higher than either of the other groups. Information processing theorists account for learn- ing by assuming that people maintain an internal reference level that serves as the basis for the self-regulation of behavioral sequences. As Guthrie (1970) points out: Given that a stimulus context is sufficient to elicit a relatively apprOpriate response, the re- sponse is modified by an error-nulling process. That is, the response is compared to the internal reference level, any error in the response is de- tected, and successive responses are corrected on the basis of the information resulting from the comparison. In this system, learning is produced by the acquisition of a memory trace and a per- ceptual trace. The memory trace is basically the S-R associative strength for given stimuli and responses. The perceptual trace, however, is the internal representation of stimuli which provides a basis for recognition of all kinds, including the recognition of responses as correct or incorrect. (p.2) Obviously, unlike the behaviorists, information processing theorists have been willing to assume a number of internal 20 functions. Specifically, these theorists have assumed that a human being is, among other things, a processor of infor- mation whose behavior can be simulated with algorithms like the ones used to solve problems with computers (Mayer, 1977). Once developed, such algorithms can be studied as models of human learning (e.g., Lindsay & Norman, 1972; Mayer, 1977; Miller, Galanter & Pribram, 1960; Newell & Simon, 1972), serve as formal statements of the learning theory, and are readily manipulatable and testable. The principal areas of interest of researchers of this school have been in the capacities and parameters of human memory, how information is encoded, stored and retrieved, and the nature of feedback (Gagne', 1977). Feedback in such a sys- tem, as Gagne' points out, is seen as the final step in the learning process: Learning is a process which appears to require the closing of a "loop" which begins with stimu- lation from the external environment. The final link of this 100p is an event which also has its origin outside the learner, in his environment. Feedback is provided by the learner's observation of the effects of his performance. This is the event that provides the learner with the confir- mation (or verification) that his learning has accomplished its purpose. ... Although feedback usually requires a check which is external to the learner, its major effects are obviously internal ones, which serve to fix the learning, to make it permanently available. (p.57) Informative test feedback, accordingly, is viewed by re- searchers in this area as having one of two effects. It can, on the one hand, serve to inform students when they have responded correctly. This role is seen as a necessary 21 function in terms Of the model, but superfluous as students are believed to know when they comprehend material and do not need feedback regarding correct responding (Kulhavy, 1977). Supposedly more importantly, however, is the role that feedback is believed to have in eliminating errors and substituting correct information in their place. Kulhavy and Anderson (1972) have hypothesized that students tend to perseverate over their responses and, as a result, need to be given an Opportunity to forget their in- correct answers before feedback can successfully identify such errors and act to replace the faulty information. Kulhavy (1977), in an extensive review Of the Delayed Re- tention Effect, concluded that their notion has been well supported and, as a result, that practitioners should: (1) insure that students will be able to comprehend the materi- al and identify errors in their cognitive structure; (2) provide feedback regarding any mistakes the learners make; and (3)insure that any feedback is given after a period of delay rather than immediately. Why feedback works - analysis. Both the behavioral and information processing school's explanations for why feedback works, as well as their subsequent research, are limited in a number Of respects with regard to their appli- cability to educational practice. The major problem in the behaviorist's applications, as Glaser (1976) points out, probably lies in their failure tO identify the parameters of 22 reinforcement that are relative to school learning: Contingencies Of reinforcement pervade the acqui- sition of competence. However, with the strong emergence Of cognitive psychology, and with awareness Of the fact that the bulk Of our knowl- edge about reinforcement is derived from animal studies in simple task situations and from human experimental contexts in which conditions con- strain subjects to employ limited behavioral pro- cesses, we are in some danger of ignoring the po- tential influence Of reinforcement on complex performance. There is, on the one hand, a strong suggestion of discontinuity in the operation Of reinforcement when moving from simple to higher- Order behaviors. On the other hand, the view that seems best supported at the moment is that the mechanisms Of reinforcement are similar at all levels Of development, but variations in re- sponse organization result in different pheno- typic manifestations (Estes, 1971). As individu- als mature, human behavior is organized into higher-order routines and strategies, and it is these large cognitive organizations whose proba- bilities Of occurence are modified by reinforcing contingencies. It is the nature of the unit of response that may distinguish the mature human learner, whereas the Operation Of the principles Of reinforcement may be similar for different species and different levels Of development and competence. (p. 20) While the behaviorists have been criticized for having as- sumed too much (e.g., Kulhavy, 1977), probably more impor- tant has been their failure to consider enough. They can be criticized mostly for having overgeneralized the effects to be expected from feedback, not having adequately speci- fied the behavior they were attempting to modify in the classroom, not having considered adequate dependent measures Of classroom learning and for ignoring too many Of the con- structs thought by a majority Of psychologists to be rele- vant to complex human learning (e.g., the capacities Of memory, learning hierarchies and developmental stages). 23 PrOponents Of information processing theory, while having focused on many Of the complexities of human learning that the behaviorists failed to consider, have also been guilty Of a number Of experimental and logical shortcomings with respect to their suggested use Of feedback in educa- tional practice. The problems in their methodology, de- scribed earlier in the present paper, served as the prin- cipal justification for the research described herein -- the Delayed Retention Effect simply has not yet received ade- quate investigation within the context Of typical classroom situations. Such problems are easily remedied, however, and documenting the effects on retention Of delaying the presen- tation Of informative feedback in the classroom is a task that is constrained only by time. If Brackbill and Kulhavy and their respective associates are correct, and delaying feedback not only fails to hinder students' retention in the classroom, but actually enhances it, serious question will have to be brought on the practices that psychologists have already recommended for school learning. An equally significant problem exists, though, re- garding what such results would allow one to conclude. Phillips (1980) recently issued apparently necessary re- minders tO the field that deducing applied prescriptions from, or support for, educational and psychological theories is quite bound to adequate use Of logic, and that psycholo- gists have Often failed in this regard. Considering the following rationale, Offered by Kulhavy (1977) as support 2A for the argument that reinforcement ought tO be discounted as a viable explanation for the Delayed Retention Effect, critical reviews Of the nature suggested by Phillips are Obviously sorely needed: The problem, Of course, is that behaviorists were a bit hasty in assigning functionally reinforcing properties to feedback. In the laboratory 3 re- inforcer is defined as some stimulus that in- creases the future probability Of the response which it follows. For example, suppose that a laboratory rat or pigeon is deprived Of food, and then required to perform some target response in order to eat. When the animal responds cor- rectly he receives a pellet Of food, and to the degree that he will emit the same response again under similar conditions, the pellet assumes re- inforcing prOperties. There is no good reason to believe that this same sequence Of events will occur in (say) a programmed text -- unless one contends that students hunger for knowledge! Obviously, classroom learners rarely perform un- der potent contingencies such as physical depri- vation, and the stimulus-response environment Of the program is constantly undergoing change, a condition at odds with the typical laboratory setting. Of course no one is trying to say that some program stimuli may not be functionally re- inforcing for some students. But to assume that the instructional behavior of most pupils is sub- ject to the control Of feedback statements is tO violate the Operant manifesto: only that which reinforces is a reinforcer. (p.213) First, while Kulhavy has apparently accepted the premise that "students do not hunger for knowledge," regardless Of the validity Of that premise (which is quite questionable), it could not have been made by proponents Of reinforcement theory since they have explicitly avoided such concepts as motivation (McKeachie, 197A). Second, he presumes that "classroom learners rarely perform under potent contin- gencies such as physical deprivation," yet Offers no support 25 for that statement (and likely could not). Finally, while he concludes that "to assume that the instructional behavior Of most pupils is subject to the control of feedback state- ments is to violate the Operant manifesto: only that which reinforces is a reinforcer," not only is his only premise regarding reinforcement the definitional one, but his own data suggests that student retention is subject to the control Of precisely such stimuli. Critical experiments have seldom represented more than the wishful hoping Of scientists and little has been Offered in the present debate to suggest that it will be any different. The question is not whether student behavior is controlled entirely by reinforcement or information but, rather, what variables and in which combinations can best lead to a more useful theory Of school learning. Both schools have Offered much promising guidance to educational psychology and to fail to consider their potential func- tional interactions at this point would seemingly be naive indeed. Educational Taxonomies Educators and educational researchers have prOposed a number Of taxonomies Of educational Objectives (e.g., Bloom, 1956; Ebel, 1965; Gagne', 1977; Merrill, 1971; Walbesser, 1965), designed to enhance both teaching and research. Their Obvious utility has long been professed by educational researchers (Gagne', 1977), and applied in 26 instructional designs (Seddon, 1978), but seldom have they actually been incorporated into the designs Of educational research (Barker & Hapkiewicz, 1979). Not only have re- searchers failed to analyze the differential effects Of the variables they were studying on various levels Of learning, but even the levels of the stimuli used in learning studies has seldom been specified. Yet while such omissions have already received an abundance Of criticism (e.g., Barker & Hapkiewicz, 1979; Gagne', 1977) which need not be reiterated here, the question Of which Of the taxonomies or levels thereof should be used in educational research is not one for which a consensus Of Opinion is likely to be immediately found. Each taxonomy suggests a number Of levels that a course's content might actually entail, but the concordance between their various delineations is as yet unknown (Seddon, 1978) and in itself is a topic for future inves- tigation. Their main weakness, namely that they have not been closely related to learning theories in general (Merrill, Kowallis & Wilson, 1981), has prompted the recent development of hierarchies that are theoretically based (e.g., Gagne' & Briggs, 197A, 1979; Gagne' & White, 1978; Merrill, Olsen & Coldeway, 1976) and which appear to be more promising as controlling factors for research. The model prOposed by Gagne' and White (1978) was utilized for the present study primarily because of its simplicity, concor- dance with previous taxonomies, close relation tO 27 information processing theory and agreement with the taxono- mies currently used for teaching chemistry (Heller, 1979; Smith, 1972). The approach taken by Gagne' and White, basically, is one in which memory structure is merely interposed as a mediating variable between instruction and performance: ... memory structures are to be related to two different domains Of learning outcomes, knowledge stating and rule application. These two categor- ies Of outcome are usually viewed as distinguish- able classes Of human performance (Gagne', 1972; Olson & Bruner, 197A) and are readily identifia- ble as classes Of Objectives in many forms Of school learning (Gagne' & Briggs, 197A). For both categories, it is possible to define reten- tion as a dependent variable and to specify a set Of Operations applicable to its assessment. Likewise, it appears that for both these cate- gories one or more meanings may be assigned to learning transfer which suggest the design Of measures for that outcome. (p.189) Smith (1972), similarly, concluded that the same two types Of outcomes as prOposed by Gagne' and White are those which are needed to enable a student to complete any given chem- istry task: (1)gaining a knowledge Of chemistry concepts and their relations and (2) learning the psychological proce- dures that allow one to manipulate various concepts and principles in order to produce the required solutions. The model was used in the present study both for Specifying the outcomes being investigated, and to enable the study Of the possible functional interaction between the theories. 28 State Anxiety Test anxiety, the supposed classically conditioned arousal Of tension and apprehension that people are thought to experience in typical classroom testing situations (Sieber, 1980), has long been considered by many to be central to both learning and performance (Mueller, 1980). Theoretically, the notion has been explained in terms Of both behaviorism (Hull, 19A3; Spence & Spence, 1966) and information processing theory (Mueller, 1980), as well as Freudian psychoanalysis (Sarason, 1980), and its attractive- ness as a construct for explaining what would otherwise be residual variance in experimental designs, has always been high. Behaviorists' interest in anxiety, in general, stemmed primarily from Hull's (19A3) theory Of learning, in which the strength or tendency Of occurence Of an organism's responses was viewed as being the multiplicative result Of its habit strengths and drives. If anxiety is a drive then an increase in that drive, according to Hull, should result in a concurrent increase in response strength. Such in- creases could facilitate learning if properly directed, but more likely would cause students to spend time attending tO task-irrelevant events if tOO high (Mandler & Sarason, 1952). Interest in the notion clearly waned under Skinner's redirection (cf., Spielberger, 1972), but was so appealing that Taylor's Manifest Anxiety Scale (1951, 1953), even today the most widely used measure Of anxiety, was 29 specifically constructed as a measure Of Hull's postulated Drive (Sarason, 1980). Interest in the construct again flourished after Spielberger (1966) differentiated between two types Of anxiety that people might experience in testing situations, namely the relatively stable personality characteristic Of trait anxiety and the transitory condition Of state anxiety in which people are thought tO be responding to perceived threatening situations (Sieber, 1980). In a recent review by Mueller (1980), however, it was readily apparent that while test anxiety has been theorized as having numerous roles in memory and information processing, little research has yet been accomplished to substantiate those notions. Its inclusion in the present study was for exploratory purposes since many Of the theoretical views suggest effects with respect tO short-term memory as Opposed tO retention (e.g., Walker, 1958), recognition as Opposed to recall, and the depth and breadth Of processing involved (Mueller, 1980), all closely related to the variables which were investigated in the present study. CHAPTER III METHODOLOGY This chapter describes the methodology that was employed in the present study, including the sampling me— thod, characteristics of the sample, instrumentation, proce- dures, hypotheses and statistical design used. Rationales for the specific sampling method, instrumentation, proce- dures and statistical methods selected are presented within each Of those respective sections. Selection Of the Sample The students who served as subjects for the present study comprised the entire pOpulations Of three sections of a freshman-level introductory chemistry course taught by the same instructor at a mid-western community college with an annual enrollment Of 10,500 students. Based upon the char- acteristics of the college's student body, in general, the average subject was a first year student, more likely to be female rather than male, 27 1/2 years Old, caucasian and en- rolled for slightly less than eight credit hours. Each stu- dent, as described in the section on procedures to follow, was randomly assigned, independently for each Of the two studies that were conducted, to one Of two experimental con- ditions on the first day Of each study. The students, as 30 31 such, had a probability Of 0.50 Of being assigned to either Of the treatment groups. Three points explain why the spe- cific populations and sampling methods selected were chosen. First, as Berliner (1975) pointed out, studies on school learning are not likely to be very believable unless they are conducted in natural classroom settings. For this reason the present study was designed so that: (1)students in regularly scheduled classes could be used as the subjects Of the investigations and (2)actually planned course content and examinations could be used as the stimuli and dependent variables, respectively. Second, in order to enhance both the testability Of the manipulations, as well as the subsequent power Of the analyses, random assignment to the treatment conditions was accomplished, independently for each of the two studies, so that "students" could serve as the units of analysis. Addi- tionally, the study was conducted on three classes so that there were three inherent replications. Third, based on Sturgis's (1978) findings, it was decided tO use measures Of recall rather than recognition as the dependent measures Of the study. Thus the selection Of a course lending itself to such measures was imperative (Mehrens & Lehmann, 1973). Three introductory college chem- istry sections, taught during the same term, on the same days and by the same instructor, were selected because their content did lend itself to such measures and any differences between them should not have adversely affected either the 32 homogeneity Of the study's variance or the general accepta- bility of the manipulations themselves. Instrumentation Content Review Quizzes. Two content review quizzes, each comprised Of 20 items answerable by the students' re- calling or deducing the one correct answer, were constructed by the course instructor. The items on the quiz for Study I (Appendix A) were designed to measure students' proficiency in stating facts, namely 10 items requiring students to give the names Of elements for which they were given the symbols, and 10 items which required students to supply the symbols for elements that they were given the names Of. The items on the quiz for Study II were designed to measure the stu- dents' ability tO apply rules, namely 10 items that required them to deduce the names Of the compounds that would be cre- ated through the combination Of various elements, and 10 items requiring them to deduce the combination Of elements that would be needed to form various compounds (Appendix B). The internal consistencies for the twO quizzes for Study I and Study II, computed using Cronbach's Alpha (Cronbach, 1951), were 0.89 and 0.9A, respectively. Informative feedback for the immediate feedback condition was given by having the answers to each question printed directly on the quizzes with a Latent Image process (Pound & Bailey, 1975). The process is one in which teach- ers, after having their tests typed onto the usual spirit 33 ditto masters, have their typists replace the normal ditto carbon with a Latent Image transfer sheet (e.g., A.B. Dick Form 97-1001), then proceed to type in the correct answers immediately adjacent to each question and, finally, to run the master on a regular ditto machine using standard paper as usual. The resulting test forms, while displaying all Of the test questions and containing each of their respective answers, totally conceal the latter until they are develOped by a Latent Image developer (e.g., A.B. Dick model 97-3100) which looks like, costs no more, and is used in the same manner as a highlighter. Informative feedback for those conditions receiving delayed feedback was accomplished by giving those students a printed copy Of the questions with their respective answers approximately twenty minutes after the time they had completed taking the quiz. Additionally, the quiz for Study I contained three items, designed by the experimenter and answerable on four- choice Likert-type scales, which attempted to measure the degree Of state anxiety the subjects may have experienced as a result Of the particular forms of feedback they received. The quiz forms used in Study II each contained the 20 item Spielberger-Corsuch-Lushene State-Trait Anxiety Inventory (Spielberger, Gorsuch & Lushene, 1970), as well as one item measuring the number Of additional high school or college chemistry courses the subjects might have previously taken. The degree Of internal consistency Obtained with the STAI, as computed with Cronbach's Alpha (Cronbach, 1951) on the 3A subjects used in the present study, was 0.89. NO measure Of internal consistency was computed for the three item measure of state anxiety develOped by the experimenter for the quiz used in Study I, as unexpected negative correlations were Obtained between the items. Examinations. The examinations for each Of the two studies consisted Of 20 items, each measuring the same gen- eral content areas as that measured on their respective content review quizzes, and again answerable with the same type Of recall formats (Appendices C and D). Addi- tionally, 10 Of the items on the examination used for Study I were precisely the same questions, in the same formats, as the students had to answer on the quiz for that study. All items, on all Of the quizzes and examinations, were scored with two points for a perfectly correct answer and one point if the answers were partially correct. Determination Of correctness was made on the basis of deriving a consensus of Opinion between the instructor, the experimenter and an as- sistant. The internal consistencies of the two examinations for Study I and Study II, as measured by the computation Of Cronbach's Alpha (Cronbach, 1951), were 0.86 and 0.93, re- spectively. Procedure The studies were conducted over a period which ex- tended for three weeks (see Table 3.1), beginning with the content review quiz for Study I and concluding with the 35 examination for Study II. All classes, as a result Of the instructors' normal, teacher-paced, combination lecture and lab-type presentations, had covered all Of the subject mat- ter relevant to the two studies immediately prior to the time each Of the studies were initiated. The experimental manipulations occurred during announced practice quizzes, administered one week prior tO the dates the classes were scheduled to take "for credit" examinations on the content areas of concern. Table 3.1.--Schematic Representation Of the Experimental Design and Timeline Day1............Day8............Day15............Day22 R1----FB -------- R2 --------------- R3----FB --------- RA (t1) (t2) Where: R1 is the initial quiz on factual material; FB is the informative feedback presented tO subjects; t1 and t2 are the delays with which feedback was pre- sented (2 seconds or 20 minutes); R2 is the administration Of the achievement test for factual material (dependent variable); R3 is the initial quiz on rule applications; and RA is the administration Of the achievement test for rule applications (dependent variable) After the students were seated at their desks on the days Of the quizzes, the instructor proceeded to pass out the quiz forms. The forms themselves, as described within 36 the section on instrumentation, contained the various ex- perimental manipulations. Random assignment to the two con- ditions for each study was accomplished by: (1)randomly or- dering the quiz forms according to a set Of random assign- ments generated by a PR1ME Computer Company Fortran Sub- routine run on a PRlME 750 computing system and (2)distrib- uting those quizzes tO the students, alternating between adjacent rows in an attempt to further help eliminate any irrelevant bias that might have been present due to the students' self-selected seating arrangements. The instructor provided each student in an immediate feedback condition with a latent image developer (A.B. Dick model 97-3100), after which he read the following to them: The quiz you are about to take is an attempt to accomplish three purposes: (1)tO serve as a re- view Of some Of the information we have covered in class; (2)to give you a better idea Of the types of questions you might find on next week's examination; and (3)to provide me with some in- formation regarding the effects Of providing or not providing feedback tO students regarding how they answer questions on a quiz. Answer each question by writing the answers on the lines provided but, if you received one Of the markers, insure that you draw a line with it between the two parentheses next to your answer immediately after writing your answer. That will reveal the correct answer. DO that for each question, but make sure that you write your answer first before finding out what the correct answer is. If you did not receive one Of the markers insure that you see me before you leave, so that you will have an opportunity tO see what the correct answers to the quiz were. Be sure that you read all instructions carefully and, upon completing the quiz, bring yours to me, personally, along with the marker if you had one. 37 The students were given as much time as they needed to complete the quiz and, upon turning their papers in to the instructor, were given additional examinations that, while unrelated to the present study, had been announced for that day. All students in the delayed feedback conditions, upon finishing all of the examinations for that period, were given the printed copy Of the questions and answers to the quiz, and were instructed to take a few minutes to review that information and to return the sheet tO the instructor before leaving the class. Due tO the findings Of previous studies that stu- dents would peek at answers if given the Opportunity (e.g., Anderson, Kulhavy & Andre, 1971, 1972), as well as the now well known problem in studies Of the present type that re- searchers simply cannot rely upon teachers tO consistently perform precise manipulations independently (e.g., Fullan & Pomfret, 1977), the experimenter assisted the instructor in monitoring the administration Of each quiz. Particular at- tention was paid to insuring that: (1)peeking did not occur; (2)feedback was received immediately where intended; and (3) delayed feedback was not only received where intended, but only for a controlled period. One week following the administration of each Of the content review quizzes, all students were administered the respective examinations for the two levels of learning, their performances on those measures serving as the depen- dent variables for the investigation. I. 1.". 2' it: . 38 Design The study can be viewed as two independent experi- ments, one for each category of learning outcome studied, and both having three replications on the factor Of primary concern (i.e., delay Of feedback; see Table 3.2). The de- pendent variables were the scores students attained on the examinations and measures Of anxiety administered, and the scores they attained on the respective pretests were used as a covariate. An additional covariate had been planned, namely the number Of college and/or high school chemistry courses subjects reported they had previously taken, but lack Of specificity, and therefore substantive variance on the measure, precluded its potential effectiveness. Table 3.2.--Experimental Design Study I: Knowledge Stating Classes Delay of Feedback 9:00 a.m. 10:00 a.m. 1:00 p.m. 2 Seconds S1 82 S3 20 Minutes SA 55 S6 Study II: Rule Applications Classes Delay Of Feedback 9:00 a.m. 10:00 a.m. 1:00 p.m. 2 Seconds S1 S2 S3 20 Minutes SA 85 S6 Hypotheses 39 The hypotheses that were of primary interest in the present study were as follows: H1: There will be a significant main effect, on H2: a measure Of retention, between the two de- lays Of feedback for knowledge stating; that is, subjects in one of the two delay conditions will exhibit superior perform- ance on the measure Of retention for stat- ing facts, given one week after the exper- imental manipulation. Reinforcement theory would predict that higher scores would be attained by subjects in the immediate feed- back conditions, but information process- ing theory would predict the Opposite re- sult. There will be a significant main effect, on a measure Of retention, between the two de- lays Of feedback for rule applications; that is, subjects in one Of the two delay conditions will exhibit superior perform- ance on the measure Of retention for apply- ing rules, given one week after the experi- mental manipulation. Reinforcement theory would predict that higher scores would be attained by subjects in the immediate feed- back conditions, but information process- ing theory would predict the Opposite re- sult. Additionally, four related questions were studied, although redundancy of the analysis permitted tests Of these for des- criptive purposes only: Q1 Q2: Will there be a significant main effect, on a measure Of retention Of familiar items (i.e., questions the subjects had encoun- tered previously), between the two delays of feedback for knowledge stating; that is will subjects in one Of the two delay condi- tions exhibit superior performance on the measure Of retention Of familiar items, given one week after the experimental manipulation? Will there be a significant main effect, on a measure Of retention Of items the subjects A0 had not previously encountered, between the two delays Of feedback for knowledge stating; that is, will subjects in one of the two de- lay conditions exhibit superior performance on the measure Of retention of unfamiliar items, given one week after the experimental manipulation? Q3: Will there be significant main effects, on the measures of state anxiety, between the two delays Of feedback; that is, will sub- jects in one Of the two delay conditions exhibit greater levels Of state anxiety when quizzed on knowledge stating and/or the application Of rules? QA: Will there be a significant interaction be- tween the degrees Of state anxiety which the subjects eXperience and the delays with which feedback is presented; that is, will subjects in at least one of the two delay conditions exhibit superior performance on the measure Of retention, dependent upon the levels of anxiety they had experienced on the quiz? Analysis Of the Data The primary hypotheses, namely that there would be significant main effects across the grOUps and studies on the delays of feedback given, were tested via two two-way completely randomized factorial analyses Of covariance (Kirk, 1968). Both independent variables, that is delay Of feedback and classes (replications), were treated as fixed factors. A family alpha Of .05 was selected for all tests. Summary All of the students from three introductory chem- istry classes at a large mid-western community college served as subjects in two experiments designed to test the effect Of varying delays Of feedback on the retention of two A1 categories of learning outcomes, namely the stating Of facts and the application Of rules. All subjects, prior tO the start of the experiments, had already enrolled in the classes and had covered all areas Of course content upon which the study's measures were based. The experimental manipulations of concern, namely whether the effects Of informative feedback would be en- hanced more by presenting such feedback immediately or after a period Of twenty minutes, were studied, treating each cat- egory of learning outcome within a separate investigation. Subjects were randomly assigned, within each Of the two studies, to one Of two conditions, namely a group that was given immediate feedback (i.e., within two seconds) and a group that was given feedback delayed by a period Of ap- proximately 20 minutes. The dependent measures were the scores subjects at- tained on examinations covering the same content areas as they were given feedback for earlier, but administered one week after the experimental manipulations had been intro- duced, as well as two measures Of state anxiety. The primary hypotheses of the study were concerned with the differential levels Of academic performance, on twO categories Of learning outcome (Gagne' & White, 1978), ex- pected as a result Of delaying or not delaying the presen- tation Of informative feedback. Additionally, four questions concerned with an as- pect Of the specificity Of such manipulations were A2 investigated, namely whether subjects, across conditions, would perform better on new items or items that they had previously encountered, and whether state anxiety would be useful for explaining the variance resulting from the ma- nipulations. CHAPTER IV ANALYSIS OF RESULTS Study I - Knowledge Stating Retention (H1). One purpose Of the present study was to test the differential effects Of delaying or not de- laying the presentation Of informative feedback on students' retention Of chemical elements and their respective symbols (i.e., factual information). Analyses were conducted on the total raw scores subjects attained on the measure of retention for factual information, as well as two compo- nents of that measure, namely those items which were iden- tical to questions that were on the pretest and those which were new. The total raw scores Of the subjects in each Of the twO feedback conditions, across the three classes, for the retention test for knowledge stating are summarized in Table A.1. Each cell shows the sample size, and the means and standard deviations Of the subjects' total raw scores on the measure Of retention for factual information. A univariate unweighted means analysis Of covariance was performed on the data, treating each factor as fixed and completely crossed, and using the total raw scores subjects attained on the pretest as a covariate. As can be seen in Table A.2, the A3 AA only factor that showed tO be significant, aside from the covariate, was the experimentally manipulated delay Of feedback [F(1,58)=7.892, p<0.007]. Delaying the presenta- tion Of informative feedback by twenty minutes resulted in superior performance, as compared with presenting it im- mediately. Table A.1.--Means and Standard Deviations Of Total Raw Score on the Measure of Retention Of Factual Informa- tion A 222:2:==:==::::::::=:22:22:22:22::2222::2222222222222223222= L Class Delay Of -------------------------------------------- Feedback 9:00 a.m. 11:00 a.m. 1:00 p.m. Mean 3A.83 37.71 32.1A 2 Seconds SD 8.59 2.97 8.86 N 12 1A 7 Mean 36.62 38.25 38.86 20 Minutes SD 2.98 2.05 1.86 N 13 12 7 Table A.2.--Summary of Univariate Unweighted Means Analysis Of Covariance on Students' Total Raw Scores for the Retention Of Factual Information Source Of Variation SS df MS F p 'EFEEEEE'ZESJQFEQEE? """"" 66333 'i 8633; 33:52 2:66? Delay Of Feedback 135.32 1 135.32 7.89 (.007 Class (Replications) AA.A0 2 22.20 1.30 n.s. Delay Of Feedback x Class 59.9A 2 29.97 1.75 n.s. Residual 99A.A7 58 17.15 A5 Similar results were found for both familiar and unfamiliar items on the retention measure. The raw scores of the students in each of the two feedback conditions, across the three classes, on items that were also asked on the pretest are summarized in Table A.3. A two-way uni- variate unweighted means analysis Of covariance was per- formed on the data, again treating each factor as fixed and completely crossed, and using the total raw scores students attained on the pretest as a covariate. Aswcan be seen in Table A.A, the only factor that showed to be significant, aside from the covariate, was the experimentally manipulated delay Of feedback [F(1,58):6.175, p<0.016]. The raw scores of the students in each Of the two feedback conditions, across the three classes, on items that were not asked on the pretest, but measured the same general content area, are summarized in Table A.5. A two-way univariate unweighted means analysis Of covariance was performed on the data, again treating each factor as fixed and completely crossed, and using the total raw scores students attained on the pretest as a covariate. As can be seen in Table A.6, the only factor that showed to be significant, aside from the covariate, again was the experimentally manipulated delay of feedback [F(1,58)=5.780, p<0.019]. Delaying the presenta- tion Of informative feedback by twenty minutes resulted in superior performance, as compared with presenting it im- mediately, for both familiar and unfamiliar items. A6 Table A.3.--Means and Standard Deviations Of Total Raw Score on the Measure Of Retention Of Familiar Factual Information Class Delay Of -------------------------------------------- Feedback 9:00 a.m. 11:00 a.m. 1:00 p.m. Mean 17.50 19.00 15.A3 2 Seconds SD 3.75 1.62 5.32 N 12 1A 7 Mean 18.15 19.08 19.29 20 Minutes SD 2.A8 1.31 1.25 N 13 12 7 Table A.A.--Summary Of Univariate Unweighted Means Analysis Of Covariance on Students' Total Raw Scores for Familiar Test Items Source of Variation SS df MS F p "EFZEEQE'EESCQEEQZS"m" 371733 "'7 77333 3378? 2:661 Delay Of Feedback 31.98 1 31.98 6.18 (.016 Class (Replications) 17.77 2 8.88 1.72 n.s. Delay Of Feedback x Class 2A.02 2 12.01 2.32 n.s. Residual 300.A0 58 5.18 :47 Table A.5.--Means and Standard Deviations Of Total Raw Score on the Measure of Retention of Unfamiliar Factu- al Information Class Delay Of -------------------------------------------- Feedback 9:00 a.m. 11:00 a.m. 1:00 p.m. Mean 17.33 18.71 16.71 2 Seconds SD 5.16 2.02 3.95 N 12 1A 7 Mean 18.A6 19.17 19.57 20 Minutes SD 1.39 1.A0 0.79 N 13 12 7 Table A.6.—-Summary of Univariate Unweighted Means Analysis of Covariance on Students' Total Raw Scores for Unfamiliar Test Items Source of Variation SS df MS F p $33-22;-ZES;;;E;Z;§"""135:5? ’7 736:6? 27:63 2:66? Delay Of Feedback 35.73 1 35.73 5.78 (.019 Class (Replications) 7.79 2 3.89 0.63 n.s. Delay Of Feedback x Class 8.A6 2 A.23 0.68 n.s. Residual 358.52 58 6.18 A8 State Anxiety. A second purpose Of the study was tO investigate the utility Of state anxiety toward explaining the effects Of varying the delay Of presenting informa- tive feedback regarding knowledge stating. Of the three questions students were requested to reSpond to, differ- ences were Observed between the experimental conditions on only one. As seen in Table A.7, when asked whether they felt that the questions on the quiz were arranged in order Of difficulty, subjects in all three classes were more likely to agree if their informative feedback had been delayed by approximately 20 minutes. A two-way unweighted means analysis of variance performed on the data, treating each factor as fixed and completely crossed (see Table A.8), showed delay Of feedback tO be the only significant factor [F(1,62)=A.796,p<.032]. Due to the fact that the three items did not systematically correlate with each other as anticipated, no attempt was made to combine them into a scale. Study II - Rule Applications Retention (H2). A third purpose of the study was to test the differential effects of delaying or not delaying the presentation Of informative feedback on students' reten- tion of how to apply the rules for systematically naming inorganic compounds. Alternate forms of one Of the in- structor's usual exams were used for the experimental ma- nipulation and measure Of retention, respectively, thus A9 Table A.7.--Means and Standard Deviations Of the Subjects' Responses to the question, "I felt that the questions on the quiz were arranged in order of difficulty" Class Delay Of -------------------------------------------- Feedback 9:00 a.m. 11:00 a.m. 1:00 p.m. Mean 3.00 3.31 3.38 2 Seconds SD 0.85 0.87 0.52 N 12 16 8 Mean 2.85 2.62 3.1A 20 Minutes SD 0.99 0.78 0 69 N 13 13 7 Table A.8.--Summary Of Univariate Unweighted Means Analysis Of Variance on Students' Responses tO the Ques- tion, "I felt that the questions on the quiz were arranged in order Of difficulty" Source Of Variation SS df MS F p 3525—3FEEEESQZE'm’m "3:2? "1' "372'? 7:56 2:635 Class (Replications) 1.3A 2 0.67 1.00 n.s. Delay Of Feedback x Class 0.85 2 0.A3 0.6A n.s. Residual A1.50 62 0.67 50 none Of the items on the posttest were familiar to the students. The total raw scores Of the students in each Of the two feedback conditions, across the three classes, for the retention test are summarized in Table A.9. A two-way univariate unweighted means analysis of covariance was performed on the data, treating each factor as fixed and completely crossed, and using the total raw scores students attained on the pretest as a covariate. As can be seen in Table A.10, the only factor that showed to be significant, aside from the covariate, was the experimentally manipulated delay Of feedback [F(1,50)=6.686, p<0.013]. Delaying the presentation Of informative feedback by twenty minutes resulted in superior performance, as compared with present- ing it immediately. Table A.9.--Means and Standard Deviations Of Total Raw Score on the Measure Of Retention Of Rule Applications Class Delay Of -------------------------------------------- Feedback 9:00 a.m. 11:00 a.m. 1:00 p.m. Mean 16.89 20.17 21.A0 2 Seconds SD 11.01 12.12 11.08 N 9 12 5 Mean 21.73 20.A6 30.A3 20 Minutes SD 9.92 11.6A 7.93 N 11 13 7 51 Table A.10.--Summary Of Univariate Unweighted Means Analysis Of Covariance on Students' Total Raw Scores for Retention Of Rule Applications Source Of Variation SS df MS F p ”REESE-78532512223 """ 52336 '7 53:26 Ti; 2676 Delay Of Feedback 673.00 1 673.00 6.69 (.013 Class (Replications) 3A9.A1 2 17A.71 1.7A n.s. Delay Of Feedback x Class 39.A1 2 19.70 0.20 n.s. Residual 5033.06 50 100.66 State Anxiety. A fourth purpose Of the study was to investigate the utility Of state anxiety toward explaining the effects Of varying the delay Of presenting informative feedback with respect to tests of rule applications. The State Trait Anxiety Inventory (STAI, Spielberger, Gorsuch & Lushene, 1970) was analyzed both as a dependent variable in a two-way univariate unweighted means analysis Of variance, and as an independent variable (using a median split) in a three-way univariate unweighted means analysis of covariance of the posttest scores. NO significant dif- ferences were found. 52 Summary Of Results 1. The hypotheses regarding differential perfor- mances on measures Of retention, according to the delays with which informative feedback was presented on a quiz given one week earlier, were all supported. Students who were given informative feedback approximately 20 minutes after they had responded tO the quiz, scored significantly higher than students who were given such feedback immedi- ately (i.e., within two seconds). That is, regardless of the level Of the information being tested, or whether the test items were new or familiar, delayed feedback enhanced retention significantly more than immediate feedback. 2. Only one Of the questions regarding differen- tial levels Of state anxiety, according to the delays with which informative feedback was presented, was supported. Students who were given informative feedback approximately 20 minutes after they had responded to the quiz used in Study I, were more likely than students in the immediate feedback condition to agree that the items on that quiz had been arranged in order Of difficulty. 3. NO evidence was found to support the existence of a functional interaction between reinforcement and infor- mation processing theories. CHAPTERS Summary and Conclusions Summary Problem. While most researchers would agree that some aspects of human learning are subject to control by reinforcement, some of the applications Of that principle which behaviorists had suggested have recently been seri- ously questioned. One such prescription has been Skinner's (195A) recommendation that feedback can serve tO reinforce students' recalling Of the answers to test questions and should, therefore, be given immediately. Information prO- cessing theorists, who have eschewed reinforcement as an eXplanation for verbal learning and problem solving, have argued that Skinner was incorrect and have Offered a good deal Of evidence to support their claim (e.g., Anderson, Kulhavy & Andre, 1971, 1972; English & Kinzer, 1966; Kulhavy, 1977; Kulhavy & Anderson, 1972; More, 1969; Phye & Baller, 1970; Sassenrath & Yonge, 1968; Sturges, 1969, 1972, 1978; Sturges & Crawford, 196A; Suber & Anderson, 1975). Their own conclusions, however, were based on studies which have been limited in a number Of respects and it was argued that the question Of whether feedback should be given im- mediately or delayed in the applied setting Of school 53 5A learning has yet tO be studied satisfactorily in the context of typical classroom situations. Purpose. The purpose of the present study was to compare the relative effects Of immediate and delayed feed- back On the retention Of two categories Of learning out- comes, the stating Of facts and the application Of rules, in the context Of an actual classroom setting. The study, specifically, was an attempt to correct for five limitations Of previous studies conducted on the same tOpic, namely: (1)the possible confounding of reinforcement and delayed feedback; (2)the use of dependent measures atypical to most classroom learning situations; (3)the use of a potentially biased measure Of retention; (A)the use of theoretically inconsistent definitions Of immediate and delayed feedback; and (5)the failure to consider and/or control for the par- ticular outcomes of learning studied. The intent was to provide valuable information for the controversies on learning theory and, equally important, to Offer a controlled comparison Of the practical applica- tions Of the Opposed prescriptions. Given the increasing questioning that has occurred in recent years of the ap- plicability Of research tO educational practice in general (e.g., Hook, 1981; Turnbull, 1979), a less equivocal under- standing Of the tOpic is necessary. Methodology. The entire populations from three sections Of a freshman-level college chemistry course served 55 as subjects in two experiments, one for each Of the outcomes of learning investigated. In each study the effects on re- tention of providing informative test feedback immediately (i.e., within two seconds) and delayed (i.e., after approxi- mately 20 minutes) were compared. Actual planned course content and examinations were used as the stimuli and de- pendent variables, respectively, and the manipulations were accomplished within the normal context Of the course. The questions to be answered were: Was there a dif- ferential performance on the two levels Of questions across the groups? If so, did the evidence solely support a behav- ioral interpretation, an information processing interpreta- tion, or was there evidence Of a functional interaction ex- isting between the two theories? Were there corresponding differences existing between the groups on a measure Of state anxiety? Results. Both of the hypotheses suggested by infor- mation processing theory were supported. Subjects given in- formative feedback approximately 20 minutes after responding to the quiz, scored significantly higher than those given such feedback immediately (i.e., within two seconds). That is, regardless Of the outcome being examined, or whether the test items were new or familiar, delayed feedback enhanced retention significantly more than immediate feedback. Only one Of the questions regarding differential levels of state anxiety, according to the delays with which 56 informative feedback was presented, led tO finding dif- ferences between the groups. Students who were given infor- mative feedback approximately 20 minutes after they had responded to the quiz used in Study I were more likely than students in the immediate feedback condition tO agree that the items on that quiz had been arranged in order Of diffi- culty. NO evidence was found to support the existence Of a functional interaction between reinforcement and information processing theories. Conclusions The hypotheses studied in this project concerned the effects on retention Of varying the delay with which infor- mative feedback is presented, for two different outcomes Of learning, namely knowledge stating and rule application, and the results clearly favored delaying the presentation Of informative feedback (as suggested by information processing theory), rather than presenting it immediately (as pre- scribed by the prOponents of reinforcement theory). Regarding the additional questions that were inves- tigated, no evidence was found to indicate either the ex- istence Of a functional interaction between reinforcement and information processing theories, or any explanatory utility of anxiety with respect to the manipulations per- formed. l" )7 Limitations. Before a formal interpretation and discussion Of the results is undertaken, a brief overview of the general utility Of the study is in order. The most Obvious problem with the present study is its lack Of sta- tistical generalizability. The specific subject matter, and the particular students that were selected for the study, were both fixed factors and the results are simply not gen- eralizable beyond the particular classes and subject matter that were used. In this regard it is hoped that future re- searchers, who might wish tO use the Bridge Argument Of Cornfield and Tukey (1956) for generalizing the results to an hypothesized population with similar characteristics, will find the description Of the sample provided to suffice. A second limitation Of the study is that the one purportedly anxiety related question about which any dif- ferences were found, was also the one question that did not correlate systematically with the other anxiety related items. That is, while asking subjects whether they thought the items on the quiz were arranged in order Of difficulty was intended as a measure Of state anxiety, it did not correlate systematically with any Of the other anxiety related items and attempting to draw conclusions from such a serendipitous finding might be questionable. Finally, while analyses of variance are typically robust against violations Of homogeneity Of variance, it should be noted that there were discrepancies in Study I due to the apparent ceiling effect. 58 Discussion. The results echoed the major previous finding Of studies on the Delayed Retention Effect, "long- term retention Of academic material following some delay Of informative feedback is superior to that with immediate feedback (Sturges, 1978, p.385)." These results are es— pecially intriguing since, unlike previous studies on the same tOpic, the present study was conducted in an actual classroom setting, used no apparent inherently reinforcing stimuli for providing delayed feedback, and did not use a potentially biased measure of retention. The findings were consistent with the perseveration-interference hypothesis proposed by Kulhavy and Anderson (1972) and clearly in con- trast to the behavioral prescription. The interpretation that any feedback serves merely to initiate an error-nulling process without regard to the mental processes involved (e.g., Guthrie, 1970), however, was not supported. With the exception Of one experiment, past studies of the Delayed Retention Effect have been the same in at least the following ways: (1)the pretest and any respective informative feedback was always presented with multiple-choice formats and (2)the retention measure always consisted Of the same items as the pretest. In the one study that differed from that procedure (Sturges, 1978), retention was measured for both: (1)the same items that were on the pretest; (2)items from the pretest that had been restated in a recall-type format; and (3)items that measured the same general content as the items on the pretest but 59 were, in themselves, novel. Sturges found no significant differences attributable to feedback in either Of the latter two measures. The present study, conversely, utilized recall formats for all items on the pretests and posttests and consistently found differences, attributable to the delay Of feedback, regardless Of the degree Of the subjects' famil- iarity with the items. For rule applications, in fact, alternate forms Of the exam were used for the pretest and posttest, yet the effect was still found. While recall and recognition were not formally compared in the present study, the indication that they should be is Obvious. A number Of theoreticians have claimed that the two are qualitatively and functionally different behaviors (see, Gagne' & White, 1978; Mueller, 1980), but reliable Observed differences between them have apparently been nonexistent or difficult tO locate (e.g., Mueller, 1980). The results Of the present study, compared with those found by Sturges (1978), suggest that recall may well constitute a qualitatively different component in the process. Such a result can be viewed, however, from at least three additional perspectives. First, there exists the pos- sibility that delayed feedback leads to increased motiva- tion. While earlier studies discovered no significant dif- ferences between the delay conditions on items that had not been presented previously, and therefore discounted the pos- sibility Of a motivational interpretation, such an 60 eXplanation Obviously cannot be eliminated given the results Of the present study. These results were especially con- vincing considering the fact that the delayed feedback groups clearly had reached the test's ceiling in the first study. Conceivably, the added difficulty Of having to recall answers rather than merely recognize them, leads to greater efforts in later study habits. Second, is the possibility that immediate feedback is perceived as being unnatural and serves to suppress learning. While the present study did not utilize a control group, and therefore Offered no test Of that hypothesis, the previous studies reviewed which did include control groups Offered no support for such a view. That is, while students may or may not perceive immediate feedback tO be natural, little if any evidence is available to indicate that such a condition serves to depress learning. Finally, there remains the serendipitous finding that students in the delay condition were more likely to perceive that the questions on the test were arranged in order Of difficulty. Since nO differences were found on any Of the accepted anxiety measures, an interpretation in- volving that construct does not seem warranted. An alter- native explanation might be that, without the availability Of feedback, students engage in a more thorough semantic analysis Of the questions (see, e.g., Sturges, 1978). Such an explanation would be consistent with the findings Of other studies that those students' confidence increases at 61 the time of retention, since they would then be more fa- miliar with the information. Recommendations for Future Research. Those who wish tO continue research on the topics investigated in the pres- ent study may find that certain instrumentation and methodo- logical changes would enhance both the degrees Of accuracy and generalizability Of the findings Of the present experi- ments. Some Of these changes were implied in the preceding discussion. Some instrumentation changes, for example, could be made with little difficulty. The most Obvious is the selec- tion Of a better covariate. Asking students how many chem- istry courses they had taken prior to the one they were cur- rently enrolled in failed to sufficiently discriminate be- tween their current levels Of comprehension regarding the task at hand. The measure which was used, namely students' scores on the studies' pretests, could have been biased if receiving feedback immediately after recalling and writing the answer to a question affected later responding on that measure. While analyses between those scores did not indi- cate such differences, a more reasonable alternative would be to administer a separate pretest prior to initiating the experimental manipulations. ‘ Another change would be tO use a measure Of state- anxiety, Such as the STAI, for both aspects Of the exper- iment. It was only used within Study II of the present ex- 62 periment, as its existence was unknown tO the experimenter prior to that time. Considering the items that were devel- Oped by the experimenter for Study I, however, a more re- warding endeavor might be tO expand on the notion of the subjects' perceived difficulty Of the quizzes. A longer, more reliable index Of that phenomenon, should definitely be used if it is to be formally investigated. Given the discrepancy between the findings of the present experiment and those Of Sturges (1978), the system- atic comparison Of using recall and multiple-choice type items on the pretest most definitely seems to be in order at this time. This could be accomplished, as Sturges did, by using the same stems on both the pre- and posttests and varying only the mode of response required. BIBLIOGRAPHY BIBLIOGRAPHY Anderson, R.C., Kulhavy, R.W. & Andre, T. Feedback Procedures in Programmed Instruction. Journal Of Educational Research, 1971, 62, 1A8-156. Anderson, R.C., Kulhavy, R.W. & Andre, T. Conditions Under which Feedback Facilitates Learning from a Programmed Lesson. Journal Of Educational Psychology, 1972, 631 186-188. Andre, T. Does Answering Higher-Level Questions While Reading Facilitate Productive Learning? Review of Educational Research, 1979, Ag, 2, 280-318. Barker, D. & Hapkiewicz, W.J. The Effects Of Behavioral Objectives on Relevant and Incidental Learning at Two Levels Of Bloom's Taxonomy. Journal Of Educational Research, 1979, 12, 33A-339. Barringer, C. & Gholson, B. Effects of Type and Combination Of Feedback upon Conceptual Learning by Children: Implications for Research in Academic Learning. Review Of Educational Research, 1979, 32, A59-A78. Berliner, D.C. Impediments to the Study Of Teacher Effectiveness (Technical Report #75-11-3). San Francisco, California: Far West Laboratory for Edu- cational Research and DevelOpment, 1975 Bloom, B.S., Englehart, M.D., Furst, E.J., Hill, W.H. & Krathwohl, D.R. Taxonomy of Educational Objectives. Handbook I: Cognitive Domain. New York: David McKay, 1956. Brackbill, Y., Adams, G. & Reaney, T.P. A parametric study Of the delay-retention effect. Psychological Reports, 1967, 29, A33-A3A. Campbell, D.T. & Stanley, J.C. Experimental and quasi- experimental designs for research on teaching. In N.L. Gage (Ed.), Handbook Of research on teachin . Chicago: Rand McNally, 1963. Pp. 171- 2A6. 63 6A Carlson, J.C. & Minke, K.A. Fixed and Ascending Criteria for Unit Mastery Learning. Journal Of Educational Psychology, 1975, 61, 96-101. Cornfield, J. & Tukey, J. Average values Of mean squares in factorials. Annals of Mathematical Statistics, 1956. 21. 907-9149. Crawford, J.C. The Effects Of Temporal Intervals of Reinforcement upon Deductive and Inductive Generalizations. Ellensburg: Central Washington State College, 1966. Cronbach, L.J. Coefficient alpha and the internal struc- ture Of tests. Psychometrika, 1951, 16, 297-33A. Deese, J. & Hulse, S.H. The Psychology Of Learning (3rd edition). New York: McGraw-Hill Book Company, 1967. Ebel, R.L. Measuring educational achievement. EnglewOOd Cliffs, New Jersey: Prentice Hall, 1965. English, R.A. & Kinzer, J.R. The effect Of immediate and delayed feedback on the retention Of subject matter. Psychology in the Schools, 1966, 3, 1A3-1A7. Estes, W.K. Reward in human learning: Theoretical issues and strategic choice points. In R. Glaser (Ed), The Nature of Reinforcement. New York: Academic Press, 1971. Eysenck, M.W. Effects of noise, activation level, and re- sponse dominance on retrieval from semantic memory. Journal Of Experimental Psychology: Human Learning and Memory, 1975, l, 1A3-1A8. Feyerabend, P.K. Science in a Free Society. Thetford, Norfolk (Great Britain): Lowe and Brydone, Ltd., 1978. Fullan, M. & Pomfret, A. Research on Curriculum and In- struction Implementation. Review Of Educational Research, 1977, 31, 335-397. Gagne', R.M. The Conditions Of Learning (2nd edition). New York: Holt, Rinehart and Winston, 1970. Gagne', R.M. Domains Of learning. Interchange, 1972, 3, 1-80 (Sagne', R.M. The Conditions Of Learning (3rd edition). New York: Holt, Rinehart and Winston, 1977. 65 Gagne', R.M. & Briggs, L.J. Principles of Instructional Design. New York: Holt, Rinehart and Winston, 197 . Gagne', R.M. & Briggs, L.J. Principles Of Instructional De- sign (2nd Edition). New York: Holt, Rinehart and Winston, 1979- Gagne', R.M. & White, R.T. Memory Structures and Learn- ing Outcomes. Review Of Educational Research, 1978, 66, 187-222. Glaser, R. Components Of a Psychology Of Instruction: Toward a Science Of Design. Review Of Educational Research, 1976, 66, 1-2A. Glaser, R. Trends and Research Questions in Psychological Research on Learning and Schooling. Educational Researcher, 1979, 6, 10, 6-13. Glass, G.V. Educational Knowledge Use. The Educational Forum, 1971, 36, 1, 21-29. Glass, G.V. The Wisdon Of Scientific Inquiry on Education. Journal of Research in Science Teaching, 1972, 2, 1, 3-18. GreenSpoon, J. The reinforcing effect Of two spoken sounds on the frequency Of two responses. American Journal Of Psychology, 1955, 66, AO9-A16. Grice, G.R. The relation Of secondary reinforcement tO de- layed reward in visual discrimination learning. Journal of Experimental Psychology, 19A8, 36, 1-16. Guthrie, J.T. Feedback and Sentence Learning. Baltimore, Maryland: Johns Hopkins University Center for the Study Of Social Organization Of Schools, 1970. Heller, P.S. A descriptive analysis Of the clinical reason- ing Of college chemistry teaching assistants in a tutorial setting. East Lansing: Unpublished doctoral dissertation, Michigan State University, 1979. Hidlum, D.C. & Brown, R.W. Verbal Reinforcement and inter- viewer bias. Journal of Abnormal and Social Psy- chology, 1956, 63, 108-111. Hockman, C.H. & Lipsitt, L.P. Delay-Of-reward gradients in discrimination learning with children for two levels Of difficulty. Journal Of Comparative and Physio- logical Psychology, 1961, 66, 2A-27. 66 Hook, J. Draft Of Reagan's Budget Plan Shows Major Slashes in U.S. Aid to Colleges. The Chronicle Of Higher Education, 1981, February, 1-13. Hull, C.L. Principles of Behavior. New York: Appleton- Century-Crofts, 19A3. Hull, C.L. Essentials Of Behavior. New Haven, Connecticut: Yale University Press, 1951. Hull, C.L. A behavior system. New Haven, Connecticut: Yale University Press, 1952. IUPAC. U.S. Presentation of the IUPAC Inorganic Rules. Journal Of the American Chemical Society, 1960, 66, p.5525. Kelleher, R.T. Discrimination learning as a function Of re- versal and nonreversal shifts. Journal Of Experimen- tal Psychology, 1956, 61, 379-38A. Keller, F.S. Goodbye, Teacher... Journal Of Applied Be- havior Analysis, 1968, 1, 79-89. Kirk, R. Experimental Design: Procedures Of the Behav- ioral Sciences. Belmont, California: Brooks Cole, 1968. Kling, J.W. & Schrier, A.M. Positive Reinforcement. In J.W. Kling and L.A. Riggs (Eds), Woodworth and Schlosberg's Experimental Psychology (3rd edition). New York: Holt, Rinehart and Winston, 1971. Kulhavy, R.W. Feedback in Written Instruction. Review of Educational Research, 1977, 61, 211-232. Kulhavy, R.W. & Anderson, R.C. The Delay-Retention Effect with Multiple-Choice Tests. Journal of Educational Psychology, 1972, 63, 505-512. Levonian, E. Retention over time in relation to arousal during learning: An explanation Of discrepant re- sults. Acta Psychologica, 1972, 36, 290-321. Lindsay, P.H. & Norman, D.A. Human informationyprocessing: An introduction to_psychology. New York: Academic Press, 1972. Lorge, I. & Thorndike, E.L. The Influence of Delay in the After Effect Of a Connection. Journal of Experimental Psychology, 1935, 16, 186-19A. 67 Mandler, G. & Sarason, 3.8. A study of anxiety and learning. Journal Of Abnormal and Social Psychology, 1952, A7, 166-173. —" Mayer, R.E. Thinking and Problem Solving: An Introduction to Human Cognition and Learning. Glenview, Illinois: Scott, Foresman and Company, 1977. McKeachie, W.J. The Decline and Fall Of the Laws Of Learning. Educational Researcher, 197A, 3, 3, 7-11. Mehrens, W.A. & Lehmann, I.J. Measurement and Evaluation in Education and Psychology. New York: Holt, Rinehart and Winston, Inc., 1973. Merrill, M.D. Necessary Psychological conditions for de- fining instructional outcomes. Educational Tech- nology, 1971, ll, 8, 3A-39. Merrill, M.D., Kowallis, T. & Wilson, B.G. Instructional Design in Transition. In F.H. Farley & N.J.Gordon (Eds), Psychology and Education - The State Of the Union. Berkely, California: McCutchan Publishing Corporation, 1981. Merrill, M.D., Olsen, J.B. & Coldeway, N.S. Research Sup- port for the Instructional Strategy Diagnostic Pro- file (Technical Report, Series NO. 3). San Diego, California: Courseware, Inc., 1976. Miller, G.A., Galanter, E. & Pribram, K.H. Plans and the Structure Of Behavior. New York: Holt, Rinehart and Winston, 1960. More, A.J. Delay Of feedback and the acquisition and reten- tion Of verbal materials in the classroom. Journal Of Educational Psychology, 1969, 66, 339-3A2. Mueller, J.H. Test Anxiety and the Encoding and Retrieval Of Information. In I.G. Sarason (Ed), Test Anxiety: Theory, Research and Applications. Hillsdale, New Jersey: Lawrence Erlbaum Associates, 1980. Newell, A. & Simon, R.A. Human Problem Solving. Englewood Cliffs, New Jersey: Prentice-Hall, 1972. Noble, C.E. & Alcock, W.T. Human Delayed-Reward Learning with Different Lengths Of Tasks. Journal Of EXperimental Psychology, 1958, 66, A07-A12. Olson, D.R. & Bruner, J.S. Learning through experience and learning through media. In D.R. Olson (Ed), 68 Media and Symbols: The forms Of expression, com- munication, and education. Seventy-third_year- booknyational Society for the Study Of Education. Chicago: University Of Chicago Press, 197A. Phillips, D.S. What DO the Researcher and the Practitioner have to Offer Each Other? Educational Researcher, 1980, 2, 11, 17-2A. Phye, G. & Baller, W. Verbal retention as a function of the informativeness and delay Of information feedback. A replication. Journal Of Educational Psychology, 1970, 61, 380-381. Piaget, J. The origins Of intelligence in children. New York: International Universities Press, 1952. Pound, L.D. & Bailey. G.D. Immediate Feedback Less Effec- tive than Delayed Feedback for Contextual Learning? Reading Improvement, 1975, 16, 222-22A. Raymond, C.K. Performance in Instrumental Learning as a Joint Function Of Delay of Reinforcement and Time Of Deprivation. Journal Of Experimental Psychology, 195A, 61, 2A8-250. Renner, K.E. Delay Of Reinforcement and Resistance to Extinction: A Supplementary Report. Psychological Reports, 1965, 16, 1, 197-198. Ryan, F.J. & Bilodeau, E.A. Countertraining Of a Simple Skill with Immediate and One Week Delays Of Information Feedback. Journal Of Experimental Psychology, 1962, 63, 19-22. Saltzman, I.J., Kanfer, F.H. & Greenspoon, J. Delay of Reward and Human Motor Learning. Psychological Reports, 1955, l, 139-1A2. Sarason, I.G. Test Anxiety: Theory, Research and Applica- tions. Hillsdale, New Jersey: Lawrence Erlbaum Associates, 1980. Sassenrath, J.M. & Yonge, G.D. Delayed information feed- back, feedback cues, retention set, and delayed retention. Journal of Educational Psychology, 1968, 22. 69-73. Schwartz, B. Psychology Of Learning and Behavior. New York: W.W. Norton and Company, 1978. Seddon, G.M. The Properties Of Bloom's Taxonomy Of Educa- tional Objectives for the Cognitive Domain. 69 Review Of Educational Research, 1978, 66, 303- 323. Shulman, L.S. The Psychology of School Subjects: A Premature Obituary? Journal Of Research in Science Teaching, 197A, 11» 319-339. Sieber, J.E. Defining Test Anxiety: Problems and Approaches. In I.G. Sarason (Ed), Test Anxiety: Theory, Research and Applications. Hillsdale, New Jersey: Lawrence Erlbaum Associates, 1980. Skinner, B.F. The behavior Of organisms. New York: Appleton-Century-Crofts, 1938. Skinner, B.F. "Superstition" in the pigeon. Journal of Experimental Psychology, 19A8, 36, 168-172. Skinner, B.F. The Science Of Learning and the Art Of Teaching. Harvard Educational Review, 195A, 36, 86-970 Skinner, B.F. Teaching Machines. Science, 1958, 128, 969- 977- Skinner, B.F. The Technology of Teaching. New York: Appleton-Century-Crofts, 1968. Smith, E.L. Analytical concepts and the relation between content and process in science curricula (Techni- cal Note 2-72-56). Los Alamitos, California: SWRL Educational Research and Development Center, 1972. Spence, J.T. & Spence, K.W. The motivational components Of manifest anxiety. In C.D. Spielberger (Ed), Anxiet and behavior. New York: Academic Press, 1966). Spence, K.W. Behavior Theoqy_and Conditioning. New Haven, Connecticut: Yale University Press, 1956. Spence, K.W. Behavior Theory and Learning: Selected Papers. Englewodd Cliffs, New Jersey: Prentice- Hall, 1960. Spielberger, C.D. Theory and research on anxiety. In C.D. Spielberger (Ed), Anxiety and behavior. New York: Academic Press, 1966. Spielberger, C.D. Anxiety, Current Trends in Theory and Research. New York: Academic Press, 1972. 70 Spielberger, C.D., Gorsuch, R.L. & Lushene, R.E. Manual for the state-trait anxiety inventory. PaIO Alto, California: Consulting Psychologists Press, 1970. Sturges, P.T. Verbal retention as a function Of the inform- ativeness and delay Of informative feedback. Jour- nal Of Educational Psychology, 1969, 66, 11-1A. Sturges, P.T. Informative delay and retention: Effect Of information in feedback and tests. Journal of Edu- cational Psychology, 1972, 63, 32-A3. Sturges, P.T. Delay Of Informative Feedback in Comput- er-Assisted Testing. Journal Of Educational Psychol- 2519 1978: 19. 378-387. Sturges, P.T. & Crawford, J.J. Retention Using Delayed Informative Feedback as a Function Of Meaningfulness Of Material and Form Of Feedback. Olympia, Washington: State Department Of Public Instruction, 196A. Suber, J.R. & Anderson, R.C. Delay-retention effect in natural classroom settings. Journal of Educational Psychology, 1975, 61, 170-173. Thorndike, E.L. Animal intelligence: An experimental study Of the associative processes in animals. Ps cho- logical Review (Monograph Supplement), 189 , _, 8. Thorndike, E.L. Animal intelligence. New York: Macmillan, 1911. Turnbull, W.W. Setting Agendas for Social Research. Edu- cational Researcher, 1979, 6, 8, 9-2A. Uehling, B.S. Arousal in verbal learning. In C.P. Duncan, L. Sechrest and A.W. Melton (Eds), Human Memory: Festschrift in honor Of Benton J. Underwood. New York: Appleton-Century-Crofts, 1972. Walbesser, H.H. An evaluation model and its application. Washington, D.C.: American Association for the Ad- vancement Of Science, 1965. Walker, E.L. Action decrement and its relation to learn- ing. Psychological Review, 1958, 66, 129-1A2. Watson, J.B. Psychology as the Behaviorist Views It. Psychological Review, 1913, 20, 158-177. Wesman, A.G. Writing the Test Item. In R.L. Thorndike (Ed), Educational Measurement. Washington, D.C.: American Council on Education, 1971. 71 Wittrock, M.C. Learning and Memory. In F.H. Farley & N.J. Gordon (Eds), Psychology and Education, The State Of the Union. Berkeley, California: McCutchan Publishing Corporation, 1981. APPENDICES APPENDIX A. PRETEST ON KNOWLEDGE STATING APPENDIX A PRETEST ON KNOWLEDGE STATING ,' 'b 11 \A . \l. -hemiscry 1:3 Jane. no. " his. nUSEEd -ate: h Of the following symbols, with pen or pencil on the line provided, w. e the name Of the element the symbol represents. Im- mediately after writing an answer, but only then, use the Image Developer you have been provided with to draw a line between the two parentheses: As ( ) Fe ( ) As ( ) Hg ( ) To the right of each Of the following elements, with pen or pencil on the line provided, write the symbol that is used to represent the element. Immediately after writing an answer, but only then, use the Image Devel- oper to draw a line between the two parentheses: Boron _____ ( ) Platinum _____ ( ) COpper _____ ( ) Potassium _____ ( ) Fluorine _____ ( ) Sulfur _____. ( ) Gold ______ ( ) Tin _____ ( ) Manganese ( ) Tungsten ______ ( ) The following section is not part Of the quiz but, rather, is an at- tempt to find out how anxious the format used for this quiz may have made you. Please answer each item, as honestly as you can, by drawing a circle around the response to the right Of the statement that most accurately reflects your feelings. Use a pen or pencil to circle_"SA" if you STRONGLY AGREE with the statment, "A" if you AGREE with it but not strongly, "B" if you DZSAGREE, or "$3" if you STRJJOLY 31333323. There are no ’ri‘nt' or 'wrcnz' answers: 5 b a. I felt quite nervous while taking this quiz. SA A D 50 b. I felt more relaxed later in the quiz than I did earlier. SA A D SD c. I felt that the questions on the quiz were arranged in order of difficulty. SA A D SD 72 -—- .o 3-—-—-—._ -... .. APPENDIX B. PRETEST ON RULE APPLICATIONS APPENDIX B PRETEST ON RULE APPLICATIONS mafia o 3cwc ou womoHo>oa oocEH any on: .mocoomsoo ocfionHOM ecu 05cc .Hfiocom no A A O>c: so» womo~o>oa mocEH ozu on: .cozu haze use .wo3mco co ocMuuwz wOuuo >~0umacoEEH V V homoa axon co oscaucouv mm: «Anom.-< nommso «Ommmx «Azo.:m hONuOmz "momozucowmm 03» on» coo3uon mumeowzuwv ancoeso wo>aan ovawuwc cacomcoE OHHOOA ansono oaeeowawom nsowooon Gun Hon MQOECHSU ocwood A>U owcomwd ouozmnonm oufizmmozm comowph: moonfisdm Eocaesao oumcoawno oacosqucu manuaamwa canton .cocu waco usn .wm3mcm cm mcwuHHI wOuum kaouowooEEH com cud: .nomo mo unoww on» Ca vooa>oum mafia on» :0 .0H .m «mononucowmm 03» may coozuoa ocaa w keno o» and: cocw>owm coon .mccoomeoo mcd3OHH0u on» new mcfiOEwOM mudw3 .Hfiocom we con :ua3 .nowo no unmww one ou coow>owm mafia on» :o .< "Open ”OEOZ N Nfiso OOHuomwm mfia auumwsozo 73 APPENDIX B (Continued) mowucooo uocv 30: Hawu mo :oxmu 90> o>nn omwsoo awumwwmmw ¢ n v m v m VVVV'V‘VVV'VVV MMMMMMMMMMM VV'VVVV MMMMMM on 2032 xwo> on» mo zoom Cu poocoonou o>ns so» unzu anomca >Hunoco: no On >~ouowmcoz .Ewuw comm wozmco ommoum NNN NNNNNNNNNNN NNNNNN amazosom H H H Had—4Hr4—4Ha4-1Hr4 HHHHHH «.maa xuumweOno Hoo o Avon H.mH Iiiltlliilliiloowww03 Ea H.5H 1111111 11111 acoucoo doom H.wH 11111111 111111 coxcaow Em H.mH 111111 :mcowum cows: doom H.vH u 1111111 1 11111 >wouuwn Ea H.mH Illnlulliulnmoo>woc doom H.~H 11111 ucuofiuc001u~om Houu H.HH 11111111 ofinmququo down H.oa 111111111111 macaxco Homu H .m I 11111 I IIIIII powwow doom H .m ImocsquUme manwmmom wo>o mowhwwo3 aducumowm Ed 1111:: 11111111 woman down it: 111111111 Once an doom 1 11111111111 Hauuowmow En initiallillluuunomcou Ed I: 11111111111 owsoom Hoou 111111111111111 Educ Hoou AHO um uoz HHHHHHH O V «mucosououn cm .30: Osman mmcwaoou uso> muooHuow aaouowsooc umOE uczu ucoeoumum 624 we unvaw on» Ca uncommow on» pcoowo maowwu o mcwzmuv an .:60 50> no .ooa opus o>m£ >cE Naov was» wOu com: guano“ on» nsowxcc 30: use ccwu ou unsound on ma .wonunw .uon nwsv onu uo undo uo: ma :ofiuuom mcdonAOu oza .U . v N N omom «Hoe .OH ~.~oz.:z .a Amoumvao .m voezum .h N «“50 oowuonwm APPENDIX C. POSTTEST ON KNOWLEDGE STATING APPENDIX C POSTTEST ON KNOWLEDGE STATING 3 b 1 1 U) (c '1 (n m 1 (I: J y i A . . A a ‘9 n O f! ,1 U I ...-- I :z..$..‘3\. pate. q 1. To tne rign ynools, with pen or pencil on the line prov e en t the symbol represents: C‘ ‘9' 3.; 5.. 2. To the r-ght of each of the fol-ox-:g elecerts, with :e' or “en:-l 3* :1e line prov1:e:, write the syrbc- that .5 used to represent :18 element. Antimony - Manganese Arsenic Platinum Boron Silver Fluorine Sodium Iron Tungsten 74 o. -.-—v -. APPENDIX D. POSTTEST ON RULE APPLICATIONS APPENDIX D POSTTEST ON RULE APPLICATIONS Fhe~iscrv-118 Nomenclatur Sane v ooooo Write formulas for the following compounds: 1. manganic phosphite 2. chromous bicarbonate 3. mercurous periodate o. arsenous sulfite 6. calcium dihydrogenphosphate 7. zinc iodate 8. lithium sodium chromate 9. manganous phosphide 10. cobaltous borate Write names for the following compounds: 1. A12(Cr207)3 2. Ba(MnO4)2 3. CuBrO 2 4. CaSO4 5. H3803 6. Mn(SO3)2 7. LiSrPO3 8. P205 9. 828(aq) 10. SnlCN)4 75 C. HICHI IES mmmu