.. . a : 1.532%; ‘ 11...... : ymxmd? .....r 1.. ‘ . ; 3 3.1L I . .1.“ . .1. r .a rib V. y a . 7f? {.1 .. t T Z; 7 . a ~ . v .r .393. 115.77.;5 1.3.. : .5 1.3.3.... 3, 7.71:»; F! .. 11.). a. :17: 1.: 2 «2... Le... : . v 1. . nut? .— w; .n .3 Z ...rx$.ur ._ 1;. .w 5 uu~ » . C... 1 n . 30.1; q :12? .. : 3.2.. as “$3.: i .u ; . an; a , Sun}... .3 I “at.“ L: 1. . 1 . v ,. «E. a ; A?» (a: ".11."; av 31.. i... ,a. a. 5‘3}? 5.. aw? :3 .1 . l urefimbfl. .r Fawn... a I , a. . . . at am} whfimgxi . .eiilaug, . I3“. .2: . 2... A t i fl‘—n..§.f. .. .3. . 11.11.... as ._ .; iv.“ ' l r 51:9 ; .113.» EL 5:. . 7. a. :EH 5.8...» 3 r2 . 3.; v. 3%.». a. 7i 2?: as . 3.3m. . .. .v. r . .. .375... 4 .. .31 3.: .;.§...r$«.., (z. .11. . w: 3.... .a a»; . 12.3. a s; . 3.5.; : all .1, 1rd at In t n ‘1 133 y .r ) 3 .x . Eh . 5w...»\2....% .. 3176.; i»... . L. . . $323.4 .. a: a $07.77.». 2. z. . )a r ‘5‘ f. .) gtsw A 3; L23. .fi sfiuasn. I 3.391 ELF; due. L v: N, , 5.23. 1‘ a . 1.3:.1 an. .Auw;q3.w<.;€v3en~rar. . . r... r . . 72:3.“ Z: W....flu. .1...» 3? . .‘F‘ I, n. ; .finfiut a 1...... 53...: . if it. 2. .. ; a»; .. 31.14. Jv>k 24.))? 4.1. _. . 1 AI— . . $1210.). ail. . 1.9.3.; . E $.39.“ 2. .x. z. . ; . i .. ' 2:.» .2 «.1 1.3.3:. hit. I“... 2 e." If. T..u\i..uae..= it“? ($9.35.! ; 1.1. , .5? 3.3... .5..$X\;..... . 51. a 5:35)), .1; 32..., +2321 5; It} 5 5...; 2...... . . .6115. $1 9. . wit a _ .1 . 2:5; . 3.... EAS iii}. 3. . iFJxaa§ora v. . . .2.)SI .. .L‘. .l , 9. f. .C 1.1 A. x... 7. . .,.. 5H. 3.3.: t . 1.. y .2‘: f.” ‘ 1...... A .x 2....3 . 3 .. “mafia”... firing“? lscfir . , a. d? :IIKP (I. :zlii 25.»... . It. , 3.35.4.4. . .. sqou : . {1.55.71 36::32 . a x . in L2»? . . 337...: 742v; A3,. ). CL C . 31355:. \I, . .2}. .52). V2321; ::.~ ‘ .. .I...ir. 1 .Jiw (1.. rFl‘ris is to certify that the thesis entitled USE OF OBJECTIVE TESTS IN THE UNITED STATES AND TURKISH EDUCATIONAL SYSTEMS, AND INDUSTRY IN GENERAL, WITH IMPLICATIONS FOR TURKEY presented by Osman Kazanci has been accepted towards fulfillment of the requirements for Ph.D. Secondary Ed. 8 Curriculum degree in Jan/L? [6% Major professor . {gate 2/5/75 \ q 0-7639 ABSTRACT USE OF OBJECTIVE TESTS IN THE UNITED STATES AND TURKISH EDUCATIONAL SYSTEMS, AND INDUSTRY IN GENERAL WITH IMPLICATIONS FOR TURKEY By Osman Kazanci The Problem The productivity rate defined as the percentage of students who pass their classes or are promoted to the successive level of educa- tion in any given year has been very low in Turkey, around 50 percent, which Turkey cannot afford. The present measurement and evaluation system in Turkish schools, besides other factors, plays an important role in contributing to this state of affairs. Since the students' achievement and progress cannot be measured and evaluated adequately with the present system, various irregularities and irremedial and unfortunate results have been occurring in practice. With a better test and measurement system the quality and productivity rate of the Turkish educational system could be higher than it is now, as pointed out by the Turkey National Commission on Education in 1961. Method of the Study The study aimed to show the possibilities for developing a better measurement and evaluation system for Turkish schools and industrial organizations in general. For this reason, the educational systems of Osman Kazanci the United States and Turkey were reviewed with a particular interest in the history and use of tests and measurement in the two countries. A description and critical analysis of the literature in the field of objective testings in the United States was made with a view to identi- fying test uses, particularly the use of objective tests in promoting learning in education, and to a lesser extent their uses in industrial organizations. The similarities and differences and their underlying causes in the two systems of education were identified. This approach was based upon the idea that things outside the schools may matter more than things inside the schools, and help govern and interpret the things inside. The author's cumulative experiences in Turkish educational system as a teacher, administrator and research worker, and his observations of the United States educational system were also utilized. The Findings A. Similarities in the use of objective tests between the United States and Turkish educational systems found were in: l) selection of students to higher educational institutions which require certain qualifications, or establishment of levels of achievement for the numbers of students to be accepted, and 2) coaching of students for school entrance examinations. B. Differences in the use of objective tests between the United States and Turkish educational systems were stated as follows: tween to: l) 2) 3) C. Osman Kazanci objective tests, in the form of teacher—made, standardized, and program examination tests, were more widely and more frequently used in the United States educational system than they were in the Turkish educational system; objective tests were widely used for such varied purposes as gradings, promoting learning, diagnosis, guidance, placement, formative evaluation and research in the United States, while they were not so widely used for these purposes in Turkey; objective tests in the United States educational system were being far more intensely subjected to scientific studies in order to utilize the tests in promoting learning in education, and to a lesser extent in industrial organizations, than they were in Turkish educational system and industrial organiza- tions; and objective tests were used unofficially, although not often, for some purposes in Turkish educational systems. Reasons for the differences in the use of objective tests be- the United States and Turkish educational systems were attributed 1) 2) 5) differences in the cultural values of the two countries; the different educational systems of the two countries since the United States has a decentralized educational administra- tion system, while Turkey has a centrally administered.educa— tional system; the evolved models of educational systems in the two countries in that the United States has developed her own unique educa- tional system, while Turkey, with some changes, carried on the old European (primarily French) school systems, in which oral and essay type written examination were one of the prevailing features of the system; the differences in teacher training, since most colleges of education in the United States require many more courses in the field of tests and measurements than the teacher training institutions in Turkey which require only a few such courses; and specialization and research in the field of tests and measure- ments, as a consequence of the teacher training model of United States education, and United States administrative structures Osman Kazanci of education, since far more attention has been paid to specialization and research in the field of tests and measure- ments in the United States than in Turkey. D. Reasons for similarities in the use of objective tests between the United States and Turkish educational systems were attributed to a) the influence of United States education on Turkish education, and b) practical and economical reasons. E. Implications for developing a better measurement and evaluation system in Turkey were: l) changes in the present Turkish examination regulations should be made so as not to prohibit teachers from using objective tests, N V changes in the curricula of Turkish teacher training institu- tions should be made so that additional or new courses in the field of tests and measurements would be added as required courses in these teacher training institutions, 3) in-service tests and measurements training for teachers should be added in Turkish schools in the form of seminars, workshops and short term courses, and .h v comparative and survey type researches in the field of objective tests and measurements should be carried out in Turkey. USE OF OBJECTIVE TESTS IN THE UNITED STATES AND TURKISH EDUCATIONAL SYSTEMS, AND INDUSTRY IN GENERAL WITH IMPLICATIONS FOR TURKEY By Osman Kazanci A DISSERTATION Submitted to Michigan State University in partial fulfillment of the requirements for the degree of DOCTOR OF PHILOSOPHY Department of Secondary Education and Curriculum T974 DEDICATION To my father M. Mustafa who has been the best teacher I have ever had, and to my wife Sayan whose love, patience, sacri- fice, and encouragement have been the cornerstone of this study. ACKNOWLEDGEMENTS My very grateful thanks are due to Dr. Harry L. Case, chairman of the doctoral committee, for his continued encouragement and generous support which made this study possible. I am especially indebted to him for his support in every way. I do wish to express my sincere appreciation to Dr. Robert L. Ebel, who served as chairman of the doctoral committee for more than two years. I deeply appreciate his understanding, advice and support from the begin- ning to the end. My grateful thanks are also due to Dr. Frederic R. Wickert who served on the doctoral committee. Words cannot express the gratitude which is his due. My special thanks are due to Dr. Carl H. Gross who served on the doctoral committee and paved the way to the success of this study. I gratefully appreciate the assistance of Dr. Ben Bohnhorst for reading and advising in preparing the final manuscript. In particular, I would like to extend my deep appreciation to Mr. Musret Karcioglu who helped me in every possible way to complete this study. Finally, I wish also to express my appreciation to the USAID for providing scholarships to earn my degrees of B. S., M. S., and Ph. D. CHAPTER II. TABLE OF CONTENTS Pa« LIST OF TABLES ........................................... V' LIST OF FIGURES .......................................... vi‘ THE PROBLEM .............................................. Introduction to the Problem ........................ .. Statement of the Problem .............................. Purpose of the Study .................................. Importance of the Study ............................... Limitations of the Study .............................. Method of the Study ................................... Order of Presentation of the Study .................... TURKISH EDUCATIONAL SYSTEM AND MEASUREMENT AND EVALUATION SECTION I THE SCHOOL SYSTEM IN TURKEY Primary Education ..................................... Secondary Education ................................... Vocational-Technical Education ........................ Higher Education ...................................... Administration ........................................ Evaluating of Learning Outcomes in Turkish Schools.... SECTION II MEASUREMENT AND EVALUATION IN TURKEY History of the Measurement and Evaluation in Turkey... Present Situation of Objective Testing ................ Teacher-made Tests ........................... . ........ Standardized Tests .................................... iv TABLE OF CONTENTS--Continued CHAPTER Pa Program Examination Tests ............................ Measurement and Evaluation in Turkish Industry ....... Summary .............................................. III. THE UNITED STATES EDUCATIONAL SYSTEM AND MEASUREMENT AND EVALUATION ........................................... SECTION I THE SCHOOL SYSTEM IN THE UNITED STATES OF AMERICA Elementary Education ................................. Secondary Education .................................. Higher Education ..................................... Administration of Education .......................... Grading System in American Schools ................... SECTION II MEASUREMENT AND EVALUATION IN THE UNITED STATES History of Educational Measurement and Evaluation in the United States ................................. Present Situation of Measurement and Evaluation ...... Teacher-made Tests ................................... Standardized Achievement Tests ....................... Program Examination Tests ............................ The Use of Objective Tests in Industry ............... Summary .............................................. IV. THE USE OF OBJECTIVE TESTS TO PROMOTE LEARNING IN THE UNITED STATES ........................................ Studies of Feedback and Class Discussion ............. Studies on Open-Book Examinations .................... Coaching Studies ..................................... Summany .............................................. V. COMPARISON, IMPLICATIONS, SUMMARY AND CONCLUSIONS ....... TABLE OF CONTENTS--continued CHAPTER Page SECTION I COMPARISON OF THE USE OF OBJECTIVE TESTS IN THE UNITED STATES AND TURKISH EDUCATIONAL SYSTEMS, AND IN INDUSTRIES IN GENERAL Reasons for Similarities and Differences in Use of Objective Tests in the United States and Turkish Educational Systems ................................ 134 SECTION II IMPLICATIONS FOR TURKEY Need for Changes in the Examination Regulations ....... l43 Need for In-service Training (Workshop and Seminars).. 145 Need for Changes in the Course Programs of Teacher Training Institutions .............................. I46 Summary and Conclusions ............................... T48 BIBLIOGRAPHY .................................................... l54 vi LIST OF TABLES TABLE I. Proportion of Students and Schooling of the Total Number of School Age Population at Every Educational Level (except higher education)--l968-1972 ............................... 2. Productivity Rate at the Middle School and Lise Level (l968-1972) ................................................ 3. A Comparison on the Use of Objective Tests in the United States and Turkish Educational Systems, and Industry in General .................................................... vii Page 129 LIST OF FIGURES FIGURE Page l. The structure of education in Turkey ...................... l6 2. A summary diagram of evaluating of learning outcomes in Turkish schools ........................................... 34 3. The structure of education in the United States ........... 63 4. General summary of the three types of testing and their usages in the United States and Turkish educational systems ................................................... I33 viii CHAPTER I THE PROBLEM ”For centuries, the Turks have been moving in the same direction. Always from east to west. There are many countries, but one civilization. National progress means participation in this civilization.” M. Kemal Atatflrk Introduction to the Problem Turkey has been engaged in the development process by modeling the Western countries for more than a century. She is the first non- western nation to seek a new existence within the political, cultural, and technological mold of the Nest. Under the leadership of AtatUrk, the founder of modern Turkey, and thereafter, to be civilized and to develop have always meant to be westernized politically, culturally, and technologically. Education, in this context, has been accepted an used as one of the most effective means of Westernization. In 1927, only 10.7 percent of the Turkish people were literate; in 1965, 48 per cent of the people knew how to read and write. By 1970 it had increas to 55 percent. In 1923, the nation had 4,894 primary schools, and 10,238 teachers teaching 342,000 students; in 1971, 132,577 teachers i 38,227 primary schools were teaching more than 5 million primary schoo children. The number of secondary schools had risen from 160 in 1923 (data on teachers not available) to 3,283 with 47,476 teachers in 1971 and 1.5 million secondary school students, as compared to 12,500 stu- dents in 1923.1 In fifty years the nation thus had achieved a more than eight- fold increase in primary schools, fifteen—fold in primary school teachers, and more than fourteen-fold in primary school students. At the secondary education level, increase in the number of schools is more than twenty—fold; in the number of students it is one hundred and twenty- fold. In spite of these increases, schooling rate (proportion of stu- dents at an age group attending school to the total age group) has still been low in comparison with the Second Five Year Development Plan targets. Table 1 shows that in the 1971-1972 academic year (with some opti— mism) more than 60 percent of the 11—14 middle school age group, almost 90 percent of the 15-17 lise (senior high school) age group and at least 94 percent of the 18-22 higher education age group did not con— tinue their education. 0n the other hand, a sizeable percentage of these students (as high as 23.5 percent at the primary level and 33 percent at the lycée—— hereafter lise-level) failed in their classes. Thus productivity rate defined as the percentage of students who passed their classes or were promoted to the successive level of education in any given year, has changed between 76.5 percent and 80.6 percent of the primary level, 1All statistics given here are taken from the Devlet Ististik EnstitUsU (State Statistics Institute), and Devlet Planma Tes—Kilati (State Planning Organization) publications. .momF ”mamx:< .Aommv cowumecmmco mcwccmFA mpmum .CMFA ucmEQOFm>mo mew m>wm vcoomm mzu Eocw umpqmu<¥ m.oF o.mmm mmv.m o.mm o.qmm wmm.m o.Nw mmo.m mmn.m Nnm_-_mm_ o.op o.mmm wmm.m 0.0m o.~mn moo.m o.mw Nmm.q m_n.m me_-0nmp o.m o.mom mmN.N o.—m o.own amm.m o.mm mmm.¢ Nmo.m omm_-mmm_ o.w o.mn_ wm_.N o.om o.mvm nqv.m o.ow nmm.¢ me.m mmm_-mmm_ Sd Si .08 8.0 Si :03 8.0 Si :08 DJ 1.0 03 DJ .40 03 DJ 1.0 03 UIO n1. dUi UIO n1. dUI UIO n1. dUi 0d 0.0.. "0 0d Dr? "0 0d 0.? no 00 8|. [.0 00 9|. ..L0 00 8|. [.0 LMOX [J U 9'. [J U 9'. [J U 9'. 60 06 60 06 50 06 U U3 U U3 U U9 m0 HMO “We/0 (I: (IT. ([7. 32: 28:8 23.2% Foocom acmucoowm Foocom xcmccoumm Foocum ALmEFLa x.mucmmzo;p cw mmczmwm .mmm_-mmm~ Aco_pmuzvm gmcmw; uqmoxmv Fm>04 Facowpmozum Aco>m pm cowum_:aoa mm< Foozum we consaz Papop asp yo chFoosum use mpcwvzum we :owusoaocm ._ BBS 68.9 percent and 70.0 percent at the middle school level, and 67.0 percen' and 69.5 percent at the lise level during the period of 1968-1972 (see Table 2). Table 2. Productivity Rate at the Middle School and Lise Level (1968- 1972). Figures in percentages* Academic Years 5C“°°‘ Leve‘ 1968-1969 1969-1970 1970-1971 1971-1972 Primary Passed 76.6 78.1 79.8 80.6 56h°°l Failed 23.5 21.9 20.2 19.4 Middle Passed 68.9 69.6 70.0 69.8 ' 5c“°°‘ Failed 31.1 30.4 30.0 30.2 ; Lise Passed 69.5 69.4 , 67.0 68.9 Failed 30.5 30.6 33.0 31.1 *Adapted from a) 1972-1973 Ilkdgretim Yillidi, Milli Editim Bakanligi Ilkfigretim Genel Mfidfirlggfi p. 10, and b) 1972-1973 Ortaoqretim, Milli Egitim Bakanligi, Ortadgretim Genel MUdUrlUgU, p. 191. Ankara, Turkey. In addition to the social and psychological consequences, the elementary school failures cost the nation (1,000,000 x 780.00 T.L. = 780,000,000.00 T.L.); secondary school failures cost (95,000 x 540.00 1 T.L. = $0,300,000.00 T.L. every year. If the money spent for auxiliary services for education was added to these figures the wastage for every 1a) 1973 Mali Yili Program--Alt Program ve Faaliyetlere Gdre Milli Egitim Bakanligi,BUtcesi Harcama Kalemleri, P1an1ama-Arastinma ve Koordinasyon Dairesi Baskanligi, Ankara, 1973. b) 67 Ilde Okul, Ogretmen. Ogrenci Sayilari, 1972:1973, P1an1ama--Arastirma ve Koordinasyon Dairesi BasEanligi, Ankara, 1973. academic year would well exceed one billion Turkish Liros, that is a 20 percent of the budget alloted to the education. Several attempts have been made at different times to increase the productivity rate in secondary schools only through changes in the rules and regulations of promotion and examination. These simply con- sisted of lowering the average for passing, or calculating the passing grade on the basis of weighted points for each subject matter, or in- creasing the number of written and oral examinations, or permitting those students who failed in only one subject with the condition that they must take a completion examination and pass it during the succes- sive classes.1 Despite these efforts, the productivity rate remained as low as 60 percent. Of course it is doubtful whether the quality in- creases when the examination regulations are changed. It is a common complaint among teachers, parents, and even among the intellectuals of the society who also passed through the same system that the schools are lowering standards. The fact is that the schools above primary education level are selective institutions.2 Although anybody who is a graduate of a pri— mary school can register for middle school and anybody who is a graduate of middle school can register for lycee, many students fail and repeat grades or are dismissed according to the regulations or drop out of school through the process of selective examinations. There are various 1Cumhuriyetin 50. Yilinda Milli Egitimimiz, Milli Egitim Basimevi, Istanbul, 1973, pp. 86—90, 96-97. 2Andreas M. Kazamias, Education and the Quest for Modernity in Turkey (London: George Allen and Unwin Ltd., 1966), pp. 134-139. reasons (socio-economic, psychological, intellectual) for failure and for being dismissed or dropping out of school. But whatever the reasons are, whether a student passes the course(s) or not is determined by the present examination system. The present examination system in Turkish schools requires oral and essay type examinations in addition to quizzes and heavy homework. The frequency of these examinations during the school year is dependent upon some conditions such as class size and teaching load of the teacher. Graduation examinations are closely supervised by the Ministry of Education. Questions for lise graduation are prepared by a central committee in the Ministry of Education. While all examinations in classrooms and at graduation are oral and essay types, selection of students for state boarding schools and higher education is made by means of objective measurement techniques. Although the regulations require the use of oral and essay type examinations, since the late 1950's some teachers occasionally use objective type tests. The motives in using objective tests vary from one teacher to another: some use them for learning practice, some for the objectivity, and some to prepare students for the state boarding schools or higher education entrance examinations. Statement of the Problem The foregoing brief observations show that the schooling rate as well as the productivity rate in the Turkish educational system is as low as the country can tolerate. The present measurement and evalu- ation system, besides other factors, plays an important role in preparing to this end. With a sound and objective measurement and evaluation system the quality and productivity rate of Turkish educational system could be higher than it is now. Because of overcrowded classes, heavy teaching loads of teachers (at least 24 hours per teacher per week in different classes and in different subjects), irregularities of administration with regard to measurement and evaluation, and non-objectivity inherent in the type of examinations employed, in addition to other variables, it is prac- tically impossible to have impartial, adequate and sound evaluations of students' achievement. In 1961, the Turkey National Commission on Education pointed out that measurement and evaluation was one of the crucial problems of the Turkish educational system: “The measurement and evaluation of student achievement, based on the old-fashioned examination regulation, especially at the secondary general and secondary technical-vocational levels is deficient in objective principles of measurement and evalua— tion. The oral and essay type examinations used at the first and second grades of middle schools and lycées (senior high schools) measure only factual knowledge which is acquired by rote; these types of examinations cannot measure adequately the level of educational attainment of students with regard to knowledge, reasoning and compre- hension, and applicability of these to real life situations. In addition, middle school, and lycée graduation examinations are far from demon— strating whether the graduates of these schools have attained the educa- tional accomplishment that they were supposed to. "Since the students' achievement and progress cannot be evaluated adequately, various irregularities, and irremedial and unfortunate results are occurring in the practice of this measurement and evaluation system."1 The situation with regard to measurement and evaluation in Turkish industry is not much different from that in Turkish education. Since the executives,administrators, managers and supervisors in industrial and governmental organizations come through the same educational system, and most likely they are not acquainted with objective measurement and evaluation techniques, the selection, training, placement, and promotion of personnel are made by means of non—objective measurement and evalua- tion techniques such as reference letters, interviews, and oral and written (essay) examinations. Because of these practices, unqualified persons may have greater chances of being selected for a job; from training efforts in valid conclusions may be drawn; not as appropriate a person may be placed in a position and not as appropriate a person may be promoted. Thus Turkey has the problem of developing a measurement and evalu- ation system that is a) objective, scientific and impartial; and b) facilitative for learning in formal learning situations. lTiirkiye Egitim Milli Komisyonu, M. E. 3., M1111 Egitim Basimevi, Istanbul, 1961, p. 103. Turkey National Commission on Education was established to survey the educational problems in Turkey and make recom- mendations for solutions on the basis of examples from developed countries such as the United States, Great Britain, West Germany, France, Italy and Japan. Purpose of the Study The purpose of this study is to compare the use of educational objective tests in the United States and the Turkish educational systems, with particular reference to the use of objective testing to {Homote learning in educational, and industrial organizations in general, in order to show the possibilities of a sound and better measurement and evaluation system which is objective, impartial and facilitative for learning in the Turkish educational system. The specific questions to which answers were sought are: 1) For what purposes are objective achievement tests used in the United States and Turkish educational systems, and industrial organizations in general? 2) How are objective achievement tests used to promote learning in the two educational systems? 3) What are the similarities and differences in the use of ob- jective achievement tests in the two countries? What are the reasons for similarities and differences in the use of objective achievement tests? 4) What are the implications for developing an objective measure- ment and evaluation system to measure and evaluate, and pro— mote learning in educational and industrial organizations in Turkey? Importance of the Study The present examination system in the Turkish educational system serves only the purpose of providing the basis of grading of students' mfifievement, and creates serious problems. Some of these problems are frequently cited as follows: 1) It is unfair to the students. The literature on essay type emmfinations indicates that: a) essay type examinations have relatively l0 low reliability because of the limited sampling of learning and sub- jectivity of scoring; b) since they may be constructed quickly and carelessly, questions may be ambiguously stated, and most likely are of unequal difficulty—~students can bluff; c) they take too much time for the students to write, and too much time for the teachers to read; and d) students' grades may be affected by halo, legibility of handwriting, spelling and grammar errors, and effectiveness of written expression.1 2) It is unfairly used by some teachers whenever they are not well prepared for the subject, or whenever they want to keep the class quiet, or whenever they want to discipline some students. The author of this study has experienced and indirectly witnessed these practices both as teacher and student, and listened to many complaints by stu- dents, teachers and parents. 3) The present examination system is unfair both to the students and teachers: classes are crowded, teacher load is heavy, a teacher teaches, most of the time, more than one class and more than one subject. Therefore it is practically impossible to read all papers carefully, to grade impartially and relatively objectively, and quickly. Students cannot get a second or third chance to demonstrate their abilities and level of achievement. It is quite possible that many good students cannot get as good a grade as they deserve. It is also possible that many unqualified students may happen to pass or even be graded highly. Many of the students feel the injustice of the system. Toward the end 1Robert L. Ebel, Measuring Educational Achievement (Englewood Cliffs, New Jersey: Prentice-Hall, 1965), pp. 84—102; see also R. L. Thorndike and E. Hagen, Measurement and Evaluation in Psychology, and Education (New York: John Wiley and Sons, 1955), pp. 35-42. ll of spring semester each year illegal actions start to occur: teachers are threatened, some may be beaten by unidentified persons; bribes are offered to teachers; pressures from local influential persons arise; and rumors are frequently heard in the town, or in the school neighbor- hood, that some students were failed on purpose in order to have enough subjects for high priced private courses to pass completion examina- tions. Some students drop out of school or run away from home. There is a need for development of better measurement and evalua- tion systems. This study is an attempt in that direction. The Turkey National Commission on Education urged the study and development of new objective measurement and evaluation techniques as they were observed in the United States, Japan, France, Great Britain, West Germany and Italy:1 1) The examination regulations of secondary general and secondary vocational—technical schools should be changed; and it should be provided that new examination regulations use objective examination techniques in the measurement of students' achievement, program, and their promotion from one grade to the next. 2) The new objective measurement and evaluation techniques provid- ed by new examination regulations must be taught to those teachers who are teaching now in Turkish schools by means of various courses and seminars. 3) During the academic year, the achievement and progress of stu- dents must be observed, measured and evaluated; and measurement and evaluation techniques must be re-examined, and new ones should be sought, if they are not functioning well. 4) These new objective measurement and evaluation techniques must be imported into the curriculum of teacher—training institu- tions at all levels; and these new techniques must be empha- sized and practiced in various ways in these institutions. 5) Guidance and counseling services must be provided. 1Turkiye Egitim Milli Komisyonu Raporu, op, cit., pp. 103-104. Limitations of the Study The study confines itself to the use of objective achievement tests with particular reference to the use of objective testing largely in education and partly in industrial organization. The use of achieve- ment tests are discussed under three rather broad categories, 1) teacher made tests, 2) standardized tests, and 3) program examination tests. The study also limits itself to the available literature on the Turkish educational system and the experience of the author. Method of the Study This is a descriptive and critical analysis of the literature in the field of objective testing in the United States with a view first to identifying their uses, and particularly the use of objective tests in promoting learning in education, and to a lesser extent in industrial organizations. The second step in the analysis is to identify the similarities and differences, and their underlying causes in the two systems of educa- tion. This approach is based upon the idea that the things outside the school matter more than the things inside the schools, and govern and interpret the things inside. This study also utilized the experience of the author. The author taught at all levels of the Turkish educational system, in addition to the administrative position he held in the Test and Research Bureau of MOE for more than four years. 13 Order of Presentation of the Study The remainder of the study includes four chapters. Chapters II and III are about the educational systems of the United States and Turkey in general, and measurement and evaluation in particular in these two systems. Chapter IV deals with the use of tests to promote learning. Chapter V gives the comparison, implications, and summary and conclusions. Chapter 11 describes a) the structure of school systems in the Turkish educational system as it exists today, b) administration and organization of education, and c) grading systems in the Turkish educa- tional system are described and to a lesser extent discussed. Section 2 of Chapter II describes and discusses a) the history of measurement and evaluation in the Turkish educational system and b) the present situation of measurement and evaluation in the Turkish educational system. The present situation of measurement and evalua- tion is discussed under three broad types of objective tests: teacher— made, standardized and program examination tests. Some available limited information is also given about the use of objective tests in industry in Turkey. Chapter III follows the same pattern as Chapter II but more in- formation is given about the use of objective achievement tests in American industry than in Turkish industry. Chapter IV describes and analyzes the use of educational objective tests to promote learning in classroom situations. The literature has been reviewed under subheadings such as "studies on the open-book 14 examination," literature on the use of "re-test examination technique," and studies on "coaching studies." In Chapter V the use of educational objective achievement tests in the United States and Turkish educational systems is compared with regard to similarities and differences. Then underlying reasons for similarities and differences are discussed. Chapter V also includes implications for the Turkish educational system with respect to developing improved objective measurement and evaluation systems for grading and promoting learning in education, and also in industry, and summary and conclusions. CHAPTER II TURKISH EDUCATIONAL SYSTEM AND MEASUREMENT AND EVALUATION The purpose of this chapter is to describe the general background and characteristics of the Turkish educational system in order to show the reader a) how students' achievement is measured and evaluated, b) what the deficiencies and difficulties are in the system in terms assessing students' achievement, c) what is being done in developing a better technique, and how it is being done. This chapter is composed of two sections. In the first section the school system, involving its organization and administration, and presently practiced measurement techniques are described. The second section deals mostly with the history of present examination technique, and objective measurement and evaluation technique in the Turkish edu- cational system. SECTION I THE SCHOOL SYSTEM IN TURKEY The present school system in Turkey follows the 5-3—3-4 pattern (see Figurel). Primary education is universal, free at public schools, compulsory until the age of 14, and for five years in duration. 15 l6 24 23 1 1 ' r 22 .2 : El : (Universities) : Technical 4 8 1'- Liberal Arts .Vocational I . 21 g I g l : Teacher- ' Egucation- 3 1 - - u 20 2 his): : | tra1n1ng . Institutes 2 . 1 g . 19 I ; l ' 1 _L I I 18 é 4 r’ r v "- 17 Lycee ' l l 5% 3 (Senior High Schools)’ Technical: Vocational 1 >,L g 16 5 Schools . Schools gégg 2 l ‘5 i : iii-.33? 1 I T l r l- T Y Y Y ' l ' I 14 l 3 13 Middle (Orta okul) Schools : 2 (General, Vocational, Technical) 1 12 l 1 ll 5 1° Primary Schools 4 9 3 8 2 7 1 Age Grade Figure 1. The structure of education in Turkey. Baccalaureat Lisediploma C .2 .p (U U 5 '0 Lu >5 S. S g Diploma U m (I) Diploma Z O u- .p m U 3 U LLI :1 E S... O. 17 Secondary education is divided into two levels. The lower level, or middle school, is for three years and free at public schools. Secondary education at the upper 1eve1--1ise and lise equivalent voca- tional and technical schools--is for three years (4 years in primary teacher training schools) and free at public schools. Higher education is from three to six years, mostly four years at universities, and it is free. No entrance examination is required for entrance to any schools up to the higher education level. The majority of schools are coeducational, but there are some schools for only boys or girls. The schools (except for universities and academies), are centrally administered and controlled by the Ministry of Education (MOE). The school curriculum for each level and type of education up to university level is the same throughout the country, although some changes in the order of presentation can be made by teachers depending upon the loca- tion of schools in different regions of the country. Although schools are centrally administered and controlled by the MOE, it seems very difficult to assess the progress in education by means of presently practiced measurement and evaluation techniques. Primary Education A child who reaches the age of six enters primary school and may hope to graduate at age twelve. He may, however, stay in primary school until age fourteen. Though the five years of primary education are com- pulsory and free in public schools, there are also some private primary schools, especially in big cities, run by private individuals or 18 companies. The amount of tuition and fees in these private schools is fixed by the NOE. Primary schools in most of the villages are one- or two-room schools which are taught by one or two teachers. Primary schools in cities, towns, and big villages usually have at least five or more teachers. In primary schools a teacher teaches all subjects at a grade level, and usually he starts from Grade I and teaches the same students until Grade V. The student-teacher ratio is 1/40 or above in many primary schools. Primary school teachers are graduates of primary teacher training schools, or lycée graduates who have passed the examination in some courses taught only at teacher training schools and have some practical teaching experience under supervision. Most of the primary school teachers, however, come from teacher training schools. Students attend- ing teacher training schools do so for four years above middle schools or for seven years above primary education. All four and seven-year primary teacher training schools are boarding schools and by law 75% of the students for seven-year schools are selected from among village primary school graduates. Although a few of the teacher training schools are for only boys or girls, most of the teacher training schools are coeducational. While four-year teacher training schools are located in cities or big towns, all seven-year primary teacher training schools are located in rural areas. Today there are 89 primary teacher training schools and 27 of these are seven-year schools.1 1Turkiyede Okul, Ojretmen, O§renci, Yeni Kayit, Mezun Sayilari 1963-1973, M.E.B. P1an1ama-Arastirma ve Koordinasyon Dairesi, ty1u1 1973, AnEara, p. 5. 19 Primary education serves four basic general objectives. These four general objectives are the cornerstones of primary schools in Turkey and they serve also the general objectives of Turkish national education. As stated in the regulations, it is the responsibility of the primary school to see that (1) Every student, as an individual, has become aware of his ability and capability to deal effectively with his environ- ment, has become a good citizen, and has developed a good personality and sound ethical and moral values; (2) Every primary school child fully appreciates the necessity of good human relations and cooperation with others and that these are inevitable aspects of social life; (3) Every primary school child comprehends that Turkey is a nationalist, republican, democratic, secular and social (welfare) state based upon universal human rights; that the Turkish Nation is an indivisible entity of land and people, and that it is an honorable and constructive member of the society of nations; (4) Every child in the school comprehends that man-power is the richest resource in the development of the country and, therefore it is necessary that these resources be developed as the best means of investment in the development of the country.1 -]Ilkoku1 programi. Milli Egitim Basimevi, Istanbul: 1968, pp. 2-8. 20 Measuring and evaluating of general objectives as well as specific objectives of primary education has a greater chance in primary schools than secondary schools, because: a) a class or grade system is followed in primary schools, rather than a course system, in that a teacher in a primary school teaches all subjects and continues, usually, with the same students until the last year, and b) primary school teachers are more and better informed about objective measurement and evaluation techniques than are secondary school teachers. Secondary Education Secondary education in Turkey has two tracks: General secondary and vocational and technical secondary. The choice of one or the other of these tracks is left to the primary school graduates or their parents. The general secondary education program is comprised of two cycles. The first cycle is called ortaokul (middle school) and it is for three years. Any student who is a graduate of a primary school with a diploma can enter ortaokul. There is an ortaokul graduation examination at the end of the third grade, and graduation is certified by an ortaokul diploma. In 1970-1971 there were 1,818 ortaokuls in Turkey, of which 112 were privately owned.1 All of them were day schools except for 29 evening-ortaokuls. Only two ortaokuls existed for exceptional children (for deaf and blind) in Turkey. Classes usually are crowded; for example lTiirkiyede Okul, Ogretmen, Ogrenci, Yeni Kayit, Mezun Sayilari, op. cit., p. 2. 21 the teacher-student ratio in 1969-1970 was 1/57.1 Ortaokul or ortaokul level teachers are almost exclusively gradu- ates of "Educational Institutes" which are for three years. Karagozogl found that more than 74 percent of the teachers in secondary schools we graduates of three-year "Educational Institutes." (By regulation, teachers at the second level-lise of secondary education are supposed to be university graduates. But since there are not enough university graduates to teach at this level, Educational Institute graduates fill the gap.) The second cycle of general secondary education is lycée (senior high school) and it is for three years. Every student with an ortaokul diploma can enter. Certain private and specialized lycées, however, require their applicants to take competitive examinations. General education is much more highly esteemed than other forms of lise-level education in Turkey. In 1969-70, 60 percent of the stu- dents in all upper levels of secondary education were at general lises. Because the general lise in the Turkish educational system is the main institution which prepares students for universities, at the end of the last year every third grade is required to take the State Lise Graduati Examinations (Devlet Lise Bitirme Imtihani). Those who successfully graduate from the lise receive the "lise diploma" and can apply for uni versity entrance examinations. 1G. Karagozoglu, The Role_9f;the Ministry Supervisors in the Educational System, Unpublished Ph.D. thesis, College of Education, Michigan State University, 1972, p. 47. 2Karagozoglu, op. cit., p. 220. 3Ib1d., p. 49. 22 Ortaokuls and lises are the most neglected institutions in the Turkish educational system, especially in the eastern part of the country, in terms of curriculum, equipment, and teachers. It is said that because of educational measurement techniques, and grades given on the basis of these non—objective techniques and observations, many unfortunate things happen every year; teachers are threatened; teachers lose their prestige; students lose the opportunity to continue their education; parents, as well as students, develop negative attitudes toward school and education, etc. Vocational-Technical Education Vocational-Technical schools have also two cycles. Vocational education in the first cycle is offered in agriculture, boys' and girls' I In 1970- technical, commercial, health, and teacher-training schools. 1971 there were 284 such schools in Turkey.2 The studies in these schools intend to provide the students with a marketable skill or to enable them to continue their education at the second cycle. Although the curriculum is similar to that followed by the general ortaokuls, there are some additional required courses or practical training. It is usually very difficult, but not impossible, for those who complete these schools to continue their education in a lise. Most of them continue their education at the second cycle of these schools. 1The Educational System of Turkey, USOE, Washington: 1971, pp. 7-8. 2Tiirkiyede Okul, Ogretmen, Ogrenci, Yeni Kayit, Mezun Sayilari, op. cit., p. 3. 23 The second cycle of vocational-technical schools admits students who have completed a middle (orta) vocational-technical or general school. In 1972-1973 there were 869 vocational-technical schools at the upper secondary level, 271 of these schools were technical schools for boys and girls. In these schools 12,422 teachers were teaching 246,755 students.] Most of the vocational schools are boarding institutions. For example, in 1970-1971, 238,877 students applied for admission to primary teacher-training schools and took entrance examinations. Of these only 17,419 were admitted.2 Students are admitted to boarding schools on the basis of a battery of achievement tests. These tests are objective; they are prepared and administered by the PAKD test specialists. A few of the students may be familiar with this kind of tests. The four-year primary teacher training school students, however, are taught some techniques of objective measurement and evaluation later in school while other school students follow almost the way, in terms of measurement and evaluation, as secondary school students. Higher Education Higher education in Turkey is for three years or more. Some pro- grams are for four years, some for six years. "Educational Institutes" are exclusively three-year institutions. There were 16 of these in 1Ib1d., p. 9. 2Karagozoglu, op. cit., p. 51. 24 1973,1 preparing 7,423 prospective secondary school teachers. Educational Institutes are boarding schools and admit students from various regions on the basis of competence examinations held by the MOE. These institutions are the main resource of the secondary school teaching staffs. They admit lise and primary teacher-training school graduates, though the lise graduates usually prefer attending universities. Lise graduates may prefer these institutions after they have failed to be admitted to the universities. Another type of higher educational institution for teachers is the Higher Technical Teacher-Training School. In this category there are three institutions: one for girls, two for boys. Teachers for secondary- 1eve1 technical schools are trained in these schools for four years. Students who have completed a secondary technical school are accepted in- to the program of these schools on the basis of competitive examinations. There are other vocational higher educational institutions, also for four years, in fields such as commerce, health, applied arts, fine arts, music and Islamic studies. Universities have a four year program, except the schools of medi— cine, which are six years. In 1972-1973 there were nine universities in Turkey: three in Ankara, three in Istanbul, and one each in Izmir, Trabzon, and Erzurum. The Turkish government, however, is planning to open new universities in several other cities like Konya, Adana and Diyarbakir. How many students should be admitted is decided by the universities themselves, although the NOE can make some suggestions. 1Tarkiyede Okul, 69retmen. BQrenCi. Yeni Kayit’ ”92““ Sayilarl’ op. cit., p. 10. 25 A11 universities in Turkey are autonomous with the exception of Atotfirk University in Eczurum and the Technical University of the Black Sea in Trabzon, which were established recently by special laws. Higher educational institutions, especially universities, are the only institutions that are free in employing objective or subjective measurement techniques in their educational activities. Selection of students to all higher educational institutions is made by objective tests. But practicing of objective measurement and evaluation techniques is almost ignored in most of the classrooms. Administration In Turkey, education at all levels is organized and administered centrally under the Ministry of Education, which exercises supervision and control over all schools. There are a few exceptions to this rule. The autonomous universities and schools under the sponsorship of minis- tries other than the MOE are administered and controlled by their respective related bodies. The curricula for these latter schools, however, are also approved by the NOE. Otherwise, the MOE is the only decision-making and controlling organization in the operation of all schools. The curricula they follow, the textbooks and teaching materi- als they use, assignments and dismissal of teachers, are all determined by the NOE. The minister is a member of the cabinet; he is a politician; he may or may not be a professional educator. He is assisted by the Under Secretary for General Education and the Under Secretary for Technical and 26 Vocational Education. All of the staffs in these offices are experienced and qualified professional educators, and they are appointed by the Minister of Education. Under the two Under Secretaries there are four- teen General Directorates which have also their subordinates. Besides the two Under Secretaries which assist the Minister of Education, there are three educational bodies that advise the Minister. The first advisory body is the National Board of Education (NBE). The NBE is the chief advisory body in the MOE; it prepares the school curri— culum on the basis of the other advisory bodies suggestions; examines and approves all textbooks; ratifies proposed regulations and legisla- tion; and provides some professional advice to the Minister of Education on educational matters. The members of the NBE are among the highest ranking professional educators. There is no written requirement about how many members there should be on the NBE. They are appointed by the Minister and approved by the President of the Republic through the Prime Minister. The second advisory body to the Minister of Education is the Com— mission of General Directorates (CGD), which is comprised of General Directors of the MOE and department heads. The CGD advises mostly on administrative procedures of the schools and disciplinary decisions at the upper level. The third advisory body to the Minister of Education is the National Council of Education (NCE) which meets every four years upon the invita— tion of the Ministry of Education. On important educational issues, however, the NCE may be invited to meet by the Minister more frequently; 27 sometimes its meetings may be prolonged. The NCE discusses and makes recommendations on educational or pedagogical issues of importance. The members of the NCE consist of representatives from the Ministry, universities, school administrators at every level, teachers, and some specialists in different fields selected by the Minister. Another important organization in the MOE is the Planning, Research and Coordination Office (its Turkish initials or PAKD). The PAKD is directly responsible to the Minister of Education. It conducts educational research, develops plans for Turkish education, and provides coordination between the Minister and other organizations in the MOE, and between the MOE and the State Planning Organization (SPO). One of its major functions is to construct and administer all state boarding school entrance examinations through its Testing Department. At the provincial level (there are 67 provinces), the administra- tive officer is the Director of National Education (DNE), who is ap- pointed by the MOE. The DNE is an experienced and qualified teacher with a diploma from at least an Educational Institute. Although the ONE is appointed by the MOE, he is attached to the Office of the Provincial Governor (Vali), and he acts as both an advisor and an assistant execu— tive to the governor. All schools in the province, except higher edu- cational institutions, are subject to the control of the ONE, but he has little authority to make decisions. His functions are mainly applica- tion and interpretation of the MOE's orders and regulations. Under the NDE there are various offices which are directly responsible to him. The central government is responsible for all public educational expenses from the building of schools and purchase of necessary equipment 28 to the paying of teacher's and others salaries. At the primary leve however, primary schools receive some local support, chiefly for the construction and maintenance of schools. Private schools, on the ot. hand, are financed through fees, income from property and investment and gifts and donations. Evaluating_of Learning Outcomes in Turkish Schools The present procedure for evaluating learning outcomes in Turk‘ schools is the outgrowth of the system's examination system which da' back to the nineteenth century. As has been discussed in the first chapter of this study, such an examination system has been creating 1 unpleasant conditions. The Turkish society has been paying its cost many years and there is not yet any study on such an evaluation tech- nique. What has been done has been just to change the scale of grad' or decide (by the MOE) that a grade of 4.0, which is a failing grade a "l to 10" scale, will be a passing grade for a certain time, certa' year. The author of this study believes that the cost of the presen' evaluation techniques of learning outcomes is unbearably heavy, and 1 complicated, too. To clarify the situation, a brief summary of grading system in Turkish educational system is presented below. The grading system in Turkish schools varies from one level to another. At the primary level it is based on a "l to 5" scale with ' and 2 lowest and failing, 3 pass or average, 4 good, and 5 excellent At the secondary schools it is based on a "1 to 10" scale, with 9 ant 29 excellent, 7 and 8 good, 5 and 6 average and passing, and 1 to 4 fail At the higher educational level it is either based on a "1 to 10" or "O to 100'' scale. In the latter case, 60 is the passing mark. Some private schools at the secondary level, and some universities use let grades like A, B, C and F.] With the exception of universities and foreign private schools, the kind of grading system which will be use in schools is decided by the MOE. The MOE may lower, as in the past, the passing mark in favor of students, or change the grading system i "l to 10", to "O to 100". The reason for such changes is mainly publ pressure. The school year in the Turkish educational system is divided ir two semesters. At the end of each semester the students at the prime and secondary levels are given grades for each of the courses they ha taken. The frequency of examinations is completely dependent a) upc the size of the class, b) the teaching load of the teacher (he must teach from 24 to 30 hours in a week) and c) the number of different 5 jects a teacher is teaching. On the other hand, every student, by regulations, should have a least three written and one or two oral examinations during each term.2 Oral examinations at the primary and secondary levels usually a used as make-up examinations for those students who were not good at written examinations. For example, if a student received a grade of 1The Educational System of Turkey, 0p. cit. 2M1111 Egitim Bakanligina Bagli Orta Dereceli Okullarin Sinif Gecme ve Imtihan deetmeligi, Milli Egitim Basinevi. Ankara, 1973, pp. 5-6. 30 on the written examination he most likely will not have any oral exami- nation. If, however, he received a grade of 1, 2, or 3, or 4 from the written examination, he most probably will be called for an oral examina- tion. Who will take an oral examination among those who receive lower grades (1 to 4 or to 5) is usually determined by the teacher either by his opening a page randomly from the "grade book“ which is kept by the teacher. The student who is called by the teacher either stands at the blackboard or at his seat while he tries to answer the question(s). He may be asked simple or difficult question(s). The difficulty level as well as the number of questions asked vary from one student to another. This practice causes many uneasy feelings among the students and parents. While one student may be given two chances for a make-up examination, another may not even get one chance, so his semester grade is determined on the basis of his low grade from the written examination. While one student is asked to answer two or three difficult questions, another one may be asked a simple and easy question--the former fails, the latter passes. It is obvious that there is serious threats to objectivity in such a practice. It would be unfair to blame the teachers. Thorndike and Hagen pointed out that: A grading system in an educational institution is a deeply in- grained part of the educational culture pattern. It is usually accepted automatically and with no more critical thought than our habits of holding a knife and fork. The new teacher is not systematically instructed in grading procedures but grows into them as a child grows into the regional pronunciation of 'water'. It seems unfortunate that our educational evaluations should be treated in such a casual fashion.1 1Thorndike and Hagen, op. cit., p. 487. 31 In addition to the over-crowded classes and heavy teaching load,1 teachers are not trained adequately in the techniques of measurement and evaluation. If they want to try some new techniques or use objec- tive tests, they are restrained from doing so by the regulations. Right after the second semester all classroom teachers at the secondary level meet under the chairmanship of the school director. The present state of each of the first and second grade students (i.e., VI and VII grades at the middle school level, IX and X grades at the lise) are discussed by the classroom teachers. If a student received 5.0 or more in all courses,he passes the class. Subject matter teachers, however, have the right to decide against any student whose first semes- ter grade was high, but second semester grade is very low, even though his grade average is 5.0 or more (i.e., 10 + 3 = 13, 13/2 = 6.5 - to 7.0). If the particular student is known as a good student in other subjects, the other teachers may try to persuade the particular teacher to pass the student. But this procedure may become a matter of bargain- ing: if a teacher may let a student pass the course, in return he may ask a favor from his colleagues to pass another student whom he knows. Sometimes well-to-do parents' influence, political influence or other types of influences may come into play, too. Classroom teachers have also the right to excuse a student's one, two or three failed courses by regulations, depending upon the student's achievement in other courses and such other factors as health problems, poor manners, etc. But in that case the teacher of the courses to be excused can use his veto 1Kazamias, op. cit., pp. 157-158. 32 power. If his answer is "no", there is nothing to do, except urge him to change his mind. By the rules, a "Turkish" course with an average grade less than 5.0 cannot be excused.1 A student whose over-all grade averages is 5 or more is passed "directly". A student having one, two, or three failed courses but who was excused by CTD (Classroom Teachers Decision) in his favor, is "passed by CTD“. On the other hand, if a student fails in up to three courses, he is allowed to take a written completion examination in August or September just before the new academic year starts. If he happens to fail again he must repeat all courses failed. After repeating the class, if he fails again in four or more courses after CMT (Class- room Teachers Meeting), he is dismissed directly and he is not given the opportunity to take completion examinations. If he fails in up to three courses he may take the completion examination, and he is dis- missed in case of failure in more than one. If he fails in only one course he may take completion examinations at the end of the successive grade level. A student who fails in more than three courses after the CTM is allowed to repeat the grade only once. The same evaluation system of students' learning outcomes is used for the third graders who are going to graduate but in a little differ- ent way. The third graders of middle school and of lise and its equiva- lent, vocational and technical schools, are also given two semesters grades the same as first and second graders. But their situation is not 1Milli Egitim Bakanligina Bagli Orta Dereceli Okullarin Sinif Gecme ve Intihan deetmeligi, op. cit., pp. 16-25. 33 discussed at CTM as to whether they passed or failed. Every third grader who receives first and second semester grades is entitled to take graduation examinations, regardless of his semester grades. By meeting the following two criteria he may take examinations from only three courses (Turkish, Mathematics, and Social Science or Science): 1) those students whose grade averages are all 5.0 or more (in case of having some courses with grade aVerage of 4, the second semester grade must be 7.0 or more from those courses) and 2) those students who may have one, two or three courses with grade average 4.0 but whose total grade average is 6.0 or more. The students whose grade averages do not fit the above two criteria and have several courses with grade averages less than 5 must take examinations from all those courses in addition to the three courses mentioned above. In order to graduate, those students who take examinations in only three courses must earn at least 5.0 from each of these courses. But the remaining students can graduate only if they receive a tptpl_average grade of 5.0 (second semester grade plp§_graduation examination grade, divided by two). If this average is less than 5, they fail in the course(s). For example: first semester grade = 2, second semester grade 4, grade average 2 + 4 = 6, 6/2 = 3; graduation examination grade for that course = 5; final grade (3 + 5)/2 = 4; so he fails. Those students who were not successful in Summer Graduation Exami- nations take examinations for failed courses in the Fall Graduation Examinations. Regardless of the prior grade averages, 5 on the examina- tion is an acceptable grade to pass the course. If they pass the courses 34 Figure 2. A summary diagram of evaluating of learning outcomes in Turkish Schools (1973-1974). Non-repeater Students (first and second graders) A -—9-GPA 3.5.0 ———>-pass directly GPA 3‘ 5.0 B-——+> but < 5.0 in -——>-may pass by CTD up to 3 courses C GPA < 5.0 in 4 or more courses ———9—fail directly D GPA 3 5.0 but < 5.0 in Turkish ———9-fail directly -——9-Comp1etion-——4> pass or Examination repeat GPA 3 5.0 but E ———>-< 5.0 in less than ———>~may fail by pass or 4 courses CTD in some ———>-Completion ———>—re eat courses Examination p Repeater Students (first and second graders) A -——>—GPA 3 5.0 ———> Pass GPA < 5.0 B -——> in 4 or more -——9-dismissed directly courses GPA < 5.0 in up to 3 -——9-Fall Completion Examination courses 1. Failure in more than one course ———>-dismissed directly 2. Failure in one course ———9—pass but responsible from the failed course 3. Pass -——9-continue next grade continued 35 Figure 2—-continued Non-repeater Students (Grade Average-GA-3’5.O, third graders) Summer Graduation Examination (SGE) If some course GA 2 4.0 SGE in Grades in Gradua— A ——-é> second semester grades_jn ———e> only 3 —-—+> all 3 -——+> tion and these courses must be > 7.0 courses courses diploma 3'5.0 Total GA'; 6.0 bpt GA in SGE in Grades in Gradua— B ———9—up to 3 courses < 4.0 -——>-on1y 3 ———>—all 3 ———9-tion and courses courses diploma 5.0 Do not meet A and B's SGE in 2nd semester grade + C -——> conditions. GA in 4 or ———>-all SGE grade = < 5 0 more courses < 5.0 courses 2 fail taken 1 I Y Fall Graduation Examination 1. GA in failed courses 3 5.0 ———> Graduation and diploma Not allowed to Wait until 2. Grades < 5.0 1n up to 3 courses ———> attend school ———> next SGE Repeat the entire 3. Grades < 5.0 in more than 3 courses -——9- third grade Repeater Students (third graders) SGE like non-repeaters Grades < 5.0 in Wait out of the A, B, or C more than 3 school until courses next SGE. D-——+> 36 which were subject to graduation examinations, they are graduated, and they get diplomas. Without this diploma no one can make the transition from middle school to lise, from lise to higher schools. If a non-repeater student of the third grade fails the Fall Gradu- ation Examinations in one, two or three courses, he is not allowed to attend the school until the following Summer Graduation Examinations. If he fails in more than three courses in the Fall Graduation Examina- tions, he must repeat the entire third grade. If a repeater student, on the other hand, fails in one or more courses he has to wait outside of the school until the next Summer Graduation Examinations (he may take examinations no more than three times).1 SECTION II MEASUREMENT AND EVALUATION IN TURKEY HistoryAOf the Measurement and Evaluation in Turkey The origins of educational measurement and evaluation in Turkey can be tracked back more than 500 years. Upon the conquest of Istanbul by the Turks in 1453, Mehmet the Conqueror ordered the opening of new schools in Istanbul. A particular school which was called Enderun Mektebi (Palace School) selected its students among non-Muslim children on the basis of certain criteria, such as good behavior, good appearance and physical condition, intelligence, character and facial beauty. '161 d. 37 Although that school selected mostly from non-Muslim children, later it became so important in the Ottoman hierarchy that children of very important persons were also accepted.1 Many of the highest ruling administrators and military leaders of the Ottoman Empire were educated in that school. From the beginning until the eighteenth century the selection criteria were kept rigorously; Ottoman Turks were among the first nations who "shunned birth", wealth and other aristocratic accoutre- ments, deliberately making education an important criterion for selec- tion, social advancement and occupational placement.2 The importance of this school, for the purpose of this study, is in the methods used to recruit and select cadre of prospective leaders of the Empire. Selection was done at different stages before they entered the school, and while they were in school by highly trained officials. Tutoring as a technique to promote their learning was used for almost every student. In fact, tutoring among Turks has been known for more than ten centuries; the Khans, Beys, Sultans and Emperors all had different tutors starting during the very early years. A guided type learning (a kind of programmed learning, as it were) had to be applied to every prospective official, to the children of well-to-do families, and also to the students of Enderun Mektebi. Students in this school were both students and pages in the service of the Sultan. As students they had to pass seven grades by receiving instruction from 1Faik Resit Unat, Turkiye Egitim Sisteminin Gelismesine Tarihi Bir Bakis, Milli Egitim Basimevi, Ankara, 1964, pp. 10-11. 2 Kazamias, op. cit., p. 25. 38 special teachers (tutors) in the necessary fields, like the arts of the courtier and of the administrator.1 Another educational institution which was marked for its advanced teaching and evaluating techniques to promote learning is the Medrese (College). These schools come after the Koranic primary school. As with the Enderun Mektebi, recruitment into the medrese, and promotion within it were based on merit rather than on family background. They provided a more advanced religious instruction than the Koranic schools. In addition to religious subjects, the course of study included other subjects like grammer, syntax, logic, metaphysics, rhetoric, geometry, arithmetic, and even medicine. The instruction was graded on the basis of written and oral evaluation, but also was individualized and each student could advance in accordance with his capability. In that sense, "individuality“ and merit of mastery learning have been long understood and practiced by the Ottoman Turks.2 But unfortunately, the meritocratic principles of selection, advancement, and occupational placement con- tinued only until the eighteenth century. At the beginning of the eighteenth century there were important changes in the patterns of recruitment and selection into major govern- mental institutions and schools, as well as changes in the power of the State. Education and achievement had lost their original importance as basic criteria of selection and advancement. Other factors such as 1Unat, op. cit., pp. 10—14. - 2A. E. Lyber, The Government of the Ottoman Empire in the Time of Suleiman the Magnificent (Cambridge. Mass.: Harvard University Press, 1913), p. 203. 39 wealth, bribery, and connections now entered the picture. Individuali instruction was given up, or used for only a few elite children; posi- tions and promotions were allocated on the basis of extra educational criteria. These changes in recruitment, selection and promotion patte contributed as major factors to the decline of the Ottoman institution and then of the Empire. Whereas, at the height of Ottoman glory, one of the necessary conditions for admission, placement, and promotion within the ranks of the rulers was an exacting and selective system of education. There was of course no equality of opportunity of educatio as it is understood today. After the second attempt to conquer Vienna in 1683, the Ottoman Turks made their first contacts with European countries in the beginni of an awakening to recognize their whole system. But the importance 0 the education was not adequately appreciated until the nineteenth cen- tury. In 1857, for the first time in the history of the Ottoman Empir a Ministry of Education was established. Different rules and regulati which were accepted at earlier times were modified and put together. French educational system was examined and several adaptations were ma In 1869 a very important event with regard to the Turkish educa— tional system, and to measurement and evaluation of educational attain ment, emerged from these previous attempts at modernizing. A comprehe sive regulation, which is still the cornerstone of today's Turkish educational system and of measurement and evaluation,was issued in 186 With this regulation (Regulation of General Public Education-—Maarif-i Unumiye Nizamnamesi) three school systems were adopted.1 The Sibyan Ok 'Unat, OE. cit., pp. 96-113. 40 (primary school) was for four years and compulsory, the Rustiye Okulu (middle school) was for four years and should be opened in towns from 500 to 1,000 households in size. Idadiye Okulu (senior high) should be operated in towns with over 1,000 households and the length of educa— tion in these schools was for three years. These two schools at the secondary level were free public schools providing general education. Sultaniye Okullari (academic high schools) however, were not free, and could be opened only in large cities and towns. These schools were mostly boarding institutions. The )ength of education could be either six years (including idadiye) or three years. The graduates of these schools, thus, had a l4-year education with an academic and some vocational-professional background. At the higher education level Darulmuallimin (Higher Teacher Train- ing), Darulfunun (University) and Higher Technical schools were to be opened. The duration of education in these institutions varied between two and four years. The Regulation of 1869 brought forward some important concepts and procedures concerning measurement and evaluation of educational attainment. Among the 198 paragraphs of the regulation,twenty-five were devoted to examinations, qualification for certification, and qualifica- tion for graduation diploma or qualification for apprenticeship. The regulation requires that grade examinations (promotion from one grade to the next upper grade), and school leaving examinations must be given in the presence of representatives of the community and education com- missions of the local governments. Examinations for the Sultaniye (lycée) school graduates were to be on three levels: (1) Examination in 41 literature, law and science must be passed in order to be accepted as an unpaid beginner in an official post (mulazemet imitihani); (2) examinations in the above three fields to pass to the paid lower level of an official post (mezuniyet), and; (3) examinations to be certified on the mastery of these fields. These examinations were written or given orally and a student had a right to take these examinations four times. Every school year a three-month period was devoted for examinations, generally between June and September. If a student could not pass the examinations in June,he could be given another examination three months later in September. If he could not pass the examination,he could repeat the same examination the following June and September. In case of failing twice in the grade promotion examinations,he had to repeat the same grade once more. At the higher educational level, the final examinations at each grade level were to be given before a three membered "examination com- mission“. Three colors (white, red and black) were used to grade a student's written and oral examination results. White represented excellence, red average success and black failing. If a student received three "white" grades from the three members of the commission,he was graded "excellent", and had all the priorities and privileges in apply- ing and getting an official post, he was considered an "honor" student; one white plus two red marks used to give him an average passing grade; three reds or two whites plus one black mark provided the student with a passing grade but no credits were given. The names of the students who passed the examinations used to be posted at different public places. 42 The last paragraph of the "examination section" is an interesting one. This paragraph requires that twenty days before the school year ends students at the upper level of secondary education were first to be screened on the basis of their achievement for a reward examination. Those who were selected for the reward examinations were tested by the school examination commission. Those examinees who were highly success- ful in these examinations were to be awarded either two silver or two bronze medals; these rewards were to be given directly by either the Minister of Education if in Instanbul or by the local Governor before the parents of the students, invited guests and government officials. Although many parts of this regulation have been changed since then, the examination system with some minor changes remained until today. Today's examination system, as discussed earlier in this study, is based upon this 1869 regulation, which was modeled after the French educational system.1 With minor changes this procedure is followed as closely as possible and is controlled by the Ministry of Education. Karagozoglu2 stated that it is the responsibility of secondary school supervisors that completion examinations and graduation examinations are conducted according to the regulations. For example, a supervisor or a group of supervisors should supervise whether the teachers' meeting of the school is held at the end of the school year properly or not; how examination questions are prepared and how they will be graded; how oral examinations are conducted, and so on. 'Naf1 Atuf, Turkiye Maarif Tarihi (Bir Deneme), Muallim Ahmet Halit Kitaphanesi; Milliyet Matbaasi, Istanbul: 1930. 2Karagozoglu, op. cit., pp. 93-95. 43 The scientific movement in Turkish education in terms of tests and measurement began with the translation of the Binet test in 1915,1 although it did not have a significant impact on the examination system. The uselessness of such a psychological measure, being merely a trans- lation, was soon recognized and in the 1930's a pragmatic and scien- tific interest in psychological and educational pUrposes began to gather momentum.2 Until the 1950's all attempts of psychological and educational measurement were led by university professors who had some education in countries abroad, such as France, Germany and England. The diffusion of such a movement was seen only in psychological clinics attached to the universities in Istanbul and Ankara, although some eleven books on testing, of which eight were translations from English, French and German, were published.3 The beginning of the second half of the twentieth century is a very important turning point in the history of measurement and evaluation in the Turkish educational system because the two most influential organizations, the Turkish Armed Forces and Ministry of Education, started to introduce the use of objective tests all over the country. Since Turkey has become a member of the NATO, the Turkish Armed Forces 1Hasan Tan, "Decelopment of Psychology and Mental Testing," in Mental Tests and Cultural Adaptation. L. J. Cronbach and P. J. D. Grenth (eds.), Mouton Publishers, The Hague, Netherlands, 1972, pp. 3-12. 21bid. 3161d. 44 have adopted and used almost all modern military testing techniques; and at the same time naturally the selection, placement and training of personnel were completely changed. It is this active involvement that started modern objective testing in Turkey. Many Turkish officials were trained in the United States and many were helped by military advisory committees, mostly from America. Although the author of this study was not able to locate any literature on the objective tests used by the Armed Forces, he has observed some testing activities in the Armed Forces and upon his induction to military service was exposed for the first time during his education, to an objective classification tests prepared (adapted) by the Armed Forces in 1953.1 During the same years another organization in the MOE was becoming actively involved in objective measurement and evaluation. The new organization, the Test and Research Bureau (TRS) was opened in a small room of the Gazi Educational Institute by the initiative of American- educated Turkish educators. Although some of the faculty of Isbantul University were selecting their students on the basis of objective examinations between 1951 and 1953,2 for the first time in the history of the Turkish educational system, objective tests began to be used all over the country. All essay type entrance examinations for Educa- tional Institutes and some higher educational institutions were abandoned 1Hasan Tan, op. cit., pp. 3-12. 2Sadrettin Celal Antel, ”Universite tercih yoklamasi ve neticelri,“ Pedagogi Bfilteni, Istanbul Universitesi Edebiyat Fakultesi, 1954, pp. 26-60. 45 and were replaced by objective tests.1 In 1957-1958 the Ankara University Faculty of Medicine, in 1959-1960 the Middle—East Technical University,2 in 1961-1962 the Faculty of Political Sciences, Faculty of Language and History and Geography, and the Academy of Social Services all started to use objective tests. These tests were either prepared, conducted and evaluated by the TRB, or by a joint committee composed of related 3 A11 faculties of Ankara faculty representatives and TRB personnel. University started to use objective tests prepared and administered by a Central Examination Commission in the 1962-1963 academic year. Istanbul University and other higher educational institutions, except Educational Institutes and Middle-East Technical University, started to use objective tests the same way in the 1964-1965 academic year. Entrance examinations for the Educational Institutes continued to be prepared and administered by the TRB, while the Middle-East Technical University prepared its own tests. Since its establishment in 1953, the TRB has prepared a number of achievement tests at the elementary and secondary school levels. It was also involved in developing some group aptitude tests based on American tests as models (Otis Quick Scoring, for example). The writer, as one of the researchers in the Bureau for five years, has not been able to locate any validity studies and standardization activities on the tests 1Ethem Ozguven, "Universite giris sinavlavinin ogrenci secimindeki rolu," Hacettepe Sosyal ve Beseri BilimlerDergisi, Cilt 3, Sayi 2, Ekim 1971, Hacettepe Universitesi Basimevi, Ankara. 2Tan, op. cit., pp. 3-12. 3Ozguven, op. cit. 46 that have been developed, although some item analyses have been done. Yet the TRB has been preparing the selection batteries and directing entrance examinations for various schools at all levels. The TRB has also helped to train a number of people from various organizations in test development and application and interpretation of tests and test results. The TRB has a large collection of tests, mostly American. For experimental purposes a considerable number of them have been used on various populations. Among these are the Otis Quick Scoring Mental Abilities, Kuder Preference Record-Vocational, MacQuarrie Mechanical Aptitude, and Differential Aptitude Tests. Although the TRB has been the leader in the testing movement (since 1970 it has merged into the Planning, Research and Coordination Bureau), there are also some other centers interested in testing and test development. For instance, the Pedagogy Institute of Istanbul University directed a standardization study of the Stanford-Binet on an experimental basis. Gazi Teachers' College developed the Gazi-Beier Test. Middle-East Technical University has an experimental adaptation of the 1960 revision of the Stanford—Binet and Wechsler Adult Intelligence Scale. Hacettepe and Istanbul Universities are working on the Wechsler Intelligence Scale for Children. Some original aptitude tests are pre- pared for university entrance examinations by the psychologists and educators_—o-.-_.—..—v— 96 aptitude and achievement tests are administered five times a year to those students who are seeking admission to college by ETS at centers in the United States and throughout the world; answer sheets are scored and reported to colleges designated by the candidates and to the candidates' secondary school, which may release scores to the students at its discretion. Admission tests for schools of special training, like the Architectural School Aptitude Test and the Test of English as a Foreign Language, are also administered by the ETS. If some high school students who wish to demonstrate their readiness for courses more advanced than those usually studied in the Freshman year (first year in a college), they are given Advanced Placement Examinations; College-level examinations can be given to evaluate the achievement of individuals seeking credit at the college level for studies pursued outside the college situation (i.e., through independent study and correspondence school). Testing at the graduate level and for the professions include testing programs like the Graduate Record Examination, Admission Tests For Graduate Study in Business, the Law School Admission Test, and the Medical College Admission Test. Tests for the first three programs are conducted by the ETS. Graduate Record Examination tests are com- posed of an aptitude test and advanced tests (Achievement) in twenty- one areas. Admission Tests for Graduate Study in Business and the Law School Admission Tests are largely aptitude tests, while the Medical College Admission Test is largely an achievement test. In these pro- grams, except the Medical College Admission Test Program, scores are also reported to the candidates. 97 The United States Government uses externally prepared, adminis- tered, and scored standardized aptitude and achievement tests. For example, for Peace Corps selection, a general aptitude test, the Modern Language Aptitude Test, and an achievement test are given to the candidates. To provide basic information for the selection of applicants to the Regular Naval Reserve Officers Training Corps, the Navy College Aptitude Test has been designed. These tests are admin- istered and scored by the ETS. Examples of testing programs for certification are the Actuarial Examination, which is prepared, scored and results are analyzed by the ETS, and the American Board of Obstetrics and Gynecology Examina- tion which is scored and test results analyzed by the ETS. Two major national testing programs should also be mentioned here. These are the National Assessment and Project Talent Programs. A number of testing agencies under the auspices of the Carnegie Corpora- tion are engaged in a program designed to evaluate educational attain- ments in the United States in nine fields: reading, writing, literature, music, art, citizenship, science, social studies, and mathematics. The purpose of the National Assessment is to gather data that will help to answer questions as to the effectiveness of education in the United States. Project Talent, on the other hand, is an attempt to identify the capabilities of American youth by providing substantial nationwide data on the aptitudes, achievements, backgrounds, and career plans of Secondary school students.1 ‘- 1Ebe1, op. cit. (1972), pp. 24-27. 97 The United States Government uses externally prepared, adminis- tered, and scored standardized aptitude and achievement tests. For example, for Peace Corps selection, a general aptitude test, the Modern Language Aptitude Test, and an achievement test are given to the candidates. To provide basic information for the selection of applicants to the Regular Naval Reserve Officers Training Corps, the Navy College Aptitude Test has been designed. These tests are admin- istered and scored by the ETS. Examples of testing programs for certification are the Actuarial Examination, which is prepared, scored and results are analyzed by the ETS, and the American Board of Obstetrics and Gynecology Examina- tion which is scored and test results analyzed by the ETS. Two major national testing programs should also be mentioned here. These are the National Assessment and Project Talent Programs. A number of testing agencies under the auspices of the Carnegie Corpora— tion are engaged in a program designed to evaluate educational attain- ments in the United States in nine fields: reading, writing, literature, music, art, citizenship, science, social studies, and mathematics. The purpose of the National Assessment is to gather data that will help to answer questions as to the effectiveness of education in the United States. Project Talent, on the other hand, is an attempt to identify the capabilities of American youth by providing substantial nationwide data on the aptitudes, achievements, backgrounds, and career plans of secondary school students.1 'Ebei, op. cit. (1972), pp. 24-27. 98 Local schools and school systems frequently use teacher-made1 and standardized tests. Teacher-made achievement tests serve the pur— poses of providing "information needed for instruction and guidance, evaluating local achievement against external standards, and stimulating and directing continuing efforts to improve curriculum and instruction in the local school."2 Standardized tests, on the other hand, are used for almost the same purposes as teacher-made tests. Their uses can be grouped into three major categories: for classroom use, for guidance, and for administration purposes. A testing program for classroom situ- ations is used to group students for instruction within a class (readi- ness tests in the first grade, achievement tests at later grades are appropriate), to guide the planning of activities for specific indi- vidual students (both scholastic aptitude and achievement tests are used), and to identify students who need special diagnostic study and remedial instruction (achievement and aptitude tests are used). Both standardized and teacher-made achievement tests are most widely used for guidance purposes. Their guidance functions are preparing evidence to guide discussions with parents about their children, building realis- tic self—pictures on the part of the students, helping the students to set educational and vocational goals, and improving counselor, teacher and parent understanding of problem cases. l 2 Ibid., pp. 534-536. Thorndike and Hagen, op. cit., pp. 422-429. 99 A survey conducted in 464 school districts in Michigan, enrolling students in grades K-12 yielded the following findings: 1) In elementary grades, the predominant uses of test results were diagnosis of learning difficulties, counseling parents, evaluation of curriculum, ability grouping, counseling students, and evaluating teaching effectiveness, with a frequency in order of mention. In secondary grades they were counseling students, diagnosis of learning difficulties, counseling parents, evaluation of curriculum, and ability groupings, in that order. 2) The most common methods of interpreting results of testing were conferences with individual students and individual parents, and summaries for administrators and teachers. 3) In the elementary grades, tests were administered once each year, in the fall or spring, by the classroom teacher in the classroom to all students. In the secondary grades, tests were generally administered once a year by the guidance counselor to nearly all students. 4) At both elementary and secondary levels, national norms were used in more than ninety percent of the districts. Local norms were also used by approximately twenty-five percent. 5) For in-service education of teachers and other personnel to use test results, building faculty meetings, departmental meetings, grade and divisional faculty meetings were the methods used. 6) Most commonly given tests were a) reading readiness (kinder— garten or first grade, once a year), b) group intelligence tests (twice in elementary grades, once in secondary schools), c) achievement test battery (twice in elementary grades, once or twice in secondary grades), d) scholastic aptitude test (once in secondary grades), and e) interest inventory (once in secondary grades). Local or school testing programs are locally initiated and di- rected. It is always undertaken with the cooperation and responsibility 1Wm. L. Schmalgemeir and R. P. Watson, Michigan Schools: The Organization and Management of Their Testing Programs--1970 (Ann Arbor, Michigan: Michigan Bureau of School Services, The University of Michigan, 1970). 100 of more than one person. It may involve only classroom teachers and their supervisors or principals, or the work may be planned and carried out with the cooperation of the entire staff of a school or school system. However, the task of selecting tests may be delegated to a committee. External testing programs are often under the direction of a private educational agency, such as the CEEB, the American College Testing Program, or the Law School Admission Test Council. These organizations are often composed of member schools and colleges; these agencies frequently contract with an independent testing agency for test development and test administration activities. The ETS, for example, performs specific services in the construction and administra— tion of College Board examinations. The steps in a testing program are usually as follows: 1) The sponsor establishes the program. An educational agency decides to sponsor a testing program for a particular purpose; a commit- tee is formed to survey methods of organizing and conducting the testing program; a contract is made with an appropriate testing agency for specific purposes. 2) The testing agency develops the program; it is pretested; testing centers are established for the actual test administration. 3) The candidates register for the examinations, and registration forms are returned to the testing agency with established test fees. 4) The testing agency processes the registration forms, a ticket of admission to a test center is prepared for each candidate and is mailed to the candidate. The test materials are also sent to each center. 101 5) The supervisor administers the tests, and all test materials, including answer sheets and test booklets, are returned to the responsi- ble testing agency. 6) The testing agency processes and reports the results. 7) Schools and colleges use the results. Then a follow-up study is conducted to determine the effectiveness of the tests in a given situation. The Use of Objective Tests in Industry The use of objective tests in American industry can be said to have started with American psychologists concentrating most heavily on problems of personnel selection and placement during World War I. In 1921, the Psychological Corporation was founded, among other purposes, to develop and distribute psychological tests to industrial and other organizations. During World War II, the extensive requirements of the military for improved personnel assessment and training procedures added further impetus to the use of objective tests in industry and other organizations.1 Objective tests are mostly used in personnel selection, placement, training and evaluation, and promotion. These tests are mostly psycho- logical rather than educational achievement tests. 1Laurence Siegel, Industrial Psychology (Homewood, 111.: Richard D. Irwin, Inc., 1969), pp. 12-15. 102 As Haire1 pointed out, use of psychological tests in personnel selection rests upon two basic assumptions: First, any given human ability follows the lines of a normal distribution. With a known probability there are a few people who are very low or very high on a particular ability, and the rest of them are distributed around the middle. Second, it is possible to construct psychological selection tests that are associated with the ability in question. Then it is possible to select men with the desired talents for a given job or posi- tion by assuming that a high or a low score on the test will be associ- ated with the presence of a high level of ability in question. If one can know and state explicitly what kind of ability is needed for a given job, and develop a test to measure such an ability he can (a) identify the people who are in the high part of the distribution for that abil- ity, and (b) select that segment of the population. One can markedly improve his labor pool through such a procedure. Implicitly or explicitly, individual differences exist, and it is important that these individual differences are recognized and measured. Then personnel testing has one specific objective: to contribute to the increasingly effective use of manpower within an organization by choosing from a number of available applicants a smaller number to be hired for a given job.2 The first step in personnel testing is to specify the dimensions to be predicted, since every personnel decision is an implied prediction 1Mason Haire, "Use of tests in employee selection," Harvard Business Review (January, 1950), pp. 42-51. 2Robert M. Guion, Personnel Testing(New York: McGraw-Hill Book Company, 1965), pp. 3-10. 103 of on-the-job behavior. Most widely used instruments in prediction are: (1) tests of general intelligence. Wechsler Adult Intelligence Scale, Otis Self-Administering Tests of Mental Ability, The Wonderlic Personnel Test, Adaptability Test, Miller Analogies Test, and Thurstone Test of Mental Alertness are examples of this group. (2) The second group of tests are aptitude tests. These tests intend to measure more specific constructs than "general" intelligence, although, with a few exceptions, they measure intellectual abilities. The Minnesota Clerical Test, Psychological Corporation General Clerical Test, and The Short Employment Tests are examples of tests to measure clerical aptitude. Differential Aptitude tests, Flanagan Aptitude Classification Tests, and General Aptitude Test Battery are the most widely used multiaptitude tests. There are many other standardized objective tests published and sold by more than 300 agencies in the United States. Achievement tests are largely developed for training purposes in industry by personnel specialists and industrial psychologists. Many organizations employ part-time psychologists, and some employ full-time psychologists, to assist them in selection, placement, train- ing and other organizational problems. As is the case for standardized educational tests, tests for industrial purposes are selected according to special purposes by qualified personnel. As is discussed in the next chapter, test results are also used to promote learning in training situations as well as selection, placement, guidance, and promotion. 104 Summar The system of education in the United States is decentralized. Every state mantains a system of free public education through the twelfth grade. Public schools are operated and controlled by local governments. School attendance is compulsory until a certain age (which varies from 16-18 from state to state). Besides public educa- tion, private educational institutions constitute a significant and vital part of American education. Elementary education generally consists of education for all children ages six through eleven in grades one through six. The cur- riculum usually planned locally. Schools attempt to individualize progress from the beginning through the elementary years. Guidance services are provided. ‘ Secondary education consists of education 14 through 17 years of age, and nearly all of the elementary school graduates enter secondary schools. A high school may offer an academic, technical or vocational curriculum or be organized as a comprehensive high school. Vocational education is an integral part of the total education program in the American high schools. It is both the responsibility of the federal government and the states. Higher education includes those educational programs which require for admission the completion of approximately 12 years of previous schooling. The institutions of higher education are various; such as, Junior colleges, community colleges, universities, institutes of tech- nology. and technical institutes. They are essentially academically 105 autonomous, and admission is open, as compared with most other countries. Achievement of American students is measured frequently by objec- tive tests which are highly developed and published by the test industry. Special Acts provide federal funds for the establishment of programs of testing, counseling, and guidance, especially in secondary schools. These Acts also provide funds for a variety of innovative and experimental programs. Objective standardized tests are in effect integral parts of the American education. CHAPTER IV THE USE OF OBJECTIVE TESTS TO PROMOTE LEARNING IN THE UNITED STATES In chapters II and III the history and use of objective tests in the United States and Turkish educational systems were discussed. In this chapter, how objective tests may be used in promoting learning is described and discussed in light of the literature on the use of objective tests. The studies reported here fall into three broad cate- gories: a) studies of feedback and class discussion of test results, b) studies on open—book or take-home examinations, and c) studies on coaching and retesting. Studies of Feedback and Class Discussion Since an important goal of any classroom instruction is to produce individuals with a maximum amount of correct and usable information and a minimum amount of incorrect and conflicting information, a major prob- leniappears to be the fixation of correct, and the elimination of er- lflbneous ideas. One means to these ends involves letting the student know what mistakes he has made in an examination; that is, giving him feedback. One of the functions (perhaps one of the most important functions) that evaluation serves is to enable students to determine how well they 106 107 achieving. Studies cited in the following pages demonstrate that when students are aware of their learning progress, their performance will be superior to what it would have been without such knowledge. McKeachie summarized studies related to feedback from examina- tions as follows: While we usually think of testing procedures in terms of their validity as measures of student achievement, their function as instruments for promoting learning may be even more important. After dismal recitals of non-significant differences between differing teaching methods, it is refreshing to find positive procedures. Curtis and Wood2 in their experimental study in the University High School, University of Michigan, during four consecutive school years (1924-1928), compared four practices in correcting examination papers. Their purpose was to determine which of the methods served best with respect to student learning and student retention of the subject matter. Under Method 1 students checked the incorrect responses on their own papers as the teacher gave the correct responses. Free discussion was allowed. Under Method 2 the teacher collected test papers and checked the incorrect items as wrong but made no corrections. The papers were later returned to the students and discussed item by item. Under 1W. J. McKeachie, "Research on teaching at the college and univer- sity level," in Gage (ed.) Handbook of Research on Teaching, 1963, AERA, p. 1154. 2F. 0. Curtis and G. 6. Wood, "A study of the relative teaching value of four common practices in correcting examination papers," §ghool Review, 37: 615-623, 1929. 108 Method 3 the teacher collected the papers and wrote in all the correc- tions. The papers were returned to the students and discussed item by item. Under Method 4 the teacher collected the papers and wrote in all the corrections, as under Method 3, but when the papers were returned to the students discussion was limited to such questions as arose in response to the teacher's direction: "I have carefully corrected all the errors on your papers. Note your errors, and ask any questions you wish to in connection with them." There was no limit on the discussion and students were encouraged to ask questions about their errors. The study was conducted in ten science classes, of which two were in the seventh grade, three in the eighth grade, two in the ninth grade, two in the tenth grade and one in the eleventh grade, altogether enroll- ing 286 students. Four parallel objective tests for each class were prepared and used as the midterm examination. The test booklets were clipped together so that one-fourth of the students began with sheet 1 and finished with sheet 4; another one-fourth began with sheet 2 and finished with sheet 1, and so forth. Each of these four sheets was treated according to the four methods described above. During the first meeting period after the examination, the four sheets were distributed to the students. After each set of sheets was treated it was collected. Each student received his own sheet 1 on which he checked the errors as the correct responses were read and dis- cussed. Sheets 2, 3, and 4 were returned to their owners for discussion. Sheets 2 and 3 were discussed item by item, but only those items of 109 sheet 4 were discussed about which the students asked questions. The same test sheets were re-administered, as a test of immediate recall, during the next class period without previous warning. Six weeks later they were re-administered a third time again without previous warning, as a test of delayed retention. In the test of immediate recall, Method 1 was slightly superior to Methods 2, 3 and 4. The results of the delayed retention study also showed that Method 1 was superior to Method 2, somewhat superior to Method 3 and consistently superior to Method 4 in promoting learning and retention. There seemed to be little choice between Methods 2 and 3. Both Method 2 and Method 3 were found to be superior to Method 4. Under Method 1, students were able to correct their own examination papers with a high degree of accuracy. In a similar but much more recent study to investigate the ques- tions, "Do teacher comments cause a significant improvement in student performance," and “If comments have an effect, which comments have more than the others, and what are the conditions, in students and classes, conducive to such an effect?"1 It was shown that brief written comments upon returned objective examinations caused a significant improvement in subsequent objective examination performance of students. Seventy-four randomly selected secondary school teachers in 12 randomly selected schools in the city of San Diego, along with 2,139 students from their daily classes in 6 grades (grades 7-12) who were unaware of the nature 1Ellis B. Page, "Teacher comments and student performance: a seventy-four classroom experiment in school motivation," Journal of Educational Psychology, 49: 173-181, 1958. 110 of the experiment, performed the experiment. The teachers administered to all students whatever objective test would occur in the usual course of instruction. After scoring and grading (A, B, C, D, F) the test papers in their customary way, and matching the students by performance, they randomly assigned the papers to one of the three treatment groups (No Comment, Free Comment, and Specified Comment groups). The "No Comment" group received no marks beyond those for grading. The "Free Comment" group received whatever comments the teacher felt were appro- priate for the particular students and tests concerned. Teachers were instructed: "Write anything that occurs to you in the circumstances; There is not any 'right' or 'wrong' comment for this study. A comment is 'right' for the study if it conforms with your own feelings and practices." The "Specified Comment" group received certain uniform com- ments designated beforehand by the experimenter for all similar letter grades, regardless of teacher or student differences as follows: Excellent! Keep it up. Good work. Keep at it. Perhaps try to do still better? Let's bring this up. Let's raise this grade! ”10057) Then teachers returned tests to students rapidly and automatically, try- ing not even to notice who the students were. The next normally scheduled objective test in the class became post-test (criterion). The results of the study can be summarized as follows: la) "Free Comment" students achieved higher scores than "Specified Comment“ students. 111 lb) ”Specified Comment" students did better than "No Comment" students. All differences were significant except that between Free Comments and Specified Comments. 2) When teachers were asked to predict the effect the comments would have on student performance, most of them had said that the better students would be more responsive than the poorer ones. The results showed, however, that good and poor students alike responded favorably to the comments. This is an interesting finding to show that poor students as well as good students can benefit from such an exami- nation technique. 3) When the class-groups from six different grades (7-12) were compared, no conclusive differences in the effect of the comments appeared between the grades (almost no evidence of interaction). The researcher commented that: When the average secondary teacher takes the time and trouble to write comments (believed to be "encouraging") on student papers, these apparently have measureable and potent effect upon student effort, or attention, or attitude, or whatever it is which causes learning to improve. One of the major elements in learning which objective tests pro- vide is the knowledge of results, and one would expect that the more information contained in the feedback, the greater its value. In a similar experiment, but in a different setting, in the Air Force,] performances benefited from return of multiple-choice tests together with information about why the alternative choice was wrong and why the 1G. R. Stone, "The training function of examinations: retest Performance as a function of critique information." USAF Personnel and ILaining Research Center, 1955, No. AFPTRC-TN-55-8. 112 correct alternative was right. The purpose of that experiment was to study the influence of the amount and kind of information received by examinees in individual critique sessions (such influence was measured by immediate and delayed retests on the same examinees). 0n the basis of a regularly scheduled objective examination, six successive classes of about 40 cadets in training for ratings as B-26 aircraft observers were divided into five subgroups for purposes of individual examination critique. Five readers handled about eight students individually by discussing examination results with them in a relatively formal manner. The readers were instructed not to attempt answers to any student's questions except by re-reading that portion of the material which pertained to the question. For the five subgroups the conditions were as follows: Condition (1): Students were informed of their total score only. Condition (2): For the items each student missed, the question and the alternative which he had chosen were read to him. No informa- tion with respect to the other alternatives was given. Condition (3): The same as condition (2) except that in addition to reading the incorrect alternative prepared material was also read which explained to the student why this choice was incorrect. This explanation did not include information concerning the correct answer. Condition (4): For each item missed the student was read the question, and the correct answer was read and explained. No mention or no reference to the incorrect alternatives was made. Condition (5): For each item missed the student was read the question, his response and why it was wrong; and the correct response 113 and why it was correct. Each of these five subgroups was, thereafter, divided into two matched groups for purposes of retest, one-half within 24 hours and the other half thirty days later. The results in summary were: 1) On the immediate (24 hours) retest, the Conditions (3), (4) and (5) all produced significant improvement in performance as compared to the Condition (1), control group. Condition (2) also showed improve- ment but it was not significant. 2) The improvement demonstrated by Conditions (4) and (5) was significantly greater than that demonstrated by Conditions (2) and (3). Neither difference within each pair, however, was significant. 3) On the 30-day retest, only Condition (5) demonstrated improved performance over the original test. 4) For the group retested at one day and again retested at thirty days, Conditions (4) and (5) both remained significantly better than the original test. On the basis of these results it was concluded that (a) there are varieties of comments which produce significant benefits in students' responses as much as 30 days following the examination, and this is a very strong sanction for properly conducted comment sessions; (b) there are varieties of comments which do not produce benefits in students performance at 30 days which are significantly greater than performance resulting from mere knowledge of test scores; and (c) as students are very prone to ask why their chosen answers are wrong rather than why 114 the correct answers are correct, some care is necessary to avoid lapsing into negative explanation. In all learning situations the learners are expected to learn whatever is taught them. If the learners are to learn skills, they have to practice the skills; but practice alone does not make perfect. Practice works only if the learner sees the results of his practice. The studies discussed so far clearly demonstrate that if students are given feedback they perform better, and the procedure promotes learning. Another study, similar to these studies, by Sassenrath and Garverick1 showed that students retained material covered in quizzes best when they received feedback in terms of the instructor's discussing the correct answers with them. This method was more effective than having students look up the answers in the book or having them check their replies against a list of answers written on the blackboard. However, all three of these methods were better than no feedback at all. The study was conducted to see if there was an increase in scores on a retention section and a transfer section of a final examination when the amount of information from feedback on midterm examinations was increased. In terms of increasing amounts of information (a) one group received no feedback, (b) a second group checked over their answers from correct ones placed on a blackboard, (c) a third group had the questions discussed by the instructor, and (d) a fourth group re-read material for questions they answered incorrectly as well as correctly. 1J. M. Sassenrath and C. M. Garverick, "Effects of differential feedback from examination on retention and transfer," Journal of Educa- tional Psychology. 56: 1965, pp. 259-263. 115 Four-hundred eighty-seven students from classes in introductory psychology participated in the experiment. The subjects were randomly assigned to three experimental and one control (no feedback) groups. On a pre-test early in the semester, the four treatment groups did not differ in initial ability level. About 120 students in each of four groups received one of the feedback treatments on three mid-term examinations. Students had three objective mid-term and one objective final test. The dependent variables for the experiment consisted of the 45-item "retention" and the 30-item "transfer" parts of the exami- nation that were common to the four groups. Analyses of the results in general showed that on both the reten- tion and transfer tests the group which had the questions discussed in class and the group which checked their answers from the correct ones on the blackboard were better than the control (no feedback) group and the group that reread the material on items they got incorrect on the three mid-term examinations. Sassenrath and Garverick recommended, as a result of their study, that teachers should discuss examination ques- tions with students or let students check over their examinations; the type of feedback is less important than the fact that the students get it. Little1 in a well-controlled investigation reported that maximum learning results from testing when students are permitted to score their own papers and when discussion of errors and remedial work follow imme- diately. The study was conducted in fourteen sections of a course in ]J. K. Little, "Results of use of machines for testing and for drill upon learning in educational psychology," in Lumsdaine and Glasser (eds.) Teaching Machines and Programmed Learning, NEA, 1960. 116 educational psychology averaging thirty students each. Each quarter, two sections of the course were run as "test-machine groups," two as "drill-machine groups,“ and the remaining three sections as control groups. Thus, there were four test sections, four drill sections, and six control sections. All sections took pre-test, mid-term, and finals. In the test machine sections each unit test was scored by the scoring and tabulat- ing machine; as each student finished punching his answer slip, he brought it to the instructor, who at once scored it, checked wrong answers, tabulated the score on a tabulation form already on the board, and returned the slip to the student. When all had finished, the test was discussed. Each student thus at once knew his score. Students were also required to take a make-up test if the first test scored below 8. In the drill sections, the same general procedure was used. But the tests were taken using the drill machines which automatically made corrections of errors and provided the opportunity for immediate progress to mastery of that test. The control sections took the chapter-unit tests by marking on an answer slip which was graded by the instructor that evening and returned ‘ the next day. There were no make-up tests. The results showed that students who immediately were appraised of their test results, and were given opportunity to correct deficiencies by make-up tests in both experimental groups, profited markedly in terms of final examination results over students who did not have such 117 advantage. The mean achievement of the drill-machine group was approxi- mately twice that of the machine scored group. Studies described so far clearly demonstrate that objective tests can be used effectively to promote learning if the user is aware of their use, and if the user is seriously interested in promoting learning. Studies on Open-Book Examinations Open-book examination is a type of examination ”in which the examinees are permitted to bring and use textbooks, references, and "1 class notes. The intention of the instructor by giving an open-book examination is to emphasize command of knowledge, as distinguished from recall of factual information. It was suggested as a reaction to the conventional examination technique. Tussing criticized the conventional examination technique in the following terms: ’ There are always people in the class who are considered the "brains" who get high grades on such an examination and yet are very impractical in the use of such information. Although this is not always true, nevertheless, such an examination does place a premium on the person who reads the book very carefully and memorizes line after line or who can repeat back the instructor's notes in a somewhat verbatim manner. This type of examination obviously does not test the end product of the course, which should be something other than factual details.... The cram- unload- and forget method is the one most commonly used by stu- dents during examination period.... However, the student who has the feeling of a high degree of responsibility and does not feel adequate to the arduous memory task in a short perigd of time, has what is commonly termed a nervous breakdown.... 'Ebei, op. cit. (1972), p. 119. 2L. Tussing, "A consideration of the open book examination," Educational and Psychological Measurement, XI (1951), 597-602. 118 In addition, Tussing gave as reasons for attempting such an examination technique that open-book examinations (a) tend to promote a course content that emphasizes application rather than a memory of specifics; (b) it places emphasis on questions dealing with reasoning rather than rote memory; (c) it would eliminate or reduce cramming; (d) lessen, if not eliminate, emotional strain during the final exami- nation period; and (e) closely resemble a life situation where the seeker of knowledge has some facts memorized, but has the opportunity of checking sources before making a report or statement. Kalish1 attempted to determine the equivalence of two approaches to the administration of examinations; namely, closed-book versus open- book. He tested three hypotheses that (a) the open-book examination will lead to fewer student errors; (b) the open-book examination will measure different abilities than those assessed by the closed-book tests; and (c) there is no correlation between student ratings of the help received from open-book examinations and their test scores. It appeared from the results that the opportunity to use text and lecture materials resulted in no differences in total errors, but the two types of examinations measured significantly different abilities. Kalish suggested that if an instructor feels that the open-book examination is a more valid measure due to the decrease in reliance on memory and detraction from cheating, the open-book approach would be the most appropriate. 1R. A. Kalish, "An experimental evaluation of the open-book exami- nation, Journal of Educational Psychology, 49: 200-204, 1958. —!—_—_w --—-- 4 —. — 119 In an evaluation of students' reaction to open-book examinations by Feldhusen1 both objective and essay type examinations were used in the open-book situation, in a junior level educational psychology course at the University of Wisconsin. After a series of examinations, all of the open-book type, he found that on the objective type two-thirds of the students (60 out of 90) reported reduced tension. With the essay type it was only slightly less (60 percent). The only other effect of open-books on objective and the essay type concerned the memorizing of factual material. For the objective type 79 percent felt it reduced memorization. On the essay type 61 percent said there was a reduction. Feldhusen summarized his students' other reactions to the open-book examinations as follows: 1) They believe open-book examinations reduce the tendency to cheat. 2) They believe students learn more during open than during closed book examinations. 3) They preferred open-book examinations. 4) They intended to try open-book examinations when they became teachers. 5) They had seldom or never taken an open-book examination before. These studies support, to some extent, the rationale of this study that an examination technique should allow for and provide the 1J. F. Feldhusen, "An evaluation of college students' reactions to open-book examinations," Educational and Psychological Measurement, 21: 637-646, 1961. 120 opportunity to promote student learning. This approach reduces both tension and memorization and promotes learning. A similar examination technique, the take-home test, has also been suggested by Ebel.1 The take-home test is similar to the open-book examination with an additional advantage and an additional disadvantage. Its advantage is the removal of the pressure of time; its disadvantage is the loss of assurance that the answers a student submits represent exclusively his own achievements. But it also has its limitation in that an instructor should have a large item pool, and he should not use the items used once again for a long time. The procedure in a take-home test in college teaching is as follows: 1) An objective achievement test is administered under supervision of the instructor as an in-class closed book test and immediately after- wards as a take-home test. Students are given two answer sheets, one to be turned in at the conclusion of the test period, the other at the next class meeting with the test booklets. Students are encouraged to discuss the test items with their friends, referring to the textbook, notes and references-~except asking the instructor. They return, the second time, more correctly checked answer sheets. Their scores on in- class and take-home re-test are combined and a single grade is given. Ebel commented that: With an ideal test, and ideal student, everyone might get a per- fect score on the take-home test. But with real tests and real students the scores are different. Few if any of them are 1R. L. Ebel, "Using tests to improve instruction," in K. Ingenkamp (ed.) Developments in Educational Testing, Vol. 2, University of London Press Ltd., London, 1968. . 121 perfect. Of course, almost all the students get substantially higher scores on the take-home test than they did on the in- class test. The greatest gains, naturally tend to be made by the students who had the lowest scores in class, but even here there are wide differences. Some come close to perfect scores. Others improve only a little. Some have time, and choose to spend much of it in the pursuit of perfection. Others have less time, or choose to spend it in other ways. The author of this study undertook an experimental study to in- vestigate the effectiveness of such a method of students' achievement at Michigan State University.2 The preliminary analysis of the study showed that those students who took the mid-term in-class test twice, one as an in-class test, one as a take-home re-test, retained more information on the final examination than those who did not take the test as take-home re-test when the same items were re-administered. Most of the students reported that this procedure was effective with regard to promoting learning. In light of these studies on open-book and take-home examinations it is reasonable to conclude that these techniques have the following advantages: 1) They stress the incentive to study both before the examination and after the examination. This method eliminates cramming to a large extent; 2) They direct the efforts of students toward the attainment of essential achievements. By taking home the test, they have the oppor- tunity to use any relevant material. One may have thousands of books 1Ibid. 2Osman Kazanci, Take Home Re-test as an Examination Technique, unpublished manuscript, Michigan State University, 1972. This manuscript will be prepared subsequently for publication separately. 122 but what use if he does not know when and how to use them! 3) Instructional experience is enhanced since the questions are discussed after the examination has been marked. From the point of view of the psychology of learning, students become more involved and active in the process by finding out how they have done on the in-class examination through discussing it with friends and by looking for the answers to the questions at hand. 4) Since every student works on the test at his own pace, and at the most convenient time for him, time pressure is reduced, test tension is reduced, and the process of testing is individualized. 5) Since the ultimate goal of any classroom instruction is to produce individuals with a maximum of correct and usable information this procedure seems to work toward this goal. Coaching,Studies Since educational as well as industrial organizations frequently select candidates on the basis of objective tests, the practice of coaching to prepare students to take examinations has developed. Coach- ing is a part of the pattern often attached in the frequently voiced criticisms of standardized testing today, that standardized testing results in excessive efforts on the part of teachers to prepare their students for tests, especially those given in connection with some external testing programs like the College Entrance Examination Board. The result, it is often claimed, is that teachers emphasize only those things covered by the test to the detriment of other important aspects 123 of their subjects or, which is even more important, they are afraid to introduce innovations in teaching techniques that might result in their students doing poorly on standardized tests. However, studies of coach- ing reviewed here generally indicate that coaching has a positive effect on subsequent test performance and that it promotes learning. French and Dear1 reported research findings on studies of coaching for the College Board Scholastic Aptitude Test (SAT). In that study, two large and very similar private secondary schools took part. The two schools were matched in size, and the general nature of the instruc- tion and the kinds of courses taken by the students were very similar. Both of the schools gave a past form of the SAT to their seniors at the beginning of the school year in September for equating purposes. During the fall one school gave its senior students weekly coaching sessions based on specially prepared exercises consisting of test items exactly like those on the SAT: mathematics problems, verbal analogies, comple- tion items, anonyms, etc. Coaching for SAT-V (Verbal section of the SAT) handled by English teachers, was based on a series of 12 practice exercises. Coaching by mathematics teachers for SAT-M (Mathematical section of the SAT) followed a similar pattern. General instructions for taking multiple choice tests were reviewed, and techniques for most efficiently solving each kind of item were practiced. All students in the coached school and in the uncoached school took the regular SAT for college entrance in March, six months later. ]J. W. French and R. E. Dear, "Effect of boaching on an aptitude test," Educational and Psychological Measurement, 19: 319-330, 1959. 124 After corrections had been made by means of analysis of co-variance for differences found at the beginning of the first testing in September, still some differences were found in average test scores of the two schools, in favor of the coached school. The coached students did better on the verbal test, and much better on the mathematical test than the uncoached students. If mathematics is considered generally a more difficult subject than verbal materials for high school students, the effect of coaching in promoting learning can be more clearly recognized. As reported by French and Dear, the same experiment was repeated on public school students. This time three public schools participated. In one school in which during a typical year about 25 percent of the students went to college, the seniors were coached on verbal and mathe— matical materials similar to those in the SAT. In the second school coaching was done only in the verbal area. In the third school, no coaching took place. The students took a past SAT in September and the regular one in March, six months later. This study was different from the first one in that there was more intensive coaching. There was individual coaching, two sessions a week with much homework. A similar significanct effect was again found. These studies carried out by the College Board and Educational Testing Service between 1951 and 1965 were followed by another study which also indicated that coaching promoted learning. Jacobs1 investi- gated the effects of coaching for the College Board English Composition 1P. I. Jacobs, “Effects of Coaching on the College Board English Composition Test," Educational and Psychological Measurement, XXVI: 55-67, 1966. 125 Test (ECT). Two hundred sixty-six juniors from six high schools volun- teered. In each school students were randomly assigned to the two groups, control (no coaching), and an experimental group. In each coached--and--control school an English teacher served as the coach. Each coach was presented with coaching material for two item types, and was told that these were item types that had appeared on the ECT in the past and were likely to occur again. The coaching was given three hours a week for six weeks. After the sixth week of coaching, all stu- dents in the study took the ETC. In the spring following the first ETC, the students in the coached--and--control schools were again given the ETC. The results of the study were as follows: In two of the six coached--and--control schools there was essential- ly no difference between coached--and--control groups. In each of the other four schools there was a statistically significant difference favoring the coached groups. Further analysis showed that the coached group achieved its superiority by answering fewer items incorrectly. The study in general showed that the students who receive coaching have an advantage with respect to increasing their learning over stu- dents who did not. Summar The review of the studies on the use of objective tests to promote learning presented in this chapter indicates that in many learning situa- tions the benefits which the learner may get through the use of objective 126 tests are determined by different test usages. If objective tests are to be used with the intention of promoting learning in any learning or instructional situations the test user should take all necessary measures and steps. First of all, tests must meet the necessary requirements of a good test; that the items in the test must be stated unambiguously; that the test must be as reliable as possible, and valid. Second, a teacher must have a good item pool for use. Third, the teacher should plan in advance how much time will be devoted to small or large group discussions as well as to individual feedback to the students, and the type of feedback. Finally, the teacher should keep in mind that the use of tests to promote learning is an indirect contribution, not a direct one. The examinations, on the other hand, are one of the best oppor- tunities for good students to demonstrate their abilities and studies; they are one of the best means of progress in the eyes of many instruc- tors and students. The studies cited in this chapter, then, indicate that there are many possible ways of bringing examinations to a happy, useful and wished state in schools. The very first thing that must be decided before everything is that "What are the things instructors are looking for in their teaching?" "Are they looking for catching students when they are not ready to punish them? ..." 0r "Are they looking for the all best possible ways of teaching their students whatever they are supposed to teach?" Then, the tests show themselves where and how they must be used. CHAPTER V COMPARISON, IMPLICATIONS, SUMMARY AND CONCLUSIONS This chapter describes the possibilities of developing a better test and measurement technique for the Turkish educational system and industry in general. For this reason a comparison on the use of objec- tive tests in the United States and Turkish educational systems in terms of measuring students' learning outcomes and promoting learning is made. The chapter also includes summary and conclusions. In the first section of the chapter a comparison is made by point- ing out similarities and differences as found in the use of objective tests in the two countries' educational systems and in industries in general. Then, underlying reasons for similarities and differences are discussed. The second section of the chapter shows the possibilities of developing a better objective test and measurement technique for Turkish educational system and industry in general as implied by the study. Summary and conclusions of the study are also included in this section. 127 128 SECTION I COMPARISON ON THE USE OF OBJECTIVE TESTS IN THE UNITED STATES AND TURKISH EDUCATIONAL SYSTEMS, AND IN INDUSTRIES IN GENERAL A summary table on the use of objective tests in the two countries' educational systems and industries is provided to give the readers a quick grasp of the findings. This table (Table 3) has been prepared on the basis of the material presented in earlier chapters, and cumulative experiences and observa- tions of the author in the United States and Turkey. It summarizes use of objective tests in the United States and Turkish educational systems and industries in general. The numbers in the table (1, 2, 3 and 4) represent elementary, secondary, and higher education and industry, respectively. Objective tests are not used for either formative evaantion or research purposes in the Turkish educational system, whereas these teacher- made tests are frequently used in the United States educational system. It is clear that there are a few similarities between the two countries in the use of teacher-made tests for formative evaluation and research purposes. Use of teacher-made tests for coaching purposes seems an important aspect of the Turkish educational system. The reason for that is the selection of students to state boarding schools on the basis of objective test results. Classroom teachers feel a responsibility for preparing their students for the successive levels of education. While these tests .HHH eee HH meoeeeeu poms cm>mc amoep< u a Acpmavcw :H u e u apmcmm u u Fm>mp Pocomumoanm Loewe: u m . mmswumeom u m ~w>mP Foozom zswvcoomm u N xPpmzm: u < pm>mp _oo;om acmacmewFN n F e m.F _ e.m N newcomee «.m.N._ N.F ¢.m “seamen—e «corp «.m.N._ N._ ¢.m :ugwmmmm iecwmem Eocmoca w m.F N F m ¢.N cowpomem IIIIIIIII IIIIIIIIIIIIII u IIIIIII 3 IIIIIII 1 IIIIIII 1 llllll 1 IIIII t lllllllll llllllllllllelllllllllllll llllll e.m N.F e.m N._ eeeeoeee ¢.m.N._ q m.N.P mcvccmmp mpoeoca ¢.m N.P ¢.m N._ mocmuwso ¢.m.— N ¢.m N._ memocmewo o. ¢.m.N.F ¢.m N.F gusmmmmm wm N.F ¢.m e m.N.P acmEmUm_a «umuwucmnceum e.m.N._ e.m N._ meeeece llllllllllllllllillllllillllllllllllllllllllllllllllllfilllItllllllllllliillllllIIJIll llllllllllllllll N._ e.m N.F me_eoeee «.m.N.P a m.N._ :osmmmmm ¢.m.N.P e m.N.F comum:_m>m m>wumscom «.m.N F v m N._ mcwccmmF mwosocm smumeicmcomm» ¢.N m._ e m.N._ mcwnmgw o u m .n o u m < cmpeo 30: :mumo 30: mm: soc mmmoaesa mc_pmmp co mmaxh xmxczh :H mmumpm copra: any :H chmcmw cw chmzucH use .msmymxm Paco?» -eoznm gmwxcah one mopepm capes: on“ c? mpmmh m>wpumnao mo mm: we» so mpnme comwcoaeou < .m mpneh 130 are "rarely used" at the elementary and secondary levels in the United States, they are "usually" used in Turkish educational system. Standardized tests are used at all levels of the United States educational system for all purposes stated. In the Turkish educational system they are “rarely" or “almost never" used, except at elementary and secondary levels and then only for coaching purposes. It should be stressed that so-called standardized tests in Turkish educational sys- tem are not real standardized tests as discussed earlier. The only similarity on the use of standardized tests between the two countries is found in higher education. Since these tests are usually program examination tests, their use is discussed in the following paragraphs. Program examination tests that, as discussed in Chapters II and III, are developed for special purposes and usually standardized are also used more frequently in the United States educational system than they are in Turkish educational system. The differences, however, in the use of program examination tests between the two countries is less obvious as compared with the differences between the two countries in use of teacher-made and standardized tests. An essential difference in use of program examination tests for selection purposes between the two countries should be kept in mind that program examination tests in the United States are used generally not to decide whether high school graduates will be accepted to colleges or universities, but they are used to select students for various scholarships and as pre-tests in order to follow up students' progress. In the Turkish educational system, these tests are used for selection purposes, they are generally the main 131 criteria whether a student will be allowed to attend certain schools, either a state boarding school or a higher educational institution. Examination of Table 3 shows that, in the United States, teacher- made tests are more often used for grading, promoting learning, forma- tive evaluation, research, and coaching purposes than they are in the Turkish educational system. While teacher-made tests are "usually” used for grading at every level of education in the United States they are ”rarely" used at the elementary and higher educational levels, and ”almost never" used at the secondary level of the Turkish educational system. It is clearly seen that objectivity in the measurement of students' achievement in the United States educational system is fre- quently practiced while it is “rarely" or “almost never” practiced in the Turkish educational system. A few instructors in elementary school and higher educational institutions' in Turkey use these for grading pur- poses. While higher educational institutions' instructors may use objective classroom tests without any interference from any source ele- mentary school teachers cannot use objective tests openly because of the examination regulations. In the United States teacher-made tests are more frequently used to promote learning at the elementary and secondary levels than at the higher educational level. In Turkey while they are "sometimes” used at the elementary level, they are ”almost never” used at the secondary and higher educational levels. It can be said that elementary school teachers are more apt to try innovations, or therefore they may be more informed about the usefulness of objective tests than teachers on other levels. The concern of the teachers in secondary and higher educational 132 schools with the learning in the classroom is reflected in different forms, mostly in the form of grade punishment. A comparison of the use of objective teacher-made tests to promote learning in the two countries clearly indicated that these tests are more often used in the United States educational system than they are in the Turkish educa— tional system. Use of teacher-made tests in formative evaluation at all levels of education in the United States is more often seen than it is in Turkey. Program examination tests for selection and placement purposes are more widely used in the United States educational system than they are in the Turkish educational system. In fact, they are "almost never” used in the Turkish schools. Turkish schools use program examination tests for selection and coaching purposes. In other words, while they are used for practical as well as scientific reasons in the United States educational system they are used only for practical purposes in the Turkish educational system. A general summary on the use of three types of testing is given in Figure 4. When a comparison is made between the two countries in terms of objective tests and their use for the purposes stated, it is clear that tests are used for all purposes, except coaching and selec- tion, more frequently in the United States educational system than they are in Turkey. In other words, the use of tests takes place on the positive side of the scale in the United States while it takes place on the negative side in Turkey. It is interesting to stress the fact that 133 .memgmxm Pecowumosvm :mwxczh use mopepm umpwc: oz» :_ mommms cwmzp use mc_ummp No momma omega we» mo accessm Fmemcme d S G d 8 ll. 3 a 3 d I... MO II. a p. 0 1.. O I. In 9 a p. 9 S 3 by a p. a 0 5 S 3 Id 9 a 3 3 3 p. w u a a p. In a I... .L. .L. u 4.1 S I“ a I: D u u 0 u ..L. a I... 3 u u “I 1+ 6 u 6 W S u.- nl? 5 p L P b h r11 1- L mi 4 q A i 1 i q a a q 113A i .7, .ii,, .\1-1t \ I I .\ \ \ I I \ \ \ ‘ ‘ s \ LT \ :1 .rl mhmmh onHm= will. 1. pmoep< .. >_mcmm ... mmeumEom -r.>P_m:m= ”zmhmo 30: mpmmh mo