INFORMATION TO USERS This material was produced from a microfilm copy of the original document. While the most advanced technological means to photograph and reproduce this document have been used, the quality is heavily dependent upon the quality of the original submitted. The following explanation of techniques is provided to help you understand markings or patterns which may appear on this reproduction. 1. The sign or "target" for pages apparently lacking from die document photographed is "Missing Page(s)". If it was possible to obtain the missing page(s) or section, they are spliced into the film along with adjacent pages. This may have necessitated cutting thru an image and duplicating adjacent pages to insure you complete continuity. 2. When an image on the film is obliterated with a large round black mark, it is an indication that the photographer suspected that the copy may have moved during exposure and thus cause a blurred image. You will find a good image of the page in the adjacent frame. 3. When a map, drawing or chart, etc., was part of the material being photographed the photographer followed a definite method in "sectioning" the material. It is customary to begin photoing at the upper left hand corner of a large sheet and to continue photoing from left to right in equal sections with a small overlap. If necessary, sectioning is continued again — beginning below the first row and continuing on until complete. 4. The majority of users indicate that the textual content is of greatest value, however, a somewhat higher quality reproduction could be made from "photographs" if essential to the understanding of the dissertation. Silver prints of "photographs" may be ordered at additional charge by writing the Order Department, giving the catalog number, title, author and specific pages you wish reproduced. 5. PLEASE NOTE: Some pages may have indistinct print. Filmed as received. Xerox University Microfilms 300 North Zeeb Road Ann Arbor, Michigan 48106 75-7258 STELLER, Bonnie J., 1928THE MARKING PROCEDURES USED BY PUBLIC SCHOOL TEACHERS IN THE STATE OF MICHIGAN. * Michigan State University, Ph.D., 1974 Education, theory and practice Xerox University Microfilms, © Ann Arbor, Michigan 48106 1974 BONNIE J. STELLER ALL RIGHTS RESERVED THE MARKING PROCEDURES USED BY PUBLIC SCHOOL TEACHERS IN THE STATE OP MICHIGAN By Bonnie J. Steller A DISSERTATION Submitted to Michigan State University in partial fulfillment of the requirements for the degree of DOCTOR OP PHILOSOPHY Department of Counseling, Personnel Services, and Educational Psychology 1974 ABSTRACT THE MARKING PROCEDURES USED BY PUBLIC SCHOOL TEACHERS IN THE STATE OP MICHIGAN By Bonnie J. Steller This study dealt with the question of establishing the status quo in the State of Michigan regarding the pro­ cedures used by teachers when assigning marks to students as indicators of their performance on a portion of the school curriculum. The intention was to define differences in the relationships between thirteen student characteristics and the teachers' personal characteristics, situational factors including subject area and grade level taught, the teachers' attitudes regarding the appropriate goals for education and the functions of m a r k s , and other procedures that are asso­ ciated with marking. A total sample of 1022 teachers representing 140 school districts and 511 schools were randomly selected to receive a mailed questionnaire. A response rate of 70.15 percent was obtained. Essentially canonical correlation analyses and chi square tests of independence were utilized for the statisti­ cal analysis of the data. Bonnie J. Steller The results indicated that teachers can be categor­ ized into two groups on the basis of whether the marks they assign are based upon objective information or upon subjec­ tive estimations. The teachers* attitudes regarding the appropriate goals for education and the usefulness of marks to teachers were found to be the basic determinants of the relative importance associated with each of the student characteristics. Approximately equal numbers of teachers reported that effort and test marks were of greatest im­ portance. The majority of teachers reported that they base marks on a combination of subject matter mastery and the students' growth. However, those teachers who favored ob­ jective marks were inclined to prefer to base marks on rela­ tive performance compared to others or a combination of relative performance and subject matter mastery. A profile of the two groups of teachers demonstrated that male teachers more frequently reported that the marks they assign are derived from objective information. In this group were also more younger teachers and teachers of upper grade students and academic subject areas. These teachers reported that the basic goal of education should be to pro­ vide students with a sound framework of substantive knowledge. The other group of teachers included primarily older fe­ male teachers of either lower grade level students or second­ ary nonacademic areas. These teachers reported that the Bonnie J. Steller primary function of education should be to socialize the children rather than to instill knowledge. Additional findings revealed that the majority of school districts provided a dual marking system with re­ ports issued to parents and students at six week intervals. The majority of teachers reported that they were satisfied with the present marking system which for many elementary teachers included parent-teacher conferences. An important concern was raised as a result of this study. Teachers, for the most part, were found to be in­ capable of defining precisely those tasks that are involved during the process of assigning marks. It, therefore, is apparent that not only do the resultant marks lack relia­ bility but also that teachers, for this reason, cannot de­ fend or explain the assigned marks. ACKNOWLEDGMENTS I would like to thank the members of the faculty at Michigan State University who contributed to my overall education* Particularly, I would like to thank the members of my doctoral committee who contributed their time and knowledge to assist me in the design and execution of this research. Included on this committee were Dr. Maryellen McSweeney, Department of Educational Psychology, and Research Design; Statistics Dr. Andrew Porter, Office of Research Consultation; Dr. Charles Eberly, Office of Evaluative Serv­ ices; Dr. Ellen Strommen, Department of Psychology; and Dr. Robert Ebel, Department of Educational Psychology, Measurement and Evaluation. I would like to offer a special thanks to Dr. Ebel, the committee chairman, without whose guidance this research would have been much more difficult. I would also like to thank Paula R. Knepper for providing both psychological encouragement and physical assistance during this period of research. My family also deserves a thanks for their encour­ agement and patience. Also a special thanks to the typist without whose services this research would have remained known only to its author. ii TABLE OP CONTENTS Page LIST OP TABLES ..................................... LIST OP APPENDICES ................................. Chapter I. II. THE MARKING PROCEDURES USED BY PUBLIC SCHOOL TEACHERS IN THE STATE OF MICHIGAN . 1 Background of the Problem ............. General Purpose of this Study . . . Specific Objectives of This Study . . Definition of Terms . . . . . . . Guiding Principles and Assumptions . . O v e r v i e w ........................... 2 9 10 12 14 REVIEW OF THE L I T E R A T U R E ............. Historical Background ................. Philosophical Arguments Regarding M a r k i n g ........................... Methodological Arguments Regarding M a r k i n g ........................... Basis for M a r k s .................... Student Characteristics Considered . . Effect of Teachers' Personal Character­ istics .............................. Effect of Situational Factors . . . Effect of Teacher Attitudes . . . . Effect of Other Aspects of the Marking Procedure ....................... School Marking Policies ............. S u m m a r y .............................. O v e r v i e w ........................... 14 16 17 18 23 28 32 37 41 45 49 52 56 60 Chapter III. Page DESIGN OF THE S T U D Y ....................... 61 Population and Sample ................. Data Collection and Instrumentation . D e s i g n ................................. ................. Analysis Methodology O v e r v i e w .............................. IV. V. DATA ANALYSES AND FINDINGS OF THE STUDY 61 66 73 80 87 . 88 Statistical Methodology ................. Statistical Findings . Canonical Correlation Analyses . . . Chi Square Tests of Independence . . Tests for A g r e e m e n t .................... Summary Tables ....................... Summary of F i n d i n g s .................... O v e r v i e w .............................. 88 90 91 110 141 145 151 155 RATIONALE FOR THE STUDY, METHODOLOGY, CON­ CLUSIONS, AND RECOMMENDATIONS FOR FURTHER S T U D Y ........................................ Rationale for the S t u d y ................. M e t h o d o l o g y .............................. O b j e c t i v e s .............................. S a m p l e ................................. Data C o l l e c t i o n ....................... Data A n a l y s i s ........................... L i m i t a t i o n s .............................. C o n c l u s i o n s .............................. Recommendations for Further Study . . . 156 156 159 .160 161 162 162 163 164 167 A P P E N D I C E S ........................................... 169 BIBLIOGRAPHY 190 ........................................ LIST OF TABLES Table 2.1. 2.2. 2.3. 2.4. 3.1. 3.2. 3.3. 3.4. 3.5. 3.6. 3.7. 3.8. 3.9. Page Changes in the Assignment of Marks from 1913 to 1961 in the Philadelphia Public .............................. School System 19 Factors Reported Considered by Teachers and the Percent of Teachers Reporting Each . . Nation-Wide Summary of Specific Methods C o u r s e s ................................ 33 40 Percent of Respondents Agreeing with, Dis­ agreeing with, and Expressing Uncertainty Toward Selected Issues .................... 42 Target Population Distribution and Sample According to Strata I through V 64 Sample Distribution According to Level of Schools by Strata I through V ......... 65 Sources of Information Regarding the Varia­ ............. bles of Interest 67 Cronbach's Alpha Reliability for Appro­ priate Groups of Items by Strata I through ....................................... V 68 Scoring Procedure for Items Dealing with Frequency of Measurement .................... 70 Distributions of Teachers' Responses by Urbanization Level According to Strata 71 . Dual Sampling and Response Rates by Strata and by T o t a l ............................. . 75 Responses from Regular Mailing and from Extra Effort by Strata and by Total . . . Multivariate and Univariate F-Ratios for Composition of Marks and Whether Response Received from Regular Mailing or from Extra Effort for Stratum I through V ......... v 76 78 Table 3.10. 3.11. 3.12. 4.1. 4.2. 4.3. 4.4. 4.5. 4.6. 4.7. 4.8. 4.9. 4.10. Page Multivariate and Univariate F-Ratios for Composition of Marks and to Whom the Ques­ tionnaires were Sent for Stratum I through ........................................ V 81 Contingency Table for Stratum and the Basis on Which Marks Are A s s i g n e d .......... 85 Multivariate and Univariate F-Ratios for Composition of Marks and Stratum . . . . 86 Correlation Matrices for Personal Factors and the Weighting Applied to Student Char­ acteristics ................................. 94 Significant Canonical Variates for Per­ sonal Factors and Student Characteristics . 96 Correlation Matrices for Situational Factors and Weighting Applied to Student Characteristics .......................... 98 Significant Canonical Variates for Situa­ tional Factors and Student Character­ istics .................................... 100 Correlation Matrices for Attitudinal Fac­ tors and Weighting Applied to Student Characteristics .......................... 103 Significant Canonical Variates for Attitud­ inal Factors and Student Characteristics . 105 Correlation Matrices for Other Aspects of the Marking Procedure and Weightings Applied to Student Characteristics . . . 108 Significant Canonical Variates for other Aspects of the Marking Procedure and Stu­ dent Characteristics ....................... 109 Chi Square Analyses for Personal Factors and Basis for M a r k s ................... Ill Chi Square Table for Sex and Basis for M a r k s ................................... 112 vi Table 4.11. 4.12. 4.13. 4.14. 4.15. 4.16. 4.17. 4.18. 4.19. 4.20. 4.21. 4.22. 4.23. 4.24. 4.25. 4.26. Page Chi Square Table for Age and Basis for M a r k s ....................................... 114 Chi Square Analyses for Situational Factors and Basis for M a r k s ....................... 115 Chi Square Table for Level of School and Basis for M a r k s .......................... 116 Chi Square Table for Grade Level Taught and Basis for M a r k s .......................... 118 Chi Square Table for Academic versus Nonacademic and Basis for M a r k s ............. 120 Chi Square Table for Subject Area Taught and Basis for M a r k s ....................... 121 Chi Square Analysis for Attitudinal Fac­ tors and Basis for M a r k s ................ 123 Chi Square Table for Goals of Education and Basis for M a r k s .......................... 124 Chi Square Table for Functions of Marks for Students and Basis for M a r k s ............. 126 Chi Square Table for Functions of Marks for Parents and Basis for M a r k s ............. 128 Chi Square Analyses for Other Aspects of the Marking Procedure and Basis for Marks . 129 Chi Square Table for Sources of Useful In­ formation and Basis for M a r k s ............ 130 Chi Square Table for Frequency of Objective Measurements and Basis for Marks . . . . 132 Chi Square Table for Use of Homework and Basis for M a r k s .......................... 134 Chi Square Table for Frequency of Quizzes and Basis for M a r k s ...................... 135 Chi Square Table for Frequency of Tests and Basis for M a r k s ...................... 136 vii Table 4.27. Page Chi Square Table for Frequency of Use of Statistical Techniques and Basis for M a r k s ........................................ 138 Chi Square Table for Use of Percentile Hanks and Basis for M a r k s ................ 140 Chi Square Table for Use of Normal Curve and Basis for M a r k s ....................... 140 Relative Weightings for Student Character­ istics .................................... 142 Rankings of Importance of Student Charac­ teristics ................................. 143 4.32. Frequency Distribution for Basis for Marks 144 4.33. Formats of Items Used for Quizzes and T e s t s ........................................ 146 4.34. Frequency of Objective Measurements . 147 4.35. Suggested Changes in Marking Procedures and P o l i c i e s .................................... 149 Frequency of Characteristics Regarding the Report Forms .............................. 150 4.28. 4.29. 4.30. 4.31. 4.36. • • v m • . . LIST OP APPENDICES Appendix Page A. Survey Questionnaire ........................ 170 B. Table— Frequency Distribution for Responses to Questionnaire Items .................... 173 Letter to Superintendents and Postcard Follow-Up to Superintendents .............. 180 Letter to Principals in Nonresponding Districts and Letter of Follow-Up to Principals in Nonrespondlrig Districts . 183 Cover Letter to Teachers, Letter of Follow-up to Teachers and Postcard FollowUp to T e a c h e r s ............................... 186 C. D. E. . CHAPTER I THE MARKING PROCEDURES USED BY PUBLIC SCHOOL TEACHERS IN THE STATE OF MICHIGAN The practice of assigning marks to students, as in­ dicators of their academic and/or social performance, has become an established tradition at all levels of education from kindergarten through graduate school. However, despite the universality of the practice, there has been an associa­ ted, continuing dissatisfaction, expressed both publicly and privately, regarding particularly the reliability of marks and the question of whether they are accurate indi­ cators of students* abilities. In 1918 when E. L. Thorndike made his statement that "whatever exists at all exists in some amount," he was re­ ferring to the belief that achievement, just like other char­ acteristics of man, can be measured quantitatively. A fur­ ther addition to this statement was advanced in 1939 by William A. McCall, who added the following: "Anything that exists in amount can be measured." The marking of student performance is at least par­ tially based upon the above stated assumption. Each teacher is entrusted with the responsibility of providing at specified 2 intervals a mark or group of marks that is an accurate and reliable indicator of each student's performance relative to a portion of the school curriculum. Each mark is then re­ corded on a permanent file, which is an accumulated record of every mark assigned by teachers during the student's en­ tire school career and is referenced by college admissions officers and future employers for many years. These marks are intended, also, to provide immediate and useful knowl­ edge to the student and his parents concerning current aca­ demic performance. One fallacy, however, becomes obvious when compari­ sons are made between marks assigned a student. assigned by different teachers comparable? Are marks The answer, of course, is expected by the student and his parents to be "yes." Background of the Problem In the second decade of this century Starch and El­ liott presented sound evidence concerning the lack of reliability of teachers' marks. 12 3 ' ' About the same time Kelly verified that not only did different teachers frequently as­ sign different marks to the same student papers but also that specific marks held widely diverse interpretations for ^David Starch and E. C. Elliott, "Reliability of the Grading of High School Work in English," School Review, XX (1912), 442-457. 2Starch and Elliott, "Reliability of Grading Work in History," School Review, XXI (1913), 676-681. 3Starch and Elliott, Mafhomat - 1 oc " Srhnnl "Reliability of Grading Work in Rauipu. YYI f1913) . 254 — 259. 3 different teachers. Marks earned for equivalent work might vary from G, good, to F-, fair minus, within a single ele­ mentary school or from 70 percent to over 81 percent between 4 different high schools. Two major concepts may explain at least a part of this variation in the mark assigned by different teachers under similar circumstances. The first such concept is the lack of agreement regarding the basis or framework within which marks are to be assigned as has been demonstrated by several studies. The second concept related to the lack of comparability of marks concerns the student characteristics that are represented by the marks. Thorndike refers to this first difference as a differ­ ence in the frame of reference according to which a mark is expressed. The first frame of reference was termed perform­ ance relative to potential and the second termed mastery. The remaining frame of reference is performance relative to a group, national, school, class, or a group defined by the par­ ticular teacher as typical. The first frame of reference he rejects because teachers are unable to assess with any degree of reliability the "gap between potential and competence." Marks assigned on the basis of mastery are suitable to only a few educational situations. On the other hand, there are difficulties associated with attempts to mark relative to ^Frederick James Kelly, Teachers' Marks: Their Variability and Standardization (New York: Teachers' College Press, 1914). 4 some group. Local autonomy virtually prohibits the use of national or even state norms. At the same time values and philosophical issues complicate attempts to develop school norms. Many teachers utilize their own concept of the "typical class," an undefinable ideal, as a class norm. Thorndike, for these reasons, does not recommend the appli5 cation of any of the alternatives. If, in fact, the major cause of the lack of compara­ bility between marks is the absence of agreement regarding the basis on which marks are to be assigned, policies at either the school level or district level, specifying the basis to be utilized by all teachers, would decrease the dif­ ference. Yet Terwillinger found variability not only between schools, with the majority of policies stating the basis as either absolute mastery of subject matter or achievement with regard to ability, but also between departments and teachers within these schools. While over 4 0 percent of the teachers reported that they based marks on achievement with regard to ability, mathematics, foreign language, and business teachers frequently reported absolute subject matter mastery as a basis. Art and physical education teachers reported a pref­ erence for achievement with regard to class and self c Robert L. Thorndike, "Marks and Marking Systems," Encyclopedia of Educational Research, edited by Robert L. Ebel (New York: The Macmillan Company, 1970), pp. 759-766. 5 improvement was the basis for marks assigned by agriculture, home economics, and dri v e r 's education teachers•® Apparently even though a policy is established at the school level, teachers do not feel bound to follow these recommendations and, therefore, marks remain not comparable even within those schools which have established policies regarding the basis upon which marks are to be assigned. The other concept responsible for the lack of com­ parability of marks concerns the student characteristics that are represented by the marks. Palm and Herron both re­ ported that some teachers intend to consider such factors as effort and attitudes when assigning marks, while others do so 7 8 unconsciously. ' Similar findings were made by Owen and Proffitt who both showed that many diverse concepts such as accuracy, regularity, the ability to apply information, subject matter mastery, and the ability to study independently might 9 10 be included in a single mark. ' 6James S. Terwillinger, "A Survey of Secondary School Marking Practices and Policies," National Association of Secondary School Principals, L (March, 1966), 1-37. 7 R. R. Palm, "A Proposed Secondary School Uniform State Marking System," M i n n . J . E d ., XV (1935), 213-215. Q J. S. Herron, "How Teachers Rate Their Pupils," Elementary School Principals Bulletin, VIII (1929), 235-239. g M. M. Proffitt, Grading in Industrial Schools and Classes, Cir. 28 (Washington, D.C.: U.S. Office of Education, 1929). XVIII ^°W. B. Owen, "Making the Grade," N a t i o n s Schools, (August, 1936), 21-22. 6 Terwillinger found these variations to be largely accounted for by the subject area which the teacher teaches. Over 50 percent of the studied teachers reported that they made use of weighting formulas when assigning marks. Business teachers allocated the greatest weight to daily work; mathe­ matics and science teachers allocated the most weight to test marks; language arts and foreign language teachers allocated the greatest weight to projects, while others gave equal weight to daily work, test marks, and projects. Significant differences were also found between teachers of academic and non-academic subject areas in their consideration of behavior, attendance, and punctu a l i t y . ^ In an attempt to encourage teachers to limit the student characteristics, that are included in marks, to cognitive factors, dual marking systems are frequently em­ ployed. Such a dual system provides for the assignment of separate and distinct marks to represent affective character­ istics in addition to the usual academic marks. In an investigation of the effects of single and dual marking systems on the academic marks of gifted sixth grade pupils, Ilousdorff and Farr reported no associated differences, thereby, concluding that a dual system provides 12 no advantage over a single mark. ^ T e r w i l l i n g e r , loc. c i t . ^ H e n r y Housdorff and S. David Farr, "The Effects of Grading Practices on the Marks of Sixth Grade Gifted Children," J. Ed. Res., LIX (December 1965), 169-172. 7 Similar conclusions were drawn by Halliwell and Robitaille, when they reported a significant correlation of .68 between assigned academic marks and a subjective mark representing effort. If a dual system is to increase the reliability and validity of academic marks, teachers must bo provided with adequate methods for the assessment of 13 affective factors. The above mentioned evidence demonstrates that variations do occur among teachers in regard to the bases upon which marks are assigned and the student character­ istics that are considered when assigning marks. Yet little is known about what factors are related to the various pro­ cedures used by teachers when they assign marks to students. The majority of studies dealing with the question of how teachers assign marks to students were conducted during the third and fourth decades of the present century. However, since that time numerous changes have taken place which may have had a significant impact on these procedures. Not only are teachers required to have an increased number of years of training but also the content of this training has undergone major revision. The expanded knowl­ edge of human growth and behavior has made teachers increas­ ingly aware of their contributions to students* development. ^ J o s e p h W. Halliwell and Joseph P. Robitaille, "The Relationship between Theory and Practice in a Dual Report­ ing Program,11 J . E d . R e s . , LVII (November, 1963), 137-141. 8 Subject matter expertise is no longer considered to be sufficient to produce the desired results within the students. At the same time students are expected to acquire during their school careers a more complete and comprehen­ sive body of knowledge than ever before. However, concerns have also been voiced that the educational processes can have serious, at times detrimental, unintended effects on the students. These concerns have led to the increased criticism of education, a part of which has been directed toward the practice of assigning marks to students. Critics have strongly and repeatedly recommended that marks be completely eliminated from the schools. On the other hand, the supporters of marking claim that marks can serve important functions to the teacher, the student, and the parents. Other techniques for the accomplish­ ment of these functions are not yet available. To eliminate marks would be to create within the school an artificial situation, wherein, students are not provided the means for judging their own strengths and weaknesses, an essential capacity for adult activities. The supporters of marking have proposed numerous suggestions regarding changes which would, they believe, im­ prove marking procedures and practices. suggestions have dealt, However, these for the most part, either with mechani­ cal considerations of form or with theoretical considerations. 9 General Purpose of this Study The purpose of this study is to examine the current situation in the State of Michigan in regard to the frame­ work within which marks are assigned and the composition of marks. Although no direct attempt is made to replicate the earlier studies conducted during the third and fourth decades of the twentieth century, this study deals with many of the same issues and, therefore, references will be made to these studies when appropriate. These issues deal with such topics as the compara­ bility of marks, the objectivity versus the subjectivity of marking, and the influence of teachers' personal character­ istics and situational factors to the framework within which marks are assigned and the composition of marks. Such information as is provided will be useful in a number of contexts. The first and most important context is in the design of alternatives to marking or in the modifica­ tion of existing systems. As the emphasis on continuing life-long education increases, in conjunction with the re­ quirements of increased technological knowledge and skills, more and more often there is a need for reliable indicators of individual's strengths and weaknesses in regard to academic and technical skills. Even with the acknowledged low relia­ bility of marks, college admissions officers must base at least a significant portion of their decisions on marks, as other means are lacking. 10 In the context of schools or school districts such information will be useful in the establishment of guidelines and the determination of policies regarding the assignment . of marks. Although institutional policies have in the past been tried and have failed, because of the lack of quality 14 control, if properly administered, such a system can insure the comparability of marks at least within its jurisdiction. In a further context that of the colleges of educa­ tion such information can serve two purposes. The first purpose is to revitalize the study of student marking instead of its condemnation based upon incomplete evidence. The second purpose is its contribution to the training of pros­ pective teachers and for in-service training of teachers. In the context of the in-service teachers, such in­ formation would assist them in the formulation and definition of their own marking procedures and, at the same time, aid in their interpretation to parents and students of assigned marks. Specific Objectives of This Study The major goal of this study is to derive an answer that is an accurate representation of the current situation as reported by teachers in the State of Michigan to the fol­ lowing question: ment 338 . How are teachers assigning marks to students? ^ R o b e r t L. E b e l , Essentials of Educational Measure­ (Englewood Cliffs: Prentice-Hal1, I n c ., 1972), pp. 337- 11 This question will be answered by synthesizing the responses to the following associated questions: What personal factors are significantly related to the weighting applied to student characteristics that are considered when assigning marks? What personal factors are significantly related to the basis on which marks are assigned? What situational factors are significantly related to the weighting applied to student characteristics considered when assigning marks? What situational factors are significantly related to the basis on which marks are assigned? What attitudinal factors are significantly related to the weighting applied to student characteristics considered when assigning marks? What attitudinal factors are significantly related to the basis on which marks are assigned? What other aspects of the marking procedure used are significantly related to the weighting applied to student characteristics considered when assinging marks? What other aspects of the marking procedure used are significantly related to the basis on which marks are assigned? Do teachers agree in regard to the relative importance of student characteristics considered when assigning marks? Do teachers agree in regard to the basis on which marks are assigned? What types of objective measures of student perform­ ance are made by teachers? How frequently are objective measurement of student performance made by teachers? What changes do teachers suggest regarding marking procedures and policies? How do school districts differ in their marking policies? 12 Definition of Terms Assigned m a r k s : This refers to the letter, number, or other symbol that appears on a report card or student progress report, which is intended to provide the student and/or parents at regular time intervals with information regarding the academic progress or relative standing of the student. Attitudinal factors; This refers to the teacher's expressed attitudes regarding the following: propriate goals of education, can successfully serve, and (1) the ap­ (2) the functions that marks (3) suggested changes or im­ provements in marking procedures and policies. Basis for M a r k s : This refers to the frame of refer­ ence within which marks are assigned. are included: Three possible bases (1) mastery of subject matter, (2) student growth when compared to himself, and (3) student performance relative to other class members. Marking policies: This refers to institutional re­ quirements within which framework the teacher must operate when assigning marks. quency of marking, system, Included are the following: (1) fre­ (2) presence or absence of a dual marking (3) types of symbols used to indicate marks, and (4) the overall format of the reporting form. Marking practices: This refers to the methods util­ ized by teachers when assigning marks to students and in­ cludes the following elements: (1) student characteristics 13 that, are considered by the teacher when making the decision to assign a particular mark, (2) relative importance or weight assigned each of these student characteristics, (3) type and frequency of objective measures of student perform­ ance, (4) frequency of use of particular statistical tech­ niques when assigning marks, and (5) sources of useful information in the decision-making process related to mark­ ing. Situational factors; To be included in this cate­ gory of interest are the following: district, (2) size of the school, the community, (1) size of the school (3) urbanization level of (4) percent of minority student within the school district, (5) grade levels included in the school, (6) grade level taught, and Personal factors: (7) subject area taught. To be included are the following characteristics of the teacher: (1) sex, (2) age, (3) teach­ ing experience as measured by the number of years taught, (4) college degrees held, (5) self estimation of the level of academic success achieved, and (6) number of college courses completed in the area of grading, measurement, or evaluation. Size: Size both in association with schools and school districts refers to the number of teachers employed. Student characteristics: To be included in this category are those factors that are considered by teachers 14 when assigning marks. attendance, (2) personal appearance, class participation, attitudes, marks, These factors are as follows: (3) behavior, (4) in- (5) amount of effort put forth, (7) neatness of work, (10) quiz marks, (8) homework, (11) optional work, (1) (6) (9) test (12) group re­ ports or projects, and (13) individual reports or projects. Guiding Principles and Assumptions The following set of principles and assumptions form the framework within which this study was conducted. 1. The major function of marks is to provide a reliable and accurate indication of student performance in regard to a portion of the school curriculum. 2. Teachers want to assign reliable and accurate marks. 3. The more reliable a mark i s , the more valid it is as an indicator of student performance. 4. Marks based upon objective measurements are more reliable than are marks based upon subjective estimations. 5. Measurements which are comprised of objective formatted items are more reliable under normal school conditions than are measurements com­ prised of other types of items. 6. A reliable mark is one which does not vary significantly for the same work between points in time or between different teachers. Overview This chapter has presented the problem statement as well as the objectives to be accomplished by the study.- 15 Chapter XX Includes a review of the major literature in the field. Chapter XXI includes a summary of the procedures that were utilized. and findings. Chapter IV presents the data analysis Chapter V presents the conclusions and recom­ mendations which were derived from the study. CHAPTER II REVIEW OF THE LITERATURE The review of literature for this study included an examination of the philosophical and methodological argu­ ments which have been presented in regard to the practice of assigning marks to students. Both the critics' and the defenders1 points of view were examined in regard to these arguments in order to present an unbiased review, although the author's biases may, nevertheless, at times be evident. This review of literature included many of the wellknown writings on the subject as well as several of the lesser known articles. Because of the large number of arti­ cles and books, which have been written on the subject of marking since 1910, only a limited amount of the material has been presented in this chapter. The material presented was chosen for the most part because of its relevance to the variables, investigated by this study. For this reason, the chapter has been primarily organized according to the variables under investigation, with the exception of the section dealing with the historical background of the prob­ lem and the philosophical arguments for and against mark­ ing. 16 17 Historical Background In the early period of education the direction of evaluation was the reverse of that taken today. The teach­ ers were evaluated in terms of the performance of their students. Teachers were held accountable for the success or failure of their students, a justifiable procedure as long as the criteria for success were highly visible. How­ ever, as the requirements for success became complex, this method was no longer practicable. By the mid-nineteenth century formal education as preparation for college had gained prominence among the upper class families. However, college admission depended upon the family's social status and their financial ability rather than upon the prospective student's scholastic abili­ ties. The use of examinations was considered beneficial primarily as indicators of the student's readiness to leave the college but not in regard to admission. With the rapid growth between 1870 and 1910 in the number of students enrolled in public elementary and high schools, new methods of informing students as to their rela­ tive performance were introduced. By 1910, however, the written reporting of student performance had gained almost universal application in the United States. Although many elementary teachers provided written descriptions of the performance of each individual student, virtually all high 18 school teachers used percentages or other similar markings to report the students' performance in the different sub­ ject areas. During the next few years elementary teachers also gradually adopted point systems or percentage mark­ ing procedures. Nevertheless, the search for improved methods was continued even to the present. Not only has a single "best" method not been developed, but there is doubt that such a method is feasible or pos s i b l e . Table 2.1 shows the changes which have taken place within the Philadelphia Public School System between 1913 and 1961 in regard to the assignment of marks.^ Many of these changes were in response to the criti­ cisms made of marks and marking procedures, ranging from philosophical arguments to methodological and technical critiques. Philosophical Arguments Regarding Marking One philosophical argument against the practice of assigning marks to a student in order to convey information regarding the student's relative performance has been pri­ marily concerned with the question of whether one individual had the right to make judgments regarding the performance of another individual. “^Howard Kirschenbaum, Sidney B . S i mon, and Rodney W. Napier, Wad-ja-get? (New York: Hart Publishing Co., Inc., 1971), pp. 45-54. 19 TABLE 2.1.— Changes in the Assignment of Harks from 1913 to 1961 in the Philadelphia Public School System.a Year Elementary High School 1910 to 1920 10 point A single overall mark 10 distinguished, 9 excellent, 8 good, 7 fair, 6 poor, 5-1 very poor Percentage for each subject 1920 to 1930 5 point Percentage equivalent 91-100 excellent, Bl-90 good, 71-80 fair, 61-70 passing, below 61 deficient 1930 to 1940 3 A B C 5 point Dropped percentage equivalent 1940 to 1950 2 point S satisfactory, U unsatisfactory plus trait checklist point outstanding, satisfactory, needs improvement Percentage equivalent 90-100 A, 80-89 B, 75-79 C, 70-74 D, 60-69 E, below 60 F 4 point Added 0 outstanding, I improvement to above system 1950 to 1960 5 point A, B, C, D, P 1961 Added checklist in reading and arith­ metic to above 5 point system 5 point A, B, C, D, E includ­ ing grades for behav­ ior and work habits Howard Kirschenbaum, Sidney B. Simon, and Rodney W. Napier, Wad-ja-qet? (New York: Hart Publishing Co., Inc., 1971), pp. 45-54. 20 Pewitt's statement that " . . . mitted purpose . . . the real, but unad­ is to maintain a caste system of su­ periority over inferiority, educated over uneducated, teacher over student, adult over child," is an example of this point of view. Complete equality was demanded for the student, insisting that the teacher had no moral authority to judge the students' performances. Increased capabilities through education or even maturation were denied. For the teacher to judge the student was to label one individual superior over another. On the other hand, Dunivan in his rebuttal to Pewitt stated that marking does not create a caste system because the individuals do not remain permanently fixed within a given station but may advance either forward or backward. Teachers have ". . . no compulsion to use grading to prove their superiority over students." The pattern of authority of which marking is only a small part should have no detri­ mental effect on students, because this is a natural phenom­ ena of all life. "Each of us has at least one person who 3 has authority over him." From the same point of view Becknell commented that any attempt at insuring absolute equality will fail because 2 Joe Pewitt, "Grading as a System of Coercion," School and Community, LVI (Feb., 1970), 20. 3 Lindell P. Dunivan, "Grading as a System of Guid­ ance: A Rebuttal," School and Community, LVI (May, 1970), 31. 21 "human beings will always build up social barriers between groups." If the role of the school is to provide the oppor­ tunity for its students to develop into good citizens, the school should create.an environment representative of adult life. Because "adults are graded in everything they do," the school should provide an environment in which the stu­ dents learn to be aware of their own strengths and weak­ nesses. Poor students should not be deceived into thinking that their work is as good as that of the A students. 4 A satisfactory answer to either of these points of view is impossible. The school exists as a part of the surrounding society and, therefore, must operate within the same framework, including both freedoms and limitations. A society in which judgments are made and estimations of worth provided will, in most cases, support a school which utili­ zes similar techniques. Therefore, to criticize teachers for marking students is to criticize the similar practice with the society of which it is a part. Another philosophical argument deals with the issue of external reward. Johnson commented that grades do not encourage students to think for themselves. They learn to memorize answers, thereby, the natural love of learning for 5 personal satisfaction is destroyed. 4George l». Becknell, "S and U Grading System," Texas Outlook, XXV (July, 1941), 18. 5 Icie F. Johnson, "Injustice of Grades," School and Community, LIV (October, 1967), 24-25. 22 Such a statement requires the assumption that the amount of learning which takes place is not related to the grade received. If this is, in fact, the situation, then the problem lies not with the mark itself, but with the pro­ cedures used in the teaching process. assignment of the mark or with the Marks, which are based upon sufficient, relevant, important evidence, are valid indicators of what and how much the students have learned.® On the other hand, De Zouche in 1945 presented an argument against the need of external rewards. Does everything under God's heaven have to be done for the sake of winning? Can it never be done for the sake of the thing itself? Life is not a matter of winning or losing. Life is a matter of living, simply, courageously, and happily with our fellow creatures. Must we be rewarded, tagged with a blue ribbon for every worthy effort we make? Even adults have to be whiddled with an award. The government gives us an E if our war production reaches a certain point. The Red Cross hands us a sticker to put in our windows to announce to our neighbors that we gave a donation. The Tuberculousis Society clips a button to our lapel for the same reason. And in St. Louis the Community Chest gives us a dear little red feather to wear in our headbands as soon as we've sent in our checks. Perhaps if we stopped giving children a gold star for brushing their teeth, we wouldn't be giving adults red feathers for contributing to the Com­ munity Che s t .7 £ Robert L. Ebel, Essentials of Educational Measurement (Englewood Cliffs, New Jersey: Prentice Hall, Inc . , 1972), pp. 313-315. 7 Dorothy De Zouche, "The Wound is Mortal," Clearing House, XIX (February, 1945), 339-344. 23 The difficulty lies in the interpretation of marks as re­ wards. Although marks may provide either a sense of satis­ faction or of insufficiency, their primary function is to report, in a factual manner, the student's level of perform­ ance . High marks should not be intended as rewards or low marks intended as punishments. Methodological Arguments Regarding Marking The earliest studies which pointed out the low re­ liability of marks were those conducted by Starch and Elliott g in 1912. It must be remembered, however, that until after 1920 marks were based upon essays, which under normal condi­ tions, seldom have been shown to have acceptably high relia­ bilities . Dissatisfaction with the then-conunonly-used practice of assigning percentage equivalent marks led the College High School of Colorado State University at Greeley under the direction of William Wrinkle to search for improved methods of marking and reporting. Wrinkle reported six fallacies associated with marking. 1. Anyone can tell what a single A B C D P mark means. 2. A student can achieve he is willing to make 3. The student*s success in his after-school life compares favorably with his success in school. any mark he wishes— if the effort. Q Starch and Elliott, op. c i t . 24 4. The student's mark is comparable to the worker *s paycheck. 5. The competitive marking system provides a worthwhile and justifiable introduction to competitive adult life. 6. School marks can be used as a means to an end without their becoming thought of by students as ends in themselves. Although these fallacies are philosophical in nature, their involvement in the search for an improved method according to Wrinkle can lead to several practical results and find­ ings, several of which are presented below in an abbreviated form. 1. Outcomes must be thoroughly analyzed so that meanings can be clearly stated. 2. The number of different forms should be kept to a minimum. 3. The basis for student evaluation must be spe­ cific. 4. Parents are likely to misunderstand a report which includes many details. 5. If students are favorable to an idea, parents will be favorable. 6. The summarization of reports is too big a task and not workable. 7. The scale type form is not satisfactory. 8. The check form is simpler and economical of space. 9. Detailed, elaborate cumulative records are not economical. 10. Conference plan is not a practical solution at the secondary level. 25 11. A five-point scale should be maintained for administrative n e eds. 12. Check forms become increasingly detailed and, therefore, impractical. 13. Objectives should be stated in terms of desired student behavior. 14. Teachers have difficulty in writing effective comments. 15. Best way to write comments is to tell what student did. 16. Reports to parents of detailed specific ob­ jectives unnecessary. From these findings was developed a one page report which included a four-point mark to each of twelve general ob­ jectives and dual checks for actual and expected achievement based on outstanding, above average, average, below average. and very poor. 9,10,11 Anderson claimed that marks are not constructive methods of communicating with parents for three reasons: (1) marks are too general, (2) marks are too impersonal, and 12 (3) marks provide for only one-way communication. q William Wrinkle, "The Story of a Secondary School Experiment in Marking and Reporting," Educational Adminis­ tration and Supervision, XXIII (October, 1937), 481-500. ^®William Wrinkle, Improving Practices (New Y o r k : Rinehart and C ^ S a m u e l R. Johnston, "Are There Better Ways of Evaluating, Recording, and Reporting Student Progress?," National Association of Secondary School Principals, XXXIV (March, 1950), 79, 89. 12 Robert H. Anderson, "The Importance and Purposes . of Reporting," National Elementary School Principals Bulle­ tin, XLV (May, 1966), 6-11. 26 Of the same viewpoint is Melby who stated that the specific observational records usually maintained by teach­ ers are ignored in the marking process in favor of a single meaningless symbol. 13 These comments have dealt with the provisions and limitations set forth by school administrators in regard to the assignment of marks to students. Teachers seldom are free to make decisions regarding such matters as the number of points to be permitted, the format of the reporting form, or the frequency of reporting. Such decisions are regarded as administrative and at best will be made by a committee of teachers acting under the guidance of an administrator. Teachers on the other hand, can wrongly affect mark­ ing in seven ways, according to Palmer. follows: (1) by abdication, These ways are as (2) by a "carrots and club" system of rewards and punishments, (3) honestly by default when a single test serves as the entire basis for a mark, (4) by being a testing zealot who advocates the survival of the fitest, (5) by changing the rules in midstream, (6) by being a psychic grader who needs no evidence, and (7) by 14 anchoring grades on the rainbow of perfection. R. C. Little reported from a survey of teachers that, while they do desire to grade constructively in a way ^ E r n e s t 0. Melby, "It's Time for Schools to Abolish the Marking System," Nation's Schools, (1966), 104. ^ O r v i l l e Palmer, "Seven Classic Ways of Grading Dishonestly," The English Journal, LI (1962), 464-467. 27 most fruitful for the growth of their p u p i l s , the search for a single "best" procedure is likely to go on for some time.15 Bates proposed that the null hypothesis be tested that, "those students whose progress in reading, arith­ metic, and spelling is reported by means of report cards will demonstrate no more achievement in these subjects than students who do not receive report cards. Although the philosophical arguments presented de­ nied the existence of a simple, easy solution, for the methodological arguments, solutions are no more easily found. Kirschinbaum, Simon, and Napier in Wad-ja-get? present the advantages and disadvantages for each of the eight most frequently used alternatives to the assignment of symbols to represent the student's level of performance. These alternatives are as follows: (2) self evaluations, dents, (3) give marks but don't tell the stu­ (4) the contract system, ance approach, grading, and (1) written evaluations, (5) the mastery or perform­ (6) pass/fail grading, (8) blanket grading. (7) credit/no credit When the number and severity of the disadvantages are compared to the advantages, a "best" procedure is not defined. 17 l5Ruth Coyner Little, editorial, "Whither Grading," National Education Association Journal (1947). 16 Bea Bates, "Can We Stop the Merry-go-round of Grades?", Arizona Tea c h e r , LV (January, 1967), 19 and 26.. 17 Kirschenbaum, o p . c i t ., pp. 292-307. 28 Therefore, the methodological arguments also remain unanswered for the present. An investigation of the current situation will provide at least some direction to the search for improved practices. Basis for Marks At least two decisions must be made by the teacher whenever a marking procedure is selected or a mark is as­ signed. The first decision concerns the basis on which the mark or marks are to be determined. proches have been suggested. ing to the following: (2) Three possible ap- Marks may be assigned accord­ (1) mastery of the subject matter, performance as compared to a predefined group, usually the other class members, or (3) growth when compared to past performance or demonstrated ability. Little concur­ rence of opinion has been exhibited in this regard. The arguments for and against marking in relation to others have dealt with the question of whether competi­ tion has a useful or acceptable place within the school. Wrinkle claimed that because it is common knowledge that all Btudents can not achieve any mark, that competitive marking is unfair to the low achiever. Although adults do compete, this competition takes place primarily among equals. Also in adult life there is more cooperation than there is competition at least between unequals. 18 18 Wrinkle, Improving, pp. 36-4 9. 29 In a reaction to this point of view Schwartz, Tiedman, and Wallace stated that "the ability to achieve high marks is just one of many ways in which pupils vary and the elimination of (competitive) marking will not thereby create a Utopian institution in which all pupils work together on a basis of complete equality." 19 In answer to fear voiced by teachers that, students who are competing only against each other and not against some absolute standard, will mutually agree to take it easy together, Maxon replied that intense competition occurs when an individual works with his peers. free enterprise system competitors In the American do not mutually agree to take it easy nor do athletes when attempting to make the team. 20 Fine showed that while elementary teachers often avoid the use of competitive marking procedures, secondary school guidance counselors insist upon this traditional method because of the weight assigned marks by college ad21 missions officers. 19 Alfred Schwarts, Stuart C. Tiedman, and Donald L. Wallace, Evaluating Student Progress in the Secondary School (New Y o r k : David McKay Co., Inc., 1957) , pp. 391-392. 20 W. B. Maxon, "Grading a Serious Matter," National Education Association Journal, XIII (October, 1964), 56-57. 21Benjamin Fine, "A.B.C. of Grading Puzzles Parents," New York Times Magazine, November 18, 1957, cited by Ann Z. Smith and John E. Dobbin, "Marks and Marking Systems," Encyclopedia of Educational Research, 3rded., edited by Chester W. Harris (New York: The MacMillan C o . , 1960), pp. 783-791. 30 If marks are to be based upon performance relative to a predefined group, the decision must be made as to whether a national, state, school, or class group will used. be Sufficient information is seldom available for the use of either a national or state group. At the same time there is seldom sufficient cooperation at the school level to make such a comparison possible. For this reason as well as for the sake of convenience the class, either in actu­ ality or as a vaguely ideal class, is the most-often-used basis for comparison. Yet enough to provide adequate such a group is'seldom large information. One alternative basis for marks is mastery of sub­ ject matter. This position was favored by Marshall. How­ ever, he claimed that the marks of subject matter mastery should clearly only include content areas per se not other judgments.22 Wrinkle also agrees that if marks are influenced by other extraneous factors, they become meaningless. He goes on to say, on t|ie other hand, that marks which include only academic information are significant only insofar as the 23 academic information is commonly recognized. According to Ebel the primary objective of education is to assist students in the acquisition of a command of 2? John C&.-JWarshall, "Evaluation: Does it Mean Any­ thing?", Clearing H o use, XLIV (May, 1968), 535-538. 23Wrinkle, "The story of," op. c i t . 31 substantive knowledge. If this is the case, then a mark should represent the degree to which this objective has been attained, i.e., how much knowledge the student possesses. However, if a different objective has been selected, the basis of marks should also be adjusted. Both Linder and Rothney maintain that the basis for marks is not as important as is the fact that the students should know and understand what basis is to be used in determining marks. 25 '26 Seldom has a recommendation been made that marks should be based upon the student's own growth as based upon past performance or capacity. Although the statement that the reliability of growth scores is extremely low, as the variance error of a difference score is the sum of the var­ iance errors of component scores, has often been made, it is highly unlikely that this can totally explain the lack of these recommendations. On the other hand, Melby stated that teachers should 27 evaluate in terms of capacity and growth. However, eval­ uation and marking need not have been simultaneously refer­ enced . ^ E b e l , op. c i t . , pp. 55-69. 2 5 Ivan Linder, "Is There a Substitute for Teachers1 Grades?," American School Board Journal (July, 1940), 25-26. 26 John W. M. Rothney, Evaluating and Reporting Pupil Progress (Washington, D . C . : National Education Association Press, 1955). 27 Melby, op. c i t . 32 Student Characteristics Considered Teachers also must decide when selecting a marking procedure or assigning a mark what student characteristics are to be represented by the mark. While the inclusion of both academic and affective information often occurs, the recommendation has been- made by Marshall that grades be limited in their composition in order to make them compar­ able between students. A system of grading should be used that would enable one to know that an A student is in the upper group of his class, a grade of B would indicate that the student is in the second group, etc. 2 8 Toliem reported that the majority of teachers in­ clude in marks certain characteristics of students, while other characteristics are included less often. Table 2.2 indicates those characteristics that were frequently reported and the percent of teachers who claimed to consider each. It should be noted, however, that several of the factors listed were previously referred to as bases for marks rather than as student characteristics. 29 Variations in the characteristics represented by a mark occur not only in regard to the inclusion of affective 28 Marshall, op. c i t . 29 William Joseph Toliem, "Factors which Affect the Decision Making of Classroom Teachers as they Evaluate Stu­ dents" (Unpublished Ph.D. dissertation, State University of New York at Buffalo, 1972). 33 TABLE 2.2.— Factors Reported Considered by Teachers and the Percent of Teachers Reporting Each. Factor Percent Progress when compared to own growth 92.1 Student interests 84.2 Class participation 82.9 Observation of student 82.9 Student-teacher conferences 80.3 Classroom tests and quizzes 77.6 Classroom behavior 75.0 Self evaluation by student 61. 8 Homework 57.9 Type of group 54.0 Progress when compared to others 44. 7 Reports by counselors, etc. 43.4 Observation of student plus logs 39.5 Achievement test results 32.9 Anecdotal records 19.7 Rating scales 11.8 Opinions of other teachers 6.5 34 factors but also in regard to the measures of cognitive skills on which the mark is determined and the relative importance attached to each. In a study by Russell and Thalman of 335 seventh and eighth grade students, a correlation of .658 was found be ­ tween average marks and the personality rating made by the teachers. The multiple correlation obtained when personal­ ity, ratings, achievement test results, and intelligence test results were correlated with average marks was .810, which indicated that 65 percent of the variance in teachers' marks could be explained by the linear relation of the three measures to the m a r k s . When teachers1 marks were categorized according to A, B, C, D, and E, significant differences were found in the personality ratings for the five groups. Russell and Thalman concluded that: If the findings of this study are true, they should be recognized and either accepted or corrected in terms of our educational philosophy . . . . Serious and permanent damage to a pupil's personality can result from continued failure in school; and if the mark results from a personality conflict between the teacher and the pupil, the act is cruel and unjusti­ fied. A challenge is made to teachers to guard against prejudice and to be alert for personality problems which may cause the pupils who have them to function at a level lower than they might achieve. Recognize those problems for what they are, but avoid allowing them to appear in the disguise of a teacher's m a r k . 30 30 Ivan L. Russell and W. A. Thalman, "Personality: Does it Influence Teachers' Marks?," Journal of Educa­ tional Research, SLVIII (April, 1955), 561-564. 35 On the other hand, in an issue of What Research Says to the Teacher, a publication of the National Educa­ tion Association, Kuthney recommended that the 3C's, char­ acter, cooperation, and courtesy, should be a part of every mark assigned. 31 However, clues are not provided as to how these personality traits are to be assessed. The only available instruments for personality measurement either require a highly skilled psychometrician for their adminis­ tration or are subjective estimation procedures. In ad­ dition, the laws of many states, including Michigan, specifically prohibit the administration of any personality, interest, or attitude instrument without the written consent of the child's parents. A study by Sobel demonstrated the difficulties associated with attempts to include affective factors as well as academic achievement in a mark. When marks do not repre­ sent solely the academic achievement of the students, the following results may be expected: will receive high marks, (1) more girls than boys (2) those receiving high marks would also score high on a measure of the amount of effort put forth, (3) those receiving high marks will have a better attendance record than will those receiving lower marks, and (4) those receiving high marks will be high in self con­ fidence and have a pleasing personal appearance. 3^Ruthney, op. c i t . Therefore, 36 each mark should represent only academic achievement rather 32 than to also include personality characteristics. Even when teachers try to provide a measure of aca­ demic performance, marks were found by Doherty to be signifi­ cantly related to school attitudes, interests, effort, and citizenship, 33 while Fleming found a relationship between marks and school attitudes, chronological age, and the 34 teacher's estimate of intelligence. Miner, in agreement with Fleming, found that varia­ tions were closely ralated to the grade level of the stu­ dents being marked. Teacher assessment of student perform­ ance in the early grades was found to be actually an assessment of citizenship and/or behavior, while the assessment of secondary pupils was a measure of cognitive skills. 35 Apparently teachers find it impossible to completely separate the affective characteristics of students from 32 Frances S. Sobel, Teachers' Marks and Objective Tests as Indices of School Adjustment (New York: Teacher's College Press, 1936). 33William V. Doherty, "A Survey of the Evaluation of Pupil Progress in Selected Elementary Schools in Ohio” (Unpublished Ph.D. dissertation, Ohio State University, 1954) . 3^Cecile W. Fleming, A Detailed Analysis of Achievement in High School (New York: Teachers's College Press, 1925), p. 69. 35Betty Crowther Miner, "Three Factors of School Achievement," Journal of Educational Research, LX (April, 1967), 370-376. 37 their academic performance, when assigning marks. This in­ ability can create serious problems relevant to the inter­ pretation and comparability of m a r k s . Because the measures of affective characteristics are subjective estimations of the degree to which the student possesses a particular char­ acteristic, different characteristics may be included with unequal importance for different students. This fluctuation also will contribute significantly to the low reliability of marks. Effect of Teachers1 Personal Characteristics The question has been raised as to whether certain personal characteristics of the teacher may contribute to the decisions made regarding the basis on which marks are to be assigned and the composition of these marks. Metzner recently pointed out that many of the varia­ tions that occur in marks are due to differences in the degree to which the students accept the middle-class orien­ tation of the school. This finding may explain why girls receive higher marks than boys and why students of higher socio-economic status receive the higher marks. He also found that teachers who come from lower socio-economic back­ grounds exhibited the same patterns of approval and disap­ proval, as did those from higher class backgrounds. In 38 fact, the former showed more authoritarianism and were less sympathetic.3 6 Although Carter in 1930 reported a study by Newton, demonstrating that women teachers assigned higher marks 37 than did men teachers, a more recent study by Arnold pre­ sented contradictory findings, indicating that for fifth and sixth grade students, there was no evidence of differ38 ential marking procedures for men and women teachers. Theimer's investigation dealt with two other personal characteristics of teachers, their own grade point aver­ age and their teaching experience as measured by number of years taught. In neither instance was a significant relationship found to the marking procedures used. 39 The personal characteristics of teachers according to these studies, do not significantly affect the procedures used when assigning marks. It would seem, however, that 36 Seymour Metzner, "Teacher Bias in Pupil Evalua­ tion: A Critical Analysis," Journal of Teacher Education, XXII (Spring, 1971). 37 Robert S. Carter, "How Invalid are Marks Assigned by Teachers?." J. of Ed. P s y ., XLIII (1952), 218-228. 3R Richard Dean Arnold, "The Relationship of Teacher's Sex to Assigned Marks and Tested Achievement among Upper Elementary Grade Boys and Girls" (Unpublished Ph.D. disser­ tation, University of Minnesota, 1966). 3Q William Charles Theimer, "The Relationship between certain Personality and Educational Factors and the Grading Patterns of Secondary School Teachers" (Unpublished Ph.D. dissertation, University of Denver, 1967). 39 their training in the area of measurement would affect their procedures. Although Mosely reported that 52 percent of the teachers studied had taken college courses dealing with re40 search methodology, testing, and measurement, Mayo found that such study taken after graduation did not explain the increase in measurement competency, demonstrated after two years of teaching experience. However, the completion of such study prior to graduation did result in moderate superi­ ority in measurement competency on the part of beginning 41 teachers. In a recent study concerning the requirements estab­ lished by a random sample of 916 colleges and universities in regard to their elementary education curricula, Roeder reported that 57.6 percent of the institutions did not re­ quire the completion of a course in measurement or evaluation. Only 31.4 percent of the institutions required the comple­ tion of a course devoted solely to this topic. Table 2.3 presents the findings of this study in regard to required 42 college courses. ^^Aubrey Howard Moseley, "A Study of Teachers' Per­ ceptions of Factors Related to Educational Research" (Un­ published Ph.D. dissertation, University of Alabama, 1966). <*1Samuel T. Mayo, Pre-service Preparation of Teach­ ers in Educational Measurement (Loyola University, 1967), pp. 62— 63. 42 Harold H. Roeder, "Teacher Education Curricula— r Your Final Grade is F," Journal of Educational Measurement, X (Summer, 1973), 141-143. TABLE 2.3.— Nation-Wide Summary of Specific Methods courses. Course Ho. Req. Sem. Hrs. One-Two Sem. Hrs. Three Sem. Hrs. Four-Five Sem. Hrs. Over Five _ Sem. Hrs. Freq. Pet. Preq. Pet. Freq. Pet. Freq. Evaluation 495 57.6 104 12.1 154 17.9 10 1.2 2 Geography 343 39.9 45 5.2 277 32.2 47 5.5 76 Pet. Freq. Combined , course Pet. Unscored Pet. „ ^ , Total Freq. Pet. Freq. Freq. Pet. .2 62 7.2 33 3.8 860 100.0 8.8 16 1.9 56 6.5 860 100.0 O Art Method Music Method 94 10.9 205 23.8 299 34.8 103 12.0 30 3.5 45 5.2 84 9.8 860 100.0 109 12.7 203 23.6 278 32.2 118 13.7 34 4.0 37 4.3 81 9.4 860 100.0 4 Physical Education 126 14.7 232 27.0 65 7.6 261 30.3 39 4.5 9 1.0 128 14.9 860 100.0 "Religion" 447 52.0 8 .9 55 6.4 36 4.2 269 31.3 0 0.0 153 17.8 860 100.0 41 Studies such as the one by Roeder indicate that teachers often are inadequately trained in the area of marking. Even without the needed knowledge and skills teachers must make frequent and far reaching decisions, when assigning marks to students, about issues concerning which even the experts do not a g r e e . Harris recently published an investigation dealing with the degree of agreement among a group of measurement specialists regarding several of these issues about which teachers must make decisions. Table 2.4 presents a part of his findings concerning several of the issues . ^ 3 If experts cannot agree as to the "best" way to assign marks, the task of the teacher becomes that of de­ vising marking or her needs. procedures which uniquely best serve his It then becomes surprising that marks are as comparable as they are. Effect of Situational Factors Studies concerning differences among the practices of teachers from schools and/or school districts of differ ent sizes and different levels or urbanization are for the most part lacking. Because highly urbanized communities generally contain larger number of minorities, the study 43 Wilbur S. Harris, "Agreement among N.C.M.E. Mem­ bers on Selected Issues in Educational Measurement," J. Ed. Measure, X (Spring, 1973), 63-70. TABLE 2.4.“ Percent of Respondents Agreeing with, Disagreeing with, and Expressing Uncertainty Toward Selected Issues. Issue 3. 6 . 8. 10. 11. 17. 19. Essay tests are more suitable than objective tests for measuring most types of learning. Agree Undecided Disagree 8 14 79 Schools which use marks should adopt and en­ force a clearly defined institutional marking policy. 80 9 11 Students1 academic marks should be determined solely by their academic achievement. 76 12 12 The majority of teachers rely too much on their subjective judgements in marking students. 69 18 13 About the same proportion of high marks should be given to classes of slow learners as to classes of rapid learners. 13 18 69 Factors like attitudes and interests should be used deliberately in determining students' marks. 15 18 67 The acquisition and utilization of knowledge are the primary aims of formal schooling. 74 7 19 TABLE 2.4.— Continued. Issue 20. 22. 23. 29. 37. 39. 40. Agree Undecided Disagree Most teachers use too few appropriate statis­ tical techniques in evaluating and marking students. 76 14 19 "Pass-fail" or "credit-no credit" is more de­ sirable than marking with three or more cate­ gories for academic classes. 22 20 58 "If something exists, it exists in some quan­ tity, and therefore it canbe measured." 64 15 21 Academic marks should be based more on achievement status than on growth or prog­ ress. 54 20 26 Teachers should attempt to evaluate and mark students in such areas as interest, attitudes, and motivation. 35 19 46 Absolute standards are more desirable than relative standards in evaluating and marking students. 37 17 46 Some type of numerical or letter marking system is essential to good educational practice. 45 15 40 44 by Fish, which concluded that when the effects of socio­ economic factors were removed no relationship was found be­ tween racial background and marks, might be an indicator of a lack of relationship between urbanization level and the marking procedures used by the teacher. 44 Another study, dealing with the urbanization factor, was conducted recently by Finn. He observed in a sample of urban and suburban elementary schools that teachers in the urban schools had slightly more experience than did those in the suburban schools, 10.7 years as compared to 9.4 years. This study also involved the grading of two essays by fifth grade teachers with additional false information, provided regarding the student's sex, race, past achievement, and intelligence. The following conclusion was reported. Teachers in the urban environment are relying to a greater extent upon information about the pupils in rating their products. While on the average, the suburban teachers rate all pupi l s ’ essays similarly regardless of race, sex, or ability in­ formation, the urban teachers do not. Yet the suburban teachers unlike their urban counterparts do not have Blacks in their c l a s s e s . ^ Terwillinger found differences in the marking poli­ cies and practices of schools of different sizes. Marks 44 Enrica Fish, "The Relationship of Teachers' As ­ signed Marks among Elementary Grades" (Unpublished Ph.D. dissertation, University of Minnesota, 1969). Jeremy D. Finn, "Expectations and the Educational Environment," Review of Educational Research, XLII (Summer, 1973), 387-410. 45 were lowered as a disciplinary measure in 40 percent of the large schools but in less than was this practice used. 10 percent of the small schools The large schools more often made provision for a separate citizenship or deportment mark. As discussed in the preceding chapter, Terwillinger also found that many of the differences associated with marking prac46 tices were related to the subject area taught. Similar findings were reported by Theimer. 47 Although significant differences have been identi­ fied on the basis of certain situational factors in regard to marking procedures, attempts to determine specific rela­ tionship are lacking. It is not known whether these dif­ ferences are consistent across schools or school districts or whether the differences are associated with particular characteristics of the situation and/or of the teacher. Effect of Teacher Attitudes The educator's beliefs regarding the worthwhile goals of education, according to Ebel, are important determ­ iners of his viewpoints regarding the role of measurement 48 and methods for its implementation. 4 6 Terwillinger, op. cit. ^ 7 Theimer, op. c i t . ^® Ebel, o p . c i t . , pp. 29-41. 46 Much of the recent controversy regarding the fail­ ure of education to live up to its responsibilities can be traced to differences in opinion regarding the appropriate goals of education. cognitive skills? Is the major goal of education to teach Are the goals of education multidimen­ sional or unidimensional? Millman made the following statement, "The key task of our schools is to maximize the amount of a subject each 49 student has 'mastered.'" On the other hand Butterworth insisted that the goals of education should be to insure that every student attains a sufficient degree of self identity to make accurate, wise choices. He further stated that this is not being accomplished because "Marks have reached the point where indeed they are means to an end and, in fact, do represent, and have substituted for, the broader outcomes of education."5® Concurrence of opinion is noted in the statement by Smith and Dobbins that "perhaps the development of (an inproved marking) system awaits wider agreement on the goals of instruction and the prupose of marking." 51 The most frequently stated functions of marks are as follows: (1 ) to motivate students, (2 ) to provide useful 49 Jason Millman, "Reporting Student Progress," Phi De1ta Kappan, LII (December, 1970), 226-230. 50Thomas W. Butterworth, "A Guide to the Evaluation of Pupil Progress with Particular Emphasis on Pupil Marks,." ERIC No. 036 814 (August, 1966). 51 Smith and Dobbin, op. cit., p. 789. 47 information to the pupil and his parents regarding academic performance, (3) to provide useful information for academic and vocational counseling, and (4) to provide information relevant to administrative functions. tioned uses of marks are to: and procedures, Less frequently men­ (1 ) appraise teaching methods (2) instruct students, (3) maintain stand­ ards of performance, and (4) provide a means of reinforcing students either positively or negatively. 52 * 53 * 54 * 55 Differences, however, exist in regard to the ques­ tion of whether marks are assigned in order to provide infor­ mation to teachers, to students, or to parents. Bolmeier reports that marks should benefit the student not the 56 teacher, while Krause reported that the traditional grading systems best served the needs of the parents but were not 52 William Clark Trow, "On Marks, Norms, and Profi­ ciency Scores," Phi Delta Kappan, XLVIIX {December, 1966), 171-173. ^ W i l l i a m A. Liggett, "Are There Better Ways of Eval­ uating, Recording, and Reporting Pupil Progress?," National Association of Secondary School Principals, XXXIV (March, 1950), 79-89. 54 R. C. Billett, Provisions for Individual Differ­ ences in Marking and Promotion (Washington, D . C . : U.S. Office of Education,1932). 55 A. D. Crooks, "Marks and Marking Systems: Digest," J. of Ed. R e s ., XXVII (1933), 259-272. A 5 ®E. C. Babmeier, "Principles Pertaining to Mark­ ing and Reporting Pupil Progress," School Review, LIX January, 1951), 15-24. useful to either the student or the school. 57 In the years between 1940 and 1970, according to Dionne, the principal functions of marks have changed along with viewpoints regard­ ing the basic goals of education. In 1940 the goal of edu­ cation was to provide equality of opportunity, therefore, evaluations were most useful in placement and counseling. In 1970 the goal of education was to maximize each potential and the functions of evaluation were expanded to include the facilitation of instruction, resource allocation, and 58 curricular management. The differences in the appropriate goals of educa­ tion and the function of marks can hardly be differentiated from the philosophical arguments. Therefore, concrete re­ plies are not likely to be forthcoming. During recent years the roles assumed by education have rapidly expanded. In the early days of the public elementary and high schools the single role of the school was to teach the student those subject areas which were not likely to be learned outside the school. Gradually, however, the school has assumed many of the responsibilities for the social and citizenry learning of its students. At the same time, however, there C *7 Dorothy Krause, "A Study of Present Practices and Attitudes Concerning the Letter Grading System" (Unpublished Ph.D. dissertation, University of South Dakota, 1969). Joseph Dionne, Future of Testing: Trends, ERIC 043 669 (March, 1970). A Look at the 49 is not universal acceptance regarding this changing role, as many people believe that the school should concentrate on the successful accomplishment of its former role rather than to assume new responsibilities. Because marks are in­ dicators of the degree to which the student has agreed to the objectives of the school, marks must reflect whatever it is that the school is trying to do. Until the role of the school is clearly defined, the role of evaluation will remain ambiguous. Effect of Other Aspects of the Marking Procedure A lack of consistency between teachers in regard to other aspects of the marking procedures has been shown. Gould, Ohlson, and Bangs and Greene all reported that from 50 to 75 percent of a mark is determined by daily work, from 25 to 50 percent by quiz marks, and from 25 to 33 per. ■ . * . 59,60,61 cent by test marks. It is important then to consider the number of tests or other components that are included in the mark. Accord­ ing to Taylor a large number of daily recitation grades must be included if this is to be an important determiner 59 Herron, op. c i t . 6 ®C. W. Bangs and H. A. Greene, Teachers Marks and Marking Systems (University of Iowa, 1930) . ®^George Gould, "Practices in Marking and Examina­ tions," School Review, XL (1932), 142-146. 50 of the assigned mark because of the low reliability associa­ ted with such measures. In order to insure a reliable mark, the mark should not be based upon only two or three test scores, unless the tests have been proven to have high re­ liabilities . Numerous studies have been conducted to examine the benefits to be gained from using test items, having various formats. However, the results of these studies have been inconclusive, and often contradictory. In a study of first grade teachers Niedermeyer and Johnson found that teachers, operating according to the mastery learning concept, preferred the use of oral measure­ ments. This method, they claimed, had several advantages: (1 ) it was easier to administer, (3) it eliminated copying, (2 ) it required less time, (4) it provided more useful in­ formation regarding the individual child's skills, and (5) the individual attention from the teacher was beneficial. 63 On the other hand, those studies conducted to de­ termine whether teachers can with some uniformity of judg­ ment mark essay type examination items have come to the fi2 J. Carey Taylor, The Reliability of Quarterly Marks in the Seventh Grade of Junior High School (Balti­ more : Johns Hopkins Press, 1931). ®3Fred C. Niedermeyer and Howard J. Sullivan, "Dif­ ferential Effects of Individual and Group Testing Strate­ gies in an Objective-U based Instructional Program," Journal of Educational Measurement, IX (Fall, 1972), 199204. 51 nearly universal conclusion that, in fact, this is not true. The range of scores obtained in this type of experiment gen­ erally includes the maximum number of possible points as well as the minimum number possible. Lawson found this to be true even when the social background, educational prepara­ tion, cultural and industrial environment, level and type of teaching demanded, and experimental background were simi64 lar for all teachers involved in the experiment. The format of items used as contributors to marks may have a significant effect on the marks assigned as well as on the decisions made regarding such marks. Numerous suggestions have been made regarding the way that marks are to be derived and/or distributed. Yet none of these methods has been demonstrated to have practi­ cal significance. Marshall, in his article, showed eleven 65 distributions which were suggested between 1904 and 1924. Others suggest the use of deviation scores and/or standard ^ D o u g l a s E. Lawson, "Scoring of Subjective Tests with Several Vairables Controlled," Elementary School Journal, XXXVIII (February, 1938), 450-457. ^M a r s h a l l , op. c i t . scores 66 67 69 69 ' ' ' Yet: the practical application of these procedures are not offered. While differences in the other aspects of the mark­ ing procedure has been demonstrated, an attempt to determine whether there is a relationship between these aspects and the basis on which marks are assigned and/or the composition of these marks is lacking. School Marking Policies Most school districts regardless of size have re­ ported that they use a five category marking system. How­ ever, the number of categories commonly ranges from two to fifteen. Starch in the early part of the century supported the use of only a few categories because " . . . on the ordi­ nary percent scale a difference of 7 percent or possibly 5 percent is the smallest that can be determined with reasonable reliability." 70 ^ G e o f f r e y P . M a s o n , Studies and Rep o r t s ; An Em­ pirical Analysis of a System of Achievement Gra d i n g , ERIC 014 131 (132), November, 1965. 67 Ralph P. Newsome, "Assignment of School Marks," Texas Outlook, XXV (August, 1941), 39-40. 6 fi E . S . Thorndike and E . O. Bregman, "On the Form of Distribution of Intellect in the Ninth Grade," Journal of Educational Research, X (November, 1924), 271-278. ®®William A. Wetzel, "The Use of the Normal Curve of Distribution in Estimating Students' Marks," School Review, XXIX (May, 1921), 373-378. 70 Daniel Starch, Educational Psychology (New York: MacMillan, 1927). 53 Bramlette offered five possible benefits to be de­ rived from the use of a two category marking system: increases emphasis on learning, marking, (1 ) (2 ) decreases emphasis on (3) encourages the poorer student, dents to evaluate themselves, and (4) forces stu­ (5) encourages better attitudes in parents who want a superior child but have instead an average c h ild . 7 1 Support for a marking system having several cate­ gories came from Ebel who stated that "Regardless of the inaccuracies of the basis for grading, the finer the scale used for reporting grades, that is the more different grade levels it provides, the more accurate the grade reports will b e . ” 7 2 In a study conducted to assess the effect of adding + and - grades to the existing five-point scale, thereby, creating a twelve point scale, Philbrick and O'Donnell found the greatest effect for those studentB at the lower end of the grade distribution. In a significant number of cases, the former C student dropped to either a C- or D+ and the D student dropped to D - . Therefore, it was concluded that the more precise scale served to lower grade point averages 71 Metle Bramlette, "Is the S and U Grading System Satisfactory or Unsatisfactory?," Texas Outlook, XXVI (April, 1942), 29-30. 72 *• Robert L. Ebel, "The Relation t>f Scale Fineness to Grade Accuracy," J. Ed. Measure, VI (Winter, 1969), 217-221. 54 and, therefore, would not increase the students' 73 cept and motivation as had been intended. self con- The introduction of the dual marking system, in which two separate sets of marks are assigned, one for aca­ demic performance and the other for affective considera-. tions, was one attempt to improve marking by restricting the elements included in the academic m a r k . ^ In an investigation of the effects of single and dual marking systems on the academic performance of gifted sixth grade pupils, Housdorff and Farr reported no associa­ ted differences, thereby, concluding that a dual system offers no advantage over a single mark. 75 Similar conclusions were reached by Halliwell and Robitaille, when they reported a significant correlation of .68 system. between the two or more marks assigned under a dual If a dual system is to increase the reliability and validity of academic marks, teachers must be provided with 73 Joseph L. Philbrick and Patrick I. O'Donnell, "Precision in Grading Practices— Panacea or Problem?," Journal of Educational Research, LXII (December, 1968), 173-176. T A John Lund, "More Truth about Marks," Journal of Education (1929), 609— 619. ^ H e n r y Hounsdorff and S. David Farr, "The Effect of Marking Practices on the Marks of Gifted Sixth Grade Children," Journal of Educational Research, LIX (December, 1965), 169-172. 55 adequate methods for the assessment of the affective fac­ tors.^® Other attempts to improve the reporting of student performance have been the substitutions of conferences for at least a part of the written reports. The Kentucky School Journal reported that 40 percent of the surveyed teachers used scheduled conferences. Three times as many elementary as secondary teachers used conferences. Conferences are used more frequently in the Middle and Western states than in the Southeast or Northeast. "Conferences open up new avenues of communication and are especially helpful in cop­ ing with subjective areas such as attitudes and inter77 actions with the group." Still other attempts have used checklists to re­ place the former methods. One point then can be conclusively stated in regard to school policies. Differences are found in this area just as they are found in all other sectors of the marking of students. The single "best" way has not yet been found. 76Joseph W. Halliwell and Joseph P. Robitaille, "The Relationships between Theory and Practice in a Dual Report­ ing Program," Journal of Educational Research, LVII (Novem­ ber, 1963), 137-141. 77 Editorial, "Report Cards or Parent-Teacher Con­ ferences: Which is Better?," Kentucky School Jou r n a l, IXL (November, 1970), 23. 56 Summary As the number of students enrolled in public ele­ mentary and secondary schools increased during the latter decades of the nineteenth century and the early decades of the twentieth, there was an accompanying need for a means of written reporting of student progress and performance in regard to how well the goals of the school were being accomplished. It was in response to this need that marking practices and policies were developed. However, despite numerous changes in an attempt to develop better methods, many criticisms of marking have been voiced. Two primary philosophical arguments have been pre­ sented against the practice of assigning marks. One of thse arguments concerns the question of the intent of marks and the other deals with the use of marks as rewards. Marks have sometimes been construed as means of maintaining a stratified social system. It has also been claimed that marks are harmful as they destroy the natural love of learn­ ing by becoming the sought-after reward. These arguments, just as many philosophical arguments, proceed from a stated assumption, in this case, that marks are not accurate, valid indicators of student performance. This assumption is not proven by the argument but is merely to be accepted as a fact. If, in fact, marks are not accurate, valid indicators 57 of student performance, then the intent of marks may be questionable and marks probably should not function as re­ wards . However, as long as marks cannot be conclusively proven to be lacking in accuracy and in validity, the claim is not supported. The methodological and technical criticisms have, for the most part, emphasized the need for well-defined, clearly interpretable marking procedures. Eight alterna­ tives to the commonly used practice of assigning symbols somewhat arbitrarily to represent performance were pre­ sented as follows: evaluation, (3) tract marking, ing, (1 ) written evaluation, (2 ) self marks not known by students, (5) mastery marking, (4) con­ (6 ) pass/fail mark­ (7) credit/no credit marking, and (8 ) blanket marking. None of these methods are without a number of advantages and disadvantages, therefore, no single "best" method has been recommended. Support has been presented for each of the three most frequently used bases for marks. Marking on the basis of comparison to the performance of others or competitive marking has received both approval, as being a more ac­ curate representation of adult life, and disapproval, as being unfair to the less able students. Marking based upon the students* mastery of the subject matter has been criticized as being appropriate to only a limited number 58 of situations and then only where specific, narrowly defined goals or objectives have been stated. On the other hand, if the primary goal of education is to assist students in the acquisition of a command of a body of substantive knowl­ edge , then marks should indicate how much knowledge the student has attained. Attempts to mark according to the growth of the individual student have not been successful due to problems regarding the measurement of change. How­ ever, this latter position has been claimed by many to be the most defensible basis for marking. Not only must teachers decide upon the basis which they will utilize when assigning marks but they must also decide upon the student characteristics that will be rep­ resented by the mark. There is much evidence to support the statement that even when teachers believe that the marks they assign represent purely cognitive functioning there is a high correlation between these marks and ratings made of affective student characteristics. The primary associa­ ted difficulty is that subjective ratings of affective characteristics possess low reliability either between points in time or between different raters. Although findings do not support the statement that differences in marking procedures are associated with the personal characteristics of the teachers such as sex, age, experience, college degree, grade point average, or the 59 number of measurement courses taken, there is some incon­ clusive evidence that situational factors such as the urbani­ zation level of the community, size of the school district, percentage of minority students, etc., may be associated with the differences. Significant relationships have been found between the grade level of the students involved and the subject area and the marking procedure used. Evidence has been presented that teachers differ regarding their beliefs concerning the appropriate goals of education and the functions which marks serve. However, the effect which these factors have in the determination of marking procedures has not been investigated directly. Nor have attempts been made to determine the relationship between marking procedures and other aspects of the marking procedure such as format of test items, frequency of testing, or the use of particular statistical techniques. Teachers must operate within the framework provided by the school administration when they assign marks. There­ fore, because school districts are known to differ in re­ gard to their policies for marking, it can be assumed that teachers must vary accordingly. Two major conclusions have been derived from this review of literature. The first is that differences exist in regard to how students are marked. These differences exist at the teacher, the school and the school district 60 level. Evidence does not clearly support the existence of relationships between these differences and the personal characteristics of the teachers, the situational factors, the attitudes of the teachers, or other aspects of the mark­ ing procedure used. The second conclusion is that no single "best" method for the reporting of student performance has been developed. Overview This chapter has presented a review of the literature relevant to the topic of study. Chapter 111 includes a summary of the procedures that were used for data collection. Chapter IV presents the data analyses and the derived find­ ings of the study. Chapter V presents the conclusion and the recommendations which were derived from the study. CHAPTER III DESIGN OF THE STUDY The purpose of this chapter is to describe the target population/ to define the sample and sampling proced­ ures, to describe the instruments used in data collection, and to specify the statistical methodology which was utilized. Population and Sample The target population included those teachers who were assigned on a regular basis to a classroom in a public school within the State of Michigan for the school year 1972-73. Teachers of grade levels kindergarten through grade twelve were included. Excluded were all teachers of special education classes regardless of type. Also excluded were all substitute teachers regardless of the length of their assignment, unless a school district officially listed their names on a roster of teachers assigned to classrooms on a regular basis. A wide variation was found within the population in regard to the total number of teachers employed by each dis­ trict, ranging from a minumum of 4 teachers per district to the maximum of Detroit with approximately 11,000 teachers. 61 62 Therefore, in order to maximize the probability of the selection of school districts at or near the extremes of the distribution as well as districts within the midrange of enrollment, stratified sampling procedures were used. The variance due to the size of the school districts on the variables of interest was expected to be reduced within each strata, thereby, improving the precision of the results. Each district within the state was allocated on the basis of the total number of teachers employed to one of five strata, four of which contained approximately equal numbers of school districts, plus one stratum that contained those nine school districts employing the largest numbers of teachers. A three-stage sampling procedure was employed. Stage one included the selection of school districts for inclusion in the study. Stage two included the selection of schools and stage three the selection of teachers to receive the mailed questionnaires. The Michigan Education Directory for 1972-73 served as the sampling frame for the first and second stages. Proportional allocation, based upon the fraction of the total population assigned to each stratum on the basis of size, determined the number of school districts to be sampled from each stratum. In this way, the proportion of the sample included from each stratum was equal to the pro­ portion of the population within that stratum. An exception 63 was made, however, in regard to the stratum containing the 9 large districts. All of the school districts allocated to that stratum were sampled. Because the number of teachers employed by the Detroit Public School District was approxi­ mately six times that of the next largest district, Detroit was subdivided into five subdistricts on the basis of geo­ graphic proximity, as determined by school addresses. The total number of teachers within each subdistrict was approxi­ mately equal to the number for the other districts within the same stratum. Each subdistrict was sampled. In this way, a total sample was created that was an accurate repre­ sentation of the State of Michigan in regard to the distribu­ tion of teachers according to the size of the districts. Table 3.1 shows the population and sample distribution accord­ ing to stratum. A simple random sample of school districts was se­ lected from each stratum with the number of districts per stratum determined according to proportional allocation as described above. The second stage involved the selection of six schools from each sampled district. Two of these schools were senior high schools, two were junior high or middle schools, and two were elementary schools, thereby, insuring that all levels were represented in the sample. The infor­ mation within the sampling frame was assumed to be correct 64 TABLE 3. 1 .— Target Population Distribution and Sample Accor< ing to Strata I through V. Stratum Stratum Boundaries Number of Districts in Population Proportion of Population Number of Districts in Sample 1-50 12 9 .242 31 II 51-100 168 .316 41 III 101-200 124 .233 30 IV 201-1000 102 .101 25 V Over 1000 __ 9 .018 13* 532 1.001 I TOTAL 140 2 ** 534 *Includes 5 subdistricts for Detroit. **Districts for which required information was not available. in regard to the classification of schools by level. When­ ever a sampled district has only one or two schools of a specified level, those schools were automatically sampled. No elementary school, however, was sampled that contained fewer than three grade levels. In the case of a district that included no junior high or middle school, only four schools were sampled, two senior high schools and two ele­ mentary schools. Whenever more than two schools of a spe­ cified level were listed, simple random sampling was used to choose those schools that were included in the study. Table 3.2 shows the allocation by school level and the number of schools per level according to strata. 65 TABLE 3. 2 .— Sample Distribution According to Level of Schools by Strata I through V. Stratum Total Number of Schools Number of High Schools Number of Junior High or Middle Schools Number of Elementary Schools I 75 30 9 36 II 131 40 25 65 III 117 30 27 60 IV 124 30 38 42 V 79 26 26 27 156 125 230 TOTAL 526* *The total number of schools (526) is not equal to the total number of high schools, junior high or middle schools, and elementary schools due to schools which included more than one level and, therefore, were included only in the total number of schools. The third stage involved the selection of two teach­ ers from each of the selected schools. Two teachers per school was considered to be adequate as teachers were to be the unit of analysis rather than schools or school districts. Teachers were chosen as the unit of analysis because pre­ vious studies demonstrated both the lack of district wide marking procedures and the resistance of teachers to abide by such procedures even when they were established. The sampling frames for this stage were rosters of teachers' names maintained by the selected school administrators. A 66 simple random sampling procedure was used to select teachers for inclusion in the study. Data Collection and Instrumentation Table 3.3 summarizes the source of information for each variable of interest. The primary source was the teachers' responses to the mailed questionnaire. This instrument was developed especially for this study. Approximately 15 minutes were required for completion of the twenty-three items included in the questionnaire. One item allowed for free response, while the others offered several options each. naire included an identification number. Each question­ A copy of the questionnaire is included in Appendix A. Table 3.4 presents the reliability coefficients of the questionnaire computed using Cronbach's Alpha for groups of items for which it was appropriate. Because a reliability coefficient would not provide useful information for the free response items used, those items were not included. Also, because a reliability coefficient computed on strictly demographic items is meaningless, these items were also ex­ cluded from the calculation. The reliability coefficients reported were based upon the particular sample of responders and, for this reason, do not imply cross validation. For these reasons, the reliability coefficients were calculated 67 TABLE 3.3.— Sources of Information Regarding the V a r i a b l e s o f Variable Source Attitudinal factors appropriate goals of education functions of marks suggested changes Questionnaire Questionnaire Questionnaire Marking policies frequency symbols used dual system overall formal Report Report Report Report Marking procedures basis type objective measures frequency objective measures frequency of use of statistics student characteristics considered relative importance o f student characteristics Interest (8 items) (9 items) (free) form form form form Questionnaire Questionnaire Questionnaire Questionnaire Questionnaire (1 item) (2 items) (3 items) (3 items) (14 items) Questionnaire (14 items) Personal factors sex ag e experience degree held academic success courses taken Questionnaire Questionnaire Questionnaire Questionnaire Questionnaire Questionnaire (1 (1 (1 (1 (1 (1 Situational factors size d i s t r i c t size school composition urbanization m i n o r i t y level grade level taught subject area taught Education Directory Education Directory Education Directory Assessment Program Report Assessment Program Report Q u e s t i o n n a i r e (1 item) Q u e s t i o n n a i r e (1 item) item) item) item) item) item) item) TABLE 3.4.— Cronbach's Alpha Reliability for Appropriate Groups of Items by Strata I through V. Item Numbers Variable I II III IV V Total Frequency of objective measurements 9-10-11 .307 .342 .455 .431 .619 .431 Format of measurements 12-13 .362 .805 .458 .439 .333 .479 16-17-18 .467 .370 .242 .470 .512 .412 Goals of education 19 .896 .871 .483 .841 .904 .799 Functions of marks 20 .840 .827 .786 .991 .863 .861 .759 .838 .544 .899 .813 .771 Frequency of use of statistics TOTAL 69 only for those items which comprise five of the measures: (1) the frequency of use of objective measurements, format used for measurement items, of statistics, (2) the (3) the frequency of use (4) the appropriate goals of education, and (5) the function of marks. Statements included within the item pertaining to the teacher*s viewpoint concerning the appropriate goals of education were divided into two categories of four state­ ments each. Four of the statements were most likely to be rated highly by someone espousing the traditional belief and the other four were most likely to be rated highly by someone favoring the progressive or humanistic belief. Scores were totalled for the eight items included in the two categories, with those in the traditional category being assigned positive values and those in the progressive category being assigned negative val u e s . This permitted each respondent to be placed at a point on a continuum, rang­ ing from High Traditional to High Progressive. Those re­ spondents who selected equivalent response for both of the categories w e r e , in this w a y , assigned to an Ambivalent position. The statements concerning the functions that marks successfully serve were separated into three categories. Three statements dealt with functions relevant to the stu­ dent himself; three dealt with the provision of useful infor­ mation to the parents; three dealt with functions relevant 70 to the teacher. As respondents were asked to indicate their position on a 0 to 5 scale, each of the three category scores ranged from 0 to 15. In order to compute a value representing the fre­ quency of the use of objective measures of student perform­ ance, the three items dealing with the frequency with which homework is assigned, the frequency of quizzes, and the frequency of longer tests were combined as shown in Table 3.5. The summation of the values for the three items yielded possible scores ranging from 0 to 14. TABLE 3.5.— Scoring Procedure for Items Dealing with Frequency of Measurement Value 0 Homework Frequency never Quiz Frequency never Test Frequency never 1 or 2 times marking period 1 2 end of unit end of unit 3 less than 1 or 2 weekly less than 1 or 2 weekly every 2 or 3 weeks 4 1 or 2 weekly 1 or 2 weekly 1 or 2 weekly 5 daily daily Similarly, to obtain a value representing the fre­ quency of use of particular statistical techniques, standard scores, percentile ranks, and a normal distribution. 71 responses were first combined into three overall categories, always, sometimes, and never, each category being assigned a value of 3, 2, and 1 respectively. In this way, a possi­ ble score ranging in value from 0 to 9 was obtained as an indication of the total frequency of use of these statisti­ cal techniques. The Michigan Educational Assessment Program Report provided information regarding the five categories into which school districts were separated on the basis of the urbani­ zation level of the community. The five categories are metropolitan, urban fringe, city, town, and rural, with 15, 24, 105, 119, and 271 school districts in the population per respective category. this study. Similar divisions were used in Table 3.6 presents the distribution of responses for this variable. TABLE 3.6.— Distributions of Teachers' Responses by Urbani­ zation Level According to Strata. Stratum Metropolitan Urban Fringe I 0 II III 0 2 36 15 54 117 32 71 196 IV V Total Number of Teachers 4 7 Town Rural 2 10 34 11 35 97 15 89 135 22 0 0 0 0 46 158 246 City 9 72 In regard to the measure concerning the composition of marks, the relative weightings assigned specific student characteristics, ranks were computed for each respondent. However, the original percentage values reported were also retained. For the other measures obtained from the question­ naires, the categories used were those included as response options. In order that quantitative types of analyses be utilized, dummy variables were created for each level of the categorical variables. Dummy variables were assigned a value of 2 if the response belonged to the particular level; otherwise, they were equal to 1. Report forms were analyzed according to school level on the following bases: (1) is a dual system used, teacher comments included, used, (2) are (3) is a check list of objectives (4) is a narrative report used, (5) how many times per / year is it issued, (6) which of the following is used; A-B-C-D-E, S-U, or S-A-I-U, (7) is it returned to teacher signed by parent, and {8) is there a place for effort and/or conduct. Responses to the free-response items dealing with suggested changes in marking procedures were divided into the following categories: rative, (3) pass-fail, (1) eliminate entirely, (4) conferences and reports, tem or school wide procedures, (2) nar­ (5) sys­ (6) more emphasis on academic. 73 (7) more emphasis on affective characteristics, (8) other, and (9) no change. Design Because many school boards control external re­ search and experimentation, the first step was to secure their approval. School superintendents, as representatives of the school boards, were sent letters stating briefly the purpose of the study, its importance, the amount of time and involvement requested of the personnel in the district, and a request made for their cooperation. A request was made for a list of the names and addresses of the teachers assigned within each of the sampled schools and for an un­ used copy of the student progress report form used in each of these schools. Appendix C contains a copy of the origi­ nal letter and a copy of the follow-up letter mailed three weeks later. Questionnaires were then mailed directly to these teachers. Appendix E includes a copy of the enclos­ ure letter sent to these teachers as well as follow-up letters. The response rate from superintendents was 55.1 per­ cent for the total sample with two superintendents declin­ ing to participate; therefore, another means was needed to make contact with teachers. To accomplish this, letters were sent to the principals of the selected schools in nonresponding districts, requesting that they distribute 74 the accompanying questionnaires. Directions were provided specifying the method by which teachers were to be chosen to receive the questionnaires. A roster of teachers' names was to be numbered consecutively, excepting special educa­ tion teachers and all other special services personnel. The accompanying questionnaires were addressed to teacher number ___ . These numbers had been randomly selected from a random number table based upon the available information regarding the number of teachers per building. Appendix D contains a copy of the letter sent to the principals and the follow-up letter sent two weeks later. Table 3.7 shows the distribution of responses for both those receiving ques­ tionnaires directly and those receiving them through princi­ pals. After the deadline for the returns from the followup letters, the response rate for teachers was 60.8 percent. From the remaining non-responders a 25 percent sample was chosen to receive an extra effort in an attempt to elicit their responses in order to determine whether the non­ responders differed on the measures of interest from the responders. A minimum of 25 non-responders were selected from each stratum to receive the extra effort, which con­ sisted of registered letters and telephone calls. Table 3.8 presents the response rate for the regu­ lar mailing and for the extra effort. It is apparent that the extra effort was more successful with teachers from the TABLE 3.7.— Dual Sampling and Response Rates by strata and by Total. iu ^ Strata II III IV Number _ Sent to _ ^ Supt. j Number . „ _ received Percent from response „ „ Supt. Number of teachers _ , sent from _ _ Supt. Number of , teachers response _ from _ Percent response Number of . teachers „ . sent to . principals Nuttber of . teachers response / from _ Percent response Total Total _ „ , „ . , , . Total Number of Number of _ „ _ Percent teachers teacher . response sent response 31 14 45.2 48 38 79.2 100 68 66.0 148 104 70.4 41* 23 56.1 {x 1 refusal) ( 6 teachers) 144 106 73.6 112 73 60.9 256 179 70.0 ■si Ul 30 20 66.7 142 99 69.7 92 63 74.0 235 167 71.4 25* 14 56.0 (+1 refusal) (10 teachers) 84 61 72.6 142 120 84.5 226 181 80.0 44.0 34 20 58.8 124 66 53.2 158 86 54.5 55.1 452 324 71.6 570 393 69.0 1022 717** 70.15 13(9) Totals tLa a ta a a x a ij-a i 140 75 *Indicates the number of superintendents who replied and declined to participate plus the number of teachers who would have been included in the study. •*Ihis total includes response from each sampled school district with the exception of those indicated in column two. TABLE 3.8.— Responses from Regular Mailing and from Extra Effort by Strata and by Total Total Strata Number of Teachers Number of Early Responses Percent Early Responses Total Number Receiving Extra Effort Number of Responses from Extra Effort Percent Response from Extra Effort I 148 93 62.9 25 11 44.0 II 256 165 64.5 25 14 56.0 III 234 139 59.4 28 28 100.0 IV 226 161 71.3 27 20 74.0 V 158 63 39.9 26 23 88.5 1022 621 60.8 131 96 73.3 Totals 77 larger school districts. This may have been caused by the fact that the smaller school districts were closer to the end of their school year than were the larger districts at the time that the extra effort was being made. However, with the exception of the two school districts for which the superintendent declined, response was obtained from all sampled districts. In order to determine whether there were significant differences between the responses to the items pertaining to the relative importance of student characteristics from the regular mailing and those elicited by the extra effort, multivariate and univariate analyses of variance were calcu­ lated. Table 3.9 presents the results of these tests, in­ cluding the computed F ratios, the degrees of freedom, and the level of significance. As is shown, no significant multivariate or univariate F ratio was found at the .01 level. Therefore, the assumption was made that the respond­ ents were an unbiased sample of the population of teachers in Michigan. The two groups, respondents from the regular mailing and extra effort, were combined in Table 3.7 which indicates the total response rate as well as the response rate from each of the two sampling procedures. A total response rate for teachers of 70.15 percent is reported. The response rate for Strata I is reported as 70.4 percent. The response rate for Strata II is 70.0 percent and for Strata III is 71.4 percent. The rate for 78 TABLE 3.9.— Multivariate and Univariate F-Ratios for Composi­ tion of Marks and Whether Response Received from Regular Mailing or from Extra Effort for Stratum I through V. F-Ratio Significance DF 13/90 Overall 1.3115 .221 Attendance Appearance Conduct Effort Participation Attitudes Neatness Homework Quizzes Tests Optional Group Reports Individual Reports .3894 .1565 .0005 1.7952 .1337 .2191 .4754 3.1034 .8958 .6450 .0244 4.1086 1.1432 .534 .693 .983 .183 .715 .641 .492 .081 .346 .424 .876 .045 .288 1/103 1/103 1/103 1/103 1/103 1/103 1/103 1/103 1/103 1/103 1/103 1/103 1/103 Overall 1.2860 .226 13/164 Attendance Appearance Conduct Effort Participation Attitudes Neatness Homework Quizzes Tests Optional Group Reports Individual Reports 1.7908 5.8821 5.6242 .2944 .1105 .6081 3.4906 .7146 2.4858 1.9959 5.3641 2.3343 2.5419 .183 .016 .018 .588 .740 .437 .063 .399 .117 .159 .022 .128 .113 1/178 1/178 1/178 1/178 1/178 1/178 1/178 1/178 1/178 1/178 1/178 1/178 1/178 .3981 .969 13/153 .0036 .0033 .0101 2.0113 1.7808 .2024 .1321 1.3935 .3772 .952 .954 .920 .158 .184 .653 .717 .240 .540 1/166 1/166 1/166 1/166 1/166 1/166 1/166 1/166 1/166 ■Overall Attendance Appearance Conduct Effort Participation Attitudes Neatness Homework Quizzes 79 TABLE 3.9.— Continued Tests Optional Group Reports Individual Reports Stratum IV -Overall Attendance Appearance Conduct Effort Participation Attitudes Neatness Homework Quizzes Tests Optional Group Reports Individual Reports Stratum V -Overall Attendance Appearance Conduct Effort Participation Attitudes Neatness Homework Quizzes Tests Optional Group Reports Individual Reports F-Ratio Significance .3259 .0431 .0030 .0812 .569 .836 .956 .776 1/166 1/166 1/166 1/166 .8031 .656 13/167 .1476 1.3820 2.4136 1.2595 .0151 1.4282 .0077 .5136 1.7210 .8812 .5704 1.0769 .7437 .701 .241 .122 .263 .902 .234 .930 .475 .191 .349 .451 .301 .390 1/180 1/180 1/180 1/180 1/180 1/180 1/180 1/180 1/180 1/180 1/180 1/180 1/180 1.1083 .366 13/72 .2310 .7538 .4426 3.4092 .1849 .7222 .6482 .0021 2.5567 .0004 5.9661 1.9576 1.2782 .632 .388 .508 .068 .668 .398 .423 .963 .114 .983 .017 .165 .261 1/85 1/85 1/85 1/85 1/85 1/85 1/85 1/85 1/85 1/85 1/85 1/85 1/85 DF 80 Strata IV is reported as 80.0, while that for Strata V is 54.5 percent. When the questionnaires were sent directly to the teachers, the response rate was higher for Strata I, II, and V, while for Strata III and IV the higher response rate was obtained from the teachers who received the question­ naires through the principals. In order to determine whether significant differ­ ences were associated with to whom the questionnaire was sent, multivariate and univariate analyses of variance were calculated. Table 3.10 shows the F-ratios for these analy­ ses, the degrees of freedom, and the significance level. As is shown, only for Stratum II was a significant multivariate difference found at the .01 level. Because this resulted only from a single univariate significance, no further consideration was made of this difference. Analysis Methodology The range of all measures was restricted by elimin­ ating those item options or values reported by fewer than two percent of the respondents, i.e., were not reported by at least two respondents in Strata I and V, by three in Stratum III, or by four in Strata II and IV, The respondents who re­ ported these options were assigned to either the next higher or the next lower category. All of these changes assigned the respondents to the category closer to the mean for th$ particular measure. 81 TABLE 3.10.— Multivariate and Univariate F-Ratios for Composition of Marks and to Whom the Question­ naires were Sent for Stratum I through V. Stratum I -Overall Attendance Appearance Conduct Effort Participation Attitudes Neatness Homework Quizzes Tests Optional Group Reports Individual Reports Stratum II -Overall Attendance Appearance Conduct Effort Participation Attitudes Neatness Homework Quizzes Tests Optional Group Reports Individual Reports Stratum Ill-Overall Attendance Appearance Conduct Effort Participation Attitudes Noatnnna F-Ratio Significance .4916 .924 .7422 .2132 .0395 .0636 1.2899 .9444 .7653 .0381 .0004 .2239 .0094 .8074 2.5558 .391 .645 .843 .802 .259 .333 .384 .846 .984 .637 .933 .371 .113 2.4093 .005* 13/165 1.3289 .3187 .5382 1.2335 .2942 .3303 1.3059 7.5443 2.9895 1.4250 .0161 .4748 .1230 .251 .573 .464 .268 .588 .566 .255 .007* .086 .234 .899 .492 .726 1/178 1/178 1/178 1/178 1/178 1/178 1/178 1/178 1/178 1/178 1/178 1/178 1/178 DF 13/90 1/103 1/103 1/103 1/103 1/103 1/103 1/103 1/103 1/103 1/103 1/103 1/103 1/103 .5941 .856 13/153 .0444 .5671 .4494 .2025 .0084 .0783 .3213 .833 .452 .504 .653 .927 .780 .572 1/166 1/166 1/166 1/166 1/166 1/166 1/166 ^Indicates significance at a = .01. 82 TABLE 3.10.— Continued. Homework Quizzes Tests Optional Group Reports Individual Reports Stratum IV -Overall Attendance Appearance Conduct Effort Participation Attitudes Neatness Homework Quizzes Tests Optional Group Reports Individual Reports Stratum V -Overall Attendance Appearance Conduct Effort Participation Attitudes Neatness Homework Quizzes Tests Optional Group Reports Individual Reports F-Ratio Significance 2.3661 .0562 .3865 .3726 .1316 .9766 .126 .813 .535 .543 .717 .324 1/166 1/166 1/166 1/166 1/166 1/166 1.2421 .254 13/167 3.9622 .0008 1.2953 1.3974 .8588 2.8497 4.2732 1.0128 .1011 .2335 .0822 1.0469 .0121 .048 .978 .257 .239 .355 .093 .040 .316 .751 .630 .775 .308 .913 1/180 1/180 1/180 1/180 1/180 1/180 1/180 1/180 1/180 1/180 1/180 1/180 1/180 .9785 .481 13/72 .1512 1.0513 .1259 5.5506 .8838 .0067 .2338 .0926 .7784 .0431 .0000 .1299 .0053 .698 .308 .724 .021 .350 .935 .630 .762 .380 .836 .998 .719 .942 1/85 1/85 1/85 1/85 1/85 1/85 1/85 1/85 1/85 1/85 1/85 1/85 1/85 DF 83 In order to answer the questions which were defined as the objectives of the study, statistical analysis tech­ niques were utilized. Kendall's coefficient of concordance was used to answer the following questions: Do teachers agree on the relative importance of stu­ dent characteristics considered when assigning marks? Chi square analyses were utilized to answer the following questions: What personal factors are related to the basis on which marks are assigned? What situational factors are related to the basis on which marks are assigned? What attitudinal factors are related to the basis on which marks are-assigned? What other aspects of the marking procedures used are related to the basis on which marks are assigned? Canonical correlational analyses were utilized to answer the following questions: What personal factors of teachers are related to the weights applied to student characteristics considered when assigning marks? What situational factors are related to the weights applied to student characteristics considered when assigning marks? What attitudinal factors are related to the weights applied to student characteristics considered when assigning marks? What other aspects of the marking procedure used are related to the weights applied to student character­ istics considered when assigning marks? 84 A Chi square one sample test was employed in order to determine whether responses regarding the basis on which marks were assigned formed a uniform distribution. Do teachers agree in regard to the basis on which marks are assigned? The following remaining questions were answered by means of summary tables. What types of objective measure of student performance are made by teachers? How frequently are objective measurements of student performance made by teachers? What changes do teachers suggest regarding marking procedures and policies? How do school districts differ in their marking policies? The first step in the analysis of the data was to determine whether differences existed between the strata according to the basis on which marks were assigned and the student characteristics that were considered when as­ signing marks. A Chi square test for independence was utilized to determine whether there was a relationship between the stratum and the basis on which marks were assigned. resulting information is presented in Table 3.11. The The results were not significant at the .05 level of signifi­ cance. Therefore, for this purpose of this study it may be said that the basis upon which marks are assigned is independent of the stratum from which the teacher was sampled. 85 TABLE 3.11.— Contingency Table for Stratum and the Basis on Which Marks Are Assigned. I II III IV V 9 23 19 23 11 85 13 25 15 17 11 81 5 2 3 5 4 19 Growth Mastery 40 76 90 83 38 327 Growth Performance 11 18 16 19 6 70 Mastery Performance 27 31 19 25 9 111 0 4 5 8 7 24 104 179 167 181 86 Growth Mastery Performance None TOTAL Total X2 = 36.115 df = 24 not significant at a = .05. In order to determine whether the stratum had a significant effect upon the student characteristics which were considered when marks were assigned multivariate and univariate F-ratios were calculated. shown in Table 3.12. These results are As shown , both the overall multi­ variate F-ratio and all univariate F-ratios indicate that the teachers' choice of student characteristics for con­ sideration when assigning marks is not affected by the stratum, based upon school district size within which they teach. 86 TABLE 3.12.— Multivariate and Univariate F-Ratios for Composition of Marks and Stratum. F Overall Attendance Appearance Conduct Effort Participation Attitudes Neatness Homework Quizzes Tests Optional Group Reports Individual Reports Significance 1.1560 .208 .8781 1.4776 .8137 1.032 1.240 1.428 .7650 1.0416 1.4855 .8210 1.1200 1.3359 .8398 .477 .207 .517 .390 .292 .223 .548 .385 .205 .512 .346 .255 .5002 DF 52/2720.945 4/714 4/714 4/714 4/714 4/714 4/714 4/714 4/714 4/714 4/714 4/714 4/714 4/714 Because the results of the Chi square test for in­ dependence for the basis for marks, the multivariate analy­ sis of variance for the overall student characteristics considered, and the univariate analysis of variance for each of the student characteristics considered indicated that the basis for marks and the student characteristics represented by marks were independent of the stratum from which the teachers were sampled, the decision was made to eliminate the consideration of strata from the remaining portions of the study. Therefore, the analyses of the data, intended to derive answers to the questions which are the objectives of the study, utilized the entire sample of 717 respondents. 87 Overview This chapter has presented a description of the population and sample, a description of the instrument used for data collection, and a description of the procedures employed for data collection and analysis. sents the data analyses and findings. Chapter XV pre­ Chapter V presents the conclusions and recommendations that were derived from the study. CHAPTER IV DATA ANALYSES AND FINDINGS OF THE STUDY In this chapter is presented an explanation of the data analyses that were conducted and the findings that were derived. The first section includes a brief description of the statistical techniques which were utilized. The second section provides the statistical findings of each data analy­ sis and the related interpretation of the findings. The third section provides an overall summary of the results of the study. Statistical Methodology Cannonical correlation analysis allowed the researcher to test the independence of two sets of variables in the mul­ tidimensional population. The test utilizes the covariance matrix for the two sets of standardized variables, from which are extracted the largest characteristic roots. These then are used to form a system of equations from which the coefficients are derived. Canonical variates, weighted composites of variables are formed so as to maximize the proportion of variability in one set of variables that is explained by the other set. These are formed as unique, orthogonal sets of coefficients 88 89 with the number of variates that are formed dependent upon the smallest number of variables in either of the two sets. From these coefficients are calculated canonical R's, the maximum value that would be obtained by correlating the two weighted sets of values. The interpretation of a canonical analysis is de­ pendent upon the signs and relative magnitudes of the co2 efficients as well as upon the canonical R , which indicates the amount of variance in one set of weighted variables that is statistically accounted for by the other set of weighted variables. Canonical correlation analyses utilize data which must be measured at least on an interval scale. On the other hand, nominal or categorical data is required for a Chi square test. The Chi square test of independence provided the researcher with a means of deter­ mining whether two variables were independent. The computed Chi square value is dependent upon the number of levels for the two variables as well as upon the sample size. By computing the Chi square value for each level of the joint distribution, and using a single degree of freedom, an indication of the relative contribution of each variable level to the overall, significant Chi square value is obtained. Thereby, the specification of those levels of the joint distribution for which there is dependence between the two variables, is possible. 90 Kendall's coefficient of concordance provided a measure of the degree to which there was agreement among the teachers in regard to the relative importance applied to each student characteristic. The interpretation of Ken­ dall's coefficient of concordance indicates whether the value is statistically different from complete disagree­ ment. Data for this test included ranks from one to thir­ teen, the number of characteristics being ranked. When the data consisted of the frequency of choice for the bases for m a rks, a Chi square one sample test en­ abled the researcher to determine whether a uniform dis­ tribution of responses was created. The interpretation of the obtained Chi square value depends upon the number of levels of the variable and the desired level of confidence. A significant value indicates that the distribution of the obtained data departs from a uniform distribution is at All analyses were performed on either the CDC 6500 or the 3600 computer at Michigan State University utilizing programs developed by Jeremy Finn of the State University of New York at Buffalo and the Computer Institute for Social Science Research at Michigan State University. Statistical Findings Findings are presented below according to the type of analysis performed with the remaining organization depend­ ent upon the questions that were to be answered by the study. 91 Whenever appropriate an a level of .05 was used as representing the desired level of confidence. For the most part due to space limitations only those findings are presented which were statistically sig­ nificant. However, in Appendices B and C the complete fre­ quency distribution for all variables of interest as used in the analyses are presented. Canonical Correlation Analyses Question 1 : What personal factors are significantly related to the weighting applied to student charac­ teristics that are considered when assigning marks? Findings; The correlation matrix presented in Table 4.1 reports the degree of relationship between the teachers* personal characteristics and the weighting applied to each student characteristic considered when assigning marks, as well as the intercorrelation for each of these measures. A consideration of the interrelationships among the personal factors showed that, as would be expected, the older teachers reported more years of experience than did the younger teachers. The older teachers also reported that they had taken a greater number of measurement courses and held higher college degrees. Similar findings occurred between experience and both degree held and number of meas­ urement courses. It was also found that male teachers were more often older than females, while female teachers were 92 less likely to hold advanced college degrees and more likely to report higher grade point averages. A consideration of the interrelationship among the weightings applied to student characteristics showed either a marked or close relationship with the following exceptions. The importance which a teacher applies to attendance was found to be independent of the importance attached to con­ duct, neatness of work, homework, test marks, and individual projects or reports, while only slightly related to the im­ portance associated with class participation and optional work. The importance applied to conduct was found to be only slightly related to the importance of effort and moder­ ately related to attitudes, quiz marks, and group reports or projects. The importance of effort was found to be only slightly related to the importance of neatness of work, homework, test marks, and individual reports on projects. The importance of student attitudes was found to be moder­ ately related to neatness of work, test marks, and indi­ vidual projects or reports. The importance associated with group reports or projects was found to be moderately re­ lated to neatness of work, homework, and test marks, while individual reports or projects were found to be moderately related to quiz marks and group reports or projects. The importance of quiz and test marks was moderately related. A consideration of the cross correlations between the teachers' personal factors and the weighting applied to 93 each student characteristic considered showed that while many of the correlations were statistically significant, none of them indicate more than a slight relationship. Table 4.2 reports the first three canonical variates that were derived from the above correlation matrix. The first canonical variate indicated a canonical R of .421 be­ tween the weighted set of student characteristics and the weighted set of personal factors. The set of student char­ acteristics demonstrated a high positive loading on neatness of work and a high negative loading on the students1 per­ sonal appearance. Moderately high positive loadings were found for group reports or projects and attendance. Mod­ erately high negative loadings were found for homework and test marks and for individual reports or projects. The set of personal factors demonstrated all posi­ tive loadings with the highest loading for sex. Moderately high loadings were found for both experience and high school grade point average. The second significant canonical variate indicated a canonical R of .253 between the two sets of measures. The set of student characteristics demonstrated a high positive loading on effort and a high negative loading on attendance. Moderately high positive loadings were found for neatness of work and optional work. Moderately high negative loadings were found for individual reports or projects, participation, and appearance. I M L f 4 . 1 . — C o r r e la t io n u t r i e e t f a r p e rio n a l f a c t o r * and th e w l q h t i m i i p p l ia d t o tt u d e n t c t u r a c t r r i t t i c i . tm -wo i-oo B^er lance .041 .713 Degree -.1*4 .IM .319 Courea* -.058 .252 .314 .285 31 -.11* -.011 -.010 .0)5 .064 Itttndincv .0)1 .04) .040 -.023 a004 -.024 appearrace -.048 .01) -.0)2 -.112 -.051 -.081 .799 Conduct -.107 -.074 -.088 -.152 -.084 .005 .590 .011 .041 .015 -.057 -.016 -.04) .984 .87* .151 Participation -.071 -.044 -.044 -.158 -.07* -.101 .17) .721 .94) .117 Attltadea o f i 0 1 .011 -.018 -.102 -.030 -.072 .871 .987 .411 .934 .424 •UtMU -.112 -.075 -.092 -.159 -.IS» -.084 .004 .402 .98) .154 .974 .4*7 Unut -.140 -.095 -.113 -.158 *.0M -.095 .041 .425 .971 .184 .948 .513 .988 Quiziei -.0)4 .021 -.020 -.09* -.040 -.048 .871 .998 .425 .9)2 .421 .995 .484 .515 Teit* -.177 -.092 -.100 -.147 -.076 -.080 .007 .587 .950 .144 .*) .47) .949 .944 .428 Optional -.10) -.048 -.091 -.141 -,0M -.095 .175 .727 .947 .320 .987 .428 .980 .977 .427 .952 Creep naport* -.011 .041 -.004 -.080 -.017 -.057 .9)0 .941 .354 .97) .512 .988 .34) .194 .981 .154 .51* livUvldual -.117 -.079 -.095 -.154 -.081 -.091 .001 .400 .982 .151 .975 .484 .998 .981 .4*3 .Ml .1*0 h 5 'j C T b 4 1 I. <" i V 0 u X b 2 b L. 1.00 2 5 7 b T 3 id 1*00 * j c k ■3 0 < 1.00 a. J — < 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 .142 Individual Clfort 1.00 95 The set of personal factors demonstrated negative factor loadings with the exception of sex, which had also the highest loading. degree. A high negative loading was found for Moderately high loadings were found for both age and experience. The third significant canonical variate indicated a canonical R of .234 between the two sets of measures. The set of student characteristics demonstrated a high positive loading for group reports or projects and a high negative loading for neatness of work. A moderately high positive loading was found for individual reports or projects. Mod­ erately high negative loadings were found for attitudes and attendance. An extremely low positive loading was found for test marks. The set of personal factors demonstrated a high nega­ tive loading for age and high positive loadings for degree and high school grade point average. A moderately high load­ ing was found for sex. With the first canonical variate 17.8 percent of the variance in the weightings applied to student characteristics was statistically accounted for by the personal characteris­ tics of the teacher. With the second canonical variate 6.4 percent of the variance was accounted for and with the third variate 5.5 percent was accounted for. It, .therefore, seems that a knowledge of the personal characteristics of the teacher particularly in regard to sex would provide a slightly accurate 96 TABLE 4.2.— Significant Canonical Variates for Personal Factors and Student Characteristics Student Characteristics Attendance Appearance Conduct Effort Participation Attitudes Neatness Homework Quizzes Tests Optional Group Reports Individual Personal Factors Sex Age Experience Degree Courses GPA Canonical R Canonical R 2 Redundancy X2 df Significance First Canonical Variate Second Canonical Variate Third Canonical Variate 2.511 -7.113 - .425 - .309 1.043 .588 8.712 -3.129 .442 -2.040 .789 2.646 -2.610 -7.438 -2.125 - .114 7.229 -2.268 1.051 3-431 1.441 - .651 -1.093 2.905 1.542 -4.522 -3.232 -2.082 1.719 1.524 -2.867 -3.563 -4.608 1.267 -1.557 .118 1.519 8.290 2.791 .433 .347 .279 .400 .143 .141 .253 .064 .297 - .629 - .148 .595 .170 .586 .234 .915 .119 .274 .125 .041 .295 .421 .178 1.366 256.57 78 .0001 - .491 118.15 60 .0001 .055 .421 71.43 44 .0056 97 prediction of the comparative importance attached to the stu­ dent characteristics considered when assigning marks. Question 3: What situational factors are signifi­ cantly related to the weighting applied to student characteristics that are considered when assigning marks? Findings: The correlation matrix presented in Table 4.3 reports the degree of relationship between the situational factors and the weighting applied to each stu­ dent characteristic that is considered when assigning marks as well as the intercorrelation matrix for the situational factors. The intercorrelation matrix for the student char­ acteristics is presented above in Table 4.1. A consideration of the intercorrelations among the situational factors revealed a close relationship between grade taught and school level as well as between grade level and subject area taught. As would be expected, it was found that more minority students were enrolled in the metropoli­ tan school districts than in rural districts. A consideration of the cross correlations between the two measures showed several slight relationships. Teach­ ers in urban school districts were found to attach less im­ portance to the students' personal appearance than did teachers in rural districts. Teachers, in districts having a high percentage of minority students, attached more im­ portance to attendance, conduct, and personal appearance than did other teachers. Teachers of nonacademic subject areas and elementary grade levels were found to attach more 98 TABLE 4.3.— Correlation Matrices for Situational Factors and Weighting Applied to Student Characteristics Level -.489 Humber District -.127 -.308 .018 Humber School -.338 -.159 -.173 -.174 1.00 .467 .048 -.050 -.082 -.208 -.818 -.025 .032 -.019 -.014 .837 1.00 Attendance .016 -.040 .165 -.027 -.045 .092 -.038 Appearance -.031 -.116 .203 .040 -.084 .079 -.039 Conduct -.015 -.071 .115 .006 -.076 .088 -.023 Effort .168 .000 .001 .017 -.081 .279 -.339 Participation .082 .054 -.037 -.036 -.025 .151 -.102 Attitudes .073 -.037 .037 .017 -.052 .189 -.141 Heatness .132 -.037 .037 -.009 -.118 .207 -.212 Homework -.080 .005 .074 -.069 -.006 -.210 .274 .011 -.056 .053 -.009 -.019 -.061 .096 -.092 -.051 .018 .003 .058 -.243 .339 Optional .094 -.023 -.011 -.007 -.079 .123 -.040 Group Reports .056 -.018 .023 .014 -.064 .066 -.025 -.058 -.017 .013 .019 -.031 .070 .206 Quizzes Tests Individual 1.00 1.00 Grade Grade 1.00 Subject Subject Urban -.118 Level Minorities Number School 1.00 Number District .295 inorities Urban 1.00 99 importance to effort, class participation, students' tudes and neatness of work. atti­ Teachers of academic subject areas attach more importance to homework and test marks. Teachers of upper grade levels also attach more importance to homework, test marks, and individual reports on projects, while teachers of lower grade levels attach more importance to effort and neatness of work. Table 4.4 reports the first four canonical variates that were derived from the correlation matrix in Table 4.3 and the intercorrelations for student characteristics, which appear in Table 4.1. The first canonical variate indicated a canonical R of .603 between the weighted set of student characteristics and the weighted set of situational factors. A high positive loading was found for test marks and a high negative loading for neatness of work. A moderately high positive loading was found for homework marks. Moderately high negative loadings were found for effort and optional work. Extremely low negative loadings were found for both appearance and attendance. The set of situational factors showed a single posi­ tive loading of moderately high magnitude for the number of teachers in the school. The level of school and grade level taught both were found to have high negative loadings. A moderately high negative loading was found for subject area taught. The second canonical variate indicated a canonical R of .392 between the two sets of measures. The set of 100 TABLE 4.4.--Significant: Canonical Variates for Situational Factors and Student Characteristics. Student Characteristics Attendance Appearance Conduct Effort Participation Attitudes Neatness Homework Quizzes Tests Optional Group Reports Individual Reports Situational Factors Minorities Number in District Urbanization Grade Taught Subject Taught Number in School School Level Canonical R 2 Canonical R Redundancy x2 df Significance First Canonical Variate Second Canonical Variate -.011 -.028 .059 -.314 .072 -.090 -.496 .349 .136 .720 -.233 -.098 .198 .363 .147 .149 .189 .025 -.350 -.191 -.244 -.179 .016 -.084 -.357 .987 -.102 -.024 -.053 -.646 -.251 .499 -.768 .603 .364 .080 -.002 -.327 -.184 .003 .102 -.309 .392 .154 2.796 813.13 338 .0001 1.181 498.67 300 .0001 Third Canonical Variate .632 .629 -.217 -.003 .092 .269 -.229 -.171 .155 .204 -.232 -.386 -.028 .061 -.031 -.623 .665 1.429 -.126 1.892 .372 .139 1.066 382.64 264 .0001 Fourth Canonical Variate .094 -.209 .444 -.416 -.767 -.195 .747 -.379 .186 .074 -.139 -.038 -.033 .075 -.308 1.464 -.383 -1.361 -.440 2.798 .326 .106 .816 278.84 230 .0153 101 student characteristics demonstrated a very high positive loading for individual reports or projects and a moderately high positive loading for attendance. Moderately high nega­ tive loadings were found for group reports or projects, at­ titudes, and appearance. Extremely low positive loadings were found for class participation and test marks. The set of situational factors showed moderately high negative loadings for urbanization and school level. The third canonical variate indicated a canonical R of .372 between the two sets of measures. The set of student characteristics demonstrated high positive loadings for attitudes on test m a rks. A high negative loading was found for group reports or projects and moderately high negative loadings for optional work, neatness of work, and conduct. The set of situational factors demonstrated very high positive loadings for school level and subject area and a moderately high positive loading for grade level taught. A moderately high negative loading was found for urbaniza­ tion . The fourth canonical variate indicated a canonical R of .326 for the two measures. The set of student char­ acteristics demonstrated a high positive loading for neat­ ness of work and a high negative loading for class partici­ pation. A moderately high positive loading was found for 102 conduct. Moderatley high negative loadings were found for effort and homework marks. Extremely low negative loadings were found for both individual and group reports or projects. The set of situational factors showed a very high positive loading for school level. A high positive loading was found for urbanization and a high negative loading for subject area taught. With the first canonical variate 36.4 percent of the variation in the weighting applied to student characteristics was statistically accounted for by the situational factors. With the remaining canonical variates 15.4, 13.9 and 10.6 percent of the variation was accounted for by the respective variate. It, therefore, seems that a knowledge of the situa- tional factors particularly in regard to the grade level of the students would provide a reasonably accurate prediction of the comparative importance attached to the student char­ acteristics considered when assigning marks. Question 5 : What attitudinal factors are signifi­ cantly related to the weighting applied to student characteristics that are considered when assigning marks. Findings; 4.5 The correlation matrix presented in Table reports the degree of relationship between the teachers* beliefs regarding the appropriate goals of education and the function of marks and the weighting applied to each student characteristic considered when assigning marks as well as the intercorrelation matrix for the attitudinal factors. The 103 TABLE 4.5.— Correlation Matrices for Attitudinal Factors and Weighting Applied to Student Characteristics. 1.00 Functions Teacher .999 Functions Student -.230 -.231 Functions Parents -.234 -.237 .366 Attendance .998 .998 Appearance .799 1.00 1.00 • — .241 .799 -.188 - .154 -.001 -.001 .001 .052 Effort .985 .985 -.240 - .238 Participation .169 .170 -.041 .016 Attitudes .871 .871 -.207 - .187 Neatness .002 .002 -.000 .058 Homework .040 .040 -.007 .066 Quizzes .872 .872 -.197 Tests , .006 .006 .040 .116 Optional .173 .173 -.042 .013 Group Reports .931 .931 -.216 - .204 -.001 -.001 .001 .060 Functions Parents Conduct individual Functions Teacher -.232 Goals 1 Functions Student o o Goals — .168 104 intercorrelation matrix for the student characteristics is presented above in Table 4.1. A consideration of the intercorrelations among the attitudinal factors showed that the teachers* beliefs regard­ ing the appropriate goals of education and their opinions re­ garding the functions of marks for teachers were almost per­ fectly related. For this reason, the correlations between these two measures and the student characteristics were identical. A slight negative relationship was found between the functions of marks for teachers, for students, and for parents. A consideration of the correlations between the at­ titudinal factors and the student characteristics showed a close relationship between the teachers * viewpoints regard­ ing goals of education and the weighting applied to attend­ ance, appearance, effort, attitudes, quiz marks and group reports or projects. A slight negative relationship was found between both functions of marks for students and parents and the weighting applied to attendance, appearance, effort, attitudes, quiz marks, and group reports. Table 4 .6 reports the first two canonical variates that were derived from the above correlation matrix. The first canonical variate indicated a canonical R of .9997 between the weighted set of student characteristics and the weighted set of attitudinal factors. A high positive load­ ing was found for appearance and a high negative loading for 105 TABLE 4.6.— Significant Canonical Variates for Attitudinal Factors and Student Characteristics. First Canonical Variate Student Characteristics Attendance Appearance Conduct Effort Participation Attitudes Neatness Homework Quizzes Tests Optional Group Reports Individual Attitudinal Factors Goals Functions Teacher Functions Student Functions Parents Canonical R 2 Canonical R Redundancy X2 df Significance .129 .615 -.044 .074 -.025 .088 -.254 -.000 .058 -.001 -.153 .208 -.151 .126 .872 -.004 -.003 .999 .999 7.688 5446.06 52 .0001 Second Canonical Variate -.624 -.823 -.347 .927 -.679 2.367 2.999 -.706 -6.291 -2.731 2.530 3.763 -.397 -7.543 7.270 -.272 -.867 .330 .109 .839 107.34 36 .0001 106 neatness of work. Moderately positive loadings were found for group reports or projects and attendance. Moderately high negative loadings were found for optional work and in­ dividual reports or projects. The set of attitudinal factors showed a high positive loading for the functions of marks for teachers and a moder­ ately high positive loading for the goals of education. The second significant canonical variate indicates a canonical R of .330 between the two sets of measures. The set of student characteristics demonstrated a high positive loading for both group reports on projects and neatness of work. A high negative loading was found for quiz marks. Moderately high positive loadings were found for optional work and for attitudes. A moderately high negative loading was found for test marks. The set of attitudinal factors showed a high posi­ tive loading for the functions of marks for teachers and a nearly equal negative loading for the goals of education. With the first canonical variate 99.9 percent of the variation in the weighting applied to student characteris­ tics was statistically accounted for by the teachers' atti­ tudes toward the goals of education and the function of marks. With the second canonical variate 10.9 percent of the variation was explained by the attitudinal factors. seems, therefore, that a knowledge of the teachers' It atti­ tudes regarding the appropriate goals of education and the 107 functions of marks would provide an accurate prediction of the comparative importance attached to the student charac­ teristics considered when assigning marks. Question 7 i What other aspects of the marking procedure used Is significantly related to the weighting applied to the student characteristics that are considered when assigning marks? Findings; The correlation matrix presented in Table 4.7 reports the degree of relationship between other aspects of the marking procedure and the weighting applied to each student characteristic that is considered when assigning marks, as well as the intracorrelation matrix for the other aspects. The intercorrelation matrix for the student char­ acteristics is presented above in Table 4.1. A consideration of the intercorrelations among the other aspects showed a slight relationship between the fre­ quency of use of statistical techniques and the frequency of use of objective measurements. A consideration of the cross correlations revealed a slight relationship between the frequency of use of sta­ tistical techniques and homework, quiz, and test marks. A slight relationship was also found between the frequency of objective measurements and homework, quiz, and test marks while this other aspect showed a slight negative relation­ ship to effort. Table 4.8 reports the first two canonical variates that were derived from the appropriate correlation matrices. 108 TABLE 4.7.— Correlation Matrices for Other Aspects of the Marking Procedure and Weightings Applied to Student Characteristics. Sources 1.00 Frequency Statistics .044 Frequency Objective .085 .315 1.00 Attendance -.036 .057 -.006 Appearance -.043 .004 -.066 Conduct -.077 .002 -.099 Effort -.069 -.155 -.277 Participation -.035 -.022 -.071 Attitudes -.057 -.069 -.117 Neatness -.085 -.020 -.059 Homework .065 .167 .312 -.001 .152 .272 .094 .204 .317 Optional -.032 .024 .019 Group Reports -.036 .033 .065 Individual -.028 .004 .042 >i Ul u o G >i Quizzes Tests Ul G H <11 4J 3 U oi ai <11 *r-> u b -t O 109 TABLE 4.8.— Significant: Canonical Variates for other Aspects of the Marking Procedure and Student Character­ istics. Student Characteristics Attendance Appearance Conduct Effort Participation Attitudes Neatness Homework Quizzes Tests Optional Group Reports Individual Reports Other Aspects Sources Information Use Statistics Use Objective Measures Canonical R 2 Canonical R Redundancy X2 df Significance First Canonical Variate Second Canonical Variate -.096 -.304 .920 .272 .084 .317 -.260 -.402 .116 .157 .097 -.006 -.134 -.047 .121 .025 .485 .018 .048 .166 -.367 -. 397 -.520 .205 -.120 -.071 .335 -.263 -.850 .583 .959 .157 .340 2.618 .097 .747 113.36 407.06 78 .0001 .102 .312 66 .0001 11 0 The first canonical variate indicated a canonical R of .583 between the weighted set of student characteristics and the weighted set of other aspects. A high positive loading was found for effort and a high negative loading for test marks. Moderately high positive loadings were found for optional work and neatness of work. Moderately high negative loadings were found for quiz marks and homework marks. The set of other aspects showed a high negative load­ ing for the total frequency of use of objective measurements. Sources of useful information were found to have the only positive loading. The second significant canonical variate indicated a canonical R of .312 between the two measures. The set of student characteristics demonstrated an extremely high posi­ tive loading for appearance and a high positive loading for class participation. found for conduct. A moderately high positive loading was A high negative loading was found for neatness of work and moderately high loadings for attendance and attitudes. An extremely low negative loading was found for optional work. The set of other aspects showed all positive loading. Sources of useful information was found to have a high loading. With the first canonical variate 34.0 percent of the variation in the weighting applied to each student charac­ teristic was statistically accounted for by the other aspects Ill of the marking procedure. With the second canonical variate 9.7 percent of the variation was accounted for. It seems, therefore, that a knowledge of the other procedures that are related to the practice of marking students would provide a reasonably accurate prediction of the comparative importance attached to the student characteristics considered when as­ signing marks. Chi Square Tests of Question 2 : What personal factors are significantly related to the basis on which marks are assigned? 1. Findings: As indicated in Table 4.9, the basis on which marks are assigned was found to be significantly related to the sex of the teachers. Table 4.10 shows the joint frequency distribution for the two measures as well as the theoretical expected frequency and the cell Chi square value. TABLE 4.9.— Chi Square Analyses for Personal Factors and Basis for Marks. X2 Sex Age Experience Degree Number Courses GPA 43.935*** 43.487** 38.482 34.408 14.310 9.815 2 **Indicates X 2 significant at a equal to .0 1 . ***Indicates X significant at a equal to .001. df 6 24 30 24 18 12 TABLE 4.10.— Chi Square Table for Sex and Basis for Marks. Growth Male (282) Mastery Frequency 24 37 Theoretical 33.82 31.86 2.85 .83 Performance 13 7.47 Growth Mastery Growth Performance Mastery Performance None 104 27 68 9 128.61 27.53 43.26 9.44 .01 14.14 *★ Chi Square Frequency Female Theoretical (435) Chi Square 62 44 52.18 49.14 1.85 .54 4.09^ 4.71^ .02 223 43 42 15 11.53 198.39 42.47 66.74 14.56 2.65 3.05 .01 6 2 ♦Indicates X^ significant at a equal to .05. 2 ♦♦Indicates X^ significant at a equal to .01. 9.17^ .01 113 Male teachers appear to base marks either on perform­ ance relative to that of others or on a combination of sub­ ject matter mastery and relative performance more often than was expected. Male teachers also appear to be less likely than female teachers to base marks on a combination of self growth and subject matter mastery. Female teachers were less likely than male teachers to use a combination of subject matter mastery and relative performance than was expected. 2. Findingss As indicated in Table 4.9, the bas on which marks are assigned was found to be significantly related to the age of the teacher. Table 4.11 shows the joint frequency distribution for the two maasures as well as the theoretical expected frequency and the cell Chi square value. Those teachers in the 46 to 55 year old group appear to less frequently use subject matter mastery alone as the basis for marks. Those teachers in the oldest age group, 56 years and older, appear more likely to use the students' growth as the basis for assigning marks than was expected. Question 4 ; What situational factors are signifi­ cantly related to the basis on which marks are assigned? 1. Findings: As indicated in Table 4.12, the basis on which marks are assigned was found to be significantly related to the type of school within which the teachers TABLE 4.11.— Chi Square Table for Age and Basis for Marks. Growth Mastery Performance Growth Mastery Growth Performance Mastery Performance None 20 - 25 (127) Frequency Theoretical Chi Square 9 15.21 2.54 15 14.33 .03 2 3.36 .55 68 58.02 1.72 14 12.38 .21 16 19.46 .61 3 4.25 .37 26 - 35 (265) Frequency Theoretical Chi Square 25 31.74 1.43 27 29.90 .28 11 7.01 2.27 120 121.06 .01 26 25.84 .00 45 40.60 .48 11 8.86 .52 36 - 45 (155) Frequency Theoretical Chi Square 24 18.57 1.59 24 17.49 2.43 1 4.10 2.35 62 70.81 1.10 13 15.11 .30 27 23.75 .45 4 5.18 .27 46 - 55 (98) Frequency Theoretical Chi Square 13 11.74 .14 4 11.06 4.50* 3 2.59 .06 48 44.77 .23 9 9.55 .03 17 15.01 .26 4 3.28 .16 15 8.74 4.83* 11 8.23 1.13 2 1.94 .03 30 33.35 .71 7 7.12 .03 5 11.18 3.45 2 2.44 .10 Frequency 56 & over Theoretical (72) Chi Square ♦Indicates 2 significant at a equal to .05. 115 were assigned, as determined by the grade levels in that school. TABLE 4.12.— Chi Square Analyses for Situational Factors and Basis for Marks • X2 df Type of School (Grade Levels) Grade Level Taught 80.061*** 24 113.965*** Subject Taught Academic - Nonacademic 191.305*** 72.668*** 30 54 Size of (Within Size of (Within Percent District Stratum) School Stratum) Minorities Urbanization Level ***Indicates X 2 12 18.390 18 33.643 24 24 27.506 35.487 24 significant at a equal to .001. Table 4.13 shows the joint frequency distribution for the two measures, as well as the theoretical expected frequency and the cell Chi square values. Teachers in either a senior high school or junior high school were less likely to base marks on the students self growth than was expected. On the other hand, teachers in these schools are more likely to base marks on a combina­ tion of subject matter mastery and relative performance. This latter finding was true for both separated and combined schools. TABLE 4.13.-~Chi Square Table for Level of School and Basis for Marks. Growth Mastery Performance Growth Mastery Growth Performance Mastery Performance None Senior High 213 Frequency Theoretical Chi Square 8 19.53 12.55“ * 20 18.45 .26 7 4.33 2.18 69 74.69 .72 20 15.94 1.59 33 25.05 5.05* 6 5.47 .39 Junior High Middle School 214 Frequency Theoretical Chi Square 9 19.53 12.55“ * 20 18.45 .26 6 4.33 2.18 70 74.69 .72 19 15.94 1.59 33 25.05 5.05* 7 6.47 .39 Elementary (329) Frequency Theoretical Chi Square 65 39.41 16.62“ * 30 37.12 1.36 3 8.71 3.74 168 150.30 2.09 25 32.08 1.56 28 50.40 9.96** SeniorJunior High (48) Frequency Theoretical Chi Square 3 5.75 1.31 9 5.42 2.37 2 1.27 .42 13 21.93 3.63 6 4.68 .37 Junior High Elementary (9) Frequency Theoretical Chi Square 0 1.08 1.08 1 1.02 .00 1 All (5) Frequency Theoretical Chi Square 1 1 0 ♦Indicates 2 .60 .27 .24 2.44 .56 .34 significant at a equal to .05. • ‘‘indicates X, . ... . ._ ,^ 1 significant at a equal to .01 ‘“ Indicates . .„ ,, 1 significant at a equal to .001. .13 .13 5 4.11 .19 0 3 2.28 .22 0 .88 .88 .49 .49 14 7.35 6.01* 10 11.00 .09 1 1.60 .23 2 1.38 .28 0 0 0 .77 .77 .30 .30 .17 .17 117 Teachers in elementary schools were found to be dis­ proportionately likely to base marks on the students' self growth. These teachers were also less likely to utilize a combination of subject matter mastery and relative perform­ ance . 2. Findings: As indicated in Table 4.12 the grade level taught by the teacher was found to be significantly re­ lated to the basis on which marks are assigned. Table 4.14 shows the joint frequency distribution for the two measures as well as the theoretical expected frequency and the cell Chi square value. Teachers of the lower elementary grade levels, grades kindergarten through fourth, were found to be more likely to base marks on the students' self growth. Teachers of grades kindergarten through second were also more likely to utilize a combination of growth and subject matter mastery and were less likely to utilize a combination of subject matter mastery and relative performance as the basis for marks. Teachers of grades five and six were less likely to use a combination of subject matter mastery and relative performance as the basis for marks. On the other hand, teachers of grades seven through twelve are more likely to use a combination of subject matter mastery and relative performance while they are less likely to use growth as the basis for marks. Teachers of grades ten through twleve were also found to be more likely to use the TABLE 4.14.— Chi Square Table for Grade Level Taught and Basis for Harks. Growth Mastery Performance Growth Mastery Growth Performance Mastery Performance None K-l-2 (127) Frequency Theoretical Chi Square 27 15.21 9.14^ 10 14.33 1.31 0 3.36 3.36 73 58.02 3.87^ 9 12.38 .92 4 19.46 12.28### 4 4.25 .01 3-4 (115) Frequency Theoretical Chi Square 22 13.77 4.91^ 14 12.97 .08 1 3.04 1.37 52 52.53 .01 10 11.21 .13 11 17.62 2.49 5 3.84 .35 5-6 (101) Frequency Theoretical Chi Square 18 12.10 2.88 9 11.39 .50 3 2.67 .04 55 46.14 1.70 9 9.85 .07 5 15.47 7.09« 2 3.38 .56 7-8-9 (176) Frequency Theoretical Chi Square 11 21.08 4.Q2* 22 19.86 .23 4 4.66 .09 71 80.40 1.10 22 17.16 1.37 40 26.96 6.30* 6 5.88 .00 10-11-12 (187) Frequency Theoretical Chi Square 6 22.40 12.0 1 ^ 26 21.10 1.14 11 4.95 7.40** 71 85.43 2.44 17 18.23 .08 50 28.65 15.91*** 6 6.25 .01 Nongraded Elementary (11) Frequency Theoretical Chi Square 5 5.03 .00 3 1.07 3.46 2 1.32 .35 0 1.24 1.24 0 .29 .29 2 ♦Indicates X1 significant at a equal to .05. ♦♦indicates ♦♦♦Indicates 2 2 significant at a equal to .01. significant at a equal to .001. 0 1.69 1.69 1 .37 1.09 119 students' performance when compared to others as the basis for m a r k s . Teachers of non-graded elementary classes were not found to be significantly different from other teachers in regard to the basis for marks. This finding may have re­ sulted from the small number of respondents in this classifi­ cation . 3. Findings: As indicated in Table 4.12, the basis on which marks are assigned was found to be significantly re­ lated to whether the teachers taught elementary students, academic, or nonacademic subject areas. Table 4.15 shows the joint frequency distribution of the two measures as well as the theoretical expected frequency and the cell Chi square values. Secondary teachers in academic subject areas more often than was expected reported that they used a combination of subject matter mastery and relative performance as the basis for marks. On the other hand, secondary teachers in academic subject areas are less likely to use growth as the basis. As reported abo v e , elementary teachers were found to be less likely to use a combination of subject matter and relative performance and more likely to use growth. 4. Findings: As indicated in Table 4.12, similar findings were made regarding the particular subject area taught, as related to the basis for marks. Table 4.16 shows TABLE 4.15.— Chi Square Table for Academic versus Nonacademic and Basis for Marks. Growth Mastery Performance Growth Mastery Growth Performance Mastery Performance None Academic (346) Frequency Theoretical Chi Square 24 41.02 7.06** 45 39.09 .89 12 9.17 .87 149 158.28 .54 34 33.78 .00 72 53.08 6.74** Nonacademic (86) Frequency Theoretical Chi Square 4 10.20 3.76 10 9.72 ,01 5 2.28 3.25 29 39.34 2.72 12 8.40 1.55 20 13.19 3.51 6.00 2.88 3.38 Elementary (281) Frequency Theoretical Chi Square 55 33.31 14.12*** 26 31.74 1.04 2 7.45 3.98* 149 128.55 3.25 23 27.43 .72 18 43.11 14.63*** 8.00 9.41 .21 •Indicates 2 significant at a equal to .05. 2 ••Indicates X1 significant at a equal to .01. 2 •••Indicates significant at a equal to .001. 10 11.58 .22 121 T A 1 U 4.16.--Chi Square Tibia for Subject t a u Taught and l u l l (or Hull. Growth Mastery Performance Growth Mastery Growth Performance Mastery Performance Hone Science (58} Frequency Theoretical Chi Square 4 6.86 1.20 6 6.55 .05 3 1.54 1.39 18 26.45 2.70 7 5.66 .32 17 8.98 7.17** 3 1.94 .SB Mathematics (691 Frequency Theoretical Chi Square 4 8.16 2.14 14 7.79 4.94* 4 1.83 2.58 22 31.47 2.85 2 6,74 3.33 22 10.68 11.99*** 1 2.31 .74 Social Studies (84) Frequency Theoretical Chi Square 6 9.96 1.59 16 9.49 4.47* 2 2.21 .02 29 38.31 2.26 10 8.20 .39 19 13.00 2/76 2 2.81 .23 Language Arts (112) Frequency Theoretical Chi square 9 14.46 2.06 8 13.76 2.41 3 1.23 .02 74 55.64 6.06* 15 11.91 .80 9 18.89 5.18* 4 4.08 .00 Foreign Language (14) Frequency Theoretical Chi Square 1 1.66 .26 1 1.58 .21 0 7 6.38 .06 0 1.17 1.37 5 2.17 3.70 0 .37 .17 Businese Comssrcial (21) Frequency Theoretical Chi Square 0 2.49 2.49 6 2.17 5.55* 0 5 9.58 2.19 1 2.05 .54 Music Art (19) Frequency Theoretical Chi Square 2 2.25 .01 1 2.15 .61 1 .50 12.38*** 8 8.67 .05 3 1.85 ‘ .71 9 3.25 10. L7** » 1 2.94 1.28 0 .56 .56 Industrial Arts (25) Frequency Theoretical chi Square 1 2.96 1.10 0 2.82 2.82 7 2.44 8.52** 5 1,87 .33 1 Physical Education Health (21) Frequency Theoretical Chi Square 0 2.49 2.49 1 2,17 .17 5 3.25 .94 4 Elementary All Subjects (284) Frequency Theoretical Chi Square 58 31.67 17.58*** 26 12.08 1.15 2 .66 2.70 .56 .56 7 9.58 .69 2 2.05 .00 2 7,53 4.06 148 129.52 2.64 23 27.71 0 •Indicate* X* significant .it i equal to .OS, ••Indicate* xj significant at 1 2) 3) 4) 5) 6) How many college courses have you taken in the area of grading or measurement? 1) none 2) 1 3) 2 or 3 4) 4 or more what is the highest college degree that you hold? 1) none 2) B.A. or B.S. 3) H.A. or H.S. 4) Ed. Spec. 5} Ed.D. or Ph.D. 7. In which one of the following grade categories do you teach mostly this year? 1) K-l-2 2) 3) 4) 5) 6) 7) 3-4 5-6 7-8-9 10-11-12 nongraded elementary special education 9. How often do you assign homework? 1) never 2) daily 3) once or twice weekly 4) less often than twice weekly 11. How often do you give longer tasts? 1) never 2) once or twice weekly 3) once every two or three weeks 4) at the end of each unit 5) once or twice each marking period 13. 10. oral problems essay short answer fill in multiple choice true false do not give testa What kind of grades did you receive in high school? 1) high 2) average 3) low 8. In which one of the following subject areas do you teach mostly this year? 1} science 2) mathematics 3) social science 4) language arts including reading 5) foreign language 6) business and/or conmercial 7) music or art 8) home economics or industrial arts 9} health and/or physical education 10) elementary, all subject areas How often do you give quitxes? (requiring 15 or less minutes) 1) never 2) dally 3) once or twice weekly 4) less often than twice weekly 5) only at the end of each unit 12. What format do you usually use for quix items? 1) oral 2) problems 3) essay 4) short answer 5) fill in 6) multiple choice 7) true false 8) do not give quisses What format do you usually use for test items? 1) 2) 3) 4) 5) 6) 7) 8) 2-5 6-10 11-20 21-40 over 40 14. On which of the following bases do you assign marks? 1) the student's growth compared to himself 2) the student's mastery of subject matter 3) the student's perfonanca compared to others 4) 1 and 2 above 5) 1 and 3 above 6) 2 and 3 above 7) none of the above 15. Which of the following sources of information do you find most helpful when assigning marks? 1) parents 2) the student himself 3) other teachers 4) student past records 5) none of these 16. Teachers sometimes use certain techniques in their efforts to measure student performance. <*ie such technique is the use of standard scores. r Goals of Education 1 . To provide the opportunity for students to learn the value of academic knowledge 0 1 2 3 4 5 To teach useful skills 0 1 2 3 4 5 3. To teach social awareness 0 1 2 3 4 5 4. To provide students with academic competency 0 1 i 178 Table Appendix B Continued. 5. 2 3 4 5 To help students become socialized individuals 0 1 2 3 4 6. 5 To assist students in their preparation for a future vocation 0 1 2 3 4 7. 5 To assist students in the development of a positive self concept 0 1 2 3 4 5 To provide the opportunity for students to become aware of societal problems 0 1 2 3 4 5 Functions of Marks for Parents 8. To indicate to the parents the social needs of the student 0 1 I II III IV V Total Total Percent 6 20 32 41 3 52 63 56 5 39 59 59 3 37 65 67 3 19 27 30 20 167 246 253 3 23 34 39 6 4 5 18 29 42 5 2 12 40 48 72 3 1 5 4 11 19 57 85 4 0 4 18 24 36 23 11 40 131 212 300 3 2 18 30 42 3 2 3 15 21 60 3 2 7 22 41 104 4 2 17 2 9 1 10 18 42 92 3 2 6 29 46 95 2 28 93 167 402 4 13 23 56 4 3 2 9 1 0 11 35 120 3 0 3 11 31 133 4 2 3 12 15 68 12 34 127 4 3 6 22 31 38 3 2 12 40 63 59 39 7 58 26 1 a 36 54 65 4 1 10 17 51 3 8 2 5 19 57 13 6 8 51 134 505 7 19 70 2 9 37 65 50 6 2 6 49 51 67 5 0 8 13 25 35 22 9 41 167 235 249 3 1 6 22 33 35 44 43 62 38 29 18 232 132 32 18 1 1 1 1 8. 179 Table Appendix B Continued. 2 3 4 5 6. To inform parents of the student's academic prog­ ress 0 1 2 3 4 5 3. To inform parents of the student's relative stand­ ing in the class 0 1 2 3 4 5 Functions of Marks for Students 4. To guide the student in planning for college or a career 0 1 2 3 4 5 To direct the learning of 7. the students 0 1 2 3 4 5 2. To punish student for mis­ behaving 0 1 2 I II III IV V Total To tail Percent 18 26 10 4 36 36 17 6 32 38 10 0 29 38 10 4 11 19 7 2 126 157 54 16 18 22 8 2 5 5 13 28 35 18 9 3 23 64 50 30 9 5 17 59 57 20 10 6 19 63 63 20 8 4 12 23 24 15 41 23 84 237 229 103 6 3 12 33 32 14 6 9 19 37 20 13 13 13 30 68 39 16 16 16 15 57 47 16 21 10 20 58 51 21 12 8 12 26 21 7 68 56 96 246 178 73 9 8 13 34 25 10 12 5 13 34 31 9 19 8 28 65 43 16 17 14 30 61 31 14 20 5 35 61 43 17 16 9 10 24 23 4 84 41 116 245 171 60 12 6 16 34 24 8 14 10 19 30 24 7 25 15 32 69 28 10 27 18 30 54 28 10 27 21 33 54 32 14 20 7 13 29 10 7 113 71 127 236 122 48 16 10 18 33 17 7 62 15 9 110 28 22 106 26 17 102 35 25 52 17 9 432 121 82 60 17 11 180 Table Appendix B Continued. 3 4 5 For Teachers 1. To indicate to the teacher the degree to which spe­ cific objectives are mastered 0 1 2 3 4 5 5, To indicate to the teacher the academic strengths of the students 0 1 2 3 4 5 9. To indicate to the teacher whether methods and mate­ rials were successful 0 1 2 3 4 5 Total Percent I II III IV V 12 4 2 11 4 4 10 6 2 9 6 4 5 3 0 47 23 12 7 3 2 7 9 10 42 23 13 12 8 20 63 52 24 11 10 16 65 46 19 13 8 21 64 53 22 11 5 10 24 27 9 54 40 77 258 201 87 8 6 11 36 28 12 6 5 14 33 37 9 9 7 22 51 67 23 12 5 18 62 57 13 12 9 22 62 60 16 8 7 10 29 22 10 47 33 86 237 . 243 71 7 5 12 33 34 10 13 9 6 38 24 14 21 13 25 48 50 22 14 11 22 52 42 26 22 17 21 50 46 25 10 6 10 20 29 11 80 56 84 208 191 98 11 8 12 24 27 14 Total APPENDIX C 181 18 2 Computer Institute for Social Science Research Michigan State University SOI Computer Center East Lansing, Michigan 48823 .Dear During recent years education has been the recipient of much criti­ cism, a part of which has centered around the grading or marking of students. Claims have been made that marks are harmful to students and that they are not accurate indicators of the students* abilities. However, little is actually known about the processes by which teachers assign marks or the relationship between personal, situational, and attitudinal factors and the marking procedures used by teachers. We answers dents. study. have designed a survey through which we hope to derive some to the question of how teachers in Michigan are marking stu­ Your school district has been selected to participate in this Your cooperation will be greatly appreciated. You are being asked to support this study in two ways. First by supplying the names and addresses of the teachers assigned on a regular basis to each of the following schools within your district. Two teachers from each of these schools will be selected! to re­ ceive mailed questionnaires pertaining to the procedures by which they assign marks. This questionnaire will require approximately 15 minutes for its completion. No further contact will be made to any teacher, unless a request is made by a teacher for a copy of the findings. S e c o n d l y y o u are b e i n g a s k e d to s u p p l y us w i t h a n u n u s e d cop y o f the s t u d e n t p r o g r e s s r e p o r t f o r m u s e d in e ac h o f the a b o ve m e n t i o n e d schools. B e c a u s e the e n t i r e s t u d y is d e p e n d e n t u p o n the c o o p e r a t i o n o f s c h o o l s u p e r i n t e n d e n t s , y o u r s u p p o r t is r e q u e s t e d . We would appreciate very much if the requested information could be sent immediately. Unless all information is received before April 20, the study can not be completed during the current school year. If you would like to receive a copy of the complete findings, please so indicate, when you send the list of teachers and the progress report forms. Respectfully, Bonnie J. Steller 183 Deer Superintendent: Recently you should have received e letter requesting your support of e study wo have designed, regarding the marking procedures used by teachers In Michigan. If you have not as yet replied, your Immediate attention would be appreciated, tt your failure so respond Is due to poHdes regarding the release of teachers' addresses, please sand us tho remaining Information. Additional procedures have been designed for districts which prefer not to even release teacher names. Please confirm your cooperation at your aertliit convenience, as without complete cooperation from school superintendents, the entire study wlN be worthless. Thank you for your help. BONNIE J. STELLER Computur Institute for Social Science Research 501 Computer Center Michigan State University East Lansing, Michigan 4 8 8 2 3 APPENDIX D 184 185 COMPUTER INSTITUTE FOR SOCIAL SCIENCE RESEARCH 501 Computer Center Michigan State University East Lansing, Michigan 48B23 Dear Sir: During recent years education has been the recipient of much criticism, a part of which has centered around the marking o r grading of students. Claims have been made that marks are harmful to students and that they are not accurate indicators of students* abilities, yet little is known about how teachers actually assign marks to students. He have designed a survey through which we hope to derive some answers to the question of how teachers in Michigan are assigning marks. Two teachers from your school have been selected to participate in this study. Enclosed with this letter are two sets of materials, a cover letter explaining the purpose of the survey, a two-page questionnaire, and a return addressed, stamped envelope, each set being addressed by a number. You are being asked to distribute these two sets of materials according to the following instructions. It is very important that these instructions be followed, as only in this way can we later make statements of our findings as they apply to all teachers within the state. In other words, we have applied random selection procedures. First you are to number consecutively the names on a roster of teachers employed in your school, omitting all special education teachers, librarians, counselors, etc. The teachers, who in this way are assigned the numbers on.the envelopes, are to be given the set of materials. If, by chance, an envelope is addressed to a number greater than any that you have assigned, the materials are to be given the last teacher on the roster. Because we plan to have no direct contact with any teacher, it would be very greatly appreciated if you would be responsible for reminding the teachers on or about May 11 that the questionnaires are to be returned before May 1 5 , as we want to complete the collection of information prior to the close of schools for summer vacation. It would also be helpful if you would send us an unused copy of the form used for reporting student progress by your teachers. If you are unable to participate in this study, please drop us a note, indicating your reasons for refusal to comply with our needs. If you would like a copy of our findings, please let us know. be available during the Fall of 1973. Thank you for your participation. Respectfully, They should 166 THIRD HAILING COMPUTER INSTITUTE FOR SOCIAL SCIENCE RESEARCH 501 Computer Center Michigan State University Baat Lansing, Michigan 48823 Dear Principal: Recently you were requested to deliver copies o f the enclosed questionnaire to two teachers In y o u r building. As yet we have not received both completed responses. Therefore* we are asking y o u to again distribute copies o f the questionnaire to the selected teachers from whom w e have received no response. We are aware o f the fact that there were unplanned delays In the mailing and delivery of these materials; therefore* we are extend­ ing the deadline to June *1. Also w e realize that the first copy of the questionnaire may have been mislaid or for some other legitimate reason been forgotten. However* as you realize 1t 1s crucial that we obtain complete responses, 1f the findings of the study are to be generalized to the entire population of teachers within the State of Michigan. If you have not sent us a copy of the student progress report form used in your school, please put 1t 1n the mall at y ou r earliest convenience. Again thank you for your support and cooperation. Please remind the teachers to whom you distribute the ques­ tionnaires that we must have them returned before J u n e H . as we realize that after that date teachers often are not available through the schools. We certainly hope that you are able to secure the coop­ eration of the teachers 1n your building. Sincerely, *- Q- - Bonnie J. Steller Enclosures __ APPENDIX E 188 Computer Institute for Social Science Research 501 Computer Center Michigan State University East Lansing, Michigan 48823 Dear Teacher: During recent years education has been the recipient of much critic cism, a part of which has centered around the marking or grading of stu-.,. dents. Claims have been made that marks are harmful to students and that they are not accurate indicators of the students' abilities. How­ ever, little is actually known about the processes used by teachers when assigning marks. We have designed a survey through which we hope to derive some ans­ wers to the question of how teachers in Michigan are marking students. Your school district was selected to participate in this study. Your school superintendent has been contacted and has indicated his willing­ ness for you to participate in this study. Your cooperation will be very greatly appreciated. You are being asked to support this study by responding to the en­ closed questionnaire. It should require approximately 15 minutes for completion. Please respond to those items directly pertaining to your marking methods according to what you actually do, rather than what you would like to do or think might be better. Other items ask for your own opinions rather than for facts. Only one response should be made to each item. Because the entire study is dependent upon the cooperationof teach­ ers such as you, your support is requested. We would appreciate very much if you could complete the question­ naire at your earliest possible convenience and return it to usinthe enclosed, addressed envelope. Those questionnaires received after May will not be included in the study. If you would like to receive a copy of the findings of the study, please either so indicate on the questionnaire by including your name and address or send us a postcard. The quesionnaires do not need to be signed unless you want to request a copy of the findings. Bie findings should be available early in September. Thank you for your time and assistance in helping us answer an important question for educators today. Sincerely, Bonnie J. Steller 189 COMPUTER INSTITUTE FOR SOCIAL SCIENCE RESEARCH 502 Computer Center Michigan State University Bast Lansing, Michigan 48823 Dear Teacher: Recently you should have received a questionnaire dealing with, the procedures by which you assign marks to students. However, as yet we have not received your completed response. We are aware of the fact that there were delays 1n the mailing and delivery of this earlier letter and questionnaire. Also knowing that teachers are very busy people particularly at this time of year, and that, for this reason, 1t 1s easy to mislay and forget something such as this, we have enclosed another copy of the questionnaire with this letter. Please complete and return 1t before June 1 so that we can analyze the responses. Remember that 1t 1s crucial that all of the teachers, w h o m we selected, do respond regardless of their opinions regarding marking. Unless we receive all of the responses, the findings of the study will tell us nothing about how teachers 1n Michigan are assigning marks but will only tell us how those who took the 15 minutes required to re­ spond are assigning marks to students. If solutions to problems confronting educations are to be found, it must be through the cooperation of all teachers. Only a few can accomplish little. Thanks again for your support and cooperation. your help. We appreciate Don't forget to' let us know 1f you would like a copy of the findings. Sincerely Bonnie J. Steller Enclosure 190 Dear Teacher, This is just a reminder that we have not as yet received your response to the questionnaire concerning your marking procedures. June 1 is not far off so please remember to take care of it at your earliest convenience. Thank You Again. Bonnie J. Steller Computer Institute for Social Science Research BIBLIOGRAPHY 191 BIBLIOGRAPHY Primary Sources Anderson, Robert. H. "The Importance and Purposes of Report­ ing." National Elementary School P r i n c i p a l s , XLV <1966), 6-11. Arnold, Richard Dean. "The Relationship of Teachers' Sex to Assignment Marks and Tested Achievement among Upper Elementary Grade Boys and Girls." (Unpublished Ph.D. dissertation. University of Minnesota, 1966)• Bates, Bean. "Can w e stop the merry-go-round of grades?" Arizona T e a c h e r , LV (January, 1967), 19 and 26. Bicknell, George L. "Sand V Grading System." XXV (July, 1941), 18. Texas O u t l o o k , Bolmeier, E. C. "Principles pertaining to marking and report­ ing of pupil progress." School R e v i e w , LIX (January, 1951), 15-24. Bramlette, Merle. "Is the Sand V Grading Syst e m Satisfactory or Unsatisfactory?" Texas O u t l o o k , XXVI (April, 1942) 29-30. Butterworth, Thomas W. "A Guide to the Evaluation of Pupil Progress w i t h Particular Emphasis on Pupil Marks." ERIC No. 036 814 (August, 1966). Chadwick, Ruth E., Rose Durham, and Marion Morse. "The Re­ p ort Card in a Nongraded School." National Elementary P r i n c i p a l s , XLVII (January, 1968), 22-28. Crooks, A. D. "Marks and M arking Systems: A Digest." Journal of Educational R e s e a r c h , XXVII (1933), 259273. Delgalvis, N. "Suggesting a Different Grading System." Pennsylvania School Journal (January, 1965). De.Zoeche, Dorothy. "The Wound is Mortal." XIX (February, 1945), 339-344. 192 The C learing House 193 Dionne, Joseph. HFuture of Testing: A Look at the Trends.” ERIC No. 043 669 (March, 1970). Doerhoff, Roy. "Grading as Easy as 1, 2, 3, 4." Co m m u n i t y , LV (April, 1969), 19. School and Doherty, William V. "A Survey of the Evaluation of Pupil Progress in Selected Elementary Schools in O h i o . ” (Unpublished Ph.D. dissertation, Ohio State Uni­ versity, 1954) . Dunivan, Lindell P. "Grading as a system of guidance: a rebuttal." School and C o m m u n i t y , LVI (May, 1970), 31. Ebel, Robert L. "The relation of scale fineness to grade accuracy." Journal of Educational M e a s u r e m ent, VI (Winter, 1969), 217-221. Editorial, "Report Cards on Parent-Teacher Conferences: Which is Better?" Kentucky School J o u r n a l , XIXL (November, 1970), 23. Benjamin Fine, "A.B.C. of Grading Puzzles Parents." New York Times M a g a z i n e , November 18, 1957, cited by Ann Z. Smith and John E. Dobbin, "Marks and Marking Systems," Encyclopedia of Educational R esearch, 3rd e d . , edited by Chester W. Harris (New York: The Macmillan Company 1960), pp. 783-791. Finn, Jeremy D. "Expectations and the Educational Environ­ ment." Review of Educational R e s e a r c h , XLII (Summer, 1972), 387-410. Fish, Enrica. "The Relationship of Teachers' Assigned Marks among Elementary Grades." (Unpublished Ph.D. disser­ tation, University of Minnesota, 1969) . Gage, N. L. "Desirable Behaviors of Teachers." cation (November, 1965), 85-95. Gould, George. School Urban Edu­ "Practices in Marking and Examinations." Review XL (1932), 142-146. Halliwell, Joseph W and Joseph P. Robitaille. "The relation­ ship between theory and practice in a dual reporting p r ogram." Journal of Educational R e s e a r c h , LVII (November, 1963), 137-141. 194 Harris, Wilbur S. "Agreement among N.C.M.E. Members on Issues in Educational Measurement.” Journal of Edu­ cational M easurement, X (Spring, 1973), 63-70. Hinely, R. T. "An Equal Chance— A Fantasy." Phi Delta K a p p a n , XLVIII (June, 1967), 523-524. Housdorff, Henry and S. David Farr. "The effect of grading practices on the marks of gifted sixth grade child­ ren." Journal of Educational R e s e a r c h , LIX (Decem­ ber, 1965), 169-172. Johnson, Xcie F. "Injustice of G r a d e s . ” School and Commu­ nity, LIV (October, 1967), 24-25. Johnston, Samuel R. "Ar there better ways of evaluating, recording, and reporting pupil progress in the junior and senior high schools?" National Association of Secondary School Princi p a l s , XXXIV (March, 1950), 79-89. Keck, Winston B. "Testing and Evaluation Practices in New England Public Schools." (Unpublished Ph.D. disser­ tation, Boston University, 1961). Kelley, Allan C. and Paul Zarembka. "Normalization of Stu­ dent Test Scores: An Experimental Justification." Journal of Educational Research, LXII (December, 1968), 160-164. Krause, Dorothy. "A study of present practices and atti­ tudes concerning the letter grading system of secondary language arts teachers.” (Unpublished Ph.D. dissertation, University of South Dakota, 1969). Lawson, Douglas E. "Scoring of subjective tests with sev­ eral variables controlled." Elementary School J o u r n a l , XXVII (February, 1938), 450-457. Liggett, William A. "Are there better ways of evaluating, recording, and reporting pupil progress in the junior and senior high schools?" National Association of Secondary School Principals, XXXIV (March, 1950), 79-89. Linder, Ivan, "Is there a substitute for teachers' grades?" American School Board Journal (July, 1940), 25-26. 195 Little, Ruth Coyner. "Whither Grading?" National Educa­ tion Association Journal (May, 1967), 95. Lund, John. "More Truth about Marks." tion (1929), 609-639. Journal of Educa­ Marshall, John C. "Evaluation: Does it mean anything?" The Clearing H o u s e , XLII (May, 1968), 535-538. Mason, Geoffrey. "Studies and Reports: An Empirical Analysis of a System of Achievement Grading." No. 014 131-132 (November, 1965). ERIC Maxson, Wilbur G. "Grading— a serious matter." National Education Association Journal, LIII (October, 1964), 56-57. McGee, H. M. "Measurement of Authoritarianism and its Re­ lation to Teachers' Classroom Behavior." Genetic Psychology Monographs (1955), 89-196. Melby, Ernest O. "It's time for schools to abolish the mark­ ing system." Nations Schools (May, 1966), 104. Metzner, Seymour. "Teacher Bias in Pupil Evaluation: A Critical Analysis." Journal of Teacher Education, XXII (Spring, 1971). Millman, Jason. "Reporting Student Progress: A case for a criterion referenced marking system." Phi Delta Kappan, LII (December, 1970), 226-230. Miner, Betty Crowther. "Three factors of school achieve­ ment." Journal of Educational Research, LX (April, 1967) , 376-376. Morris, Lucile, "Evaluating and Reporting Pupil Progress." Elementary School Journal, LIII (November, 1952), 144-149. Mosely, Aubrey Howard. "A Study of Teachers' Perceptions of Factors Related to Educational Research." (Unpublished Ph.D. dissertation, University of Alabama, 1966) . Newsome, Ralph P. "Assignment of School Marks." look, XXV (August, 1941), 39-40. Texas Out­ 196 Niederme y e r , Fred C. and Howard J. Sullivan. "Differential effects of individual and group testing strategies in an objective based instructional program." Journal of Educational M e a s u r e m e n t r IX (Fall, 1972). 199-204. Palmer, Orville. "Seven Classic Ways of Grading Dishonesty." The English J o u r n a l , LI (1962), 464-467. Pewitt, Joe. "Grading as a system of coercion." Co m m u n i t y , LVI (February, 1970), 20. School and Philbrick, Joseph L. and Patrick I. O'Donnell. "Precision in Grading Practices— Panacea or Problem?" Journal of Educational Research, LXII (December, 1968), . 173-176. Pope, Billy Newton, "Factorial Study of Teacher Marks and Standardized Tests. (Unpublished Ph.D. disserta­ tion, University of Houston, 1965). Purkey, Ernest. "An Analysis of Test Data and Teachers' Marks over Twelve Years of School Attendance." (Unpublished Ph.D. dissertation, University of Missouri, Columbia, 1966). Ribon, David M. "A Grade-Weighting System Investigated." National Association of Secondary School Principals, L (November, 1966), 94-98. Roberts, Bernice R. "A Comparison of Teacher Judgements and Multi-factor Test Scores as Predictors of Pupil Capacity." (Unpublished Ph.D. dissertation, Wayne State University, 1958) . Roeder, Harold H. "Teacher Education Curricula— Your final grade is F." Journal of Educational M e a s u r e m e n t , X (Summer, 1973), l4l-143. Roelfs, R. M. "Trends in Junior High School Progress Re ­ porting ." Journal of Educational R e search, XLIX (December, 1955), 241-249. Russell, Ivan L. and W. A. Thalman. "Personality: Does it influence teachers' marks?" Journal of Educational R e s e a r c h , XLVIII (April, 1955T7 561-564. Sparks, Merla. "Alternative to the Traditional Grading Sys­ tem." English Journal (October, 1967), 119-121. 197 Starch, Daniel and E. C. Elliott, "Reliability and Grading Work in History." School Raview, XXI (1913), 676681. Starch, Daniel and E. C. Elliott. "Reliability of Grading Work in Mathematics." School Rev i e w , XXI (1913), 254-259. Starch, Daniel and E. C. Elliott. "Reliability of Grading of High School Work in English." School Review, XX (1912), 442-457. Tees, Arthur Thomas. "In Defense of Grades." School and C o m munity, LV (October, 1968), 12-13. Terwillinger, James S. "A Survey of Secondary School Mark­ ing Practices and Policies." National Association of Secondary School Principals') L (March, 1966) , 1-37. Theimer, Jr. William Charles. "The Relationships between Certain Personality and Educational Factors and the Grading Patterns of Secondary School Teachers." (Unpublished Ph.D. dissertation, University of Denver, 1967). Thorndike, E. L. and E. 0. Bergman. "On the form of dis­ tribution of intellect in the ninth grade." Journal of Educational Research, X (November, 1974), 271-278. Toliem, William Joseph. "Factors which Affect the Decision Making of Classroom Teachers as they Evaluate Stu­ dents." (Unpublished Ph.D. dissertation, State University of New York at Buffalo, 1972) . Trow, William Clark. "On Marks, Norms, and Proficiency Scores." Phi Delta K a p p a n , XLVIII (December, 1966), 171-173. Wetzel, William A. "The use of the normal curve of dis­ tribution in estimating s tudents’ marks." School R e v i e w , XXIX (May, 1921), 373-378. Wrinkle, William, "The Story of a Secondary School Experi­ ment in Marking and Reporting." Educational Ad ­ ministration and Supervision, XXIII (October, 1937) , 481-500. 198 Secondary Sources Bangs C. W. and H. A. Greene. Teachers' Marks and Marking Systems (University of Iowa, 1930). Ebel, Robert LEssentials of Educational Measurement (Englewood Cliffs, New Jersey: Prentice Hall, Inc., 1972). Ebel, Robert L. Measuring Educational Achievement (Engle­ w o o d Cliffs, New Jersey: P r e n t i c e - H a l l , 1965). Fleming, Cecile W. A Detailed Analysis of Achievement in High School (New York: Division of Publications, Teachers College, 1925). Goodland, John and Robert H. Anderson. The Nongraded Elementary School (New York: Harcourt, Brace and C o . , 1959) . Kirschenbaum, Howard, Sidney B. Siman, and Rodney W. Napier. Wad-ia-get? (New York: Hart Publishing Co., Inc., 1971). Mayo, Samuel T. Pre-service Preparation of Teachers in E du­ cational M e a s u r e m e n t . (Toyola University, 1967). Odell, C. W. "Marks and Marking Systems." Encyclopedia of Educational Research (New York: Macmillan, 1950). Rothney, John W. M. "Evaluating and Reporting Pupil Prog­ ress." What Research Says to the Teacher, (March, 1955) . Ryans, D. G. Characteristics of Teachers (Washington, D. C . : American" Council on Education, I960). Schwartz, Alfred and Steward C. Tiedmann. Evaluating Stu­ dent Progress in a Secondary School (New York: David McKay Co., Inc., 1957). Smith, Ann Z. and John E. Dobbin. "Marks and M a r k ing Sys­ tems ." Encyclopedia of Educational Research (New York: Macmillan, 1960), 783-791. Sobel, Frances S. Teachers' Marks and Objective Tests as Indices of School Adjustment (New York: Division of Publications, Teachers College, 1936). Starch, Daniel. Educational Measurement millan, 1916). Starch, Daniel. millan, Educational Psychology 1927) . (New York: (New York: Mac­ M ac­ Taylor, J. Corey. The reliability of quarterly marks in the seventh grade of junior high scho o l . (Baltimore: John Hopkins Press, 1931). Thorndike, Robert L. "Marks and Marking Systems." Encyclo­ pedia of Educational Research (New York: Mac­ millan, 1970) , 759-766. Traxler, Arthur. Techniques of Guidance and Brothers, 1945). (New York: Harper Wrinkle, William. Improving Marking and Reporting Practices (New York: Rinehard and Co., Inc., 1947).