INFORMATION TO USERS This was produced from a copy or a document sent to us for microfilming. While the most advanced technological means to photograph and reproduce this document have been used, the quality is heavily dependent upon the quality of the material submitted. The following explanation of techniques is provided to help you understand markings or notations which may appear on this reproduction. 1.Thc sign or “ target" for pages apparently lacking from the document photographed is “Missing Page(s)". I f it was possible to obtain the missing pagc(s) or section, they arc spliced into the film along with adjacent pages. This may have necessitated cutting through an image and duplicating adjacent pages to assure you of complete continuity. 2. When an image on the film Is obliterated with a round black mark it is an indication that the Film inspector noticed cither blurred copy because of movement during exposure, or duplicate copy. Unless we meant to delete copyrighted materials that should not have been filmed, you will find a good image of the page in the adjacent frame, 3. When a map, drawing or chart, etc., is part of the material being photo* graphed the photographer lias fotlowcd a definite method in “sectioning” the material. It is customary to begin filming at the upper left hand comer of a large sheet and to continue from left to right in equal sections with small overlaps. If necessary, sectioning is continued again-beginning below the first row and continuing on until complete. 4. For any illustrations that cannot be reproduced satisfactorily by xerography, photographic prints can be purchased at additional cost and tipped into your xerographic copy. Requests can be made to our Dissertations Customer Services Department. 5. Some pages in any document may have indistinct print. In all cases we have filmed the best available copy. University, Microfilms International 300 N /E E B ROAD. AN N A R B O R . Ml 48106 IB B ED FO R D ROW. LO N D O N WC1R 4EJ, EN G LA N D 7 91 76 77 BORNSTEINt TINA J . A FOLLOW-UP STUDY» COMPARING GRADUATESa AND SUPERVISORS* RATINGS OF THE EFFECTIVENESS OF MICHIGAN STATE UNIVERSITY TEACHER EDUCATION PROGRAMS f 1 9 6 9 —1 9 7 6 1 • MICHIGAN STATE UNIV ER SITY* P H . D . * Unhcrdtv . Mtatirems International jo o * 1976 a iin o w , annahbom , m m ho o @ 1978 TINA J . BORNSTEIN ALL RIGHTS RESERVED PLEASE NOTE: In a l l cases th is m aterial has been filmed 1n the best possible way from the availab le copy. Problems encountered w ith th is document have been id e n tifie d here with a check mark . 1. Glossy photographs ________ 2. Colored Illu s tra tio n s ________ 3. Photographs with dark background________ 4. Illu s tra tio n s are poor co p y________ 5. P rin t shows through as there 1s te x t on both sides o f page _________ 6. In d is tin c t* broken o r small p rin t on severalpages throughout 7. T ig h tly bound copy w ith p r in t lo s t 1n s p in e ________ 8. Computer printo ut pages w ith in d is tin c t p r i n t ________ 9. Page(s) _ _ l acki ng when material received* and not a v a ila b le from school or author ________ 10. P a g e (s )_________seem to be missing In numbering only as te x t follows ________ 11. Poor carbon copy________ 12. Not o rig in al copy, several pages with blurred 13. Appendix pages are poor c o p y _________ 14. O riginal copy with l ig h t type _________ 15. Curling and wrinkled pages________ 16. O th e r____________________ Universe Microfilms International 300 N ZEES 3 0 .. ANN ARBOR. M l J8t06<313l 761-4700 ___ ______ type _____ ___ A FOLLOW-UP STUDY, COMPARING GRADUATES' AND SUPERVISORS' RATINGS OF THE EFFECTIVENESS OF MICHIGAN STATE UNIVERSITY TEACHER EDUCATION PROGRAMS (1969-1976) By T1na Bornsteln A DISSERTATION Submitted to Michigan State University 1n p a rtia l fu lfillm e n t of the requirements fo r the degree of DOCTOR OF PHILOSOPHY Division o f Student Teaching and Professional Development ABSTRACT A FOLLOW-UP STUDY, COMPARING GRADUATES' AND SUPERVISORS' RATINGS OF THE EFFECTIVENESS OF MICHIGAN STATE UNIVERSITY TEACHER EDUCATION PROGRAMS (1969-1976) By T1na Bornsteln Purpose The major purpose of this study was to determine the degree to which data gained from supervisors' judgments of graduates con­ trib u te to the assessment of Michigan State University's teacher education programs. Methodology Comparisons were made between graduate and supervisor ratings of graduates' a b ilit y to apply selected teaching s k ills and o f th e ir level o f commitment to the teaching profession. Supervisors' knowl­ edge and ratings o f teacher education programs were also examined. The instruments used to collect data included the "Survey o f Michigan State University College of Education Graduates," "Follow-Up Study o f Michigan State University Graduates—Supervisor Survey," and the "Success Rating Chart," used to determine graduates' scores for the student teaching experience. Tina Bornsteln The graduates in the study were selected using a s tr a tifie d random sampling procedure. Sixty Individuals were selectod fo r each group, which represented the Intersection among the fiv e programs and four graduation In te rv a ls . The sample o f supervisors was generated by the respondents 1n the graduate sample. Each graduate was strongly encouraged but not required to name his/her Immediate supervisor. Of the 269 supervisors 1n the sample, 236 returned questionnaires, fo r a return rate of 88 percent. Supervisors' and graduates' ratings were compared to deter­ mine relationships and sig n ific a n t differences among programs and years o f graduation. Whenever feasib le, ratings o f Individual Items were combined to form subscale ratings. In some o f the analyses, supervisors ratings were treated Independently, whereas 1n others they were compared to the graduates' s e lf-ra tin g s . Ma.lor Findings 1. Supervisors and graduates generally agreed on the Impor­ tance o f the 11 specified teaching s k ills , 2. The results o f the graduates' and supervisors' ratings of graduates' performance of specified teaching s k ills Indicated they did not agree about the graduates' level o f performance. 3. Supervisors' ratings of the graduates' performance o f specified teaching s k ills did not d iffe re n tia te among graduates from d iffe re n t programs. Tina B ornsteln 4. There were no differences among programs, as measured by supervisors' ratings o f the graduates' commitment to teaching. 5. The E .I.P . was ranked highest (a) on supervisors' accu­ rate Id e n tific a tio n o f the program from which the teacher had gradu­ ated, (b) as a factor fo r hiring p o te n tial, and (c) for b etter preparing graduates fo r classroom teaching. The Regular Program ranked lowest fo r (a) hiring potential and (b) fo r preparing graduates fo r classroom teaching, with Overseas and Cluster programs drawing neutral responses on these Items. 6. The graduates' student teaching reports by th e ir cooperat­ ing teachers had a very low correlation with the supervisors' ratings o f the graduates' performance of specified teaching s k ills . 7. The graduates' student teaching reports by th e ir cooperat­ ing teachers also had a very low correlation with the supervisors' ratings of the graduates' commitment to teaching. 8. No s ta tis tic a lly sig n ifica n t differences were found between the general patterns of graduates' se lf-ratin g s and the supervisors' ratings o f performance s k ills among programs and years. 9. I t was recommended that supervisors' ratings should be Included 1n future follow-up studies to evaluate teacher education programs. More research should be conducted to discover why super­ visors' and graduates' ratings o f graduates' performance s k ills d iffe red so dram atically from one another. ACKNOWLEDGMENTS I am extremely thankful to the chairman of my committee, Dr. Henry Kennedy, whose In te re s t, support, and f le x ib ilit y have helped me to re a lize my goals. In add itio n, I would lik e to thank Dr. Robert Scrlvens, Dr. Paul Slocum, and Dr. James T o lliv e r fo r serving on my committee. I am especially thankful to Dr. Jacqueline Caul, who o rig i­ nally Inspired me to take th is d irectio n , and to Dr. Donald Freeman, who kept me moving along the rig h t path. I am also grateful to Franclne Tompkins, Sherry Ralston, and Jeewaratnam Moses, who donated th e ir time to help me c o lle ct data and have been encouraging and supportive friends. F in a lly , this dissertation Is dedicated to my husband, Fred, who has always challenged me to see the other side. 111 TABLE OF CONTENTS Page LIST OF TA B LES........................................................................................... v11 Chapter I. II. III. INTRODUCTION ................................................................................ 1 Statement o f the Problem........................................................ Purposes ................................................................................... Importance of Study ................................................................ Generalizations ........................................................................ Background ................................................................................ Regular Student TeachingProgram .................................. Cluster Program .................................................................... Overseas Program ................................................................ Elementary Intern Program ................................................ Competency-Based TeacherEducation .............................. Procedure.................................................................................... Q uestions.................................................................................... H yp o th eses................................................................................ D efin itio n of Terms ................................................................ Organization o f the Study .............................................. 3 3 4 5 6 6 7 8 8 9 10 12 13 14 15 REVIEW OF RELATED LITERATURE ................................................ 18 A Review o f Trends In Evaluation of Teacher Education Programs ............................................................ A Review o f Research Concerning Supervisors' Ratings of Teachers and Teachers' Self-Ratings . . A Review o f Follow-Up Studies Using Evaluations From Graduates and Their Supervisors ...................... Summary................................................................. ;..................... 34 42 DESIGN OF THE STUDY.................................................................... 49 Description of Sample ............................................................ Measuring Instruments ............................................................ V a l i d i t y .................................................................................... R e l i a b i l i t y ................................................................................ Data C o lle c tio n ........................................................................ 49 51 53 53 54 1v 18 28 Chapter Page Treatment and Analysis o f Data ........................................ Question 1 ........................................................................... Question 2 ............................................................................ Question 3 ........................................................................... Question 4 ........................................................................... Question 5 ........................................................................... Assumptions............................................................................... L im ita tio n s ............................................................................... Summary....................................................................................... IV. V. PRESENTATION OF THE FINDINGS 55 55 56 57 57 58 58 58 60 ................................................ 62 F in d in g s .................................................................................... Question 1 ............................................................................ Question 2 . .................................................................... Question 3 ........................................................................... Question 4 ........................................................................... Question 5 ........................................................................... Summary........................................................................................ General Hypothesis 1 General Hypothesis 2 General Hypothesis 3 ........................................................ General Hypothesis 4 .................................................... . 62 62 66 69 73 76 77 77 78 78 78 SUMMARY AND CONCLUSIONS............................................................ 80 Summary........................................................................................ Design of the S tu d y ................................................................ Summary and Discussion o f the Research Findings . . . Graduates' and Supervisors' Ratings of Importance o f Teaching S k ills .................................... Graduates' and Supervisors' Ratings of Graduates' Performance o f Teaching S k ills ................................ Supervisors' Ratings of Graduate S k ill Performance to Determine Differences Among P ro g ra m s ............................................................................ Supervisors' Ratings of Graduates' Commitment to Teaching to Determine Differences Among Programs. P ro g ra m s ............................................................................ Supervisors' A b ility to Id e n tify Programs ................ Relationships Between Supervisors' Ratings and the Cooperating Teachers' Success Ratings of the Graduates' Student Teaching Experience . . . General Patterns o f Graduates' and Supervisors' Ratings o f Performance S k ills .................................... General Findings ............................................................ Including Supervisors' Ratings 1n Evaluations of Teacher Education Programs ........................................ 80 81 82 v 82 82 83 84 84 86 87 88 89 Chapter Page Conclusions................................................................................ Recommendations........................................................................ Implications ............................................................................ 89 91 92 APPENDICES.................................................................................................... 96 A. INITIAL SAMPLE S IZ E ..................................................................... 97 B. GRADUATES AND SUPERVISORS: TOTAL SAMPLE RETURNS BY YEARS AND PROGRAMS ................................................................ 99 C. GRADUATE QUESTIONNAIRE ............................................................. 101 ........................................................ 110 E. FIRST COVER LETTER TO GRADUATES ............................................. 115 F. SECOND COVER LETTERS TO GRADUATES: BY PROGRAM ................. 117 G. FIRST COVER LETTER TO SUPERVISORS ......................................... 123 H. REMINDER COVER LETTER TO SUPERVISORS ................................. 125 I. THIRD COVER LETTER TO SUPERVISORS ........................................ 127 J. SUCCESS RATING CHART ................................................................. 129 K. SUCCESS RATING SCORES OF GRADUATES' STUDENT TEACHING EXPERIENCE: BY PROGRAM ........................................ 131 L. SUPERVISORS' GENERAL COMMENTS: BY PROGRAM ......................... 136 M. GRADUATES' SELF-RATINGS AND SUPERVISORS' RATINGS OF GRADUATES' PERFORMANCE OF SPECIFIED TEACHING SKILLS, BY YEAR OF GRADUATION............................................ 140 MEAN RATINGS: SIX SOURCES OF CONTRIBUTIONS TO THE GRADUATES' PRESENT TEACHING PERFORMANCE ........................ 142 D. SUPERVISOR QUESTIONNAIRE N. BIBLIOGRAPHY ................................................................................................ vi 144 LIST OF TABLES Table 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. Page Mean Ratings: Graduate and Supervisor Ratings o f the Importance o f Specified Teaching S k ills ............................ 63 Relationships Between Subscales o f Graduate and Supervisor Ratings and Graduates' Student Teaching Success Ratings by Their Cooperating Teachers ................ 65 Factor Analysis With Varlmax Rotation of Subscale R a tin g s ............................................................................................ 67 Differences Among Programs Based Upon Supervisors' Ratings of Graduates' Performance S k ills ........................ 67 Differences Among Programs Based Upon Supervisors' Ratings o f Graduates' Commitment to Teaching ................ 68 Supervisors' Overall Ratings of Graduate Competence and Commitment to Teaching .................................................... 69 Supervisors' Knowledge of Michigan State University Teacher Education Programs .................................................... 70 Supervisors' Perceived and Actual Knowledge o f Michigan State University Teacher Education Programs .................... 72 Supervisors' Ratings o f Graduates fo r the Hiring and Preparation of Classroom Teachers ................................ 74 The Relationship Between the Success Ratings Given by the Graduates' Cooperating Teachers During Student Teaching and Graduates' and Supervisors'Ratings . . . 76 General Patterns of Differences Between Supervisors' Ratings and Graduates' Self-Ratings of Performance S k i l l s ............................................................................................ 77 Graduates' Self-Ratings and Supervisors' Ratings of Graduates' Performance o f Specified Teaching S k ills , by Year o f Graduation ................................................................ 141 Mean Ratings: Six Sources o f Contributions to the Graduates' Present Teaching Performance....... ....................... 143 v ii CHAPTER I INTRODUCTION Teachers have a tremendous Impact on the q u ality o f elementary and secondary education and must be well prepared to meet the respon­ s ib ilit ie s of a classroom. Realizing the e ffe c t a teacher can have, i t 1s Imperative that educators design unique, high-quality teacher education programs. Many in s titu tio n s tra in teachers, and most of these In s titu tio n s o ffe r a variety o f train in g programs, but chances are not a ll of these programs, or the In s titu tio n s themselves, are equally e ffe c tiv e . Therefore, i t 1s Imperative th a t these In s titu ­ tions gather data on the q u a lity o f th e ir programs with the Intention of Improving them. Because the success of an education In s ti­ tution can best be measured by the success o f Its graduates, 1t would seem that a f i r s t step 1n conducting a program evaluation would be to assess the performance of that program's graduates. "A major and continuing problem o f teacher education 1s the evaluation of Its product--namely the teacher."1 Sandefur suggested th at program evaluation can be based p a r tia lly on the evaluation of teachers who are graduates o f these programs. Presently, In s titu ­ tions are not conducting enough program evaluation research, as w ill be documented la te r. Colleges of education must put greater e ffo rt into program evaluation. Needed 1s a systematic, valid procedure 1 2 to tes t how well teachers perform the tasks they have been trained to perform. Teacher educators must evaluate th e ir graduates and adjust th e ir teacher preparation programs on the basis o f that evalua­ tion Information. To develop e ffe c tiv e programs we need continuing follow-up studies o f graduates' and th e ir employers' (supervisors') evaluations o f the In s titu tio n 's pre-service programs, combined with Input from fa c u lty , administrators, local school d is tr ic ts , and teacher organizations. In reporting on the nursing preparation program a t Amarillo College 1n Texas, Peterson concluded, “The employer follow-up has the potential capacity for determining ju s tific a tio n o f Increased expenditures, progressive program development and acco u ntability." 2 Supervisors have a first-hand understanding o f the graduates and can provide useful evaluations because of th e ir unique relationship with the teachers in th e ir schools. Some d is tin c tiv e outcomes th at might be gained from a supervisor follow-up are: - Rating graduates' professional and generic s k ills - Assessing the competence o f specific teacher education programs -Comparing the professional q u a litie s of program graduates with those of the general population o f teachers - Providing supervisors the opportunity to recommend areas o f Improvement or programs to be developed - Developing employer contacts and assisting the in s titu ­ tion with Its public relations e ffo rts 3 Statement o f the Problem In an e ffo r t to evaluate and Improve existing programs, the Division o f Student Teaching and Professional Development at Michigan State University conducted an extensive follow-up study of a sample o f College o f Education graduates from selected programs. The cur­ rent dissertation was designed to work 1n conjunction with the Graduate Follow-Up Study—to focus on supervisors' judgments o f the graduates 1n the study and to compare these judgments with the graduates' own judgments. (See page 14 fo r d efin itio n s of terms.) The research supported the assumption that both graduates and th e ir supervisors can contribute valuable postgraduation Information, providing a more global view o f the Impact Michigan State University teacher education programs have had on teacher performance and pro­ fessionalism. Purposes The major purpose o f this study was to determine the degree to which data gained from supervisors' judgments of graduates con­ trib u te to the assessment o f Michigan State U niversity's teacher education programs. More s p e c ific a lly , the study sought to deter­ mine the relationship between graduates' and supervisors' ratings of graduates' performance o f specific teaching s k ills and the Importance o f these teaching s k ills . Eleven generic teaching s k ills were Iden­ t i f ie d and relevant questions were asked of both the graduates and th e ir supervisors. Relationships were drawn between supervisors' perceptions of the graduates' a b ilit y to apply these s k ills in the 4 classroom, and the graduates' perceptions o f th e ir own a b ilit y to apply these s k ills . The second purpose o f the study was to Id e n tify s im ila ri­ tie s and differences In the re la tiv e effectiveness o f various under­ graduate teacher preparation programs, as suggested by data provided by graduates and supervisors. The th ird purpose was to analyze ratings provided by teachers and supervisors on the fiv e teacher education programs being evaluated to Id e n tify s im ila ritie s and differences among these fiv e programs. The fiv e programs under consideration were the Regular Student Teaching Program, Elementary Intern Program, Cluster Program, Over­ seas Program, and the Competency-Based Teacher Education Program. The fourth purpose of the current research was to determine the re la tiv e effectiveness of various undergraduate teacher prepara­ tory programs, as measured by supervisors' ratings o f graduates' commitment to teaching and performance of specified teaching s k ills , and the supervisors' knowledge and rating o f the value of specific teacher preparation programs at Michigan State U niversity. F in a lly , the f i f t h purpose was to determine the consistency among the ratings o f the graduates and th e ir supervisors and the evaluations o f the graduates w ritten by th e ir cooperating teachers during the student teaching experience. Importance o f Study This study Is Important fo r the following three reasons. F ir s t, 1t 1s customary to obtain follow-up Information from graduates, 5 but i t 1s worthwhile to Investigate whether data provided by super­ visors' perceptions o f graduates' commitment to the profession, per­ formance o f specific teaching s k ills , and assessment of the teacher education programs are congruent with conclusions suggested by s im ila r data from the graduates. Second, the results o f this study o f super­ visors' ratings o f Michigan State University graduates could assist in determining how much time, e ffo r t , and money should be directed to gathering and reviewing supervisors' opinions 1n future follow-up studies. Third, to Insure the continuing Improvement o f teacher education programs, i t is necessary consistently to evaluate e x is t­ ing programs. Adding supervisors' ratings to graduates' s e lf-ra tin g s In follow-up studies 1s possibly a more comprehensive method o f evaluating teacher education programs than 1s examining graduates' ratings alone. Generalizations The findings of this study may have an Impact beyond the lim its o f the study I t s e lf . Any serious e ffo r t to maintain q u a lity undergraduate programs must Include the experience and opinions of th e ir graduates, so researchers should be open to any kind o f data that could contribute to this undertaking. W ill follow-up studies be more powerful i f they Include data from supervisors? The results o f the present study indicate this method could be a model fo r others doing sim ilar follow-up studies. This study might provide Important Information to Michigan State U niversity's College o f Education about Its teacher education programs. The study might 6 also provide useful data fo r National Council fo r Accreditation o f Teacher Education (NCATE) and other forthcoming evaluations o f teacher education programs. In addition, the data could provide relevant information about which programs students should be advised to pursue. Results could also suggest which programs universities should continue to support. Background The College of Education provides professional programs fo r teachers a t a ll levels o f education. The fiv e programs being evaluated in this study are teacher education programs offered through that College's Department of Teaching and Professional Development. A b r ie f description of these fiv e programs follows. Regular Student Teaching Program The Regular Program 1s a four-year plan and includes an assignment to a student teacher center fo r 11 weeks o f student teach­ ing. This instructional a c tiv ity provides fo r actual classroom teaching experience and is an extension of the campus-based portion of the preparation program. Every student earning a Michigan elementary or secondary provisional teaching c e rtific a te through Michigan State University spends one academic quarter (1n the case o f Special Education candidates, two quarters) 1n this super­ vised laboratory experience In a cooperating school 1n one o f the fourteen student teaching centers maintained by the College of Education.3* ♦Correction: There are now eight student teaching centers. 7 Students wanting to teach 1n any of the majors offered a t M.S.U. may meet graduation requirements through the Regular Student Teach­ ing Program. Depending upon the student's major, sp ecific course- work, including general, major, minor, and professional education 1s required, culminating in the student teaching experience, and usually followed by a social foundations course to complete the requirements fo r graduation. The student teacher 1s placed In a school and 1s assigned to a supervising teacher; they cooperatively make plans to provide the student with meaningful experiences during the 11-week period. Depending 1n part on his/her readiness for teaching, the student f i r s t observes, Increasingly assumes classroom re s p o n s ib ilitie s , and for a portion of the time teaches a f u ll load. A university coordi­ nator frequently v is its the student In the school to provide guidance, In stru ction , and evaluation. Cluster Program The requirements for the Cluster Program are the same as fo r the Regular Student Teaching Program. The differences occur during the student teaching experience, where, 1n the Cluster Program, a group o f 10 to 12 student teachers 1s assigned to one school build ­ ing. A school faculty member 1s released one-half time to develop and d irec t a program fo r these student teachers; the h alf-tim e salary Is reimbursed to the school d is tr ic t by the university. Having a clu ster of students teaching 1n the same building provides increased opportu­ n itie s fo r supervision, communication, seminars, and evaluation. 8 This program was devised at Michigan State University as a model that would broaden the learning experiences o f the student teachers. Planned student contact with several teaching models, a highly Individualized experience, con­ tact with a variety o f school community a c tiv itie s , and greater Involvement o f the public school cooperating s ta ff are primary elements of th is program. 4 Overseas Program The Overseas Student Teaching Program has substantially the same pre-student-teaching requirements as the Regular Program. The University arranges fo r groups o f students {15 to 25) to be assigned to overseas American schools for th e ir term of student teaching. The experience 1s f u ll time, following the regular university time requirements. "A regular Michigan State University faculty member from the Department o f Student Teaching 1s assigned as coordinator of the program to work with the host school administration and faculty 1n planning, supervising and Implementing a program o f experiences fo r 5 each student teacher." Programs have been conducted 1n Madrid, Rome, Guadalajara, The Hague, Brussels, Lankenheath, and Belize. Teacher candidates are expected to engage in the usual a c tiv itie s of student teaching, observation, special program work, and student a c tiv itie s 1n addition to p artic ip atin g 1n available cross-cultural experiences. Elementary Intern Program As an a lte rn ative to the Regular Program in elementary education, students may ele c t the Elementary Intern Program ( E .I.P .) during the f i r s t or second term o f th e ir sophomore year. In th is program the 9 student spends two terms 1n a cooperating center during the ju n io r year, and does supervised Intern teaching the fourth year. While Intern teaching, the student earns a salary of approximately $6,000. The E .I.P . "alsoIncludes the following two main deviations from the Regular Teacher Preparation Program. Methods classes are taught 1n a block In the off-campus centers and In conjunction with some observa­ tion on the part o f the student; and the Elementary Intern spends two g summers on campus," so as to graduate w ithin a four-calendar-year period. Benefits o f the E .I.P . Include Increased classroom exposure and the Intern teaching experience. Including extensive guidance and supervision by a master teacher (an Intern consultant). Competency-Based Teacher Education The program Includes a two-term sequence of 20 term hours of c re d it, one term of which 1s student teaching. During the f i r s t term, students spend one-half day weekly 1n a school and two hours weekly In a teacher education laboratory on campus. The second term provides fo r a fu ll-tim e school experience. C lin ic a l consultants supervise both terms o f fie ld experi­ ence. The c lin ic a l consultant Is a classroom teacher relieved fo r a portion of his/her teaching load to provide th is supervision. The university reimburses the school fo r the released time, providing an opportunity fo r the consul­ tan t to work together with the university s ta ff on In-service tra in in g and program development.7 During this time students 1n the program complete the study and are evaluated on 14 selected major competencies within 5 major areas of teaching re sp o n s ib ility. Benefits of the program Include Increased school exposure and d irect application of learned s k ills and theory V 10 in micro-teaching and in the actual classroom. Final evaluation is based on the student teachers' performance o f the specified s k ills . Pre-student-teaching experiences are the same in the Regular* C luster, and Overseas programs, depending on the in d ivid u al's major and minor. The E .I.P . is d iffe re n t, in th at the student spends two terms of the ju n io r year 1n the cooperating center, taking methods classes and student teaching, followed by intern teaching the senior year. In the C.B.T.E. program, the e n tire experience comprises a two-term, 20-cred it course that includes the social foundations m aterial. Procedure In 1977-78, the Division of Student Teaching and Professional Development conducted the Graduate Follow-Up Study to assess gradu­ ates' opinions about the existing teacher education programs and th e ir suggestions for improvement. ment variables In the study: There were two s ig n ific a n t tre a t­ (1) date o f graduation and (2) student teaching program in which the individual was enrolled. academic years were selected fo r study: and 1975-76. The following 1969-70, 1971-72, 1974-75, These graduation intervals were selected to provide a base fo r p lo ttin g trends in selected aspects o f professional develop­ ment. These p a rtic u la r years were also chosen because o f the av a ila ­ b i l i t y of a sample fo r 1969-70 from the statewide study, "The Impact of Student Teaching Programs Upon the Cooperating Public Schools 1n Michigan."8 The 1974-75 and 1975-76 school years were selected so as to Include the C.B.T.E. program, and 1971-72 was selected 11 to insure r e a lis tic intervals to span the seven years of the study. The fiv e student teaching programs selected fo r study were: Regular, Cluster, Overseas, Elementary Intern Program ( E . I. P . ) , and Competency-Based Teacher Education (C .B .T .E ,). The 5x4 matrix resulting from the cross between teacher preparation programs and years of graduation had a to ta l of 20 c e lls . Because 1t is a com­ p aratively new program, there were no graduates of the C.B.T.E. program fo r the years 1969-70 and 1971-72. were empty. Thus, these two cells A random sample of 60 individuals from each o f the other 18 ce lls participated 1n the graduate study. sample size was 18 x 60 = 1,080 individuals. The to ta l graduate (See Appendix A .) The primary focus of the current study was to examine the contributions that graduates' supervisors can make to a more thorough evaluation of the fiv e teacher education programs. This study was based upon the data collected from the Immediate supervisors of the gradu­ ates included in the Graduate Follow-Up Study. The supervisor data were compared s p e c ific a lly to the graduate data concerning the (1) perceived Importance of specified teaching competencies, (2) perceived performance o f specified teaching s k ills , and (3) per­ ceived level of commitment to the teaching profession. Supervisors were also asked to report th e ir knowledge of the five teacher edu­ cation programs. This study also compared the supervisors' ratings to the rating o f the graduates' fin a l evaluation by th e ir cooperating teachers during the student teaching experience. 12 Questions The study was designed to answer the following sp ecific ques­ tions as to whether or not supervisors' ratings should be included in graduate follow-up studies evaluating teacher education programs: I. II. III. IV. V. To what extent w ill ratings provided by supervisors d if f e r from those provided by graduates? a. W ill ratings of the "importance" o f specific teaching s k ills provided by Michigan State University graduates d iff e r from those provided by th e ir supervisors? b. To what extent w ill ratings o f "performance" of specific teaching s k ills provided by Michigan State University graduates d if fe r from those provided by th e ir supervisors? W ill judgments provided exclusively by supervisors assist in determining differences among teacher education programs? a. W ill supervisors' ratings of the graduates' performance of specified teaching s k ills d if fe r among the fiv e teacher education programs? b. Will supervisors' ratings of the professional commitment of graduates to teaching d iff e r among the specified teacher education programs? Does a supervisor's knowledge of specific teacher education programs d if f e r from one program to another? W ill there be a s ig n ific a n t relationship between the super­ visors' ratings of the graduates and the ratings of graduates by th e ir cooperating teachers during the student teaching experience? a. Will supervisors' ratings of the graduates' commitment to teaching d iffe r from sim ilar ratings provided by th e ir cooperating teachers? b. W ill supervisors' ratings of graduates' performance of specified teaching s k ills d if f e r from lik e ratings pro­ vided by th e ir cooperating teachers? Are the variations among programs suggested by data provided by graduates the same as variations among programs suggested by data provided by supervisors? 13 Hypotheses The study was based on the premise that supervisors' judg­ ments should be included in graduate follow-up studies, to evaluate teacher education programs more accurately. To validate this b e lie f, the following hypotheses were tested: H*: Ratings provided by supervisors w ill be s ig n ific a n tly d iffe re n t from those provided by graduates. This hypothesis w ill be regarded as true i f there is a sig­ n ific a n t difference between supervisor and graduate judgments of the importance o f specific teaching s k ills and of the graduates' perfor­ mance of these specific teaching s k ills . H2 : Information provided by supervisors w ill be valuable in determining differences among the specified teacher edu­ cation programs. This hypothesis w ill be regarded as true 1f there is a sig­ n ific a n t difference among teacher education programs, as measured by supervisors* ratings of graduates' professional commitment to teach­ ing and th e ir performance of specified teaching s k ills . H3 : Supervisors' knowledge o f teacher d iffe r among programs. education programs w ill This hypothesis w ill be regarded as true i f there 1s a sig­ n ific a n t difference among programs as measured by the supervisors' a b ility to id e n tify the sp ecific teacher education programs from which the teachers graduated. H*: There w ill be a s ig n ifica n t correlation between the super­ visors' ratings o f graduates and the ratings of graduates by th e ir cooperating teachers during the student teaching experience. 14 This hypothesis w ill be regarded as true i f a sig n ifican t s ta tis tic a l relationship is found between cooperating teachers' ratings and supervisors' ratings of graduates' conmltment to teaching and performance o f specified teaching s k ills . H&: Variations among programs suggested by graduates' judg­ ments w ill be s ig n ific a n tly d iffe re n t from variations among programs suggested by supervisors' judgments. This hypothesis w ill be regarded as true 1f data provided by graduates and those provided by supervisors suggest s ig n ific a n t d if ­ ferences among programs. D e fin itio n o f Terms Competence—Abi 1itv to apply the essential principles and techniques of teaching to practical situ ation s. Cooperating Teacher—The classroom teacher who supervised the graduate during his/her student teaching experience. Graduates—Those who completed a specified teacher education program and received a degree from Michigan State U niversity. Performance—Actual accomplishment as distinguished from potential a b ilit y , capacity, or aptitude. Professional Commitment—The degree to which a teacher has attained the specialized attitudes and dedication that characterize commitment to teaching, as judged by the supervisor. Rating—An estimate, made according to some systematic pro­ cedure, of the degree to which a person or thing possesses a given ch aracteristic. 15 Teaching S k ills —Specific actions that the teacher has learned to perform with ease and precision; may be e ith e r cognitive, psychomotor, or a ffe c tiv e performance (e .g ., the a b ilit y to re la te to students). Supervisors—Persons id e n tifie d by the graduates in the sample as the Immediate supervisors to whom they are responsible. This could include p rin cip als, assistant p rincipals, department heads, or any other supervisory person designated In that p a rtic u la r school system. Organization of the Study In the f i r s t chapter an introduction to and statement of the problem were presented, along with a b r ie f description o f the proce­ dures employed in the study. The major questions to be tested were stated, and significance o f the research, generalizations, and d efinition s pertinent to the study were discussed. Chapter I I , the review o f related lite r a tu r e , contains three sections: a review of trends In evaluation of teacher education programs, a review o f research on supervisors' ratings of teachers and teachers' s e lf-ra tin g s , and a review of follow-up studies using evaluations from graduates and th e ir supervisors. The design of the study 1s detailed In Chapter I I I . Included are a description of the sample, information on the measuring in stru ­ ments, and an explanation o f the procedures used in collecting and analyzing the data. The hypotheses to be tested are stated and the s ta tis tic a l analysis is described. 16 The findings o f the study are reported In Chapter IV. Conclusions and recommendations, as well as implications fo r further studies, are presented In the fin a l chapter. 17 Footnotes—Chapter I ^J. T. Sandefur, Illu s tra te d Model fo r Evaluation of Teacher Education Graduates (Washington, D.C.: AACTE, 1970), p. 1. 2 Larry K. Patterson, Graduate Employee Follow-Up (Am arillo, Amarillo College, 1977), p. 9. 3 Hugo David, e d ., Toward Excellence in Student Teaching (Dubuque, Iowa: Kendall/Hunt Publishing Co., 1973), p7 l . 4 Charles Jackson, "A Study o f Selected Student Teaching Experiences Reported by Michigan State University Cluster Program and Conventional Program Student Teachers” (Ph.D. d issertatio n , Michigan State U n iversity, 1971), pp. 16-17. 5 Banks Bradley, "Overseas Student Teaching: A Follow-Up Study Report as an Assessment o f In te rc u ltu ral Experiences in Student Teaching” (East Lansing: Michigan State University Department of Student Teaching, 1975), p. 1. Texas: ^Robert Arends, "A Comparative Study o f the Graduates o f the Michigan State University Elementary Intern Program and the Regular Teacher Education Program” (Ph.D. d is se rta tio n , Michigan State Uni­ v e rs ity , 1969), p. 7. ^Michigan State University and p artic ip atin g school dis­ t r ic t s , "P .O .I.N .T .E ." (East Lansing: Division o f Student Teaching and Professional Development, 1978), p. 3. Q Deans and Directors of Teacher Education in Michigan, Student Teaching Programs: Questions and Answers. "The Impact of StVdentTteaching Programs Upon the Cooperating Public Schools in Michigan" (June 1970). CHAPTER I I REVIEW OF RELATED LITERATURE An exhaustive examination, including a thorough lib ra ry search and d irect correspondence with the individuals and institutions pres­ ently involved in research pertaining to teacher education, revealed several studies focusing on evaluations of programs that were somewhat related to the current study. However, none of those studies included the d iv ersitie s of procedures and designs necessary to give a basis for answering the specific questions posed in Chapter I . This chapter con­ tains three sections designed to provide the reader with the background necessary for understanding the conceptual framework of this study. The three sections include a review o f trends in evaluation of teacher edu­ cation programs, a review o f research concerning supervisors' ratings of teachers and teachers' s e lf-ra tin g s , and a review o f follow-up studies using evaluations from graduates and th e ir supervisors. A Review of Trends in Evaluation of Teacher Education Programs' The review of trends in evaluation of teacher education programs includes a look at the guidelines and recommended standards fo r accreditation of basic and advanced teacher preparation programs. Using these standards as a broad base from which to develop actual evaluation models, a review of experts' opinions outlines the direc­ tion that evaluation of teacher education programs should take. 18 19 Also presented In this section are two models that suggest actual procedures for the effective evaluation o f teacher education programs. Educators on the national level are continuously s triv in g to Improve the q uality o f teaching in th is nation. For example, a major goal of the American Association o f Colleges fo r Teacher Edu­ cation (AACTE) 1s to set up procedures th at w ill assure the public that accredited programs "meet national standards of q u a lity ," that "children and youth are served by well prepared personnel," and that the teaching profession 1s advanced "through the Improvement of preparation programs."^ The new standards of the AACTE establish a relationship between "the nature of programs, and the teaching a b ility of th e ir graduates, and the values which should inform f s ic l 2 e ffo rts toward Improving programs." The National Council fo r Accreditation of Teacher Education (NCATE) has set certain standards fo r educators. The following statement prefaces standards 6.1 and 6.2 on evaluation: The ultimate c rite rio n fo r judging a teacher education pro­ gram Is whether 1t produces competent graduates who enter the profession and perform e ffe c tiv e ly . An In s titu tio n com­ mitted to the preparation o f teachers keeps abreast o f new developments 1n the evaluation of teacher education person­ nel and engages 1n systematic e ffo rts to evaluate the q u a lity of Its graduates . . . when they complete th e ir programs of study, and a fte r they enter the teaching profession.3 Standard 6.2 reads: "The In s titu tio n regularly evaluates its teacher education programs and uses the results of Its evaluation 1n the modl4 f1cation and improvements of those programs." Teacher educators agree that evaluation of teacher education programs and th e ir curricula 1s essential to ensure q u a lity preparation 20 of teachers. Heath claimed, "Whether as a developmental function, as an aid to the practicing educator, or as fundamental research, evaluation o f the interaction between a curriculum and it s environ5 ment seems e s se n tia l." Mayhew agreed that "the establishment, opera­ tio n , and evaluation o f the curriculum ought to be one of the central resp o nsib ilities of college facu lties and academic administration. It is the vehicle through which the In s titu tio n seeks to make its most s ig n ifica n t impact on the lives o f students." g Woodring concurred: "In spite o f the fa c t that projects 1n teacher education are, by th e ir very nature, d if f ic u lt to evaluate, the problem o f evaluation must be accepted as a major resp o nsib ility o f a ll experimental pro­ jects 1f we are to know the extent o f th e ir success."^ Piyush Implied the importance o f program evaluation when he wrote, "The q uality of education in schools hinges prim arily on the q u a lity of teachers. High levels o f professional competence o f teachers are crucial fo r the successful dissemination o f human knowledge from one Q generation to the next." The necessity o f evaluation has been established and con­ firmed, but extensive program evaluations are not being undertaken. There are many reasons fo r th is . Woodring claimed, Never before 1n history has a major nation provided so much education fo r so many fo r so long. Unfortunately the empha­ sis on quantity has not been accompanied by a s im ila r empha­ sis on q u a lity ; we provide more education but 1t 1s not a ll th at clear that we provide b etter education than other nations, or an education th at 1s nearly as good as we can provide, with our vast resources.9 Besides emphasizing quantity rather than q u ality in education, edu­ cators are confused about the methodology to follow fo r successful 21 evaluation. Evaluation 1s more than a mere compilation of factual data; "1t Implies a system of values and decisions about values Involving human judgment."*® Evaluation requires decisions about accepted practices and th e ir possible Improvement. Sandefur claimed th at since re lia b le evaluation tools have been developed there 1s no excuse for not proceeding to evaluate teacher education programs. I t 1s evident th a t teacher education In s titu tio n s have largely Ignored the evaluation o f th e ir graduates. This fa ilu r e has been due prim arily to the profession's In a b il­ it y to determine what constitutes e ffe c tiv e teaching. For­ tunately both of-these conditions which have prohibited evaluation have been removed and that teacher education In s titu tio n s must now move ahead with systematic approaches to evaluating th e ir products.H He offered two premises: 1. A s u ffic ie n t body o f research now exists from which in fe r­ ences may be drawn, and substantiated, on the characteristics of good teaching and good teachers. The findings of research on teaching and learning form a configuration that is subject to order and can be Incorporated Into Instructional schemata. 2. Classroom observational systems and other evaluative tools have been developed which enable educators to assess teaching behavior in a systematic fashion. The next Issue of methodology to contend with 1s: evaluate the graduates? Mho should The current study u tiliz e d supervisors' evaluations and graduates' self-evalu atio n s, but the researcher cannot ignore the worth o f input from other sources. A controversial argument fo r accountability 1s that teacher ratings should be based 22 on student performance. This argument could very well be applied to the evaluation o f teacher education programs based on the performance of pupils In the graduate's classroom. Herbert stated: I t has often been argued that the v a lid ity o f the evalua­ tion o f a teacher preparation program Increases 1f the ev i­ dence is collected as close as possible to the fin a l product — the changes 1n the pupil. Though Indisputable in theory, th is argument does not work 1n practice. While we should do more and b etter research on which teacher behaviors result 1n changes In pupil beha­ v io r, i t is not expedient to evaluate teacher preparation programs, by such changes 1n the schools where the teachers find employment. Pupil changes occur to a great number of d iffe re n t individuals, each of unknown personality, unpredict­ able cultural conditioning and Idiosyncratic response. The reaction to any teacher cannot necessarily be a ttrib u ted to the teacher much less the teacher's preparation.12 Evaluators of teacher education programs suggest using con­ trib ution s from the pupils, peers, supervisors, and the s e lf-ra tin g s o f graduates. Another method o f evaluation 1s the evaluation of graduate teaching performance by trained observers. The major focus of Michigan State U niversity's graduate follow-up study was s e lfevaluation. Kaufman supported th is approach: "The most common approach to the evaluation of Instruction now 1s through student 13 evaluation of th e ir learning experiences." He also ju s tifie d s o lic itin g supervisor ra tin g s, c itin g s p e c ific a lly a study evaluat­ ing South Alabama U niversity's teacher train in g program: Two instruments, severally addressed to two groups of respondents, 1n the ultim ate analysis, converge on an appraisal of the work of the University 1n the s k ills , Insights, knowledge essential to e ffe c tiv e teaching. The selection of the two sources o f feedback (graduates and th e ir supervisors) on South Alabama's teacher train in g program is apt and ju d ic io u s .14 23 An extensive discussion of the v a lid ity o f supervisor ratings and teacher se lf-ra tin g s 1s Included 1n the second section of this lite ra tu re review. The process o f evaluation 1n education Is a measure of the achievement of objectives and of the possible need fo r re d e fin itio n or modification o f these objectives. There Is a need fo r the evalua t1on process as an aid 1n identifying and abandoning outmoded and obsolete practices while promoting relevant and useful programs. Baer endorsed the need fo r evaluation o f teacher education programs. He wrote: Increasingly those Individuals and In s titu tio n s working with teachers-to-be are feelin g and assuming resp o n sib ility fo r the performance o f th e ir graduates. Recognizing th at teach­ ing success Is d ire c tly proportional to the q u a lity o f teacher preparation, In s titu tio n s are seeking to measure the e ffe c tiv e ­ ness o f th e ir programs.'5 Grommon conducted a survey of 392 colleges o f education to discover the extent, nature, and value o f follow-up programs fo r program evaluation, and concluded that not enough evaluation 1s being done. He revealed that o n e -fifth of the u n iversities studied had conducted follow-up studies of secondary programs, and one-half had conducted studies of elementary programs. types of studies— formal and Inform al. He Id e n tifie d two Informal studies, he said, are extremely unreliable; "fewer than 10% performed formal follow up studies o f the 392 colleges surveyed."^® Based on his research, Grommon made the following suggestions fo r follow-up improvements: (1) people 1n charge o f follow-up studies are now assigned the task as extra duty, whereas f u ll time should be devoted to such research; 24 (2) researchers must substantiate th e ir findings and make the appropriate changes based on these resu lts; and (3) follow-up studies should be conducted fo r more than firs t-y e a r teachers. A more recent survey, by Peques, revealed that 42 percent of NCATE-accredited In s titu tio n s are not conducting follow-up evaluations. She contended that "Parent In s titu tio n s appear to be exerting only minimal pressure on tra in in g programs to conduct follow up evaluations. The external pressure provided by NCATE, NASDTEC and Regional Accrediting Associations may not be potent enough factors 1n Influencing the extent of follow-up evaluation p ra c tic e s ." ^ It becomes evident that despite the Increase 1n Information supporting the value o f follow-up studies, the actual practice o f conducting such studies has increased l i t t l e 1n the past decade. Sandefur recommended a p ractical model fo r conducting follow up studies fo r the evaluation o f teacher education programs. Depend­ ing on the a v a ila b ility o f time, money, and human resources, an in s titu tio n can adapt and f i t th is model to Its own p a rtic u la r needs. Sandefur cautioned th at "any model fo r evaluating the product of teacher education w ill be Inadequate and Incomplete. The problems are too great and the knowledge about evaluation too lim ited to allow 18 the presentation o f a model which 1s not subject to c ritic is m ." He suggested, "The proposed evaluative data can be derived from four categories: (A) career lin e data; (8) d ire c t classroom observation; (C) p u p il, peer, and supervisor evaluations; and (D) standardized 19 measures." He defined these four categories as follows: 25 Career Line Data: Demographic Information, program success (6PA), continued professional preparation, and perception o f the q u ality of th e ir teacher preparation programs'are examples of the data that constitute career lin e data. A ll these data, collected from the p a rtic ip atin g teachers, are concerned with th e ir personal background, professional development, and perceptions of. the prepara­ tory program. D irect Classroom Observation: A category o f data that has been shown to be most important in the study of teacher e ffe c tiv e ­ ness is d ire c t observation of the teacher and students in the class­ room. This observation should be systematic and provide data s u it­ able fo r assessing a teacher's competence. P upil, Peer, and Supervisor Evaluations: I t seems reasonable to c o lle c t data from the sources most closely associated with the teachei— students, peers, and supervisors. "Rating scales have the advantage o f allowing the researcher to use a human observer to descrlbe characteristics of another person." 20 Sandefur found a d e fin ite relationship between these sources, lending credence to the thesis that "supervisors as well as pupils can consistently id e n tify these 1mportant teaching behaviors." 21 Standardized Measures: Sandefur discussed the drawbacks of using standardized achievement tests to measure student gain, and did not recommend using such tests as product measures In evaluating teacher education programs. However, more research is needed 1n th is area to determine those teaching competencies and characteris­ tic s th a t can and should be evaluated. Sandefur suggested that 26 standardized measures, especially the authoritarianism measure, be used to assess certain personality characteristics that seem desir­ able in teachers. However, he pointed out the inconclusiveness of research in this area. Sandefur summarized the development o f his model: The e ffo r t to design a model fo r the evaluation o f teacher edu­ cation graduates was based on two major premises: that a suf­ fic ie n t body o f research was now in existence from which generalizations on good teaching and good teachers can be drawn, and th at classroom observational systems and other eval­ uative tools had been developed which enabled educators to evaluate systematically the product o f teacher education pro­ grams 1n the lig h t o f research findings. The overriding premise was, of course, the position that In s titu tio n s o f teacher educa­ tion had h is to ric a lly Ignored the whole area o f evaluation but were now required to face th is Issue because o f the new stan­ dards Implemented by the National Council for Accreditation of Teacher Education.22 In an attempt to help Pennsylvania State University keep b etter track o f its teacher education graduates, Peshkopla developed 23 a model to be used In conducting follow-up research. He suggested that such research should be conducted 1n the following stages: Stage A: Secure a l i s t o f prospective teacher education graduates--prior to graduation--to Insure correct addresses. Stage B: Generate a sample of prospective teacher education graduates. Stage C: Orient the sample of prospective teacher education graduates— student Involvement w ill increase response. Stage D: Collect data from the sample and evaluations from th e ir supervisors. Generally the purpose o f [submitting] the evaluation scale to employers was to obtain feedback about graduates th at would assist faculty with program improvement. The questionnaire 27 fo r the graduates and the teacher evaluation scale fo r th e ir employers were designed to discover: 1. Names and addresses o f the persons d ire c tly responsible fo r the supervision o f former graduates who are teaching. 2. The percentage o f graduates who actually enter teaching 1n September Immediately a fte r graduation. 3. The percentage who re je c t teaching completely and who decide to seek more a ttra c tiv e opportunities. 4. The e ffe c t which geographic constraints, student teach­ ing or other factors have on job placement. 5. Evaluations of f i r s t year teachers by th e ir employers. 6. Opinions from graduates concerning th e ir teacher educa­ tion programs.24 Stage E: Analyze Information from the Teacher Questionnaire and supervisors' evaluations. Stage F: Improve the plan fo r next re p e titio n . Stage G: Conduct a cost analysis o f the plan. Realizing that the cost factor Is often the major deterrent to education In s titu tio n s ' conducting follow-up studies, Peshkopla devised a fle x ib le , reasonably economic model. He concluded, Educators agree th at follow-up of graduates 1s Important but have found th at operating a system o f regular follow-up of graduates 1s not only costly but adm inistratively cumbersome. The q u ality o f return fo r the to ta l e ffo rt expended usually 1s disappointing. This fie ld te s t should be Important to NCATE and teacher education In s titu tio n s 1n general fo r i t demonstrates a cost e ffe c tiv e follow-up system fo r gathering information about teacher graduates which can be described in NCATE lite ra tu r e and adapted fo r use by any college or u n iv e rs ity .25 U niversities must not lose sight of the purpose of follow-up research. The actual process o f conducting such studies 1s very demanding 1n terms of human resources, and 1s also very expensive. The goal, again, 1s to make necessary Improvements 1n existing teacher education programs. Johnson substantiated th is statement: 28 I t must be stated th at a study of this type 1s meant to pro­ vide feedback regarding the educational experiences o f under­ graduates 1n the college of education. Evaluation o f the results o f th is feedback must result 1n m odification, both q u a lita tiv e and q u a n tita tiv e , o f the experiences provided students. I f this feedback 1s not examined and used as a c rite rio n fo r curriculum change and teaching emphasis, the purposes for performing this Investigation have not been met.26 I f the results of a given study provide d e fin ite Information fo r the a lte ra tio n o f a program, the results must be heeded, or the study was fo r naught. A Review of Research Concerning Supervisors* Ratings oT teachers and teachers' Self-Ratings A s ig n ific a n t number o f researchers have analyzed the v a lid ity o f supervisors' ratings of teachers and teachers' s e lf-ra tin g s . The purpose of th is section Is to review the lite ra tu r e concerned with using supervisors' and graduates' ratings 1n evaluating teacher edu­ cation programs. There are also other reasons fo r using graduate and supervisor ratings. Swartz reported, "A proclaimed ju s t if ic a ­ tion of the evaluation procedure 1s to serve adm inistrative purposes ( i . e . , promotion, tenure, salary adjustment) and to help teachers to 27 improve the instructional process." Also reviewed in th is section 1s research conducted on the v a lid ity of supervisors' and teachers' perceptions o f the teacher's performance. This review 1s Important because the current study was Intended to analyze supervisors' ratings and graduates' s e lf-ra tin g s 1n relatio n to th e ir contribution to the improvement o f teacher education programs. In his a r tic le on teacher evaluation, McAfee posed the f o l­ lowing questions: 29 I f you were to take a group o f teachers and th e ir inrnediate supervisors and ask them the same questions about the job being done by the teachers, how closely would they agree? Would they agree as to the importance o f the objectives, and how well the objectives are attained? Would they agree as to how well and how e ffe c tiv e ly the teacher works with d i f ­ ferent types o f students, his supervisor, other teachers, parents, and the community In general (add professionalism and training)? Would they agree as to the effectiveness of the teacher's a b ilit y to communicate with students, motivate them, and help them to improve th e ir self-image?28 McAfee attempted to answer these questions through a study he conducted fo r Southern Illin o is U niversity. He asked 1969-70 and 1970-71 teaching graduates to evaluate themselves and th e ir performance 1n re latio n to an Inventory of 41 Items designed to measure teacher performance and personal ch aracteristics. The graduates furnished th e ir supervisors' names, and the supervisors were asked to evaluate the graduates using the same Instrument. The results o f the survey were varied. Teachers and super­ visors did not agree about who (the teacher or the supervisor) set the objectives fo r the course being taught by the teacher. In regard to meeting th e ir objectives, supervisors gave the teachers a higher rating (88 percent) than the teachers gave themselves (68 percent). Both parties agreed that the teachers had an above-average under­ standing of students. They also concurred on the following teacher performance and personality ch aracteristics: an above-average relationship with th e ir supervisors; an average relationship with parents o f students; and above-average ratings on personal char­ a c te ris tic s , teachers' a b ilit y to rate themselves, and planning s k ills . Of the 51 Items in the survey, teachers and supervisors agreed on 35 percent, or 17 items. McAfee stated, 30 I t is apparent from the response to the Items on the ques­ tionnaire by both teachers and supervisors that they do not often agree on th e ir estimates of the teachers' a b ilit ie s , professional s k ills , a ttitu d e s , public relations and knowl­ edge, and use o f various methods and techniques of teaching.29 The lack of agreement between supervisors and teachers 1n McAfee's study damaged the v a lid ity o f both responses. He concluded, "with such a wide variation between teachers' and supervisors' responses 1t seems possible that e ith e r the teachers or the supervisors or both are incapable of correctly evaluating the teachers' performance, back­ ground, and a b i l i t i e s . " ^ Crisp conducted a study to determine how experienced secondary school English teachers evaluated themselves 1n knowledge of English and 1n English teaching a b ilit ie s . He found, On the basis o f the data obtained 1n this survey, teachers with more years of teaching experience do 1n fa c t tend to rate themselves higher 1n given areas o f knowledge 1n English and knowledge and s k ill 1n the teaching of English than do teachers with less experience.31 Cook and Richards id e n tifie d "one of the most d if f i c u l t prob­ lems facing supervisors o f teachers and principals o f schools where beginning teachers are performing 1n actual classroom situ atio n s, 1s 32 that o f determining the success of teacher behavior." In th e ir study to reveal the dimensions o f principal and supervisor ratings o f teacher behavior, they interpreted the results to mean that the rating scales generated data that were more a re fle c tio n o f the ra te r's point o f view than of a teacher's actual classroom behavior. They found that the teachers' ratings may re fle c t the expectations o f the evaluator more than the actions of the teacher. Regardless of how these results are In terp reted , however, i t 1s clear 31 that Information about a teacher's performance based on rating scales should be Interpreted with caution.33 The purpose o f Carey's study was to examine the v a lid ity of the practice o f Interpreting teachers' perceptions o f th e ir perfor­ mance as an Indication o f th e ir actual performance on specified teaching s k ills . Many state departments of education, u n iv e rs itie s , and teacher centers use th is procedure to assess students and teachers on id e n tifie d s k ills , to evaluate existing train in g programs, and make decisions about forthcoming teacher tra in in g . Carey compared teachers' se lf-ra tin g s to trained observers' ratings. He stated, Using teachers' perception scores to predict actual per­ formance scores on teaching s k ills appears to be an In valid practice. This finding held true for three d iffe re n t types o f questions, namely, recall o f verbal Information, concept Id e n tific a tio n , and problem solving questions.34 He warned, . . . Basing Inservice on teachers' said need does not appear to be v a lid . The acquisition o f accurate data concerning teachers' competence on specified teaching s k ills Is not the only Ingredient that should be used to Id e n tify 1nserv1ce train ing p r io r it ie s .35 The evidence presented by the four researchers discussed above demonstrates the human fac to r of misperception 1n evaluating teaching performance. Herbert summarized the lim ita tio n s o f super­ visor ratings: Information of th is kind 1s r e la tiv e ly accessible since 1t can be gathered d ire c tly or by Interview , or by other tech­ niques, not as d ire c t. Such Information, however, 1s subject to a number of lim ita tio n s . Procedures and c r ite r ia fo r eval­ uating teachers vary from d is t r ic t to d is t r ic t , and frequently the evidence on which ratings are based 1s very meager or second hand. The personality o f the principal also seems to 32 have a substantial e ffe c t on the ratings o f a teacher's a b ilit y and social competence. In addition, school dis­ tr ic ts and college supervisors do not agree in th e ir ratings o f teachers. Perhaps the attempts to divide teachers into types based on p ro file s o f a ttrib u te s they have in the p rin ­ cip als' judgment may be more v a lid , but the evidence is not strong.3® Hardebeck's study was designed to determine the degree to which teachers mastered Individualized instruction by comparing trained education observers' evaluations and teachers' s e lfevaluations. According to th at researcher, Both observations and surveys are acceptable methods to assess and describe characteristics or situations. Obser­ vations by trained observers have the advantage o f poten­ t i a l l y high inter-observer r e l ia b i l it y . Surveys have the advantage o f re la tiv e low cost and lack of geographical re s tra in ts .37 Hardebeck fu rth e r claimed. Researchers often gather data by sending questionnaires, opinlonnaires, or other s e lf reporting forms to the popu­ la tio n being studied. Is 1t reasonable to expect r e lia ­ b i l i t y from responses which come from individuals untrained in the use o f the p a rtic u la r instrument and possibly unin­ formed as to the intended meaning o f terminology used 1n the instrument?3® A p a rtia l answer to Hardebeck's question was presented by • Worle, who studied the effects of train in g on the variance 1n teachers' ratings. factors. He found that variance was Influenced by two F ir s t, greater variance existed when raters used unsealed items than when they used scaled Items. This supported H arris, Bessent, and McIntyre's claim that when a ra te r had specific beha­ vio ral items upon which to focus his attention the ratings tended to become more uniform than when only general terms were offered. 39 Second, tra in in g did a lte r variance. 33 Swartz conducted a mail survey to study the differences among group ratings of the same Instru cto r. A population o f 72 teachers o f d iffe re n t backgrounds, teaching In the fie ld of trade and Ind ustrial education, was selected. Each Instructor rated him* s e lf and was rated by one school adm inistrator, a supervisor, two teaching colleagues, and pupils. The results revealed that the school adm inistrators, supervisors, and colleagues had sim ilar views, whereas students' ratings and teachers' se lf-ratin g s did not reveal many s im ila r itie s . Swartz concluded, " I t Implies that the emphasis o f teaching effectiveness Is placed d iffe re n tly by d iffe re n t groups, and thus ratings from a single group of raters would not reveal a 40 to ta l picture o f the teaching effectiveness of an In s tru cto r." Swartz cited some o f the related research supporting his study: A 1969 study by Johnson and Radebaugh found th at adminis­ tra to rs could e ffe c tiv e ly evaluate and Id e n tify superior teachers. Owens, 1n 1971, concluded that administrators, teachers, and college 41 supervisors perceived most areas of teacher competence s im ila rly . Results of the research on teacher evaluation are extremely c o n flic tin g . For the purposes o f the current study, 1t would have been negligent to avoid the findings c itin g the lim itatio n s of supervisors' ratings and teachers' s e lf-ra tin g s . The two major lim itatio n s appear to be nonsystematized ratings and human misperceptions. In the current study, both the supervisor and the graduate rated teaching performance with the same instrument. The researcher understands the possible dynamics of perceptions and p a r tia lly compensated by comparing the evaluations with the student 34 teaching evaluations. The research supporting the use o f supervisors' and teachers' evaluations of teaching performance lends credence to the current study. The next section Includes a review o f follow-up studies using evaluations from graduates and th e ir supervisors, and th e ir Im pli­ cations fo r the present research. A Review of Follow-Up Studies Using Evaluations From Graduates and Their Supervisors This section o f the lite ra tu r e review cites recent follow-up studies s o lic itin g evaluation data,from both graduates and th e ir supervisors. Kaufman supported this approach to the evaluation of teacher education programs: A variation that seems to us most appropriate 1s the use of dual samples. In th is case you are Interested 1n comparing the responses o f one or more samples from one or more popu­ latio n s. We recommend to the reader th at your survey design be set up to accommodate both graduates and th e ir Immediate supervisors. I f you query both samples, you can compare th e ir responses to see 1f there 1s agreement or disagreement. In this p a rtic u la r case o f dual sampling we are re a lly using the sample from one population to generate the sample from another population. This is p erfectly acceptable and 1n this case necessary 1f you are to match the responses of the gradu­ ates to the responses o f th e ir Immediate supervisors.42 The purpose o f Plyush's study was to follow up graduates o f a preservice field-based program a fte r they had taught one to fiv e years. The sample was composed o f 86 graduates employed as fu ll-tim e science teachers 1n Ohio during 1974 and 1975. They had been teach­ ing fo r a period ranging from one to fiv e years, a t d iffe re n t levels {elementary, ju n io r high, and senior high school), and in d iffe re n t settings (urban and suburban). The s o lic ite d responses came from 35 administrators, students, and the teachers themselves. Preservice data included each subject's score on the Science Classroom A c tiv i­ tie s Checklist and h is/h er grade point average. The data analysis revealed there were no sig n ific a n t d if f e r ­ ences in changes of teachers' views regarding the appropriate types of classroom a c tiv itie s , or in the types of a c tiv itie s implemented by the teachers with one to fiv e years of teaching experience. The graduates' views toward inquiry-oriented teaching and the use of such a c tiv itie s 1n the classroom remained stable a fte r fiv e years of teaching. The administrators' views regarding science teaching and the support given science teachers were a strong independent pre­ dictor variable. I t is Important to take into account the Independent variables 1n teacher education, especially when attempting to evaluate a teacher education program by the performance o f its graduates. Piyush stated, Teaching-learning is a complex process. The participants in this process, a teacher and a learn er, bring to a classroom th e ir own personality ch aracteristics, unique motivations, and expectations. I t is easy to decide on rational state­ ments about what a learner and a teacher should do in the classroom. I t is exceedingly d i f f i c u l t , 1 f not Impossible, to block th e ir unique personality characteristics from playing th e ir role in i t . In addition, a v a ila b ility o f mate­ r ia ls and physical f a c ilit ie s influence the performance of the teacher as well as the le a rn e r.43 Although these Independent variables cannot be controlled fo r in the research evaluating teacher education programs, they must be con­ sidered in the fin a l analysis. Adams conducted a p ilo t study to fie ld te s t a theoretical model fo r the evaluation o f a teacher education program, using 36 Sandefur's model (described e a r lie r 1n th is chapter). Forty In d i­ viduals were observed during th e ir student teaching* toward the end o f th e ir f i r s t year of teaching, a fte r three years, and fin a lly a fte r fiv e years o f teaching. The results Indicated that studies using Sandefur's model can be successful. 1. Adams concluded: Elementary teachers became less authoritarian a fte r one year of teaching. 2. There was no s ig n ific a n t difference between the coop­ erating teachers' ratings o f student teaching behavior and peer and supervisor ratings a fte r one year of teaching. 3. The secondary supervisors' ratings were lower than the cooperating teachers' ratings fo r the teaching dimension "re la ­ tions with students." 4. Pupil ratings during student teaching and a fte r the f i r s t year-o f teaching did not d if fe r s ig n ific a n tly from one another. 5. Classroom Interactions fo r elementary and secondary school teachers did not change s ig n ific a n tly a fte r one year o f teach44 1ng experience. Coyne reported the results o f a study comparing the conven­ tional teacher education program and the new Missouri Western Con­ tinuum Program. The new program comprised 54 weeks o f classroom experience startin g with the sophomore year, and replaced tr a d i­ tional education courses with seminars with faculty and school per­ sonnel . The Information collected included scores on proficiency examinations, student questionnaires, and school administrators' 37 evaluations of students' performance as observed on video tapes. The T -te s t for paired groups was used to analyze the data. The findings of the study were: 1. The supervisors' evaluations of the students in the new program were sig n ifica n tly superior to those o f the comparison group. 2. The students in the new program had s ig n ific a n tly superior attitudes toward the concurrent education courses than did the comparison group. 3. Perceptions (visual and w ritten ) of the students 1n the new program concerning the analysis of the teaching situations included in a questionnaire were s ig n ific a n tly superior to those of 45 the students 1n the conventional programs. In a closely related fie ld , Patterson so lic ited employer data to analyze a nursing preparation program at Amarillo College in Texas. He employed a two-phase follow-up method; the f i r s t phase was a personal interview, and the second phase a questionnaire mailed to supervisors. Patterson praised the employer response: "Not only were there no objections, but without exception the employers thought 1t was an excellent opportunity to provide relevant Information fo r 46 performance evaluation." Further, Patterson recommended the questionnaire over the Interview method: This study has u tiliz e d the personal Interview and the mall out questionnaire as a data gathering device, and i t is our recommendation to Implement the mall out questionnaire designed fo r computer analysis. The personal interview has 38 proven very costly, time consuming, and has not given the s ig n ifica n t personal dimension I n i t i a l ly anticipated. Most employers [supervisors] find 1t Inconvenient to arrange a 15-20 minute personal Interview to provide the necessary data; on the other hand, most w ill take time to complete a short questionnaire and return 1t by m a ll.4? Rande Smedley concurred: The mall survey has the advantage of being able to collect large amounts of data from large segments of the popula­ tion Inexpensively and fo r this reason enjoys wide popular­ ity 1n areas where large sums of money are not available for research.48 Patterson concluded, The employer follow-up report appears to be a very valuable tool fo r developing an accountability model for Individual major departments and school divisions. In addition some other Important areas fo r which employer follow-up data may provide sig n ifica n t documentation are (A) State and Federal reports, (B) Educational Association accreditation evalua­ to rs, (C) Grant Proposals {Federal, State, and P rivate), (D) Public Relations Improvement (newspapers), (E) Increase enrollments (Impressive employment d a ta ).49 Copley conducted evaluative research comparing three types of teachers: those who were trained as teachers, those who were trained as teachers without student teaching, and those with no fo r­ mal tra in in g . The sample comprised (1) 22 lib e ra l arts graduates with no formal education classes, (2) 38 lib e ra l arts graduates with some education courses and no student teaching, and (3) 40 education gradu­ ates. Copley ju s tifie d the study by saying: This study was not conducted to prove a point. I t was con­ ducted to gather evidence to evaluate professional education courses. The study was not conducted to silence academician c ritic is m . The results hopefully w ill be enlightening to both academicians and the educationist.50 Principals rated the subjects on the following six dimensions: (1) exhibits understanding of people, (2) uses effective communication 39 s k ills * (3) possesses classroom management s k ills * (4) secures e ffe c tiv e teaching results* (5) is considerate of pupils* and (6) is f a i r 1n relations with pupils. S ignificant differences among the three groups on these Items separated group three from groups one and two* demonstrating that the principals rated the education graduates s ig n ific a n tly higher than the teachers 1n groups one and two. There was a s ig n ific a n t difference among principals' ratings of beginning teachers 1n terms o f th e ir professional education preparation, but there was no sig­ n ific a n t difference in other types o f academic preparation 1n terms o f personality characteristics or physical and emotional health. Goldenberg prefaced his study o f the relationships between p rincip als' and teachers' perceptions o f the q u a lity o f college preparation fo r teaching competence by s ta tin g , "Since evaluation Is a necessary Ingredient of any program preparing teachers, 1t 1s Increasingly appropriate that studies o f this nature be designed and executed 1n order to determine program effectiveness and future 51 goals." His study was designed to determine: (1) how elementary school principals perceive the undergraduate preparation o f th e ir teachers* (2) how graduates perceive th e ir undergraduate education* and (3) whether principals' and graduates' perceptions are congruent. The sample Included 136 fir s t-y e a r teachers and 134 p rin cip als. The researcher evaluated two programs—a regular teacher education program and a competency-based teacher education program. Goldenberg's conclusions Indicated "principals and th e ir teachers view the preparation of teaching 1n a d iffe re n t manner." 52 40 Principals took a more global view of teacher preparation and were prone to view the product of an undergraduate teacher education program, rather than the program i t s e lf . Principals did not per­ ceive the various components of preparing teachers, but rather viewed teacher education in its t o ta lit y . On the other hand, teachers viewed preparation as being composed of several major elements, because they were d ire c tly involved in the process. One major difference between the two groups was th at teachers viewed the programs as being somewhat more e ffe c tiv e in preparing them to teach than did th e ir p rincipals. This study revealed that graduates' responses were much more helpful than p rin cip als' re p lies . Goldberg id e n tifie d a flaw in his ques­ tionnaire as not rating teaching performance, but rather asking teachers and principals fo r th e ir views concerning undergraduate preparation fo r teaching. Johnson conducted a study of graduates of the University of South Alabama to assess th e ir reactions to th e ir jobs and how they f e l t about th e ir preparation, and to estimate the university's ade­ quacy 1n developing th e ir s k ills and insights. Johnson sent ques­ tionnaires to a ll of the Spring 1969 graduates, and received a 31 percent return. 1. Some o f the results were: Seventy-four percent o f the alumni were very sa tisfied with th e ir education, 28 percent were sa tis fie d with th e ir education, and 5 percent were somewhat d is s a tis fie d with th e ir education. 2. A m ajority claimed they had learned teaching s k ills 1n the preservice preparation. 41 3. Graduates f e l t they needed the following s k ills but had not learned them during preservice education: lectu rin g , advising students, Interpreting and using results from standardized te s ts , and working e ffe c tiv e ly with groups of students in extra-class re la ­ tionships. 4. Teachers id e n tifie d classroom d iscip lin e as th e ir big53 gest problem. In the same study, Johnson e lic ite d p rincip als' evaluations to ascertain the strengths and weaknesses o f the graduates surveyed. He determined the r e lia b ilit y of the rating scale "by comparing the relationship between p rincip als' ratings of the teachers and the ratings of those teachers by th e ir supervisors In practice teaching." 54 Johnson found a s ig n ifica n t difference between these two ratin g s, which he attrib u ted to the fact th at the two tracking experiences were Inconsistent. He also speculated that the two parties defined teaching d iffe re n tly , and that principals observed less than super­ vising teachers. Jarvis's study was Intended to measure the effectiveness of a vocational education program. The study Included two groups— one trained 1n a four-week summer program and the other untrained. Questionnaires were administered to the two control groups; a ques­ tionnaire was also given to the administrators 1n d ire c t supervision of the subjects. The purposes o f the study were to determine 1f the course content actually met the needs o f the fir s t-y e a r vocationaltechnical education teachers and to make recommendations fo r program improvement. 42 Jarvis concluded, " I t 1s the opinion of the Investigator that fo r the most part the administrators f e l t th at this was a very CC beneficial program fo r new s t a ff members." The administrators helped strengthen the program with th e ir overwhelming support. further suggested th at the train in g program be mandatory. They Jarvis's study revealed weaknesses 1n firs t-y e a r teachers and made recommen­ dations fo r program Improvement. The studies reviewed 1n this section suggested recent trends In the evaluation of teacher education programs. follow-up studies were conducted a fte r 1970. A m ajority of the Realizing the Impor­ tance of and need to emphasize the evaluation o f teacher education programs, the 1978 AACTE meeting's main focus was on the methodology fo r the development o f program evaluation. The feedback received from such studies can be used to Improve preservice programs. As the results become more r e lia b le , through thorough investigation and re p lic a tio n , colleges o f education w ill have more and more d irect Indications o f where to Improve or change th e ir programs. Summary In summary, a number o f researchers have attested to the Importance o f follow-up studies in evaluating teacher education pro­ grams. This lite r a tu r e review has defined the role major national organizations have taken, In firm support of program evaluation. Two fle x ib le follow-up models, developed by Sandefur and Peshkopla, designed to apply to any college o f education's evaluation pro ject, were outlined. The researcher also focused on the v a lid ity of 43 graduates* s e lf-ra tin g s of teaching performance and supervisors' ratings o f graduates' teaching performance. The research concerned with developing follow-up studies to evaluate teacher education pro­ grams or actual studies Implementing the suggested models advocated using supervisors' opinions. In contrast, other researchers have questioned the v a lid ity of graduates' and supervisors' ratings of teaching performance. The m ajority o f studies warned against placing too much credence In e ith e r supervisors' or graduates' judgments. The consensus o f th is research 1s that neither group 1s capable o f judging performance accurately. The results of these two types of studies are Inconsistent, and therefore serve as a caution 1n Interpreting the current study's findings, f in a lly , this chapter surveyed recent research th at s o lic ite d graduate and supervisor responses to evaluate teacher education programs. The methodology of that research le n t credence to the current study. In his Handbook for Evaluations o f Academic Programs. Roth summarized the essence o f the current study: In order to e ffe c tiv e ly evaluate a program, 1t 1s neces­ sary to c o lle c t feedback Information from the graduates of that p a rtifu la r program. Gathering this Information through follow-up studies thus forms an essential part o f program evalua­ tio n . The Importance o f the u tiliz a tio n of follow-up studies 1s based on the fa c t that these data are derived from Individuals who have already completed the program. These individuals thus have a d iffe re n t perspective than those 1n the program, since they have completed the e n tire sequence and may have teaching experi­ ence. Follow-up studies, however, need not only be 1n terms of the viewpoints o f graduates. Evaluation by supervisors . . . can also be part of this follow-up. In general, resources d ictate that the f i r s t type [view­ points o f graduates and supervisors] is more feasib le and thus more commonly u tiliz e d in evaluations of teacher educa­ tion programs.56 44 A thorough description of research procedures used 1n col­ lecting and analyzing the data Is presented In Chapter I I I . Follow­ ing th is , the findings o f the study are reported In Chapters IV and V. 45 Footnotes—Chapter I I ^AACTE, Reconmended Standards fo r Teacher Accreditation of Basic and Advanced Preparation Programs fo r Professional School Personnel (March 1970). ERIC ED 037 423. 2 John Herbert, "A Research Base fo r the Accreditation of Teacher Preparation Programs" (paper presented at the AERA, Minneapo­ l i s , 1970), p. 82. 3 NCATE, Standards fo r Accreditation o f Teacher Education (Washington, D.CT1 toCATE, 1977), p. 10. 4 Ib1d., p. 11. 5 Dan Kaufman and others, "Assessment o f Teacher Preparation Programs" (unpublished paper, June 1974), p. 280. ERIC ED 095 176. 6 Ib id ., p. 188. 7 P. Woodring, New Directions In Teacher Education (New York: Fund fo r the Advancement” or tducation, I9b 7j, p. 62. O Swami Piyush, "A Follow-Up Study fo r Evaluation of Pre­ service Secondary Science Teacher Education Program a t Ohio State University" (unpublished paper, February 1975), p. 20. ERIC ED 113 200. g Woodring, p. 62. ^Kaufman and others, p. 2. ^ J . T. Sandefur, Illu s tra te d Model fo r the Evaluation of Teacher Education Graduates (Washington, b.C .: AACffe, 1970), p. 2. ERIC Eb 080 485.---------------12 13 Herbert, p. 8. Kaufman and others, p. 16. 14Ib id ., p. 38. 15 Thomas Baer, "Teacher Preparation—What Graduates T e ll Us" (unpublished paper, July 1975), p. 3. ERIC ED 103 402. ^ A lfre d Grommon, e d . . The Education of Teachers of English fo r American Schools and Colleqes, Vol. V, NCTE Curriculum Series (April T$69), p. 43. EfclC ED 024 687. 46 ^Wennette Peaues, "A National Survey of Teacher Education Follow-Up Practices" (AACTE convention paper, 1978), p. 1. 18 Sandefur, p. 11. 19Ib1d., p. 17. 20Ib1d. 21Ib1d., p. 56. 22Ib1d., pp. 5-6. 23 Theodore Peshkopla, "How to Keep Better Track of Our Teaching Education Graduates?" Journal o f Colleqe Placement (Sprinq 1976): 61- 63. ------------------------------- ------------------------ 24lb id ., pp. 62-63. 25Ib1d., p. 63. 26 G ranville Johnson, "Evaluation o f the University o f South Alabama College o f Education Teacher Training Program by Analysis of Its Alumni, 1971 (unpublished paper, University of South Alabama, Mobile, June 1972), p. 42. ERIC ED 060 050. 27 Ned Swartz, "Divergent Perceptions of Teaching E ffective­ ness by D ifferen t Groups of Raters" (unpublished paper, NCME, A pril 1975), p. 3. ERIC ED 104 959. 28 David McAfee, "Evaluation of the Teacher: Do Teachers and Supervisors Agree?" High School Journal 58 (May 1975): 336. 29Ib id ., p. 339. 30Ib1d. 31 Raymond Crisp, "The Professional Competency o f Illin o is Secondary School English Teachers: A Report o f the Self-Evaluation of Experienced Illin o is Secondary School English Teachers" (Washing­ ton, D.C.: O ffice o f Education, December 1968), p. 44. ERIC ED 029889. 32 Martha Cook and Herbert Richards, "Dimensions of Principal and Supervisor Ratings o f Teacher Behavior," Journal o f Experimental Education 41 (Winter 1972): 11. 33Ib1d., p. 13. 34 Lou Carey, "An Investigation of the V a lid ity of Using Self-Evaluation Instruments to Id en tify Instructional Needs" (unpub­ lished paper, December 1977), p. 6. ERIC ED 142 579. 47 35Ib1d. 36Herbert, p. 14. 37 Richard J. Hardebeck and others, "Individualization of Instruction by Vocational and Nonvocatlonal Teachers and Self-Reports Compared With Observations"(unpublished paper, Texas Education Agency, 1974), p. 21. ERIC ED 131 202. 38Ib 1 d ., p. 28. 39Ib 1 d ., pp. 24-25. 40 Swartz, p. 6. 41 Ib id ., pp. 4-6. 42 Kaufman and others, p. 68. 43P1yush, p. 42. 44 Ronald Adams, Follow-Up and Evaluations o f Teacher Education Graduates. Final Report (Bowling Green, Kentucky: Western Kentucky U n iversity, 1974), P* 68. 45 Charles Coyne, Comparative Analysis of Teacher Education (S t. Joseph, Missouri: Missouri Western State College, 1970), p. 75. 46 Larry Patterson, Graduate Employer Follow-Up Studies (Am arillo, Texas: Amarillo College, 1 9 /7 ), p. 20. 47Ib1d. 48 Rande H. Smedley and George H. Olson, "Graduate Follow-Up Studies: How Useful Are They?" (Washington, D.C.: ARA, 1975). ERIC ED 109 431. 49 Patterson, p. 20. 50 Patrick Copley, "A Study of the Effect of Professional Education Courses on Beginning Teachers" (unpublished paper, Southwest Missouri S tate, 1975), p. 1. ERIC ED 098 147. 51 Ronald Goldenberg, "The Relationships Between Principals' and Teachers' Perceptions o f the Quality o f College Preparation fo r Teaching Competency " (unpublished paper, ATE, March 1975), p. 3. ERIC ED 131 075. 52Ib1d., p. 14. 48 ^Johnson. 54Ib1d., p. 33. ^Bob Jarvis and William Stevenson, "The Effects of the 1971 UO-Tech New Teacher Training Program on the Teaching A c tiv itie s of Nineteen Beginning Vocational Teachers in Oklahoma" (unpublished paper, July 1972), p. 25. ERIC EO 069 907. 56 Robert Roth, Handbook fo r Evaluation of Academic Programs (Washington, D.C.: University Press of America, 1978J, pp. 235-36. CHAPTER I I I DESIGN OF THE STUDY This study was designed to test the degree to which data gained from supervisors could contribute to the assessment of fiv e specified Michigan State University teacher education programs. In the f a l l of 1977, the Division of Student Teaching and Professional Development In itia te d a Graduate Follow-Up Study to evaluate teacher education programs. During the planning of that study, the question was raised whether a second follow-up study s o lic itin g responses from the graduates' Immediate supervisors would provide enough valuable Information to ju s tif y the additional cost. The current study 1s th at extension of the Graduate Follow-Up Study. This chapter discusses the design o f that Investigation. The population from which the sample was drawn 1s described and the nature of the sample is specified. data are d etailed . Procedures fo r gathering the Both research and s ta tis tic a l hypotheses are stated, and the models chosen to te s t them are described. Description of Sample Two hundred s1xty-n1ne supervisors were Involved 1n th is study. Respondents In the Michigan State University Graduate Follow-Up Study generated the supervisor sample. 49 Each graduate 1n that study 50 was asked to name h is/h er Immediate supervisor, but was given the option to f i l l out and return the questionnaire omitting the super­ visor's name. The graduates 1n the follow-up study were randomly selected by groups, divided by the p a rtic u la r program and year 1n which they participated 1n student teaching. Five teacher education programs were compared over the following academic years: 1969-70, 1971-72, 1974-75, and 1975-76. The origin al graduate follow-up sample comprised 1,080 subjects, 60 In each o f the 18 c e lls . The t a lly of the fin a l sample was 994 subjects, not 1,080 as o rig in a lly designed. I f questionnaires were returned as nonforwardable from the second m ailing, those sub­ jects were eliminated from the to tal sample. An e ffo r t was made, how­ ever, to maintain the origin al sample size by randomly selecting new subjects to replace those who could not be reached 1n the second mail­ ing. Of these, 86 were nonforwardable, making the fin a l sample size 994 subjects. (The 18 c e lls resulted from a 5x4 matrix from the cross between teacher preparation programs and years of gradua­ tio n , minus two ce lls from the CBTE program fo r the years 1969-70 and 1971-72.) F ifty -fo u r percent o f the graduate sample (536 respondents) responded to the questionnaire. Forty percent of the 536 graduate respondents never had taught or were no longer teaching and were not able to submit a supervisor name. Only 10 percent of the graduates who were s t i l l teaching chose not to submit th e ir supervisor's name, but did return the questionnaire. F ifty percent, exactly one h a lf, 51 o f the graduate respondents submitted supervisor names and addresses. Of the 269 subjects in the supervisor sample, 236 returned question­ naires fo r a return rate of 88 percent. Measuring Instruments Two questionnaires were developed, one fo r the graduate follow-up study and the other fo r the supervisor follow-up study. Both instruments were constructed during spring term, 1977, and have undergone minor revisions. As stated above, the f i r s t questionnaire was administered to a random sample of graduates and provided data regarding: (1) employment h is to rie s ; (2) perceptions o f the 1nflui ence of selected individuals and ch aracteristic features of the student teaching programs on subsequent classroom performance; and (3) perceptions of the graduates' a b ilit y to apply selected teaching s k ills , th e ir rating of the importance o f these selected teaching s k ills , and th e ir rating of the Influence the student teaching program had on the development of each s k i ll . The supervisor questionnaire was administered to supervisors o f those graduates who supplied th e ir names in the f i r s t question­ n aire. The instrument was designed to provide evidence of (1) the graduates' a b ilit y to apply selected teaching s k ills , (2) the gradu­ ates' level of commitment to the teaching profession, and (3) super­ visors' knowledge and ratings o f the teacher education programs. A r e l ia b i l it y analysis tes t was applied to the question­ naires, computing an in te r-ite m matrix o f performance s k ills th at 52 correlated items into subscales that em pirically and lo g ic a lly f i t together. A th ird instrument, the "Success Rating Chart" (developed by West^), was used to compare the supervisors' ratings of graduates' professional commitment to teaching and performance of specified teaching competencies, to evaluations o f graduates by th e ir cooperat­ ing teachers during the student teaching experience. The graduates' student teaching evaluation was assigned a success rating number. Seven was the lowest t a lly 1n a c e ll 1n which both the graduates and th e ir supervisors responded to the questionnaires. The c e lls with more than seven respondents were reduced by a random selection pro­ cedure, using a table of random numbers. The student teaching recom­ mendations were obtained by the researcher with the cooperation of the Student Placement Services. A panel o f three evaluators, who had extensive experience In teaching and 1n supervising student teachers, separately assigned each graduate a success rating number. Each panel member was given the self-explanatory "Success Rating Chart" (see Appendix J) and a 11st of guidelines. lines fo r the raters were to: Some of the guide­ (1) assign the graduate a rating based on the cooperating teacher's comments, (2) take the cooperating teacher's comments a t face value, and (3) give the graduate the lower score 1f undecided between two scores. The three panel members' ratings were averaged to the closest Integer, resulting in one fixed number per graduate. These numbers were then compared to the super­ visors' rating scores o f graduate competence and commitment to teaching. 53 The r e l ia b i l it y o f the "Success Rating Chart" appears to be very high. (See Appendix K.) In the sample of 126 subjects, the three raters unanimously agreed on the graduates1 success rating numbers 54 percent o f the time. In a ll the other cases, there was agreement by two ra te rs , with the th ird ra te r scoring e ith er one point above or one point below the consensus ratin g . In no case did a ll three raters d if fe r on a p a rtic u la r ratin g . V a lid ity The two main Instruments of this study were designed by a panel o f professionals 1n the teacher education f ie ld . This team ca re fu lly screened each Item to Insure that the Instruments did pro­ vide a reasonable measurement o f what they purported to measure. This procedure should Insure that the Instruments have a reasonable level o f content v a lid ity . The panel Included Dr. Donald Freeman, Dr. Henry Kennedy, and Dr. Banks Bradley, faculty members at Michigan State U niversity, and Grace Iverson, research consultant from the Lansing School D is tr ic t. R e lia b ility Internal consistency was determined through the s p H t-h a lfs correlation c o e ffic ie n t fo r two separate subscales o f the survey, namely commitment to teaching and performance o f specified teaching s k ills . 54 Data Collection The procedures fo r collectin g and handling the data are Illu s tra te d by the following calendar outline: August 1977--The graduate sample was selected. September-December 1977— Four months were spent retrievin g graduates' addresses from student teaching f i le s , the MSU Alumni Association, and outdated MSU phone books. December 1977—The f i r s t graduate surveys were mailed on Decem­ ber 10. The answer sheets were coded, so that when they were returned the researcher could readily Id e n tify the c e ll to which data belonged. January 1978—Graduates' names were checked o ff as the answer sheets were returned. - Those graduates submitting supervisors' names were Id e n ti­ fie d , coded, and catalogued 1n a separate f i l e box. - Approximately 100 questionnaires were returned by the post o ffic e as bearing In correct, nonforwardable addresses. - New subjects were randomly selected to replace those 100 subjects whose questionnaires had been returned by the post o ffic e . - The new subjects were sent questionnaires on January 15. - The researcher mailed an e n tire second package on January 16, with a new cover le t t e r {s p e c ific a lly Id entifyin g th e ir programs and In d ivid u ally signed by the d irector of the specific program) to the graduates who had not responded to the f i r s t m ailing. (See Appendix F .) 55 February 1978—The supervisor surveys were mailed on February 1; the answer sheets were catalogued and coded to id e n tify in d i­ viduals who did not respond. March 1978—On March 7, a second questionnaire was sent to each of the second group of graduates who had not returned questionnaires. - The supervisors who had not yet responded were sent a reminder le t t e r on March 18. - The reminder le tte rs to the supervisors were followed two weeks la te r by an e n tire second questionnaire packet. April 1978—The collectio n o f data continued u n til the c u t-o ff date o f A pril 30. May-June 1978—The data were progranmed on the computer using the S ta tis tic a l Package fo r the Social Sciences. Treatment and Analysis of Data The following null hypotheses were formulated in an attempt to answer the main question of th is study: Should supervisors' ratings be Included In graduate follow-up studies evaluating teacher education programs? Question 1 To what extent w ill ratings provided by supervisors d if f e r from those provided by graduates? This question was tested by the following null hypotheses: Ho, : The correlation between graduates' and supervisors' ratings o f the importance o f specified teaching s k ills does not d if fe r s ig n ific a n tly from zero. 56 Hew: The correlation between graduates' and supervisors' ratings o f graduates' performance o f specified teaching s k ills does not d iffe r s ig n ific a n tly from zero. Analysis; Data regarding the f i r s t research question were analyzed by computing a correlation c o e ffic ie n t between supervisor and graduate subscale scores. Eleven Items on the supervisor ques­ tionnaire dealt with Importance o f specified teaching s k ills and graduates' performance o f these teaching s k ills . Subscale scores were determined to correlate the same Items appearing on both the graduate and supervisor surveys. Question 2 W ill judgments provided exclusively by supervisors assist In determining differences among teacher education programs? This question was tested by the following null hypotheses: Hop.: There 1s no s ig n ific a n t difference among programs as judged by supervisors' ratings of graduates' performance of specified teaching s k ills . HOpt.: There is no s ig n ific a n t difference among programs as judged by supervisors' ratings o f graduates' commitment to teaching. Analysis: Data regarding the second research question were subjected to an analysis o f variance to compute the significance of differences among ratings of programs, years, and the Interaction of the two. The to ta l scores o f supervisors' subscale ratings were computed fo r each pertinent Item on the questionnaire. That 1s, supervisors' responses to commitment questions became one subscale score fo r graduates' commitment to teaching, and so on fo r performance. 57 Question 3 Does a supervisor's knowledge of specific teacher education programs d iffe r from one program to another? This question was tested by the following null hypothesis: Ho,: There 1s no s ig n ifica n t difference among programs as judged by supervisors' a b ility to Id e n tify the specific teacher education programs from which the teachers graduated. Analysis: The data were analyzed by using a chi-square tes t to determine s ig n ifica n t differences 1n knowledge of programs, as judged by the supervisors. Question 4 W ill there be a s ig n ifica n t relationship between the super­ visors' ratings o f the graduates and the ratings o f graduates by th e ir cooperating teachers during the student teaching experience? This question was tested by the following null hypotheses: Ho. : The correlation between supervisors' ratings of the graduates' professional commitment to teaching and the general ratings o f the graduates by th e ir cooperating teachers during the student teaching experience does not d if f e r s ig n ific a n tly from zero. Ho-b: The correlation between supervisors' ratings o f the graduates' performance of specified teaching s k ills and the general ratings of the graduates by th e ir cooperating teachers during the student teaching experi­ ence does not d iffe r s ig n ific a n tly from zero. Analysis: Data regarding the fourth research question were analyzed by computing a correlation c o e ffic ie n t fo r a performance subscale and a commitment to teaching subscale o f supervisors' ratin g s, and comparing the rating on each subscale to the success rating of the graduates' student teaching performance. 58 Question 5 Are the variations among programs suggested by data provided by graduates the same as variations among programs suggested by data provided by supervisors? In addition to testing the preceding hypotheses, an e ffo r t was made to Id e n tify differences among programs that were suggested by data provided by graduates and supervisors. Data were analyzed by comparing the pattern o f resu lts; the analysis was lim ited to the results o f the performance subscales. The analysis was conducted by subtracting the graduate performance subscale scores from the super­ visor subscale scores fo r performance s k ills . These scores were subjected to an analysis of variance te s t to compute the s ig n if i­ cance of differences between ratings of programs, years, and the Interaction between the two. Assumptions This study was developed with certain basic assumptions. They are lis te d below: 1. The supervisors had observed the graduates and knew th e ir performances well enough to rate them. 2. Graduates and supervisors were careful and honest 1n f i l l i n g out the forms. 3. The questionnaires accurately tested the hypotheses. L1mitations The following factors were considered to be lim itatio n s of the study: 59 1. The results o f the study were based on supervisors' ratings and graduates' s e lf-ra tin g s rather than on observations. Although the cited research Indicated these ratings are not the best method o f evaluating teacher education programs, such ratings were the only viable source o f information fo r this p ro ject, considering cost, human resources, time, and geographic constraints. 2. The v a lid ity of supervisors' ratings and graduates' se lf-ra tin g s has not been established, and is a lim ita tio n of this study because of the possible question o f the accuracy of such ratings. 3. The study was lim ited by at least two unavoidable con­ taminating variables, which could not be controlled: (a) Current addresses fo r some of the original random sample of graduates were unavailable; (b) Some graduate respondents did not provide super­ visors' names. Those graduates who did not submit supervisors' names may have had relationship d iffic u ltie s with th e ir supervisors, prob­ lems with self-confidence, or f e l t th e ir supervisors had nothing substantial to o ffe r to the study. There was no way to determine whether this lim ita tio n affected to ta l scores. 4. The findings of the study can be generalized only to those fiv e programs and years specified in the study. Any e ffo rts to generalize beyond the study are questionable. 5. Inferences cannot be drawn regarding comparative q u ality o f the programs studied, because of the differences 1n students entering the various programs. Program differences may also have 60 resulted from students being attracted to specific programs. There was no way to control fo r differences 1n academic q u a lific a ­ tio n s, personal ch aracteristics, motivations, or reasons fo r entering various programs. Sumnary Five hundred th ir ty -s ix graduates and 236 supervisors par­ ticip ated 1n th is study. The Instruments used to c o lle c t data were the "Survey of Michigan State University College o f Education Graduates," "Follow-Up Study of Michigan State University Graduates--Supervisor Survey, “ and the "Success Rating Chart," which was used to determine graduates' scores fo r the student teaching experience. The graduate questionnaire was sent to a random sample of MSU graduates, followed by the supervisors' questionnaire sent to the supervisors whose names the graduates had provided. Analysis of variance and correlation coefficients were used to analyze responses to the research questions. Chapter IV contains the results of the s ta tis tic a l analysis of the data. Footnotes—Chapter 111 Bradley West, "A Study of Computer-Scored Group Holtzman Inkblot Variables as Related to Student Teaching Success, Major Teaching Fields, and Sex" (Ph.D. d issertatio n , Michigan State U niversity, 1969), p. 36. CHAPTER IV PRESENTATION OF THE FINDINGS The results of the s ta tis tic a l analysis o f the data are pre­ sented 1n this chapter. The procedures followed were In accordance with the research design outlined 1n Chapter I I I . Each research question Is stated as a null hypothesis, followed by a narrative account of the research findings. Tables summarizing the findings fo r each research question accompany the discussion. Findings Question 1 To what extent w ill ratings provided by supervisors d iffe r from those provided by graduates? This question was translated Into two null hypotheses; the f i r s t was: Ho, : The correlation between graduates' and supervisors' ratings o f the Importance o f specified teaching s k ills does not d if fe r s ig n ific a n tly from zero. Table 1 shows th at there was a correlation of >.883 between graduates' ratings and supervisors' ratings o f the Importance o f specified teaching s k ills . zero (a = .0 1 ). This correlation was s ig n ific a n tly d iffe re n t from A series o f tests was applied to te s t the s ig n if i­ cance o f differences between Item means. were found fo r the following items: S ignificant differences knowledge o f educational theory 62 63 Table l.--Mean ratings: graduate and supervisor ratings of the 1mpor tance o f specified teaching s k i ll s . 3 Teaching S k ills Knowledge of educational theory and practice Graduates jf N Supervisors X N T -te s t Alpha 1.20 401 .82 224 7.54 .01 Knowledge o f subject matter .33 339 .46 225 2.82 .01 A b ility to establish rapport with students .20 400 .22 225 .58 N.S. Ab111 ty to communl cate wl th parents and other teachers .53 401 .52 226 .22 N.S. A b ility to formulate instruc­ tional goals and objectives .79 339 .66 223 2.48 A b ility to provide a wide variety o f Instructional strategies and materials .57 339 .53 226 * 4* N.S. A b ility to c o lle c t and in te r­ pret data regarding student needs and achievement .71 400 .72 224 ♦ • * N.S. A b ility to maintain active student p articip atio n in classroom tasks .46 403 .50 226 .42 N.S. A b ility to recognize and deal e ffe c tiv e ly with problems In student d iscip lin e .31 402 .46 224 1.65 N.S. Abl 11 ty to use e f fe c ti ve questlonlng and Interaction tech­ niques in the classroom .63 401 .71 225 .79 N.S. Abl 11 ty to eva 1uate one' s own classroom and general pro­ fessional performance .49 395 .61 219 1.31 N.S. Note: .02 Relationship between graduates' and supervisors' ratings of Importance of specified teaching s k ills : Ray » .883b a0 * c ru c ia l, 1 a Important, 2 = lim ited relevance, 3 = nonessential . L Correlation score o f to ta l mean ratings between graduates and supervisors. 64 and practice, knowledge o f subject m atter, and a b ility to formulate instructional goals and objectives (see Table 1 ). Although only 3 o f the 11 Items showed significance, based on t-te s ts o f mean d if ­ ferences, the correlation o f to tal mean ratings was very high. Based on the high relationship between supervisors' and graduates' ratings of the importance o f specified teaching s k ills , the null hypothesis was rejected. The second null hypothesis stated: Ho,.: The correlation between graduates’ and supervisors' ratings of graduates' performance of specified teaching s k ills does not d iff e r s ig n ific a n tly from zero. Two correlations were of special in terest in testing the hypothesis. F ir s t, are supervisors' ratings of graduates' performance consistent In and of themselves? As can be seen 1n Table 2, the high correlation (>.747) between the supervisors' ratings of graduate performance and graduate commitment to teaching suggested th at supervisors' ratings were consistent per se. The second and main question pertaining to this null hypothesis asked whether supervisors' ratings were highly related to graduates' s e lf-ra tin g s of performance. The relationship between graduates' s e lf-ra tin g s o f th e ir performance of specified teaching s k ills and supervisors' ratings o f that same performance resulted 1n a correlation of > -.0 0 4 . (See Table 2 .) Since th is correlation was extremely low, the null hypothesis was accepted. The fin a l analysis th at had a d ire c t bearing on Question 1 was a factor analysis of the correlation matrix o f the fiv e subscales used in th is study. Three subscales were based on graduates' ratings: Table 2 .—Relationships between subscales of graduate and supervisor ratings and graduates' student teaching success ratings by th e ir cooperating teachers. 0) (1) Graduates' ratings of satisfaction with student teaching (2) Graduates' self-ratings of s k ill performance (3) Graduates' ratings of s k ill competence: contributions from student teaching (4) Supervisors' ratings of graduates' s k ill performance (5) Supervisors' ratings o f graduates' commitment to teaching (6) Graduates' cooperating teachers' student teaching general ratings 1.000 (2) (3) (4) (5) (6) .237 .608 .018 .033 .186 1.000 .304 -.004 -.043 .033 1.000 .064 .003 .167 1.000 .747 .128 1.000 .151 1.000 66 (1) graduates' ratings o f satisfaction with student teaching, (2) graduates* se lf-ra tin g s o f s k ill performance, and (3) graduates' ratings o f s k ill competence gained from student teaching. scales were based on the supervisors' ratings: Two sub­ (1) supervisors' ratings o f graduates' s k ill performance and (2) supervisors' ratings o f graduates' commitment to teaching. I f there were clear d if f e r ­ ences 1n graduates' and supervisors' ratings, one would anticipate th at a factor analysis with a two-factor solution would d iffe re n ­ tia te between these two sources. As shown In Table 3, the graduate subscales were most highly loaded on factor 2, and the two supervisor subscales were most highly loaded on factor 1. This analysis pro­ vided substantive evidence that the graduates' and supervisors' ratings were d is tin c t. I f the fiv e subscales were to be condensed Into two subscales, one o f these would Involve only graduates' ratings and the other would Involve only supervisors' ratings. Question 2 W ill judgments provided exclusively by supervisors assist 1n determining differences among teacher education programs? This question was tested by two null hypotheses, the f i r s t of which stated, Ho2 : There is no s ig n ific a n t difference among programs as judged by supervisors' ratings of graduates' performance of specified teaching s k ills . Table 4 presents supervisors' mean ratings of s k ill performance among graduates o f the five teacher preparation programs Included 1n the study. An examination o f these mean ratings suggested that they were 67 quite sim ilar; thus 1t 1s not surprising that the corresponding F -ra tlo (>.344) was not s ta tis tic a lly s ig n ifica n t (a = .8 5 ). There­ fo re , the null hypothesis was accepted. Table 3 .—Factor analysis with varlmax rotation of subscale ratings. c , Subscales Factor Analysis -----------------------£-----------Factor 1 Factor 2 Graduates1 ratings o f satisfaction with student teaching .031 .693 -.02 8 .347 Graduates' ratings o f s k ill competence: contributions from student teaching .036 .875 Supervisors' ratings of graduates' s k ill performance .856 .022 Supervisors' ratings of graduates' commitment to teaching .874 -.0 1 8 Graduates' s e lf-ra tin g of s k ill performance- Table 4 .—Differences among programs based upon supervisors' ratings o f graduates' performance s k ills . Supervisors' Scores Program J N 1.16 45 .82 55 Cluster 1.08 54 Overseas 1.06 C.B.T.E. 1.00 54 16 Regular E .I.P . Grand mean - 1.02 F (4 , 195) = .334 alpha = .85 (N .S .) 68 The second null hypothesis stated: HOpb: There Is no s ig n ifica n t difference among programs as judged by supervisors' ratings o f graduates' conmltment to teaching. Table 5 presents the supervisors' mean ratings of commitment to teaching among graduates o f the fiv e teacher education programs. As can be seen from the tab le, no s ta tis tic a lly s ig n ifica n t differences were found among programs, based on supervisors' ratings o f graduates' commitment to teaching (a = .3 7 ). The null hypothesis was accepted. Table 5 .— Differences among programs based upon supervisors' ratings of graduates' commitment to teaching. Supervisors' Scores Program y N Regular .66 45 E .I.P . .60 45 Cluster .71 48 Overseas .49 C.B.T.E. .71 54 16 Grand mean s .63 F (4, 195) = 1.065 alpha = .37 (N .S .) As a complement to this analysis o f the subscales, the supervisors were asked to give a single overall ratin g —on competence and commitment to teaching—of the graduates whom they supervised. Table 6 shows that the chi-square te s t was applied to see 1f fr e ­ quencies could be d istributed by chance. cant difference No s ta tis t ic a lly s ig n if i­ among programs was found fo r competence (o = .828) 69 or fo r commitment to teaching (a * .921). The low significance levels shown 1n Table 6 agree with the data shown 1n Tables 4 and 5, in that both support the decision to accept the null hypothesis. Table 6 .--Supervisors' overall ratings of graduate competence and commitment to teaching.3 Program Supervisors' Ratings of Competence Commitment X N X N Regular 1.49 48 1.00 46 .94 49 49 Cluster 1.06 52 .78 .91 Overseas .81 57 .78 58 C.B.T.E. 1.28 18 .97 18 E .I.P . Grand Mean ** Ch1-square (12) « alpha = 1.12 7.423 .828 (N .S .) 53 .89 5.907 .921 (N .S .) a0 = outstanding (top 10 percent o f a ll teachers), 1 = strong (top 25 percent o f a ll teachers), 2 = above average, 3 = below average. Question 3 Does a supervisors' knowledge o f specific teacher education programs d iff e r from one program to another? This question was translated Into the following null hypothesis: Ho3: There 1s no s ig n ifica n t difference among programs as judged by supervisors' a b ilit y to Id e n tify the specific teacher education programs from which the teachers graduated. Table 7 presents the data concerning supervisors' knowledge of Michigan State University teacher education programs. The table 70 Table 7 .—Supervisors1 knowledge o f Michigan State University teacher education programs. Question Answer Supervisor Response -------- ------------ --------N % Did th is teacher begin his/her professional career under your supervision? Yes No 96 123 43.8 56.2 Regular Yes No 41 3 93.2 5.8 E .I.P . Yes No 39 7 84.8 15.2 Cluster Yes No 48 7 87.3 12.7 Overseas Yes No 48 7 87.3 12.7 C.B.T.E. Yes No 18 0 100.0 0.0 Total Yes No 194 24 89.0 11.0 P rio r to th is survey, were you aware that th is teacher had graduated from Michigan State University? Program Ch1-square (4) = 4.18 alpha - .38 (f 71 shows that less than h a lf of the graduates (43.8 percent) began th e ir teaching under that supervisor's direction. More than 89 percent of the supervisor respondents knew th e ir teacher had graduated from Michigan State University, regardless of the program in which the teacher had been enrolled. The corresponding chi-square value (4.18) revealed no sig n ifican t differences among programs (a « .3 8 ). In other words, most supervisors knew th e ir teachers had graduated from Michigan State University, but there were no distinguishable d if f e r ­ ences among programs. Table 8, column A, shows that the supervisors were most aware of teachers' having graduated from the Regular and E .I.P . programs; 61 percent were unable to Id e n tify the program from which the teacher had graduated. To check the accuracy of these t a llie s , a cross- tabulation was computed on this Item. Table 8, column B, reveals that most of the supervisors of teachers from the E .I.P ., Overseas, and Cluster programs correctly id e n tified the program from which th e ir teachers graduated. Over one-half, or 19 of 33, incorrectly Id en tified the Regular Program. (That is , the supervisor marked the Regular Program, which was not the one from which the teacher had graduated.) Table 8, column C, shows that the greatest number of teachers whose supervisors did not know from which program they had graduated had been 1n the Cluster Program (75 percent) or the C.B.T.E. Program (72 percent). The percentage fo r the Cluster Program was based on 41 responses and was therefore comparatively stable; however, the corresponding percentage for the C.B.T.E. Program was based on only 15 responses and therefore may have questionable s ta b ility . Table 8 .--Supervisors' perceived and actual knowledge of Michigan State University teacher education programs. (A) Supervisors Responding to the Identification of the Specific Program in Which the Graduate Participated N % Program (Bl) (B2) Supervisors Who Correctly Identified the Graduate's Program Supervisors Who Incorrectly Identified the Graduate's Program N % N % (C) Supervisors,by Pro­ gram, Who Said They Did Not Know From Which Program thefa Teacher Graduated N % Regular 33 75 14 32 19 43 26 59 E .I.P . 25 54 22 48 3 6 21 46 Cluster 10 18 10 18 41 55 Overseas 11 20 9 16 2 4 31 75 C.B.T.E. 2 11 • • • • 2 11 13 72 132 61 • * • * • * « ♦• «• • * »• L Don't know • ’Bl + B2 = A. ^Column C distributes the 132 "don't know" responses over the five programs. 73 Table 9 shows th at the ch1-square value o f 18.84 was s ig n ifi­ cant at > .01 fo r supervisors' judgments of specific programs con­ trib u tin g to the chances of graduates' being hired in that d is t r ic t . I t appears th a t, in the supervisors' opinion, graduates from the E .I.P . were more apt to be hired in a p a rtic u la r d is tr ic t than were graduates from other programs. The table also shows that a teacher's having graduated from the Regular Program was not a major factor in the supervisor's decision to hire that teacher. Because only two supervisors chose the C.B.T.E. program, the expected frequencies in that category were extremely unstable. As shown also in Table 9, the ch1-square value of 35.62 was sig n ifica n t at .0000 fo r differences among programs, based on super­ visors' judgments about which program b etter prepared the graduate for classroom teaching. Once again, supervisors rated the E .I.P . as preparing teachers fo r the classroom b etter than the other pro­ grams. Supervisors indicated th at the Regular Program did not b etter prepare students to be classroom teachers, whereas they were not sure of the train in g provided in the Cluster and Overseas programs. The C.B.T.E. program was not included in this analysis because of the low number of respondents 1n that category. Because significance levels were achieved fo r the supervisors' id e n tific a tio n o f programs, the null hypothesis was rejected. Question 4 W ill there be a s ig n ific a n t relationship between the super­ visors' ratings of the graduates and the ratings of graduates by th e ir cooperating teachers during the student teaching experience? Table 9 .—Supervisors' ratings of graduates for the hiring and preparation of classroom teachers, by program. Supervisor Response Question Program Yes N Do you feel that graduates from this program have a greater chance of being hired in your d is tric t than graduates of other programs at Michigan State University? Not Sure % Regular E .I.P . Cluster Overseas C.B.T.E. 3 - 1 5 12 + 50 7 33 * » 50 * ■ % totals 27 • • 1 Regular E .I.P . Cluster Overseas C.B.T.E. 1 5 13 + 54 3 15 1 6 % totals 22 * * Chi-square (8) = 35.62 Note: 35 - 29 48 + 63 * • % N + 10 5 4 6 1 + - 41 50 21 19 38 50 31 Sig. - .01 Chi square (8) = 18.84 Do you feel that graduates from this program are better prepared as classroom teachers than gradu­ ates of other programs at Michigan State University? % N 7 7 10 10 No • • 12 8 15 12 • • 60 - 33 + 75 + 75 • • 57 7 3 2 3 2 + 35 13 10 19 + 100 21 Sig. = .0000 A "+" sign reflects a disproportionately high frequency (where expected frequencies are based on column totals for the entire sample) and a sign reflects a disproportionately low fre­ quency. I f the percentage for a specific program is within 10 percent of the row average, there is no further designation. 75 This question was translated In to two null hypotheses. The f i r s t stated: Ho. : The correlation between supervisors' ratings of the graduates' professional commitment to teaching and the general ratings o f the graduates by th e ir cooperating teachers during the student teaching experience does not d if f e r s ig n ific a n tly from zero. Table 10 shows that there was a low correlation between the super­ visors' and cooperating teachers' general ratings of the graduates' commitment to teaching (>.15) and between supervisors' and cooperat­ ing teachers' overall ratings o f graduates' commitment to teaching (> .0 1 ). Hence the null hypothesis was accepted. The second null hypothesis stated: Ho4b: ^he correlation between supervisors' ratings of the graduates' performance o f specified teaching s k ills and the general ratings of the graduates by th e ir cooperating teachers during the student teaching experi­ ence does not d iffe r s ig n ific a n tly from zero. The correlation coefficients fo r Null Hypothesis 4b can also be found In Table 10. There 1s no evidence that the null hypothesis can be rejected* as Table 10 shows th at the correlation between super­ visors' and cooperating teachers' general ratings o f graduates' s k ill performance was >.03 and the correlation between supervisors' and cooperating teachers' overall rankings o f graduates' competence in teaching was >.09. The null hypothesis was therefore accepted. 76 Table 10.—The relationship between the success ratings given by the graduates' cooperating teachers during student teaching and graduates' and supervisors' ratings. Rating Correlation Between Graduates' and Supervisors' Ratings and Success Ratings Given by the Graduates' Cooperat1 ng Teachers Graduates' ratings o f s a tis ­ faction with student teaching .19 Graduates' s e lf-ra tin g of s k ill performance .38 Supervisors' ratings of graduates' s k ill performance .13 Supervisors' ratings of graduates' commitment to teaching .15 Graduates' ratings of s k ill competence contributed by student teaching .17 Supervisors' overall competence ranking .09 Supervisors' overall commitment ranking .01 Question 5 Are the variations among programs suggested by data provided by graduates the same as variations among programs suggested by data provided by supervisors? To determine 1 f the general patterns of graduates' s e lfratings were s ig n ific a n tly d iffe re n t from supervisors' corresponding ratings o f s k ill performance, the graduates' s e lf-ra tin g s of s k ill performance were subtracted from the supervisors' ratings on that measure. The d iffe re n t scores were then analyzed using an analysis 77 of variance te s t, with years and programs serving as the Independent variables. This analysis exhausted another avenue to determine 1f there were any significant differences among programs, as judged by graduates and th e ir supervisors. Table 11 reveals no differences 1n the patterns of graduates' and supervisors' ratings across eith er programs or years. Table 11,—General patterns of differences between supervisors' ratings and graduates' self-ratin g s of performance s k ills . --------------------------------------Significance Alpha Significance of: 4 Main effects Year F (7) F (3) Program F (4) Two-way Interactions fo r year and program .796 .592 - 1.251 * .454 .293 = F (10) * .686 .769 .737 Summary General Hypothesis 1 A. A s ta tis tic a lly sig n ifica n t correlation was found between graduates' and supervisors' ratings o f the importance o f specified teaching competencies. B. The null hypothesis was rejected. No s ta tis tic a lly sig n ifica n t correlation was found between graduates' se lf-ratin g s of teaching performance and supervi­ sors' ratings o f graduates' teaching performance. was accepted. The null hypothesis 78 General Hypothesis 2 A. No s ta tis tic a lly sig n ific a n t differences were found among programs, as measured by supervisors' ratings of graduates' performance of specified teaching s k ills . B. The null hypothesis was accepted. No s ta tis tic a lly sig n ific a n t differences were found among programs, as measured by supervisors' ratings o f graduates' commit­ ment to teaching. The null hypothesis was accepted. General Hypothesis 3 A. S ta tis tic a lly s ig n ific a n t differences were found among programs, as measured by supervisors' a b ility to Id e n tify the spe­ c if ic teacher education programs from which the teachers had graduated. The null hypothesis was rejected. General Hypothesis 4 A. No s ta tis tic a lly s ig n ific a n t correlation was found between supervisors' ratings o f the graduates' professional commitment to teaching and the general ratings o f the graduates by th e ir cooper­ ating teachers during the student teaching experience. The null hypothesis was accepted. B. No s ta tis tic a lly s ig n ific a n t correlation was found between supervisors' ratings o f graduates' performance o f specified teaching s k ills and the general ratings o f the graduates by th e ir cooperating teachers during the student teaching experience. null hypothesis was accepted. The 79 Chapter V contains a general summary of the study, a dis­ cussion of the research findings, conclusions, and recommendations for fu rth er study. CHAPTER V SUMMARY AND CONCLUSIONS Presented In th is chapter are a review o f the problem, a summary of the procedures employed to c o lle c t the data, a summary and discussion o f the research findings, conclusions drawn from the findings, and recommendations fo r fu rther study. Summary The problem o f the study was to determine whether supervisors' ratings should be Included in graduate follow-up studies evaluating teacher education programs. Five research questions were considered: 1. To what extent w ill ratings provided by supervisors d if f e r from those provided by graduates? 2. W ill judgments provided exclusively by supervisors assist in determining differences among teacher education programs? 3. Does a supervisor's knowledge o f specific teacher education programs d if f e r from one program to another? 4. W ill there be a s ig n ifica n t relationship between the supervisors' ratings of the graduates and the ratings of graduates by th e ir cooperating teachers during the student teaching experience? 5. Are the variations among programs suggested by data pro­ vided by graduates the same as variations among programs suggested by data provided by supervisors? 80 81 Design o f the Study The study was designed to tes t the degree to which data gained from supervisors' judgments contribute to the assessment o f fiv e specified Michigan State University teacher education programs. Comparisons were made between graduates' and supervisors' ratings of the graduates' a b ility to apply selected teaching s k ills and th e ir level of commitment to the teaching profession. In addition, In fo r­ mation concerning the supervisors' knowledge and ratings o f teacher education programs was s o lic ite d . The Instruments used to c o lle c t data included the "Survey of Michigan State University College o f Education Graduates," "Follow-Up Study of Michigan State University Graduates—Supervisor Study," and the "Success Rating Chart," used to determine graduates' scores on the student teaching experience. The graduates In the study were selected using a s tr a tifie d random sampling procedure. Sixty Individuals were selected fo r each group, where groups represented the Intersection between the fiv e programs and four graduation in te rv a ls . Respondents 1n the graduate sample generated the supervisors' names. Each graduate was asked to name h is/h er Immediate supervisor. Although th is response was strongly encouraged, 1t was not mandatory. Of the 269 subjects 1n the supervisor sample, 236 returned questionnaires, fo r a return rate o f 88 percent. Supervisors' and graduates' ratings were compared to deter­ mine relationships and s ig n ific a n t differences among programs and years o f graduation. Whenever fe a s ib le , ratings o f individual items 82 were coirtbined to form subscale ratings. In some of the analyses* supervisors' ratings were treated Independently, whereas 1n others they were compared to the graduates' s e lf-ra tin g s . Summary and Discussion of the Research Findings Graduates' and Supervisors' Ratings o f Importance of Teaching S k ills A strong correlation was found between graduates' and super­ visors' ratings o f Importance o f specified teaching s k ills . Both groups agreed that the 11 s k ills were Important to teaching. Every mean score (except graduates' ratings of knowledge o f educational theory and practice) f e l l between "Important" and "cru c ia l." Both groups viewed the a b ilit y to establish rapport with students as most crucial to success 1n teaching, and knowledge of educational theory and practice as least cru c ia l. Graduates' and Supervlsors' Ratings o f Graduates' Performance~oT Teaching S k ills An extremely low correlation was found between graduates' s e lf-ra tin g s on the performance of specified teaching s k ills and the supervisors' ratings of the graduates on th is same measure. Whereas both groups agreed th at these s k ills were important, they were Inconsistent 1n th e ir judgments o f how well the graduates per­ formed the s k ills . Because there was no way to predict perfor­ mance ratings fo r the graduates solely on the basis of th e ir s e lf-ra tin g s or the supervisors' ratings, I t was necessary to co lle ct data from both groups. Without d ire c t observations, 1t 1s impossible to 83 determine which group was more accurate in judging graduates' s k ill performance. Therefore, i t must be concluded th a t, because the two sets o f ratings were inconsistent, program decisions based solely on the ratings o f one o f the groups would be incomplete. McAfee's research, cited in Chapter I I , supported th is position: "With such a wide variation between teachers' and supervisors' responses i t seems possible that e ith e r the teachers or the supervisors or both are incapable o f co rrectly evaluating the teachers' performance, back­ ground, and a b ilitie s ." ^ In fa c t, there is some basis fo r questioning the v a lid ity of e ith e r source. Do supervisors observe a teacher's performance often enough to render valid judgments? Do teachers observe the perfor­ mance o f th e ir colleagues often enough to have a solid basis for comparing th e ir own performance with that of others? On the other hand, there is reason to believe each source 1s uniquely v a lid . Supervisors' ratings would probably be more objective than teachers', being based on a wider norm group, I . e . , a ll the teachers 1n the building. do. The teachers, however, know c le a rly what they can or cannot Their judgments are based on a to tal picture of th e ir personal strengths, weaknesses, and actual classroom performance. Supervisors' Ratings o f Graduate S k ill Performance to Determine Differences Among Programs' Supervisors' ratings of the graduates' performance of speci­ fied teaching s k ills tended to be comparatively high, with mean ratings ranging from "outstanding" to "strong." The mean ratings 84 also tended to be consistent across programs. Those modest d if f e r ­ ences in program means that did occur were not s ta tis tic a lly s ig n if i­ cant. These data suggest that i f there were large'and meaningful differences among programs, as measured by graduates’ performance levels, i t should have been possible to refute the null hypothesis. Therefore, i f there were differences among programs, they were prob­ ably comparatively small and In s ig n ific a n t. Supervisors’ Ratings o f Graduates’ Commitment to Teacninq to Determine Pi ffe rerices Among Pro grains' S ta tis tic a lly s ig n ific a n t differences were not found among programs fo r supervisors' ratings of graduates’ commitment to teach­ ing. The results indicated that the mean differences among programs, based on supervisors' ratings of graduates' commitment to teaching, were so small that the supervisors' ratings o f graduates' commitment to teaching did not distinguish among programs. This conclusion was reinforced by the supervisors' separate ratings o f graduates' overall competence and commitment to teaching. Supervisors' A b ility to Id e n tify Programs Some s ta tis tic a lly s ig n ific a n t differences were found among programs, based on supervisors' judgments. Although 56 percent o f the graduates did not begin th e ir professional careers under th e ir present supervisors, more than 85 percent o f the supervisors knew th e ir teachers had graduated from Michigan State U niversity. Although most o f the supervisors were aware that th e ir teachers were Michigan 85 State graduates, only 39 percent of these supervisors attempted to Id e n tify the specific teacher education program 1n which the teacher had particip ated. The supervisors accurately Id e n tifie d graduates from the E .I.P ., Overseas, and Cluster programs. The two super­ visors who said th e ir teachers were C.B.T.E. graduates were Incorrect, as were more than h a lf o f the supervisors who believed th e ir teachers had graduated from the Regular program. Nineteen supervisors thought th e ir teachers had participated 1n the Regular program, when 1n fact they had not. I t 1s d if f ic u lt to make judgments about the Regular program based on these data. One p o s s ib ility 1s that because the Regular program is the oldest and most common, the supervisors 1n question f e l t safe 1n guessing this answer. The Regular program had enough appeal fo r the supervisors to Id e n tify , yet the frequency o f misjudgments was much higher fo r this than for the other programs. The abnormally and Inconsistently high proportion of correct and incorrect judgments made i t v ir tu a lly Impossible to compare the Regu­ la r program with the other programs on th is dimension. For the Regular, E . I.P ., and Cluster programs, s lig h tly more than h a lf o f the supervisors could not id e n tify the specific program, whereas two-thirds of the supervisors of Overseas and C.B.T.E. pro­ gram graduates did not know the specific program from which the teacher had graduated. Of the remaining programs, supervisors cor­ re c tly Id e n tifie d graduates from the E .I.P . most frequently. The other three programs (Overseas, Cluster, C .B .T .E .) were roughly equal on th is dimension. 86 Supervisors' opinions regarding hiring potential and q u a lity of classroom preparation also pointed to a sig n ifica n t difference among programs. Supervisors who Id e n tifie d th e ir teachers as being from the C.B.T.E. program were not included in the analysis because they had incorrectly id e n tifie d the program. Supervisors f e l t that graduates from the E .I.P ., as compared to the other three programs, were the best prepared and had the greatest potential fo r being hired. Supervisors' opinions o f graduates from the Cluster and Overseas programs were somewhat lower; th is seemed to imply that supervisors were not certain whether graduating from these programs Influenced employment potential or evidenced b etter classroom prepara­ tio n . Supervisors' opinions o f the Regular program were the least favorable and seemed to suggest that graduating from th is program might resu lt 1n a lower potential fo r job opportunities and lower levels of classroom preparation. Relationships Between Supervisors' Ratings and the Cooperating Yeachers' Success Ratings of the Graduates1' Student teaching Experience No s t a tis tic a lly s ig n ifica n t correlations were found between the supervisors' ratings of graduates' commitment to teaching and the cooperating teachers' ratings o f graduates' student teaching experi­ ence. The correlation between supervisors' ratings of graduates' performance o f specified teaching s k ills and the cooperating teachers' ratings o f the graduates' student teaching experience was also very low. These results indicated th at predictions o f how a supervisor 87 w ill evaluate a teacher's performance or commitment to teaching* based on student teaching reports, are suspect. The researcher could not uncover any other research to support or refute these findings. Given that this conclusion is not sup­ ported by additional research evidence, there are two ways 1n which the low correlations may be interpreted: (1) Graduates change between student teaching and classroom experience and (2) Student teaching reports, or supervisors' ratings, or both, are not v a lid measures of teaching performance. I f I t can be assumed that both o f these ratings are f a ir ly accurate measures or true scores, 1t can also be assumed th at graduates' actual performance and commitment levels Immediately a fte r student teaching cannot be used to predict th e ir level of per­ formance and commitment a fte r graduation. In other words, a highly committed young teacher may not be as committed fiv e years a fte r graduation, and vice versa. General Patterns of Graduates' and Supervisors' Ratings o f Performance S k U lF No s ta tis tic a l significance was found fo r graduates' and supervisors' ratings o f performance s k ills among years or programs. These findings concurred with the other findings, which revealed that neither group's ratings determined differences among programs. These data were analyzed to Investigate whether there was an interaction between the differences 1n patterns of ratin g s. The results Indicated that the patterns were very closely aligned, revealing no interactions among programs and/or years. 88 General Findings Two additional findings, which were not suggested by the questions posed in th is study, may be of in terest to the reader. The f i r s t related to supervisors' ratings of graduates' performance s k ills , broken down by years. A s ta tis tic a lly s ig n ifica n t difference was found among these ratings, as can be seen in Table 12 (Appendix M). I t appears that the supervisors favored the more experienced teachers over the less experienced teachers. The second finding concerned graduates' and supervisors' ratings of factors contributing to classroom performance. Six sources of influence were id e n tifie d on the questionnaire, and both groups were asked to rate the extent to which each source contributed to the graduates' present teaching performance. The correlation among the mean ratings fo r these six sources was comparatively high, as can be seen 1n Table 13 (Appendix N). Although supervisors' ratings of a ll six sources tended to be somewhat lower than graduates' ratings, both groups tended to agree that "interactions with co l­ leagues" had moderate influence, and that "undergraduate education courses," "inservice programs," and "graduate education courses" had lim ited Influence. The difference between groups in mean ratings of the influence of student teaching was comparatively large. Graduates seemed to feel th at student teaching had contributed more to th e ir present teaching performance than supervisors f e l t i t had contributed. 89 Including Supervisors1 Ratings in Evaluations of Teacher Education Programs The major question addressed by this study asked: "Should supervisors' ratings be Included in graduate follow-up studies evaluat ing teacher education programs?" This study provided substantive evidence that the answer to this question is yes. There was a n eg li­ gible correlation between graduates' and supervisors' ratings of the graduates' performance o f specified teaching s k ills . This study did not determine which group's ratings were more accurate, but i t did demonstrate th at the ratings were d iffe re n t. A researcher could also use peer and student ratings o f graduates to strengthen the foundation of evaluating teacher education programs. Another Impor­ tan t finding of this study revealed that graduates' student teaching reports did not predict how the graduates employed as teachers would be rated by th e ir supervisors. The questions on the supervisors survey, which asked them to id e n tify and rate programs, Indicated th at the supervisors, as par­ ticip a n ts 1n the teacher education process, should have the rig h t and opportunity to express th e ir views on teacher preparation to the colleges and u n iversities that control th at process. Supervisors' extremely high questionnaire return rate (88 percent) also indicated th e ir in te re s t in teacher preparation. Conclusions On the basis o f the data gleaned from th is study, the f o l­ lowing conclusions can be drawn: 90 la . Supervisors and graduates generally agreed on the Impor­ tance of the 11 specified teaching s k ills . 1b. The results o f graduates' and supervisors' ratings o f graduates' performance of specified teaching s k ills Indicated they did not agree about the graduates' performance. 2a. Supervisors' ratings of the graduates' performance of specified teaching s k ills did not d iffe re n tia te among graduates from d iffe re n t programs. 2b. There were no differences among programs, as measured by supervisors' ratings o f graduates' commitment to teaching. 3. The E .I.P . was ranked highest (a) on supervisors' accu­ rate Id e n tific a tio n of the program from which the teacher had gradu­ ated, (b) as a facto r fo r h iring p o te n tial, and (c) for b etter preparing graduates fo r classroom teaching. The Regular Program ranked lowest fo r (a) hiring potential and (b) fo r preparing graduates 1n classroom teaching, with the Overseas and Cluster programs drawing neutral responses. 4a. The graduates' student teaching reports by th e ir cooperat­ ing teachers had a very low correlation with supervisors' ratings of the graduates' performance o f specified teaching s k ills . 4b. The graduates'student teaching reports by th e ir cooperat­ ing teachers also had a very low correlation with supervisors' ratings of the graduates' commitment to teaching. 5. No s ta t is tic a lly s ig n ific a n t differences were found betwee the general patterns of graduates' s e lf-ra tin g s and supervisors' ra t­ ings o f graduates' s k ill performance among years or programs. 91 6a. The f i r s t general finding was th at there was a re la tio n ­ ship between the number of years a teacher had been teaching and .supervisors' ratings of graduates' performance; this demonstrated that supervisors rated more experienced teachers higher than new teachers on the performance measure. 6b. The second general finding was th at there was a high correlation between supervisors' and graduates' ratings o f contribu­ tions to teaching; the graduates rated the student teaching experi­ ence as somewhat more in flu e n tia l than did the supervisors. 7. Based on the preceding conclusions, supervisors' ratings of graduates should be Included in follow-up studies evaluating teacher education programs. Recommendations On the basis of the foregoing research findings, the follow ­ ing recommendations are made: 1. Further research rep licatin g this study over a longer period o f time would help to corroborate the findings of this study, and o ffe r more conclusive information regarding the C.B.T.E. program. 2. Further research 1s recommended to focus on observations of graduate performance by trained observers, as a more accurate measure o f the graduates' actual teaching performance. This method would require the selection of a r e a lis tic economic subsample. 3. Further research, s o lic itin g data from graduates and th e ir pupils and colleagues, would provide additional information to assist teacher educators in evaluating teacher education programs. 92 4. A more complete follow-up study should be undertaken Incorporating a ll of the preceding recommendations, as well as a sample o f graduates who are prepared before leaving college, to p artic ip ate 1n a longitudinal follow-up study to evaluate the teacher education program. 5. Research should be conducted th at closely examines the relationships between graduates' student teaching evaluations and th e ir present supervisors’ ratings. In addition, relationships could be drawn between graduates' s e lf-ra tin g s o f performance and th e ir student teaching evaluations. 6. Further research 1s recommended to develop a more valid measure fo r determining graduates' teaching performance and commit­ ment to the profession. 7. I t 1s also recommended th at the questionnaires used 1n this study be redesigned to be programmed more easily fo r computer analysis. Imp!ications In addition to the specific recommendations stated above, the study has three Important Implications fo r graduates, th e ir supervisors, and teacher educators In general. F ir s t, i t 1s customary fo r graduates' student teaching evalua­ tions to remain with th e ir credentials throughout th e ir teaching careers. According to the findings of th is study, these student teaching reports did not re la te to graduates' performance a fte r they had had some teaching experience. Perhaps the graduates were 93 Improving, becoming b etter teachers as they gained more experience, although this probably is not always the case. Therefore, the student teaching evaluation probably does not remain a meaningful evaluation tool a fte r a graduate has had teaching experience. Hiring o ffic ia ls should heed the results of th is study and c a re fu lly weigh both cooperating teachers' and supervisors' recoranendations before h iring a new teacher. Because of the lack of research 1n this area, i t is Impossible to determine 1 f one rating is more accurate than the other. The second Important Im plication o f the study 1s an obvious need fo r a ll Individuals involved in education to be trained in evalua­ tion procedures. The research cited has numerous examples o f the fa ilu re of teachers accurately to evaluate th e ir own performance, and the in a b ility o f supervisors to evaluate teachers' performance. I f educators continue to base important decisions such as tenure, promotion, h irin g , and f ir in g , o r, as in th is study, assessments of teacher education programs, on evaluations, more accurate measures must be developed and procedures standardized fo r the evaluation process. The th ird im plication of th is study was derived from the supervisors' overwhelming response to the study. Eighty-eight per­ cent o f the supervisors returned th e ir questionnaires by the deadline, and another 7 percent returned questionnaires la te . a 95 percent return rate. That amounted to The supervisors' comments and general feelings Indicated (a) pleasure 1n having had the opportunity to p artic ip ate in such a study; (b) general lack of knowledge about the 94 specific teacher preparation programs, but a d e fin ite in te re s t 1n learning; and (c) appreciation th at Michigan State University had communicated with them and asked fo r th e ir contributions to the Improvement of teacher education programs. Twenty percent o f the supervisors took the In it ia t iv e to w rite a short note In addition to the questionnaire, expressing th e ir feelings about Michigan S tate, the graduates' teacher education programs, and the opportunity to express themselves. found 1n Appendix L .) (A representative sample of comments can be The general feeling o f the supervisors was expressed simply 1n one comment: "Thanks fo r the opportunity of r a t in g ________ At the very le a s t, th is study opened doors with the super­ visors of Michigan State University graduates throughout the United States and the world, Including Canada, South America, A u s tra lia , and Europe. The education o f people should be a community e f f o r t , each facet (government, u n iv e rs itie s , and school d is tric ts ) contributing Its expertise to the others. This study has opened the doors of communication, and they must remain open. 95 Footnotes—-Chapter V ^Davld McAfee, "Evaluation of the Teacher: Do Teachers and Supervisors Agree?" High School Journal 58 (May 1975): 336. APPENDICES APPENDIX A INITIAL SAMPLE SIZE 97 APPENDIX A INITIAL SAMPLE SIZE Program 1969-70 1971-72 1974-75 1976-76 Regular 60 60 60 60 240 Cluster 60 60 60 60 240 Overseas 60 60 60 60 240 E .I.P . 60 60 60 60 240 0 0 60 60 120 240 240 300 300 1,080 C•B.T C p- Knowledge of educational theory and practice 2 3 u £ 0 a s 1 2 3 A 1 2 3 A Ability to establish rapport with students 1 2 3 A Ability to conaunleate with parents and other teachers 1 2 3 A Ability to formulate instruc­ tional goals and objectives 1 2 3 A Ability to provide a wide variety of Instructional strategies and materials 1 2 3 A 1 2 3 A Ability to euilntaln active student participation In classroom tasks 1 2 3 A Ability to recognise and deal effectively with problems In student discipline 1 2 3 A Knowledge of subject matter 53 Ability to collect and Interpret data regarding student needs and achieve­ ment Ability to use effective questioning and Interaction techniques In the class­ room Ability to evaluate one's own classroom and general professional performance * a , Little or So answer the three questions which follow each knowledge or skill area listed below. (Mark the response which best ex­ presses your view on the answer sheet. Moderate T5-53TnstnIctIonsr“n < 75 79 1 2 3 A SC 82 1 2 3 A Outstanding - top 10X of all teachers Strong “ top 25Z of all teachers 109 83. Please provide the naae end address of the principal or supervisor In the school in which you are currently working. Naaet________________________________________________________ Address 84. GENERAL COMMENTS: __________ APPENDIX D SUPERVISOR QUESTIONNAIRE 110 APPENDIX D SUPERVISOR QUESTIONNAIRE FOLLOW-UP STUDY OF M.S.U. GRADUATES - SUPERVISOR SURVEY Name____________________________________________________ Date Address Aa a part of a follow-up atudy of graduatea of the College of Education at Michigan State, we would appreciate your cooperation in evaluating the performance of . Your responses to thla survey will be confidential. Results will be reported collectively rather than bv schools or Individuals. 1-6 Please Indicate the extent to which you agree with each of the following statements which refer to professional activities of thla teacher. (Please mark the corresponding spaces on the answer sheet which has been provided.) 1. Strongly Disagree THE TEACHER WHOSE NAME APPEARS ABOVE... Seeks active Involvement with students outside the classroom setting 2 Establishes cooperative relations with colleagues and various support personnel In the building 2 Is receptive to "promising" new Ideas or approaches to teaching 2 A. Maintains appropriate professional conduct and appearance 2 5. Actively participates in various in-service activities auch as workshops and teacher comnlttees 2 Assumes a leadership role within the informal social structure of the school 2 Is resourceful in creating and using available instructional materials 2 Completes professional assignments and responsibilities In a competent and dependable manner 2 2. 3. 6. 7. 8. 111 112 Items 9 and 10 ask you to compare this teacher with other teachers in his/her field 9. Where would this teacher rank in overall competence as a teacher? 1. 2. 3. A. 10. Outstanding (top 107 of all teachers) Strong (top 25% of all teachers) Above average Below average Where would this teacher rank In level of commitment to the teaching profession? 1. 2. 3. A. Outstanding (top 10% of all teachers) Strong (top 25% of all teachers) Above average Below average 113 Instructional Please answer the two questions which follow each knowledge or skill area listed below. (Mark the number on your answer sheet which best expresses your view.) A. B. To what extent is this skill essential to success In teaching? c un u i l l Knowledge/Skill . i How would you rate this teacher's ability to apply this know­ ledge or skill In the classroom? m *3 0 6 CL H O « H U 6O ft w M(ft » Of> ■85 e 0 jfttM 0 ^ 3w -a * f° -5 g *> o S |< V) N U V «e va E g 2 1 2 3 A 1 2. 1 2 3 A Knowledge of educational theory and practice 11 Knowledge of subject matter- 13. 1 2 3 A 1A. 1 2 3 A Ability to establish rapport' with students ■15, 1 2 3 A 16. 1 2 3 A Ability to connunlcate with parents and other teachers 17. 1 2 3 A 18. 1 2 3 A Ability to foraulate instruc­ tional goals and objectives -19. 1 2 3 A 20. 1 2 3 A Ability to provide a wide variety of Instructional strategies and naterlals 21 . 1 2 3 A 22, 1 2 3 A Ability to collect and --Interpret data regarding student needs and achlevenent. 23. 1 2 3 A 2A, 1 2 3 A Ability to nalntaln active student participation In classroom tasks 25. 1 2 3 A 26. 1 2 3 A ■27. 1 2 3 A 28. 1 2 3 A Ability to use effective questioning and inter­ action techniques In the classroom. 29. 1 2 3 A 30. 1 2 3 A Ability to evaluate one's own classroom and general professional performance 31. 1 2 3 A 32. 1 2 3 A Ability to recognise and dealeffectlvely with problems In student discipline 114 Items 33-37 1 In your Judgment* how much have each of the following con______ trlbuted to this individual’s performance as a classroom teacher? 33. 34. 35. 36. 37. Student teaching Undergraduate education courses In-service programs in the schools Interactions with colleagues Craduate education courses 38. Did this teacher begin his/her professional career under your supervision? 1. yes 2. no 39. Prior to this survey, were you aware that this teacher graduated from Michigan State University? 1. yes 2. no 40. This teacher graduated from one of the following teacher preparation pro­ grams at Michigan State University. If you are aware of which program, please check the appropriate box. If you have no knowledge of the program she/he graduated from, please check the "don't know" category. 1. Elementary Intern Program (E.I.P.) 2. Competency-Based Teacher Education Program (C.B.T.E.) 3. Overseas Student Teaching Program 4. Cluster Student Teaching Program 5. Regular (Conventional) Program 6. Don't Know (Skip to item 43) DO NOT ANSWER QUESTIONS 41 and 42 If you checked "Don't Know" in Item 35 (skip to Item 43) 41. Do you feel that graduates from this program have a greater chance of being hired in your district than graduates of other programs at H.S.U.? 1. yes 2. not sure 3. no 42, Do you feel that graduates from thiB program are better prepared as class­ room teachers than graduates of other programs at M.S.U.? 1. yes 2. not sure 3. no 43. GENERAL COMMENTS: APPENDIX E FIRST COVER LETTER TO GRADUATES 115 APPENDIX E FIRST COVER LETTER TO GRADUATES M IC H IG A N STATE U N IV E R S IT Y COLLEGE OF EDUCATION • DIVISION OF EAST LAH tlN C • MICHIGAN • MEM STUDENT TEACHING AND FSOFEttlONAL DEVELOPMENT EKICUON HALL Dear Participant, Aa a part of our continuing effort to laprove teacher education program* at Michigan State University, ve are conducting a follow-up study of paat atudent teachara. You are a part of a randomly eelected sample from thla group. Fron your reeponeea to the encloaed questionnaire, we can Identify atrengtha and weaknesses of our teacher education program. In thla way graduates will have a significant Input In our afforts to lnprovs the ongoing progran. Ve therefore urge you to take 15 to 20 nlnutes of your tine to complete the questionnaire. The final question on the survey asks you to nans your principal or supervisor. A shorter questionnaire will be sent to hla/her. Although this survey will ask your supervisor to rate certain aspects of your performance, the purpose is clearly to evaluate the success of our student teaching program as seen by administrators. Ve will therefore never analyze or report data for Individual teacharaI If for any reason you would rather not cooperate In this phase of the study, pleass complete the questionnaire, omitting only item 183. All data from both the teacher and supervisor surveys will be published In group form only. A specific respondent will never be identified by name by the research team. Thus all personal Information will be kept strictly confidential. Ve sincerely appreciate your cooperation in thla Important study. Respectfully. I Director INSTRUCTIONS PLEASE! 1. Carefully record your response to each Item In tha appropriate apace on ths IBM answer sheet. 2. Use a soft (#2) lead pencil. 3. Vrite your name at the top of the IBM answer sheet end questionnaire. 4. Detach page 8 of tha questionnaire (which Includes additional c o M e n t s you wish to maksT and enclose It and tha IBM answer sheet In ths return addressed envelope. Discard or keep the first seven paaea of the questionnaire I 5. Enclose a aelf-addrassed stamped envelope If you wish a copy of the final report. 116 APPENDIX F SECOND COVER LETTERS TO GRADUATES: BY PROGRAM 117 APPENDIX F SECOND COVER LETTERS TO GRADUATES: BY PROGRAM M IC H IG A N STATE U N IV E R S IT Y IO IIH .I o r r o t CATION . OtVlMOS O f EMT LANSING • MICHIGAN • «M1« STLUEN1 TMCHIMG AND PROnSHONAt OEVKOPMENl I HICKSON H A ll January 16, 1978 Dear Participant, In December, you should have received a letter requesting your partici­ pation In a major study of the undergraduate teacher preparation programs at Michigan State University. Although many Individuals have returned the questionnaires, we have not yet received your response. Because the number of returns from former participants in theCBTE student teaching program is comparatively small, we fear that you and other graduates of this program may not be adequately represented In our efforts to Improve undergraduate programs. We therefore urge you to take 15 to 20 minutes of your time to complete this Important survey. * The earlier letter also described our desire for you tosupply the name of your supervisor so that we may evaluate the success of our undergraduate programs as seen by administrators. Data from both the enclosed question­ naire end the supervisor survey will be kept strictly confidential and will be published in group form only. If for any reason you would rather not Include your supervisor's name, please complete the questionnaire, omitting only Item 183. If you have already returned the questionnaire, please disregard this letter. We sincerely appreciate your cooperation In this important study. Respectfully, Robert Hatfield Professor * On the chance that you may have inadvertantly misplaced the original letter, we have enclosed another copy of the questionnaire as well as a stamped envelope. INSTRUCTIONS Please: 1. Carefully record your response to each Item in the appropriate space on the IBM answer sheet. 2. Use a soft "142) lead pencil. 3. Write your name at the top of the IBM answer sheet and questionnaire. A, Insert the questionnaire in the enclosed envelope and return. 5.' Enclose a self-addressed envelope If you wish a copy of the final ________ report.______________ ______________ 118 119 M IC H IG A N STATE U N IV E R S IT Y t o i l tot or iw c a h o n M to tN T teaching IRICKSON HAIL and . oiw u o n or m o ru iia vA t lA ll LANAINb • MICHIGAN • AMit o h iio p m f n t January 16, 1978 Dear Participant, In December, you should have received a letter requesting your partici­ pation in a major etudy of the undergraduate teacher preparation programs at Hichigan State University. Although many Individuals have returned the questionnaires, we have not yet received your response. Because the number of returns from former participants in theoverseas student teaching program is comparatively small, we fear that you and other graduates of this program may not be adequately represented in our efforts to Improve undergraduate programs. We therefore urge you to take IS to 20 minutes of your time to complete this important survey. * The earlier letter also described our desire for you tosupply the name of your supervisor so that we may evaluate the success of our undergraduate programs as seen by administrators. Data from both the enclosed question­ naire and the supervisor survey will be kept strictly confidential and will be published in group form only. If for any reason you would rather not include your supervisor's name, please complete the questionnaire, omitting only item 083. If you have already returned the questionnaire, please disregard this letter. Ve sincerely appreciate your cooperation in this important study. R e a n e e rfu llv . » ^ Banks Bradley Associate Professor BB/cg * On the chance that you may have inadvertantly misplaced the original letter, we have enclosed another copy of the questionnaire as well as a stamped envelope INSTRUCTIONS Pleaset 1. Carefully record your response to each item in the appropriate space on the IBM answer sheet. 2. Use a soft (42) lead pencil. 3. Write your name at the top of the IBM answer sheet and questionnaire. 4. Insert the questionnaire in the enclosed envelope and return. 5. Enclose a self-addressed envelope if you wiBh a copy of the final r e p o r t . 120 M IC H IG A N STATE U N IV E R S IT Y c o iu o t or e d u c a t io n « o m ci or the dean iait uniwo >mooow iiid • e u o u o h k a il January 16, 1978 Dear Participant, In December, you ahould have received a letter requeatlng your partici­ pation In a major study of the undergraduate teacher preparation programs at Michigan State University, Although many Individuals have returned the questionnaires, we h-ive not yet received your response. Because the number of returns from former participants in thecluster student teaching program Is comparatively small, we fear that you and other graduates of this program may not be adequately represented In our efforts to Improve undergraduate programs. Ve therefore urge you to take IS to 20 minutes of your time to complete this important survey. * The earlier letter also described our desire for you tosupply the name of your supervisor so that wo may evaluate the success of our undergraduate programs as seen by administrators. Data from both the enclosed question­ naire and the supervisor survey will be kept strictly confidential and will be published in group form only. If for any reason you would rather not include your supervisor's name, please complete the questionnaire, omitting only item #83. If you have already returned the questionnaire, please disregard this letter. Ve sincerely appreciate your cooperation in this important study. Keith Goldhammer Dean * On the chance that you may have inadvertantly misplaced the original letter, we have enclosed another copy of the questionnaire as well as a stamped envelope INSTRUCTIONS Pleaset 1. Carefully record your response to each item in the appropriate space on the IBM answer sheet. 2. Use a soft (#2) lead pencil. 3. Write your name at the top of the IBM answer sheet and questionnaire. 4. Insert the questionnaire in the enclosed envelope and return. 5. Enclose a self-addressed envelope if you wish a copy of the final r e p o r t . 121 M IC H IG A N STATE U N IV E R S IT Y ust tm u o t o t tD ucA tioN • o m c t o f m u m a ro . m cko a n «m ii m a n ■ n io u o N h a u January 16, 1978 Dear Participant, In Decenber, you should have received a letter requesting your partici­ pation In a major study of the undergraduate teacher preparation programs at Michigan State University. Although many individuals have returned the questionnaires, we have not yet received your response. Because the number of returns from former participants In the regular student teaching program Is comparatively small, we fear that you and other graduates of this program may not be adequately represented In our efforts to Improve undergraduate programs. We therefore urge you to take IS to 20 minutes of your time to complete this important survey* * The earlier letter also described our desire for you to supply the name of your supervisor so that we may evaluate the success of our undergraduate programs as seen by administrators. Data from both the enclosed question­ naire and the supervisor survey will be kept strictly confidential and will be published In group form only. If for any reason you would rather not Include your supervisor's name, please complete the questionnaire, omitting only item #83. If you have already returned tho questionnaire, please disregard this letter. We sincerely appreciate your cooperation in this important study. tr Keith Goldhammer Dean * On the chance that you may have Inadvertantly misplaced the original letter, we have encloaed another copy of the questionnaire as well as a stamped envelope. INSTRUCTIONS Please: 1. Carefully record your response to each item In the appropriate space on the IBM answer sheet. 2. Use a soft (#2) lead pencil. 3. Write your name at the top of the IBM answer sheet and questionnaire. A, Insert the questionnaire In the enclosed envelope and return. 5. Enclose a self-addressed envelope If you wish a copy of the final report. 122 M IC H IG A N STATE U N IV E R S IT Y COLLEGE OF EDUCATION DEPARTMENT OF ELEMENTART AND SPECIAL EDUCATION EAST LANSING • MICHIGAN • M il January 16, 1978 Dear Participant, In December, you should have received a letter requesting your partici­ pation in a major study of the undergraduate teacher preparation programs at Michigan State University. Although many individuals have returned the questionnaires, ve have not yet received your response. Because the number of returns from former participants in the Elementary Intern Program is comparatively small, ve fear that you and other graduates of this program may not be adequately represented in our efforts to improve undergraduate programs. We therefore urge you to taka IS to 20 minutes of your time to complete this Important survey. * The earlier letter also described our desire for you to supply the name of your supervisor so that ve may evaluate the success of our undergraduate programs as seen by administrators. Data from both the eneloaed question­ naire and the supervisor survey vlll be kept strictly confidential and vlll be published in group form only. If for any reason you vould rather not Include your supervisor's name, please complete the questionnaire, omitting only item #83. If you have already returned the questionnaire, please disregard this letter. We sincerely appreciate your cooperation in thla important study. fin tn R f r fn 11 v * James E. Snoddy lalrman Elementary and Special Education * On the chance that you may have Inadvertantly misplaced the original letter, ve have enclosed another copy of the queetlonnaire aa veil as a stamped envelope INSTRUCTIONS Please: 1. Carefully record your response to each Item In the appropriate apace on the IBM answer sheet. 2. Use a soft (#2) lead pencil. 3. Write your name at the top of the IBM ansver sheet and questionnaire. A. Insert the questionnaire In the enclosed envelope and return. 5. Enclose a self-addressed envelope If you vlah a copy of the final r e p o r t . APPENDIX G FIRST COVER LETTER TO SUPERVISORS 123 APPENDIX G FIRST COVER LETTER TO SUPERVISORS M IC H IG A N STATE U N IV E R S IT Y co il ice or in t r .A ito N • o n iu o v or E A lT tA N M M . • MICHIGAN • ( U K f T t l lM A l I f A tlllN G AN H r K O f fllt O S A L O fV U O P M IX T ERICKLON H A IL January 16, 1978 Dear Supervisor, As a part of our continuing effort to improve teacher education programs et Michigan State University, we arc conducting a follow-up study of former students. An Important phase of this study will be to evaluate the success of our undergraduate programs as seen by those who currently supervise our graduates. The teacher who is identified on the enclosed questionnaire voluntarily provided your name and address as their direct supervisor. From your responses we can gain some insight into thQ activities and teaching performance of our former students. In this way, graduates and their supervisors can have a significant input in our efforts to lmprovo ongoing programs. Ve therefore urge you to take approximately 10 minutes of your time to complete the questionnaire. Your response to the questionnaire will be analyzed and reported by undergraduate programs only. Thus all personal information will be kept strictly confidential and will never be analyzed or reported for individual teachers, supervisors, or school systems. Ue sincerely appreciate your cooperation in this important study. Respectfully, W. Henry Kennedy Director of Student Teaching INSTRUCTIONS Tlease 1. 2. 3. A. 3. Carefully record your response to each item in the appropriate space on the IBM ansver sheet. Use a soft (#2) lead pencil. Urite your name at the top of the IBM answer sheet and questionnaire. Insert the questionnaire in the enclosed envelope and return. Enclose a self-addressed stamped envelope if you wish a copy of the final report. 124 APPENDIX H REMINDER COVER LETTER TO SUPERVISORS 125 APPENDIX H REMINDER COVER LETTER TO SUPERVISORS M IC H IG A N STATE U N IV E R S IT Y co m r.f: o f n x a t i * . w v iiio n or V It T W M I V A I H I V j CA1T LAMING • MICHIGAN • O M l ANTI m o m il O N A L D f.\T lO fM K N T tftlC K lO N H A II March 8, 1976 Dear Supervisor, Recently, you should have received a letter requesting your partici­ pation In a major study of the undergraduate teacher preparation programs at Michigan State University. Your name was provided by one of the grad­ uates in our study. Although most have returned the questionnaire, we have not yet received your response. Because the total number of supervisors In our study Is comparatively small, we are anxious to maxtmlee the nunber of returned questionnaires. This will Insure that supervisor evaluations of the success of our under­ graduate programs Is based upon a representative sample. Such a sample will provide critical Information regarding the activities and teaching performance of our former students. If you have already returned the questionnaire, please disregard this letter. Ve sincerely eppreclate your cooperation In this Important study. On the chance that you may have Inadvertantly misplaced the original latter, we will send you another copy of the questionnaire in approximately two weeks. Sincerely » Dr. HencyKennedy Henlcg/Kennedy Director HK:dme 126 APPENDIX I THIRD COVER LETTER TO SUPERVISORS 127 APPENDIX I THIRD COVER LETTER TO SUPERVISORS M IC H IG A N STATE U N IV E R S IT Y to L ttc r or r d u c at io n . m v iiio n or CAST IANMNG ' MICHIGAN • IM 1I JTVW.NT TEACHING AND PR om ilO N A L DIATlOTM rNT fAtCKWJN HA1.1 March 22, 1978 Dear Supervisor, Recently you should have received a reminder that ve aeek your cooperation in completing a questionnaire which is part of a follow-up study of M.S.U. graduates. Because ve have not heard from you, ve assume you have misplaced the questionnaire. As our earlier letters have indicated, participation by you and others will ensure an adequate representation of supervisors in our study. Another copy of the questionnaire is therefore enclosed as veil as a stamped addressed envolope. The directions below should assist you in completing this form. If you have already returned the ansver sheet, please disregard this letter. We sincerely appreciate your cooperation in this Important study. Respectfully, W. Henry Keqi^dy Director Instructions Please 1. Carefully record your response on each item in the appropriate space on the IBM Answer Sheet. 2. Use a soft (#2) lead pencil. 3. Write your name at the top of the IBM Ansver Sheet. 4. Insert the ansver sheet in the enclosed envelope and return. 5. Enclose a self-addressed stamped envelope if you vlBh a of the final report. 128 copy APPENDIX J SUCCESS RATING CHART 129 APPENDIX J SUCCESS RATING CHART Success Rating Number Interpretation Percentage of Students Likely to Receive This Number 1 ONE OF THE VERY BEST STUDENT TEACHERS I HAVE EVER SEEN. Assign rating 1 to the MOST EXCEPTIONAL and OUTSTANDING student teacher of a ll. I f you judge that a student is of ABSOLUTELY OUTSTANDING ACCOMPLISTMENT and w ill make a potentially GREAT and PROFOUND effect on students, assign rating 1. 2% or about 15 out of 850 2 HIGHLY SUCCESSFUL. Assign rating 2 to those of somewhat lesser overall ability than rating 1 but nevertheless represent ACCOMPLISHED and OUTSTANDING student teaching performance. The highly success­ ful student would rate close to rating 1 but is not one of the best student teachers you have ever seen. 131 or about 110 out of 850 3 SUCCESSFUL. Assign rating 3 to COMPETENT student teachers. Most should receive this rating and do not show the outstanding qualities of ratings 1 and 2. About 7Q%or most or about 595 out of 850 4 LESS SUCCESSFUL. Assign rating 4 to those student teachers who have some problems and rate below the middle, that Is , competent and successful, group in your center. 13X or about 110 out of 850 5 PASSED BUT SHOULD NOT BE IN TEACHING. Assign rating 5 to those students who you feel OUGHT to fa ll—they really are not inclined to teaching— but because of various reasons should not receive a falling grade. CRITERIA OF SUCCESS A. Working With People B. Establishing Class­ room Climate C. Planning Instruction D. Managing Instruction E. Cotrmand of Subject and Teaching Materials F. Personal Qualities G. Professional Qualities H. General Effectiveness as a Teacher 2% or about I f out of 850 APPENDIX K SUCCESS RATING SCORES OF GRADUATES' STUDENT TEACHING EXPERIENCE: BY PROGRAM 131 APPENDIX K SUCCESS RATING SCORES OF GRADUATES' STUDENT TEACHING EXPERIENCE: BY PROGRAM Rater I.D . # n o r Final Score Regular 1969/70 2 4 7 11 13 19 26 2 4 3 3 3 2 2 3 4 3 3 3 3 3 3 4 3 3 3 3 2 3 4 3 3 3 3 2 2 2 4 3 2 4 3 3 3 3 3 3 3 3 3 3 3 3 3 4 3 3 3 3 3 3 4 3 3 3 3 3 2 2 3 3 3 3 2 3 1 3 3 3 3 2 2 1 3 3 2 3 4 2 3 3 3 2 4 3 3 2 3 3 2 3 3 3 3 E .I.P . 1969/70 34 35 36 45 50 51 52 Cluster 1969/70 59 61 66 73 75 76 82 3 3 2 3 1 2 1 2 Overseas 1969/70 87 90 95 98 100 101 105 3 4 2 3 3 3 3 132 133 I.D . # — _______________________________________A — B Final Score C ___________________________ Regular 1971/72 116 120 122 125 126 132 133 3 2 3 3 3 2 4 3 3 3 3 3 3 3 3 2 3 3 3 3 3 3 2 3 3 3 3 3 3 3 3 2 3 3 3 3 3 2 2 3 3 2 3 3 2 2 3 3 3 3 4 3 3 3 2 2 3 4 3 3 3 3 2 2 4 3 3 3 2 2 3 4 3 3 3 2 2 199 201 2 3 2 3 2 3 2 3 202 2 2 2 2 207 208 211 219 2 2 3 3 3 3 3 3 3 3 3 3 3 3 3 3 E .I.P . 1971/72 138 139 142 144 148 150 158 2 3 3 1 3 3 3 Cluster 1971/72 162 165 173 175 176 179 188 Overseas 1971/72 Regular 1974/75 228 232 233 236 243 244 247 3 1 2 2 2 3 3 3 2 3 3 3 3 3 3 2 3 3 2 3 3 3 2 3 3 2 3 3 134 1.0. # Rater A B Final Score C E .I.P . 1974/75 252 257 262 272 275 281 2 3 2 2 3 2 2 3 2 2 3 2 3 3 3 2 3 2 2 3 2 2 3 2 3 3 2 2 3 2 1 3 3 3 3 3 3 2 3 3 3 3 3 3 2 3 3 3 3 3 3 2 2 3 2 2 3 2 2 1 3 3 3 2 2 2 2 3 2 3 2 3 2 2 3 2 3 2 2 2 3 5 3 3 2 2 3 3 5 3 3 3 2 3 2 5 3 3 2 3 3 3 5 3 3 2 2 3 3 2 3 4 2 2 2 3 3 3 3 2 2 2 3 3 4 4 3 2 2 3 3 3 4 2 2 2 Cluster 1974/75 288 293 303 306 307 308 310 Overseas 1974/75 320 322 323 328 329 335 340 C.B.T.E. 1974/75 343 346 348 356 358 369 372 Regular 1975/76 385 389 393 397 399 402 403 135 I.D . # ______________________________________ — A — Final Score C ___________________________ B E .I.P . 1975/76 413 414 415 417 418 420 426 2 2 2 2 3 2 3 3 2 3 2 3 2 3 3 2 3 2 3 3 3 3 2 3 2 3 2 3 3 3 3 3 2 3 1 3 3 3 3 2 3 2 3 2 2 3 3 3 2 3 3 3 3 2 3 2 2 2 2 3 3 3 2 2 2 3 3 3 1 2 2 2 3 3 3 3 2 3 2 3 2 3 3 2 3 2 3 2 3 3 2 3 2 3 2 3 Cluster 1975/76 433 435 436 444 452 455 456 Overseas 1975/76 466 469 488 489 490 494 502 1 2 2 2 3 3 3 1 1 C.B.T.E. 1975/76 514 518 519 522 523 526 528 3 2 3 2 3 2 2 APPENDIX L r SUPERVISORS' GENERAL COMMENTS: BY PROGRAM 136 APPENDIX L SUPERVISORS' GENERAL COMMENTS: BY PROGRAM Regular Program: The Regular Teacher Education Program received 11 positive comments concerning the graduates, as judged by th e ir super­ visors. Below 1s a sample o f these comments: " _______ 1s a fin e teacher, most cooperative, and works well with students and s ta ff." " I find M.S.U. does well 1n Teacher Educat1on--period." "This has been the most d if f ic u lt form I have ever completed. Is a very good teacher, I feel MSU may have contributed to this but I think he would be a good teacher coming from any teacher education school." . . In the short time she has been with us, ________ 1s doing an excellent job teaching." Two less positive comments were received from supervisors judging graduates from the Regular Student Teaching Program: "This teacher was prepared In a subject area and does not have a very good knowledge of dealing with students or developing sequential s k ills within her content area." " does not 'measure up’ to other M.S.U. grads we've employed and consequently would not be issued a contract for '78-79. However, due to his la te s ta rt and the 19 days we've had to miss school, we're going to make an attempt to correct d e fi­ ciencies and see I f he could f i t 1n here." Elementary Intern Program: The E .I.P . received eight positive comments concerning the graduates, as judged by th e ir supervisors. Below 1s a sample o f these comments: " was one o f the most outstanding teachers I have ever worked w ith. She Is a c re d it to her profession and to M.S.U." " E .I.P . 1s one o f the b e tte r teacher train ing approaches 1n the country. I feel that the methods classes s t i l l have a long way to go." 137 138 "________ 1s an outstanding young teacher. She 1s well grounded in theory as well as In practice. Herclassroom performance 1s much b etter than any teacher o f her lim ited experience should be expected to perform." Cluster Program; The Cluster teacher education program received five positive comments concerning the graduates, as judged by th e ir super­ visors. Below 1s a sample of these comments: "Thanks fo r the opportunity o f 'ra tin g ' ________ . I honestly con­ sidered her to be one o f my best teachers ever. ( I have sixteen years experience as a p rin c ip a l.) probably had more innate ta le n t than most of us put together 1n our building. Her a rt room was the best example of a work center that I have experienced. Thus, I have rated her quite high. However, besides being very talented she could get kids Involved. Perhaps, she learned the la t t e r at Michigan State." " 1s a to ta lly dedicated, excellent teacher. She has applied fo r an elementary administrator position in our d is tr ic t and 1 am highly recommending her fo r the position." The following are three of four comments th at were less positive than the preceding two. "________would be a superior d iffic u lty 1n high school." college professor. He has some "________ has had a bad year. Techniques used 1n the classroom were poor. He has been receptive to constructive critic is m and 1s Improving." "I am very much in favor of the C.B.T.E. approach or Intern pro­ gram approach fo r the train in g of future teachers." Overseas Program: The Overseas teacher education program received 18 positive coninents concerning the graduates and other teacher edu­ cation programs, by supervisors. Below is a representative sample of these comments: "________ is what he 1s prim arily because of the person________ 1sT I f he re fle c ts M.S.U. then M.S.U. is #1; i f he simply re fle c ts him self, then he 1s #1." "In a ll respects, th is individual has been a positive contribut­ ing member o f the s ta ff . Experience and c re a tiv ity have developed constantly." 139 "Excellent teacher. Professional and dedicated." " I have nothing but the highest, most enthusiastic regard fo r the E .l.P . program. My experience has convinced me th a t, given a choice, I would be highly biased to choose a teacher fo r n\y building that had been trained 1n th is program. Their grasp o f the r e a litie s o f the profession 1s p ra c tic a l, th e ir background of experience 1s broad and th e ir acceptance of guidance and cooperative sharing Is very high. Friday nig ht, or I'd say more." Competency-Based Teacher Education Program: The C.B.T.E. Program received one comment from a responding supervisor: "General education courses generally are o f l i t t l e consequence in helping out new teachers; theory has very lim ited application without a practical base as a control." APPENDIX M GRADUATES' SELF-RATINGS AND SUPERVISORS' RATINGS OF GRADUATES' PERFORMANCE OF SPECIFIED TEACHING SKILLS, BY YEAR OF GRADUATION 140 APPENDIX M GRADUATES' SELF-RATINGS AND SUPERVISORS' RATINGS OF GRADUATES' PERFORMANCE OF SPECIFIED TEACHING SKILLS, BY YEAR OF GRADUATION Table 12.--Graduates' s e lf-ra tin g s and supervisors' ratings of graduates' performance of specified teaching s k ills , by year of graduation. Graduate Self-Ratings N X N 1969-70 1.04 47 .94 47 1971-72 1.06 55 1.05 55 1974-75 1.11 54 • to CD X Supervisor Ratings 54 1975-76 1.13 57 1.28 57 Grand mean » F (3 , 195) = .453 alpha .71 « 1.06 1.09 2.97 .03 141 APPENDIX N MEAN RATINGS: SIX SOURCES OF CONTRIBUTIONS TO THE GRADUATES' PRESENT TEACHING PERFORMANCE 142 APPENDIX N MEAN RATINGS: SIX SOURCES OF CONTRIBUTIONS TO THE GRADUATES' PRESENT TEACHING PERFORMANCE Table 13.—Mean ratings: six sources of contributions to the graduates' present teaching performance. Mean Rating9 Graduates X Supervisors N X N .68 405 1.73 215 Undergraduate education courses 1.61 404 1.87 2.5 Inservice programs 1n the schools 1.58 400 1.71 214 .85 405 1.05 213 Graduate education courses 1.56 379 2.22 212 Other undergraduate education courses 1.72 404 • • • Student teaching Interactions with colleagues • • • Correlation between graduates and supervisors' ratings of contribu­ tions to teaching: RA J f « .560 aO = strong influence, 1 = moderate Influence, 2 = lim ited Influence, 3 « l i t t l e or no influence. 143 BIBLIOGRAPHY 144 BIBLIOGRAPHY AACTE. Recommended Standards fo r Teacher Accreditation of Basic and Advanced Preparation Programs fo r Professional School Personnel. March 1970. EfclC Eb' 037 4237 Adams, Ronald. Follow-Up and Evaluation o f Teacher Education Graduates. Final Report. Bowl 1ng Green, Kentucky: Western Kentucky Uni­ v e rs ity , 1974. Arends, Robert. "A Comparative Study of the Graduates o f the Michigan State University Elementary Intern Program and the Regular Teacher Education Program." Ph.D. d issertatio n, Michigan State U n iversity, 1969. Baer, Thomas. "Teacher Preparation—What Graduates T ell Us." Unpublished paper, July 1975. ERIC ED 103 402. Bradley, Banks. "Overseas Student Teaching: A Follow-Up Study Report as an Assessment o f In te rc u ltu ral Experiences 1n Student Teaching." East Lansing: Michigan State University Department o f Student Teaching, 1975. Carey, Lou. "An Investigation of the V a lid ity of Using Self-Evaluation Instruments to Id e n tify Instructional Needs." Unpublished paper, December 1977. ERIC ED 142 579. Cook, Martha, and Richards, Herbert. "Dimensions of Principal and Supervisor Ratings o f Teacher Behavior." Journal o f Experi­ mental Education 41 (Winter 1972). Copley, Patrick. "A Study of the Effect of Professional Education Courses on Beginning Teachers." Unpublished paper, Southwest Missouri State, 1975. ERIC ED 098 147. Coyne, Charles E. Comparative Analysis of Teacher Education. St. Joseph, Missouri: Missouri Western State College, 1970. Crisp, Raymond. "The Professional Competency of Illin o is Secondary School English Teachers: A Report o f the Self-Evaluation o f Experienced Illin o is Secondary School English Teachers." Washington, D.C.: O ffice of Education, December 1968. ERIC ED 029 889. 145 146 David, Hugo, ed. Toward Excellence In Student Teaching. Iowa: Kendall/Hunt Publishing Co., 1973. Dubuque, Deans and Directors o f Teacher Education in Michigan. Student Teaching Programs: Questions and Answers. "The Impact of Student Teach­ ing Programs Upon the Cooperating Public Schools In Michigan." June 1970. Goldenberg, Ronald. "The Relationships Between P rincipals' and Teachers' Perceptions o f the Quality o f College Preparation fo r Teaching Competency." Unpublished paper, ATE, March 1975. ERIC ED 131 075. Grommon, Alfred H ., ed. The Education of Teachers of English fo r American Schools and Colleges. Vol. V. NCTE Curriculum Series. A pril 1969. EklC ED 024 687. Hardebeck, Richard J . , and others. "In d ividu alization of Instruction by Vocational and Nonvocatlonal Teachers and Self-Reports Compared With Observations." Unpublished paper, Texas Educa­ tion Agency, 1974. ERIC ED 131 202. Herbert, John. "A Research Base fo r the Accreditation of Teacher Preparation Programs." Paper presented a t the AERA, Min­ neapolis, 1970. ERIC ED 040 124. Jackson, Charles. "A Study o f Selected Student Teaching Experiences Reported by Michigan State University Cluster Program and Conventional Program. Student Teachers." Ph.D. d is se rta tio n , Michigan State U niversity, 1971. Jarvis, Bob, and Stevenson, W illiam. "The Effects o f the 1971 UO-Tech New Teacher Training Program on the Teaching A c tiv itie s of Nineteen Beginning Vocational Teachers In Oklahoma." Unpub­ lished paper, July 1972. ERIC ED 069 907. Johnson, G ranville. "Evaluation o f the University o f South Alabama College of Education Teacher Training Program by Analysis o f Its Alumni, 1971." Unpublished paper. University o f South Alabama, Mobile, June 1972. ERIC ED 060 050. Kaufman, Dan, and others. "Assessment of Teacher Preparation Programs." Unpublished paper, June 1974. ERIC ED 095 176. McAfee, David T. "Evaluation o f the Teacher: Do Teachers and Super­ visors Agree?" High School Journal 58 (May 1975): 336-42. Michigan State University and p articip atin g school d is tr ic ts . "P .O .I.N .T .E ." East Lansing: Division o f Student Teaching and Professional Development, 1978. 147 NCATE. Standards fo r Accreditation o f Teacher Education. b.C .: NCATE, 1977. Patterson, Larry. Graduate Employer Follow-Up Studies. Texas: Amarillo College, 1977. Washington, Am arillo, Peques, Wennette. "A National Survey o f Teacher Education Follow-Up Practices. AACTE convention paper, 1978. Peshkopia, Theodore. "How to Keep Better Track o f Our Teaching Education Graduates?" Journal of Colleqe Placement (Spr1nq 1976): 61-63. Piyush, Swami. "A Follow-Up Study fo r Evaluation of Preservice Secondary Science Teacher Education Program at Ohio State U n iversity." Unpublished paper, February 1975. ERIC EO 113 200. Roth, Robert. Handbook for Evaluation o f Academic Programs. ton, D. cT: University Press o f America, 1978. Washing­ Sandefur, J. T. Illu s tra te d Model fo r Evaluation of Teacher Educa­ tion Graduates. Washington, D.C.: AACTE, 1970. Smedley, Rande H ., and Olson, George H. "Graduate Follow-Up Studies: How Useful Are They?" Washington, D.C.: ARA, 1975. ERIC ED 109 431. Swartz, Ned K. "Divergent Perceptions o f Teaching Effectiveness by . D iffe re n t Groups of Raters." Unpublished paper, NCME, April 1975. ERIC ED 104 959. West, Bradley. "A Study o f Computer-Scored Group Holtzman Inkblot Variables as Related to Student Teaching Success, Major Teaching Field s, and Sex." Ph.D. d issertatio n , Michigan State University, 1969. Woodring, P. New Directions in Teacher Education. fo r the Advancement of Education, 1957. New York: Fund