WRITTEN EVALUATION AT msrm MORRILL COLLEGE: iMPLEMENTmG AN ommmwm woman Dissertation for the Degree 0f P‘h. D. MECHiGAN STATE UNWERSW NEIL HENDERSOH CULLEN: 1973 I: I ‘ )1 '.' Ms“... yJ V 5.1.1. 1 ‘ K’A‘. 4-1;,- m"-‘Mx 7.1L..- . 3 J . a' malgan State University m—-— \ ,7 A—\ BINSWG BY H0118 & SDNS' 800K BINDERY mc. 111 LIBRARY chERs w SPRIh‘GPORY, men’s-.1». 1 :Jl ABSTRACT WRITTEN EVALUATION AT JUSTIN MORRILL COLLEGE: IMPLEMENTING AN ORGANIZATIONAL INNOVATION By Neil Henderson Cullen The present study was undertaken to: 1) create a model of organ- izational innovation by synthesizing what is presently known about organizational change and organizational behavior; 2) use the model to develop and analyze the case history of written evaluation,1 an organ- izational innovation adopted by Justin Morrill College (JMC) in 1970; 3) determine the degree of implementation of written evaluation in Justin Morrill as of Winter Term, l973 and compare it with full imple- mentation as defined in Chapter I of the present study; 4) generate specific hypotheses concerning the implementation of organizational innovations. The organizational innovation model developed is a modification of the "Innovation-Decision Process" outlined by Everett Rogers and Floyd Shoemaker2 and includes the following elements: l) an innova- tion, 2) advocate(s) of change, 3) a complex organization in its environment, 4) a communication network, 5) time. Thus, if organiza- tional innovation is successful, it is a process of planned change in an organizational setting during which the system members move from Neil Henderson Cullen initial knowledge of the innovation through the stages of persuasion, adoption, implementation and incorporation. Methodology Four basic research methods were used to develop the case study of written evaluation at Justin Morrill: l) participant-observation; 2) nonreactive, unobtrusive measures; 3) attitude surveys of faculty and students; 4) an analysis of the degree of completion of the written evaluation forms that faculty and students use to assess stu- dent course performance. For the form analysis, a sample of 389 completed written evalu- ation forms was pulled from student folders, approximately 50 from each of the eight terms in which the innovation had been in effect. One generalized null hypothesis was generated, and twelve research hypotheses were formed in order to test the generalized null. The generalized null stated that: G.H.o There is no difference in the degree of comple- tion among the written evaluation forms. The statistical models chosen for analysis were the Chi Square of independence and the Spearman nonparametric correlation analysis. Both permit one to determine if there is a statistical association between two variables. In addition, frequency counts, means and standard deviations were calculated in order to compare use of the written evaluation form among the five independent variables of time, faculty employment status, student class, grade point average, and percent of completed forms on file. Neil Henderson Cullen Findings The form analysis revealed uneven use of the written evaluation form by faculty and students in the sample. Faculty, however, com- pleted the form more thoroughly than students. The twelve research hypotheses were tested, providing the basis for rejecting the generalized null hypothesis. Faculty and students did not vary in their use of the evaluation form over time. Fulltime faculty com- pleted the form more thoroughly than parttime faculty. Faculty in Language, Inquiry and Expression, and Field Study completed the form more thoroughly than those in Natural Science, Social Science and the Humanities. In addition, students in Language and Humanities classes completed the forms to a higher degree than students in other classes, and lowerclassmen completed the evaluation forms more thoroughly than upperclassmen. JMC had not fully implemented the written evaluation system by Ninter Term, l973. In brief, there was not a completed evaluation form for each JMC student in every JMC course, many graduates with more than fifty credits under the JMC written evaluation system did not request profiles, many faculty did not distribute the evaluation forms on the first day of class to review course objectives and the goals of written evaluation, many faculty did not offer guidance to students as they wrote self-evaluations, and some students did not read their completed evaluation forms. Nonetheless, a profile was written for each student who requested one and advisors did use the evaluation forms to aid their student advising. Neil Henderson Cullen The organizational innovation model proved a useful framework for developing the case history of written evaluation in Justin Morrill College as well as providing the basis for a series of generalizations concerning implementing innovations in complex organizations. 1Faculty in JMC write evaluations for each student in their classes rather than awarding them numerical grades. The students also evaluate their class performance on the same form which contains a course description, course objectives and bases for evaluation. The evaluations are placed in individual student advising folders and at the student's request, are later summarized into a brief profile. The students may have the profiles of their work at JMC forwarded to prospective employers or graduate schools along with their Michigan State transcripts. 2Everett Rogers with Floyd Shoemaker, Communication of Innova- tions (New York: The Free Press, l97l), p.102. WRITTEN EVALUATION AT JUSTIN MORRILL COLLEGE: IMPLEMENTING AN ORGANIZATIONAL INNOVATION BY Neil Henderson Cullen A DISSERTATION Submitted to Michigan State University in partial fulfillment of the requirements for the degree of DOCTOR OF PHILOSOPHY Department of Administration and Higher Education 1973 To Justin S. Morrill-- the innovator who made this study possible ii ACKNOWLEDGEMENTS The writer wishes to express his appreciation to those persons who were of assistance during the planning and writing of this dissertation. To Halter Johnson, thesis co-director and Chairman of the writer's Doctoral Committee, for his helpful criticisms and aid throughout the writer's doctoral study. To Everett M. Rogers, thesis co-director, for his inspiration and encouragement at critical periods during the planning and writing of the dissertation. To Paul Dressel and Ruth Useem, members of the author's Doctoral Committee, for their willingness to read and critique the manuscript with care. To Gordon Rohman, Dean of Justin Morrill College, for his encour- agement to complete the dissertation, his unending interest in new ideas, and his ability to spark the imaginations of those around him. without his support JMC might well not have attempted written evalua- t on. To the JMC faculty, the author's colleagues for the past five years, who continually illustrate their willingness to implement new ideas, even under somewhat difficult circumstances. To Dorothy Rhines who typed the first draft of this dissertation. Without her assistance and attention to detail, the thesis would not have been completed. To Betsey for being Betsey and Rebecca Ann for her smiles and laughter. iii CHAPTER I. II. III. IV. TABLE OF CONTENTS THE PROBLEM. . . . . . . . . . . . . . . . . . . . . . . Introduction. ..... . . . . ........... Purpose . . . . . . . . Thesis. . . . . . . . . . . . . . . flmmw. .. ... ... ... ... ... ... ... The Innovation. . . . . . . . Full Implementation or Incorporation. . . . . . . . . Glossary of Key Words and Phrases ....... . . . Overview. . . . . . . . . . . . . . . . . . ..... LITERATURE REVIEW AND THEORY . . . . . . . . . . . . . . Introduction. . . ..... . . . . . . . . . . . . . Complex Organizations ....... . ........ The Change Process. . . . ..... . ..... . . . Social Interaction Perspective. . . . . . . Research, Development and Diffusion Perspective . . . Problem-Solver Perspective. . . . . . . . . . . . . . A Model of Organizational Innovation. . . . . . . . . Sumnary....... ........ . ..... .. METHODOLOGY AND DESIGN ..... . . . . . . . . . . . . Introduction. . . . . . . . ..... . . . . . . . . Measures and Instruments. . . . . . . . . . . . . . . Design. . . . . .............. . . . . . Hypotheses. . . . . . . . . . . . . . . . . . . . . . Analysis. . . . . . . . . . . . . . . . . . . . . . . smry O O O O I O o O o O O o O O O O O O 0 O o o 0 HISTORICAL DEVELOPMENT AND DATA ANALYSIS . . . . . . . . Introduction. . . . . . ........ . . . Justin Morrill College as an Organization . ..... Adopting Written Evaluation . . . . . . . . . . . . . Implementing Nritten Evaluation . . . . . . ..... The Degree of Implementation. . . . . . . . Discussion of the Degree of Implementation. . . . . . Summary. . . . . . . . . . . . . . . ........ iv 109 III III 132 146 168 195 TABLE OF CONTENTS--Continued CHAPTER Page V. SUMMARY AND CONCLUSIONS ........ . . . . . . . . . 205 Summary . . . . . ...... . . . . ..... . . . 205 Conclusions . . ................. . . 212 Research Implications . . . ........ . . . . . 223 BIBLIOGRAPHY .......................... 225 APPENDICES. O O O O I O O O 0 O O O O 0 O O O 0 O O O O 0 9 O 0 23] 1. Categories and Questions to Analyze the JMC Written Evaluation Forms . . . . . . . . . . . . . . . . . . . . 231 2. Faculty and Student Attitude Surveys, Results, and Cover Letters. . . . . . . . . . ..... . . . . . . . 234 3. JMC Written Evaluation Forms: Fall Term, 1970 and Winter Term, 1971-Winter Term 1973 . .......... 249 4. JMC Student Profile--Two Examples ..... . . . . . . . 253 5. Written Evaluation Proposal Distributed to JMC Faculty and Students April 23, 1970. . . . . . . . . . . . . . . 256 6. Written Evaluation Proposal Forwarded to the University Curriculum Committee, May 7, 1970 ..... . . . . . . . 262 7. A Provisional Glossary for the JMC College Goals on the Written Evaluation Forms ....... . . . . . . . . . 267 8. Guidelines for the Instructor for Using the Written Evaluation Form. . . . . . . . ........ . . . . . 271 9. Proposed Policy and Form for Writing Student Profiles. . 274 10. Suggestions for Improving the Written Evaluation System and Examples of Faculty Evaluations. . . . . . . . . . . 278 11. Summary of Fall 1972 Meeting on Written Evaluation and Sample Evaluation Forms. . . . . . . . . . . . . . . . . 283 12. Fall 1972 Proposal for Modifying the JMC Written Evaluation System. . . . . . . . . . . . . . . . . . . . 291 TABLE OF CONTENTS-~Continued APPENDICES 13. Procedural Guidelines for Using the Modified Written Evaluation Forms. . . . . . . . . . . . . . . . . . . . 14. Suggestions for Using the Modified Written Evaluation Fom. O O O O I O 0 O O O O O O O O O O O C 0 O O 0 0 0 15. Guidelines for Writing Course Descriptions, Objectives and Bases for Evaluation-~An Excerpt from a 4/25/73 Memorandum by Neil H. Cullen. . . . . . . . . . . . . . vi Page 295 299 302 TABLE 3.1 4.1 4.2 4.3 4.4 4.5 4.6 4.7 4.8 4.9 LIST OF TABLES The breakdown of the 100 student sample for the 1971 attitude survey. . . . . . . . . . . . . . . . . . . . . Raw completion scores by term for the categories and questions used to analyze the Justin Morrill College written evaluation form. . . . . . . . . ........ Frequency of completion by term for the categories and questions used to analyze the Justin Morrill College written evaluation form. . . . . . . . . . . . . . . . . Raw completion scores by faculty status for the cate- gories and questions used to analyze the Justin Morrill College written evaluation form. ..... . . . . . . . Frequency of completion by faculty status for the cate- gories and questions used to analyze the Justin Morrill College written evaluation form. . . . . . . . . . . . . Raw completion scores by knowledge area for the cate- gories and questions used to analyze the Justin Morrill College written evaluation form. . . . . . . . . . . . . Frequency of completion by knowledge area for the cate- gories and questions used to analyze the Justin Morrill College written evaluation form. . . . . . . . . . . . . Raw completion score by class for the categories and questions used to analyze the Justin Morrill College written evaluation form. . . . . . . . . . . . . . . . . Frequency of completion by class for the categories and questions used to analyze the Justin Morrill College written evaluation form. . . . . . . . . . . . . . . . . Faculty completion score for the Justin Morrill College written evaluation form by academic term and frequency of low, medium and high scores . . . . . . . . . . . . . vii Page 93 170 171 172 172‘ 173 174 175 175 180 LIST OF TABLES-~Continued TABLE 4.10 4.11 4.12 4.14 4.16 4.17 4.18 Student completion score for the Justin Morrill College written evaluation form by academic term and frequency of low, medium and high scores . . . . ..... . . . . Chi square values, degrees of freedom and significance levels for the responses to questions 3, 9, 11 and 13 on the instrument used to analyze the Justin Morrill College written evaluation form by the variable, academic term. . . . . . . . . . . . . . . . . . . . . . Faculty completion score for the Justin Morrill College written evaluation form by faculty employment status and frequency of low, medium and high scores . . . . . . . . Student completion score for the Justin Morrill College written evaluation form by faculty employment status and frequency of low, medium and high scores . . . . . . . . Chi square values, degrees of freedom, and significance levels for responses to questions 3, 5, 7, 8, 9 and 14 on the instrument used to analyze the Justin Morrill College written evaluation form by the variable, faculty status . . . . . . . . .‘. . . . . . . . . . . . . . . . Faculty completion score for the Justin Morrill College written evaluation form by academic term controlling for faculty employment status--fulltime faculty only . . . . Faculty completion score for the Justin Morrill College written evaluation form by academic term controlling for faculty employment status--parttime faculty only . . . . Faculty completion score for the Justin Morrill College written evaluation form by knowledge area and frequency of low, medium and high scores . . . . . . . . . . . . . Student completion score for the Justin Morrill College written evaluation form by knowledge area and frequency of low, medium and high scores . . . . . . . . . . . . . viii Page 180 181 182 182 183 184 184 186 186 LIST OF TABLES--Continued TABLE 4.19 4.20 4.21 4.22 4.23 4.24 4.25 4.26 Chi square values, degrees of freedom, and significance levels for the responses to questions 1-4, 6-11, and 15 on the instrument used to analyze the Justin Morrill College written evaluation form by the variable, knOWI Edge area 0 O O O O O O O O O O O O O O O O O O O O 0 Student completion score for the Justin Morrill College written evaluation form by class and frequency of low, medium and high scores. . . . . . . . . . . . . . . . . Faculty completion score for the Justin Morrill College written evaluation form by class and frequency of low, medium and high scores. . . . . . . . . . . . . . . . . Chi square values, degrees of freedom, and significance levels for the responses to questions 3-5, 7, 9-11, and 13 on the instrument used to analyze the Justin Morrill College written evaluation form by the variable, class. . Student completion score for the Justin Morrill College written evaluation form by academic term controlling for student status--1owerclassmen only. . . . . . . . . . . Student completion score for the Justin Morrill College written evaluation form by academic term controlling for student status--upperclassmen only. . . . . . . . . Spearman Rank Order Coefficients and significance levels for the variable pairs, student grade point average with student completion score for the Justin Morrill College written evaluation form ang_student completion score with faculty completion score . . . . . . . . . . . . . . Number and frequency of Justin Morrill College written evaluation forms on file for the student sample by term . ix Page 187 189 189 190 191 191 193 194 LIST OF FIGURES FIGURE Page 2.1 Stages typically included in models of change within three schools of research. . . . . . . . . . . . . . . . 42 CHAPTER I THE PROBLEM Introduction1 As A190 0. Henderson notes, change is no stranger to the American educational scene.2 In reviewing the history of higher education in the United States, one is struck not by the absence of change but rather by its omnipresence. The Land Grant Act, the elective curricu- lum, the community college, the basic college concept, student rebellion, experimental colleges, free universities-~all these and more dot the most recent history of higher education. Indeed, change and specific innovations have been principal characteristics of many colleges and universities. Yet, in spite of the frequent application of new concepts in colleges and universities, few human processes have been less examined and are less understood than planned organizational change.3 Changes are introduced, judged successes or failures primarily on anecdotal evidence, continue or fail, and little more is known about the process 1Please see the glossary of terms at the end of this Chapter if questions arise as to the definition of specific words or phrases. 2A190 0. Henderson, The Innovative Spirit (San Francisco: Jossey- Bass, Inc., 1970), p. 3. 3See glossary of terms at end of Chapter I. 1 of change. Why was the change recommended? Who made the decision to adopt it? Was the change implemented? Why was it successful? Why did it fail? Did it fail? Such questions are infrequently asked and even less frequently answered. People ignore the process surrounding specific innovations, and repeat the pattern of either moving on to the next change or retreating to the status quo. The question is, why? Matthew Miles offers a partial explanation: The dominant focus in most contemporary change efforts tends to be on the content of the desired change, rather than on the features and consequence of change processes. . . . We need to know, for example, why a particular innovation spreads rapidly or slowly, what the causes of resistance to change are in educational systems, and why particular strategies of change chosen by innovators succeed or fail.1 In brief, innovators often concentrate more on the what to the exclu- sion of the hgw_of change.2 Success or failure is attributed to the idea alone, and little is learned about the change process which might be applicable under similar circumstances in the future. One must remember that while content is important, a myopic concern with it may be self-defeating and even lead to increased rigidification of complex systems.3 1Matthew B. Miles, Innovation in Education (New York: Teachers College Bureau of Publications, Columbia University, 1964), p. 2. 2The following authors address themselves to this problem: Ronald Havelock et_gj,, Planning,for_lnnovation (Ann Arbor: Institute for Social Research, 1969), p. 17. Everett Rogers with Floyd Shoemaker, Communication of Innovations (New York: The Free Press, 1971), pp. 78-7977 Warren Bennis, The Leaning Ivory Tower (London: Jossey-Bass, Inc., 1973), p. 139. 3See Warren Bennis' account of the Meyerson administration at SUNY Buffalo for an excellent example of the backlash that concentrating on content alone can create. Ibid.. pp. 112-145. Change is a developmental process rather than a single event, and when it occurs in an organizational setting such as a college or university, it can be quite complex. Yet few studies have explored the organizational innovation process as a whole, and the resultant lack of detail makes it appear deceptively simple.1 A variety of case studies might be helpful to give persons a sense of the complexity of organization change, but as Ronald Havelock gt_gl, note, there are remarkably few case histories of utilization (i.e., the adoption and implementation of innovations): 0f the thousands of dissemination and utilization events that take place each year, it is unsettling to find so few docu- mented in such a way that others could learn from them. This deficiency in the literature was one of the factors that thwarted our efforts to code, analyze, and compare utilization processes across studies and fields. There are three further shortcomings in change theories which frustrate persons wanting to help organizations respond to the new demands being made upon them. Studies which have examined change as a process have concentrated on events and actions leading up to the adoption of a change and ignored the sequence of events related to implementation which occurs after the decision to adapt.3 Warren Bennis makes this point emphatically: 1Neal Gross et a1., Implementing_0rganizational Innovations (New York: Basic-BbBEs, Inc., 1971), pp. 1, 42-43. 2Ronald Havelock §t_gl,,‘gp, 213,, p. 17. 3Neal Gross £31., 93. _<_:_i_t_., pp. 1, 22. What we know least about--and what continually vexes those of us who are vitally concerned with the effective utilization of knowledge-~is implementation.. As I use the term, "implementa- tion" encompasses a process which includes the creation in a client system of understanding of, and commitment to, a particu- lar change which can solve problems, and devices whereby it can become integral to the c1ient--system's operations.I Secondly, the innovations examined are frequently simple techno- logical changes such as vaccines or fluoridation of water rather than complex philosophical concepts such as "general education" or "student centered learning" so common to higher education.2 Persons can identify and either reject or accept the former whereas abstract ideas are diffi- cult to translate into workable models and even more difficult to implement. Finally, as Bennis argues, theories of organizational change are simply not designed for persons who want to manage change. What I object to--and I include the "newer" theories of neo- conflict, neo-functionalism, and neo-evolutionary theories-- is that they [theories of social change] tend to identify and explain the dynamic interactions of a system without providing a clue pertaining to the identification of strategic levers for alteration. They are theories suitable only for observers of social change, not theories for participants in, or practi- tioners of, social change. They are theories of change and not theories ofc ”hanging Thus, there are at least five weaknesses in contemporary theories of social and organizational change from the perspective of one 1Warren Bennis, Changing Organizations (New York: McGraw-Hill, Inc., 1966), p. 175. 2See Gross et al., 0 .cit., pp.l ,22, and Rogers with Shoemaker, .gp. cit. , p. 79 to corro rate “this point. 3Bennis, Changing Organizations, p. 99. interested in planning organizational innovations: 1) Many researchers concentrate on the content of a specific innovation rather than empha- sizing that change is a process, occurring over time and demanding thorough strategies for adoption and implementation; 2) Few researchers have examined the change process in organizational settings. The result is an over-simplified view of organizational innovation; 3) Most research on diffusion of innovations has analyzed or recounted the efforts leading to adoption rather than those related to implementa- tion; 4) Most innovation research deals with physical or technological innovations rather than with ideas or social practices; 5) Few theories of complex organizational change give explicit guidance to the person interested in managing change in an educational institution.1 Purpose This study is a case history, and it will address the five weak- nesses in theories of social change identified above. That is, the case study will: 1) depict the adoption and implementation of a specif- ic educational innovation and thereby emphasize the developmental process of organizational change; 2) call attention to the various organizational variables which may affect the history of the innovation; 3) illustrate as precisely as possible the degree of implementation of 1For exceptions to this generalization see: R. Chin "The Utility of System Models and Developmental Models for Practitioners," in Bennis et Ll., The Plannigg of Change (New York: Holt, Rinehart and Winston, _1969), pp. 297 3 Ronald Havelock at Ll., Planning¥Educationa1 Innovations (Ann Arbor: Institute for Social Research, 1971). the innovation and try to identify factors which both aided and hindered the implementation process; 4) examine an educational innovation de- signed to improve student learning and not a simple technological improvement and 5) generate specific hypotheses concerning the imple- mentation of organizational innovations which should prove helpful to those persons who want to manage change in educational institutions or other formal organizations. More generally, the purpose of the case study is to develop a com- plete history of the adoption and implementation of a specific innova- tion in a complex institution so that one can view the process in its entirety and generate plausible explanations for the determined degree of implementation. The case study as a whole will identify factors which block and ease the implementation of organizational innovations. 3321's The principal problem of the study is the same one which faced Neal Gross e;_gl, in their case study of the catalytic role model for teachers-~15 the implementation of organizational innovations affected by factors other than those which precede adoption, and if so, what are those factors? It is the thesis of this study that once the decision to adapt has been made, the degree to which a complex organization utilizes an innovation is influenced by: l) the manner in which the organization chose to adopt the innovation; 2) the characteristics of the innovation itself--its complexity, relative advantage, compatibility and observability;1 3) the characteristics of the organization (e.g., goals, structure, roles, the organization's environment) and the members of the organization (e.g., education, background, educational philosophies); 4) the amount of nurturence received by the innovation during its period of implementation. If the thesis is incorrect and implementation depends only on getting the organization to adopt the innovation, then one would expect full implementation to occur during the first term of use and continue throughout the history of the innovation. The data gathered to develop the case study will give clues as to the validity of the thesis. Theory The theories used in the thesis will be the communication of inno- vations theory develOped predominantly by Everett Rogers and Floyd Shoemaker2 and complex organizational theory as discussed by a variety of theoreticians, e.g., Bennis, Slator, Etzioni, Parsons, Lippitt, Likert, Chin, Benne.3 The theoretical framework of Rogers and Shoe- maker is particularly useful since it calls attention to the essential elements of the adoption process. They note that there are four parts to diffusion: 1Rogers with Shoemaker, 92, 215,. pp. 22-23. These are four of the five characteristics of innovations discussed by Rogers and Shoemaker. It is the contention of this thesis that only these four affect imple- mentation. They are defined in the Glossary at the end of Chapter 1. 2Rogers with Shoemaker, 92, 213. 3See Chapter II for Specific citations and further writers. l) the innovation itself with its characteristics: relative advantage to potential adopters, the innovation's compatibil- ity with the prediSpositions of the adopters, the degree to which the innovation may be used on a limited basis, the complexity of the innovation, and finally how easily the re- sults of the new idea can be observed, 2) the communication of the innovation (taking source, messages, effects and receiver into account), 3) the social system in which the innovation is communicated or diffused (including the organizational structure, its environ- ment, the members, norms, roles), and 4) the time the adoption process occurs (ranging from initial knowledge of the innovation through confirming that adoption or rejection was the correct decision). Rogers, himself, says it more briefly, [there] are four key elements: (1) an innovation, (2) communicated in certain channels, (3) to members of a sacial'system,7(4) who adopt it over a period of time. Although Rogers is discussing diffusion, or the spread of innova- tions, in the above quotations, the process culminates in either the adoption or rejection of a new idea or an adaptation of a new idea. Thus, it seems clear that the same four elements influence the adoption and ultimately the implementation of innovations in complex organiza- tions. Nonetheless, one needs to supplement communication of innova- tions' theory when analyzing the process of implementation, the main interest of the present investigation. Although Rogers and Shoemaker identify implementation as the step which logically follows adoption, they offer no generalizations or strategies which might aid the incorpo- ration of an innovation in an on-going organization.2 To complement 1Everett M. Rogers, "The Communication of Innovations in a Complex Institution," in Educational Record, Winter 1968, vol. 49, no. 1, p. 68. 2Rogers and Shoemaker, gp, 213., pp. 310-316. their theoretical framework, this study will rely heavily on the obser- vations of Neal Gross e;_gl,, who discuss the implementation of innovations in detail.1 Regardless of the minor emphasis placed on implementation by Rogers and Shoemaker, however, it is the thesis of this study that all of the elements important to the adoption process--the characteristics of the innovation, communication and the traits of the organization and its members-~plus the adoption process itself influence the degree to which organizational innovations are implemented. Ideally, the present study should illustrate that generalizations once thought only applicable to adoption also help explain the degree of implementation. In Chapter II, the study will attempt to "forge a convergence"2 between the various theories of organizational change and communication of innovations theory. After a discussion of the elements of complex organizations and the various theories of change, communication of innovations' theory will serve as a basis for developing a new model of organizational innovation. The Innovation At the end of Spring Term, 1970, Justin Morrill College (JMC) adopted a system whereby instructors write evaluations for each student 1Neal Gross e__a_l_., 92. gig. 2Rogers with Shoemaker, 92, cit., p. 299. Rogers and Shoemaker use this phrase to describe a simfiTar attempt on their part in Communi- cation of Innovations. 10 in their classes in lieu of awarding numerical grades. Prior to adop- tion, instructors simply awarded each student a numerical grade (4.0- 0.0) for the term's work and gave the grades to the Assistant Dean so that he could forward them to the Registrar's office. If instructors provided more detailed evaluation directly to the students, they did so at their own initiative. JMC adopted the innovation to achieve three major objectives: 1) Aid communication between an instructor and a student as both eyeluate the student's performance in a pgrticular course. The systemincorporates detailed evaluation as part of the learning process and encourages the student as well as the teacher to participate in the evaluation. 2) Highlight_§pecific course objectives and those college ob'ec- tives stressed in a course. The evaluation forms enable J C to emphasize the importance of integrating the College courses. 3) Provide a summary of a student's work in JMC which, although brief, is more specific than grades or a grade point average. Written evaluation in JMC has two basic elements: 1) the written evaluation form used for each student in each course (see Appendix 3) and 2) a summary of a student's course work in JMC called a "Profile of Competencies" (see Appendix 4). The form leaves room for a course description, the course objec- tives, and the bases used to evaluate student performance. In addition, the college objectives are listed, and a grid is provided for students and faculty to assess how well the student did on relevant College objectives.2 Space is left for each student's name, his student number, 1Neil H. Cullen, “Written Evaluation in Justin Morrill College," in Ideas from JMC, May 1973, p. 2. 2The grid design was abandoned at the beginning of Spring Term,l973. 11 the instructor's written evaluation of the student's performance and the student's written assessment of his own performance. The instruc- tor should distribute the forms at the beginning of each course in order to preview the course content and objectives with the class members . At the end of the term, the instructor redistributes the forms so that the students can evaluate their own performance if they wish. After collecting the forms and reviewing each student's course work, the instructor writes an evaluation of each student, comments on his ability to do honors work, and awards a "Pass" or "No-credit." The instructor then gives the evaluation forms and the grade cards to the Assistant Dean. He, in turn, places copies of these forms in student folders for individual students to examine at their convenience. Both faculty and students attempt to write their respective evalu- ations in terms of course, college and, in the case of students, personal objectives. They may also comment on the quality of a student's exami- nations, papers, or oral presentations and mention specific strengths and weaknesses in his performance. The Profile of Competencies is a summary of a student's achievement in Justin Morrill as evidenced by the completed written evaluation forms. The profiler is a person unfamiliar with the student, and he relies predominantly on the instructors' evaluations. Using the College objec- tives as an organizing scheme, he normally comments on the degree to which the student can: 1) communicate effectively; 2) acquire, evaluate, and synthesize information; 3) work independently; 4) work in groups; 5) demonstrate creativity; 6) demonstrate intercultural and self awareness; 12 and 7) solve problems. The profiler does not comment upon skills that neither the faculty nor the students have mentioned in the individual evaluations. A profile is written only at the student's request and may not be seen by persons outside the college without his consent. He may keep the profile for personal reference and have the Assistant Dean forward it to prospective graduate schools and employers. Justin Morrill encourages students to add the profile to their vita, since it comments directly on their competencies. The ultimate reader-~such as a graduate school admissions officer or an employer--is told that the profile is not a recommendation of a single individual, but rather a summary of the skills demonstrated in numerous courses taught by different instructors. Full Implementation or Incorporation In order to determine the degree of implementation of written evaluation at Justin Morrill, one must understand what constitutes full implementation from the outset. Gross e§_gl, refer to full implementa- tion as incorporation of the innovation into the ongoing organization.1 The following elements taken as a whole constitute full implementation of the innovation, written evaluation: 1) A completed evaluation form for every student in every JMC course in both the Dean's folder and the student's advising folder. To be complete each evaluation form must have: 1Gross e§_gl,, _p,‘gj§,, p. 17. 13 1.1 a course description, i.e., something beyond the general and specific course title, 1.2 bases for evaluation, 1.3 course objectives and college objectives stressed, 1.4 student performance on relevant College objectives checked on the grids by both instructor and student,1 1.5 a written evaluation of student performance by the instruc- tor commenting on objectives met; the quality of examinations, papers and class participation; and strengths and weaknesses, 1.6 a written evaluation of student performance by the student commenting on objectives met; the quality of examinations, papers and class participation; and strengths and weaknesses, 1.7 a "Pass" or a "No-credit" circled, and 1.8 the instructor's signature. 2) Requests for profiles from all students who have more than 50 credits under the written evaluation system. 3) A complete profile for each student who has requested one. 4) Distribution of evaluation forms to all students in each JMC course on the first day of the class to discuss the course description, the course and college objectives, the bases for evaluation and the purpose of the written evaluation system. Each instructor having the students add their names and student numbers to the top of the forms, and then collecting the forms for redistribution at the end of the term. ] 1973. The grid design was abandoned at the beginning of Spring Term, 14 5) Redistribution of the forms later in the term, with each instructor explaining that the student self-assessment is voluntary but encouraged. The instructor offering some guid- ance in the process of self-evaluation. 6) Every student reading every written evaluation of his course performances which are on file in the Student Advisinngenter. 7) Use of the completed written evaluation forms by faculty to aid in the advising of students. Glossary of Key Words and Phrases1 Adoption process--see "Innovation-decision process" and Change process." Change--synonymous with "Innovatibn." Change process-~for this study, it is synonymous with "Planned organi- zational change." It involves several stages: knowled e of the innovation, persuasion that the innovation is sound, the decision to adopt, confirmation that the decision to adopt was correct (occurs during attempted implementation), incorporatign or full implementation into the ongoing organization. (R)(G)Z_ N.B. As used in Chapter II, it refers to the dissemination-utilization process (see below). Compatibility--the degree to which persons perceive an innovation being consistent with their values, experiences and needs. (R) 1Most of the definitions came from either Rogers and Shoemaker or Neal Gross et 21, An (R) will designate a definition taken from the former, a (G) a definition from the latter. 2Parallels, but is more detailed than the process outlined by Gross et 21,, p. 17 and Lewin's model. Gross identifies three stages-- initiation, attempted implementation and incorporation. Lewin also identifies three--unfreezing, changing, refreezing. The process is identical to Rogers' and Shoemaker's Innovation—Decision process with the addition of "Incorporation" as a final stage. 15 Complex organizations-~social systems that are deliberately and rationally designed to achieve certain predetermined goals and are characterized by prescribed roles, an authority structure and a formally established system of rules and regulations to govern the behavior of its members. They also have informal networks, norms and social relationships among their members. (R)(G) Complexity--the perceived difficulty in understanding and using an innovation. (R) Degree of implementation--the extent to which, at any given point in time, the organizational behavior of members conforms to the guidelines of an organizational innovation. (G) In this investi- gation, degree of implementation refers to the extent to which the organizational behavior of faculty and students matches that outlined under "Full Implementation or Incorporation" in Chapter I. Diffusion--the process by which an innovation is communicated through a social system over time. (R) Dissemination and utilization--the spread (diffusion) and use (adoption and implementation) of new ideas. In all, it covers the stages of research, development, diffusion and adoption (see Chapter 11). Formal organizations--see "Complex organizations." Incorporation--full implementation of an innovation in an on-going organization. Innovation—~an idea, object or practice perceived as new by an indi- vidual, group of individuals or an organization. The idea need not be "objectively" new to be perceived as such. (R) Innovation-decision process--see "Change process." This study adds incorporation as a fifth stage to Rogers' four stage process. Innovativeness--the degree to which an individual is relatively early in adopting new ideas or practices. (R) Innovator--a person who is venturesome and willing to take some risks to try new ideas. He/she is frequently marginal in influence within his social system. (R) Institutions--see "Complex organizations." Observability-—the degree to which one can see the results of an innovation. (R) 0rganization--see "Complex organizations." 16 Organizational change-—behavioral change with reference to role per- formance, the authority structure, the division of labor or the goals of an organization. Individual change does not necessarily cause behavioral change as an organizational member. (G) Organizational innovation-~an innovation introduced into an organiza- tional setting. Also a synonym for "organizational change" and "planned organizational change." Organizational innovation process-~used interchangeably with "Planned organizational change." Planned organizational change--involves deliberate efforts to introduce change(s) into an organization. Refers to the total process that follows after efforts are made to alter organizational behavior through the introduction of an innovation. (G) If successful it involves both adoption and implementation processes. See "Change process" for stages. Relative advantage--the perceived rewards or drawbacks associated with adopting an innovation. (R) Role--a set of expectations applied to the behavior of the occupant. (G) Trialability--the degree to which an innovation may be tried on an experimental basis. (R) Overview In Chapter II, the three major research traditions regarding change theory are reviewed along with the work of some complex organizational theorists. An attempt is made to merge these theories and create a model of organizational innovation. Chapter III includes the research methodologies, a description of the sample and testable hypotheses re- lated to the evaluation form completion analysis, the rationale for using the case study format and the statistical models to analyze the data from the form completion analysis. Chapter IV contains the history of the adoption and implementation of written evaluation in Justin Morrill 17 and a description of the main organizational characteristics of JMC. In addition, it has a discussion of the form completion analysis, the principal determinant of the degree of implementation of written evaluation in Justin Morrill College. CHAPTER II LITERATURE REVIEW AND THEORY Introduction To understand the process of organizational innovation, one must first be conscious of the various elements which comprise a complex or formal organization. These and the manner in which they interrelate and influence organizational activity will be reviewed in the first part of the chapter. Within this review most of the characteristics unique to colleges and universities will be mentioned. The three principal theoretical approaches to diffusion of innovations and planned change will then be discussed. This review of the literature will form a basis for establishing a theoretical framework or model to discuss the adoption and implementation of innovations in complex organizations. Complex Organizations Formal organizations are complex systems, and systems have several important characteristics that Robert Chin identifies: It is helpful to visualize a system by drawing a large circle. We place elements, parts, variables, inside the circle as the components, and draw lines among the components. The lines may be thought of as rubber bands or springs, which 18 19 stretch or contract as the forces increase or decrease. Outside the circle is the environment, where we place all other factors which impinge upon the system. Chin's description highlights the boundary ("a large circle"), an imaginary line which separates and distinguishes the system from its environment. The boundary is comprised of the characteristics which give a unique identity to the system--perhaps membership qualifications or particular objectives. Chin also emphasizes that there are a variety of system components interconnected with and influencing one another; hence, a change in one may well lead to a change in another. Finally he says there is tension and therefore motion in the system; in fact, all human systems are in constant motion, and any analysis of a system at a single point in time is somewhat distorted. Stated more succinctly, a system is marked by coordinated activity to reach a common goal, "interaction, interdependency, and integration of parts and elements."2 A system is also open, i.e., although it is differentiated from its environment, it cannot function independently. Some organizational theory in the past erred in regarding a system as closed and therefore depicting the "enterprise as sufficiently independent to allow most of its problems to be analyzed with reference to its internal structure 1Robert Chin, “The Utility of System Models and Developmental Models for Practitioners," The Planning of Change, eds. Warren G. Bennis, Kenneth D. Benne, Rebert Chin (New YEFk: Holt, Rinehart and Winston, Inc., 1969), p. 300. 2Ibid., p. 300. 20 and without reference to its external environment."1 A passage by Goodwin Watson discussing the influence of technology on education illustrates the impact that the environment may have on a system. The impact on education has been manifold. Accelerating scientific advance has brought the “explosion of knowledge" which forces revision of curricula. Technological changes have elimi- nated some traditional occupations and created demands for new kinds of training. Rapid communication and transportation have made the ethnocentric curriculum of American schools an anachron- ism. New opportunities are presented to teachers by new media: films, projectors, tape-recorders, kinescopes, T.V. and closed circuit T.V., microfilm, computers, and other instruments. As technology has raised standards of living, it has also made college education economically possiblezfor more pupils and has changed their secondary school demands. Nevertheless, it is important to remember that a system does not simply move at the whim and fancy of its environment; the boundary is more a semi-permeable membrane than a sieve. Open system theory simply stresses there are interrelationships between a system and its environ- ment. Daniel Katz and Robert Kahn state briefly the most general char- acteristics of open systems: All such systems involve the flow of energy from the environ- ment through the system itself and back into the environment. They involve not only a flow of energy but a transformation of it, 1E. C. Trist, "On Socio-Technical Systems," The Planningiof Change, eds. Warren Bennis e5_gl, (New York: Hall, Rinehart and Winston, Inc., 1969), p. 270. 2Goodwin Watson, "Toward a Conceptual Architecture of a Self- Renewing School System," Change in School Systems, ed. Goodwin Watson (Washington, D. C.: NationalTTraining Laboratories for the Cooperative Project for Educational Development, 1967), p. 108. 21 an alteration in energic form the precise nature of which is one definition of the system itself.1 In addition to the cycle of importing energy (input), transform— ing it (through-put) and exporting the product (output), open systems also have several other traits. They acquire what Katz and Kahn call negative entropy, i.e., they take more energy in than they expend in order to counter entropy, the tendency for systems to degenerate. A university, for example, does not expend all its money on the pro- duction of credit hours but rather keeps some in contingency funds to deal with crises and some for faculty leaves to permit regeneration of the staff. In addition, state universities enroll more students than are capable of earning degrees, knowing they will generate additional revenue while on campus which supports those students who complete their education. Open systems process information as well as energy from the en- vironment. To do so they establish a coding mechanism which rejects data irrelevant to system objectives and accepts relevant data. When such information is related to the system's own functioning (the cycle of input, throughput and output), it is known as feedback. A univer- sity, for instance, is responsive to new knowledge of many kinds but quite deaf when asked to become an advocate for one side of a contro- versial political issue. Universities also tend to be interested in 1Daniel Katz and Robert L. Kahn, The Social Psychology of Organi- zations (New York: Joyn Wiley and Sons, l966), p. 453. n.b. the following discussion on the characteristics of Open systems is based on Katz and Kahn, pp. 19-29. 22 the marketability of their graduates; such feedback may have a marked effect on future curricula. Open systems attempt to preserve their character and identity through the feedback mechanism. Through adapting to malfunctioning, they seek to obtain a steady state or homeostasis. The equilibrium achieved is a dynamic one, however, and not static. In fact, the sys- tem may grow in order to cope with changing external forces. This tendency of dynamic equilibrium that systems exhibit prevents them from flying apart--a university which enrolls increasing numbers of students increases in turn the size of the faculty and physical facili- ties, a college which creates a Center for Black Studies recruits more Negroes, and a school which does not attract enough students is either reorganized or cut in size. A change in one variable necessitates a change in another or others in order to maintain balance and direction. In the process of achieving homeostasis or a steady state, a system differentiates and elaborates itself, i.e., systems become increasingly specialized and complex. Over time, universities have become increas- ingly departmentalized to handle the demand for more specific knowledge, and as a result, have developed elaborate hierarchies of control. In an earlier period a basic division between Arts and Sciences, on the one hand, and the professional schools on the other seemed sufficient. Now, at schools like Michigan State, there are separate Colleges of Humanities, Social Science, Natural Science, General Education, Urban and Ethnic Studies and many more. 23 Finally, an open system is noted for equifinality, or the ability to "reach the same final state from differing initial conditions and by a variety of paths."1 Thus, in a university setting a wide variety of students ("Initial conditions") attain the bachelor's degree through a variety of different programs. Increased demand for product control may decrease the amount of equifinality since the demand for control will limit the variety of inputs and the number of paths to a goal. Thus, the greater the decentralization of control in a complex organi- zation, the greater the pr0pensity for equifinality. Examining the tendencies that systems exhibit while continually repeating the cycle of gaining input from the environment and trans- forming it to output, one is struck by the fact that some-~homeostasis, coding and feedback--tend to give the system stability and help it main- tain its intended direction, whereas others--equifinality and negative entropy--seem to assure that the system will have the energy to change. When a system begins to regularize its operations in order to gain greater predictability of human behavior, it becomes a formal organiza- tion. One can have a fairly loose schedule in a one room school house, but in a university with 40,000 students, it is important that the professor teach the predetermined subject matter at the assigned time and place if the system is to work at all efficiently. Katz and Kahn specify the characteristics which are associated with human systems in general and complex organizations: 11bid.. p. 26. 24 The most generalized description of the various substructures [of an organization] is that of the role system. A role system is a set of functionally specific interrelated behaviors generated by interdependent tasks. Their role enactments are appropriate to the system requirements and not necessarily to the personality expression of the individual. The forces which maintain the role system are the task demands, the shared values, and the observance of rules. Organizations develop out of more primitive groupings in which these first two forces may have been dominant, but they grow by formal elaboration of the third factor of rule enforcement. The formulation of rules and the sanctions of rewards and punish- ments result in an authority §ystem for the organization. Though the authority system invokes sanctions, it is also supported by the nature of the task demands and the shared values of the group. Thus organizational norms and values continue to be supportive of the role system. The values of the system are a justification and idealization of its functions. The most generic norm is that of legitimacy, i.e., an acceptance of the rules of the game because people acquiesce in the belief that there must be rules. (Emphasis added) Thus, most generally, a system is a structure of roles held together by task demands, shared values and the observance of rules as enforced by an authority system. Roles are created from the behaviors necessary to perform certain specialized tasks associated with reaching the goals of the organization, i.e., they describe the organizational and not the personal behavior expected of individual members. Gradually, these roles are normalized and specialized so that both the occupants and those with whom they work share the same set of behavioral expectations for any one role.2 In a stable organization, the nature of the task demands, the shared values and the rules of the group support such expectations. 1 2 Ibid.. pp. 455-456. Havelock giggly gp_. _c_i_t_., p. 2-25. 25 In a university setting, for instance, a very complex structure of roles exists consisting predominantly of faculty, students and administrators. The administrators are part of a fairly traditional hierarchy of control consisting of several levels-~president, vice- president, Dean, Associate Dean--each representing an area of control. However, faculty are not merely workers under the control of manage- ment. Rather they are a group of relatively autonomous professionals with loosely defined roles who are accustomed to wielding a great deal of influence in the policy decisions of the administration. The stu- dents have a unique role since they are both consumer and, more recently, low level participant. Thus, in a collegial environment, there are really three role structures needing the maintenance of task demands, shared values and the observance of rules. Whereas the terms, task demands, shared values, rules and regula- tions are self-explanatory, "norm" is a much less understood term. As Havelock e$_gl, define it, "Norm" is a very broad and general concept which can be used to describe any attitude, belief, value or mode of behavior which is held in common by the members of a group.1 Thus, a norm is a standard to which members of the group are expected to conform, and norms can be a very powerful force--either supporting or undercutting the role structure of an organization. It is not unusual for writers to use the term norm, to refer to roles (seen as behavioral norms) or shared values (seen as value norms) or both. 26 The authority system mentioned in the Katz and Kahn quotation en- forces the rules and regulations of a system by sanctions. In most modern organizations, the sanctions are rewards rather than punish- ments, and they may be tangible or intangible (e.g., money, status, prestige). The resulting hierarchy of control describes who makes and implements decisions and may range from an authoritarian to democratic mode. Regardless, "the essence of authority structure is the acceptance of directives as legitimate, i.e., either the acquiesence or approval by people of the rules of the game."1 The authority structure, rules and regulations, system norms and the structure of roles range over a continuum among organizations from very rigid and closely defined expectations of behavior at the one extreme to very loose and broadly defined expectations of behavior at the other. For instance, as organizations grow larger, they frequently become concerned about control and develop elaborate authority struc- tures (either centralized or decentralized), many rigid rules and regulations, and a structure of roles with a high degree of specializa- tion. In small organizations, on the other hand, coordination and control are more easily accomplished, the authority structure may be quite simple, there may be few rules and regulations, and the role structure may permit extensive individual definition and initiative. Organizations vary on the above control continuum according to their mission also. The United States Army, for example, wants to be able to predict the behavior of its members to a high degree and thus leaves 1Katz and Kahn, 92, 915,, p. 44. 27 little flexibility to individual role occupants. In universities, on the other hand, faculty have traditionally had extensive autonomy in determining their instructional behavior. The degree to which the above elements of organizational struc- ture--authority structure, rules and regulations, system norms, task demands, shared values and role structure--are mutually supportive is quite important for organizational stability. For instance, if a new leader tries to impose a rigid, centralized and tightly controlled authority structure onto an organization whose member norms support loosely defined rules with a high degree of individual initiative, con— flict is inevitable. To regain homeostasis or stability, either the leader, the members, the authority structure or the member norms must change. The conflict may, at one extreme, cause the collapse of the system or it may, at the other, force small changes in all four ele- ments as movement toward a new equilibrium point occurs. In addition to being aware of the above structural characteristics of an organization, one must also view a human organization as a socio- ] i.e., it has a group of human beings working together technical system, to accomplish a task or tasks involving some technology. These two aspects of a human system present a fundamental dichotomy between human needs and aims on the one hand and production needs and objectives on the other. At least two authors other than Trist have discussed this dichotomy. Katz and Kahn draw distinction between production inputs, "the materials and energies directly related to the through-put or the 'Trist, gp. 312., p. 272. 28 work that comprises the activity of the organization in turning out a product," and maintenance inputs, "the energies and informational con- tributions necessary to hold the people in the system and persuade them to carry out their activities as members of the system."1 Talcott Parsons refers to both the instrumental (production) needs of an organization and the socio-emotional (maintenance) needs of the people who comprise it.2 As a result of these two dimensions of an organization, theorists itemize certain functional imperatives that all organizations must meet if they are to remain stable and healthy. Katz and Kahn refer to five functions for an organization: (1) production subsystems concerned with the work that gets done; (2) supportive subsystems of procurement, disposal and institutional relations; (3) maintenance subsystems for ty- ing people into their functional roles; (4) adaptive subsystems, concerned with organizational change; (5) managerial subsystems for the direction, adjudication, and control of the many sub- systems and activities of the structure.3 Talcott Parsons uses somewhat different terminology (Goal attain- ment, adaptation, integration and pattern maintenance with the need for three supra-systems, managerial, institutional and technological for the sake of coordination), but he too is concerned with identifying the same set of tasks which any organization in a changing environment must accomplish. 1Katz and Kahn, 22:.219-4 p. 454, 2Chandler Morse, "The Functional Imperatives," The Social Theories of Talcott Parsons, ed. Max Black (Englewood Cliffs, New Jersey: Phentice-Hall, Inc., 1961), pp. 113-114. 3Katz and Kahn, 92, 913,, p. 37. 29 Goals and the production subsystem designed to attain these goals have a profound effect on organizational behavior: the task demands initially define the roles, determine the need for an elaborate or simple division of labor (structure of roles), shape the system norms, and provide the basis for system coding. Normally, the goals and supportive production system provide the fundamental basis for organi- zational identity. For instance, the goals of higher education are normally vaguely stated (provide a, "liberal education"), multiple (transmit knowledge, create new knowledge, socialize students, provide public service) and conflicting (teach students, do research, permit individual student autonomy, provide housing for 20,000 students).1 Such goals encourage a high degree of equifinality, very general rules and regulations, a decentralized authority structure, few system-wide shared values and sub-system shared values which conflict with one another, a wide variety of task demands and system norms which support entrepreneurship, independence and a minimum of over-all coordination. In brief, the goals contribute heavily to a university's identity. To attain goals and maintain the production subsystem, an organi- zation must provide the proper support. It must maintain an appro- priate quality and quantity of inputs, allocate outputs to the larger environment and be certain that the organization maintains its legitimacy 1Adapted from a list of education goals in Matthew B. Miles, "Some Properties of Schools as Social Systems," Change in School Systems, ed. Goodwin Watson (Washington, D.C.: Natihnal Training Laboratories for the Cooperative Project for Educational Development, 1967), p. 6. 30 in the eyes of people external to the organization. A large university fulfills these needs primarily through its admissions office, a place- ment bureau and a Board of Trustees respectively. An organization must also maintain the commitment of the individual members to the enterprise. This process entails not only the intelligent use of rewards (raises, promotions, praise from superiors, inclusion in the influence system) but also the recognition of the need for rest and recuperation after intense periods of goal attainment and the desire for personal growth and development on the part of individuals. Organiza- tions simply cannot ignore these basic human needs if they are to main- tain their members' motivation. In a university setting maintenance behavior on the part of a college might entail all of the following: recruiting new faculty suitable for the college and orienting them to their new work mfiljeg, promoting outstanding faculty, recommending a teacher for a special fellowship, permitting and encouraging trips to professional conferences, providing sabbatical leave with pay, estab- lishing a school year which lasts only nine months. In a changing environment an organization must devote a share of its energy to the function of adaptation or run the risk of entropy. As Katz and Kahn emphasize: External changes in taste in cultural norms and values, in competitive organizations, in economic and political power--all these and many others reach the organization as demands for intern- al change. To refuse to accede to such demands is to risk the possibility that the transactions of procurement and disposal will be reduced or refused, or thatlthe processes of maintenance will become increasingly difficult. 1Katz and Kahn, 925 913,, p. 42. 31 Thus, the best universities have adaptive units such as those recommended by John Gardner, Ernest Palola and Everett Rogers.1 These units anticipate and are responsive to changes in the world of higher education and the environment at large. At their best they can forecast such developments as the increasing demand for a college education by culturally deprived and disadvantaged students, the need for degree programs of varied pace and the extensive use of television and film as media for instruction. Special units are needed for adaptation since the other subsystems are preoccupied with their own tasks. Finally, there is a need for a managerial subsystem to coordinate the efforts of persons involved in other subsystems (i.e., to assure that energy is devoted to both production and maintenance functions) and to maintain the legitimacy of both the regulatory and authority systems.2 The regulatory mechanism is simply a formalization of the feedback process discussed earlier and assures that the organization will be guided by information about its efficiency and effectiveness. For instance, universities are now obsessed with cost-effectiveness planning based on such measures as the number of credit hours produced per full-time equivalent faculty member and the comparative cost of 1John Gardner, Self-Renewal, the Individual and the Innovative Society (New York: Harper and Row, Publishers, 1963), p. 76. Ernest Palola and William Padgett, Planningfor Self-Renewal (Berkeley, California: Center for Research and Development in Higher Education, University of California, 1971), p. 96. Everett Rogers, "The Communication of Innovations in a Complex Institution," Educational Record, vol. 49, no. 1 (Winter 1968), p. 76. 2Katz and Kahn, gp,lgig., p. 43. 32 undergraduate versus graduate education. Through a regulatory mechan- ism, the university administration can establish certain minimum cost-effectiveness standards. Management also maintains the authority structure, i.e., the process by which decisions are made and imple- mented. In a university setting, for instance, it is the administra- tion's responsibility to assure the authority structure remains somewhat coincident with prevailing system norms. Otherwise, the structure's legitimacy may be severely undercut, and decision-making will become a much more difficult process. By viewing an organization as an open system with two basic dimensions--production and the maintenance of human motivation-~one can avoid the traditional arguments between the human relations school of management (McGregor, Argyris, Thompson)1 which stresses the socio- emotional needs of organizational members and the scientific school of management (Taylor, McMurray)2 which stresses the production needs of an organization and the desirability of controlling members' behavior. Avoiding the argument does not remove the dilemma, however. The most ubiquitous characteristic of formal organizations is the conflict be- tween behavior necessary for production (role or organizational behavior) lDouglas McGregor, The Human Side of Enterprise (New York: McGraw- Hill, Inc., 1960). Chris Argyris, "Personality and Organization Theory Revisited," Administrative Science Quarterly, vol. 18, no. 2 (June, 1973), pp. 141-167. Victor Thompson, "The Innovative Organization,“ Organizations and Human Behavigr, ed. Fred 0. Carver and Thomas J. Sergiovanni(New York: McGraw-Hill Book Co., 1969), pp. 392-403. 2Frederick W. Taylor and Robert N. McMurry as cited in Warren G. Bennis, Changing Organizations (New York: McGraw-Hill Book Co., 1966), pp. 66-67, 71-72. 33 and that exhibited by organizational members during periods of main- tenance. Talcott Parsons sumarizes the behavior patterns associated with each organizational dimension. During the production phase, members should: 1. practice NEUTRALITY; i.e., use their relationships for pursuing the goal and not for gaining personal gratification; 2. concentrate on PERFORMANCE; i.e., deal with their fellows on the basis of their contributions to the organization; 3. practice SPECIFICITY; i.e., restrict their relationships with organizational members to matters of business; 4. practice UNIVERSALISM; i.e., ignore the impact of their relationships to their fellows outside the criteria imposed by the task setting.1 In universities, for instance, it is important that professors evaluate students on the basis of their performance and not on their personal like or dislike of the student. And, too, administrators should promote instructors on the basis of their professional competence, not on their agreement or disagreement with an instructor's political beliefs. During maintenance functions, members should: 1. practice AFFECTIVITY; i.e., allow emotionally charged rela- tionships to develop with fellow members; 2. concentrate on QUALITY; i.e., value other members for their own sake, not only for their contributions to the organi- zation; 3. practice DIFFUSENESS; i.e., broaden their relationships with one another to include a wider range of personality than exhibited during business matters; 1Henry A. Landsberger, "Parsons Theory of Organizations," The Social Theories of Talcott Parsons, ed. Max Black (Englewood Cliffs, New Jersey: Prentice-Hall, Inc., 1961), pp. 219-220. 34 4. practice PARTICULARISM; i.e , value fellow members for their uniqueness as human beings.1 In college settings, coffee lounges permit faculty and students to unwind and relate to one another as human beings rather than as role occupants. In healthy collegial environments, motivation is frequently regenerated as faculty and/or students begin to appreciate the problems that others face as they perform their roles. Naturally, organizational behavior cannot be divided so neatly in reality, but it is quite evident that both types of behavior are required of individuals in any complex organization. For this reason, even though successful organizations are generally marked by their stability, tension and conflict are common attributes of organiza- tions. Recognizing the inevitability of conflict enables healthy organizations to make adjustments routinely and move to a new point of equilibrium. Unhealthy systems, on the other hand, frequently have difficulty because they choose to ignore the basic dichotomy inherent in organizations, and permit conflicts to reach crisis proportions. Organizations process information (coding and feedback) as well as energy from the environment. For example, while seeking to influ— ence students' attitudes and behavior and produce new knowledge, universities process messages related to these two activities. As- Ronald Havelock emphasizes, the "through-put" of messages describes the life of an organization as much as the "through-put" of material or persons.2 In addition to operational feedback, which has already 'Ibid.. pp. 219-220. ZHaVETOCk 9;;- a_]_o ’ 9‘2. Eig- 9 p . 6‘4. 35 been discussed under regulatory mechanisms above, there are two basic types of communication flow of interest to the present study: Horizontal communication, which occurs between members of an organiza- tion on roughly the same level of the hierarchy of control (authority structure) and vertical communication, which occurs between upper and lower levels on the hierarchy of control. In organizations with many levels in the hierarchy of control (bureaucracies), vertical communication is normally downward and con- sists of job instructions, job rationale, procedures and practices, performance feedback or indoctrination.‘ That is, it is almost always informational rather than problem-solving. Upward flow is rare indeed, and when it does occur is also informational: ". . . the subordinate can report to his boss about what he has done, what he thinks needs to be done, what those under him have done, what his peers have done, his problems and the problems of his unit, and about matters of organiza- tional practice and policy."2 Upward flow is almost always distorted in a rigid hierarchy since the superior controls the job security of the subordinate. Thus, most often, the subordinate tells the superior what he wants to hear and what the subordinate wants him to know. Conmunication also occurs among peers or horizontally in complex organizations, and recent studies have shown it to aid the purposes of 3 an organization. Horizontal communication may well rigidify a 1Katz and Kahn, 92, cit., p. 239. 21bid., p. 245. 3Trist, gg. _c_i_t_., p. 278. 36 hierarchy of control, since when done in the extreme, people only talk to other people having identical roles and status. In most instances, however, extensive peer communication signifies an active communication flow within the organization. As organizations become increasingly authoritarian and have centralized control, horizontal communication is discouraged in order to isolate individuals in their various roles and force them to rely more on their superior for information. Universities are marked by both vertical and horizontal flow of information, but within any single unit horizontal or peer communica- tion predominates. This dominance is a function of decentralized con- trol and the socio-emotional support that organizational members obtain from peers. Communication patterns common to universities are particularly important to the present study, since innovations come into organizations as messages and must be processed by the existing comnunicati on network. Organizational members with their individual value systems, attitudes, beliefs, backgrounds, preferences and behavioral patterns also influence organizational functioning. In analyzing a particular event, it may be as important to be familiar with the role occupant as the role structure. Whether the members are homogeneous or hetero- geneous and whether their preferences are compatible with organizational norms, structures and goals are usually important factors. A homogene- ous membership whose preferences support organizational norms, struc- tures and goals can lend greater stability to an organization whereas the opposite member characteristics may promote at least temporary disequilibrium. 37 Although the maintenance subsystem of the organization attempts to maintain member motivation, the conflict between production demands and individual needs may be so great as to spawn the birth of an in- formal system among the members. In such groups members "interact, make decisions of their own, and cooperate among themselves, and so find gratification for their needs for self-determination and self- expression."1 The emergence of informal organizations is really determined by the degree to which members feel their individuality, self-expression and self-determination are stifled by role requirements. Informal groups can hinder or facilitate the production of an organi- zation depending on how oppressive they find the role structure and role enforcement. In sum, the organizational context is indeed complex. As an open system, it is marked by the energic cycle of input, through-put and output, the coding of information, the processing of feedback on its own functioning, negative entropy to resist running down, homeostasis to maintain a dynamic equilibrium, differentration and elaboration, and equifinality, the tendency to reach the same final state from differing conditions. In addition, an organization is a structure of roles main- tained by task demands, shared values, system norms and rules enforced by an authority system. It is a socio-technical system with specific functional imperatives--production, maintenance, adaptation and manage- ment-~in order to cope with the inherent dichotomy between individual human needs and the production requirements. It consists of human 1Katz and Kahn, 99, 913,. PP. 80-81. 38 beings with their own needs, values, attitudes, backgrounds and pref— erences who frequently establish an informal group to fulfill their desires for self-control and self-expression. Finally, it has com- munication networks for processing information internally. When the above elements work in concert, the most dominant char- acteristic of an organization is its stability.1 This is not to say that the system is unchanging, for as has been made clear, the suppor- tive and adaptaive subsystems enable an organization to adapt to a changing environment; the only unchanging organization is a dead one. The stability, instead, is a dynamic equilibrium which may be marked by a somewhat different identity at different points in time (modified goals, a streamlined production process, broader membership qualifica- tions). The term stability is used to emphasize that when there is neither an internalrun‘environmental crisis, organizations are not noted for sudden lurches in new and different directions. Detroit will continue to manufacture automobiles unless there is a technological breakthrough that forewarns the automobile's obsolescence or a crisis of the proportion of World War II, when Detroit shifted to armament production. Nevertheless, some organizations change more readily than others, and it is interesting to note the differences between those human 1See Daniel E. Griffiths, "Administrative Theory and Change in Organizations," Innovation in Education, ed. Matthew Miles (New York: Bureau of Publications, Teachers College, Columbia University, 1964), p. 425; see also, Goodwin Watson, 0p. cit., p. 107. 39 systems which seem to accept and use new information (innovations) easily and those which do not. It is generally conceded that the more open a system, the more susceptible it is to influence by new informa- tion from the environment. That is, a system which has a loosely defined coding scheme, a role structure with general behavioral expec- tations which permit individual initiative, general rules and regula- tions, norms of openness to external influence, an authority system which permits wide membership participation and influence, rewards for innovative behavior, a degree of membership turn-over, occasional change in leadership, an open communication network, and an emphasis on the adaptive and supportive subsystems equal to that on production and maintenance will be more likely to be aware of and adopt relevant inno- vations. Or, in the phrase of J B Lon Hefferlin, a system which permits the "dangers of instability"1 which result from an openness in many areas of the environment, is the organization most likely to try new ideas. It is important to note that the above generalizations do not say that changing organizations are good or bad; nor do they imply anything about the quality of specific changes or whether organizations which adopt innovations more readily than others do an adequate job of implementing innovations. These are important qualifications since the research evidence which gives rise to the generalizations is based pre- dominantly on one type of measure--the number of changes that a specific 1d B Lon Hefferlin, Dynamics of Academic RefOrm (San Francisco: Jossey-Bass Inc., 1969), p. 163. 40 organization has adopted during a particular time period."2 Additional research evidence to support some of the above generali- zations regarding organizational change will be reviewed later in this chapter. Before moving on, however, another cautionary note is neces— sary. A person interested in introducing innovations to organizations should not expect the generalizations to hold under all conditions; there is simply no empirical research which reveals the differential impact of the characteristics of an organization under varying condi- tions. As J B Lon Hefferlin states: . . . no one factor--no one specific characteristic-- appears to be either sufficient or an invariably necessary ele- ment in accounting for the differences that do exist among institutions in their amount of reform. Neither residential leadership [management] nor faculty collegiality degree of participation and influence] nor high faculty turnover by them- selves appear to contribute unilaterally to the process. Instead a whole network of factors-~attitudes procedures, mechanisms, pressures--appear to be involved.5 IFor research of this type see: Hefferlin, 99, cit. Griffiths, 99. E” pp. 425-436. Paul R. Mort, "Studies in Educational Innovation from the Institute of Administrative Research: An Overview," Innovation in Education, ed. Matthew Miles, pp. 317-328. Richard Carlson, Adgption of Educational Innovations (Eugene, Oregon: Center for theTAdvanced Study Bf'Educational Adfiinistration, University of Oregon, 1965). 2For a detailed accounting of the research evidence sup orting the generalizations regarding innovative (i.e., rapidly changing) organiza- tions see: Havelock 99_91,. Pp. 6-1 through 6-40. 3Hefferlin, 93. 919.. p. 189. 41 The Change Process1 Ronald Havelock 99._l. offer an excellent review of the three models of change stemming from three separate research traditions concerned with the change process. They are: 1) The Social Interaction (S-I) Perspective, 2) The Research, Development and Diffusion (R, O & D) Perspective, and 3) The Problem-Solver (P-S) Perspective. As can be seen from Figure 2-1, they all deal with different aspects of what Havelock terms the dissemination-utilization process, i.e., the change process in its entirety, from the generation of new knowledge, a new practice or new technology to the incorporation of the innovation in an ongoing organization or system. Social Interaction Perspective The 5-1 model (termed Communication of Innovation's Theory in Chapter I) emphasizes and describes the diffusion stage of the change process and concentrates particularly on the rate at which individuals rather than social systems adopt particular innovations. As the 1All of the available research has not been covered due to the high volume (Havelock e;_al., review over 4000 studies and Rogers with Shoe- maker reviews over 1500). An attempt was made to cover both the major theorists and major empirical researchers in the area. Please see the following bobks for more extensive bibliographies: Ronald Havelock et al., Plannin for Innovation (Ann Arbor: Institute for Social Research, 1969). Everett Rogers with Floyd Shoemaker, Communication of Innovations (New York: The Free Press, 1971). Warren Bennis, K. Benne, R. Chin, The Planning 9f Change (New York: Holt, Rinehart and Winston, Inc., 1969). 42 .mNLaLS .85 482 6:39 meuom coe mpzuwpmcH "Lonc< =coccH comimcwccmpm ..Fm um xuopm>mz upmcom "moeaom .cuemommc Lo m_oo;Um muggy cwgpwz mmcmcu Lo mpmvos cw umn:_u:w >__muwaau mmmmpm .—.N we:m_m _ $350 3 _ _ covmaeuvo «_nwmmo n cowwmuehmcmrw3uru _ co 11m co 4. co «2.3.2: W cozoucmuoifié ..oe .9151“ o>_uccemupm emu gum a meowuapom w_p+mmoa1muo: u>n w miwmmr. 1 1 1 1 1 1 1 1 A _ uauqu .cvmuso ;»_z ma_u_aowea yea “_eoo ;._Paawmu1 . casamm< ” Am1av avcmcowuopwm :mwppuumm1 m:o.u=_om Lou cues~$Wle|1¢J guacamom . eo>Fom 1 11M~m0cuown . u_mam _ 1eu—aoga _ vow:— cowuoufl_aco_»=uepmcu covuaou< cowuo__auwc~ p.322 co_pnapo>u umocmucm up»: Ana: .mv mmocogoz< :_ugh sumo emsuuo covmavm—o muceumcoewn.1 ouo=_o>m a pump "mempaogm «new, a «gangs: 1 ommxuaa a coocwmc 1mm>cH uxcwaccm acuEQopm>oa commoo m ucw>:_ uwmvucupum urn :ueuomwm pzmza04u>uo mm 41 1 a A~1mv _ _ ..... 1 1 1 1 1 1 1 1 1 1 1 1 1 1 coFuuoLmucH “ ewesmm< »m_>euu< ...- - ........ vagamm< "x. . 1 u upssmm< . ..Fuom r :o'msem_o meow . . ScoEQoFm>mo _ . coguomux " co?» ou< use copmammeo «emamm~m>mo someumum -suuoomoa L c» 393m 5 33am 5 333 .3 egg 43 Havelock figure makes evident, researchers and theorists in this tradition do not discuss the invention, research and development stages of the change process but rather concern themselves with condi- tions and characteristics which lead to diffusion and adoption. In ,their analysis they most frequently examine the traits of the innova- tion, the channels of communication used, the social system and its members, and the stages through which an adopter of an innovation goes. Research from the 5-1 perspective has been done in agriculture (Lionberger, Rogers, Ryan and Gross), education (Mort, Carlson), and medical sociology (Coleman, E. Katz, Menzel) to name just a few fields.1 Everett Rogers is by far the most prolific writer in the field and has written or co-authored two compendia of research on the communication of innovations.2 In both he identifies the four critical elements involved in the diffusion and adoption process: Crucial elements in the diffusion of new ideas are (l) the innovation (2) which is communicated through certain channels (3) over time (4) among the members of a social system. 1Herbert F. Lionberger, Adoption of New Ideas and Practices (Ames, Iowa: Iowa University Press,’l960). Bryce Ryan and Neal C. Gross, “The Diffusion of Hybrid Seed Corn in TWo Iowa Communities," Rural Sociology, vol. 8 (1943), pp. 15-24. Paul Mort, o . 919, Richard Car son, 0 . cit. Coleman, Katz and enEET, "Social Processes in Physicians' Adop- tion of a New Dru ," Social Chan e, ed. Amitai and Eva Etzioni (New York: Basic Books, 1964], pp. 439-454; see also, . Rogers with Shoemaker, 99, 913,. pp. 48-70 for a complete listing of research traditions. 2Everett M. Rogers, Diffusion of Innovations (New York: The Free Press, 1962); see also, Rogers with Shoemaker, 99, 919, 3Rogers with Shoemaker, 99, 919,, p. 18. 44 The elements identified are similar to those mentioned by other researchers in the 5-1 tradition. For example, Katz 91_91, define diffusion as: the (1) acceptance, (2) over time, (3) of some Specific item--an idea or practice, (4) by individuals, groups or other ado tin units, linked to (5) Specific channels of comnunication, (6) to a soeial structure and (7) to a given system of values, or culture. It is important to note, as do Katz and Rogers, that time is an explicit variable in the diffusion process. Including time explicitly emphasizes that adoption is a process involving several phases rather than a single act. The discussion which follows depends on Rogers' model of diffusion since it is less intricate than the model of Katz 9.12.9.1.- The innovation "is an idea, practice or object perceived as new by an individual"2 or social unit. That is, it does not matter if an idea is objectively new so long as it is new to the individual or social system in question. A wide range of specific innovations have been studied: teaching methods (Paul Mort, Richard Carlson),3 new drugs (Coleman 99“91,),4 stone axes (Sharp),5 water boiling (Williams)6 1E. Katz, M. Levin, and H. Hamilton, "Traditions of Research on the Diffusion of Innovations," American Sociological Review, vol. 28 (1963), p. 237. 2Rogers and Shoemaker, 99, 911,, p. 19. 3Mort, 99. 9_i_t_.; see also, Carlson, 99. ci 4Coleman 9991., 99. 919. 01 it., p. 335. (1 As cited in Rogers with Shoemaker, 99, 61bid., p. 2. 45 and condoms (Rogers).1 As can be seen, they vary from the concrete to the abstract, and this difference among innovations as well as others affect their rate of diffusion and adoption. Rogers and Shoemaker list five characteristics of innovations: 1) "Relative Advantage is the degree to which an innovation is perceived as better than the idea it supersedes."2 Note that it is the perceived versus the objective advantage that is important. The perceived advantage may be lower cost, a time saving, higher prestige or more happiness, but regardless, if an innovation appears an improve- ment over a present idea or practice, adoption will occur more rapidly. 2) "Com9atibility is the degree to which an innovation is per- ceived as being consistent with the existing values, past experiences, and needs of receivers. An idea that is not compatible with the pre- valent values and norms of the social system will not be adopted as rapidly as an innovation that is compatible." When an innovation is introduced into a complex organization, as is the case with the present study, it is important that the innovation be compatible with current organizational structures also, e.g., the role structure, rules and regulations and the authority system.3 3) "Complexity is the degree to which an innovation is perceived as difficult to understand and use. Some innovations are readily 1Lecture notes of Everett Rogers, Fall 1971. 2Rogers with Shoemaker, 99, 919,, p. 22. This and the following quotations specifying the characteristics of innovations came from pp. 22-23. 3Gross e_t_91_., 99. 919., p. 198. 46 understood by most members of a social system; others are not and will be adopted more slowly." Generally, innovations which demand little new learning by the receiver will be adopted more rapidly than innova- tions needing new skills. (There is not as much empirical support for this generalization as the other four. This may well be due to the fact that complexity 999H99 is sometimes highly regarded in modern cultures. For example, complex games are popular among intellectuals because they are considered a challenge to master. Both bridge and chess continue to be adopted, perhaps because of their complexity.) 4) "Trialability is the degree to which an innovation may be experimented with on a limited basis. New ideas which can be tried on the installment plan will generally be adopted more quickly than inno- vations which are not divisible." 5) "Observabili9y is the degree to which the results of an inno- vation are visible to others. The easier it is for an individual to see the results of an innovation, the more likely he is to adopt." For example, the results of educational innovations are frequently dif- ficult to observe, and many persons are reluctant to abandon tradi- tional patterns until shown that the new one is superior.1 Rogers and Shoemaker have summarized an extensive amount of empiri- cal support for the effect of the above characteristics on the diffusion and adoption process with the exception of the trait, complexity.2 O . 1See Henry Brickell, Organiz1n New York State for Educational Chan e (Albany, New York: State Education Department, 1961), pp._27-30, or t e role of demonstration in educational innovation. 2Rogers with Shoemaker, 99, 919,, pp. 350-352. 47 The list of characteristics is not exhaustive but includes those which have most often been identified by researchers as having an effect on diffusion and adoption. Among other traits that Havelock 99_al. suggest as useful for classifying innovations, one type seems a particularly useful addition to the Rogers and Shoemaker series. It is stated in the form of a question, "How much change in receiver?"1 and has four categories-- change in size and scope of operation, acquiring new skills, changing goals and changing values and orientations. In essence, Havelock 99 91. are attempting to identify the behavioral consequences of adop- tion. Stated differently, one might ask, “Just how disadvantageous, incompatible and complex is the new idea?" For the purposes of the present study, one might change the above four categories to the various components of a complex organization identified earlier in this chapter. One could then ask, how much change does adoption of the innovation require in the role structure, the task demands, the system norms, shared values, the rules and regulations, the authority system, and personal member preferences, and hypothesize that the greater the amount of change required by the innovation in any or all of the categories, the less likely that adoption will occur. At the very least, one would hypothesize that the need for major change in any or all of the above organizational categories would slow the adap- . 2 t1on process. 1Havelock 9991., 99. 919., p. 8-46. 2Supported by the tendency for systems to seek homeostasis, dis- cussed earlier in the present chapter. 48 The second element in the diffusion process is communication, "the process by which messages are transmitted from a source to a receiver . . . with a viewpoint of modifying the behavior of receivers."1 The channel serves as the vehicle for the message. In the case of the change process, an innovation is the message and the flow of the innovation to the receiver can be affected by (1) the channel used for communication (2) the system in which it occurs (3) the relationship between the source and the receiver. There are two basic channels for communication: mass media (newspapers, radio and television) and interpersonal (face-to-face contact). Mass media channels seem most effective when the desire is simply to inform persons of the existence of innovations, whereas inter- personal channels are more powerful when persuading persons to adopt.2 As was shown in the earlier part of this chapter, a system may affect the communication of an innovation in several ways. The author- ity structure may limit knowledge of the innovation to the downward, vertical communication network, or the system may be more open, and the innovation will be discussed in horizontal as well as vertical networks. In addition, certain role occupants may influence the flow of an innovation through a system. For instance, it has been suggested that the power elites of social systems serve as "gatekeepers to prevent 1Rogers with Shoemaker, 99, 919,. PP. 23-24. 2M. Becker, "Factors Affecting Diffusion of Innovations Among Health Professionals," American Journal of Public Health, vol. 60, no. 2 (2/70), p. 296. 49 restructuring innovations from entering a social system, while favoring functioning innovations that do not immediately threaten to change the system's structure."1 Opinion leaders (see below for characteristics), those persons who exert influence in the informal system of an organization, may hinder or facilitate communication of an innovation within a system. If they denigrate an idea, the horizontal communication network soon serves to spread skepticism among most members. Finally, the communication process may be influenced by the rela- tionship between the source of the new idea and the receiver. In most cases involving innovations, the source and receiver are heterophilous, i.e., they differ on such attributes as background, beliefs, values and culture. This difference forms what Rogers terms the heterophily gap and can frequently act to impede the diffusion process.2 However, if the source can overcome this barrier and come to be viewed as credi- ble and trustworthy, effective communication may occur.3 Most effective communication occurs between a homophilous source and receiver, i.e., when the two are similar in such traits as background, values and cul- ture. For instance, an Opinion leader is able to exert influence because he is homophilous with group members as well as having their respect. 1Everett M. Rogers, "Social Structure and Social Change," American Behavioral Scientist, vol. 2 (June 1971), p. 773. 2Rogers with Shoemaker, 99, 919,, p. 15. 3Havelock 991-11., 99. 919., p. 5-16. 50 Time is the third important element in the diffusion and adoption process. Rogers and Shoemaker point to three areas in which time is a factor: The time dimension is involved (1) in the innovation- decision process by which an individual passes from first knowl- edge of the innovation through its adoption or rejection, (2 in the innovativeness of the individual, that is, the rela- tive earliness-lateness with which an individual adopts an innovation when compared with other members of his social system, and (3) in the innovation's rate of adoption in a social system, usually measured as the number of members of the system that adopt the innovation in a given time period. Rogers (prior to 1968) and others (Lionberger,1 Beal and Bohlenz) use the phrase, adoption process, to refer to what Rogers and Shoemaker dub the innovation-decision process in the above quotation. As can be seen in the Social Interaction row of Table 2-1, writers itemize five stages--awareness, interest, evaluation, trial and adoption. These stages are relatively clear-~during initial awareness, information about the innovation is incomplete, and if one is interested, he begins to seek a fuller understanding. Upon doing so, he evaluates the new idea in relation to his present practice and decides whether or not it is advisable to try it. After a trial he either rejects the innovation in favor of his present practice or adopts the idea in lieu of his present practice. In addition to renaming the process, Rogers and Shoemaker condense the original five step model to one with four functions--knowledge, 1Rogers with Shoemaker, 99, 919,, pp. 24-25. 2Lionberger, 99, 919, 3George M. Beal and Joe M. Bohlen, The Diffusion Process, Special Report #18 (Ames, Iowa: Iowa State College, 1957). 51 persuasion, decision and confirmation. The knowlegge function occurs when the individual is exposed to the innovation's existence and gains some understanding of how it functions. The persuasion function occurs when the individual forms a favorable or unfavorable attitude toward the innovation. The decision function occurs when the individual engages in activi- ties which—Tead to a choice to adopt or reject the innovation. The confirmation function occurs when the individual seeks rein- forcement for the innovation-decision he has made, but he may reverse his previous gecision if exposed to conflicting messages about the innovation. The newer model is preferable to the former one since it clarifies that either rejection or adoption are possible alternatives in the proc- ess and emphasizes that an individual may change his mind about the innovation after his initial decision. Rogers and Shoemaker modify the phases in the innovation-decision process somewhat when describing the manner in which social systems adopt or reject relevant innovations. They see two basic types of organizational innovation-decisions: authoritative and collective. The first consists of knowledge, persuasion, decision, communication and action whereas the second involves stimulation, initiation, legitimation, decision and action. Pure authority decisions might be better termed "authoritarian" since as Rogers and Shoemaker view the process, the leader or authority structure of an organization learns of an innova- tion and becomes thoroughly familiar with it, develops a favorable atti- tude toward it, decides to adopt, communicates the decision to organiza- tional members and makes certain that the members implement the decision.2 The members have no voice in the process. Collective \ 1Rogers with Shoemaker, 99, 919,, p. 25. 21bid-. pp. 304-313. 52 decisions, on the other hand, ensue when a member or members are stimu- lated by a new idea or practice. They in turn, excite others in the system who initiate a specific proposal and wider discussion. After the idea receives legitimization from experts and opinion leaders, the authority structure is activated to make a decision and assure that the new practice is implemented.1 In the purest form, collective inno- vative decisions involve all members in the decision-making process. However, they occur more rapidly when there is a power concentration within a system.2 It is helpful for the purposes of the present study to think of Rogers and Shoemaker's two types of system innovation-decision proces- ses as existing at the opposite ends of a single continuum, represent- ing the minimum (pure authority) and maximum (pure collective) amount of member participation and influence in the decision process. Recalling the discussion of complex organizational behavior earlier in the present chapter, it is unlikely that many organizations would follow either of the two extremes as outlined above. Authoritarian decisions in all but the most coercive of environments tend to raise the resent- ment of members because the decisions ignore the human desire for 3,4 influence and self-control. The 1971 events in the Lordsville, Ohio, * llbid . pp. 275-291. 21bid., p. 334. 3Katz and Kahn, 99, 919,, p. 80. 4Chris Argyris, "Individual Actualization in Complex Organizations," Organizations and Human Behavior, eds. Carver and Sergiovanni (New York: McGraw-Hill Book Co., (1969), pp. 189-90. 53 Vega automobile plant where workers sabotaged the assembly process in response to a work speed-up is a clear illustration of what may happen when management does not consider member desires. Pure democracy is equally unviable since it may well lead to stalemates in controversial areas. In fact, most system innovation-decisions properly fall some— where along the continuum, and the better ones seem to involve a high degree of member participation in at least the decision stages of the process. This beneficial effect of member participation in decision- making has many components: member satisfaction with the decision is higher if the members have helped make the decision;1 persons who have been influential in the adoption process are more likely to use the innovation;2 persons who participate in and wield influence during the decision-making have a higher commitment to the resulting deci- 3 the process of shared decision-making activates the force of sion; peer pressure once a decision is made;4 the process permits the members to make clear the problems they foresee as potential users of the innovation;5 the process is consistent with most persons' desire for 1Rogers with Shoemaker, 99, 919,, p. 286. 2Nan Lin 99 91,, The Diffusion99f an Innovation in Three Michigan High Schools: Institution BuildingLThrough Change(East Lansing, Michigan: Institute for International Studies in Education and Depart- ment of Communication, Michigan State University, 1966), p. 2. 3Edith Pelz, "Discussion, Decision Commitment and Consensus in 'Group Dicision,'" Human Relations, vol. 8 (1955), pp. 251-274, as cited in Havelock £91., 99. 919., p. 5-2. “Ibid.. p. 5-2. 5Rogers with Shoemaker, 99, 919,, p. 287. 54 self-expression and self-control.1 The message seems clear; innovations are more likely to be adopted and implemented if the potential users are permitted to participate and exert their influence in part of the innovation—decision process. Time is also a factor in distinguishing the types of adopters since some persons adopt earlier than others in the same social system. Rogers and Shoemaker identify five categories--(l) innovators, (2) early adopters, (3) early majority, (4) late majority, and (5) laggards--which range from the most innovative individuals to the least as determined by speed of adoption.2 For the purposes of the present study, it is important only to know how persons in the first two categories compare to others in the social system. Innovators tend to be marginal to their social sys- tems, are more venturesome, more cosmopolite and more willing to risk possible setbacks for the sake of trying new ideas than other members of their social system. Those persons who adopt early (including inno- vators) have more educatiOn, are more literate, have higher social status, have greater empathy, are less dogmatic, are better able to deal with abstractions, are more rational, are less fatalistic, are more intelligent, have a more favorable attitude toward change, take more risks, have higher levels of achievement motivation, have higher aspirations, are more highly integrated into their social system (with 1Katz and Kahn, 99, 919,, p. 80. 2Rogers with Shoemaker, p. 287. the exception of innovators), are more cosmopolite, have greater exposure to mass media, have greater exposure to interpersonal comnuni- cation channels, seek more information, know more about innovations, exert more opinion leadership, and are more likely to belong to a system with modern norms (see below) than later adopters.l Time is also a factor in the rate of diffusion. It has been shown through a variety of studies2 that diffusion in any single social sys- tem involves extensive interpersonal communication once the adoption process begins with a few individuals, and that this social interaction speeds the adoption process. Members of a social system comprise the last important element in the diffusion process. Diffusion research has examined the impact of member characteristics such as education and age (see previous summary concerning early adopters) and system norms on the diffusion process. Rogers and Shoemaker develop two ideal types of system norms--traditiona1 and modern--to differentiate systems which view change favorably and are therefore likely to adopt innovations from those resistant to change. They characterize the norms as follows: Traditional social systems can be characterized by: 1. Lack of favorable orientation to change. 2. A less developed or "simpler" technology. 1Ibid.. PP. 174-196, a summary of the generalizations made from 228 empirical studies. 2Elihu Katz, "The Social Itinerary of Technological Change: Two Studies on the Diffusion of Innovation," The P1annigg_of Change, ed. Warren Bennis, K. Benne, R. Chin (New YorEY' Holt, Rinehart and Winston, Inc., 1969). 56 3. A relatively low level of literacy, education, and under- standing of the scientific method. 4. A social enforcement of the status quo in the social system, facilitated by effective personal relationships, such as friendliness and hospitality, which are highly valued as ends in themselves. 5. Little communication by members of the social system with outsiders. Lack of transportation facilities and communica- tion with the larger society reinforces the tendency of individuals in a traditional system to remain relatively isolated. 6. Lack of ability to empathize or to see oneself in others' roles, particularly the roles of outsiders to the system. An individual member in a system with traditional norms is not likely to recognize or learn new social relationships involving himself; he usually plays only one role and never learns others. In contrast, a modern social system is typified by: A generally positive attitude toward change. A well developed technology with a complex division of labor. A high value on education and science. . Rational and businesslike social relationships rather than emotional and affective. . Cosmopolite perspectives, in that members of the system often interact with outsiders, facilitating the entrance of new ideas into the social system. 6. Empathic ability on the part of the system's members, who are ablelto see themselves in roles quite different from their own. th-i 0 01 As Rogers and Shoemaker emphasize, these norms actually represent two extremes on the same continuum, and most social systems fall some- where between the two ends. Each ideal type represents a summary of the characteristics that researchers have found typical of systems which have, on the one hand, resisted innovations, and on the other, welcomed them. It is interesting to note that the modern norms as described by Rogers and Shoemaker are quite similar to the series of generalizations made earlier in the present chapter to characterize the type of formal 1Rogers with Shoemaker, 99, 919,. pp. 32-33. 57 organization which is open to change. This parallel would seem to lend further support to the hypothesis that the more avenues for influence that a formal organization leaves open, the more often it will adopt innovations. Two figures play important roles in the majority of the social interaction studies--the opinion leader and the change agent. Opinion leaders are those persons influential in the informal networks of a social system due to their "technical competence, social accessibility and conformity to the systems norms."] They have the respect of their fellow members and are usually more exposed to external communication, more cosmopolite, more innovative and have higher social status than other members of the system.2 They are critical to the innovation- decision process, because their support of a new idea can spell its success whereas their cynicism almost always dictates its doom. The fact that opinion leaders are important to the adoption process offers further support to the notion that some organizational members should participate in the decision process in order to help overcome initial resistance, if any exists, and to increase the likelihood of adOption. The change agent is an advocate for the innovation in question, and he is usually sponsored by a change agency; thus he is the link between two social systems.3 He might be an agricultural extension 1Ibid., p. 35. ZIbi 31bid., p. 228. O. o 58 agent linking a university system to a farming community, a birth con- trol specialist linking a population planning agency with a rural Indian village or a consultant linking an organizational development group with a small college. Frequently, the change agent is quite heterophilous with the client system, and part of his task is to over- come that barrier so that he can be influential. Borrowing heavily from Lippit 99 91,, and Rogers and Svenning,1 Rogers and Shoemaker identify seven tasks for the change agent: (1) He "deVelops the need for change," i.e., he helps the client system identify its needs in the members' terms, and urges members to view the needs as soluble problems; (2) he "establishes a change relationship," i.e., through empathy he tries to establish his credibility, his expertise and his trustworthiness (stated differently he develops a client orientation); (3) he "diagnoses the problem," i.e., he must determine why existing alternatives are not solving the problem; (4) he "creates the intent to change in the client," i.e., after exploring alternative solutions with his clients, he tries to motivate them if they are not yet ready for action; (5) he "translates intent into action," i.e., along with his clients he designs an actual program of change which is acceptable to them; (6) he "stabilizes change and prevents discontinuances," i.e., he supports the clients as they engage in new behavior so that IRonald Lippit, Jeanne Watson and Bruce Westley, The Dynamics of Planned Change (New York: Harcourt, Brace and Co., Inc., 1958) as cited in Havelock 99_91,, 99, cit.. pp. 10-55 through 10-59. Everett Rogers and Lynne-Svenning, Managing Change (Burlingame, California: Operation PEP, Mimeo Report,—1969). 59 they do not become discouraged by the difficulty of changing; (7) he "achieves a terminal relationship,§ i.e., he enables the clients to continue the new behavior without his assistance. There is a good deal of empirical evidence to suggest that change agent success is closely associated with his degree of effort, his client orientation, his efforts to increase the client's ability to evaluate innovations, and his program's compatibility with client needs.1 These findings tend to support the above recommendations for change agent behavior. The strengths of the social interaction perspective are obvious: it has an extensive amount of empirical data to support its generaliza- tions; it identifies all the elements critical to the adoption process; it illuStrates that both personal and social system variables affect the diffusion process. However, from the vantage point of the present study, there are at least three weaknesses: there is no discussion of adoption by formal organizations; there is only implicit recognition that implementation and incorporation are important phases of the innovation-decision process; it concentrates more on sender and change agent activity than on the problems the innovating unit may have in adjusting to the change. Research, Development and Diffusion (R, D 8 D) Pergpective As is evident from the outline in the second row of Figure 2-1, the R, D & D perspective includes all phases of the change process-- 1Rogers with Shoemaker, 99, 919,, pp. 380, 382. 60 research, development, diffusion and adoption. The motivating force behind most of the RD&D theorists is the desire to see more new ideas reach the application stage, to turn theory into practice. Their entire approach argues that change should be a rational and planned rather than a serendipitous process.1 Unlike the social interaction perspective, there is no one model accepted by most writers--instead there are several, each with its own special emphases. Nevertheless, as Havelock 99_91, show in their examination of the research, the "Theory-Practice Continuum" of Egon Guba and David Clark provides a sound base for discussion. As Guba and Clark view it, planned change contains the four stages mentioned above and in Figure 2-l--research, development, diffusion and adoption. 132999999_has as its basic objective the advancement of knowlegge,"2 Research is really tangential to the more active part of the change process since the results may lie dormant or be used by a developer. Guba does not see the researcher as sharing any responsibility for implementing ideas. Rather, he should investigate, conceptualize and evaluate under controlled conditions to produce reliable and valid results. His work becomes the clay that persons in the developing sub- systems mold. 1 2Egon G. Guba, "Development, Diffusion and Evaluation," Knowledge Production and Utilization in Educational Administration, ed. Terry L. Eidell andTJoanne M. Kitchell (Eugene, Oregon: Center fer the Advanced Study of Administration, University of Oregon, 1968), p. 42. Havelock 9991,, 99. 919., p. 10-39. 61 Development, or what some persons term applied research, is the second stage of the theory-practice continuum, and Guba identifies its basic objective as, "the identification of operatingnproblems and the formulation of solutions to thoseproblems."1 The developer, unlike the researcher, is an applier of knowledge; he invents solutions based on research, expert advice and his own experience and designs an appro- priate model for use in a normal environment. As Guba describes the process for educational development, it unfolds as follows: The developer, through a continuous monitoring of opera- tional data (akin to process control), identifies particular operational problems which require solution. He invents a solu- tion by transmitting, translating, or transforming already exist- ing solutions, by synthesizing solutions from known but previously uncombined components, or by creating solutions de novo. In all of these processes he may look to research for gu1dance but re- search will be but one of several competing inputs. The invented solution is engineered into usable form, and finally is tested in a rea school situation. Its use is then warranted in the schools. "Diffusion has as its basic objective the creation of awareness about new developments and the9provision of opportunities for their assessment along_9hatever dimensions practitioners may deem necessary."3 Diffusion, then, makes the innovation available and comprehensible to the ultimate users through explanation and demonstration. The diffuser, in Guba and Clark's model of change, is quite similar to the change agent as depicted by Rogers and Shoemaker, for he has the responsibility 'Ibid. 2bd., p. 46. 3Ibid.. p. 42. 62 of assuring that the target system gives the innovation a fair trial. However, the diffuser appears more change agency than client system oriented in Guba and Clark's model. "Adoption has as its basic objective the adaptation of a develo9- ment to the local situation and its installation therein."1 For the purposes of the present study, one of the most important contributions of the Guba and Clark theory-practice continuum is its recognition that use of an innovation is not guaranteed by a decision to adopt. For Guba and Clark, use of an innovation entails three phases: a local 99191_wherein the organization can see if the new concept is suitable for local conditions; installation during which the innovation may be modified, members may be trained to use the new idea correctly, facili- ties may be changed and the organizational structure may be adjusted to support the new ideas and associated member behavior; and institutionali- 999199_when the new practice becomes an integral part of the ongoing organization.2 As a final point of emphasis, one should note that Guba considers evaluation a critical component in each of the four stages. Unless testing appropriate to each of the four environments is undertaken, the 1>lanned change process will have as its substance, knowledge and programs 0f low qualitya In devising his own model for planned change, Henry Brickell Stresses a point only implicitly made by Guba and Clark--the various \ 1Ibid.. p. 43. 21bidog PP- 49-51. 63 stages of the change process may well require different types of social systems, different types of people and different skills. In setting forth his argument, Brickell urges the total separation of the design environment, the evaluation environment and the dissemination environ- ment. As he states: . . . The ideal circumstances for the design of99_improved instructional approach are artifiElhl: enriched, and free. At their best, they provide a group of highly intelligent people, a somewhat limited problem, time to concentrate on a solution, ample money and resources, freedom to try almost anything, the likelihood that the solution will be used somewhere, and the prospect of personal recognition if the problem is solved. The more artificial, enriched, and free the setting, the most distinc- tive the innovation it is likely to produce. . . . The ideal circumstances for the evaluation of a new instructional approach are controlled, closely observed, and un- free. ’At their best they provide conditions in Which the forces which might influence the success of the new approach can be controlled when possible, and kept under close surveillance when actual control is impossible. The freedom which is essential in searching for a good design is destructive in the making of a good evaluation. . . . The ideal circumstances for the dissemination of a new approach througnndemonstration are those which are ordinary, 999nriched,and normal. At their best, they are exactly like the everyday situations in the observer's own school and community. Anything which the observer could label "abnormal" or "unrealistic"-- such as the enriched conditions necessary for good design or the controlled conditions necessary for proper evaluation-- 5 sufficient to rob the observed program of persuasive effect.T Industrial organizations are certainly not unfamiliar with the Pattern that Brickell suggests; the automobile industry, for example, ¥ 1Henry M. Brickell, "State Organization for Educational Change: a Case Study and a Proposal," Innovation in Education, ed. Matthew B. Ariles (New York: Bureau of Publications, Teachers College, Columbia University, 1964), pp. 498-499. 64 long ago separated itself into the subsystems described above. Education, however, has taken only the phase of pure research seriously while expecting on-going schools to generate, evaluate and adopt new teaching methods in addition to doing a competent job of educating large numbers of students. Although Brickell seems to overstate his case in urging the total separation of design, evaluation and demonstration (as Guba and Clark argue, appropriate evaluation must accompany each stage of planned change if workable ideas are going to be produced), the thrust of his argument is profound: aspects of the change process require different behavior of people and different kinds of support, One might ask, for example, are organizations with the modern norms described by Rogers or the system open to its environment described earlier in the present chapter as suitable for implementing and incorporating innovations as they are for adopting them? One can only hypothesize given the present state of knowledge concerning organizational change, but Brickell's observations certainly give one pause for thought. Three other theorists in the RD&O perspective develop somewhat different models of dissemination and utilization. Richard Carlson, in a paper presented at the National Conference on the Diffusion of Educa- titonal Ideas, terms the process the "natural history or life cycle of an innovation" which consists of "the story of the invention, develop- m811t, promotion, adoption, diffusion, and demise of the innovation, along "1 th an account of the problems encountered and solutions developed in i"til“oducing and maintaining the innovation in the [social] setting, and 65 the unanticipated consequences growing out of its use."1 Harbons Singh Bhola2 uses the Guba-Clark theory-practice continuum and adds a final stage called "Service and Support," emphasizing the need to help an adopting system adjust to the new demands of an innova- tion and to evaluate the innovation's effectiveness. Bhola is one of the few theorists and researchers who tries to cope with the problems raised by implementation. Matthew Miles in developing his model of planned change concentrates on developing a strategy that will lead to adoption. Borrowing heavily from the work of Everett Rogers, he describes his model as follows: We thus have four ways in which a strategy may be initiated. Chronologically, we may then think of a series of stages which occur prior to the actual adoption of an innovation by a target system. These stages include: (1) desi n--the innovation is invented, discovered, made up out of w o e cloth, produced by re- search and development operation, etc.; (2) awareness-interest-- the potential consumers of the innovation, that is members of the target system, come to be aware of the existence of the designed innovation, become interested in it, and seek information about its characteristics; (3) evaluation--the consumers perform a kind of mental trial of the innovation, and form pro/con opinions about its efficacy in accomplishing system goals, its feasibility, and its cost; (4) trial--the target system engages in a (usually) small-scale trial of the innovation, in order to assess its con- sequences. If these are favorable, the innovation tends to be adopted, and the strategy is complete. 1Richard 0. Carlson, Summary and Critique of Educational Diffusion W (mimeograph copy 6f a speech given at the National Conference 0r1 the Diffusion of Educational Ideas, Michigan State University, March 26-28, 1968), p. 2. 2Harbons Singh Bhola, A Theory of Innovation Diffusion 9nd Its AEPJication to Indian Education and Comnunity Develoment. Ph.D. Thesis (Columbus, Ohio: Ohio State University, 1965), p. 49. 3Matthew 8. Miles, "Educational Innovation: the Nature of the Frauen," Innovation 111 Education, ed. Matthew B. Miles (New York: Egreau of WchationS, Teachers College, Columbia University, 1964), - 19-200 66 Unlike Guba and Clark, Bhola and Carlson, Miles shows little aware- ness of or concern with the problems associated with implementation; Miles' change strategy and model are complete when the system decides to adopt. In developing their own synthesis of the literature on planned change, Havelock 99_91, develop a typology of linking roles which gives an action element to the R080 perspective.‘ Essentially, the typology is a much more sophisticated depiction of the concept of change agent than either Rogers2 or Lippitt 9991,3 include in their models of change; each linking role (with one exception) describes the activities of a person who aids the dissemination-utilization process at different stages. In other words, the "change agent" need not be just one person or set of persons. Several people, both inside and outside the user organization,4 must serve as advocates or nurturers of change, if imple- mentation and incorporation are to occur eventually. Ronald Havelock lists the following roles: (l) the conveyor or person who makes poten- tial users aware of the innovation; (2) the consultant, the person who identifies problems and assists a system in designing solutions (Rogers' and Lippitt's change agent); (3) the trainer to help persons 1See also Ronald Havelock, "Dissemination and Translation Roles," Iglowledge Production and Utilization in Educational Administration, eds. 'T- L. Eidell and J. M. Kitchel (Eugene, Oregon: Center for the Advanced (Sigudy of Educational Administration, University of Oregon, 1968), pp. -1l9. 2Rogers with Shoemaker, 99, 919,, pp. 229-230. 3Lippitt 99 91.. 99. c_i_t_., pp. 10-55 through 10-59. 4 ti That is, the organization which ultimately implements the innova- On, 67 change behaviors in accordance with the innovation; (4) the 199999, i.e., the informal (opinion leaders) and formal (administrators, officers) leaders of an organization who can arrange the organiza- tional support needed to implement innovations; (5) the innovator, the person who initially stimulates internal interest in a specific change; (6) the defender, the person who sensitizes persons to the possible negative consequences of adoption. He actually slows change but makes it more likely that the changes adopted will be relevant to the system.1 As is inmediately apparent, the persons in these Six roles give momentum to the dissemination-utilization process. Without their efforts, a single innovation might well fail at any of several points in the chain. The various RD&D theorists differ somewhat in their approach. Guba and Clark offer by far the most thorough accounting of dissemination- utilization, and Bhola builds on their model. Both Miles and Brickell, on the other hand, offer truncated versions of the process which all but ignore the special problems associated with implementation. Brickell does, however, call special attention to the fact that different organi- zational characteristics and different member skills are needed for different stages in the dissemination-utilization process. There are also Similarities: None of the models is based as heavily on empirical ‘ 1Havelock mentions three additional linking roles which are less relevant when considering, as does the present study, adoption by a Single complex organization. These are the knowledge producers, the practitioners and the users; the first is the source of knowledge and second two, users, and are thus less active in aiding the change process. See Havelock 9991., 99. 919., p. 7-4. 68 research evidence as that of Rogers and Shoemaker; the emphasis tends to be intersystem and interorganizational rather than intraorganiza- tional--a weakness from the perspective of the present study; all models represent predominantly rational approaches to change revealing an apparent assumption that organizations will adopt new ideas if they are demonstrated superior to present practices. Due to this commitment to rational persuasion, there is little discussion (with the exception of Havelock 99 91.) concerning the possible effect of system values, norms and structures. Problem-Solver (P-S) Perspective Ronald Havelock's name for the third school of theorists is quite appropriate since their ultimate aim is to instill a problem-solving approach in organizations rather than to describe how a single innova- tion is adopted and implemented. Some do describe how the adoption of specific innovations can be facilitated1 while others concentrate entirely on problem-solving as a process which enables continuous organizational change.2 Regardless, the approach is normative rather than empirical, and the following three quotations reveal the penchant for radical organizational change, or what has come to be known as organizational development (00): 1Watson, 99, 919,, pp. 106-115. 2Matthew Miles and Dale Lake, "Self Renewal in School Systems: A Strategy for Planned Change," COnce99s fer Social Change, ed. Goodwin Watson (Washington, D. C.: National Training Laboratories for the Cooperative Project for Educational Development, 1967), pp. 81-88. 69 . . . The process of 00 can be defined as the creation of a culture which institutionalizes the uSe of‘VariouS'SOCiEl technologies to regulate the diagnOSiS‘ahd‘Change'Of'ihterper- sonal, group,iand interggpup9behav10rs, especially those behaviors Félatedl9g org9nizational decision-making, communicatiOn and 919nnihg.‘ 00, when effective, results in an organization which has processes, norms, procedures, and member Skills required for con- tinuous adaptation and thereby continuous optimal fulfillment of its goals. It becomes "a system or framework within which con- tinuogszinnovation, renewal, and rebirth can occur" (Gardner, 1962a . The kind of change we seek is growth and development of the school system--not growth in size, but growth toward increased problem-solving effectiveness, greater potential for action, and greater capacity for adaptation and change.3 In the words of Palola and Padget,4 among others, planned change Should lead to a self-renewing organization, one which according to Warren Bennis, has a: flexible and adaptive structure, utilization of member talents, clear and agreed upon goals, norms of openness, trust and cooperation, interdependence, high intrinsic rewards, and transactional controls, i.e., members of the unit should have a high degree of autonomy andsa high degree of participation making key decisions. [Sic] lHarvey A. Hornstein et 91,, eds., Social Intervention (New York: The Free Press, 1971), p. 343. 2Paul C. Buchanan, "The Concept of Organizational Development, or Self-Renewal, as a Form of Planned Change," Conce99s for Social Change, ed. Goodwin Watson (Washington, D. C.: National Training Laboratories for the C00perative Project for Educational Development, 1967), p. 2. 3Miles and Lake, 99, 919., p. 82. 4Ernest G. Palola and William Padget, Planning_for Self-Renewal (Berkeley, California: Center for Research and Development in Higher Education, University of California, 1971). 5Warren G. Bennis and Philip E. Slater, The Temporary Society (U.S.A.: Harper and Row, Publishers, 1968), p. 105. 70 Thus, Bennis, Slater and other theorists in the P-S tradition point to the characteristics of the open organization described earlier in the present chapter as being the aim of organizational deve10pment since these traits of Openness and participation enable a system to satisfy its member needs and achieve its goals. The steps involved in establishing a self-renewing or problem-solving system reveal a much more thorough-going change process, than simply assuring the use of valid scientific knowledge (RD&D) or helping a system adopt a single innovation (S-I). Ronald Havelock summarizes the most typical steps in the third row of Figure 2-1. Theorists in the problem-solver school are more client oriented than those in either the R030 or 5-1 traditions. It is not that the latter two sets of theorists are unconcerned with the users of innova- tions; it is simply that whereas they normally view the change process as a means for better utilization of knowledge (R080) or the diffusion of new practices (S-I), the problem solvers view the process as a means of enabling systems to cope with a rapidly changing society. AS a result of these different emphases, the language of the problem-solvers is more normative than that of the other two traditions and frequently is written as advice to change agents rather than aS an analysis of a change event.1 1For example of normative writing see: Rensis Likert, "The Nature of Highly Effective Groups," Organizations and Human Behavior, eds., garver and Seigiovonni (New York: McGraw-Hill Book Co., 1969), pp. 56-367. 71 As in the S-I model of Rogers and the writings of Ronald Havelock, the change agent is an essential ingredient in the problem-solving approach. Those functions for change agents listed under Rogers' theory are fully applicable with greater emphasis on client orientation and the necessity of collaborative planning. The change agent is a "facilitator, helper, objective observer and specialist in how to diagnose needs, how to identify resourdes, and how to retrieve from expert sources."1 To review, he works in a collaborative fashion with the client system and develops a need for change, establishes a change relationship, diagnoses the system's problems, examines alternative solutions, establishes goals and intentions for action, transforms intentions into actual change efforts, stabilizes the change and achieves a terminal relationship. Goodwin Watson has worked out an even more detailed process for planned organizational change. Like the guidelines for change agents offered by Rogers and Lippitt 99_91,, Watson's process is based on Kurt Lewin's description of change in three major stageS--unfreezing of present behaviors, patterns and norms, moving to new centers of equilib- rium and refreezing the new patterns of behavior and structure.2 Watson's process entails members of a system sensing the need for change, screening out the less important problems and giving higher priority to more important ones, doing a thorough diagnosis of the 1Havelock 9991,, 99. 919., p. 7-6. 2Edgar H. Schein, "The Mechanisms of Change," The Planning of Change, eds., W. Bennis, K. Benne, R. Chin (New York: Holt, Rinehart and Winston, Inc., 1969), p. 98. 72 present state of the system and its members, inventing alternative remedies, weighing the various suggestions that emerge, identifying a course of action, introducing the innovation, incorporating the change into the on-going system, evaluating the innovation's effects, and finally modifying the innovation to make it more effective for the system in question.1 Others (e.g., see Miles and Lake,2 Jung and Lippitt3) have developed problem-solving strategies Similar to that of Watson with the same ultimate aim of creating self-renewing systems. To achieve this aim, a change agent and/or an organization have a number of tech- niques at their disposal which have been created by theorists in the R080 and problem-solver traditions. Robert Chin and Kenneth Benne break these techniques down into three major categories: rational-empirical, normative-re-educative and power-coercive.4 The rational-empirical techniques rest on the assumption that "men will follow their rational self-interest once this . .5 1S revealed to them.’ As would be expected, the R080 theorists rely 1Watson, "Toward a Conceptual Architecture of a Self-Renewing School System," pp. 110-115. 2Miles and Lake, 99, 919, 3C. C. Jung, Robert Fox, Ronald Lippitt," An Orientation and Strategy for Working on Problems of Change in School Systems," Chan e in School Systems, ed. Goodwin Watson (Washington, D. C.: National Training Laboratories for the Cooperative Project for Educational Development, 1967), pp. 68-88. 4Robert Chin and Kenneth D. Benne, "General Strategies for Effecting Changes in Human Systems," Ihe Planningnof Change, eds., W. Bennis, K. Benne, R. Chin (New York: HoltlZRinehart and Winston, 1969), pp. 32-59. 51bid., p. 34. 73 almost exclusively on these types of strategies which include making information about innovations available, providing demonstrations of the new practice in believable settings (Brickell) and providing link- ing agents between research systems and user systems (Havelock). Normative-re-educative strategies are primarily the contribution of problem-solver theorists and practitioners and rest on assumptions different from those which support rational-empirical techniques. Not that the rationality of man is denied, for it is obvious that rational- empirical techniques are perfectly adequate at times. Nonetheless, men are also "guided in their actions by socially funded and communicated meanings, norms and institutions, in brief by a normative culture."] Hence, as Benne and Chin point out, Changes in patterns of action or practice are, therefore, changes, not alone in the rational informational equipment of men, but at the personal level, in habits and values as well and, at the sociocultural level, changes or alterations in normative structures and in institutionalized roles and relationships, as well as 1n cogn1t1ve and perceptual or1entat10ns. Thus, of all the persons writing about change and innovation theory today, it is the problem-solver theorists who have called attention to the importance of examining all the elements of formal organizations as well as the way in which they interact before determining which innova- tion is appropriate, how best to introduce it and how best to support it. Lewin termed such an examination a force field analysis3 and Watson — 116111.. p. 43. 21bid. 3Kenneth D. Benne and Max Birnbaum, "Principles of Changing," 199_ Plannin of Chan e, eds. W. Bennis, K. Benne, R. Chin (New York: Holt, Rinehart and W1nston, 1969), pp. 328-329. 74 includes it early in his previously outlined strategy as, "doing a thorough diagnosis of the present state of the system and its members." (See above.) Normative-re-educative strategies include: (1) modifying the existing structure of an organization through the use of temporary, single-issue task forces which can define and solve a problem and then - dissolve so that member expertise can be used again in different groups for different problems;] (2) using individual counseling so that an individual may gain "new insights, overcome his insecurities, [experi- ence] his world as a less threatening place . . . and [therefore] be able to relate to his fellows more effectively";2 (3) fostering the growth of groups through group counseling3 and the use of sensitivity training.4 These two strategies are designed to improve interpersonal conlnunication within organizations and sensitize persons to the impact of group norms, shared values and behavioral patterns; (4) the system- atic use of feedback on operations and group discussion. As Matthew Miles 99 91, explain the process: 1Matthew Miles, "On Temporary Systems," Innovation in Education, ed. M. Miles (Bureau of Publications, Teachers College, Columbia Univer- sity, 1964), pp. 437-490. 2Katz and Kahn, 99, 919,, p. 394. 3Elliot Jaques, The Changjng_§ulture of aefactory‘(New York: Holt, Rinehart and Winston, Inc., 1952), as cited in The P1anning_9f Change, p. 46. 4Warren Bennis and Edgar H. Schein, "Principles and Strategies in the Use of Laboratory Training for Improving Social Systems," The Ejanning of Change, eds. W. Bennis, K. Benne, R. Chin (New YorRTT'Holt, Rinehart and Winston, 1969), pp. 335-356. 75 Survey feedback is a process in which outside staff and members of the organization collaboratively gather, analyze, and interpret data that deal with various aspects of the organiza- tion's functioning and its members' work lives, and using the data as a base, begin to correctively alter the organizational struc- ture and the members' work relationships. (5) retraining individuals so that they can learn the new behaviors and values associated with either an innovation or the new goals of an organization. For example, persons writing about planned change fre- quently call attention to the critical role played by the chief admin- istrator(s) in facilitating the change efforts. The administrator may promote--or prevent--innovation. He cannot stand aside or be ignored. He is powerful not because he has a monopoly on imagination, creativity, or interest in change--the opposite is common--but gimply because he has the authority to precipitate a decision. The manner in which an administrator "precipitates a decision" and the sUpport he gives the decision can be critical elements in organiza- tional change, and retraining efforts are sometimes directed at persons in leadership roles to sensitize them to the influence of participation in decision-making and organizational structure on organizational behavior, (6) modification of a major structural variable such as the authority structure to affect behavior and norms of the members.3 1Matthew Miles at Ll., "The Consequence of Survey Feedback: Theory and Evaluation," The_ Planning of Change, eds. W. Bennis, K. Benne, R. Chin (New York: Holt, Rinehart and Winston, 1969), pp. 457- 478. 2Brickell, 99,0 cit, 503; see also, Litwak, 99, 919,. Pp. 361- 362, and Griffit 99, cit., pp. 433- 434. 3Katz and Kahn,o 99, cit., pp. 425-433. The authors discuss the Morse-Reimer experiment in which the authority structure was decentral- ized to increase the job satisfaction and productivity of employees. It increased the former, but the researchers could not obtain a good measure of productivity. 76 As is clear, the six techniques listed are of two general types, those which affect members directly and those which modify the struc- ture of the organization.1 The most effective organizational change plans use both techniques since they can be mutually supportive. The techniques also have three common elements: the presence of a change agent, the use of group discussion and/or extensive participation by group members, and the gaining of support of both formal and opinion leaders. In essence, the techniques, when used in concert, try to assure as broad a base of support for the new practice as is possible. Benne and Chin name their last category of techniques, "power- coercive," and this set of approaches assumes that for a given situa- tion, the persons must be forced to change. For the purposes of the present study, these techniques are less relevant than the rational and normative-re-educative.2 They include such examples as strikes, work stoppages, court orders, the withdrawing of financial support and passive resistance and usually involve the use of political, economic or moral sanctions. The problem-solver perspective has several strengths: It stresses that both human and structural variables must be manipulated if 1Dennis P. Slevin, "The Innovation Boundary: A Specific Model and Some Empirical Results," Administrative Science Quarterly, vol. lb, no. 4 (lZ/7l), p. 515, uses the same division between people and structure techniques of change. 2n.b. This is not to say they are not powerful and relevant when considering strategies for change. Indeed, the power distribution in Some organizations may be such that only a power confrontation will Create change. Nonetheless, the interest of the present study is adop- tIon and implementation by an organization willing to change. Therefore, rational and normative-re-educative techniques are more relevant. 77 organizational development is to occur (rational persuasion alone may not be a powerful enough force to modify organizational behavior); it itemizes specific strategies which may be helpful in introducing and maintaining an organizational innovation; it makes readily apparent that the process of organizational innovation is necessarily long and difficult requiring both behavioral and structural modification. Nevertheless, there are also weaknesses. Although it is evident that some strategies are based on empirical findings (e.g., the impact of peer pressure), models like Watson's have not been tested empirically, nor has the differential effectiveness of the various strategies been tested. The perspective tends to be too normative. For instance, it seems to push collaboration and group participation regardless of exist- ing norms and structure of an organization. The writers' concern for personal growth would seem to prevent them from offering much aid to more authoritarian organizations. Finally, from the perspective of the present study, the problem-solver strategy is too global in scope. The strategists seek the creation of a new organizational culture rather than merely seeking how to facilitate the adoption and implementation of a single innovation. A Model of Organizational Innovation Thus far, four theoretical traditions have been reviewed: l) The literature on complex organizational behavior for the purpose of itemizing the various elements which shape all organizational behavior including planned change; 2) the Social Interaction literature which 78 specifies the four principal elements of the change process (the inno— vation, communication channels, members of a social system and time) as well as analyzing the adoption process; 3) the Research, Develop- ment and Diffusion tradition which outlines in detail the various stages in the dissemination and utilization process--research, develop- ment, diffusion and adoption-~and stresses that change should be a rational and planned process; and 4) the Problem-Solver approach to change which, recognizing the impact of organizational variables, stresses a systems' approach to change, calling for change in the be- havior and attitudes of system members as well as change in the organizational structure itself. This approach is ultimately more concerned with creating self-renewing organizations than with introduc- ing specific innovations to individual systems. For the purposes of the present study--analyzing the adoption and implementation of an innovation within a single complex organization-- the social interaction perspective (Everett Rogers' Communication of Innovations Theory) is most useful as a structure upon which to build with the help of insights from the other three theoretical traditions. This perspective highlights the critical elements of the change process (innovation, communication channels, a social system, time), omits reference to research and development which are not of interest to the present study and is easily adapted to apply to complex organizational change. To modify Roger's model for the purposes of the present work, the "Diffusion Process" outlined in Chapter I is renamed the "Organiza- tional Innovation Process,“ and includes the following elements: 79 1) an innovation, 2) advocate(s) of change, 3) a complex organization in its environment, 4) a communication network, 5) time. Thus, if organizational innovation is successful, it is a process of planned change in an organizational setting during which the system members move from initial knowledge of the innovation through the stages of persuasion, adoption, implementation and incorporation.1 Note that the members of the organization must go through the five stages regard- less of whether the innovation decision is authoritative or collective in Rogers' terminology. Members may be told to adopt, implement and incorporate or they may participate in the innovation decision-making process. In either case, if organizational innovation is completely successful, they will either behave in accordance with the innovation decision or leave the organization. If some members do not adhere to the innovation decision, then implementation and incorporation must be considered less than complete. The first element in the above process, the innovation, is identi- cal to that mentioned in the Rogers and Shoemaker model, i.e., one defines an innovation as an idea, practice, or item perceived as new by an organization which may be classified according to its relative advan- tage, compatibility, complexity, observability and trialability. As the process unfolds it is quite likely that the innovation will be 1Please note that the process is successful and the author has changed the decision and confirmation stages of Rogers' paradigm to ado tion, implementation and incor ration. This chan e was made to emp asize the last two stages of t e process, one of t e primary inter- ests of the present study. 80 adapted to achieve an appropriate fit with the particular organization. Rarely does successful organizational innovation occur without adapta- tion of the innovation itself. The second element, advocate(s) for change, conforms to the Havelock typology outlined earlier in this chapter. There may be one advocate or several with different skills facilitating adoption and implementation at different stages of the process. 'A complex organization in its environment' replaces 'members of a social system' in the Rogers and Shoemaker model. Thus an organiza- tion's characteristics as an open system, its role structure, task demands, shared values, system norms, rules, authority system, members, and the manner in which it copes with the inherent dichotomy between member needs and the requirements of production must all be taken into account when examining the adoption and implementation process. The phrase 'a communication network' replaced 'channels' since it is more inclusive. An organization not only receives information through channels (which the coding process may affect) but also processes it through vertical and horizontal communication networks. The relative openness of these networks may well affect the adoption and implementa- tion of innovations. Finally, time is included as an explicit variable as it was in the Rogers and Shoemaker model to stress that organizational innovation is a process involving several stages-~knowledge, persuasion, adoption, imple- mentation and incorporation. As mentioned in the Chapter I glossary of terms, these stages parallel the Lewin stages of unfreezing, moving and 8] refreezing. Implementation and incorporation were added as stages to emphasize the importance of "refreezing" to the change process. Thus, when the innovation is major, the process of organizational innovation is long and involved and most frequently involves modification of both the innovation and the organization. The work of Guba and Clark in the RD&D tradition and the writings of Goodwin Watson were very influential in the decision to add imple- mentation and incorporation as explicit stages to Rogers' paradigm. However, it was the research of Gross §t_al, on the implementation of innovations which made the need most apparent. In their work, Gross et_gl, analyzed the degree of implementation of a catalytic role model for teaching1 which had been adopted by an elementary school. In their analysis six months after adoption, the authors discovered that in spite of low initial resistance to the innovation and a favorable atti- tude toward change in general, the teachers "devoted only a small proportion of their time [about 12%] to efforts to perform in accord with the new role model, and their performance when they made such efforts was of low quality."2 Gross gt_al, lay the blame for the failure to implement at the feet of management. The director's change strategy was deficient in two respects: 1Gross §t_gl,, p. ll. A complete definition is available on p. ll. Essentially, the catalytic role model required teachers to become facili- tators of individual learning rather than conveyors of knowledge. ZGross £321.. 92. 91., p. 119. 82 (1) it failed to take account of difficulties to which teachers would probably be exposed when they attempted to imple- ment the innovation, and (2) it contained no provisions for feedback mechanisms to identify and cope with barriers and 1 problems arisingduring the period of attempted implementation. More specifically, management: (1) did not provide a clear picture of the new role demands associated with the innovation; (2) did not make organizational arrangements compatible with the innovation; (3) did not provide the retraining experiences necessary to enable teachers to per- form the new role; (4) did not provide the necessary resources (instruc- tional materials) to implement the innovation and (5) did not provide either the moral support or rewards in order to maintain the teachers' motivation and willingness to implement the innovation. Rather than supporting the implementation process, the director deemed the following sufficient: (I) explain the philosophy and objectives of the innovation to teachers through several written documents; (2) give teachers maximum freedom to carry it out; and (3) delegate responsibility to an administrative subordinate (the assistant director) to see that the innovation is implemented.2 It is enlightening to examine the Gross gt_al, findings in light Of the organizational innovation model suggested above and the preceding literature review. The fact that the catalytic role model was complex, incompatible with organizational arrangements, and called for dramatic behavioral change on the part of students and teachers obviously influ- enced the degree of implementation. These factors did not affect 1Ib1d., p. 201. 2Gross gt_gl,, 92, 213,, p. l9l. 83 adoption since it was an authority innovative-decision, and the director either ignored or was not aware of these traits associated with the catalytic role model. The fact that the communication network did not provide adequate feedback on how the innovation was working inhibited implementation, and the director's use of written memoranda obviously proved inadequate in clarifying the innovation. Several organizational variables seemed to affect implementation: without a retraining program, the norms of teacher control and teacher as the source of information remained dominant; the structures of tight class schedules, age-grouping, and report cards were incompatible with the concepts of openness and progress on the basis of individual performance: the leadership was non- existent during implementation as has been pointed out, and none of the teachers had enough system sense to know how to attack the emerging problems. There was no advocate of change after initial adoption; the director was absent and the assistant director to whom he assigned implementation did not support the innovation. No one within the system conceived of organizational innovation as a process occurring over time. The director assumed adoption and implementation were synonymous, and teachers felt resentment at his lack of support. Had anyone internally been aware of the many variables involved in planned change, they might well have been able to reduce the level of frustration by simply point- ing to the difficulty of the process and giving reassurance to the participants (i.e., looking at the process as one with many stages which take time to unfold and effort to assure). Will the proposed model of organizational innovation prove useful to persons seeking either to understand the change process or to manage 84 change in complex organizations? Robert Chin suggests five questions as guidelines for examining the worth of a conceptual model.1 1) "[D]oes the model account for the stability and continuity in the events studied at the same time that it accounts for change in them?" The fact that the innovation is being introduced to a complex organi- zation permits one to answer this question positively. Open system theory includes the principal of homeostasis, which gives an organiza- tion stability, and takes into account the tension between the human and production needs of a system which guarantees a certain degree of conflict and change. In addition, whereas rules and regulations serve the needs of existing goals, an advocate for change can push for any number of system modifications. In brief, the model points to several sources of stability and change. 2) "[WJhere does the model locate the 'source' of change?" Change may result from a system imbalance (e.g., members dissatisfied with the rigidity of the authority hierarchy), from the external environment (e.g., the saturation of the teaching job market) or from an advocate for change either inside or outside the organization. In the case of the organizational innovation model, there is always an advocate for change involved either at or immediately after the knowledge stage, since the process is an intentional one. 3) "[WJhat does the model assume about how goals and directions are determined?" It makes no assumptions since the answer is determined by the organizational structure. In a successful authoritarian 1Robert Chin, pp. 913., p. 309. 85 organization, the innovation might well be most appropriately imple- mented in an authoritarian manner. HoweVer, given the fact that most organizations today are concerned about the personal growth of their employees and do encourage some degree of participation in the decision-making process, most planned organizational innovation efforts will proceed with those members who will be implementing the innovation, participating in and influencing the innovation-decision. I 4) "[Djoes the model provide the change agent with levers or handles for affecting the direction, tempo and quality of these processes of change?" The model points to many levers, e.g., the characteristics of the innovation, the norms of the members, the skills of the members, the structure of the organization--all clearly influence the organiza— tional innovation process. What is not generally known, however, is the differential influence of the variables.1 If, for instance, an innovation is incompatible with system norms, does one change the inno- vation or the group norms? Changing the innovation may be simpler, but will it then be worth introducing? Will changing the norms have a deleterious impact elsewhere in the system? These are questions that must be considered in light of the particular situation since there are no generalizations which apply. The model very clearly provides levers, but which ones should be pulled remains a question for the change advo- cates to answer in context. 1There is some evidence concerning the impactcn’variables, but it is limited. For example, interpersonal channels seem more influential than mass media ones at the persuasion stage, and empathetic change agents are more effective than those who are not empathetic. 86 5) "How does the model 'place' the change agent in the scheme of things?" The advocates for change are centrally located and have the range of responsibilities outlined by Ronald Havelock (conveyor, con- sultant, trainer, administrator, opinion leader, innovator, and defender). As is evident from Havelock's typology, what the advocates do depends on the stage of the organizational innovation process. Summary In the first part of the chapter, the characteristics of complex or formal organizations and the manner in which they interrelate are re- viewed. To review, the organizational context is, indeed, complex. As an open system, it is marked by the energic cycle of input, through-put and output, the coding of information, the processing of feedback on its own functioning, negative entropy to resist running down, homeostasis to maintain a dynamic equilibrium, differentation and elaboration, and equifinality, the tendency to reach the same final state from differing conditions. In addition, an organization is a structure of roles main- tained by task demands, shared values, system norms and rules enforced by an authority system. It is a socio-technical system with specific functional imperatives--production, maintenance, adaptation and manage- ment--in order to cope with inherent dichotomy between individual human needs and the production requirements. It consists of human beings with their own needs, values, attitudes, backgrounds and preferences who frequently establish an informal group to fulfill their desires for self-control and self-expression. Finally, it has communication networks for processing information internally. 87 When the above characteristics work in concert, they enable an organization to maintain a dynamic equilibrium (homeostasis) marked by controlled change. Generally speaking, the more open a system along the dimensions itemized above, the more often it will adopt relevant innovations. Nevertheless, the differential impact of these organiza- tional elements under varying conditions is simply not yet known. New empirical research is needed to illustrate whether, for example, decentralizing the authority system has as much impact on increasing innovation as introducing longer and more frequent vacation periods. After the review of organizational theory, three separate research traditions concerned with the change process were reviewed. They are: l) The Social Interaction Perspective (S-I), 2) The Research, Develop- ment and Diffusion Perspective (RD&D), and 3) The Problem-Solver (P-S) Perspective. As can be seen from Figure 2-l, they all deal with dif- ferent aspects of what Ronald Havelock terms the dissemination-utiliza- tion process, i.e., the change process in its entirety, from the generation of new knowledge, a new practice or new technology to the incorporation of the innovation in an on-going organization or system. The S-I model (termed Communication of Innovation's Theory in Chapter I) emphasizes and describes the diffusion stage of the change process and concentrates particularly on the rate at which individual rather than social systems adopt particular innovations. As the Havelock figure makes evident, researchers and theorists in this tradi- tion do not discuss the invention, research and development stages of the change process but rather concern themselves with conditions and 88 characteristics which lead to diffusion and adoption. In their analysis they most frequently examine the traits of the innovation, the channels of communication used, the social system and its members, and the stage through which an adopter of an innovation goes. These stages are knowledge, persuasion, decision and confirmation. In most instances of successful innovation opinion leaders and change agents facilitate the innovation-decision process. The Research, Development and Diffusion (RD&D) perspective covers the entire change process including, according to Egon Guba, research (investigation, conceptualization, evaluation), development (invention, design, field testing), diffusion (explanation, demonstration), and adoption (trial, installation, institutionalization). Researchers and theorists in this tradition argue for the conception of change as a rational and planned process, and have as their principal goal the more frequent and improved use of scientific knowledge. They wish to narrow the gap between theorticians and practitioners. Finally, it is noted that the various subsystems doing research, development, diffusion and adoption have different organizational characteristics. Therefore, linkage agents are needed if new theories are to be adopted and imple- mented. Persons writing from the Problem-Solver Perspective (P-S) are more interested in instilling a problem-solving approach in organization to facilitate organizational self-renewal than in describing how a single innovation is adapted and implemented. This bias for what has come to be known as 'organizational development' or 'OD' gives the writing a 89 normative flavor, and it is often advice for changing rather than research on the change process. Nevertheless, these authors do point to the influence of organizational structure and members on the change process, and show that organizational innovation entails more than merely persuading a system to adopt a new idea or practice. The four theoretical traditions are used to devise a model of organizational innovation. Rogers and Shoemaker's diffusion process is renamed the "Organizational Innovation Process," and it includes the following elements: l) an innovation, 2) advocate(s) of change, 3) a complex organization in its environment, 4) a communication net- work, 5) time. Thus, if organizational innovation is successful, it is a process of planned change in an organizational setting during which the system members move from initial awareness of the innovation through the stages of knowledge, persuasion, adoption, implementation and incorporation. The elements are defined and the model discussed in relation to a previous study of implementing an innovation in a complex organization. CHAPTER III METHODOLOGY AND DESIGN Introduction The case study of written evaluation at Justin Morrill College covers the period from initial discussion in Winter Term, l969 through its state of utilization in Winter Term, l973. The case study attempts to answer the question--to what degree is written evaluation implemented in JMC as of Winter,1973? Several methods of research are used in order to assure a thorough accounting of the implementation process and to lessen the possibility of error. As Webb §£_al, point out: The most persuasive evidence comes through a triangulation of measurement processes. If a proposition can survive the onslaught of a series of imperfect measures, with all their irrelevant error, confidence should be placed in it.1 Measures and Instruments The case study is developed through the use of four main research methods: 1) participant-observation; 2) nonreactive, unobtrusive measures; 3) attitude surveys; 4) an analysis of the degree of comple- tion of the evaluation forms upon which faculty and students assess student course performance (see Appendix 1). 1Eugene J. Webb gt_gl,, Unobtrusive Measures: Nonreactive Research in the Social Sciences (Chicago: ‘Rand McNally & Co., 1965), p. 3. 90 91 Participant-observation: The author has been an instructor in Justin Morrill College since the Fall of 1968. He served on the College Advisory Council which advised the Dean to forward a proposal for using written evaluation to the Michigan State University Curriculum Committee in Spring Term, 1970. During Fall Term, 1971, he became the Dean's staff assistant and was assigned the task of surveying faculty and student opinion toward the written evaluation system. In addition, he conducted numerous informal interviews with students and faculty con- cerning their use of and response to written evaluation and directed all of the efforts of improving the written evaluation system after Fall Term, 1971. Nonreactive, unobtrusive measures: Records such as minutes, reports and memoranda have the advantage of being unobtrusive measures of data, i.e., they "do not require the cooperation of a respondent and [they] do not themselves contaminate the response."1 In addition, they are usually accessible and inexpensive. Never- theless, as Webb _th_1, observe, unobtrusive measures are not without their faults since archives are frequently subject to "selective deposit and selective survival."2 In the case of the present research, the fact that the author kept records himself, searched the files of several other persons and was a participant-observer alleviates the problem of selective deposit and survival to some degree. 1Ibid.. p. 2. 21bid., p. 85. 92 Research methods one and two above provide most of the historical information regarding the adoption of written evaluation as well as background information on faculty, student and administrative charac- teristics and the organizational setting. The specific documents examined include: a) minutes of the College Advisory Council, the Program Administration and Coordination Committee, the College Curricu- lum Committee and the College Educational Policies Committee from Fall, 1968-Spring, 1973; b) various memoranda from Fall, 1968-Spring, 1973; c) official College documents relating to both Justin Morrill's goals and the written evaluation system; d) all research done on the college, its students and its faculty from Fall, l965-Spring, 1973; e) issues of the §hggt, the college newsletter, relating to the innovation from Fall, l968-Spring, 1973. Four attitudinal surveys--two each for JMC faculty and JMC students: During Winter Term, 1970, faculty and student opinion toward the written evaluation system was sampled. The faculty return was low (16/45); the student sample was 50 with 49 usable questionnaires. However, records were not kept on how the student sample was drawn. In the Fall of 1971, the two surveys were repeated deriving specific categories of response from frequent responses to the open-ended questions on the 1970 surveys (see Appendix 2 for the two 1971 questionnaires and results). As in 1970, the 1971 faculty study was a population survey, and 40 of the 44 full-time JMC faculty responded. No part-time faculty com- pleted the questionnaire. A sample of 100 students was selected, 82 of whom responded. The sample was stratified by year and sex and randomized 93 within the appropriate categories of each variable. The same proportion of freshmen, sophomores, juniors and seniors were in the sample as were in the Fall, 1971 JMC student population. First term freshmen and freshmen with less than three courses under written evaluation were excluded from the sample since they were not yet familiar with the system. The same proportion of men and women as existed in the 1971 JMC student population was included in the sample. Table 3.1. The breakdown of the 100 student sample for the 1971 atti- tude survey. L021. Dale—1 m non-first term freshmen 11% 63 22 (4) 41 (7) sophomores 40% 237 83 (14) 154 (26) juniors 31% 181 79 (13) 102 (18) seniors __1_8_g 1_l__2_ 36 (6) 76 12 100% 593 220 (37) 373 (63) Freshmen in the college who were in their second or third terms but who did not have at least 10 credits or 3 courses under the P-N system were not included in the sample. Also excluded were any students who had recently changed their major and new students who were transfers from another college or university. The information gathered from these attitudinal surveys will be used to examine trends only, and not analyzed statistically. Degree of completion analysis: A systematic analysis of the degree to which both faculty and students completed the written evaluation 94 forms (see Appendices l and 3) will be the primary indicator of the degree of implementation of written evaluation in Justin Morrill College. In addition, the percent of forms on file for the student sample dis- cussed below and the proportion of students reading evaluation forms will serve as indices of implementation. The latter measure comes from the 1971 student attitude survey and from a count of student advising folders used by the students during the first two weeks of Spring Term, 1973. For the evaluation form analysis, a sample of 46-50 forms was chosen for each term the system was in use from Fall, 1970 through Winter, 1973. Summer terms were excluded since few, if any, JMC courses are taught during Summer session. This number represents about 8% of the JMC students who take at least one JMC course in any one term and approximately 3% of the total number of forms completed in any one term. The small sample size per term seems justified in light of the relatively homogeneous populations to which the researcher wants to generalize--JMC students and faculty teaching in JMC (800 and 40 respectively). In addi- tion, the Chi Square of Independence and the Spearman Rank Order Corre— lation, the statistics for analysis, are powerful ones for the data, and muCh of the analysis will be done on the total sample of 389 forms. There is also less chance for a tabular error with a relatively small sample. Finally, taking time (it took approximately seven hours to pull and code each sample of 50) and cost (each form had to be copied) into consideration, it seemed most feasible to limit the sample size to fifty forms per term. 95 The Dean's file of student folders was the source of completed. evaluation forms. The population, however, is forms completed by stu- dents ggg_faculty, and the analysis will be generalized to all JMC faculty and all JMC students. Eighty to eighty-five names were chosen at random from the JMC major list for each term, Fall, 1970-Winter, 1973 in order to obtain a sample of 50. There are two reasons for the large initial number: a) JMC stu- dents take approximately 50% of their courses outside the college. It was quite possible that a JMC student in the original eighty to eighty- five member sample might not have taken a JMC course during the term in question and would not have a form on file. In such a case, his name was deleted from the sample: b) Some students in the eighty to eighty- five member sample might have transferred out of JMC, in which case there would be no folder on file. In such cases, their names were de- leted from the sample. The range of eighty to eighty-five names for each term was determined after pulling the Fall, 1970 sample and dis- covering about 35 missing forms and folders. When no form was available for a student who had taken a JMC course during the particular term, the form was coded to indicate that no items had been completed. This coding decision is considered appropriate since the aim is to generalize to JMC faculty and students at large and the degree to which they complete the forms. If some faculty do not hand in the forms, this lack of completion should be reflected in the sample. Over the eight terms, thirty-four forms were missing and coded as being entirely incomplete. 96 In an attempt to gain internal validity, i.e., to make it unlikely that chance produces any observed differences among the variables, a systematic sampling technique was used. The interval chosen was 01—13 and the random number table1 was entered at line 8, second column of group 6, obtaining number 09. The ninth name on each of the eight JMC major lists (one for each term) was then used to begin selecting the 80-85 member sample for each term. After going to the individual stu— dent's folder, the first form from the appropriate term found in the folder was chosen to avoid any undue bias in form selection. The sample of forms yielded information on both the faculty and students. 0f the 389 students in the sample, 252 or 65% are under- classmen (freshmen and sophomores) whereas 136 or 35% are upperclass- men (juniors and seniors).2 The population figures are lower and upper- classmen are approximately 60% and 40% respectively for the period from Fall 1970 through Winter 1973. During any one term approximately 78% of the JMC students taking JMC courses are underclassmen while 22% are upperclassmen. The relatively high proportion of upperclassmen in the sample is helpful to this particular study, since one of the aims is to determine if system familiarity is an important variable in form com- pletion (see research hypothesis, number 6, below), and in using the Chi Square statistic, one needs at least five observations per cell. 1Sidney J. Armore, Introduction to Statistical Analysis and Infer- ence (New York: John Wiley and Sons, Inc., 1966), pp. 498-499. 2The students were coded either upper or lower class according to the term they took the course for which the form was completed. There is one missing value for the variable, class. 97 Students in the sample have a mean grade point average of 2.99 with a standard deviation of .6.1 The cumulative grade point average as of each student's last term in JMC was used to calculate this sample mean. The mean grade point average of JMC students in univer- sity courses is 3.0 for the period from Fall 1970 through Winter 1973. Thus it appears the sample is representative of the JMC student popula- tion in terms of academic performance. A further indication of this representativeness is the fact that 5.4% of the students in the sample received "No-credit“ in the course for which the evaluation form was completed. 4.4% of JMC students received "No-credit" in all JMC courses during the period from Fall 1970 to Winter 1973.2 Of the 3553 faculty included in the sample, 243 or 69% are full—time and 112 or 31% are part-time.4 Sixty-eight (19%) were in Natural Science, sixty-eight 123/389 persons with 0.0 cumulative grade point averages in the sample were coded as having 3.0 grade point average since in each case the 0.0 indicated the student had not yet taken any university courses. 3.0 was substituted since 3.0 is the mean JMC student grade point average in university courses for the period from Fall 1970 through Winter 1973. 25.4%15 the adjusted frequency and is used since there are 34 miss- ing values for the variable, grade. The grades in the JMC courses were obtained from the evaluation forms, and as explained earlier, 34 of the forms were missing from student folders. The actual sample frequency of "No-credit" is 4.9% including the 34 missing values. The assumption is made that the frequency of "No-credit” in the missing values is no great- er than 5.4%. 3There are 34 missing values for the variable, faculty. There are 34 missing forms and the faculty names were obtained from the evaluation forms. 4Part-time faculty have joint appointments with both JMC and another MSU college. Full-time faculty are hired only in JMC and have at least a half-time appointment. The full-time category also includes a few stu- dent teachers who are enrolled in JMC. 98 (19%) were in Social Science, 111 (31%) were in the Humanities, thirty- nine (11%) were in Languages, forty-seven (13%) were in Inquiry and Expression and twenty-two (6%) were in either Independent Study or Senior Seminar. In the JMC curriculum the proportion of courses taught by part-time faculty varied between Fall Term, 1970 and Winter, 1973, but never went as high as 31%.] During the three year period, part-time faculty offered between 10% and 25% of the sections in JMC in any given term. The relatively high proportion of part-time faculty in the sample is helpful for the present study since one of the aims is to determine if faculty employment status is an important variable in form completion (see research hypothesis, number 3, below). There are no exact figures available concerning the proportion of sections offered in the various knowledge areas in JMC. However, it is clear from the course descriptions, credit hour data and class lists that in any given term more students enroll in the Humanities than in any other single area. Then, in descending order, enrollments are heaviest in Social Science, Natural Science, Language, Inquiry and Expression and Independent Study. 1For the purpose of this study, it is more appropriate to compare the proportion of full and part-time faculty in the sample to the proportion of courses offered by each in the JMC curriculum since a single faculty member may teach one or several courses. In addition, the faculty names in the sample come from the written evaluation forms and are likely to reflect the proportion of forms completed by part and full-time faculty rather than the proportion of part and full-time faculty in the JMC population. 99 All of the above sample data give the research some hope of external validity, i.e., the sample is, indeed, quite representative of the two populations to which the data on form completion may be generalized--the faculty and students of JMC. In addition, there are enough part-time faculty and upperclass students to enable comparisons in degree of com- pletion with their counterparts. The percent of forms handed in by faculty was calculated by deter- mining the number of forms on file for one term's work for a particular student compared with the number of courses he took that term. A mean was then calculated from the individual percentages. A special instrument entitled, "Categories and Questions to Analyze the JMC Written Evaluation Form" (see Appendix 1), was developed to determine the degree to which faculty and students completed the written evaluation forms. Following the lead of Neal Gross gt_gl,] the instru- ment attempts to measure the actual behavior of faculty and students as compared to their attitude toward the innovation. In this manner one can determine how closely attitude and behavior conform to one another and whether either or both support the objectives of written evaluation as outlined in Chapter I. The measuring instrument has three sections, each dealing with one aspect of the written evaluation forms. In section one, the following items are coded: student number, whether the student was an upper or lowerclassman, the knowledge area of the course, the instructor's name, whether the instructor was full or part-time, the term the course was 1Gross gt_al., op, cit. TOO taught, the cumulative grade point average of the student during his last term in attendance at Justin Morrill, and the number of forms on file compared with the number of JMC courses taken by a student for the term in question (recorded as a percentage). In the second and third sections are questions dealing with the degree to which faculty and students completed the form. They were derived from the first two of the three system objectives listed in Chapter I. All are "yes-no" questions, and a yes response by the coder to all questions would reflect 100% completion by both faculty and student. Questions 1-9 deal with sections of the form that the instructor should complete, and questions 10-15 deal with those sec- tions the student should complete. The faculty completion score is the total number of yes responses to questions 1-9, while the student com- pletion score is the number of yes responses to questions 10-15. The two completion scores are indices of the overall degree of completion by faculty and students. As Webb 99_91, point out, such indices can be valuable to make meaningful comparisons across time and social space.1 In this case the author intends to make comparisons among various aca- demic terms. Certain questions represent an attempt to judge the quality of the written evaluation paragraphs done by both faculty and students. Stated differently, these questions are an attempt to quantify quality. If an evaluation statement does not merit a yes response to at least two of the questions 6-9, it is unlikely to be very good. Likewise a lWebb _e__a_1_., 99. cit., p. 6. 101 student evaluation which does not receive a yes response to at least two of the questions 12-15 is probably a poor one. The risk of moving from quantity to quality seems justified in this case since it is unlikely that someone can write a good evaluation while ignoring a student's performance on college and course goals, his work in class, tests and papers and his strengths and weaknesses. TWo persons, the researcher and an assistant, used the instrument to code the 355 individual written evaluation forms as well as the 34 missing forms. To check the reliability of coding, a sample of twenty- five forms was coded separately by each person. Examining the results, the coders determined that they were in agreement on 95% of the items. After clarifying why the small amount of disagreement did exist, the specificity of some questions was increased. The researcher coded 80% of the forms. When the assistant did code, the researcher coded at the same time. Questionable items were discussed, and in the few instances where disagreement occurred (on perhaps 5 out of a possible 5325 items), the researcher's decision was final. Specific classifications during coding were possible, and any future research should use the following guides to the specific questions, 1-15: a) A course description includes more than a general or specific course title, e.g., "Sociology 250A, Organizational Behavior," is not a course description. Nor are reworded course titles, course description, e.g., "Sociology 250A, Organizational Behavior: this course will consider human behavior in an organizational setting," is not a course description either. 102 b) In questions 6 and 12, the evaluations were coded "yes" if they d) included imglicit reference to at least two college goals. The following words--insight, organizational ability, analysis and interpretation--refer to either synthesis or evaluation, two of the college goals. In some instances, instructors mentioned a student's ability "to form his own opinions" without using the word, synthesis, or a student's ability "to think clearly in the face of conflicting views' without using the word, evalua- tion. In both cases, the comments refer implicitly to a college objective. Questions 7 and 13 were coded yes only if the comments went be- yond comments on class discussion, papers, examinations and reading since these categories are covered in questiOns 8 and 14. For example, a comment such as, "he wrote an excellent final paper" does not mention a course objective, but the com- ment "he wrote an excellent final paper analyzing Milton's. poetry in the political context of Seventeenth Century England" mentions two college objectives (analysis/synthesis and quality of writing) and a course objective (relating literature to the political context). To be coded positively in questions 6, 7, 12 or 13, evaluations had either to state explicitly that particular objectives had been met or comment on how well a student performed a particular task or skill. For example, the sentence, "he wrote a final paper" does not comment on a student's writing skill whereas the phrase, "a writer of superb skill and range" does. 103 e) Phrases like, "met all my course objectives" Were coded nega— tively unless the instructor mentioned specific objectives elsewhere on the form. f) Questions 9 and 15 were coded positively only if 99t9_strengths 999_weaknesses were mentioned. The phrase "has no weaknesses" comments implicitly on both strengths and weaknesses. The evaluation form completion analysis, the percent of evaluation forms on file, the number of students who requested profiles, the number of students who read the completed evaluation forms and some results from the 1971 attitude surveys determine the degree to which JMC has implemented written evaluation. The form analysis will be the primary indicator since it provides the most thorough accounting of faculty and student use of the written evaluation system. Design All measurement devices-~participant observation, informal inter- views, unobtrusive and nonreactive records, the attitude surveys and the form analysis-~contribute to developing the case study of the written evaluation in Justin Morrill. Initially, in Chapter IV, the study will describe the climate for educational change in Justin Morrill at the time of adopting written evaluation (external environment, organizational context, the characteristics of the faculty, students and the Dean). Then it will relate the history of the innovation in JMC from initial awareness to the present stage of implementation. Next, the degree of implementation will be established through examination of the 104 two 1971 surveys, the form analysis, the percent of completed written evaluation forms on file, data regarding student profiles and anecdotal data. Finally, in Chapter V, the study will offer specific hypotheses concerning the implementation of organizational innovations using the organizational innovation model developed in Chapter II, the research reviewed in Chapter II, and the case history of written evaluation dis- cussed in Chapter IV. Thus, the design is predominantly descriptive although some generali- zations concerning implementing innovations in complex organizations will be made on the basis of the case study. The design is also hypothesis generating rather than hypothesis testing with the exception of the hypotheses tested in the evaluation form analysis. The case study format seems appropriate for a number of reasons: 1) The study will generate plausible explanations for the degree imple- mentation, 999_test hypotheses concerning implementing innovations in general. The case study seems ideally suited for such an aim;1 2) Both adoption and implementation of innovations are complex processes which occur over time. The case study approach permits one to view the processes in total rather than having to examine isolated variables;2 3) It is relatively inexpensive, relying heavily on records easily available; 4) It permits one to examine both attitudes and the degree of 1Julian L. Simon, Basic Research Methods in Social Science (New York: Random House, 1969), p. 278. 2Several authors (Havelock, Miles, Bennis) have noted that organiza— tional change must be viewed as a complex process occurring over time. Havelock points out that there are few case studies which permit one to observe the process as a whole. 105 implementation at different points in time for the sake of comparison; 5) One can use a natural as opposed to an artificial setting: 6) Many of the data gathering procedures can be unobtrusive; 7) One can use a variety of data gathering procedures which enable cross-checking the validity of information; 8) It permits continued close rapport between the observer and the observed.1 Hypotheses The testable hypotheses relate only to the form analysis to be done in Chapter IV and not necessarily to the general thesis as stated in Chapter I. The generalized null hypothesis is: G.H. : There is no difference in the degree of completion among the written evaluation forms. Since both students and faculty complete sections of the form, two sub-null hypotheses were developed from the generalized null. These hypotheses provide a means to develop systematically a series of test- able or research hypotheses. H 1: All faculty complete the written evaluation forms to the 0 same degree. H 2: All students complete the written evaluation forms to the 0 same degree. During some academic terms, more extensive efforts were made to assure that faculty and students completed the written evaluation forms. Therefore the following research hypotheses were derived: 'Items 3-8 adopted from a similar 1151; in Neal Gross 99 11;, _p_. 91_t_., pp. 42-43. 106 There is a relationship between the degree of faculty com— pletion of the evaluation forms and the term in which the evaluation forms are used. S : There is a relationship between the degree of student com- pletion of the evaluation forms and the term in which the evaluation forms are used. Since some faculty teach both in JMC and in another college and are therefore less familiar with the written evaluation system than fulltime faculty, it was felt that there would be a relationship between faculty employment status and both the faculty and student completion scores. The following research hypotheses reflect this prediction: S3: There is a relationship between faculty employment status and the degree to which faculty complete the written evalu— ation forms. S4: There is a relationship between faculty employment status and the degree to which students complete the written evaluation forms. It was also decided to examine the impact of faculty employment status over time on the degree to which faculty complete the forms. Hence, the following research hypothesis was formed: S o 5. There is a relationship between the degree to which full and parttime faculty complete the written evaluation forms and the term the faculty complete them. Faculty in Natural Science, Inquiry and Expression, Language, and Field Study have some uniformity in instruction in their various pro- grams and do more planning as a staff than do faculty in the Social .Sciences and Humanities. Faculty in the latter fields plan their courses more independently. Due to the high degree of interaction of faculty in the first four knowledge areas above and the amount of group planning, 107 it was felt that faculty in these knowledge areas would have reached some agreement on how best to use the written evaluation forms and hence complete them more thoroughly and encourage students to complete them more thoroughly than faculty in the Social Sciences and the Humanities. The following research hypotheses were developed to test this predic- tion: S6: There is a relationship between the knowledge area in which faculty teach and the degree to which they complete the written evaluation forms. S7: There is a relationship between the knowledge area in which faculty teach and the degree to which students complete the written evaluation forms. It was felt that students would improve in their use of the forms as they became more accustomed to the process of self-evaluation. Thus upperclass students should complete the evaluation form more thoroughly than do students less familiar with the system and by so doing encourage instructors to do more thorough evaluations. The following research hypotheses reflect this prediction: S8: There is a relationship between student status and the degree to which students complete the written evaluation forms. 5 o 9. There is a relationship between student status and the degree to which faculty complete the written evaluation forms. It was also decided to examine the impact of student status over time on the level of student completion scores. Hence the following research hypothesis was formed: S There is a relationship between the degree to which upper and lowerclassmen complete the written evaluation forms and the term the students complete them. 10‘ Since most educators feel that better students are more likely to take evaluation seriously, the following hypothesis was generated: 108 811: There is a positive relationship between a student's grade point average and the degree to which he completes the written evaluation form. Since the instructors are responsible for encouraging students to evaluate their own performance, it was anticipated that faculty who complete forms thoroughly will have a greater proportion of students who complete the forms thoroughly. The following research hypothesis reflects this prediction: 512: There is a relationship between the degree to which faculty complete the written evaluation forms and the degree to which students complete the written evaluation forms. Analysis Since the instrument should determine the degree of completion of the individual items on the evaluation form and the overall degree of completion, the analysis will include frequency counts, means, and standard deviations as well as the statistical tests for the twelve hypotheses just itemized. To test hypotheses 1-10 the Chi Square test of independence will be used. It is powerful enough to work with the nominal and non- parametric data of the present research. Further, it is the appropriate statistic to test the lack of statistical association of two variables SO‘long as one does not seek the degree of association between the two variables.1 * 1Norman H. Nie et al., Statistical Packaggfor the Social Sciences (New York: McGraw-HfiTl-EookTCo.,—1970), p. 2 5 109 Hypotheses 11 and 12 will be tested through a non-parametric correlation analysis since the researcher is measuring the linear rela- tionship between two variables in each case. The summary statistic will be the Spearman rank-order correlation coefficient.1 Since the data for all hypotheses are non-parametric, the assumption of normality does not need to be made. Summary The case study of written evaluation at Justin Morrill College will depend on four basic research methods: 1) Participant-observation; 2) Nonreactive, unobtrusive measures; 3) Attitude surveys of faculty and students; 4) An analysis of the degree of completion of the written evaluation forms that faculty and students use to assess student course performance. Together, it is hoped that they will permit the develop- ment of an accurate picture of the use of written evaluation at JMC and give some clues for the present degree of implementation. In its en- tirety, the case study should give rise to some generalizations regarding implementing innovations in complex organizations. The form completion analysis is the most complex of the four re- search techniques. A sample of 389 completed written evaluation forms was pulled from student folders, approximately 50 from each of the eight terms in which the innovation has been in effect. All of the forms were analyzed using a series of questions to determine the degree to which both faculty and students completed the evaluations. For the purpose of lIbidu p. 4. 110 further analysis, one generalized null hypothesis was generated, and twelve research hypotheses were formed in order to test the general- ized null. The statistical models chosen for analysis were the Chi Square of independence and the Spearman nonparametric correlation analysis. Both permit one to determine if there is a statistical association between two variables. In addition, frequency counts, means and standard deviations will be calculated in order to compare use of the written evaluation form among the five independent variables of time, faculty employment status, student class, grade point average, and percent of completed forms on file. The purpose of the case study of written evaluation in Justin Morrill is to generate plausible hypotheses for the degree of implemen- tation determined. If the general thesis as stated in Chapter I is correct, these hypotheses will relate to the manner of adoption, the characteristics of the innovation, the characteristics of the organiza- tion and its members and the amount of nurturance given the innovation during its implementation. CHAPTER IV HISTORICAL DEVELOPMENT AND DATA ANALYSIS Introduction In the present chapter, Justin Morrill College (JMC) is described in general terms and then its characteristics as a formal organization in a university setting are examined, including mention of the traits of the faculty, students and administration. The history of the adoption and implementation of the JMC written evaluation system is recounted including some comment on the impact of the various organizational ele- ments on the process. Finally, the present state of implementation of written evaluation is determined by examining the degree to which faculty and students have completed the written evaluation forms, the percent of forms on file, the number of forms read by students, and the number of student requests for profiles. Justin Morrill College (JMC) As An Organization Justin Morrill is a small (approximately 800 students, 30 full- time faculty), residential, liberal arts college located on the campus of Michigan State University. It was created in 1965 to cope with the anonymity which students were experiencing on campuses as large as Michigan State's (40,000 students, 2500 full-time faculty) and to provide 111 112 an alternative B.A. program within which new modes of instruction could be attempted. To take advantage of the university setting, the college is semi- autonomous, permitting students to take their general education and some elective courses in JMC and their field of concentration courses from departments in the University at large. Faculty have their offices in Snyder-Phillips, the dormitory in which the college is located, and in Baker Hall, an office building nearby. Many classes are offered in Snyder-Phillips, and the building has "several lounges, a grill, dining facilities, and a small library, each of which is often the locus of extended interactions among students, faculty and members of the adminis- tration in JMC, plus persons from outside the college."] Students have a wide range of choice within a curriculum organized around two sets of objectives, one concerned primarily with content and the other highlighting the learning skills students should obtain. JMC's content objectives are found in the majority of the college requirements--45 credits in the humanities, natural sciences and social sciences, a two-year competency in a foreign language, 40-45 credits in a field of concentration. The learning skills are communicating effectively; acquiring, evaluating and synthesizing information; working independently; working in groups; and demonstrating creativity, inter- cultural awareness, self-awareness and an ability to solve problems. 1R. V. Farace 99_al., The Communication §y§tem of Justin Morrill College (East Lansing,_MHchlgan: Department oflCommunication, College of Communication Arts, Michigan State University, 1970), p. 1. 113 Each course in JMC addresses itself to some of the stated process skills as well as dealing with a particular content area. During the discussion of JMC as a formal organization which follows, it is important to keep two thoughts in mind: The discussion will be of JMC from the l969-70 academic year until the present time and will not mention qualities which may have been more typical of the college during earlier periods. Secondly, although JMC has been in a state of flux since 1969, much of the description may sound as if it is a static organization. As was pointed out in Chapter II, this illusion of stasis is one of the inherent problems in describing the attributes most typical of an organization over time; unfortunately the result is more a snap- shot than a moving picture. Like most colleges and universities, the role structure in JMC is not well integrated, i.e., there is a low degree of interdependence, and due to the small size of the college, there is also a low degree of specialization. Thus, the student can be quite autonomous in defining the particular path he will take to earn a B.A. from Justin Morrill (it is unlikely that any two JMC students have taken identical programs since the college's inception in 1965), the faculty member1 has wide discretion in determining the subject matter and style of his courses, and the Dean is free to justify any one year's offerings within the rather broad parameters represented by the phrase, "experimental, liberal education." This is not to say that freedom is unbounded and there is 1Unless noted otherwise, the reader may assume that the words, “faculty member" and "faculty" refer to full-time faculty in JMC. 114 no role structure, but only that the degree of interdependence is less, say, than an English department where students must take and faculty must teach certain content areas, and the department chairman must assure that the integrity of the major is upheld. Stated differently one might say that JMC exhibits more equifinality than a department or college with a more narrowly defined mission. As noted above, JMC does not have much specialization in role structure, i.e., the behavior associated with particular roles is not severely limited, and individual initiative is encouraged. Students, although separated by class (freshmen, sophomores, juniors and seniors), are not particularly restricted in the courses they take after their freshman year. Some students have even served in the roles of assistant teacher and seminar leader, and many have participated as full voting members on all JMC committees. Faculty, too, are differentiated into sub-categories--instructor, assistant professor, associate professor and full professor--but the role expectations within JMC are predomi- nantly the same for all levels. While expected to teach primarily in specific knowledge areas, most JMC faculty cross disciplinary lines when their competence and student demand warrant it. In addition, all levels of JMC faculty advise students as they determine their fields of con- centration, serve on committees which plan the future direction of the college, and serve on university committees. There are only two adminis- trators in the college, the Dean and the Assistant Dean.] Whereas the 1The Dean also has a staff assistant with the loosely defined role of system coordinator. 115 Assistant Dean's role is quite specialized (coordinates advising, processes grades, keeps student records), the Dean occasionally teaches, serves on many committees and advises some students in addition to administering the college. At times when administrative chores peak, the Dean simply asks a faculty member or two to do a particular task or to assist him temporarily. In sum, although the normal role distinctions among students, faculty and administrators exist in JMC, they are less sharply drawn than in many colleges. And, the small size of JMC discourages the differentiation and elaboration of roles, resulting in a rather loosely defined role structure with high individual initiative. JMC's goals are quite typical of most colleges and universities in that they are vague and somewhat conflicting. These characteristics, of course, give rise to and reinforce the loose role structure described above. At one time or another since its birth, JMC has had a variety of goals--experimentation, innovation, liberal education, general education, flexibility, an integrated curriculum, a residential community of faculty and students, nonconformity (the mandate to be different and not duplicate other curricula), international education, and cross-cultural education.1 As is readily apparent, all of the above words and phrases leave much room for interpretation. What, for instance, is the meaning of, 1See the following documents for goal statements: Guidelines for the Justin Morrill College; A Report from the Ad Hoc Committee on the Feasibil1ty of a New, Sem1-Autonomous College; Minutes of the Michigan State University Board of Trustees, Justin Morrill College, a publicity brochure, 1972. 116 "experimentation?" Does it imply, as some would argue, the stating of objectives, the use of control groups where possible, and the careful evaluation of results? Or does it merely mean the trying of new ideas and practices, a definition some would associate with innovation rather than experimentation? In addition, some of the goals conflict with one another if taken in the extreme. It is difficult to be infinitely flexible and have a tightly integrated curriculum. It is also difficult to provide a liberal education while not conforming to any traditional university norms. Of course, the goals need not conflict, but they do open the possibility of disagreement because of their ambiguity. Naturally enough, the task demands associated with the above goals are many and varied, and JMC's history reflects different emphases at different times. Nonetheless, the goals that seem to have been most powerful in influencing the college are those of innovation, liberal education, flexibility, nonconformity and community. The rapidity with which the college has introduced new courses, new teaching styles, new governance structures and curricular modifications attests to the impact 1 The speci- of the goals of innovation, flexibility and nonconformity. fic student learning objectives as well as the content distribution requirements (see above) are examples of the concern for providing a liberal education. Finally, the off-campus weekends, the faculty re- treats and the high amount of participation in governance all reflect lSee term by term course descriptions, the four major governance models used by JMC and the requests for curricular modification brought before the University Curriculum Committee by JMC. 117 the desire to build an effective community in Justin Morrill. In brief, the above five goals and their relative ambiguity have influenced the teaching, curriculum development, advising, and governance-~the four major task demands associated with them. As implied during the discus- sion of roles, however, there does exist a normal, if somewhat flexible, division of labor-~people do teach predominantly in one program in order to provide an appropriate content range for liberal education, faculty do advise students who are interested in their area of competence, upperclassmen do advise lowerclassmen and there are two specialists in administration. Within the area of curriculum development and governance, however, there is much less specialization. Generally speaking, the JMC social system has more modern than traditional norms. That is, on the continuum of ideal types described in Chapter 11, there is a positive attitude toward change, a relatively complex division of labor, a high value placed on education and the scientific method, cosmopolite perspectives "in that members of the system interact with outsiders, facilitating the entrance of new ideas into the social system,"1 and an "emphathic ability on the part of [JMC's] members, who are able to see themselves in roles quite different from their own."2 Nonetheless, JMC falls somewhere toward the tradi- tional end of the continuum when it comes to the style of interpersonal 1Rogers with Shoemaker, 99, 919,, p. 33. 2Ibid. n.b., empathy, or the ability to place oneself in the shoes of another person, was one of the major goals stressed by faculty during the period when the international and cross cultural themes were pre- valent. Teachers in both language and field study still stress this skill heavily in their classes. Note also that one of the learning skills that JMC students should develop is intercultural awareness. 118 relationships most favored. Rather than engaging in business-like and rational social relationships, most faculty and students place a high value on relationships that go beyond mere organizational or role behavior within peer groups.] This norm extends to the relationship between faculty and students also. For instance, in the Farace 99_91, study of communication patterns in JMC, one of three students reported a "close friend" on the faculty and two of three had a "good friend."2 Faculty are slightly more ambivalent about how friendly relationships between faculty and students should be. In the opinion survey, twelve agreed, five expressed ambivalence, seven disagreed, and five strongly disagreed with the statement, "Professional distance between faculty and students should be maintained in the classroom."3 Still, the norm of informal interpersonal relationships is obviously felt by many faculty. Essentially, faculty and students in JMC value "affective personal relationships, such as friendliness and hospitality."4 Talcott Parsons might note that they prefer the pattern variables most normally associ- ated with the socio-emotional versus the production needs of an 1This preference was clearly illustrated in two opinion question- naires distributed in conjunction with the recent Provost evaluation of JMC. Most faculty (26/30) either agreed or strongly agreed with the statement, "JMC fosters a strong sense of community among faculty that cuts across academic disciplines." When asked to characterize their relationships with fellow JMC students, a sample of 85 students used such phrases as "a shared concern for personal growth" and "a caring for and about one another“ most frequently. 2Farace e_t_al., 92. _c_i_t_., p. 38. 3 4 Faculty Opinion Survey related to Provost Evaluation. Rogers with Shoemaker, 99, 919,, p. 32. 119 organization. In JMC's case however, such behavior aids one of the stated goals also--community building. Within JMC persons are suspicious of using the scientific method or a rational problem solving technique to the exclusion of emotional considerations. For instance, the list of desirable student learning skills includes the ability to work in groups and self-awareness in the hope that students will develop a sense of the part that attitudes, values and emotions play when people interact with one another. Another norm of JMC supports a high degree of interaction among faculty and students both within and outside the classroom,1 and is complemented by a norm of high participation in decision-making for both faculty and students. Major decisions in JMC almost always involve a high proportion of the faculty and several students, and when they do not, there is a great deal of resistance and ill will.2 The final two norms relate to the faculty social system rather than the students'. The first supports a certain degree of noncomforming, individualistic, risk-taking-behavior and fits with the JMC goal of noncomformity with the university system at large.3 Thus, the invention lBoth faculty and students report interaction between the two as one of the desirable aspects of being a part of JMC. Farace et a1. ., 99, cit. pp. 37- 38 illustrate the high degree of interaction among faculty and— students. 2Witness the administrative decision to eliminate the JMC language program and its aftermath in Spring, 1973. 3n.b., by nonconformity, the author wants to emphasize that JMC has a mandate to be different and not duplicate any other models in the uni— versity. The original guidelines for the college made this desire for difference clear, and it breeds a certain disdain for the customary. 120 of new ideas and practices is looked upon favorably by faculty peers so long as they do not severely disrupt the work of others. The second norm, typical of practically all university settings, is that of faculty autonomy with regard to their classroom behavior. Faculty in JMC, as elsewhere, desire as little interference and external control as possible when it comes to what and how they teach. Internally, the few rules and regulations complement the role struc- ture and goals of JMC. Students must earn 180 credits (45 in the social sciences, natural sciences and humanities, 12 in Field Study, 8 in Inquiry and Expression, 40-45 in a field of concentration, 50-70 elec- tive credits), live in the dormitory one year and meet the demands of various instructors. Faculty abide by the rights and responsilities in effect for all MSU faculty, write course descriptions, meet their classes, evaluate student performance, advise students who request advice, and turn in grades on time and fulfill their committee responsibilities. Within these general regulations, both students and faculty exert exten- sive individual initiative and self-control, and the administration enforces the rules to a minimal degree. The authority system of JMC is based on the clear understanding that the Dean ultimately either makes all decisions or has the right to veto in the instances where faculty share authority. His decisions in turn are subject to scrutiny and approval by the Provost, the President and the Board of Trustees. In the area of curricular change, the faculty and student committee system outside the College and Academic Council1 1The Academic Council consists of elected faculty and student repre- sentatives from individual colleges (2/3 faculty, 1/3 students), Deans, and is chaired by the MSU President. 121 become involved in the approval process. The JMC administrative hierarchy is unusually flat, i.e., it con- sists of only two persons, the Dean and Assistant Dean.1 The Dean allocates the budget, coordinates the curriculum, and serves as the principal college liaison to the university. The Assistant Dean manages student records and schedules classes. For personnel matters and policy making, there are two important advisory committees. The Personnel Committee (3 faculty, 3 students) advises the Dean on faculty promotions, retention and raises. The Advisory Council (3 faculty, 3 students) advises the Dean on all major policy issues related to personnel, budget and program development. The fact that these committees are representative and advisory clearly illustrates that there is a power concentration in the JMC authority structure which enables decisions to be made even in the absence of consensus among faculty and students. Although there has always been a power concentration within JMC, the degree of participation in decision making by faculty and students has been high. In the case of faculty, only a few have chosen not to make their influence felt. 0n the other hand, few students (perhaps 1This has not always been the case. At one time there were Program Directors to facilitate the coordination of curricular offerings in specific knowledge areas and an Associate Dean to add a helping hand in the areas of budget, program coordination, and general system mainte- nance. However, both positions were eventually eliminated because of the anti-bureaucratic, informal bias of JMC. Some persons argued that these positions simply placed gatekeepers between the Dean and his faculty and students, and that the positions would diminish the flexi- bility of the college. Thus increased efficiency was sacrificed for the sake of informality. 122 20 at any one time) actually exert influence, but they do enjoy equal membership with the faculty on all committees.1 The reward system for Justin Morrill faculty (raises, promotions, peer admiration, administrative praise, respect of students) is con- sistent with the open, innovative, participative and student oriented nature of the college. Good teaching, extensive interaction with stu- dents, high participation in JMC and university governance, the genera- tion of new and popular courses, and community building efforts receive rewards. The publication of books and scholarly articles in professional journals, while not ignored, are not seen as important to the welfare of JMC. As is evident from the discussion of roles and the authority struc- ture, JMC does not have an elaborate or highly specialized division of labor to meet the functional imperatives of goal attainment, maintenance and adaptation. Partially as a result of this lack and partially as a result of management's profound concern for adaptation and innovation, JMC has not exhibited a very stable equilibrium between production and maintenance (human) needs since 1970. Before discussing this problem, however, it should be made clear that certain basic needs for stability are fulfilled. The Assistant Dean's role is one of maintaining the efficiency of JMC. He assures that student records are updated, that students are fulfilling requirements, that the advising system is functioning, that 1As a 1972 student opinion survey makes clear, however, many stu- dents feel powerless since they are not directly involved in the advisory process. 123 faculty are writing course descriptions and writing evaluations, that the instructors can locate adequate teaching Space and that JMC's record keeping is coincident with that of Michigan State. In addition, faculty motivation is maintained through raises, promotions, awards, leaves of absence, the praise of students and the inherent satisfaction that comes from offering successful courses. . The Dean's staff assistant1 works with the elected representatives of the faculty who serve on college and university committees to assure that the governance system operates smoothly and that university dead- lines are met. The supportive mechanisms of student recruitment and graduate placement are handled for JMC by university offices. In brief, the college continues to provide an adequate B.A. program for under- graduates. Nonetheless, the Dean spends most of his time assuring that Justin Morrill does not become irrelevant to the educational needs of the 1970's rather than coordinating and maintaining a balance among goal attainment, maintenance and adaptation. He stresses innovation and adaptation as the most important elements of Justin Morrill and has all the attributes that Everett Rogers assigns to the innovator. The Dean is marginal to the JMC social system,2 more venturesome, more cosmopolite in terms of educational literature and more willing to risk possible 1The author of the present study. 2Marginal in two senses: (1) He seeks more change than the faculty desire; hence, his behavior is not normative; (2) He personally does not implement many of the innovations but rather oversees their implementa- tion. Obviously, he is of central importance in the authority system. 124 setbacks for the sake of trying new ideas than other members of JMC. Being an innovator, the Dean likes to move quickly and is frustrated by the degree to which high participation by faculty and students can slow down the adoption process. As a result he sometimes makes decisions before the advisory network can reach an agreement, and this behavior runs counter to the norm of participation so prevalent in the faculty and student subcultures. The Dean is most interested in encouraging major system change (e.g., introducing a competency based curriculum and eliminating the traditional tri-partite division of knowledge) whereas the faculty are most interested in course level innovation.1 These two levels of interest occasionally interfere with one another. Finally the Dean is more interested in creating an organization in which change can occur frequently and easily than with the fate of any single innovation. He, like the problem-solvers discussed in Chapter II, would ultimately like to create a self-renewing organization. The leadership and management characteristics of the JMC Dean have certain organizational consequences. His push for innovation in an organization that is relatively open to change and has faculty who favor change enables the college to attempt a wide array of new educational ideas. In the past three years, JMC has offered two different freshman programs, tried two distinct governance models, introduced a written evaluation system for all students in JMC courses, attempted to develop 1In the Faculty Opinion Survey related to the Provost evaluation of JMC, five questions regarding experimentation revealed a faculty pref- erence for course level innovation. 125 a model for goals based planning, adOpted two versions of nodular scheduling,1 modified the general education requirements, conducted a major all-college evaluation, and introduced innumerable new courses in addition to continuing the normal tasks of an undergraduate degree granting college. In spite of the positive aspects of the Dean's leadership, his management style coupled with a general aversion to bureaucratization within JMC have led to a minimum of maintenance behavior. Systematic Operational feedback, aiding the implementation of innovations and evalu- ation of new practices are simply not stressed as heavily as adopting innovations. Neither the faculty nor the Dean have enough energy to devote to these tasks, and until 1972-73, no one else was given the role(s) except on an overload basis. In addition, the Dean does not realize how few major innovations the faculty can implement and still continue their daily routine of teaching, advising and committee work—- the major tasks of the organization. Instead of smoothing the way for implementation the Dean frequently tries to shift the faculty's atten— tion to another new idea which he is anxious to have JMC attempt. On occasion, the result is frustration of the Dean at the faculty's reluc- tance to consider seriously the new idea and frustration of the faculty at the Dean's lack of appreciation for their present efforts. Since both the faculty and the Dean tend to concentrate their efforts on adaptation and production, the human problems associated with 1In modular scheduling students enroll for one or two courses at a time for 3-5 weeks rather than ten weeks, the aim being to work more intensely with fewer distractions. 126 innovation are generally overlooked. Most of the time, this blindspot is unimportant because the Dean, faculty and students are deeply in- volved in and tremendously excited by the work of JMC. By the end of each academic year, however, the Dean, faculty and those students directly involved in governance are weary due to the fact that all have been engaged in the tasks of production and maintenance while simultane- ously trying to meet the demands for innovation and adaptation. There is an imbalance between production and human needs, and a new point of equilibrium is usually identified each subsequent Fall Term. In fitting with the norms of participation and interpersonal com- munication and the relatively low integration of the role structure, there is an extensive horizontal communication network in JMC in which all full-time faculty and some students participate. The faculty net- work is aided by the size of the group (approximately 30 full-time people) and by the weekly faculty seminar to discuss topics of mutual concern. Part-time faculty (those who teach only one course a term or less and are on loan from another department) are often not in the net- work due to their infrequent appearance in the college, their absence from the faculty seminar and the fact that their primary loyalty is normally with their home department. The student network for normal college issues is hampered by several factors: there are 800 JMC stu- dents; over half live off campus; few upperclassmen (22%) attend JMC classes; few students are directly involved in the advisory process (approximately 20); most students are interested in their own course of study and not in shaping the future of the college; there is a rapid 127 turnover in members of JMC through enrollment, graduation and transfer. In spite of the efforts of the 99999, a college newsletter run by stu- dents, and the Communication Center, an office established expressly for facilitating communication of college issues among students, a recent student opinion survey revealed that 12% of the sample never learned of decisions, 8% learned of decisions when they were in effect, 25% learned of decisions just after they were made, 38% learned of pending decisions when plans were submitted for student discussion, and 15% participated either directly or indirectly in making the decisions.1 Compared to many colleges, these figures probably reveal a relatively integrated communication network among students. Most full-time faculty came to JMC because they were attracted by the creative and innovative character of the Dean and the goals of the College. Hence, it is not surprising to learn that an outside consultant visiting the college in connection with the 1973 Provost evaluation found JMC faculty, "warm, open, permissive, concerned with helping stu— dents formulate their own life style, disrespectfbl of academic and political authority, creative, non-quantitative and above all, committed to individual student growth."2 In addition, the faculty share many of the early adopter characteristics that Rogers and Shoemaker noted. They are well educated (all have at least the M.A. and approximately 50%, the Ph.D.), literate, empathetic (see above), can deal well with 1From the Student Opinion Survey in conjunction with the Provost eval- uation of JMC, based on a random sample of 85 JMC students. 2Harold Hodgkinson, from a personal conversation with the author. 128 abstractions, are rational, are not fatalistic, are intelligent, have a favorable attitude toward change, take risks (coming to JMC rather than staying in a more established department), are achievement motivated, are well integrated into the JMC social system, are cosmopolite (read extensively in educational fields and have professional associations beyond JMC), have exposure to interpersonal communication channels, seek information, know about many innovations in higher education and are members of a college with predominantly modern norms. Naturally, all JMC faculty are not identical on the above traits; for example, some are less empathetic than others, some are less integrated into the social system. In general, however, JMC faculty exhibit many of the characteristics associated with early adopters, and these traits comple- ment the goals, structure and norms of the college.1 JMC faculty do differ in personal life style, intellectual inter- ests, age and personal background and are thus not entirely homogeneous. However, for the interests of the present study, they exhibit remarkable homogeneity in areas associated with the early adoption of innovations. Students, also, are attracted to JMC by its stated goals and empha- sis on personal growth and development. They are not entirely homo- geneous since they have different backgrounds, lifestyles, and intel- lectual interests. Nor does the group remain constant since each year there are numerous new enrollments, graduates and transfers. 1Many of the early adopter characteristics are typical of most uni- versity professors. The important point with JMC facu ty, however, is that the traits complement system norms and structure whereas this would not be the case in a more bureaucratic, narrowly defined college. 129 Nevertheless, JMC students are similar in many respects. They tend to be bright1 non-quantitative,2 liberal politically,3 moderately theo- retical,4 to prefer imaginative exploration to logical analysis,5 to be anti-authoritarian with a high need for autonomy and independence,6 and are willing to express impulses.7 In spite of their desire for autonomy and independence, JMC students also express a strong like for community caring for other persons.8 They also express a strong prefer- ence for student participation in college decision-making,9 and many view their own value systems as being open, tolerant and flexible.10 1They have scored consistently above the MSU average on verbal abil- ity as measured by the CQT and the SAT. Their university grade point average is consistently higher than the university average. Proportion- ately, JMC has a higher percentage of its students in Honor's College than any other MSU college. Seniors in the graduating class of 1969 scored very high on the Graduate Record Examination compared to a nation- al sample: 89th Percentile in Social Science, 80th Percentile in Humani- ties and 79th Percentile in Natural Science. 2JMC students score consistently below the MSU average on mathe- matical ability as measured by the CQT and SAT. 3Walter Shaw, senior student Perceptions of Justin Morrill Colle e, Class of 1969 (East Lansing, Michigan: Michigan State University, 1969), p. 6(mimeographed); also, note the high participation of JMC students in the 1970 strike activities and the campaigns of liberal politicians. 4Qggibus Personality Inventory given to JMC student sample in 1969 and 1970. Ibid. Ibid. 71bid. 0101 8Student Opinion Survey in conjunction with the Provost evaluation of JMC. 91bid. 10Shaw, 99, 919,, p. 18. 130 Many of these characteristics were observed in JMC students by a con— sultant who visited Justin Morrill in 1973. He found the students with whom he talked, warm, open, autonomous, interested in self- exploration, anti-authoritarian, creative and collaborative.1 In addition to the above traits, the JMC students most active in the governance process exhibit many of the early adopter characteris- tics. They tend to be empathetic, have a favorable attitude toward change, take risks, are achievement motivated, are more integrated into the JMC social system than their peers, are cosmopolite (most have been overseas), have exposure to interpersonal communication channels, seek information and are familiar with some innovations in higher education.2 Like the JMC faculty, the students have many characteristics which sup- port JMC norms and goals. They express a strong desire for community (although many are not directly involved), desire autonomy, enjoy the freedom that comes with a highly flexible curriculum, and express a preference for open value systems. The environment, as well as the internal elements of the organiza- tion, has affected JMC. In 1965 when JMC was created, educators wanted to alleviate the impersonal nature Of mass education, money (State, federal and private) was available to support new ideas, higher educa- tion enjoyed unprecedented public support and was expanding rapidly, and 1Harold Hodgkinson, a personal conversation with the author. 2These generalizations concerning the most active JMC students are derived from the author's close assoc1ation with all the JMC students involved in the governance process during the past 5 years. 131 there was an ever-increasing supply of undergraduate students. In this supportive atmosphere experimental colleges like Justin Morrill emerged in large numbers around the United States for the express purpose of implementing new ideas in education. The question asked of these insti- tutions was not, how effective is your new college, but rather, how many new practices have you initiated, and how many more do you have planned. In 1973, a different environment exists. JMC is criticized as an elitist institution because it tends to attract bright, middle class students and is more expensive than many undergraduate programs at Michigan State. Funds are tight, and some feel that innovative programs are unnecessary frills which should be eliminated so that faculty in more traditional areas can receive suitable salary increases. In the wake of the student unrest of the 1960's, the public is skeptical of higher education, critical of the institution's inability to control student behavior, and wondering why tax dollars should be spent to sup- port student destruction of public property. And, student enrollments ' are decreasing. The question most frequently asked of experimental colleges in the 1960's has turned into a demand--prove to us that your new practices are more effective than the old and worth the additional expense. JMC remains more reflective of the late 1960's environment as the description earlier in the present chapter makes clear, but the college has not been unaffected by the recent societal shifts. In all likeli- hood, the college known for the frequency of its innovation in the 132 1960's will gradually be placing much more emphasis on economy and evaluation in the 1970's. This change in emphasis is quite clear in the 1973 Provost Evaluation Report of JMC. To summarize, the organizational elements of Justin Morrill, the occasional imbalance between human and production needs, the member characteristics and the environment of the late 1960's all contribute to make the college a system which, in the words of Lon Hefferlin, permits the "dangers of instability."] It welcomes change in a variety of ways, and its brief history reveals an organization which stresses innovation and adaptation. As will be illustrated later in the present chapter, these characteristics of Justin Morrill played an important role during the adoption and implementation of the written evaluation system. Adgptigg Written Evaluation The written evaluation system which JMC adopted at the end of Spring Term, 1970 did not embody ideas which were totally new to either the college or Michigan State University. In the late 1960's several colleges and universities around the country began to implement non- traditional grading practices to replace either in whole or in part systems they had been using.2 In addition, MSU had adopted a 1 2Francis H. DeLisle, The Impact of the Pass-Fail System (East Lansgngs Michigan: Michigan State University, 1969), p. lemimeo- grap ed . Lon Hefferlin, 99. 9_i_t_., p. 163. 133 credit-no-credit (CR-NC) option for limited use by students in the hope of encouraging them to explore unfamiliar intellectual areas. JMC's Field Study staff had been using pass-fail (P-F) grading since the program's inception in 1966 and began writing evaluations of student performance in 1969. During the 1969-70 academic year, JMC began using pass-no-credit (P-N) grading for the independent study sections of all its courses.1 Thus, the Dean, faculty and many JMC students were aware of the concepts of ungraded learning and written evaluation well before 1970. Nevertheless, the first formal proposal for JMC to move to an un— graded system did not surface until Winter Term, 1969, from an off- campus faculty-student planning weekend. The proposal suggested that a pass-no-credit grading system be initiated for all JMC courses, that no evaluations be written, and that a "P" be the equivalent of at least a 2.5 on a 4.0-0.0 numerical scale. The proponents felt that grades pro- vide a coercive force for learning which is not consistent with a college which professes to "ask the student to take responsibility for his own education."2 As the document argued, a P-N system should help students integrate "emotional growth, self-awareness, and the traditional pursuit of knowledge and occupation,"3 an implicit goal of JMC.4 1P—N grading for the independent study sections of JMC courses was first used during Spring Term 1970. 2Winter Weekend Planning_Committee on the Role of Grades, mimeo- graph, p. 2. 3Ibid. 4JMC provides the general education component of an undergraduate's program. Traditionally, general education courses offer a blend of cog- nitive and affective learning. 134 The Winter Weekend proposal gave impetus to two further recommenda— tions. During the Spring of 1969, JMC proposed to the University Curri- culum Committee that the college adopt P-N grading for the independent study sections (one credit each) of all its courses. In Winter Term, 1970 the College Curriculum Committee (C.C.C., 5 faculty and 5 students) began discussion of a no-fail grading system in earnest, and proposed: That JMC request permission to adopt, on an experimental basis, a system of grades running from 4.5 to 2.0. Any work below this level would receive an N. This system would be used in all JMC courses except those specifically approved for P-N grading. The C.C.C. did not present the proposal and accompanying rationale to the college at large during Winter Term, but did use it as a focus for their deliberations on revised grading in JMC. The members invited Willard Warrington, the Director of Evaluation Services at MSU and an influential faculty member in university governance, to discuss with them the merits of the proposal and to offer his estimation of whether the idea could be approved at the university level. Dr. Warrington was encouraging and urged the committee to consider a more radical departure from the traditional numerical grading procedures. During the discus- sion which ensued, several points were agreed upon: 1) Simply eliminat- ing grades below 2.0 would inflate the JMC grade point average and reduce its credibility; 2) Grades do provide some incentive for many students, as well as a general picture of their performance level. Therefore, if grades are removed, some form of evaluation must replace them; 3) Graduate schools and employers will expect some evaluation of student 1GradingProposal, JMC CurriCulum Committee,(March 1970). 135 performance; 4) JMC is in an appropriate locale to offer a non-tradition- al grading system since it is small, has an experimental mission, and has faculty willing to spend the additional time evaluating student performance.1 After two more meetings, one on April 10 and the other on April 17, 1970, the C.C.C. and the Director of Evaluation Services concurred that the college should adopt a system in which all JMC students would re- ceive either a pass or no-credit and a written evaluation of their course performance for each course taken in Justin Morrill. A sample evalua- tion form was discussed as were the possible problems associated with moving to a P-N, written evaluation system. Some of the questions were: Will graduate schools and employers discriminate against our graduates? Will our students be eligible for Honor's College? Will our students qualify for honoraries such as Phi Beta Kappa? Will students do the minimum to get through courses? Will the system prevent JMC students from receiving or maintaining financial assistance? Will the system improve the educational atmOSphere in JMC? By the end of the April 17 meeting, the members of the C.C.C., the Dean, the Associate Dean and the Chairman of the College Advisory Council agreed that a formal pro- posal and rationale should be presented to the JMC faculty and students for their response. On April 23, 1970 a formal proposal (see Appendix 5) was distributed to all JMC faculty and students. As the document noted, if the proposal lMinutes of the 4/3/70 C.C.C. meeting and conversations with the participants. 136 passed: All JMC instructors would provide written evaluation [sic] for each student in JMC courses. These written evaluations would be recorded and/or summarized and made available to other agencies at the request of students. Only "Pass" and "No-Credit" would be recorded on the MSU transcript with an explanatory note indicating the nature of our evaluation system. The document also included a sample evaluation form, two sets of instructions--one for instructors and one for students, a copy of the minimum academic progress scale, and a detailed rationale. The evalua- tion form contained space for the student's name, his student number, a course title and description, the student's objectives in taking the course, the bases for evaluation, the student's self-evaluation, the instructor's evaluation of student performance, and three separate sec— tions to list the college, program and course objectives. The college objectives1 were pre-printed on the form, and a grid was printed next to the objectives listed so that both students and faculty could check the student's level of performance. The instructions asked the students to put their name and stu- dent number on the form, to state their objectives for taking the course, and at the end of the term to evaluate their course performance if they wished. The instructors, under the proposed system, would include on the forms a course description, course objectives, an evalua- tion of the student's performance on both college and course objectives 1A slightly different set from those identified earlier in the present chapter as student learning objectives. Within JMC, the expres- sion "college objectives" is used interchangeably with "student learning objectives." 137 and a Pass, No-Credit, or Incomplete. The rationale stated that the system would provide individualized evaluation rather than ranked grading, permit both faculty and students to share in the evaluation process, encourage clearer course descriptions and objectives, high- light college and program objectives, allow evaluation based on college, course and student objectives, give more detailed evaluation, provide a basis for a student profile at the end of four years, eliminate the failing grade, and fit the experimental nature of Justin Morrill. As may be apparent, the 1970 proposal is similar but not identical to the description of the innovation given in Chapter I, the 1973 version of written evaluation in JMC. On April 29, 1970 the College Curriculum Committee members held a hearing to respond to questions and listen to the suggestions of faculty and students. Fewer than twenty-five of a possible eight hundred persons attended the hearing, but those present were interested in and supported the proposal. Responses were also requested through the §9999_(the college newsletter) and memoranda during the next few days. In light of the absence of negative feedback, the College Advisory Council, through a poll of its members, endorsed the proposal and ad- vised the Dean to forward it to the University Curriculum Committee for approval. An excerpt from a memorandum written by the Dean summarizes the activities leading up to the College Advisory Council endorsement: The JMC Curriculum Committee began discussing the matter two months ago, and their minutes have been distributed weekly. They held an open meeting to which all interested faculty and students were invited, and then an open hearing last week. The proposal has been discussed in The Sheet on three occasions, and the entire proposal with rationale and sample evaluation 138 form was distributed to all students and faculty on April 23. The Chairman of the Advisory Committee sent out a special memo to faculty pointing out the need to consider the pr0posa1 carefully and communicate their response. The Curriculum Committee has investigated all questions as they have arisen, and the full committee or the Chairman has talked with Dr. Blackington of the Honors College, Dr. Warrington of Evaluation Services, Mr. Dykema of Scholarships and Financial Aid, and representatives of the Teaching Certification Office. As a result of all of this we have had virtually no negative response from anyone, which is remarkable for any proposal of this significance. The individuals mentioned above were all sup- portive and most were enthusiastic. The JMC students and faculty comprising the Curriculum Committee support the proposal unani- mously. All students who attended the open hearing were in favor of the proposal. The College Advisory Committee has approved the proposal.1 On May 7, 1970, JMC forwarded a modified version of the proposal (see Appendix 6) to the University Curriculum Committee, anticipating approval so that the new system could be implemented Fall Term 1970. Thus, in a period of fourteen days, JMC had moved from the initial formal presentation to faculty and students to a decision to adopt a written evaluation system which required significant alterations in faculty and student behavior. The University Curriculum Committee approved the proposed system as a two year experiment on June 18, 1970. If one were to examine only the charatteristics of the innovation as a means of determining the likelihood of adoption, it would be dif- ficult to explain the rapidity with which JMC decided to adopt written evaluation on a trial basis. For although there are some obvious ad- vantages to written evaluation in a college like JMC, there are also some severe drawbacks. 1Dean's memorandum to faculty, May 4, 1970. 139 The thoroughness of the evaluation and its potential high quality gave the proposed P-N, written evaluation system a high degree of relative advantage over the former numerical grade. In addition, the proponents of the innovation hoped that faculty would clarify course and program objectives, and the students would better understand the bases for evaluation in a particular course. Individualized evaluation would replace the emphasis of ranking students that letter grades tend to produce. Proper use of the evaluation form would highlight the goals of the college more effectively and stress the integrated nature of the curriculum. Conlnunication between faculty and student might well be improved. In sum, there would be potential system as well as individual gains by the introduction of P-N, written evaluation. , Nevertheless, some persons were also aware of the possible disad— vantages. Faculty would have to spend far more time writing course descriptions, identifying course objectives, orienting new students to the system's objectives, keeping student records and completing forms. The Assistant Dean would have to spend additional time coordinating the new system, arranging for forms to be copied, filing forms, assuring that profiles are written, and orienting part-time faculty to the system. All of the additional time would be accompanied by additional expense-- printing and copying of forms, more staff help for filing, and more file space to say nothing of the expense associated with professors writing lengthy evaluations. With greater individualized evaluation come less comparable measures; some students might want to know how they rate in comparison with others, some might be upset to learn that 140 "progress made" as well as "level of achievement" may influence the evaluation, and still others might be frightened away at the prospect of having to earn at least the equivalent of a 2.0 to receive a Pass. Persons outside JMC would be uncertain how JMC students compare with others. In some respects the innovation seemed quite compatible with the JMC milieu. JMC has small classes (20-35 students), and faculty could probably manage writing up to 100 evaluations per term. The college prides itself on trying new ideas, and few other educational institu- tions had attempted written evaluation in all courses. In addition, most faculty are more concerned with developing an atmosphere which promotes intellectual and emotional growth than with learning produced by competition and stress. At its best the system should make more apparent to the students that JMC faculty are interested in them as individual learners and do not want simply to dispense knowledge. But, the proposed system was clearly incompatible in other respects. Having faculty write course descriptions, course objectives, and evaluations in a similar manner on a common form and asking them to meet common deadlines for course descriptions and evaluations con- flicts directly with the norm of faculty teaching autonomy. One would predict at least some resistance by faculty due to this clash. And faculty were not skilled in writing course objectives or in identifying through college objectives the principal aims of their courses, anymore than students were accustomed to evaluating their own performance. In addition, some JMC students readily admitted that they performed at a 141 higher level when pressured to earn a high grade and did mediocre work when left on their own. Yet, none of these factors seemed to slow the adoption process. As is readily apparent from the length of the evaluation form and the accompanying rationale, the system is quite complex and difficult to implement. Questions such as the following had to be posed: What does one send the Office of the Registrar? What should be sent to graduate schools and employers? How does one orient new students and faculty? How can the system be monitored? How can one evaluate the system's effectiveness? The fact that JMC was able to adopt the written evaluation system on a two year, trial basis and the existence of numerous variations on pass-fail, pass-no-credit and written evaluation systems certainly sped the adoption process. However, it was clear from the beginning that some results of moving entirely to written evaluation would be difficult to observe. One could certainly determine whether there were course objectives written, whether faculty and students wrote evaluations and whether it was possible to write profiles, but how does one assess the impact of written evaluation on the educational environment? For instance, it would be difficult to show with any non-reactive measurers that the new evaluation system facilitated, hindered or had no effect on the learning of students. Nonetheless, indirect measures were pos- sible. Since students would continue to receive grades in university courses and could receive a 'no-credit' in JMC courses, it would be possible to see if the student's all-university grade point average 142 changed after instituting P-N and whether there were less 'no-credit' grades given than grades below 2.0 under the former system. In addition, the Curriculum Committee knew that attitudes could be measured, and that if faculty and students perceived an improvement in the learning environment of JMC, written evaluation might be a useful innovation. In examining these conflicting aspects of the innovation, one would have guessed that internal disagreements over the merits of written evaluation would have at least prolonged the innovation decision and perhaps led to rejection. However, in light of the fast adoption rate, there must have been organizational and environmental factors which minimized the impact of the innovation's negative characteristics. Two important factors were the presence of a University Curriculum Conmittee May deadline if JMC wanted to implement written evaluation during the 1970-71 academic year, and the work of various advocates for change.1 For the innovation-decision under discussion, it is difficult to identify a single innovator since the general concept of ungraded learning had been present in the JMC environment for so long. The Dean mentioned the possibility of JMC modifying its evaluation system in his 1966 Annual Report. Several key students began advocating a shift to ungraded learning as soon as the Credit-No-Credit option became a reality at MSU (Fall, 1968), and some faculty began considering alternatives to numerical grading during the same period. To use the terminology of collective innovation-decisions, certainly all these persons served to "stimulate" interest in non-traditional grading. 1See Chapter II for the list of change advocate categories. 143 The Director of Evaluation Services, in consulting with the JMC Curriculum Committee, was an important advocate for change and served as a "legitimizer" for a much more radical departure from numerical grading than had been thought possible by anyone within Justin Morrill. The members of the C.C.C., who initiated the no-fail proposal outlined earlier in the present chapter, soon developed a document proposing that JMC adopt a written evaluation system for all its courses. Interestingly, as they did so, the Committee members became more message oriented. During the persuasion process, they spent long hours dis- cussing proposals, visiting external agencies, anticipating consequences, writing documents and persuading peers. As their time commitment grew along with familiarity of the strengths of the no-fail concept, so did their commitment to implement the P-N system. Being stimulator- initiators placed them on the horns of a dilemma, having to remain true to their cause (the role of a stimulator) and also responsive to the will of the JMC community (the role of an initiator). In the end they interpreted a limited amount of positive response and an absence of negative feedback as an indication of greater agreement than probably existed as the time of adoption. In addition to innovators, initiators and an external legitimizer, both formal and Opinion leaders played the important role of legitimiz- ing the proposal. The Dean, Associate Dean, Chairman of the Advisory Council and Chairman of the Curriculum Committee all supported the April 23 document and urged its adoption. Still other faculty and some stu- dents also concurred with the proposal and through their informal contacts advocated its acceptance. 144 Another factor which helped truncate the innovation-decision process by shortening the persuasion stage was the inability of the horizontal communication network to handle normal messages during Spring Term 1970. TWo crises, one internal and one external, had simply saturated the net- work, and few persons outside the JMC advisory system learned more than the most general aspects of the written evaluation proposal. On April 12, 1970 there was a food riot in the Snyder Cafeteria which precipi- tated a series of major student complaints concerning dormitory manage- ment and the relationship of Snyder-Phillips to Justin Morrill College. What came to be known as the "Sny-Phi Commune“ formed almost immediately, and there were nightly meetings of 200 or more students in the down- stairs' lounges. The basement doors between the Snyder (men's) and Phillip's (women's) dormitories, which had always been locked at mid- night, were "liberated," and the students declared the dormitories "co-educational." Many began to shift their belongings to the alternate dormitory, and bathrooms were made "co-ed" to accommodate the new living arrangements. To resolve the many issues raised in connection with the residential environment, the MSU Provost created a commission composed of the Dean of Justin Morrill, the Dean of Students at MSU, a member of the MSU President's staff, an Assistant Provost, the manager of MSU Residence Halls, and five residents of Snyder-Phillips (four of whom were JMC stu- dents). This commission met almost daily from April 17 until the issuance of their report on May 15, 1970. 145 On April 30, 1970, Richard Nixon announced the Cambodian "incursion" which led to nation-wide protest centering particularly on large uni- versity campuses like MSU. On May 4, 1970, students at both Kent and Jackson State were killed. The resulting strike and Vietnam Teach-out1 coupled with the Commune and Commission activities made carrying on the normal business of Justin Morrill virtually impossible. More than a few faculty members commented at the end of Spring Term, 1970 that they had memoranda and correspondence on their desks which had gone unread since mid-April. In those piles of unread literature, of course, were several items concerning the newly adopted innovation, written evalua- tion. The inability to get messages into the communication network and JMC's basic familiarity with the ungraded learning idea assured that few persons would recognize the full implications of adopting the specific written evaluation system being proposed. The general non-response from the JMC community permitted the authority structure (consisting in this instance of the Dean, the College Advisory Council and the Curriculum Committee) to assume that most persons in JMC supported the proposal and authorize the decision to adopt. Thus, under the given circumstances (high involvement in other issues, no negative feedback), the presence of a power concentration within the JMC authority structure enabled a deci— sion to be made in the absence of complete agreement. 1Several JMC faculty and many students spent innumerable hours de- veloping historical information on the Indo-Chinese War and going door- to-door in East Lansing and Lansing to discuss the war with community residents. 146 In sum, several organizational and environmental factors facilitated the adoption of written evaluation in spite of the innovation's incom- patible and negative aspects. And, although several faculty and students resented the fact that the decision had been made during a term with so many distractions, most accepted the decision and seemed quite willing to help implement the new system. This general acceptance may well have been due to the recognized legitimacy of the authority structure, the knowledge that every effort had been made to solicit opinions and advice, the lack of awareness of the behavioral implications of the grading model, and the sense that the innovation was indeed compatible with the experimental, personalized nature of Justin Morrill. Implementing Written Evaluation On May 18, 1970, the Chairman of the Curriculum Committee gave the Dean a list of tasks that needed completion if the system were to be fully operable during Fall Term 1970. The list is reprinted in its entirety since it gives a much more complete picture of the complexity of the proposed system than any description which was generally avail- able before the decision to adopt: 1. Statements must be prepared for publication in a. the MSU catalog b. JMC brochures or literature 2. A statement must be prepared to accompany the MSU transcript. 3. Some effort must be made to interpret this system to a. MSU Admissions officers b. High school counselors (through whatever channels available to us) c. The departments on campus most frequently involved with teacher certification of our students \l 05 01 h o 12. 147 d. Part-time JMC faculty-~especially those who have not beenlin the college during the discussion of the pro« posa Summer orientation sessions should include interpretation of the system for in-coming freshmen. The Assistant Dean should inform transfer students of the system. A letter should be sent to parents of present students explaining the system. The committee suggests a separate letter sent to current JMC students during the summer, notifying them of action taken on the proposal. Procedural arrangements must be completed with a. Honors College b. Scholarship Office c. Teacher Certification Office (State of Michigan) Clarify status with Phi Beta Kappa Prior to the end of fall term, the Evaluation Form needs to be approved and printed. Prior to the end of fall term, the form of the Profile must be completed, approved and printed. This Profile may be needed a. when a student transfers out of JMC b. when a student applies for scholarship or loan c. when a student graduates ALL JMC faculty must understand procedures and uses of the forms. The committee would encourage further discussion of the system.1 The above listing covers the steps necessary to initiate the system but does not detail the work needed to maintain it; all steps were eventually taken.2 During Fall Term, 1970 the faculty approved a modi- fied version of the evaluation form3 and developed individual program and course objectives so that they could be preprinted on the forms. Even though the faculty were not accustomed to writing objectives, they 1Barbara Ward, Memorandum to Dean Rohman. 2Some steps (modification of the JMC literature, developing the pro- file form and finalizing arrangements with Honors College) were not completed until after Fall Term, 1970. 3See Appendix 3. 148 worked relatively independently of one another with the exception of faculty teaching in the same programs (e.g., Inquiry and Expression, Field Study). As the form analysis at the end of the present chapter will show, the failure to reach agreement on how objectives and course descriptions were to be written led to uneven use of the form by faculty. Other than the letter which they received from the Dean during the summer, students depended primarily on individual instructors and ad- visors to orient them to the new evaluation system. But due to the low amount of internal discussion among faculty preceding adoption, most faculty were just getting used to the system themselves and unable to offer very clear or consistent guidelines. The instructions to students and faculty that comprised part of the 4/23/70 proposal were available, but these were rather general and certainly did not tell students how to write personal objectives or evaluate their learning experience in light of them, a task which few had probably ever attempted. In addi- tion, since the forms were being developed during Fall Term, many faculty were unable to distribute them until it was time for students to complete the voluntary self-evaluations. Thus, in all but a few instances, the student attempt at using the new system was preceded by only the most general of discussions. Although most faculty and students were attracted to the concept of written evaluation and willing to try the new system, it soon became clear to the members of the Curriculum Committee how little most persons knew about the Justin Morrill model. The Chairman and others did their 149 best to recount the adoption process, reiterate the rationale and encour- age faculty to share ideas on how best to implement it during the first term, but some faculty felt that the system should not have been approved without additional internal discussion of its merits.1 To aid the implementation process, the Dean arranged two meetings for part-time faculty to discuss with them the evaluation form, and he wrote a provisional glossary for the student learning objectives listed on the evaluation form. It was his hope that the glossary would provide, "minimum definition to each of the seven goals in order that all faculty (and students) [would] have a sense of some common 'ballpark.'"2 As the faculty struggled to write course descriptions, develop and rank in order of importance program and course objectives, itemize and give priority ranking to the bases for evaluation, check off the student's level of performance on all relevant objectives, write an evaluation of the student's performance based on college, program and course objectives and explain the new evaluation system to students, there were grumblings of discontent and occasional screams of anguish. In addition, it was evident that copies of the forms would be needed for both the Dean's file and the student's advising folder, and the cost of duplicating a three page form would be prohibitive. ISee Minutes of the College Advisory Council, lO/15/70 and 10/22/70. 2Dean's memo to JMC faculty, 12/4/70, see Appendix 7. 150 By the end of Fall Term, 1970, the Associate and Assistant Deans 1 and a set of were working on a less complicated evaluation form explicit instructions which would be given to all faculty.2 Although some members of the advisory system felt it unwise to modify the form before another term's trial, the financial arguments and the frustra- tion of many faculty made the Dean feel he should act promptly. After a faculty seminar discussion in which most faculty expressed a prefer- ence for the modified evaluation form designed by the Associate and Assistant Deans. the Dean approved the new form for use in Winter Term, 1971. There were some complaints about the fact that the new form was never formally approved, but the Advisory Council concurred with the Dean in viewing all versions of the form as experimental and therefore not requiring formal approval so long as there had been the opportunity for discussion.3 Of course, the new form made it more difficult to achieve two of the original objectives--to have students establish specific goals for taking a course and evaluate their performance in light of those objec- tives, and to stress the integrated nature of the curriculum by stating student learning, program, and course objectives in the same format. These were not consciously rejected by the faculty, but rather, over- looked in their desire to simplify what had become an extremely tedious process. 1See Appendix 3, evaluation form used from Winter 1971-Winter 1973. 2See Appendix 8. 3See Minutes of the Advisory Council, 1/21/71. 151 The first of two sets of opinion surveys were distributed during Winter Term, 1971 to sample faculty and student opinion. The faculty response was disappointingly low (16/45), but yielded some interesting results. Most felt the new system permitted more thorough evaluation (12/16) but was much more time consuming (15/16). Few felt that the system had had any impact on the organization of their courses (5/16) or desired to continue separating program and course objectives (5/16). In general, they favored the written evaluation paragraph (ll/16) over the check-off section (O/l6), and disliked the original form's com- plexity.1 They divided evenly over the question of whether or not they perceived any change in student attitude and performance. In the student sample (50), many (23/50) felt their instructors liked the new system, though others (12/50) remarked that faculty Opinion varied among instructors, and some (7/50) perceived their instructors as confused by the evaluation process. Most students found the new system more thorough (38/50), preferred the written evaluation paragraph to the check-off section (42/50), and thought there was merit in student self-evaluation (43/50). A minority (12/50) felt that their course performance had been affected in any way by written evaluation. Of this group, six reported that they worked harder on research and out-of-class projects and four said they worked less in uninteresting classes. IIn response to the open ended question, "What is the most serious difficulty with the system thus far?" most answers complained of the form's complexity and the time consuming nature of the system. 152 The wide variation in both student and faculty perceptions and attitudes would indicate that the use of the new evaluation form was less than adequate. This estimation is confirmed by the analysis of the Fall, 1970 evaluation forms at the end of the present chapter. Due to the delay in devising and printing the new evaluation forms, the faculty did not distribute them until the end of Winter Term. Once again, then, most students were not oriented very well to the new sys— tem, and the process was made more confusing to some by the introduction of a new form. Unfortunately, the practices of the first two terms established a pattern of delay which many faculty followed through Winter Term, 1973. In many classes, the primary vehicle for encouraging discussion of the system's objectives, its mechanics, student learning objectives and course objectives--the form itself--did not reach the student until the course was almost complete. Indeed, this practice was condoned and encouraged by the instructions for faculty which were 1 and some faculty to this distributed at the beginning of each term, day view the form merely as a record-keeping device, irrelevant to course needs. During Spring Term, 1971, the evaluation system continued to func- tion much as it had during Winter. A faculty sub-committee met to create a form for writing profiles and a profile policy.2 In essence, 1The instructions (see Appendix 8) note that item 1 (name, student number, etc.) may be completed when the student writes his evaluation at the end of the term and that the primary purpose of the course descrip- tion is "to provide anyone reading this evaluation x years from now with some idea as to what the course was about." 2 See Appendix 9. 153 the committee recommended that profiles be written at the student's request if he had more than six written evaluation forms on file. In instances where there were six or less evaluations, the student could ask the college to send capies to an agency in lieu of having a profile written. The subcommittee recommended further that,"the college hire an outside person to write profiles, in order to attain the maximum degree of objectivity and consistency,"] and that this person use the student learning objectives as an organizing scheme for the profile. Although the recommendations were never formally approved by either the Advisory Council or the Dean, the Assistant Dean, a member of the sub-committee, used the guidelines as students began requesting profiles. Until Fall 1972, when JMC hired an outsider to write pro- files, they were written by a staff assistant, the student's advisor or the Assistant Dean. In the Fall of 1971, the Dean hired a staff assistant and asked him to assume the task of gathering faculty and student opinion toward the written evaluation system, so that a report could be prepared to present to the University Curriculum Committee in February, 1972. At that time, JMC would have three avenues Open to it. It could inform the University Curriculum Committee that it wanted to return to the numerical grading system, it could request a continuation of the experi- mental period so that more data could be gathered or it could request that JMC be permitted to adopt written evaluation on a permanent basis. 1Recommendations on the Writing of Profiles, p. 2. 154 The surveys1 revealed some interesting implementation problems. Use by faculty continued to be uneven, through Fall 1971, the fourth term in which written evaluation had been in effect. Although most concurred that written evaluation permitted more thorough evaluation of student performance (31/40), many had not modified their instruc- tional approach in any way (14/40), most (37/40) spent far more time evaluating students, many did not list course or program objectives on the form (16/40), half (20/40) did not use the form to review college or course objectives with their students, and some felt there was still confusion on the part of some faculty and students as to the objectives of written evaluation (6/40).2 In addition, many students participating in the student survey viewed their instructors as confused by written evaluation (37/82), almost as many reported that their instructors did not discuss the purpose of written evaluation (20/82) as reported that their instructors did (26/82), and more than a third said that instruc- tors did not emphasize the JMC learning objectives (33/82). Finally, anecdotal evidence available by the end of Fall Term, 1971, revealed that some faculty were writing evaluations which were no more informa- tive than a numerical grade, others were not giving the evaluation forms to the Assistant Dean so that they could be duplicated and filed, and still others were having great difficulty in writing adequate evaluations.3 1See Appendix 2. 2Response to open ended question, "What did you find the most serious difficulty with the system so far?" 3See Chapter III for sample details and Appendix 2 for results. 155 The student survey (82/100) respondents) revealed wide variation in the opinions and attitudes of JMC students. As many reported that they were less motivated (22/82) as said they were more motivated (21/82) by written evaluation. Half reported they worked less in uninteresting courses (41/82) whereas more than a third believed they were learning more under written evaluation (30/82). A significant minority (25/82) felt that they worked harder in graded courses, but the vast majority expressed a preference for some form of a non-graded evaluation system (64/74). Several questions on the survey revealed an ambivalent student attitude toward the utility of the JMC student learning objectives, the check-off process of evaluation and the student self-evaluation.1 In addition to the varied results of the student opinion survey, anecdotal evidence available during Fall Term, 1971 suggested that many students were not writing self evaluations or check- ing off their level of performance on the relevant student learning objectives. In essence, the two surveys indicated that use of written evalua- tion in JMC was no more uniform in Fall Term, 1971 than it was during Fall, 1970.2 In fact, at least in the eyes of students, faculty varied even more in their commitment than they did in Fall Term, 1970. The level of frustration had diminished markedly, however, with the introduc- tion of the new form and the greater routinization of procedures. 1See questions 7, 8 and 11 on the student opinion survey, Appendix 2. 2This hypothesis is confirmed by the form analysis at the end of the present c apter. 156 By Winter 1972, written evaluation was not a prominent issue of discussion in Justin Morrill. Instead, four other topics were vying for faculty and student attention: 1) A new governance model which had taken all Fall Term to design was adopted at the beginning of Winter Term, 1972; 2) The Provost's evaluation of JMC began during Winter 1972, and teams of students and faculty spent long hours gather- ing data for two separate reports. In addition, the Dean was writing his own analysis of the history of JMC since 1965; 3) During Winter Term, 1972 JMC tried to develop a goals based planning model for the college. The attempt entailed weekly two to three hour meetings which continued into Spring Term; 4) After extensive internal discussions and meetings during Winter Term, 1972, JMC adopted a modular scheduling plan for the 1972-73 academic year. After Fall Term, 1971, the Dean's staff assistant made two attempts to promote discussion about written evaluation among faculty and to create a greater commitment to using the system as designed. In the first attempt on January 13, 1972, the faculty (19 of 35), the student members of the Advisory Council and some members from the newly created Student Evaluation Team met to discuss the results of the two opinion surveys reviewed above and the uneven use of written evaluation that they revealed. By the end of the discussion all those present agreed that JMC should ask the University Curriculum Committee for a three year extension of the written evaluation experiment in order to, "obtain more data on the use of written evaluation internally and to determine the effect of written evaluation on seniors who have taken 90-100 credits 157 within JMC as they seek employment or entrance to graduate schools."1 In addition, all those present recognized the need for internal agree- ment on such issues as: the relation of JMC learning objectives to course objectives, the use of the written evaluation form in class, the relationship between written evaluation and instructional style or course design, and the question of how consistent faculty need be in using written evaluation. Unfortunately, the press of time and other issues (see above) prevented those at the meeting from either identify- ing apprOpriate courses of action or setting aside time for future discussions. Unable to find any available time earlier, the Dean's staff assistant arranged a meeting on May 30, 1972 to discuss some of the issues raised during the 1/13/72 meeting. In spite of the long interval between meetings, the timing for a discussion seemed appropriate since faculty would soon be writing evaluations for students in their Spring Term courses, and as he stated in a note to faculty, it was more im- portant than ever that they: -share ideas on how to use the forms and evaluate the student; -decide which goals [they] want to pursue and which [the ] would prefer to abandon or de-emphasize as a result of their] experience; -come to some agreement as to where [they] want to be consistent in [their] use of the system and where [they] want to allow for variation. . . . 1Prggress Report on the JMC Writte9_Evaluation System given to the University Curriculum Committee on 2/15/72. JMC did make such a request on 2/15/72 and received approval in May, 1972. 2Neil H. Cullen, memo to faculty on written evaluation. 158 Twenty faculty were able to attend the session. To begin, the staff member writing profiles for students applying to Honors' College offered a series of quite specific recommendations for improving the use of the evaluation system as well as a set of "good" and “bad" evalua- tions.] The faculty then discussed the system's objectives as described in the original written evaluation proposal,2 but were unable to agree on either their priority ranking or how they should be interpreted for current use. One person, stating the thoughts of several others, noted that the faculty who were free to meet on May 30 had no authority to make decisions anyway, so it might be wise to postpone any efforts to improve the written evaluation system until Fall Term. Those attending the meeting did agree to take the profile writer's advice under con- sideration as they wrote Spring Term evaluations, but did not feel that an endorsement of those present should have a binding effect on all faculty. Thus, although, many faculty were more aware of the problems associated with written evaluation, there existed no more agreement on goals or procedures than before the meeting. Trying to identify a possible course of action before the end of the school year, the Dean's staff assistant distributed a summary of the meeting that same day encouraging all faculty to adhere to the pro- cedural suggestions that most faculty at the meeting had supported. Specifically, he urged them to complete all sections of the form even if 1 2 See Appendix 10. See Appendix 5, Rationale, items 1-8. 159 it meant using phrases like 'no comment' or 'do not know', write a course title 999 a course description, weight the bases for evaluation [e.g., 2 short papers (25%), 1 final exam (50%), class participation (25%)], evaluate student performance on the basis of stated course objectives, encourage student self-evaluation, and hand the completed forms to the Assistant Dean 999999 the beginning of the subsequent term. In addition, he urged faculty to give course descriptions and objectives to his secretary before the beginning of classes so that they might be preprinted on the evaluation forms and used the first day of Fall Term classes. No one followed the suggestion. In the Fall of 1972, the Dean's staff assistant coordinated a major effort to clarify the major goals of written evaluation and improve the system's use. However, once again written evaluation had to compete for the attention of faculty with other issues of major sig- nificance: the new governance model was not entirely satisfactory and was undergoing modification, the first modules (fifteen in number) were beginning and proved very time consuming, and the Provost's evalu- ation team was completing the JMC Evaluation Report. In an attempt to break the pattern of distributing written evalu- ation forms at the end of the term and to encourage faculty to write thorough course descriptions and objectives, the Dean's staff assistant sent the following memorandum to faculty on 9/18/72: . . . please be certain to discuss how you intend to use the evalu- ation forms in each of your classes early in the term. Such a discussion will familiarize the students with your particular goals for evaluation as well as the more general college goals for written evaluation. It is essential that freshmen and non-JMC 160 students learn about the system in each class they take in order to learn the difference between grading in JMC and the university. If you wish either your course description or your course objectives typed on the back of the evaluation forms for your classes, please give the write-up to Mrs. Rhines in 151 Snyder and she will modify the form for you. At his urging, more faculty did distribute the forms on the first day of classes to review with students the objectives of written evalu- ation, and one faculty member had Mrs. Rhines type course objectives on the written evaluation forms for his classes. Immediately after JMC had approved the amended version of its temporary governance model, the Dean's staff assistant chaired a dis- cussion on written evaluation in the Faculty Seminar (10/27/72). Twenty of thirty faculty were able to attend, and the discussion centered around the issue of whether the faculty wanted to continue, abandon or modify the written evaluation system in use Fall Term 1972. Most of those present concurred that the system had not yet been given a fair test due to the continuing distractions of an innovative college, and that if JMC were to keep written evaluation, the model would need streamlining. A sample evaluation form1 was distributed and discussed as a possible means of improving the system and lessening the time burden on faculty. As noted in the summary of the meeting2 (distributed to all faculty and the student members of the advisory system), use of the sample form would have several advantages: 1) The form contains a 1See Appendix 11, evaluation form confirming all 3 objectives. 2Appendix 11. 161 thorough course description which would be informative for students and the profiler; 2) The faculty member would state the student learning objectives stressed in a particular course and specific course objec- tives, thus emphasizing the interrelationship between his course and the total curriculum; 3) The bases for evaluation are linked with specific objectives, thus clarifying the instructor's expectations for student performance; 4) Preprinting the course description, the ob- jectives and the bases for evaluation on the evaluation form would save faculty time at the beginning of the term. In addition, both faculty and students could refer to them while writing evaluations, thus saving time at the end of the term. In some cases, a faculty statement as brief as "has met all my stated objectives'I might suffice; 5) Since part-time faculty would also write descriptions, objectives and bases for evaluation, their use of the form might improve; 6) The forms would be available for the first day of classes to encourage discussion of course objectives and the goals of the written evaluation system. Those present at the 10/27/72 seminar also discussed the problems associated with profiling and the fact that JMC still had not hired someone from outside the college to write the profiles for graduating and transferring students. Two faculty had been writing profiles for students applying to Honors' College, but they could not assume the task for graduates and transfers as well. The Assistant Dean agreed to continue handling the profiles on an 99_999_basis until the issue had . been resolved by the Dean and the Advisory Council. 162 Students and faculty were invited to respond to the written evalu- ation form which was distributed along with the 10/27/72 meeting summary. Few did respond, however, so the staff assistant decided to introduce the suggested form and rationale to the formal advisory struc- ture to promote internal discussion. He asked the Steering Committee1 to make a series of recommendations to the Advisory Council so that a decision could be made regarding modification of the written evalua- tion system before Winter Term, 1973. The staff assistant explained to the Steering Committee that in order to improve written evaluation, JMC needed to: modify the form to include better course descriptions and objectives and to make it easier for faculty to use, reduce the original eight rather general system objectives to three more specific objectives so that the students and faculty could keep them clearly in mind, and reaffirm a set of student learning objectives which could serve as an integrating force both for written evaluation and for the JMC curriculum. The Steering Committee found the rationale for change sound, and distributed the proposal included as Appendix 12 to Advisory Council members2 on 11/20/73. The proposal urged the Advisory Council to approve the sample evaluation forms described above as a replacement for the one in use 1n.b., normally, the Advisory Council as described earlier in the present chapter. However, during much of 1972-73 JMC was Operating a temporary, consensus model governance system. The Steering Comittee cleared issues for the Advisory Council which consisted of all full- time JMC faculty (30) and twelve students. The Advisory Council and the JMC Dean approved educational policy at the college level. 2n.b.. for Fall Term 1972, all JMC faculty and 12 students. 163 Fall Term, 1972; to reaffirm the three major objectives of written evaluation itemized in Chapter I, i.e., to aid communication between instructor and student, to highlight specific course and college objec- tives, to provide a summary of a student's work in JMC; and to endorse a statement confirming the college's commitment to a set of student learning objectives as well as particular content requirements. The statement was similar to the one discussing student learning objectives early in the present chapter and noted that the objectives served as a unifying force for both the written evaluation system and the JMC curriculum. To facilitate the discussion process, the staff assistant asked each member of the Steering Committee to talk with specific faculty members to obtain their responses to the proposals and to solicit any suggested changes. In addition, the Steering Committee approved two steps which would precede the 11/30/72 Advisory Council meeting: 1) Each faculty member would take 5-10 minutes in at least one of his classes prior to November 28, 1972 to discuss the sample evaluation form with JMC students. A list of specific questions was given the faculty; 2) The Advisory Council would meet on November 28, 1972 to discuss the student response and modify the suggested form if necessary. The staff assistant also outlined in detail the steps that would need to be taken if the Advisory Council approved the proposal on 11/30/72.1 These steps were included as part of the proposal which the 1See Appendix 12 for the specific steps described. 164 Steering Committee distributed on 11/20/73 so that faculty would know how the system change would affect them. Thus, prior to the approval of any changes, every attempt was made to familiarize JMC faculty and students with the consequences of system alteration and to have them participate in the change process. Participation among faculty and students on the 1972 Advisory Council was high, but several points need to be stressed: 1) Only one-third of the faculty picked up sample evaluation forms to distribute to their classes for discussion. Thus, many JMC students were not familiar with the proposed changes; 2) The suggested changes in written evaluation were not only competing, with modular scheduling and the Provost's' evaluation for faculty and student attention, but also with the very emotional issue of student led courses. Once again, written evaluation was of less significance than the crisis of the moment; 3) On 11/30/72, the JMC faculty and the twelve students on the Advisory Council unani- mously endorsed the three Steering Committee proposals regarding written evaluation. However, the discussion of written evaluation came at the end of an enervating, three hour, evening meeting. Even though the amount of preceding discussion and participation undoubtedly facilitated approval, it probably would have taken a very controversial issue to have postponed adjournment much longer that evening. Early 1. Winter Term, 1973, the JMC faculty (17 of 30) and two part-time faculty met to discuss the sample course descriptions and objectives which several faculty had written during the holidays. This discussion was held to aid faculty in writing their Spring Term course 165 descriptions and objectives which were due at the end of January. On the basis of this meeting the Dean's staff assistant developed a series of eleven procedural guidelines which faculty could use in developing their course descriptions and objectives for Spring Term.1 Essentially these guidelines reviewed all the steps necessary for faculty to assure that they would have enough evaluation forms at the beginning of Spring Term for each of their Spring Term classes. JMC hired a part-time profile writer at the beginning of Winter Term to start writing profiles on a demand basis. The Dean's staff assistant and the Assistant Deen developed a revised version of the profile form which appears in Appendix 4. This procedure relieved both the Assistant Dean and academic advisors of the time consuming process of summarizing from seven to twenty-five written evaluations for pro- files. All but two faculty members teaching in JMC Spring Term 1973 created individualized evaluation forms for their Spring Term classes. When distributing these forms to the faculty at the end of Winter Term, the Dean's staff assistant wrote a covering memorandum encouraging faculty to follow similar procedures in using the forms so that students would be clear as to the objectives of written evaluation.2 In Spring Term, when faculty were preparing course descriptions for the subsequent Fall, the Dean's staff assistant led a Faculty Seminar devoted to reviewing the course evaluation forms which had been 1See Appendix 13. 2See Appendix 14. 166 developed for Spring Term and identifying the characteristics common to the better forms. The discussion resulted in a set of guidelines which faculty could use when writing course descriptions, course objectives and bases for evaluation.1 At the end of the seminar, the staff assistant distributed a set of questions to aid faculty as they wrote their Spring Term student evaluations.2 During Spring Term the Assistant Dean and the Dean's staff assis- tant devised a procedure whereby, at the beginning of each term, both full and part-time JMC faculty will receive a set of explicit guidelines along with a request for course descriptions, objectives and bases for evaluation. In addition, the Dean and a small group of JMC faculty will meet with the part-time faculty at the beginning of each term to review the purposes and procedures associated with written evaluation. In examining the implementation process as a whole up to the present point in time, several facts are clear: 1) The behavioral impli- cations of the JMC written evaluation system were not very clear either at the point of adoption or in Fall Term, 1970 when the implementation stage was in progress. In Neal Gross' phrase, no one provided a clear picture of the "new role demands associated with the innovation." The lack of participation and discussion prior to adoption permitted the proposal to pass with fairly general guidelines for students and faculty. To use the change agent typology of Ronald Havelock, there were no 1See Appendix 15. 2See Appendix 1, questions 6-9. 167 vocal "defenders' to insist on a detailed explanation of the innova- tion prior to the adoption decision; 2) The innovation continues to be compatible with JMC's desire for individualized and thorough evalua- tions but incompatible with the faculty desire of autonomy and inde- pendence with regard to instruction. In addition, the norms of informality and the loosely defined role structure may be even further strained as attempts are made to gain greater uniformity in using written evaluation. As Henry Brickell would note, the innovative qualities of the JMC environment may well not be appropriate for imple- menting and evaluating the effectiveness of written evaluation; 3) Nurturance and maintenance of the innovation have at best been inter- mittent. During the first two terms, the Dean, the Associate Dean and the Chairman of the Curriculum Committee attempted to facilitate implementation. During the next four terms few efforts at improvement were made, and those that were, were ignored due to more pressing problems. During the 1972-73 academic year, nurturence and maintenance efforts increased markedly, but the majority were what Robert Chin would call "rational-empirical," i.e., they relied heavily on the written memorandum and rational argument and minimally on retraining, restruc- turing or reeducative efforts. For instance, faculty did not receive much guidance in the writing of objectives and students were not trained to engage in self-evaluation. The assumption prior to 1973 was that both could do the task without further training; 4) Written evaluation has, since its inception, been competing for the attention of faculty with numerous other innovations, crises and the normal task demands of 168 an undergraduate college; 5) During the 1972-73 academic year more efforts were made to have the college system support the innovation (e.g., the preprinting of forms, the creation of clear guidelines). However, one of the most important aspects of the organization, the reward system, was not activated in support of written evaluation. Prior to Spring Term, 1973, use of written evaluation was not very visible; neither good nor bad efforts were seen by anyone other than the individual student, the Assistant Dean and the profile writers, and they made their observations public only occasionally. Stated in organizational behavior terminology, Justin Morrill had not established an adequate operational feedback mechanism. As a result, a faculty member's efforts at written evaluation had minimal effect on the deliber- ations of the Personnel Committee, his worth in the eyes of the Dean, his status among his peers, or his reputation as a teacher. From the preceding discussion, it is obvious that JMC is using written evaluation under somewhat difficult circumstances. It now remains to determine the degree to which the college has succeeded in implementing the written evaluation system it adopted in the Spring of 1970. The Degree of Implementation As was explained in Chapter III, a variety of measures will serve as indices of the implementation of written evaluation in Justin Morrill: the evaluation form completion analysis, the percent of evaluation forms on file, the number of students who requested profiles, the number of 169 students who read the completed evaluation forms and some results from the 1971 attitude surveys. However, the form analysis will be the primary index of implementation. A special instrument entitled, "Categories and Questions to Analyze the JMC Written Evaluation Form” (see Appendix 1) was developed for the form analysis, and the general results from this instrument are dis- played in Tables 4.1-4.8. Table 4.1 shows the number of "yes" and "no" responses for each question in each term the form was in use between Fall Term, 1970 and Winter Term, 1973 and also illustrates the grand totals for each question. In addition, Table 4.1 gives the mean faculty and student completion scores for each term as well as the grand means. The faculty completion score is determined by totalling the number of yes responses to questions 1-9 and can range from zero to nine. The student completion score is computed by adding the number of yes responses to questions 10-15 and ranges from zero to six. Finally, the last row of Table 4.1 gives the number of P grades awarded under the "yes" column and the number of N grades awarded under the "no" column as well as the grand totals in these categories. Table 4.2 parallels Table 4.1 exactly, giving the frequency counts for each raw score in Table 4.1 excluding the faculty and student mean completion scores. Whereas Table 4.1 displays the raw scores for the various items in the analysis instrument by term, Tables 4.3, 4.5 and 4.7 show the raw scores by faculty status, knowledge area and class respectively. Tables 4.4, 4.6 and 4.8 show the corresponding frequency counts for faculty status, knowledge area and class. Tab1e 4.1. 170 Raw completion scores by term for the categories and ques- tions used to analyze the Justin Morrill College written evaluation form. -—-—..--- Questions & Categories ... -l —‘ + —J on + + tomVOlU'l-th—i ----n on _a N (1)—1...: GUI-D ‘- G) 5'71 Terms F'71 W'72 5'72 F'72 H'73 Totals Y N Y N v N ’ Y N Y N Y N 19 28 18 31 17 29 19 30 26 22 23 27 19 164 225 31116 33 16 31 15 35 14 39 9 41 9 37 289 100 “*1 9238 12 37 16 30 12 37 8 40 13 37 8 42 109 280 16 1 35112 42 34 12 35 14 38 10 35 15 37 290 99 42 8 421 5 46 41 5 44 5 40 8 46 4 44 6 345 44 30 20j27 23 26 25 21 27 22 23 25 18 32 30 186 203 23 21126 21 28 17 29 26 23 19 29 26 24 26 183 206 33 30117 37 12 34 12 39 10 32 16 38 12 43 7 286 103 21 17; 29? 15 32 26 23 23 23 16 33 20 28 31 19 16 34 168 221 5.5 g 4.7 .3 5.2 5.1 5.1 5.4 5.2 5.2 27 23‘ f 30.17 32 17 26 18 31 23 25 25 25 23 27 198 191 28 22? 29 18 32 17 29 17 32 24 24 27 24 194 195, 6 44. 6 41 40 4,42 4 45 5 43 7 43 20430 7‘43 48 L341 27 23124123 27 22 1 14'32 16 33 19 29 22 28 13,37 1629227 14 36 12 35 10 39 10 36 11 38 10 38 11 40 871302 7 43 8 39 42 6 40 6143 8 40 11 39 10140 4146 571332 2.2 2.3 2.4 1.5 1.5 1.9 2.0 1.5 1.9 431 2 411 2 47 l1 311 4 431 21 38 3 44 1 21 431 3 3361 19 Key: F - Fall W - Winter S - Spring + . D1fference among terms Yes No Grade FC - Mean Faculty Completion Score SC - Mean Student Completion Score * - 34 Missing Values significant at .05 level according to X tests--see text for details. 171 Table 4.2. Frequency of completion by term for the categories and ques- tions used to analyze the Justin Morrill College written evaluation form. Terms Questions & Categories + \OCDNOU'l-FwN-d + m n + + dd—ldu—Ja—l macaw—4o a. U) 53 ('3 F'7O H'71 5'71 F'71 H'72 5'72 F'72 H'73 Y N Y N Y N Y N Y N Y Y N Y N Totals Y 46 54 4O 60 37 63 37 63 39 61 54 46 46 54 38 62 42 58 84 16 66 34 67 33 67 33 71 29 81 19 82 18 74 26 74 26 62 38 19 81 25 75 35 65 25 75 17 83 26 74 16 84 28 72 68 32 75 25 86 14 74 26 71 29 79 21 7O 30 74 26 74 26 84 16 89 11 94 6 89 11 9O 10 83 17 92 8 88 12 89 11 4O 6O 43 57 47 53 54 46 55 45 48 52 36 64 6O 40 48 52 54 46 45 55 43 57 37 63 53 47 40 6O 52 48 52 48 47 53 66 34 64 36 75 25 74 26 80 20 67 33 76 24 86 14 74 26 42 58 32 68 53 47 50 50 33 67 42 58 62 38 32 68 43 56 54 46 64 36 65 35 44 56 37 63 48 52 50 50 46 54 51 49 56 44 62 38 65 35 37 63 35 65 50 50 54 46 4O 60 50 50 12 88 13 87 18 82 9 91 18 92 10 90 14 86 14 86 12 88 54 46 51 49 55 45 30 70 33 67 40 6O 44 56 26 74 42 58 28 72 26 74 20 80 22 78 22 78 21 79 20 8O 20 80 22 78 14 86 17 83 14 86 13 87 12 88 17 83 22 78 8 92 15 85 96 4 195 98 2 90 101 96 4 93 7 96 4 94 6 95 FC - See Raw Score Table 4.1 W - Winter N - No SC - See Raw Score Table 4.1 S - Spring G - Grade * - Adjusted Frequency +Difference among terms significant at .05 level according to X2 tests--see text for details. Kay: F - Fall Y - Yes Faculty Status Faculty Status 172 Table 4.3. Raw completion scores by faculty status for the categories and questions used to analyze the Justin Morrill College written evaluation form. rf_ r__ 3 Questions & Cate ories I ~k a: if T v: ‘ 1 2 3 4 5 7 8 9 FC 10 ll 12 l3 14 15 SC F Y 117 204 89 200 240 132 149 210 127 6 0.133 127 34 107 47 39 2 0 N 126 39 154 43 3 111 94 33 116 111 116 209 136 196 204 P Y 47 85 20 90 105 54 34 76 41 4 9 65 67 14 55 4O 18 2 3 N 65 27 92 22 7 58 78 36 71 47 45 98 57 72 94 T Y 164 289 109 290 345 186 183 286 168 5 7 198 194 48 162 87 57 2 1 N 191 66 246 65 10 169 172 69 187 57 161 307 193 268 2981 Key: F - Fulltime JMC faculty FC - Mean faculty completion score P - Parttime JMC faculty T - Totals Y - Yes N - No * Difference between full and parttime faculty significant at .02 level and below. SC - Mean student completion score Table 4.4. Frequency of completion by faculty status for the categories and questions used to analyze the Justin Morrill College written evaluation form. 1 Questions 5 Categories = 9: * T i 1: a i l 2 3 4 5 6 7 8 9 FC 10 11 12 l3 14 15 S . F Y 48 84 37 82 99 54 61 86 52 _ 55 452 14 44 19 16 _ N 52 16 63 18 1 46 39 14 48 45 48 86 56' 81 84 P Y 42 76 18 80 94 48 3O 68 37 _ 58 6O 13 49 36 16 _ N 58 24 82 20 6 52 70 32 63 42 4O 87 51 64 84 T Y 46 81 31 82 97 52 52 81 47 _ 56 55 14 46 25 16 _ 1N 54 191 69 18 31 48 48 19 53 44 45 86 54 75 84 Key: F - Fulltime JMC faculty FC - See raw score table 4.7 P - Parttime JMC faculty SC - See raw score table 4.7 T - Totals Y - Yes N - No * Difference between full and parttime faculty significant at .02 level and below. 173 Questions & Categories Table 4.5. Raw completion scores by knowledge area for the categories and questions used to analyze the Justin Morrill College written evaluation form. Knowledge Area Nat.Sci. Soc.Sci. Human. Lang, I & E Misc. Totals . Y N Y N Y N Y N Y N Y N Y N +1 32 36 19 49 43 68 32 7 19 28 19 3 164 191 +2 54 14 53 115 96 15 37 2 29 18 20 289 66 +3 8 60 20 148 20 91 33 6 13 34 15 109 246 +4 55 13 56 112 89 22 36 3 45 2 9 13 290 65 5 65 3 6533 110 1 38 1 46 21 1 345 10 +6 27 41 34 '34 67 ,44 15 24 31 16 12 10 186 169 +7 11 57 3O 38 50 161 34 5 41 6 17 5 183 172 +8 42 26 60 8,?98 13 27 12 39 8 20 2 286 69 +9 18 50 34 34,1 54 57 21 18 33 14 8 14 168 187 ch, 4.6 5.4 ' 5.6 7 6.3 ‘ 6.4 5.7 +10 38 3o 39 29: 60 51 31 8 25 22 5 17 1 198 157 +11 37 31 35 133; 72 39 25 14 20 27 5 17 1 194 161 12 7 61 9159 ' 21 90 1 3 36 6 41 2 20 ' 48 307 13 29 39 28 140 58 £53 22 17 19 28 6 16 162 193 14 15 53 15 153 33:78 10 29 1o 37 4 18 87 268 +15 8 6O 7 61 18 193 14 25 8 39 2 20 57 298 SC 2.0 g} 2 O 2.4 2.6 1.9 1.1 2.1 GL62 i64i4105 6 3613 46 o 2210 335j19 Key: Misc. - Ind. Study, Field Study and Senior Seminar FC - Faculty Completion Score Mean SC - Student Completion Score Mean G - Grade Y - Yes N - No +Difference among knogledge areas significant at .016 level or below according to X tests--see text for details. Questions & Categories 174 Table 4.6. Frequency of completion by knowledge area for the categories and questions used to analyze the Justin Morrill College written evaluation form. Knowledge‘Area Nat.Sci. Soc.Sci. Human. Lang, I 8 E Misc. - Totals 9] Y N v N v N Y N r N Y N 1 Y ' N +1 47 53 28 72 39 61 82 18 4o 60 87 13 46 54 +2 79 21 78 22 86 14 95 5 62 38 91 9 81 19 +3 12 88 . 29 71 18 82 85 15 28 72 68 32 31 69 +4 81 1971 82 18 8o 20 92 8 96 4 41 59 82 18 5 96 4 96 4 99 1 97 3 98 2 95 5 97 3 +6 40 6O 50 50 6O 40 39 61 66 34 54 46 52 48 3 +7 16 84 i 44 56 45 55 87 13 87 13 77 23 52 48 3 +8 62 38 E 88 12 88 12 69 31 83 17 91 9 81 19 1 +9 27 73—350 50 49 51 54 46 70 3o ~361_64 47 53 PC? ‘ 3 ' ' ' ' 1 ‘ ' 1 +101 56144157 43 54 46 79 21 53 47* 23 77 56 44 1 +111541916 1 51 49 65 35 64 36 43 57 23 77 55 45 1 +12, 10L991 13 87 19 81 8 91 13 87,y, 9 91,; 14 87 i 13; 433:57 41 59 52 48 56 44 4o 60;_ 27 73 1 46 54 14,_22 78 22 78 [+30 70 26 74 21 79 18, 82 25 75 1 +151 12 188 10 90 1 16 84 36164 17 83: 91 91 1 16 84 i T ‘1 1 1 5C; ' ' ' ' “ ' é ' 95 F—wt ' 01 at f . T ' r 63 91 91 945 6 951‘ 5 9219 8 100 0| 1001* o Key: Misc. - Independent Study, Field Study and Senior Seminar FC - See Table 4.3 SC - See Table 4.3 G - Grade Y - Yes N - No + Difference among knowledge areas significant at .016 level or below according to X2 tests--see text for details. 175 Table 4.7. Raw completion score by class for the categories and ques- tions used to analyze the Justin Morrill College written evaluation form. Questions & Categories * 'k * T i i * * 212 3 4 5 6 7 8 9FC 10111213141550 I Y 1101192 80 203 231 119 131 188 130 5 5 140 139 30 118 58 42 2 1 3 N 142 60 172 49 21,133 121 64 122 112 113 222 134 94 210 ‘9 5'1ng 54,96 29 86114167 52 98 3847 57 5518 44 2815161 1N2 82' 40 107 50,221 699 84 38 98 79' 81 118 92 108 121 1 T'11164 288 109 289'345;186'183]286 168 5 2 197 194 48 162 86 57 1 Q 1 - . T11112241100 279 99 431202 20§llOZ 220 191 194 340 226 302 331 4] Key: I - Lowerclass JMC students FC - Mean faculty completion score 11 - Upperclass JMC students SC - Mean student completion score T - Totals Y - Yes N - No *Difference between classes significant at .04 level or below according to X2 tests-~see text for details. Table 4.8. Frequency of completion by class for the categories and questions used to analyze the Justin Morrill College written evaluation form. 1 Questions & Categories ; i a: * at 1'2 3 4 5 6 7 8 9FC10111213141550 I Y 44i_761 32 81 92| 471 52 75 52 _ 561 55 12I 47123' 171_ .. N? 56 241 68' 19 8 5314825481 44146 88 53 77' 83] m ’ . :5 II Y? 40 71 21 63 34. 49 38: 72 28 _ 42,_4O 13 32 21' 1L:_ 1 N 601 29 79 37 16. 51 62; 28 72. 58' 60; 87 68? 79’ 89 1 T Y 42' 74128 741L89 48 47; 74 431 _ 51 501 129942: 22 15 _ 1 N, 581126172126 5253126 5H 49 50 88158 78 85 1 Key: I - Lowerclass JMC students FC - See raw score table 4.5 II - Upperclass JMC students SC - See raw score table 4.5 T - Totals - Y - Yes N - No *Difference betyeen classes significant at .04 level or below according to X tests--see text for details. 176 Examining the total frequency of completion by question in Table 4.2 gives a fairly clear picture of over-all utilization of the written evaluation form. For only five of the fifteen questions are there more "yes" than "no" re5ponses: "Are there bases for evaluation?" (74% yes); "Has the instructor checked off the student's level of performance on the relevant college and/or course objectives?“ (74% yes); "Has the instructor written an evaluation of the student's performances?" (89% yes); "Has the instructor commented upon agy_of the following types of student activities: class participation, class preparation, work on papers, test performances?" (74% yes); and "Has the student checked off his level of performance on the relevant college and/or course objec- tives?" (51% yes). 0f the five sections of the evaluation form to which the questions refer, faculty are responsible for four and the students, one. Looking at the remaining ten questions, one finds that six have affirmative answers forty to fifty percent of the time: "Is there a course description, i.e., more than a general or specific course title? (42% yes); "Has the instructor assessed the student's performance on at least two general college objectives in his/her written evaluation?" (48% yes); "Has the instructor assessed the student's performance on explicit course objectives in either the grid or the written evaluation?" (47% yes); "Has the instructor commented on both strengths and weaknesses in the student's performance?" (43% yes); "Has the student assessed his/ her performance on explicit course or personal objectives in either the grid or the written evaluation?" (42%). These six questions refer to four sections of the form for which faculty are responsible and two which students should complete. 177 On the remaining evaluation form sections, 28% of the faculty listed specific course objectives (question 3); l2% of the students assessed their performances on at least two college objectives (ques- tion 12); 22% of the students commented upon class, paper or test performance (question 14) and 15% of the students remarked on both strengths and weaknesses in their performance (question 15). If one examines the adjusted frequencies] in the total column of Table 4.4 for the same set of questions, the basic pattern remains the same though three additional questions receive more "yes" than "no" responses.2 In general, faculty completed the form to a higher degree than students, but even among faculty, use of the form was quite uneven. In only three areas (bases for evaluation, writing the evaluation, and commenting on student learning activities) did a clear majority of faculty complete the form as intended. In all but one of the remaining categories (the listing of course objectives), about half of the faculty completed the item adequately and half did not. The faculty and student mean completion scores confirm the above pattern of uneven faculty and student use and the fact that faculty completed the form more thoroughly than did students. The mean faculty completion score for the entire sample is 5.23 out of a possible 9 with 1Table 4.4 reflects the frequencies of form completion with the 34 forms which had each of the fifteen questions answered "no". As was noted in Chapter III, there were 34 instances in which the researcher was unable to find a completed evaluation form for a course taken by a student in the sample. 2Questions 6, 7 and 11. See Tables 4.3 and 414. 3The median for the faculty is 5.6, the mode is 6. 178 a standard deviation of 2.4. For the faculty completion mean scores by term, the standard deviation ranges from 2.7 in Fall 1970 and Spring 1972 to l.8 in Spring Term 1971. Thus, through Winter Term, l973, wide variation in faculty usage continued. The mean student completion score for the entire sample is 1.91 out of a possible 6 with a standard deviation of 1.9. The standard deviation for student completion mean scores by term ranges from 2.0 in Fall 1971 and Spring l972 to 1.7 in Winter l973. From these figures, it is clear that student use of the form also varied greatly and was lower than the level of faculty use. It is also interesting to note that the mode, or most common completion score for faculty in the sample, is 6 whereas the modal re5ponse for students is O. In Chapter III, one generalized null hypothesis related to form completion was stated. G.H. : There is no difference in the degree of completion among the written evaluation forms. Since both students and faculty completed sections of the evalua- tion form, two sub-null hypotheses were developed from the generalized null which in turn provided the bases for a series of research hypotheses. The sub-null hypotheses are: H l: All faculty complete the written evaluation forms to the same degree. H 2: All students complete the written evaluation forms to the same degree. The first two research hypotheses were stated as follows: 1The median for the students is 1.6, the mode is 0. 179 S]: There is a relationship between the degree of faculty completion of the evaluation forms and the term in which the evaluation forms are used. 52: There is a relationship between the degree of student completion of the evaluation forms and the term in which the evaluation forms are used. The results of the two Chi Square tests of independence (see Tables 4.9 and 4.l0) used to test these hypotheses show that there is not a significant relationship between the variable, term and either the faculty or student completion score.1 Thus one cannot reject the null hypotheses stating that there is no relationship between the term of completion and the degree to which faculty and students completed the form. Nonetheless, the distribution of scores among the low, medium and high levels of the completion scores does reflect the familiar pattern of greater faculty than student use of the written evaluation forms. In spite of the lack of statistical association between the variable, term and the faculty and student completion scores, there are significant differences among terms for the responses to questions 3, 9, ll and 13. The raw scores and frequency counts are asterisked in Tables 4.1 and 4.2, and the Chi Square values, degrees of freedom, and significance levels are in Table 4.ll. Since there are both full and part-time faculty in JMC, the follow- ing two research hypotheses were generated to test the relationship 1The original 10 dimension variable, faculty completion score and 6 dimension variable, student completion score were reduced to the 3 di- mensions of low, medium and high. The dimensions were decreased in order to increase the number of scores per cell in the X tables. 180 Table 4.9. Faculty completion score for the Justin Morrill College written evaluation form by academic term and frequency of low, medium and high scores. Term F'70 T_w'71 5'71 F'7l N'72 Y 5'72 F‘72 N'73 rTotals 1L1N 1o 1 8 4 6 6 f 8 5 6 ' 53 , - % 20 g 17 ; 8 , 13 12 , 17 10 12 13.5 3. M7" 16 ,1 26 33 ; 27 32 = 23 29 . 37 213 § 3 32 . 55 67 ‘ 59 65 ‘ 48 58 3 54 , 55 ’ ‘ '. T T T 6"; ”in 24 A 13 12 13 11 r 17 16 1 17 123 z 481 28 25 28 23 35 32 34 31.5 Key: Faccomp - Faculty Completion Score L - Low Faccomp, Values 0, 1 or 2 M - Medium Faccomp, Values 3, 4, 5 or 6 H - High Faccomp, Values 7, 8 or 9 2 N - Raw score X = 18.8, 14 D of F % - Column frequency Significance < .17 Table 4.10. Student completion score for the Justin Morrill College written evaluation form by academic term and frequency of low, medium and high scores. Term F'70_fi_w'71_fi‘S'7l F'71 H'72 r S'72 F'72‘ N'73 Totals LN 20 18 1 17 23 3o 1; 26 22 f, 30 l 191 #73 40 L 38 1 35_; 61 61 1 54 44 1 6o :1 50 gMEN 25 f 23g 24 12 15 ”L 16 22 184L155 3;} 50 ; 49 ' 49 26 31 . 33 44 36 4o EH1" 5; 6 a; 6 4 1 6 6 2 43 1 1% 1o 1 13 164 13 8 13 11 121 4 11 Student Completion Score Low Studcomp, Values 0, 1 Medium Studcomp, Values 2, 3 or 4 High Studcomp, Values 5, 6 2 Raw score X = 20.3, 14 D of F Column frequency Significance < .12 Key: Studcomp 322:3!" lllll 181 Table 4.11. Chi Square values, degrees of freedom, and significance levels for the responses to questions 3, 9, 11 and 13 on the instrument used to analyze the Justin Morrill College written evaluation form by the variable, academic term. Questions X2 df p 3 38.9 7 < .0001 9 17.3 7 < .02 11 17.9 7 < .01 13 17.8 7 < .01 between faculty employment status and the faculty and student completion scores: 53: There is a relationship between faculty employment status and the degree to which faculty complete the written evalu- ation forms. S : There is a relationship between faculty employment status and the degree to which students complete the written evalu- ation forms. The results of the two Chi Square tests of independence (see Tables 4.12 and 4.13) used to test these two hypotheses show that there is a significant relationship between faculty employment status and the faculty completion score and there is not a significant relationship between faculty employment status and the student completion score. Fulltime faculty completed the written evaluation form more thoroughly than did parttime faculty. Thus, whereas one can reject the null hypothe- sis associated with research hypothesis S he cannot reject the null 39 hypothesis stating that there exists no relationship between faculty status and student completion score. 182 Table 4.12. Faculty completion score for the Justin Morrill College written evaluation form by faculty employment status and frequency of low, medium and high scores. Faccomp Fulltime Faculty Parttime Faculty, Totals N % N % N % Low 5 2 14 13 19 5 Medium 140 58 73 65 213 60 High 98 40 25 22 123 35 Faccomp - Faculty completion score X2 = 23.5 2 D of F N - Raw score Significance < .0001 % - Column frequency Low Faccomp - Values 0, 1, 2 Medium Faccomp - Values 3, 4, 5, 6 High Faccomp - Values 7, 8, 9 Table 4.13. Student completion score for the Justin Morrill College written evaluation form by faculty employment status and frequency of low, medium and high scores. a Studcomp Fulltime Faculty, Parttime Faculty Totals N % N % N % Low 113 46 44 39 157 44 Medium 102 42 53 47 155 44 High 28 12 15 14 43 12 Studcomp - Student Completion Score X2 = 1.63 2 D of F N - Raw Score Significance < .44 % - Column Frequency Low Studcomp - Values 0, 1 Medium Studcomp - Values 2, 3, 4 High Studcomp - Values 5, 6 183 There are also significant differences between full and parttime faculty in the manner in which they completed specific sections of the evaluation form and a statistical association between faculty status and the way in which students completed one section of the form. The raw scores and frequency counts are asterisked in Tables 4.3 and 4.4, and the Chi Square values, degrees of freedom, and significance levels are in Table 4.14. Table 4.14. Chi Square values, degrees of freedom, and significance levels for the responses to questions 3, 5, 7, 8, 9 and 14 on the instrument used to analyze the Justin Morrill College written evaluation form by the variable, faculty employment status. Questions X2 df p 3 11.8 1 < .0006 5 5.3 l < .02 7 28.2 1 < .0001 8 15.7 1 < .0001 9 6.9 l < .009 14 10.2 1 < .001 It was also decided to examine the impact of faculty status over time on the degree to which faculty completed the evaluation forms. Hence, the following research hypothesis was formed: S o 5. There is a relationship between the degree to which full and parttime faculty complete the written evaluation forms and the term the faculty complete them. The results of the Chi Square tests of independence (see Tables 4.15 and 4.16) are somewhat suspect since there are less than five 184 Table 4.15. Faculty completion score for the Justin Morrill College written eva uation form by academic term controlling for faculty employment status--fulltime faculty only. 1959_ F'70 H'71 5'71 F'7l H'72 5'72 F'72 H'73 Totals L N o i—' 2 2'?) o o o o 5 z o 1 8 7 o 3 o o o 2 D_ M N 9 15 17 22 20 15 20 22 140 §,.”.% 36 60 63 71 62 52 57 56 58 511915 8 8 9 11 14 15 17 98 li_64 32 30 3o 34 48 43 44 . 4o Faccomp - Faculty Completion Score x2 = 21.5 14 D of F L - Low Faccomp - Values 0, 1, 2 BQZIZ I III Medium Faccomp - Values 3, 4, 5, 6 High Faccomp - Values 7, 8, 9 Raw Score Column Frequency Table 4.16. Significance < .09 (n.b. - too few scores in several cells to lend much credence to significance level) Faculty completion score for the Justin Morrill College written evaluation form by academic term controlling for faculty employment status--parttime faculty only. 1333! F'70 w'71 . s'71 F'71 f w'72 5'72 F'72 T w'73¢ Totals L16 5 2 , 1 ' 1 ' 2 1 L1 1 1 g 14 . z 25 11 1 5 1o 14 8 9 17 ; 13 g_ MlN 7 11 #1 16 :5 5 12 8 9 5 1 73 §,_ % 35 4. 61 j 76 % 50 86 67 82 1 83 65 : HN8; 5 4y: 4 o 3 1 o 25 z 40 i 28 1 19 40 ,y o 25 1 9 o 22 Key: See Table 4 15. x2 = 19.14, 14 o of F Significance < .16 (n.b. - Too few scores in several cells to lend much credence in level of signifi- cance) 185 values in several of the cells. The tests do not reveal a significant relationship between the variable, term, and faculty completion score when controlling for faculty status. One cannot, therefore, reject the null hypothesis stating that there is no relationship between the variable, term and faculty completion score when controlling for faculty status. Nonetheless, in examining the figures in Table 4.15, there does seem to be a trend among fulltime faculty for higher comple- tion rates during the last three terms of use. To test the influence of a faculty member's knowledge area on faculty and student completion scores the following two research hypothe- ses were formed: S o 6' There is a relationship between the knowledge area in which faculty teach and the degree to which they complete the written evaluation forms. S7: There is a relationship between the knowledge area in which faculty teach and the degree to which students complete the written evaluation forms. The results of the two Chi Square tests of independence (see Tables 4.17 and 4.18) used to test the two hypotheses reveal a significant relationship between the knowledge area in which faculty teach and both the faculty and student completion scores. Faculty in the language, Inquiry and Expression and Field Study programs1 completed the evaluation 1The Miscellaneous program category was created by collapsing the Field Study, Independent Study and Senior Seminar categories into one dimension of the variable, knowledge area. This collapsing was done to assure more scores per cell for the X2 analysis. Even with the collaps- ing of items, there are still five cells in one table and one in the other with less than five scores. 0f the 13 forms in the high completion category in Table 4.17 under the heading "Misc.", 8 are Field Study forms, and only 9 Field Study forms appear in the sample. 186 Table 4.17. Faculty completion score for the Justin Morrill College written evaluation form by knowledge area and frequency of low, medium and high scores. Knowledge Area _r r, Nat.Sci. Soc.Sci. Human. Lang. I & E Misc. Totals ’L1_y_ 8 4 4 o 2 1 19 ' % 12 6 4 0 4 5 5 3M N 49 44 73 12 27 8 213 § % 72 65 66 31 57 36 60 ”211, N 11 20 34 27 18 13 123 1% 16 29 30 69 39 59 35 Key: Faccomp - Faculty Completion Score X2 = 43.5 10 D of F L - Low Faccomp, Values 0, 1, 2 M - Medium Faccomp, Values 3, 4, 5, 6 H - High Faccomp, Values 7, 8, 9 N - Raw Score % - Column Frequency Significance < .0001 Table 4.18. Student completion score for the Justin Morrill College written evaluation form by knowledge area and frequency of low, medium and high scores. cit-d Knowledge Area Nat.Sci. Soc.Sci. Human.fi_ Lang. I & E Misc. Totals LVN 31 33 41 3 11 25 16 157 % 46 49 37 28 53 73 44 E?” N 30 29 57 20 15 4 155 .8, #% 44 43 51 51 32 18 44 EEH' N 7 6 13 8 7 2 43 1 11%,; 1o 8 12 1 21 15 9 12 Key: Studcomp - Student Completion Score X2 = 19.2, 10 D of F L - Low Studcomp, Values 0, l BQZIZ VIII Medium Studcomp, Values 2, 3, 4 High Studcomp. Values 5, 6 Raw Score Column Frequency Significance < .04 187 forms more thoroughly than did faculty in the other knowledge areas. The faculty in Natural Science did not complete the forms to the degree expected. Students in Language and Humanities courses completed the forms more thoroughly than students in other courses. In light of the Chi Square results, one can reject the null hypothesis suggesting that there is no relationship between the knowledge area in which faculty teach and the faculty and student completion scores. There are also significant differences among the various knowledge areas for the completion of specific aspects of the evaluation form. The raw scores and frequency counts are starred in Tables 4.5 and 4.6, and the Chi Square values, degrees of freedom, and significance levels are in Table 4.19. Table 4.19. Chi Square values, degrees of freedom, and significance - levels for the responses to questions 1-4, 6-11, and 15 on the instrument used to analyze the Justin Morrill College written evaluation form by the variable, knowl- edge area. Questions X2 df p 1 46.7 5 < .0001 2 20.7 5 < .001 3 87.9 5 < .0001 4 33.8 5 < .0001 6 13.9 5 < .02 7 87 5 < .0001 8 27 5 < .0001 9 23.7 5 < .0002 10 18.9 5 < .002 11 18.2 5 < .003 15 14.9 5 < .01 188 To test the relationship between student class level and the faculty and student completion scores, the following hypotheses were generated: S8: There is a relationship between student status and the degree to which students complete the wr1tten evaluat1on forms. S : There is a relationship between student status and the degree to which faculty complete the written evaluation forms. The results of the two Chi Square tests of independence (see Tables 4.20 and 4.21) used to test the two hypotheses show that there is a significant relationship between the variable, class and the stu- dent completion score and there is not a significant relationship between class and the faculty completion score. However, the relationship between class and student completion score is opposite to that expected, with lowerclassmen completing the evaluation forms more thoroughly than upperclassmen. And, although the difference is not statistically sig- nificant, Table 4.21 reveals that faculty tended to evaluate lowerclass- men more thoroughly than upperclassmen. As a result of the Chi Square analysis, one can reject the null hypothesis suggesting there is no relationship between student status and student completion score but not reject the null hypothesis associated with research hypothesis 59. Upper and lowerclassmen also completed specific sections of the evaluation form in a significantly different way, with lowerclassmen being consistently more thorough in their responses. In Spite of the lack of a significant relationship between faculty completion score and class level, there is also a statistical association between class level 189 Table 4.20. Student completion score for the Justin Morrill College written evaluation form by class and frequency of low, medium and high scores. Studcomp Lowerclassmen Upperclassmen Totals N 7 N 7— N Low 110 44 81 60 191 50 Medium 114 45 40 30 154 40 High 28 11 15 10 43 10 Studcomp - Student Completion Score X2 = 10.1, 2 D of F N - Raw Score Significance < .006 % - Column Frequency Low Studcomp - Values 0, 1 Medium Studcomp - Values 2, 3, 4 High Studcomp - Values 5, 6 Table 4.21. Faculty completion score for the Justin Morrill College written evaluation form by class and frequency of low, medium and high scores. Faccomp Lowerclassmen Upperclassmen Totals N % N % N % Low 28 11 24 18 52 13 Medium 137 54 76 56 213 55 High 87 35 . 36' 26 123 32 Faccomp - Faculty Completion Score X2 = 4.7, 2 D of F N - Raw Score Significance < .097 % - Column Frequency Low Faccomp - Values 0, l, 2 Medium Faccomp - Values 3, 4, 5, 6 High Faccomp - Values 7, 8, 9 190 and the manner in which faculty completed several sections of the form; once again, lowerclassmen got more thorough faculty responses than did upperclassmen. These results are seen in the responses to the questions used for the form analysis. The raw scores and frequency counts are asterisked in Tables 4.7 and 4.8, and the Chi Square values, degrees of freedom and significance levels are in Table 4.22. Table 4.22. Chi Square values, degrees of freedom, and significance levels for the responses to questions 3-5, 9-11 and 13 on the instrument used to analyze the Justin Morrill College written evaluation form by the variable, class. Questions X2 df p 3 4.25 1 < .04 4 13.0 1 < .0003 5 4.7 l < .03 7 6.2 1 < .01 9 19.1 1 < .0001 10 6.0 1 < .01 11 7.1 1 < .01 13 7.0 l < .01 It was also decided to examine the impact of student status over time on the degree to which students completed the written evaluation forms. Hence, the following research hypothesis was formed: 310: There is a relationship between the degree to which upper and lowerclassmen complete the written evaluation forms and the term the students complete them. The results of the Chi Square tests of independence (see Tables 4.23 and 4.24) are somewhat suspect since several of the cells have less than five scores in them. The tests do not reveal a significant 191 Table 4.23. Student completion score for the Justin Morrill College written evaluation form by academic term controlling for student status--1owerc1assmen only. Term 3 F'70 H'71 S'71 F'71 H'72 S'72 F'72 N'73 Totals; 1 24, ‘1 ' “UN 10 1o 12 15 19 16 1 131 15 110 1 1 I z 27 31 32 1 58 63 53 43 1 52 44 1 l 1 ‘ 1r 5 1 1 : Egg M N 22 18 21 d-- 8 T” 8 1~ ll 14 %, 12 1 lltlmy1 pg; % 60 56 55 1, 31 27 1__ 37 47 1 41 45,4, . r T T ‘ '3‘“ N 5 4 5 i 3 .y 3 1 3 3 2 2843 V1 74113 13 13 11 1 1o 1 1o 10 11 1 Studcomp - Student Completion Score X2= 18.1, 14 D of F L - Low Studcomp - Values 0, 1 Significance < .20 M - Medium Studcomp - Values 2, 3, 4 (n.b. too few scores in H - High Studcomp - Values 5, 6 several cells to lend much N - Raw Score credence to significance % - Column Frequency level) Table 4.24. Student completion score for the Justin Morrill College written evaluation form by academic term controlling for student status-~upperc1assmen only. ::: 1 1 ISEE. i L . F'701, N'71 S'7lr F'71 N'72 - 5'72 F'72 N'73ylotal§_j 11LN1018 51131111101915181; 1 fi 1 ‘7 fi 7 11 %83 53 46 65,1581 56' 45', 711 601 gg-‘Mrerl, 5 3 4y, 7 5 8 61 401 101 7— 1 ' l 13. . % 17 34 27 20 37 28 40,_1 29 29 11:7 1 Fx‘H' N 0 2 3 3 l yyy 3 3 4r 0 15 Adz o 13 271 1511 51 16 151 o 11 Key: See Table 4.23. x2=13.5, 14DofF Significance < .49 (n.b. too few scores in several cells to lend much credence to significance level) 192 relationship between term and student completion score when controlling for class level. One cannot, therefore, reject the null hypothesis suggesting no relationship between the term in which students completed the evaluation forms and the degree they completed them regardless of class level. Two further research hypotheses were stated in Chapter III and tested by means of calculating the Spearman Rank-Order correlation coefficient, a non-parametric measure. The hypotheses are: 511: There is.a positive relationship between a student's grade po1nt average and the degree to wh1ch he completes the written evaluation form. 512: There is a relationship between the degree to which faculty complete the wr1tten evaluat1on forms and the degree to which students complete the written evaluation forms. As can be seen in Table 4.25, there is not a significant positive relationship between grade point average and student completion score whereas there is a positive statistical association between the faculty and student completion scores. One cannot, therefore, reject the null hypothesis suggesting no relationship between grade point average and the degree of student form completion, but one can reject the null stating that there is no relationship between the degree to which students completed the written evaluation forms and the degree to which faculty completed it. In sum, the analysis shows both the generalized null and the two sub-null hypotheses to be untenable. It is clear that there is a dif- ference in the degree of completion among the written evaluation forms students and faculty varied in their use of the forms.' Only one 193 Table 4.25. Spearman rank order coefficients and significance levels for the variable pairs, student grade point average with student completion score for the Justin Morrill College written evaluation form agg_student completion score with faculty completion score. Variable Pair Coefficient Significance Level GPA with Studcomp .0317 < .267 Studcomp with Faccomp .2991 < .001 Studcomp - Student Completion Score Faccomp - Faculty Completion Score GPA - Student Grade Point Average independent variable, term, seemed not to affect significantly the de- gree to which faculty or students completed the forms. In addition to the evaluation form analysis, several other indices indicate that implementation of written evaluation was less than com- plete as of Winter, 1973. When the sample of student evaluation forms was pulled for each term, the number of forms on file compared to the number of courses taken was also recorded. As Table 4.26 illustrates, in only one term did the percent of forms on file reach as high as 89. Thus, it is clear that JMC had a continuing problem of assuring that all student course performances were evaluated. Prior to Summer, 1973, seventy-three graduates earned more than fifty credits under the JMC written evaluation system. Of these, twenty- six or 36% requested profiles. This low percentage may increase in the future since, beginning Spring Term, 1973, students who reached senior status received a letter encouraging them to send profiles along with 194 Table 4.26. Number and frequency of Justin Morrill College written evaluation forms on file for the student sample by academic term. Term No. of Forms/No. of Courses Percent Fall 1970 106/128 78 Winter 1971 111/129 86 Spring 1971 97/124 . 78 Fall 1971 102/123 83 Winter 1972 100/117 85 Spring 1972 79/97 81 Fall 1972 92/103 89 Winter 1973 84/103 82 Total 771/924 83 their transcripts to graduate schools and prospective employers.1 About 93 percent of the students sampled in the Fall 1971 opinion survey claimed to be reading their evaluations by faculty. To check on this figure, during the first two weeks of Spring Term 1973, a record was kept of the number of student advising folders2 which students checked out. During the two week period, 325 students requested to see their advising folders. Assuming that approximately 650 JMC students3 were enrolled in JMC courses during Winter Term 1973 and recognizing that some of the students were not checking their folders out to read their evaluations, a figure of 325 indicates that less than 50 percent 1A letter from the Dean's staff assistant has been sent to all JMC students reaching senior standing at MSU since Spring Term 1973. 2The student advising folders have copies of the completed written evaluation forms. 3Average Winter Term enrollment of JMC students. 195 of the JMC students enrolled during Winter Term 1973 had read their evaluations by the third week of the subsequent term. Because of the marked discrepancy between the Fall 1971 and the Spring 1973 figures, further data must be gathered before either figure can be verified. Regardless, it is clear that some JMC students did not read the evalu- ations after they were written. Discussion of the Degree of Implementation Examining the form analysis, the percent of forms on file, the number of graduates requesting profiles and the number of students read- ing evaluations, it is obvious that as of Winter, 1973 Justin Morrill's use of written evaluation fell somewhat short of full implementation as defined in Chapter I. In light of this fact, it is also obvious that implementing organizational innovations involves something more than deciding to adopt an innovation. To review briefly, there was not a completed evaluation form (as defined in Chapter I) for each JMC student in every JMC course, all students with more than fifty credits under the JMC written evaluation system did not request profiles, many faculty did not distribute the evaluation forms on the first day of class to review course objectives and the goals of written evaluation, many faculty did not offer guidance to students as they wrote self-evaluations, and some students did not read their completed evaluation forms. Nonetheless, a profile was written for each student who requested one and advisors did use the evaluation forms to aid their student advising. The organizational factors, the environment, the innovation's characteristics and the amount of nurturance described earlier in the 196 present chapter all played a part in hindering incorporation of written evaluation in Justin Morrill. Yet, as was also indicated earlier in this chapter, other aspects of the same set of variables enabled JMC to implement written evaluation to the degree it had by Spring Term, 1973. To review the factors which aided and hindered implementation, it is helpful to offer explanations for the findings itemized in the present chapter. It is not surprising to discover that students and faculty used the system unevenly in light of the rapid adoption process in the chaotic conditions of Spring, 1970. In addition, the communication network over- load led to an adoption decision which was more authoritative than col- lective. As a result, many persons were ignorant of the details of written evaluation and some were resentful that a decision had been made. Nonetheless, the nurturance efforts of the Dean, the Associate Dean and the Chairman of the Curriculum Corrmittee coupled with the compatible aspects of written evaluation and the willingness of JMC faculty to try a new practice enabled the system to be implemented Fall Term, 1970. Use continued to be uneven and rather low through Winter, 1973 due to the distractions of an innovative college,1 the general lack of mainte- nance after Winter, 1971 and before Fall, 1972, the complexity of the system requiring extensive coordination, the amount of time it takes to write evaluations, the system's incompatibility with faculty teaching lThat is, the continuing press by the Dean and others to adopt addi- tional innovations, the continual reorganization of college governance, the Provost's evaluation, and other pressures mentioned in the first part of the present chapter. 197 autonomy and JMC's loose role structure and the low visibility of the results resulting in an absence of operational feedback and a lack of rewards for writing superior evaluations. Nor is it surprising to find that faculty completed the evaluation forms more thoroughly than students. After all, JMC faculty are paid to teach, are committed to trying educational innovations, have been in JMC since written evaluation began, and have been directly involved in any efforts to improve the written evaluation system. Students, on the other hand, are generally unfamiliar with self-evaluation, have not received much guidance from faculty in the process of self-evaluation, and, with each passing year, are less likely to have been involved in the decision to adopt.1 In addition, although most JMC students seem to be open to new ideas and have a favorable attitude toward change, they are not necessarily committed to any specific educational innova- tion. In analyzing the first set of research hypotheses (S1 and 52), it was interesting to learn that the variable, term, did not significantly affect either the faculty or student completion score. With the greater efforts at improving the written evaluation system during the 1972-73 academic year, it was expected that there would be a trend of greater faculty completion. The lack of significant relationship may indicate 1It is interesting to note that although the variable, term, was not associated with a significant difference among student completion scores, student use of written evaluation was highest during Fall 1970, Winter 1971 and Spring 1971, terms when the largest number of students likely to have been involved in the decision to adopt were enrolled in JMC. 198 the general weakness of rational-empirical efforts which were used heavily during Fall, 1972 and Winter, 1973 (The Dean's staff assistant made extensive use of memoranda and rational discussion). Or it may be an indication that other issues were once again draining faculty energies away from written evaluation. When controlling for faculty status, however (see Table 4.15), fulltime faculty seemed to complete the form more thoroughly during the last three terms than at anytime since Fall, 1970. Nevertheless, the difference is not statistically significant. The significant difference among terms on the question of whether faculty listed specific course objectives is directly attributable to the change of forms in Winter 1971. Looking at the third row of Table 4.1 or 4.2, one can see that only in FalL 1970 did a majority of faculty write specific course objectives on the evaluation form. After Fall, 1970, writing course objectives was not clearly obligatory. The 1 occurs be- significant difference among terms on questions 11 and 13 cause of better student use of written evaluation during the first three terms. Students using the forms during that period were much more likely to have been involved in or aware of the decision to adopt, and this familiarity may have led to better student completion in the first three terms. The form analysis showed that fulltime faculty completed the evalu- ation form more fully than did parttime faculty, but that faculty 1(11) Has the student written an evaluation? (13) Has the student assessed his/her performance on explicit course or personal objectives 199 employment status did not affect the degree to which students completed the form.1 This difference was expected since JMC faculty are more committed to individualized evaluation, more familiar with the JMC written evaluation form and have had greater involvement with the efforts to improve written evaluation than parttime faculty. The fact that student use of written evaluation was no better in the classes of fulltime than parttime faculty would indicate that faculty in general were simply not encouraging students to engage in self-evaluation or aiding them in the process. The analysis of hypotheses S6 and S7 concerning the relation of knowledge area to form completion shows that faculty in Language, Inquiry and Expression and Field Study completed the form more thoroughly than faculty in Natural Science, Social Science or the Humanities. This finding was expected since the faculty in the former programs aim for some uniformity in instruction and do more staff planning than do faculty in the Social Sciences and the Humanities. It was felt that faculty doing extensive planning as a staff would probably also develop a common strategy for implementing written evalu- ation. It was surprising to find that the Natural Science faculty did not complete the form more thoroughly since they, too, work closely together and have developed a fairly tight program. The poorer than 1There was a significant difference between full and parttime faculty in the answer to question l4--Has the student commented on any_ of the following t es of student activities . . .? The author can find no reasonable exp anation for this difference. 200 expected showing of Natural Science may be due to the high number of parttime faculty that teach in the Natural Science program. In addition to the influence of staff planning, the degree of faculty completion may have been influenced by the number of students faculty had to evalu- ate each term. Faculty in the Natural Sciences, Social Sciences and Humanities often have more students in their classes than do faculty in other knowledge areas and may, therefore, feel more pressed for time at term's end. The analysis of hypotheses S6 and S7 also illustrates that students in Language and Humanities classes completed the evaluation forms more fully than students in other courses. It would seem, then, that faculty in these programs facilitated and encouraged student self- evaluation more than faculty in other knowledge areas. Of course, stu- dent completion remained quite 1ow. Only in language courses did more than 70 percent of the student sample fall in the medium and high com- pletion categories. As shown in Tables 4.7, 4.8 and 4.20, lowerclassmen completed the evaluation forms more thoroughly than upperclassmen. This finding is the reverse of the one anticipated; it was felt that the more familiar stu- dents became with written evaluation, the better they would use it. Apparently the upperclassmen did not take the system as seriously as did lowerclassmen or at least did not find it worth the effort to evaluate their own performances. This difference between upper and lowerclass use of written evaluation may be due partially to the fact that upper- classmen are developing their fields of concentration and thus, are more 201 interested in their university courses than in the few courses they take in JMC. Once again, it is important to note that in general, student completion scores were quite low; lowerclassmen also completed the form inadequately. Student use of written evaluation did not vary with ability, if one accepts the grade point average as an indicator of student academic ability. However, the analysis shows an association between high faculty completion scores and high student completion scores. Apparently faculty who approached written evaluation seriously did influence the manner in which students completed the form. This hypothesis is also supported by the relatively high student completion scores in language classes. Summary In the present chapter Justin Morrill College was depicted as a small, innovative, liberal arts college with a loosely defined role structure, a low level of differentiation and elaboration, vague and somewhat conflicting goals, varied task demands, relatively modern norms with a strong preference for informal interpersonal relationships and teacher autonomy, few rules and regulations, an authority system with a high amount of faculty and student participation, a reward system con- sistent with the tasks of a teaching innovative college, an emphasis on adaptation as opposed to maintenance, an innovative Dean who prefers rapid change, a faculty who exhibit many early adopter characteristics, and students who are bright, liberal, anti-authoritarian, impulsive and 202 express a high need for autonomy and independence. In brief, JMC is an open system which welcomes change in a variety of ways, and its short history reveals an organization which stresses adaptation and innovation. The adoption process covering the period from initial awareness to Spring Term, 1970 when JMC forwarded a proposal to the University Curricu- lum Committee was then reviewed. In sum, several organizational and environmental factors facilitated the adoption of written evaluation in spite of the innovation's incompatible and negative a5pects. And, although several faculty and students resented the fact that the decision had been made during a term with so many distractions, most accepted the decision and seemed quite willing to help implement the new system. This general acceptance may well have been due to the recognized legiti- macy of the authority structure, the knowledge that every effort had been made to solicit opinions and advice, the lack of awareness of the behavioral implications of the grading model, and the sense that the innovation was indeed compatible with the experimental, personalized nature of Justin Morrill. The implementation process which occurred over the next three years was then reviewed. In looking at the process as a whole several facts became clear: 1) The behavioral implications of written evaluation and the new task demands were not clarified in the Fall of 1970; 2) The innovation continues to be incompatible with JMC's loosely-defined role structure and the desire for faculty teaching autonomy yet compatible with JMC's desire for individualized evaluation; 3) Written evaluation has not received enough maintenance; 4) Written evaluation has, since its 203 inception, been competing for the attention of faculty with numerous other innovations, crises and the normal task demands of an undergradu- ate college; 5) The results of written evaluation are not highly visible. As a consequence, faculty have neither been adequately re- warded for doing superior evaluations nor discouraged from doing care~ less ones. In addition, until recently, the low visibility of the results hampered the development of a suitable operational feedback mechanism. In sum, JMC was using written evaluation under difficult circumstances through Spring Term, 1973. Finally, a variety of measures were used to determine the degree of implementation of written evaluation in JMC as of Winter Term 1973. Examination of Tables 4.1-4.8 reveals the general results of the form analysis which shows uneven use by faculty and students in the sample. Faculty, however, completed the form more thoroughly than students. The twelve hypotheses discussed in Chapter III were tested, providing some interesting results. Faculty and students did not vary in their use of the evaluation form over time. Fulltime faculty completed the form more thoroughly than parttime faculty. Faculty in Language, Inquiry and Expression, and Field Study completed the form more thoroughly than those in Natural Science, Social Science and the Humani- ties. In addition, students in Language and Humanities classes com- pleted the forms to a higher degree than students in other classes. And, lowerclassmen completed the evaluation forms more thoroughly than upperclassmen. 204 The form analysis, the percent of evaluation forms on file, the number of students who read the completed evaluation forms, and some results from the 1971 attitude surveys all revealed that JMC had not attained full implementation as defined in Chapter I by Winter Term 1973. To review briefly, there was not a completed evaluation form (as defined in Chapter I) for each JMC student in every JMC course, all students with more than fifty credits under the JMC written evaluation system did not request profiles, many faculty did not distribute the evaluation forms on the first day of class to review course objectives and the goals of written evaluation, many faculty did not offer guidance to students as they wrote self-evaluations, and some students did not read their completed evaluation forms. Nonetheless, a profile was written for each student who requested one and advisors did use the evaluation forms to aid their student advising. CHAPTER V SUMMARY AND CONCLUSIONS Summary Current theories of social and organizational change have at least five weaknesses from the perspective of one interested in planning or managing organizational innovations: 1) Many researchers concentrate on the content of a specific innovation rather than emphasizing that change is a process, occurring over time and demanding thorough strate— gies for adoption and implementation; 2) Few researchers have examined the change process in organizational settings. The result is an over- simplified view of organizational innovation; 3) Most research on diffu- sion of innovations has analyzed or recounted the efforts leading to adoption rather than those related to implementation; 4) Most innovation research deals with physical or technological innovations rather than with ideas or social practices; 5) Few theories of complex organiza- tional change give explicit guidance to the person interested in manag- ing change in an educational institution. In general, the present study was designed to address these five weaknesses by developing a case history of the adoption and implementa- tion of a Specific innovation in a complex institution so that one could view the process in its entirety and generate plausible hypotheses for the determined degree of implementation. 205 206 More specifically, the present study was undertaken to: 1) create a model of organizational innovation by synthesizing what is presently known about organizational change and organizational behavior; 2) use the model to develop and analyze the case history of written evalua- tion, an organizational innovation adopted by Justin Morrill College (JMC) in 1970. Using the written evaluation system, faculty in JMC write evaluations for each student in their classes rather than award- ing them numerical grades. The students also evaluate their class performance on the same form which contains a course description, course objectives and bases for evaluation. The evaluations are placed in individual student advising folders and at the student's request, are later summarized into a brief profile. The students may have the profiles of their work at JMC forwarded to prospective employers or graduate schools along with their Michigan State transcripts; 3) deter- mine the degree of implementation of written evaluation in Justin Morrill as of Winter Term, 1973 and compare it with full implementation as defined in Chapter I of the present study; 4) generate specific hypotheses concerning the implementation of organizational innovations using the organizational innovation model developed in Chapter II, the research reviewed in Chapter II, and the case history of written evalu- ation discussed in Chapter IV. Four theoretical traditions were reviewed: 1) The literature on complex organizational behavior for the purpose of itemizing the various elements which shape all organizational behavior including planned change; 2) the Social Interaction literature which specifies the four 207 principal elements of the change process (the innovation, communication channels, members of a social system and time) as well as analyzing the adoption process; 3) the Research, Development and Diffusion tradition which outlines in detail the various stages in the dissemination and utilization process-~research, development, diffusion and adoption--and stresses that change should be a rational and planned process; and 4) the Problem-Solver approach to change which, recognizing the impact of organizational variables, stresses a systems approach to change, calling for change in the behavior and attitudes of system members as well as change in the organizational structure itself. This approach is ultimately more concerned with creating self-renewing organizations than with introducing specific innovations to individual systems. For the purposes of the present study--ana1yzing the adoption and implementation of an innovation within a single complex organization-- the social interaction perspective (Everett Rogers' Communication of Innovations Theory) was found most useful as a structure upon which to build with the help of insights from the other three theoretical tradi- tions. Modifying Roger's model for the purposes of the present work, the "Diffusion Process" outlined in Chapter I was renamed the "Organizational Innovation Process," and includes the following elements: 1) an innovation, 2) advocate(s) of change, 3) a complex organization in its environment, 4) a communication network, 5) time. Thus, if organiza- tional innovation is successful, it is a process of planned change in an organizational setting during which the system members move from initial knowledge of the innovation through the stages of persuasion, adoption, 208 implementation and incorporation.1 Note that the members of the organi- zation must go through the five stages regardless of whether the inno- vation decision is authoritative or collective in Rogers' terminology. Members may be told to adopt, implement and incorporate, or they may participate in the innovation decision-making process. In either case, if organizational innovation is completely successful, they will either behave in accordance with the innovation decision or leave the organi- zation. If some members do not adhere to the innovation decision, then implementation and incorporation must be considered less than complete. Four basic research methods were used to develop the case study of written evaluation at Justin Morrill: 1) participant-observation; 2) nonreactive, unobtrusive measures; 3) attitude surveys of faculty and students; 4) an analysis of the degree of completion of the written evaluation forms that faculty and students use to assess student course performance. Together, they provided a picture of written evaluation in JMC and gave some clues for the present degree of implementation. In its entirety the case study provided the basis for generalizations regarding implementing innovations in complex organizations. The form analysis was the most complex of the four research tech- niques. A sample of 389 completed written evaluation forms was pulled from student folders, approximately 50 from each of the eight terms in 1Please note that the process is successful and the author has changed the decision and confirmation stages of Rogers' paradigm to adoption, implementation and incorporation. This change was made to emphasize the last two stages of the process, the primary interest of the present study. 209 which the innovation had been in effect. All of the forms were analyzed using a series of questions to determine the degree to which both faculty and students completed the evaluations.1 For the purpose of further analysis, one generalized null hypothesis was generated, and twelve research hypotheses were formed in order to test the generalized null. The generalized null stated that: G.H. : There is no difference in the degree of completion among the written evaluation forms. The statistical models chosen for analysis were the Chi Square of independence and the Spearman nonparametric correlation analysis. Both permit one to determine if there is a statistical association between two variables. In addition, frequency counts, means and standard devi- ations were calculated in order to compare use of the written evaluation form among the five independent variables of time, faculty employment status, student class, grade point average, and percent of completed forms on file. So that readers might have a contextual backdrop for the history of written evaluation, Justin Morrill was described as a complex organi- zation. JMC is a small, innovative, liberal arts college with a loosely defined role structure, a low level of differentiation and elaboration, vague and somewhat conflicting goals, varied task demands, relatively modern norms with a strong preference for informal interpersonal rela- tionships and teacher autonomy, few rules and regulations, an authority system with a high amount of faculty and student participation, a reward 1See Appendix 1 for the instrument used to analyze form completion. 210 system consistent with the tasks of a teaching innovative college, an emphasis on adaptation as opposed to maintenance, an innovative Dean who prefers rapid change, a faculty who exhibit many early adopter characteristics, and students who are bright, liberal, anti-authori- tarian, impulsive and express a high need for autonomy and independence. In brief, JMC is an open system which welcomes change in a variety of ways, and its short history reveals an organization which stresses adaptation and innovation. The adoption process (knowledge, persuasion, decision to adopt) covered the period from the writing of a skeletal proposal in Winter Term, 1969 until the end of Spring Term, 1970 when JMC forwarded a pro- posal for using written evaluation to the University Curriculum Com- mittee. Several organizational and environmental factors facilitated the adoption of written evaluation in spite of the innovation's incompatible and negative aspects. And, although several faculty and students resented the fact that the decision had been made during a term with so many distractions, most accepted the decision and seemed quite willing to help implement the new system. This general accept- ance may well have been due to the recognized legitimacy of the author- ity structure, the knowledge that every effort had been made to solicit opinions and advice, the lack of awareness of the behavioral implications of the grading model, and the sense that the innovation was indeed compatible with the experimental, personalized nature of Justin Morrill. The implementation stage was reviewed from Fall Term 1970 through Spring 1973. In looking at the process as a whole several facts became at {/11 211 clear: 1) The behavioral implications of written evaluation and the new task demands were not clarified in the Fall of 1970; 2) The innova- tion continued to be incompatible with JMC's loosely defined role structure and the desire for faculty teaching autonomy yet compatible with JMC's desire for individualized evaluation; 3) Written evaluation received too little maintenance prior to Winter Term, 1973; 4) Written evaluation has, since its inception, been competing for the attention of faculty with numerous other innovations, crises and the normal task demands of an undergraduate college; 5) The results of written evalua- tion are not highly visible. As a consequence, faculty have neither been adequately rewarded for doing superior evaluations nor discouraged from doing careless ones. In addition until recently, the low visi- bility of the results hampered the development of a suitable operational feedback mechanism. In sum, JMC was using written evaluation under difficult circumstances through Spring Term, 1973. Finally, a variety of measures were used to determine the degree of implementation of written evaluation in JMC as of Winter Term, 1973. Tables 4.1-4.8 illustrate the general results of the form analysis which revealed uneven use of the written evaluation form by faculty and stu- dents in the sample. Faculty, however, completed the form more thoroughly than students. The twelve hypotheses discussed in Chapter III were tested, providing the basis for rejecting the generalized null hypothesis. Faculty and students did not vary in their use of the evalu- ation form over time. Fulltime faculty completed the form more thoroughly than parttime faculty. Faculty in Language, Inquiry and Expression, and 212 Field Study completed the form more thoroughly than those in Natural Science, Social Science and the Humanities. In addition, students in Language and Humanities classes completed the forms to a higher degree than students in other classes. And, lowerclassmen completed the evaluation forms more thoroughly than upperclassmen. The form analysis, the percent of evaluation forms on file, the number of students who requested profiles, the number of students who read the completed evaluation forms, and some results from the 1971 attitude surveys all revealed that JMC did not attain full implementa- tion as defined in Chapter I by Winter Term, 1973. To review briefly, there was not a completed evaluation form (as defined in Chapter I) for each JMC student in every JMC course (17% of the forms sampled were missing from student folders), all graduates with more than fifty credits under the JMC written evaluation system did not request profiles (36% did through Spring Term, 1973), many faculty did not distribute the evaluation forms on the first day of class to review course objec- tives and the goals of written evaluation, many faculty did not offer guidance to students as they wrote self-evaluations, and some students did not read their completed evaluation forms. Nonetheless, a profile was written for each student who requested one and advisors did use the evaluation forms to aid their student advising. Conclusions If one were only to examine the degree to which JMC implemented written evaluation by Winter Term, 1973, he might conclude that the 213 college had been remiss in meeting its obligations and criticize the Dean and faculty for their general neglect of the innovation. However, in light of the history traced in the present case study, such an analysis can be considered at best only partially accurate and certainly not very helpful to an organization struggling with implementation. It is true that Justin Morrill's Dean placed greater emphasis on adaptation and a rapid pace of innovation than on maintenance and other task demands, but he could argue persuasively that he was trying to meet one of the stated goals of an innovative college--to adOpt new educational practices which might better meet the needs of students in the 1970’s. It is also true that JMC faculty expended more energy in designing new courses, teaching and advising than in conforming to the behavioral expectations associated with written evaluation, but they, too, could argue that these tasks were more important than implementing a single innovation. Thus, merely criticizing the people in a system misses the point since they represent only one aspect of the implementation process. One must also determine the impact of the existing organizational variables itemized earlier in the present Chapter (e.g., goals, task demands, roles, norms, the authority system), the characteristics of the innovation itself (relative advantage, compatibility, complexity, observability), the manner in which the communication network is func- tioning (e.g., the presence or absence of operational feedback, the amount of horizontal communication about the innovation), the present state of the external environment (e.g., is it in crisis, stable, 214 threatening, supportive), and the presence or absence of persons nurtur- ing the innovation. In short, one should examine four of the five elements1 identified in the organizational innovation model developed in Chapter II. Obviously the model for organizational innovation is not a recipe guaranteeing successful change. Rather, the model can help planners and managers to change by identifying the elements that may be influ- encing the fate of a particular innovation at any point in time. Without such a model, one might easily overlook an important factor while trying to determine why an innovation is or is not functioning. Only through an examination of all elements can a change agent be at all confident in identifying the variable or variables which seem most influential during a stage of the organizational innovation process (knowledge, persuasion, adoption, implementation, incorporation). For instance, from the present case study, it is clear that during the persuasion and adoption stages of written evaluation, the chaotic environmental conditions, the overloaded communication network and the power concentration in the college authority system were more influential than the negative characteristics of the innovation itself. Lacking an awareness of all the variables at play during Spring Term, 1970, one might not be able to reach such a conclusion. 1Excluding the variable, time, since after adoption, one is clearly involved in the implementation stage. Incorporation does not occur until implementation is complete. 215 The case study of written evaluation permits one to offer several generalizations regarding organizational innovation, many of which rely heavily on the literature review in Chapter II. These generalizations are offered as untested hypotheses and do not necessarily hold under all conditions. As was emphasized above, one must examine all the elements in the organizational innovation process before being able to determine which variable is most influential at a particular point in time. 1.) There are functions (stages)_in the organizational innovation process. These functions are knowledge, persuasion, adoption, imple- mentation, and incorporation. The case study shows that there are definite periods in the history of a successful innovation which corre- spond roughly to the above categories. Naturally, the stages are not as distinct as a typology makes them sound, but although they overlap somewhat, their presence makes clear that the passage of time is an important element in the organizational innovation process. JMC became aware of the written evaluation and Pass-No-Credit concepts during the years, 1967-69 (knowledge), considered two preliminary proposals culmin- ating in a final proposal distributed to faculty and students in April, 1970 (persuasion), forwarded the final proposal to the University Curri- culum Committee with the endorsement of the Dean, the Curriculum Commit- tee and the Advisory Council in May, 1970 (decision to adopt) and has been attempting to implement written evaluation since the Fall of 1970. If JMC implements written evaluation more fully and asks the University Curriculum Committee to approve the system as a permanent part of the 216 JMC curriculum, it may be said that JMC has incorporated the innova— tion into the ongoing organization. The functions parallel Lewin's steps of unfreezing (knowledge), moving (persuasion, adoption), and refreezing (implementation, incorporation), and adding the steps of implementation and incorporation emphasizes that refreezing is just as important to successful organizational innovation as changing. Stated somewhat differently, every organization must establish a new point of equilibrium after a major change of one or more organizational elements. Indeed, implementation of written evaluation in JMC was less than com- plete in Winter Term, 1973, precisely because the college had not encouraged the freezing of the new role behaviors associated with the innovation; a new equilibrium had not been established. 2.) The characteristics of an innovation have an effect on its degree of implementation. More specifically: (a)_The clarity of the behavioral expectations associated with the organizational innovation is positively related with its degree of implementation;1 (b) The relative advantage of an organizational innovation is positively related to its degree of implementation;(c) The compatability of an organiza- tional innovation isppositively related to its degree of implementation; (d) The complexity of an organizational innovation is negatively related to its degree of implementation; (e) The observability of an organizational innovation is positively related to its degree of 1This generalization also supported by: Neal Gross gt_gl,, pp, £13,, p. 214. 217 implementation.1 From the beginning, the behavioral expectations for faculty and students associated with written evaluation were vague and somewhat conflicting. This lack of clarity impeded implementation and only during the 1972-73 academic year did JMC begin to identify precisely where faculty should use the system uniformly and where they should exert individual initiative. The thoroughness of written evalu- ation (relative advantage) and its compatibility with JMC's innovative mission and faculty desire to evaluate students individually facilitated implementation. However, the system's high cost and time consuming nature (relative disadvantages), its incompatibility with the norm of faculty teaching autonomy and JMC's loosely defined role structure, its complexity necessitating high maintenance and coordination, and the low visibility of its results all hindered full implementation by Winter Term, 1973. 3.) The presence of change advocates facilitates a collective organizational innovation process. Specifically innovators stimulate interest duripg_the knowledge function, initiators develop a specific proposal during the persuasion stage, legitimizers facilitate the deci- sion to adgpt, and formal leaders nurture the innovation during,imple- mentation. Change advocates aided the written evaluation system in JMC through the initial stages of implementation, but abandoned it soon thereafter. Assuming that the system was Operating efficiently, the Dean was soon advocating the adoption of additional innovations, the 1The phrasing for these generalizations parallels that of: Rogers with Shoemaker, pp, 213., pp. 350-352. 218 Associate Dean returned to fulltime teaching and the Chairman of the Curriculum Committee was helping manage other curricular change issues. The Assistant Dean did provide the minimal coordination necessary to keep the system going. Although opinion surveys were run and grade data gathered, no one made a concerted effort to improve JMC's use of written evaluation-until Fall, 1972. By Winter Term, 1973, the efforts of the Dean's staff assistant had not produced any marked improvement in student and faculty use of written evaluation. However, the major system alteration (a new evaluation form) did not take effect until Spring Term, 1973, and further measures of the degree of implementation may reveal support for the idea that the degree of implementation in- creases as formal leaders nurture the innovation.1 There seems little doubt from the case study that more concerted support by formal leaders in JMC would have led to fuller implementation by Winter Term, 1973. 4.) The amount of,pperational feedback related to an organizational innovation is positively related to its degree of implementation. Organizational innovations rarely work well as originally introduced to a complex organization since local conditions vary widely among systems. To aid the implementation process, during which both the innovation and the prevailing organizational arrangements undergo some change in order to attain a new point of equilibrium (homeostasis), organizations must develop adequate mechanisms for operational feedback. During the first six terms written evaluation was used in JMC, operational feedback was 1This generalization is supported by the case study of Neal Gross 9331., _p. _c_i_t_., pp. 212-216. 219 primarily anecdotal, and conflicting reports were common. Modifications in the innovation and organizational arrangements were delayed since a clear direction for movement was not evident. Once the initial data from the form analysis and the comments of profile writers became avail- able, the JMC Advisory Council agreed to some significant system changes for Spring Term, 1973. Further adaptation of both the innovation and organizational elements may be possible in light of the evidence con- tained in the present study. 5.) The amount of interpersonal communication regarding an organiza- tional innovation in a complex system is ppsitively,related to the degree of implementation of the innovation. The low visibility of the results of written evaluation in JMC coupled with the continuing distractions of other innovations and task demands inhibited the sharing of ideas among faculty on how they were writing course descriptions, course objectives, bases for evaluation and the evaluations themselves. The form analysis in Chapter IV showed that in programs where idea sharing was common, faculty completed the evaluation forms more thoroughly. Increased idea sharing about written evaluation among JMC faculty in general might well have improved the degree of implementation of written evaluation by Winter Term, 1973. 6.) The rate of adoption of organizational innovations is positively related to the degree of power concentration in the authority system of a complex organization. The case history of written evaluation illus- trates that the power concentration represented by the Dean, the Advisory Council and the Curriculum Committee enabled the adoption decision to be 220 made in spite of the general lack of participation of faculty and students and the chaotic environment of Spring, 1970. Without such a power concentration, the low involvement of the JMC community, the environmental conditions and the incompatible aspects of written evalu— ation would probably have prolonged the persuasion stage indefinitely. 7.) The degree of implementation of an organizational innovation is positively related to the amount of participation by the implementers in the decision to adgpt. Although the rate of adoption would have been decreased by greater faculty and student involvement in the decision to adopt (see generalization 6), their participation would probably have led to more thorough use of written evaluation by Winter Tenn, 1973. Before granting their endorsement, faculty would have wanted to know what additional demands on their time the innovation would make. In all likelihood, increased participation prior to adoption by the imple- menters would have led to more clearly defined behavioral expectations and a system more compatible with the college and its faculty. 8.) An prggrization's degree of Openness to its environment is positively related to its propensity,to adopt innovations. More specifi- cally; (a) Organizations with a loosely defined coding,scheme are more likely to adopt innovations; (b),Organizations with vague and conflicting goals are more likely to adopt innovations; (c) Organizations with loosely defined role structures are more likely to adopt innovations; (d) Organizations with general rules and regulations are more likely to adgpt innovations; (e)_Organizations with norms of openness to external influence are more likely_to agppt innovatione; (f) Organizations which 221 encourage membership_participation in the authority,§ystem but do not practice consensus decision-making are more likely to adopt innovations; (g) Organizations which reward innovative behavior are more likely to adppt innovations; (h) Organizations with an open communication network are more likely to adopt innovations; (1)0rganizations whose manager(s) place(s) heayy emphasis on the adaptation function are more likely to adopt innovations; (j) Organizations whose members exhibit early adopter characteristics (see Chapter II) are more likely to adppt innovations. Justin Morrill exhibits all of the above characteristics, and together, they have led the college to adopt a large number of innova- tions at a fairly rapid rate (see Chapter IV for a full accounting of innovations adopted between Fall 1969 and Spring 1973). 9.) An organization's degree of openness to its environment is negatively related to its propensity,to implement innovations. One of Justin Morrill's greatest assets for adoption--its openness--proves its undoing when it comes to organizing a concerted effort to implement innovations. Prior to Spring Term, 1973, JMC was simply unable to apply enough of its energy to written evaluation to increase the degree of utilization. The present study tends to support Brickell's notion that an environment suitable for innovation is not suitable for controlled experimentation. That written evaluation had been implemented to the degree it was in Winter Term, 1973, is a tribute to the willingness of many JMC faculty to try new practices and their concern for giving extensive feedback to students. 10.) An orgarjzation in a neutral or supportive but changing envir- onment is more likely to adopt innovations. The changing educational 222 environment of the late 1960's supported the rapid rate of adoption in new colleges like Justin Morrill (see Chapter IV), and Michigan State University, while containing its share of nay-sayers, permitted JMC a maximum degree of autonomy and freedom to fulfill its experimental mission (see the goals of JMC, Chapter IV). The generalizations and the case study give rise to the following guidelines for those persons interested in planning or managing change in educational institutions. One should remember that they are merely strategies which are likely to facilitate implementation under normal circumstances. Following them, and carrying a rabbit's foot, might lead to successful innovation: 1.) Where possible select innovations which have a high degree of relative advantage, are compatible with organizational arrange- ments and member preferences, are not overly complex, may be used on a trial basis and whose results are highly visible. 2.) Involve the implementers in the decision to adopt so that they can help clarify the behavioral expectations associated with the innovation and increase their commitment to the success of the innovation. Avoid concensus decision-making, however, since it is almost always impossible to gain complete agreement on a given direction. 3.) The implementers and the administrator(s) should bear in mind that during implementation, both the innovation and the organiza- - tional arrangements are likely to change somewhat as a new equi- librium is established. An adequate operational feedback mechanism 223 should be created in order to assure that the innovation is not so altered as to be worthless and that the organizational arrangements are not so modified as to detract from other task demands. 4.) Rational-empirical techniques during implementation must often be accompanied by retraining, restructuring and re-educative techniques. Unless persons have the necessary skills, they cannot fill the roles demanded by an organizational innovation, and incompatible system arrangements (e.g., the reward system, con- flicting task demands) may undercut an innovation regardless of other sources of support. 5.) Be certain there are enough change advocates both before adoption and during implementation. Formal leaders must nurture the innovation and give substantive and moral support to the implementers. Such support is essential as persons experience the inevitable frustrations associated with trying new roles and behaviors. 6.) Minimize the number of competing task demands and innovations. 7.) Accept the fact that organizational innovation is normally a long and tedious process requiring extensive energy and coordina- tion if incorporation is the ultimate goal. Research Implications The logical next step for research regarding implementation is the analysis of a sample of completed evaluation forms from Spring Term, 224 1973 and each term during the 1973-74 academic year to determine if the continued efforts at improvement have had a more significant effect on written evaluation after Winter Term, 1973. In addition, it would be helpful to get an additional reading of faculty and student attitudes toward written evaluation to see if it enjoys the same degree of popu- larity in spite of its time demands. As was pointed out early in the present study, case studies gener- ate but do not test hypotheses. To test the generalizations offered in this chapter and to examine the utility of the organizational innovation model, some comparative studies should be undertaken. Neal Gross et_al, state the need succintly: . . . studies will need to be designed that introduce the same innovation into a number of organizations that vary on one or more organizational characteristics, for example, average age of the staff, degree of staff autonomy, or the influence of the external environment on the functioning of the organization. Through an analysis of the types of obstacles that arise in the organizations examined, valuable data would be obtained on organizational conditions that may influence the implementation process.1 1Neal Gross et_al,, pp, 513,, p. 206. BIBLIOGRAPHY BIBLIOGRAPHY Argyris, Chris. "Individual Actualization in Complex Organizations," in Fred Carver and Thomas Sergiovanni (eds.), Organizations and Human Behavior. New York: McGraw Hill Book Co., 1969, 189-199. . "Personality and Organization Theory Revisited." Administrative Science Quarterly, Vol. 18, No. 2 (June 1973), 141-167. Armore, Sidney J. Introduction to Statistical Analysis and Inference . . . for Psychology and Education. New York: John Wiley and Sons, Inc., 1966. Beal, George M. and Bohlen, Joe M. The Diffusion Process. Special Report #18. Ames, Iowa: Iowa State College, 1957. Becker, Marshall H. "Factors Affecting Diffusion of Innovations Among Health Professionals," American Journal of Public Health, Vol. 60, No. 3 (February, 1970),_294-304. Benne, Kenneth D. and Birnbaum, Max. "Principles of Changing,“ in Warren G. Bennis, Kenneth D. Benne, Robert Chin (eds.), The Planning of Change. New York: Holt, Rinehart and Winston, Inc., 1969, 328-334. Bennis, Warren G. (ed.). American Bureaucracy. U.S.A.: Aldine Pub- lishing Co., 1970. . Changing_0rganizations. U.S.A.: McGraw-Hill Book Co., 1966. . The Leaning,Ivory Tower. London: Jossey-Bass Publishers, 1973. Bennis, Warren G., Benne, Kenneth D. and Chin, Robert. The Planning of Change. New York: Holt, Rinehart and Winston, Inc., 1961. Bennis, Warren G..and Schein, Edgar. "Principles and Strategies in the Use of Laboratory Training for Improving Social Systems," in Warren G. Bennis, Kenneth D. Benne, and Robert Chin (eds.), The Planning of Change. New York: Holt, Rinehart and Winston, Inc., 9,3 - o 225 226 Bennis, Warren G. and Slater, Philip E. The Temporary Society. New York: Harper and Row, Inc., 1968. Bhola, Harbans Singh. "A Theory of Innovation Diffusion and its Appli- cation to Indian Education and Community Development." Unpublished Ph.D. dissertation, Ohio State University, 1965. Black, Max. (ed.). The Social Theories of Talcott Parsons. Englewood Cliffs, New Jersey: Prentice-Hall, Inc., 1961. Brickell, Henry M. Organizing New York State for Educational Change. Albany, New York: State Education Dept., 1961. . "State Organization for Educational Change: A Case Study and a Proposal," in Matthew Miles (ed.) Innovation in Education. New York: Teachers College, Columbia UniVérsity, 1964, 493-532. Buchanan, Paul C. "The Concept of Organizational Development, or Self- Renewal as a Form of Planned Change," in Goodwin Watson (ed.), Concepts for Social Change. Washington, D.C.: National Training Laboratories for the Cooperative Project for Educational Develop- ment, 1967, 1-9. Carlson, Richard. Adoption of Educational Innovations. Eugene, Oregon: Center for the Advanced Study of Educational Administra- tion, 1965. . "Strategies for Educational Change: Some Needed Research on the Diffusion of Innovations." Conference on Strategies for Educational Change, United States Office of Education, Washington, D.C., 1965. . "Summary and Critique of Educational Diffusion Research." Paper presented at the National Conference on the Diffusion of Educational Ideas. East Lansing, Michigan, 1968. Carver, Fred and Sergiovanni, Thomas. Organizations and Human Behavior: Focus on Schools. New York: McGraw-Hill Book Co., 1969. .452 Chin, Robert. "The Utility of System Models and Developmental Models for Practitioners," in Warren G. Bennis, Kenneth D. Benne, Robert Chin (eds.), The Planning of Change. New York: Holt, Rinehart and Winston, Inc., 1969, 297-312. Coleman, James, Katz and Menzel. "Social Processes in Physicians' Adoption of a New Drug," in Amitai and Eva Etzioni (eds.) Social Change. New York: Basic Books, 1964, 439-454. Cullen, Neil H. "Written Evaluation in Justin Morrill College." Ideas from JMC, l973. 227 DeLisle, F. H. "The Impact of the Pass-Fail System." East Lansing, Michigan: Michigan State University, 1969. (mimeographed) Eidell, Terry L. and Kitchell, Joanne M. (eds.), Knowledge_Production and Utilization in Educational Administration. Eugene, Oregon: University of Oregon, Center for the Advanced Study of Educational Administration, 1968. Farace, R. B. et_al, The Communication System of Justin Morrill Colle e. East Lansing, Michigan: Department of Communication, Michigan State University, 1970. Gardner, John W. Self Renewal: The Individual and the Innovative Society. New York: Harper and Row, Inc., 1963. Griffiths, Daniel E. "Administrative Theory and Change in Organizations," in Matthew Miles (ed.), Innovation in Education. New York: Teachers College, ColumbiE'University, 1964, 425-436. Gross, Neal, Giaquinta, Joseph B., Bernstein, Marilyn. Implementing, Organizational Innovations, A Sociological Analysis of Planned’ Educational'Change. NewfiYork: BaSic Books, Inc., 1971. Guba, Egon G. "Development, Diffusion and Evaluation," in Terry L. Eidell and Joanne M. Kitchell (eds.), Knowledge Production and Utilization in Educational Administratifin. Eugene, Oregon: University of:0regon, Center for the Advanced Study of Educational Administration, 1968, 37-63. Havelock, Ronald. "Dissemination and Translation Roles," in T. L. Eidell and J. M. Kitchell (eds.), Knowledge Production and Utiliza- tion in Egppational Administration. Eugene, Oregon: Center for the Advanced Study of Educational Administration, University of Oregon, 1968, 64-119. . Planning Educational Innovations. Ann Arbor, Michigan: Institute for Social Research, 1971. Havelock, Ronald et_al, Planning for Innovation. Ann Arbor, Michigan; Institute for Social Research, 1969. Hefferlin, J B Lon. Dynamics of Academic Reform. San Francisco: Jossey-Bass, Inc., 1969. Henderson, A190 0. The Innovative Spirit. San Francisco: Jossey-Bass, Inc., 1970. Hornstein, Harry A. SE _1. Social Intervention. New York: The Free Press, 1971. 228 Jung, Charles C., Rox, Robert and Lippitt, Ronald. "An Orientation and Strategy for Workin on Problems of Change in School Systems," in Goodwin Watson (ed.1, Change in School_§ystems. Washington, D.C.: National Training Laboratories for the Cooperative Project in Educational Development, 1967, 68-88. Katz, Daniel and Kahn, Robert. The Social Psychology,of Organizations. New York: John Wiley and Sons, Inc., 1966. Katz, Elihu. "The Social Itinerary of Technological Change: Two Studies on the Diffusion of Innovation," in Warren Bennis, Kenneth Benne and Robert Chin (eds.), The Plannin of Chan e. New York: Holt, Rinehart and Winston, Inc., 1969, 230-254. Katz, E., Levin, M., and Hamilton, H. "Traditions of Research on the Diffusion of Innovations," American Sociological Review, Vol. 28 (1963), 237-252. Likert, Rensis. "The Nature of Highly Effective Groups," in Fred Carver and Thomas Sergiovanni (eds.), Organizations and Human Behavior. New York: McGraw-Hill Book Co., 1969, 356-367. Lin, Nan et_al. The Diffusion of an Innovation in Three Michigan High Schools: Institution Building,Througanhange. East Lansing, Michigan: Institute of International Studies in Education and Department of Communication, Michigan State University, 1966. Lionberger, Herbert F. Adoption of New Ideas and Practices. Ames, Iowa: Iowa University Press, 1960. . "Organizing for Implementing Changes in Education: Some Implications from Agriculture and Diffusion Research." Paper pre- sented at the Seminar on the Development and Coordinating of Research in Vocational and Technical Education, Columbus, Ohio: Ohio State University, 1966. McGregor, Douglas. The Human Side of Enterprise. New York: McGraw- Hill Book Co., 1960. Miles, Matthew. (ed.). Innovation in Education. New York: Teachers College, Columbia University, 1964. . . "Innovation in Education: Some Generalizations," in Matthew Miles (ed.), Innovation in Education. New York: Teachers College, Columbia University, 1964, 631-662. . "Some Properties of Schools as Social Systems," in Goodwin Watson (ed.), Change in School Systems. Washington, D.C.: National Training Laboratories for the C00perative Project for Educational Development, 1967, 1-29. 229 Miles, Matthew and Lake, Dale. "Self-Renewal in School Systems: A Strategy for Planned Change," in Goodwin Watson (ed.), in Concepts for Social Change. Washington, D.C.: National Training Laboratories for the Cooperative Project for Educational Develop- ment, 1967, 81-88. Miles, Matthew et 91, "The Consequence of Survey Feedback: Theory and Evaluation:1i in Warren Bennis, Kenneth Benne, Robert Chin (eds.), The Planning of Change. New York: Holt, Rinehart and Winston, Inc., 1969, 457-467. Mitchell, William G. "Communication of an Educational Innovation in an Institution of Higher Learning." Unpublished Ph.D. dissertation, Michigan State University, 1970. Mort, Paul R. "Studies in Educational Innovation from the Institute of Administrative Research: An Overview," in Matthew Miles (ed.), Innovation in Education. New York: Teachers College, Columbia University, 1964, 317-328. Nie, Norman, Bent, Dale H., Hull, C. H. Statistical Package for the Social Sciences. New York: McGraw-Hill Book Co., 1970. Palola, Ernest G. and Padgett, William. Planning for Self-Renewal. Berkeley, California: Center for Research and Development in Higher Education, University of California, 1971. Rogers, Everett M. Diffusion of Innovations. New York: The Free Press, 1962. . "The Communication of Innovations in a Complex Institution," Educational Record, Vol. 49, No. 1 (Winter 1968), 67-77. . "The Communication of Innovations: Strategies for Change in a Complex Organization.“ Paper presented at the National Confer- ence on Curricular and Instructional Innovation for Large Colleges ang Universities, East Lansing, Michigan: Michigan State University, 9 6. . "Social Structure and Social Change." American Behavioral Scientist, Vol. 2 (June 1971), 767-782. Rogers, E. and Jain, Nemi C. "Change in Small Schools." Paper pre- sented at a National Working Conference of N-FIRE, Denver, 1969. . "Needed Research on Diffusion Within Educational Organizations." Paper presented at the National Conference on Diffusion of Educa- tional Ideas. East Lansing, Michigan: Michigan State University, 1968. 230 Rogers, Everett M. and Shoemaker, F. Floyd. Communication of Innova- tions: A Cross-Cultural Approach. New York: The Free Press, 1971. Rogers, Everett M. and Svenning, Lynne. Managing Change. Burlingame, California: Operation PEP, l969 (mimeograph report) Ryan, Bryce and Gross, Neal C. "The Diffusion of Hybrid Seed Corn in Two Iowa Communities." Rural Sociology, Vol. 8 (1943), 15-24. Schein, Edgar H. "The Mechanisms of Change," in Warren G. Bennis, Kenneth D. Benne, Robert Chin (eds.), The Planning of Change. New York: Holt, Rinehart and Winston, 1969, 98-107. Shaw, Walter. "Senior Student Perceptions of Justin Morrill College, Class of 1969." East Lansing, Michigan: Michigan State University, 1969. (mimeographed) Simon, Julian L. Basic Research Methods in Social Science. New York: Random House, 1969. Slevin, Dennis P. "The Innovation Boundary: A Specific Model and Some Empirical Results," Administrative Science Quarterly, Vol. 16, No. 4 (12/71), 515-532. Thompson, Victor. "The Innovative Organization," in Fred 0. Carver and Thomas J. Sergiovanni (eds.), Organizations and Human Behavior. New York: McGraw-Hill Book Co., 1969, 392-403. Trist, E. C. "On Socio-Technical Systems," in Warren G. Bennis, Kenneth D. Benne, Robert Chin (eds.), The Planning of Change. New York: Holt, Rinehart and Winston, Inc., 1969, 269-282. Watson, Goodwin (ed.). Change in School Sy§tems. Washington, D.C.: National Training Laboratories for the Cooperative Project for Educational Development, 1967. . Concepts for Social Change, Washington, D.C.: National Training Laboratories for the Cooperative Project for Educational Development, 1967. . "Toward a Conceptual Architecture of a Self-Renewing Social System," in Goodwin Watson (ed.), Change in School Systems. Washington, D.C.: National Training Laboratories for the COOpera- tive Project for Educational Development, 1967, 106-115. Webb, Eugene J., Campbell, Donald T., Schwartz, Richard D., and Sechrest, L. Unobtrusive Measures: Nonreactive Research in the Social Sciences. Chicago: Rand McNally and Co., 1966. APPENDICES APPENDIX 1 CATEGORIES AND QUESTIONS T0 ANALYZE THE JMC WRITTEN EVALUATION FORMS 231 - -.--..-.—-— (1-6) (7) (8) (9)-(12) (13) (141-(15) (16)-(18) Questions (20) l. (21) 2. (22) 3. (23) 4. (24) 5. (25) 6. 232 APPENDIX 1 CATEGORIES AND QUESTIONS TO ANALYZE THE JMC WRITTEN EVALUATION FORMS Student Number Lowerclass (l) Upperclass (2) Natural Science (1), Social Science (2), Humanities (3), Language (4), Inquiry & Expression (5), Field Study (6), Independent Study (7), Senior Seminar (8). Name of Instructor (9,10,11) Full Time 1 Part Time_2_(12) Fall '70 1 Winter '71 2 Spring '71 3 Fall '71 4 Winter '72 5 Spring '72 6 Fall '72 7 Winter '73 8 G.P.A. to nearest 10th Percent of forms handed in for designated term (000-100) 15 there a course description, i.e., more than a general or specific course title? vase). No.81. Are there bases for evaluation? Yes (1) No (2) Are there specific course objectives listed, either sepa- rately or at the end of the list of general objectives? Yes (1) No (2) Has the instructor checked off the student's level of per- formance on the relevant college and/or course objectives? Yes_1_u_ ~o_<2L Has the instructor written an evaluation of the student's performance? Yes_U_L ~o_1_2_1_ Has the instructor assessed the student's performance on at least two general college objectives in his/her written evaluation? Yes (1) No (2) 233 (26) 7. Has the instructor assessed the student's performance on explicit course objectives in either the grid or the written evaluation? Yes (1) No (2) (27) 8. Has the instructor commented upon any_of the following types of student activities: Class partic1pation, class prepara- tion, work on papers, test performance? Yes_Ll_l_ ~o_1_2_L (28) 9. Has the instructor commented on both strengths and weak- nesses in the student's performance? Yes (1) No (2) (29) NUMBER OF YES RESPONSES IN ITEMS 1-9; DEGREE 0F COMPLETION SCORE FOR FACULTY. (30) 10. Has the student checked off his level of performance on the relevant college and/or course objectives? YeSLL NOJZL (31) 11. Has the student written an evaluation of his own perform- ance? Yes (1) No (2) (32) 12. Has the student assessed his/her performance on at least two general college objectives in the written evaluation? Yes (1) No (2) (33) 13. Has the student assessed his/her performance on explicit course or personal objectives in either the grid or the written evaluation? Yes (1) No (2)__ (34) 14. Has the student commented upon any_of the following types of student activities: Class participation, class prep— aration, work on papers, test performance? Yes (1) No (2) (35) 15. Has the student commented on both strengths ang_weaknesses in his/her performance? Yes (1) No (2) (36) NUMBER OF YES RESPONSES TO ITEMS 10-15: DEGREE OF COMPLE- TION SCORE FOR STUDENT. (37) Pass No Credit APPENDIX 2 FACULTY AND STUDENT ATTITUDE SURVEYS, RESULTS, AND COVER LETTERS 234 235 APPENDIX 2 FACULTY AND STUDENT ATTITUDE SURVEYS: RESULTS, AND COVER LETTERS TO: FROM: Neil Cullen RE: Written Evaluation System At the beginning of Winter Term 1971, we asked that JMC faculty complete a questionnaire on the written evaluation system. As you will recall the system is a two-year experiment, and we must make a presentation to the University Curriculum Committee concerning the effectiveness of written evaluation. Since we need a current reading of faculty opinion, we would like you to complete a similar questionnaire to help assess whether familiarity has bred fondness, contempt or some- thing in-between. It is extremely important that we obtain honest and complete results from the survey if we are to make an accurate reading of its effective- ness. We hope that no one will feel that JMC must eventually submit a positive evaluation of the system. If it is not working well, it is important to assess why. It is also important that we get responses from all faculty. For this reason we have asked you to sign the form. In adETtion, knowing the name of the respondent will permit us to deter- mine if there are differences of opinion according to program, length of time using the form, amount of teaching time in JMC, etc. Since timing is important, please return the completed questionnaire to Mrs. Rhines in 151 Snyder, no later than Friday noon, October 15, 1971. 10/6/71 dr 236 FACULTY RESPONSE TO THE gmc WRITTEN EVALUATION SYSTEM N=4O 1. Have you sensed any change in student attitude or student performance that might be attributed to the new evaluation system? Yes 27 No 8 If yes, please check those changes you have observed below and add any changes you have Observed but are not included in the list. Affected Some Affected Few Affected Most Students - Students - £flgnge, Students 50% + About 113 Less than 1/10 __9__ More Anxious 4 5 _24_Less Anxious l7 1 5 l _lZ_Less Competitive 13 4 ._l§_More Indifferent 1 5 l 6 _12_Worked Less 1 8 1 9 _19_Worked More 3 7 __3_ igher Caliber Work 1 5 7 _lQ_Lower Caliber Work 3 2 5 __§_Less Self-Disciplined 2 2 l 3 __§_More Self-Disciplined l 5 7 13 More Cooperative 6 l 5 l Please add any further comments. * Student takes more risks, more experimenting, more Opinion giving. More attention to process vs. content on part of student. Less anxiety over competition and grade seeking. Good students do as well or better, poor students less well than before. Greater anxiety on the part Of some due to not knowing or a change in bases for evaluation. Less ansiety and it lowered enthusiasm for hard work. * *Frequent response. 237 2. DO you believe that the new evaluation system allows for a better picture of a student's performance than a numerical, letter or other scaled system? Yes 31 NO 6 If no, why not? If yes, please examine the list below and check those characteristics which you feel make a "better picture." Please add characteristics not included. 25 More Detailed 30 Can Reflect Weaknesses and Strengths 30 More Individualized 24 Can Reflect Progress Made vs. Ranking on an Absolute Scale. 17 Forces Teacher to Evaluate More than Intellect Alone. Further Comments? New system may tell the student more than he wishes to know. More difficult to compare students. Who is going to read the greater detail? Why not Simply add written evaluation to a grading system? Only know a few students well enough to give more information than would be reflected in a grade. 3. Did the new evaluation system lead to a change in the organization of your course? Yes 21 No 14 If yes, please check any items below which reflect a modification you made. Please add any not listed. 11 Kept More Complete Records 2 Changed Course Content. 12 Developed Clearer Objectives 9 Changed Design of Exams 8 Changed Instructional Style Further Comments? Asked for more feedback from students--papers, reports, exams, etc. Can get students to try new experiences without fear of being graded. I am freer to criticize Since I don't have to worry about ranking a given paper with others. I sense that students demand more goal setting and organization on part of teacher not less. I have had to Spend more time motivating students. 4. 238 Did the new evaluation system require additional time on your part? 11.9.8. 11.9. During the term 23 11 At end of term 37 None 0-15 Min 15-30 Min 30-60 Min More Increased Time per Student: During Tenn 6 7 4 3 6 End of Term 7 1 I4 10 2 If yes, check how the additional time was Spent. Please expand list if necessary. 14 Record Keeping 34 Writing Evaluations Paragraphs 15 Student Conferences 8 *Distributing and discussing forms in class 00 you have suggestions for reducing the time spent? * Evaluate in written form by request only. * Key phrase file; sample comments. * NO, good evaluation takes time, and it is worth it. Did you find the "Guidelines for the Instructor" provided with the evaluation forms helpful in using the evaluation form? Yes 30 NO 8 Why or why not? *Clarified and should make for more consistency among faculty. No aid on how to use form at beginning of term. How does one avoid "pat" phrases? Did you find the modified version of the evaluation form an improvement? Yes 36 NO 4 Why or why not? Yes - *Simpler and shorter, not repetitious No - Eliminated student Objectives and expectations Not enough rm. for course Objectives Had just gotten used to first one 7. 10. 239 Did you find the JMC General Objectives listed on the evaluation form provided some useful categories of evaluation? Yes 30 No 8 Why or why not? Yes, we must keep college goals in mind; reminders of where courses fit. Yes, we must get beyond course Objectives alone. Yes, remind us of what we should be evaluating. NO, they are irrelevant to course goals. No, they are Mickey Mouse. NO, they are too general. Should JMC General Objectives continue to be listed on the evaluation form? Yes 28 No 7 Why or why not? Yes, they help organize form, provide "continuity in evaluation from course to course, term to term, etc." Yes, we Should consider them more when designing courses. Yes, but instructor Should develop own Objectives. NO, stick to course Objectives Only. Did you list course and/or program Objectives on the form? Yes 23 No 16 If so, where did you put them and what format did you use? Yes, under #5, on back, under #2, in #4, at top, separate sheet Not enough room for them NO, discuss them only Sometimes Did you find the "check-Off" section of the evaluation form useful? Yes 26 NO 10 What are its strengths and weaknesses? Stren ths Weaknesses Good for profiling Must evalfiate person vs. per- *Simple, clear formance; unjust Forces evaluation in several categories *Just another form of grading *Compliments written paragraph *Not enough categories Too simple *I write on the lines 11. 12. 13. 14. 240 Did you find the written paragraph evaluation useful? Yes 35 NO 4 What are its strengths and weaknesses? Strengths Weaknesses *Complements check-off *One just uses 'pat' phrases *More flexible & individualized *Faculty are not thorough Can comment on personal growth *Time consuming as well as skills Who reads it? Can comment both on performance NO real criticism and capacity I don't know most students well enough to write one Did you use the reverse side Of the form for any purpose? Yes 8 NO 28 If so, for what purpose? Yes, program Objectives Yes, course objectives Yes, continue written comments Did you use the evaluation form early in the term to review college and course Objectives with your students? Yes 18 No 20 For any other reasons? 00 you think that, as forms go, this one could be improved? Yes 25 No 5 If yes, how? *-More space to 2, 5 & 6 - drop #4 -Use as supplement to numerical grades - More Space for #3 *-Include more room for course Objectives and bases for evaluation -Eliminate all but written paragraph 15. 16. 17. 18. 241 Do you think the student benefits from the student self-evaluation? Yes 25 NO ‘7 Why or why not? *It forces self-examination, not comparison with others. If he takes it seriously, a big if. No, most do a poor job and don't care. *No, they evaluate course vs. their own performance 00 you think the faculty member benefits from the student self- evaluation? Yes 27 NO 8 Why or why not? Yes, offers student perspective on himself, sometimes different from faculty. Yes, if he is honest. NO, I don't read them No, student takes it as a joke Should the student evaluations be on the same form or on a separate form than the faculty evaluation? Same l7 Separate 12 Why? *Same, convenient. Same, for counterpoint. Same, feedback should be shared between student and faculty. Separate to assure honesty. Separate to assure separate evaluations. *It doesn't matter What did you find was the most serious difficulty with the system so far? (Be specific) Not knowing whether it disadvantages the students in jobs, etc. Using 'pat' phrases for most students, not any more thorough than grades. Not enough information given in practice by most faculty. Not done on time, student never bothers to read them. *Timell Evaluating the mid-range students. *Confusion as to intent Of system on part of students and faculty. Many students do the minimum under this system. 19. 20. 21. 242 What did you find were the most positive results of the system SO far? (be specific) *Ideally the criteria for evaluation are made explicit. *The thoroughness Of the evaluation. *More attention to learning on part of students and faculty. *Personalizing evaluation vs. competitive evaluation highlights di- vergent learning styles, backgrounds and makes Obvious that comparative grading is inappropriate. It is forcing a re-evaluation on the part of students and faculty alike of their goals for learning. *Less Anxiety. Objectives--course, program and college-~are clarified and high- lighted. . How many terms have you used the written evaluation system? Terms Persons 1 4 2 4 3 l7 4 12 6 1 Approximately how many students have you evaluated by using the written form? Most have evaluated more than 80 students. All but one have evaluated at least 10 students. Since this data will be analyzed in a variety of ways, please sign the questionnaire. General Comments: Several persons mentioned that we should seriously consider adding written evaluation to a graded system. What does one do with students not motivated by our present system? Some persons found change of form frustrating since they had just gotten used to first. Reasons for change unclear. 243 MICHIGAN STATE UNIVERSITY East Lansing ..Mlchigan. 48823 Justin S. Morrill College November 2, 1971 Dear As you know, JMC is in the midst of a two year experiment designed to determine whether or not a P-N, written evaluation system is an effective means of measuring student achievement in courses. You are one of several students we are asking to complete the enclosed questionnaire to help us answer an important question: Should JMC cOntinue written evalu- ation or return to the more familiar letter grading_§yStem? Since we do not have the staff to question every JMC student, it is especially important that you complete the questionnaire. Without your honest and thorough responses it is unlikely that we can obtain an accur- ate reflection Of student Opinion. Please do not sign the questionnaire. It is important only that we know your class standing and your planned field Of concentration in addition to your responses to the various questions. Please return the completed questionnaire to Mrs. Dorothy Rhines in 151 Snyder no later than 5:00 p.m., Monday November 8, 1971. She will ask your name so that we may know who has not returned the questionnaire. Thank you for your help. Sincerely, Neil H. Cullen NHC:dr P.S. --It should only take lO-15 minutes--honest! --and your efforts will have an impact on the eventual decision. 244 STUDENT RESPONSE TO THE JMC WRITTEN EVALUATION SYSTEM (N=82 1. How has the evaluation system affected your attitude in JMC courses? (check)the appropriate blank(s) and add additional categories if needed 7 not at all 30 consider taking a wider 44 more self-directed varTety of courses _fl§_1ess competitive 9 *Less Anxious 22 less motivated 21 more motivated 2. How has the evaluation system affected your performance in JMC courses? (see above instructions) _lfl_not at all 14 work more cooperatively with fellow students _41_do less work in dull courses _2§_do more work outside class _§Q_learn more __4__1 earn less 3. How has the new evaluation system affected your attitude or perform- ance in non-JMC courses? 19 not at all 23 improved attitude because there are fewer grades to 48 d1sl1ke pressure for grade worry about in university courses 25 work harder in courses where there is a letter grade 1 work less in university course *Work harder for grade Since GPA still important and can pass JMC course with minimum effort. *Frequent response. 245 4. How has the evaluation system affected your classmates in JMC courses? __§_not at all __g__more work _3_§__less competitive __3_l___more idea sharing _22_less motivated __2_more motivated 13 less work 5. How have your instructors responded to the evaluation system? 6 not at all 20 most do not discuss the 4 most dislike it purpose of the system 31 most like it 37 confused 26 most discuss the purpose of the system 6. How does the written evaluation system allow for a better picture Of a student's performance than a numerical, letter or other graded system? __flrjt doesn't 71 It reflects weaknesses and _§9_1t Permits evaluation of strengths more competencies 54 it is an individualized not a ranked system 64 it allows comments on progress made as well as level Of skills 7. Were the JMC General Objectives listed on the form useful? How or why not? 18 they were not 19 they were irrelevant to the 12 summarize the aims of course education 9 some applied to each class 16 set specific expectations for students 33 the teacher did not emphasize the JMC Objectives. 246 8. Did you find the "check-Off" system of evaluation useful? How or why not? 19 it was useless 18 it forces the student and - . professor to remember the -Z§—;fiagg:sgftd?}}gxezgg important areas Of evalua- tion throughout the term 21 _it Offers a good summary —Of skill levels 13 it is little different from grading 9. Did you find the "paragraph“ of evaluation written by the professor helpful? How or why not? 1n 14 the professor doesn't —know the student well 63 —iEeificfig::_ personal than enough to comment 61 it allows more complete _analysis of performance 8 it is too subjective 10. How can the written evaluation be improved? 3 _shorten 15 longer 22 _add more space for student —expectations 36 eliminate the “check-Off" 4 eliminate the evaluation paragraph 11. What is the merit Of student self-evaluation? 8 _there is none 13 the student's view does not 43 helps students establish effect the ultimate evaluation objectives for his education __4_encourages honest self _§fl_adds balance to the teachers appraisal evaluation 46 _emphasizes the student' S —responsibility for his education 12. 13. 247 Does the written evaluation system seem more appropriate for some courses than others? Yes 34 NO 39 If yes, which courses seem best ang_least suited for written evalu- ation? B_e_§_t_ Least _B_e_si fist __7_ _14_ Natural Science __9_ _l_9___ Languages _QEL_ __l___Social Science _20__ __ Humanities _33;_ 5 Field Study Now that you have had experience with both graded and non-graded evaluation systems, which do you prefer and why? Ung,(without qualif.) Ung.(w. qualifliGr. 'Combo 43 *less pressure 7 *but I work 10 *I work hard- 14 -for grad. *more evaluation harder in er schools, we *set own goals graded *P-N allows need some *individualized courses you to do grades nothing -use grades I'm brain- for motiva- washed tion, in some *I know where courses I stand 14. 15. 16. 17. What is the most serious difficulty with the written evaluation sys- tem thus far? *Interference of GPA pressure and N.G. written evaluation system, I feel schizoid. *Confusion on part Of both faculty and students *Profs don't know students well enough *Profs don't take system seriously What is the best aspect of the written evaluation system thus far? *Less Pressure *More evaluation, better rapport *Less competition *Individualized *Home self-definition of goals How many credits have you taken under the written evaluation system? 2 O-6, 8 7-12, 10 13-18, 11 19-24, 29 25-35, 30 36-90 How many university credits have you taken? 22 15-30, 6 31-45, 21 46-90, 21 9O upwards -.L s an...— 248 18. Did you read your evaluation forms after they were completed? Yes 69 NO 5 Some 4 Please check appropriate blank Class level fr. 18 so. 35 jr. 18 sr. Sex 26 m 49 f Planned field of concentration (check more than 1 if appropriate, e.g., natural science, education) __29__1anguages ___J§_ fine arts ___;3__performing arts __18_ humanities ___Z__natural Science __3_5_social science 18 education Repeated Comments Profs don't take evaluation (paragraph and check-Off) seriously. Evaluation paragraphs too Short, not descriptive of performance, less certain of where I stand than in a graded course. Profs don't know students well enough to write evaluations. 1/10/72 Sf APPENDIX 3 JMC WRITTEN EVALUATION FORMS: FALL TERM, 1970 AND WINTER TERM, 1971-WINTER TERM 1973 249 -L 250 APPENDIX 3 JMC WRITTEN EVALUATION FORMS: FALL TERM, 1970 AND WINTER TERM, 1971-WINTER TERM,1973 JMC WRITTEN EVALUATION FORM, FALL 1970 Student Class: Fr. So. Jr. Sr. Student No. Term, l9___ Instructor Course Title 8 Number Credits Brief Description: Student's Objectives in Taking the Course and Background in this Area: Bases for Evaluation: NO. No. Written Assignments Discussions Lead Reading Examinations Papers Class Attendance Projects Other-Specify: Visual/Oral Pre- sentations Student's Evaluation Of Personal Goals set: Instructor's Evaluation of Specific Competencies/Deficiencies Demon- strated and Of Student's Growth During the Term: Would you foresee difficulty in further study in this area? Has the student been working up to the level of his capacity? Yes No Action Taken Regarding Credit: Pass No Credit From what you know of this student through this course, would you recom- mend him for Honors College or other honors organizations? (Phi Beta Kappa, Mortar Board, etc.) Yes No JMC Objectives 1. Demonstrated Communication Skills--written Spoken. 2. Demonstrated ability to acquire information. 3. Demonstrated ability to evaluate information. 4. Demonstrated ability to Synthesize/Integrate knowledge. 5. Demonstrated ability to study independently. 6. Demonstrated ability to work in groups. 7. Demonstrated creative ability. 8. Demonstrated intercultural awareness. Program Objectives 1. To be determined by program 2. Director and Staff. 3. Course Objectives 1. To be determined by 2. Instructor of course° 3. 251 Fall Term, 1970 Student Faculty Evaluation Evaluation cum we) r—U) r—m as. .05. 88 8= ‘PU '0-8 E '0 Q’f— f5 f0 0.!- E (J: E E <.: 0v- n- 41-, 'l— ‘l- «OJ «5 3.5 So .5 3.5 'So * z z 244 z z z-u * * l i f f *Instructions: In this box, rank the emphasis given in this course to the stated objectives, giving the highest ranking objective the lowest number (1) and the lowest ranking objective the highest number. 252 WINTER 1971-WINTER 1973 JUSTIN MORRILL COLLEGE - CONFIDENTIAL - COURSE WRITTEN EVALUATION 1) Student (print) Student No. Class: (circle one) Fr. So. Jr. Sr. JMC Student? Yes No Course No. JMC JMC Discipline Section #____Credits____ Instructor Term 197 2) Title and Brief Course Description: 3) Bases for Evaluation: 4) General Objectives A. Student Self- B. Instructor Evaluation (voluntary) Evaluation Demonstrated: l 2 3 4 1 2 3 4 1. Skill in writing . . . . . . . . Skill in speaking. . . . . . Ability to acquire informa- tion . . . . . . . . . . . . Ability to evaluate inform- ation. . . . . . . . . . . . . Ability to synthesize information. ........ Ability to study inde- pendently. . . . . . . . . . Ability to work in groups. . Creative ability . . . . . . 0 01 b CON 0 Otomxl m 0 Key: 1. Objective outstandingly met; 2. Objective met; 3. Objec- tive not met; 4. Not applicable. 5) Instructor's written evaluation Of student's performance in course objectives: 6) Student's voluntary self-evaluation of personal and course goals met. (Not an instructor or course evaluation.) 7) Would you recommend this student for Honors College or other honors organizations? 8) Action taken: (circle one) Pass No Grade Instructor's signature APPENDIX 4 JMC STUDENT PROFILE--TWO EXAMPLES 253 254 APPENDIX 4 JMC STUDENT PROFILE--TWO EXAMPLES STUDENT PROFILE STUDENT: MSU STUDENT NUMBER: DATE: March 13, 1973 Attended Justin Morrill College (JMC), Michigan State University from Fall term 1969 to present . Earned 153 credits at Michigan State, 96 of which were on the 4.0 numerical scale with a cumulative grade-po1nt average of 3.70 . Transferred 0 credits to Michigan State University from other institutions. Earned 60 credits of "P" (Pass) in JMC courses under the written evaluat1on system, a grading system used in all JMC courses. Each in- structor completes a written evaluation to sfiifilement the "P" or "N" (no grade-no credit) which he or She gives the student. Eleven (11) written evaluations appear in the student's college records. Aprofile summary of these written evaluations appears below. Katharyn's work in Justin Morrill has been consistently good. Her strengths lie in the areas of evaluation, synthesis and acquisition of knowledge; one social science teacher noted that Katharyn "showed the ability to find and integrate material from outside Of the course," another commended her "outstanding'l performance, while another men- tioned her ability to apply knowledge gained in one situation to another. Katharyn handled independent study successfully in several courses. Her generally quiet manner inhibited her effectiveness in some courses; one sociology professor stated Katharyn barely met course Objectives since her visible performance was "minimal" and another questioned her "passive acceptance" Of class materials. In general, however, Katharyn's con- scientiousness seemed to more than compensate for her minimal verbal participation. Elizabeth Cullen Charles K. Niles Profile Writer Assistant Dean Justin Morrill College Justin Morrill College N.B. This profile is npt_a letter Of recommendation but rather a summary Of student competencies. The profile writer does not know the student, is not a member of the college staff, and bases the profile solely on the information available on the written evaluation forms. 255 STUDENT PROFILE STUDENT: MSU STUDENT NUMBER: DATE: April 2, 1973 Attended Justin Morrill College (JMC), Michigan State University from Fall term 1971 to present . Earned 77 credits at Michigan State, 37 of which were on the 4.0 numerical scale with a cumulative grade-point average of 2.72 . Transferred 0 credits to Michigan State University from other institutions. Earned 36 credits of "P" (Pass) in JMC courses under the written evaluation system, a grading system used in all_JMC courses. Each in- structor completes a written evaluation to supplement the "P" or "N“ (no grade-no credit) which he or she gives the student. Nine (9) written evaluations appear in the student's college recordS} A profile summary Of these written evaluations appears below. Charles' academic performance varied from acceptable to excellent. He demonstrated a "striking capacity to express himself effectively in both oral and written form" in a literature course, wrote a "well thought out and well written" paper in Natural Science, and "made invaluable contributions to class" in History. Professors noted his creative ability and strong narrative prose, and cited examples of skillful acquisition, analysis and synthesis of knowledge. On the other hand several teachers mentioned Charles' disruptive behavior in class; in one he was termed "negative" and "argumentative," in another "insensitive." In these classes his academic work met Objectives but his behavior prevented him from realizing the potential others saw in him. Elizabeth Cullen Charles K. Niles Profile Writer Assistant Dean Justin Morrill College Justin Morrill College N.B. This profile is not a letter of recomnendation but rather a sunmary of student competencies. The profile writer does not know the student, is not a member of the college staff, and bases the profile solely on the information available on the written evaluation forms. APPENDIX 5 WRITTEN EVALUATION PROPOSAL DISTRIBUTED TO JMC FACULTY AND STUDENTS APRIL 23, 1970 256 257 APPENDIX 5 WRITTEN EVALUATION PROPOSAL DISTRIBUTED T0 JMC FACULTY AND STUDENTS APRIL 23, 1970 TO: JMC Students and Faculty FROM: Barbara Ward for the Curriculum Committee DATE: April 23, 1970 The attached proposal is currently under study by the JMC Curriculum Committee. Since implementation Of this proposal would effect every student and faculty member Of the college, we feel it is imperative to solicit the best available thinking about the proposal and its implications. You can assist the committee by carefully studying the attached proposal and supporting documents and then thoughtfully responding to them. The committee has scheduled an open hearing on the proposal for Wednesday evening, April 29, 8:00, 118 Physics-Astronomy Building. Committee members will be present to answer questions and/or to clarify statements in the documents. Primarily, however, we are asking for your response and providing a time when you may be heard. The committee will be "hearing". If you cannot attend the meeting, any committee member will be happy to talk with you or to receive your written comments about the proposal. The committee is Optimistic about the new potentials this proposal seems to Offer. We look forward to your involvement in the examination Of this possible future for JMC. Committee Membership: Jim Goatley Diana Scholberg Bill Halvangis Roger Stimson Cindy Keils Herm Struck George Lupone Wendy VanSyckle Fay Maffei Barbara Ward, Chairman David Winter, ex Officio 111 258 Proposal: All JMC instructors would provide written evaluation for each student in all JMC courses. These written evaluations would be recorded and/or summarized and made available to other agencies at the request of students. Only "Pass“ and _ "No-Credit" would be recorded on the MSU transcript with an explanatory note indicating the nature of our evaluation system. Rationale: 1. The committee recognizes a difference between numerical grading and individualized evaluation. The former is more likely to compare stu- dent with student, ranking all progress within some kind Of common continuum. It would seem preferable to provide a context and environ- ment within which individual student responses and learning experi- ences were encouraged; where recognition of Specific strengths and weaknesses, competencies and deficiencies would be a valuable part of the education process. Students and faculty would jointly be involved in the evaluation process. This is a unique and integral aspect of the proposal. The evaluation form was designed in response to the expressed concerns of students and faculty. As noted, it is titled a "student-course evaluation". Use of this form may facilitate a clearer definition by the instructor of his course Objectives, method Of evaluation, teaching style and expectations for students. It will also make the goals of the college and Of programs within the college more visible. Students may enroll in a course for different reasons and with vastly different backgrounds in the area. One may expect the course to support his area of concentration. Another may wish to tap an un- known discipline while another may be attracted to a particular in- structor or time Of day. Evaluation of a student's progress should be made in light of his stated purposes and/or goals for the course with consideration given to his competence, experience or knowledge prior to enrolling in the course. The individualized written evaluation system will produce considerably more--not 1eSS--information about the student's abilities. During the years in college, this may be of valuable assistance to academic 259 advisors as they work within a curriculum which presents many altern- atives but requires many choices. The additional information will assist faculty who may desire to write letters of recommendation for a student's acceptance into honors college or honorary organizations. Faculty could then support the recommendations with reference to Specific qualities, demon- strated abilities and Objectives as reflected by the evaluation form. At the end of four years, a letter of recommendation could be written, drawing from the evidence on the forms and the intent Of the evalua- tion process. This letter, Signed by the Associate Dean and the Advisor, would become the Official college recommendation and could be used in conjunction with the MSU transcript at the student's request in applying for a job or admission to graduate School. The committee believes that the F grade serves no positive educational purposes. If one wishes to think in terms of penalty for lack of accomplishment, there is penalty implicit in the loss of time and money for the student. The grades Of 1.0 and 1.5 are seldom satis- factory tO professor or student as they reflect a dubious level Of accomplishment. Work at that level would receive an N-No Credit. A proposed minimum level equivalent to the present 2.0 is suggested for the P-Pass. Having been commissioned as an experimental college, JMC is a logical place within the university to design and implement a different evalu- ation system. If such a system were approved for a specified length Of time, our students and faculty could then evaluate its effective- ness and feasibility (both demonstrated and projected) and make this evaluation available to the university. 260 (sample of possible instructions) Michigan State University Justin Morrill College STUDENT-COURSE EVALUATION SHEET Instructions to Students At the beginning of the term you are requested to indicate your name, class standing, and student number on an evaluation sheet for each course. The Sheet will provide the course number and title, a brief description, several bases for final evaluation in meeting set Objectives, and a number of suggested college, program, and course Objectives. In the space provided you will indicate your specific reasons for selecting the course, your personal objectives, and an indication of the background which you may have in the area in which the course is offered. At the termination of the term you will have the option of sharing with the instructor before he records his evaluatiOn of your involvement your own evaluation concerning the degree to which the experience has assisted you in meeting the Objectives set. You may record your personal evaluation of your individual progress in statement form on the front page as well as to check on the back page the degree to which you feel you have met college, program, and course Objectives. It is possible that some of the Objectives listed may not-apply to the specific course. It is also possible that some of the Objectives thought to apply in the course planning may not seem as applicable to you after taking the course. This information will assist in evaluating the course as well as your experience with it. It will also permit for an overview of your general college experience when a number Of forms from a variety Of courses are reviewed. Instructions to Faculty Before requesting students to complete any part of the form please complete the form with the necessary indication of course number and title, credit involved, term given, and brief description. In addition, record the general and Specific Objectives which you have set for the course in the space provided on the back. These are to be determined in keeping with the goals Of the college, the specific goals of the pro- gram within which the course is Scheduled, and your own goals relative to style and content of the course. This information can be preprinted on the forms. At the beginning of the term, pass the form to the students in order for them to indicate their reasons for enrolling. 261 At the end Of the term you are requested to have those students, who may wish, indicate their own evaluation based on the objectives set. Briefly indicate your evaluation Of his experience based on the same objectives, state the specific competencies and/or deficiencies demon- strated in respect to these Objectives, and the progress evident by the student during the course. It may prove helpful to the student and his adviser if you would indicate the additional work or experience for which you could now recommend him. Indicate whether or not the Student is eligible to receive credit. The form will be used by the student's adviser in assisting him to select courses within the remainder of his college program. As he approaches senior status, it will be used to develop a progress sheet which can be used in certifying his experiences and skills for planning beyond graduation. APPENDIX 6 WRITTEN EVALUATION PROPOSAL FORWARDED TO THE UNIVERSITY CURRICULUM COMMITTEE, May 7, 1970 262 263 APPENDIX 6 WRITTEN EVALUATION PROPOSAL FORWARDED TO THE UNIVERSITY CURRICULUM COMMITTEE, May 7, 1970 On July 11, 1968, a Revised Grading System report was approved by the Board of Trustees. That report reads, in part: "Departments and colleges shall be encouraged to introduce modifi- cation in grading systems or new grading systems provided they are approved by the University Curriculum Committee and the Office of the Provost." (Page 31) It is within the spirit and guidelines of this report that our proposal for a written evaluation system for Justin Morrill College courses is submitted. AS an experimental college, JMC seems a logical place with- in the university tO design a different evaluation system. The advan- tages of an individualized written evaluation system most clearly relate to better response and feed-back for the student, and better description of a student's abilities and achievements for the public. We believe it is important and responsible for the college to change its system of student evaluation for a period Of time, as an experiment, and to substitute for the numerical scale a system that will not only reflect but encourage more unique and individualized responses of stu- dents to the academic challenges presented within the college. We do not seek less evaluation, but more and better evaluation: the problem is not that there is too much evaluation but that it is inappropriate for the varied and unique responses Of real people. And not only is it inappropriate as a method of description, it has the subtle but Signifi- cant effect of changing the very nature of the learning experience it- self. When both students and faculty are working toward a product that can be ranked by some sort of common measurement and expressed on a single continuum of scores this has the effect of encouraging comparable responses, in the form of standard examinations and term papers, etc. Whereas, if the student's product will be evaluated individually, in terms of the Skills and ability demonstrated rather than as a position on a continuum, we may achieve an environment and context which truly en- courages individual response, greater motivation, andairelationship between student and faculty that is significantly different. In our judgment it is irresponsible to reduce the amount of evaluation provided for our students. Not only does society expect a fair evalua- tion of the learning achieved, but faculty response and feed-back is critical in the educational process itself. For this reason we are opposed to a system Of grading in which instructors can fulfill their duties by Simply declaring that the student has passed or not-passed. 264 Evaluation is not performed well if it is simply tacked on to an aca- demic course Of study. It must be developed in conjunction with the particular and unique goals of the course. An individualized, written system will produce an evaluation that will be more appropriate to the nature Of our students. An individual evaluation cannot possibly provide less information than the existing numerical system, and we believe it will supply considerably more information, and in the proc- ess allow for a breakthrough in creative, individualized behavior by both faculty and students in response to the educational goals Of the co ege. The grade level of the present JMC student body suggests that the pro- posed system would not in any way lower the standards of the evaluation process as recorded in grades within the college. A review Of the grade point averages of the graduates (177) from Winter 1968 through Winter 1970 indicates that 54% had averages over 3.0; 16% were above 3.5; and only 12% fell between 2.0 and 2.5. A study of the total grades given within JMC courses during the Fall of 1969 (excluding the Single credit sections attached to most courses and which are generally directed toward independent study) revealed that the total grades below 2.0 were less than 4.5%, with many of these going to the same students. Half of these were at the 1.5 level. The conclusion is that 95% of JMC grades are now above 2.0. Other random studies indicate that JMC students have tended to do slightly better in their University courses. These are taken usually at the junior and senior level and are usually related to the student's area of concentration which is taken entirely within the university and outside of the college. Proposal: Faculty would provide written evaluation for each student in all Justin Morrill College courses. These written evalua- tions would be recorded and/or summarized and made available to other agencies at the request Of the student. Only "Pass" and "NO-Credit" would be recorded on the MSU transcript with a brief explanatory note indicating the nature of our evalu- ation system. A one-page student Profile of Competencies-- summarizing his work in the college--would be prepared during the last term of a student's attendance in the college. This would be given to the student and also kept on file in the college so that it could be made available to the public at the request Of the student. This system is to be used for two years and then evaluated by JMC faculty, students and administration. Rationale: 1. The college recognizes a difference between numerical grading and individualized evaluation. The former is more likely to compare student with student, ranking all progress within 265 some kind of common continuum. It would seem preferable to provide a context and environment within which individual student responses and learning experiences were encouraged; where recognition of specific strengths and weaknesses, competencies and deficiencies would be a valuable part of the education process. Students and faculty would jointly be involved in the evalu- ation process. This is a unique and integral aspect of the proposal. The proposed and tentative evaluation form was designed in response to the expressed concerns of students and fafiulty. As noted, it is titled a "student-course evalu- at on. Use of this form may facilitate a clearer definition by the instructor of his course Objectives, method of evaluation, teaching style and expectations for students. It will also make the goals of the college and Of programs within the college more visible. Students may enroll in a course for different reasons and with vastly different backgrounds in the area. One may expect the course to support his area of concentration. Another may wish to tap an unknown discipline while another may be attracted to a particular instructor or time of day. Evaluation of a student's progress should be made in light Of his stated purposes and/or goals for the course with con- sideration given to his competence, experience or knowledge prior tO enrolling in the course. The individualized written evaluation system will produce considerably more--not less--information about the student's abilities. The freedom within our curriculum places a heavy burden on the academic adviser and yet at the present time the adviser has inadequate information about the student. This additional information may assist the adviser in per- forming his function more effectively. The additional information will assist faculty who may desire to write letters of recommendation for a student's acceptance into honors college or honorary organizations. Faculty could then support the recommendations with reference to Specific qualities, demonstrated abilities and Objectives met as re- flected by the evaluation form. At the end of four years or at the time of transferring out Of the college, a letter of recommendation (Profile Of Compe- tencieS)would be written, drawing from the evidence on the forms and the intent of the evaluation process. This letter, Signed by the Associate Dean and the Adviser, would become 266 the official college recommendation and could be supplied in addition to the MSU transcript at the student's request (to the college) when applying for a job or admission to graduate school. Some questions concerning the role of a grade in the motiva- tion of students are still unanswered. Students have said they can get "good grades" without studying and also that they can put effort into studying which is not reflected by the numerical grade. In some instances, the grade has be- come more important than any learning which may lead to it. The written evaluation system would take the pressure for grades Off of students and would seem to allow for learning situations where the student's motivation could be more clear- ly understood. We believe this proposal to be a truer reflection of the educational philosophy of the college. In addition, it may provide the university with information concerning the value of this mode Of evaluation for undergraduate education. At the end Of the two year period, our students, faculty and administrators will evaluate the effectiveness and feasibility (both demonstrated and projected) Of this system and make this evaluation available to the university. We have re- quested the assistance Of the Office Of Evaluation Services for this study. In order to prepare for an evaluation Of the experiment at the end of the two year trial the following steps are anticipated: 5/5/70 3. C. e. A complete statistical breakdown of the records of the present freshmen and sophomore classes as a comparative base of the quality of two classes which have been processed under the numerical system within their JMC core courses; a comparison within the two classes of the work attempted under both sys- temS. An analysis of the evaluation sheets for all students processed in the system in order to independently assess the general level Of performance and to draw general comparisons with the above. An inventory of the faculty and students, at selected inter- vals to determine attitudes toward the system. A review Of GRE scores, honors received, and admissions to graduate schools. . Constant review of progress with the Office of Evaluation Services. APPENDIX 7 A PROVISIONAL GLOSSARY FOR THE JMC COLLEGE GOALS ON THE WRITTEN EVALUATION FORMS 267 268 APPENDIX 7 A PROVISIONAL GLOSSARY FOR THE JMC COLLEGE GOALS ON THE WRITTEN EVALUATION FORMS JUSTIN MORRILL COLLEGE . MSU Office of the Dean . 143 Snyder Hall December 4, 1970 TO: JMC Faculty FROM: Dean SUBJECT: A Provisional Glossary for College Goals on Evaluation Forms The seven college goals listed on the new JMC evaluation forms re- quire some explanation. What follows is a preliminary attempt to give minimum definition to each Of the seven goals in order that all faculty (and students) will have some sense Of a common "ballpark." Some of the goals are more self explanatory than others. We ask all faculty to consider the following "glossary" of things that we think are included in each of the seven goals, and to adapt and add what may be necessary to evaluate students on each. Remember that no single course need relate to every goal. In fact, most courses will touch on only a few. As we refine the total evaluation system, we will also be refining our sense of definition of each of these goals. In addition, we will over time probably add to or subtract from the list. We welcome your help either in further defining what we presently have, or in adding to or subtracting from it. Since these are obviously "ideal" categories, we imagine that faculty will apply them to student performance with tact and understand- ing. We are attempting to provide some target areas for consideration, not final definitions Of student behavior. We are trying to move JMC from a content-oriented, discipline- centered program toward a "competency-directed" four-year learning experience. That is, we are trying to make certain competencies (i.e., skills, abilities) such as these seven goals, not only the real end of the JMC program, but also the very bases for ongoing as well as ultimate evaluation of students in JMC. DGR:pw Attch. 1. 269 An expanded repertoire Of communication skills Includes: being perceptive in listening as well as speaking-- writing--traditional skills of form and content in Speaking and writing (grammar, spelling, paragraph development, thesis finding, etc.) as well as less Often mentioned skills in having something to say and saying it honestly and with an authentic voice (not game playing, word pushing, grade getting behavior)--able to receive as well as to give information and sensitive to clues and cues that go beyond formal communication (awareness of feelings of self and others)--aware of non-verbal communication as well as verbal. Able to acquire knowledge Includes: knowing where the knowledge is, e.g., knowledgeable persons (ability to distinguish real experts from false ones among students as well as faculty), knowing how to use libraries and demonstrating that they do use them--how to read a book--how to read a page (information plus sensitivity to more subtle reading for import), able to learn from--not simply absorb--experience, evidence of some system wherein "events" and "experience" get transformed into "knowledge" (i.e., useable concepts), able to listen to others, including formal lectures, and TV, and profit from them, able to interview, take part in discussion (give and take). Able to evaluate information Includes: evidence of a style that tests information and experience instead of merely absorbing it or swallowing it whole, evidence that he is not easily persuaded or swayed by information, by rumor, by fads, by peer culture or faculty culture, not authority-oriented, develop- ment of a questioning attitude (a Missouri attitude, "Show me“), ability to use forms of evaluation such as logic, checking with others, cross-referencing with other "authorities," weighing evidence by common sense, by uncommon sense, willingness to check intuition against empirical data, and empirical data against intuition. Ability to synthesize and integrate knowledge Includes: evidence of combinations of ideas, insights and experi- ences such as the combination Of information and insights among various classes and courses (takes an idea from French and uses it in psychology, or from geography and applies it in natural science, etc.)--evidence of willingness tO and ability for making theses or hypotheses, 1,e,, conclusions derived from experience or reading (either as conclusions or ideas to be tested)--evidence of a Style that tends constantly to pull things together rather than simply to let them lie apart-~evidence in things such as independent study and Field of Concentration of design, purpose, unity, theme--Ability to see One in the face of the Many. 5. 270 Ability to study independently Includes: Is not frightening with the responsibility to frame a course of study for himself, can actually follow through on solo projects, doesn't need constant prodding and propping, outcomes of such 5010 study are Significant, something important is bitten off and chewed, courage to face an unknown unattended by others, as well as courage to discover and hold independent thoughts (those not supported by his peer culture or the faculty culture). Ability,to study in groups Includes: willing and able to seek the assistance of others when that is called for--willing to submit to the needs of a group, able to listen, when necessary, and to Speak when necessary, able to follow when required, and to lead when required, doesn't sit back and let John do it when JOhn shouldn't and can't, but able to let John do it when John can and will, learns from others, is COOpera- tive, democratic, relates to other kinds Of personalities and styles, and is able to live with group decisions and support them outside a group when that is necessary even though the results may not be his "ideal" solution. Able to be creative Includes: relates to the ability to synthesize and integrate, but now extends to the ability tO make new use of Old information, ideas, and action, willingness and ability to tolerate ambiguity, to settle for questions when answers aren't possible, to understand that it takes time and patience and study in order to prepare for insight and understanding--"necessary conditions" for understanding, but not "sufficient" ones in themselves--ability to seek for fresh solutions, an Openness to experience, to freshly see, feel, touch, hear, smell, taste the great world, an openness to other persons, a willingness to be "born again” (i.e., to change mind, or self, or identity, or Opinion) in the face of new insight, not the skill necessarily to create something novel (a painting, a piece of music, a poem, or any artifact), but an attitude that permits fresh insight and understanding to happen when it is ready to happen, Openness, courage in the face of ignorance, ability to have fun, to play with ideas (i.e., to suspend the law Of gravity). an ability to employ an "as if" attitude or "what if" attitude in the face of assumed truth, authority, dogma, etc. DGR:pw 12/7/70 APPENDIX 8 GUIDELINES FOR THE INSTRUCTOR FOR USING THE WRITTEN EVALUATION FORM 271 Item 1 Item 2 Item 3 Item 4 Item 5 Item 6 Item 7 Item 8 272 APPENDIX 8 GUIDELINES FOR THE INSTRUCTOR FOR USING THE WRITTEN EVALUATION FORM Preliminary information. Can be completed by the student when he responds to items 4 and 6 at the end of the term. Course number, JMC discipline, section number and credits are indi- cated on your yellow class list. It is important that these be correct. For instructor, have the student fill in your full name. Put the title of the course as stated in the JMC course descrip- tions and also a brief description. The purpose is to provide anyone reading this evaluation X years from now with some idea as to what the course was about. Here state the criteria and appropriate weights of such which you will use to arrive at a PASS or NO GRADE. Examples might be final exam, term paper, class participation, written assign- ments, etc. TO be completed by the instructor and on a voluntar basis by the student at the end of the term. The student should be aware that this information will be in front of you when you do the final evaluation. Hopefully, if completed, the student self-evaluation will assist the instructor in his evaluation. Space is alloted for the instructor to insert additional Objec- tives beyond the JMC Objectives listed. The instructor should use this Space for written evaluations of the student's performance in course goals. Early in the term the instructor should provide the student with a set of goals or otherwise inform him of the categories of evaluation. Student self-evaluation to be voluntarily completed at the end of term. This is the student's evaluation of his own performance in the course, not of the instructor or the course. Students may wish to do the latter on the forms available in the JMC Advising Center, 11 Snyder. Your response and comments here will assist the college in making recommendations for Honors College, etc. The convenient screen- ing device, the grade-point average is no longer available. Circle Pass or NO Grade. Students who receive a Pass will also receive the appropriate academic credit for the course. Those for whom you designate No Grade will ppt_receive credit and an N 1/13/71 273 will be reflected on their permanent MSU record. An N is not a failing grade and it in no way effects the student's grade- point average in courses taken outside the college. Academic progress in JMC is based on the MSU Minimum Academic Progress Scale and also credits earned per credits attempted. If you give a Pass on this form, the same should be indicated on the final grade card; likewise with the No Grade. The brief course description, basis for evaluation, and the course number, discipline, section, instructor, and credits may be mimeographed on all the forms for a class, or dictated for the students to copy. In addition if you wish to use goals with a scale for response in number 5 these may be mimeographed on the forms. Return the written evaluations with your grade cards at the end of the term. APPENDIX 9 PROPOSED POLICY AND FORM FOR WRITING STUDENT PROFILES 274 275 APPENDIX 9 PROPOSED POLICY AND FORM FOR WRITING STUDENT PROFILES May 12, 1971 TO: Barbara Ward, Chairman, and members of the Justin Morrill College Curriculum Committee FROM: Ad hoc committee on student profiles; Members: Josephs, Hachadorian, Niles, Scholberg, Struck Information: The following statement appears on the transcript of any student graduating from Justin Morrill College Fall 1970 or after: "P" (Pass) or "N" (NO Grade) grades given in Justin Morrill College Fall 1970 and after are supplemented by a student- instructor written evaluation. A Profile of Competencies will be written by the college at the time of graduation, transfer or as circumstances necessitate. This Profile will be made available to persons authorized by the student upon written request to the: Assistant Dean, Justin Morrill College, Michigan State University, East Lansing, Michigan 48823. Recommended Changes in Written Evaluation System Proposal Passed Spring 1970: Our committee recommends that the written evaluation system proposal passed in the Spring of 1970 by the University Curriculum Committee be amended as follows: 1. Under "Description of Written Evaluation System" the second paragraph would be changed to read: A Profile summarizing work in Justin Morrill College will be written at the request Of the student who is leaving (or has left) the college for purposes of transfer or graduation, according to guidelines developed by the college. 2. Under "Rationale for adopting this new system" item #6 would be changed to read: "A Profile summarizing work in Justin Morrill College Will be written at the request Of the student who is leaving (or has left) the college for purposes of transfer or graduation. This profile will be written utilizing information contained within course written evaluations in the student's academic record. This profile will be supplied to any agency with the 276 specific authorization of the student. On students or parents request, a profile will also be supplied to parents of students in accordance with university policy. A profile also will be provided for the student at his request. Recommendations on the Writing Of Profiles: Our committee recommends that the college write profiles on a re- quest basis only. Requests for profiles will be honored only when the specific authorization of the student is evident. Profiles will not be written if the student has fewer than six (6) written evaluations-ih'his academic record. The committee felt that a profile written from fewer evaluations would not be accurate. However, the student who has fewer than 6 written evaluations and desires a profile could request that the college send copies of all_his written evaluations to the agency he requests. We also recommend that the college hire an outside person to write profiles, in order to attain the maximum degree of Objectivity and con- sistency. The committee feels this person Should be hired as soon as possible and be compensated on a per profile or per hour basis. The demand for profiles has been negligible during the academic year 1970-71, however it is the feeling Of the committee that this demand will increase next year and thereafter and level off during 1973-74. The committee feels that the profile writer Should be an individual who could assume the responsibility for a considerable length of time. The idea of a graduate assistant was not satisfactory because of probable turnover. The idea of a non-JMC faculty spouse did appeal to the committee for a variety of reasons: Added Objectivity probable stability of appointment, could do profiles at Short notice, could do profiles at home and this would probably be more amenable to the irregular compensation which would be based on the profile demand. A graduate assistant might be uneasy accepting a position which did not guarantee him a monthly salary. Thus the college would have to find work for him when there were no profiles to write. The Assistant Dean and the profile writer would Sign the profiles. Students wishing to contest their profiles would see the Assistant Dean. Profile Writing: The profile writer should attempt to reflect patterns which appear to be developing in the written evaluations. The writer should include evaluations for courses where the student received an "N". The writer Should give regard to the recency of the evaluations. The writer should also read the student self-evaluation and put it in per- spective relative to the instructor's written evaluation. We conclude by recommending that the organizational format of the profile Should be such that the writer uses the General Objective cate- gories from the written evaluation form. After the introductory 277 information and statement on the profile, the first paragraph would be a summary of the student's written evaluations on his Skill in Writing; the second paragraph would be a summary of his Skill in Speaking evalu- ations. These would be followed by: Ability to Acquire Information Ability to Evaluate Information Ability to Synthesize Infonmation Ability to Study Independently Ability to Work in Groups (participation and leadership) Creative Ability Additional Abilities as defined by certain instructors Summary of Pertinent additional comments. If there is no information on a certain category, the category would be completely omitted. Therefore this would prohibit the profile writer from using a mimeographed form upon which to write profiles. The commit- tee felt that if a form was used and then the writer did not respond to each category it would have a negative connotation. eg. Creative Ability Blank The introductory information and statement would be as follows: Student Name M.S.U. Student Number Home Address Attended Justin Morrill College, Michigan State University, from term , year through term , ear . Earned term credits With a grade-point-average of or credits carried on the numerical scale. Earned credits of "Pass" in Justin Morrill College courses under the written evaluation system. written evaluations (containing the instructor's written evaluation and the student's self-evaluation) appear in the stu- dent's college records. A Profile sunmary of these written evalu- ations appears below: PROFILE APPENDIX 10 SUGGESTIONS FOR IMPROVING THE WRITTEN EVALUATION SYSTEM AND EXAMPLES OF FACULTY EVALUATIONS 278 279 APPENDIX 10 SUGGESTIONS FOR IMPROVING THE WRITTEN EVALUATION SYSTEM AND EXAMPLES OF FACULTY EVALUATIONS In reviewing several of the completed written evaluations for stu- dents being considered for Honors College, I have noticed many weaknesses in our completion of the forms. The three major problems I see with them are: (l) absence of course and program Objectives; (2) lack Of consistency among grid, written comments, and honors recommendations (it is hard for me, and the students too, to understand why an instructor rates a student very highly in the written section and on the grid but does not recommend the student for Honors College or other honors, or consistently checks "objective met" but does not indicate any areas where the student could improve or does well.); (3) lack Of Specificity in written comments. So I would like to offer the following suggestions for completing the written evaluation forms. 1. Type or print clearly with a black pen any information you put on the form. Much of the handwriting is, literally, impossible to read. Remember that the copy the students, the profiler, Phi Beta Kappa, and Honors College profilers see is a duplicate of the original and most of the duplicates are even more diffi- cult to read than the original. 2. Answer all questions and fill in all sections Of the form. Check all objectives in the grid even if it means a column of "not applicables." Be sure to answer the question regarding Honors College and other honors--whether you respond "yes," "no," "possibly," "not at this time," "insufficient information," etc. 3. Strive for some consistency in your appraisal of the student in the three areas of written comments, grids, and honors recom- mendations. 280 4. Include course and program Objectives and evaluate the student in terms of the objectives. 5. Reread some evaluations you have written four or five weeks after you have completed them. At the end of the term when we are cranking out evaluations, it is difficult to evaluate their quality. However, after you get some distance from the course and the individual students, you may be able to get a better view Of how well you are evaluating. 6. Program directors Should spend more time with part-time faculty teaching courses in their areas to acquaint them with the written evaluation system (or have them talk with Chuck or Neil). I have come across forms completed by faculty from other departments with simply the word "pass" circled at the bottom. Directors should make it clear to part-time faculty that using the written evaluation in their JMC courses is part of their teaching responsibility in the College. One of the more disappointing sections of the evaluation forms has been item no. 5, the "instructor's written evaluation of student's per- formance in course Objectives." Since this section is so important to the students and since the use of it has been so disappointing, I would like to devote a separate set of suggestions to it. The two greatest problems with the written comments are the lack of specificity and the absence of any correlation with course and program Objectives. For example, comments like "this student met all my course Objectives" are not very helpful, especially when no course Objectives are stated.. Also, general phrases like "good exams," "adequate project," and "weak papers“ do not tell the student much about his strengths and weaknesses and the ways in which he has improved. 1. Be as concrete and specific as possible. In what ways are his exams and papers good or bad? How can he improve in that area? What progress has he already made? 2. Deal with several different aspects of his performance, par- ticularly those you included in item no. 3, "Bases for Evalua- tion": class participation (both quantity and quality if important), papers, exams, class projects, interest in and enthusiasm for the material, ability to grasp concepts, areas where he did well, areas where he could improve, his potential for doing more advanced work in the discipline or field, attendance, student's self-evaluation (if you consider it appropriate), etc. 3. Evaluate the student in terms of the course and program objec- tives. 4. Evaluate the student in terms of the general JMC objectives which you stress in your course. 281 I have included two groups of sample written comments. Group one contains examples of comments that were not very useful to me as a "profiler" nor appear to be very helpful for the student. Group two contains examples Of written comments that provide Specific descrip- tions of the student's work in several areas, including the course objectives. EXAMPLES OF POORLY WRITTEN EVALUATIONS "Unfortunately, his paper and exam did not manifest a satisfactory accom- plishment." "A very good performance with respect to participation, involvement and interest. Also wrote a very good final paper." "I believe that gained in her knowledge and appreciation of the history of in a significant way." " did a good job in the course, both in her written papers and with very worthwhile contributions to class discussions." "This student's paper was very good but not quite on target in terms of this course--her final exam met the Objectives adequately." (NO objec- tives for the particular course were stated.) "Good achievement in all areas. A competent and diligent student who seems to enjoy her work." (NO areas listed and this faculty member answered no to the question regarding recommending this student for Honors College and other honors organizations.) "______performed well. She did competent and learned work. She appears to have a good grasp of the ." " more than met all my objectives for the course. Her work overall was extremely fine, I think She's an excellent student. A final synthe- sizing paper, in addition, was very nicely handled." "______Showed considerable improvement from the first to the second exam and wrote a very creditable term paper." EXAMPLES OF USEFUL STATEMENTS IN THE WRITTEN EVALUATION "______writes well, has a good vocabulary, has difficulty expressing feel- ings well, needs to organize more carefully." " '5 work in this course was of very high quality. There is a clear and dramatic pattern Of improvement in her papers. Her final (term) 282 paper, [title] is an excellent piece of work. A blend of history, law, and philosophical analysis, the paper considers the recent case of People v. Jondreau in the Michigan Supreme Court. The factual sections Of the paper are thoroughly researched and informative. The philoSOphi- cal sections-~analyses of various sorts Of rights: natural, legal, moral, and special--are first-rate. 's philosophical work here is all the more remarkable when one considers that the concept of a right was not one of the primary topics of the course. She did an enormous amount of clear-headed and a sophisticated work on her own on this. The overriding Objective for the student in this course was to develop his capacity to think deeply and well about the philosophical under- pinnings Of a number of current moral and political issues. has more than adequately fulfilled this Objective." " did an excellent job in this course and has laid a sound founda- tion for any additional work she might want to do in this field. In class discussions She was less active than many, though she was showing improvement in this regard. The comments she did make were generally of significant value--should have more confidence in her own ability, trust herself more. Her written work showed a good capacity to empa- thize and a sensitivity to subtle (and frequently informal) aspects of social interaction. She is also beginning to Show some theoretical ability and sensitivity to the effects Of perspectives. Especially good in analyzing/understanding influences of culture on the individual. Would benefit from more work on stratification and on analysis at total society level. " made very positive contributions to the class. She was atten- tive and energetic. Her creative spurts were enjoyed by the entire group. She seemed to pull together the material which was assigned reading and came to some conclusions about their relevance for her. She wasuat ease with the group and facilitated the participation by others. " writes clearly and competently; she tried several different kinds of papers this term. In all Of them she used a lot Of detail, but in a couple of them, her thesis seemed insufficiently supported by enough details." (Only weakness of this evaluation was the lack of Objectives and bases of evaluation.) APPENDIX 11 SUMMARY OF FALL 1972 MEETING ON WRITTEN EVALUATION AND SAMPLE EVALUATION FORMS 283 FT- 284 APPENDIX 11 SUMMARY OF FALL 1972 MEETING ON WRITTEN EVALUATION AND SAMPLE EVALUATION FORMS TO: Members of the Advisory Council FROM: Neil RE: Faculty seminar meeting 10/27/72 to discuss JMC's written evaluation ' system. A summary and some personal observations. r- We discussed quite a few topics related to written evaluation, so this summary may ramble a bit. I will try to organize it by topic. A. I'm-1 Why are we discussing P-N,,Written Evaluation? The evaluation paragraphs written at the end of Spring Term Showed a marked improvement over previous terms. However, in general, the forms still do not have adequate course descriptions, course objectives or bases for evaluation. Some faculty and many students do not complete the grids provided for assessing the degree to which a student attains college objectives. In February of 1972 the University Curriculum Committee approved an extension of our experiment until June 1975. We still may discontinue it for cause at any time. However, if we wish to continue it, we need to improve our use of the written evaluation forms. The Form We discussed two modified forms which Sandy prepared prior to the seminar. They included a course description (aim and content), texts used, and a list of college Objectives stressed with an accompanying basis for evaluating whether the student met each Objective stressed (see attached "Form Reaffirming 3 Objectives"). The ensuing discussion made clear to me that we have 3 answers to the question, why do we have an evaluation form? 1) to aid communication between teacher and learner as they evalu- ate the learner's performance in a particular course; 2) to highlight specific course objectives and those college objectives stressed in a particular course. From each com- pleted evaluation form a student learns if he has met or failed to meet the stated objectives; 285 3) to provide a permanent record of a student' 5 performance in each JMC course he takes SO that if the student wishes, some- one can summarize his demonstrated skills for review by: a. Honors College b. Prospective employers, graduate schools, etc. The college objectives offer an organizing scheme for the pro- file. Alternatively, at the student's request, the college can supply all his evaluation forms to prospective employers, etc. Collecting all the student's evaluation forms as a summary of his JMC work and/or writing a profile emphasize the notion that a student's education in JMC is more than a collection Of unrelated courses. During the discussion, I interpreted some persons comments to mean that they stressed 1 above while I interpreted others as stressing 2 above. Few of us are involved in 3 above, but it is a stated Objective of the written evaluation system. The fact that we have three answers to a single question necessarily makes JMC's written evaluation system complex, and, I believe, partially explains why faculty and students use the form in such varied ways. I think we must make a decision now. Either we reaffirm our commit- ment as a college to the complex system we devised (in which case each faculty member commits himself to filling out the evaluation form with all three Of the above Objectives in mind), or we admit that we are unwilling [unable (?)] to attend to all three _Objectives and state number 1 above as the only major Objective of the system. I would like people to read the attached forms with the above dichotomy in mind. One is designed SO that we can reaffirm all three Objectives. It has several strengths: 1) It includes the course description which a faculty member writes for distribution the term prior to teaching the course. He would not need to write anything else for the evaluation form providing the description is fairly concise. 2) It asks the faculty member to state only those college goals which he chooses to stress in his course rather than asking him to check those "not applicable" from the entire list. The natural tendency in the latter case is to consider all the college Objectives applicable whether one stresses them or not. If we really do want to analyze our curriculum to determine whether or not it provides adequate opportunities for a student to attain the college objectives, teachers Should determine which objectives they stress in particular courses. 286 3) It couples with each stated objective the manner in which the teacher anticipates basing his evaluation of the students performance. 4) In the suggested format, each teacher would number his stated objectives. By using corresponding numbers, he could evaluate the student's performance on each objective. He might also write a paragraph. Since the objectives are on the form, a statement such as 'has met all course objectives' might suffice. One other alternative occurs to me. Should a teacher desire to use the grid format for his evaluation, he could place the grid plus stated objectives in the space provided for the teachers evaluation. 5) The entire form with the exception of the evaluation comments themselves, can be completed before classes begin. He can duplicate enough for each class and have them ready to distribute for discussion purposes on the first day of class. 6) It should improve the use of our system by part-time faculty since they would have forms explicitly designed for their courses. I think the primary weakness of the form would be its complexity. Although it or some other variation would streamline our present system, it would still take some effort and coordination to use it effectively. The second form assumes that we modify our system to stress the first objective stated above plus the minimum information for Honors College. In effect we would make no claim concerning the articulation of course and college objectives, and permit individual faculty to deter- mine the manner in which they use the form. He would file completed forms as we do now so that they could be forwarded on the student's re- quest. He would not attempt to write profiles. The second form also has several strengths: l) It reflects manageable goals and requires minimal coordination. 2) Faculty would be free to write course descriptions and/or objec- tives in language they find most appropriate without worrying whether it is similar to other course descriptions. 3) Faculty could write brief or long evaluations. 4) It eliminates the problem of finding someone to write profiles. The second form's weakness is apparent. It does not ask faculty to consider how the course they are planning contributes to JMC curricular objectives nor does it ask a faculty member to determine if a student is 287 meeting college objectives. But perhaps the form is both realistic and desirable because it does not ask the above of teachers. Confidentiality We label our written evaluation form 'confidential' yet, Academic Advisors, Academic Assistants and all JMC faculty have access to a stu- dent's file. In addition, in order to eliminate the process of writing profiles for students being considered by Honors College, Sandy would like us to consider sending the forms themselves to Honors College. Honors College would like this procedure and it would save abundant man- hours internally. If we move in this direction, each student must grant permission for his forms to be forwarded. Question--do we want to keep the completed written evaluation forms any more confidential than any other item in the student's advising folder? The group at Friday's meeting did not try to resolve this issue. Profiling At present, Sandy Harden writes profiles for those students being considered for Honors College. All others are written on demand-~Chuck Niles eitherwrites them himself or asks the student's advisor to write them. The latter procedure does not coincide with the recommendation of a long forgotten sub-committee--that the "Profiles of Competencies" be done by someone unfamiliar with the student so that they are objective summaries of the information on the evaluation forms only. The group that met Friday morning (about 15 faculty) felt that Gordon should devise a temporary method of writing profiles adhering to the recommendations of the sub-committee (members: Josephs, Minter, Niles, Scholberg, Struck--l97l). The Steering Committee is presently considering the issue and would welcome any advice others might have. If you complete this preposterously long memo and examine the attached forms, please give any suggestions you might have to me. Let me emphasize that the forms are a first attempt at modification--feel free to criticize, praise, tear-up, stomp on, etc.--only please give me your comments. I really believe that written evaluation is one of the most significant innovations we've tried and think we should improve the way we use it. Some questions for you to ponder: a. How do we get a thorough discussion of this issue among students? b. Do either of the forms seem an improvement over our present system? c. What changes would you recommend? d. Where do we go from here? Action by the Advisory Council? Discussion and hearings? A Seminar meeting? Ask the Steering Committee to prepare a document for action by the A.C.? ll-2-72 dr 288 SAMPLE SAMPLE FORM REAFFIRMING ALL 3 OBJECTIVES - FOR DISCUSSION ONLY JUSTIN MORRILL COLLEGE - CONFIDENTIAL - COURSE WRITTEN EVALUATION l) Student (print) Student No. Class: (circle one) Fr. Soph. Jr. Sr. JMC Student? Yes No Course No. JMC 389A JMC Discipline IDC Section #_1. Credits_fi_ Instructors Josephs & Harden Term Spring 1973 2) Title and Brief Course Description: THE EXPERIENCE OF HEMINGHAY AND FAULKNER This course examines the two American writers who dominated the period between the two world wars. We will read and criticize novels and short stories of each as a basis for understanding human relationships. The major style of the course will be discussion, in groups of varying sizes, with student discussion leaders and with each student serving a turn as a process observer in relation to the dynamics of group interaction. Dr. Harden will emphasize interpersonal skills, Dr. Josephs will emphasize literary and creative skills. Each student will be asked to create an original skit illustrating the central interpersonal insight of the works we study. There will be no written examinations. Texts: Hemingway: The Sun Also Rises, The Old Man and the Sea and short stories Faulkner: The Sound and the Fury, The Bear and short stories Objectives: Bases of evaluation: 1. Demonstrated ability to evaluate --Quality of discussion participation information 2. Demonstrated ability to work in --Process observation of group groups dynamics 3. Demonstrated creative ability --Quality of skit 4. Demonstrated understanding of the --Development of theme in the skit ideas central to the course 3) Instructors's Evaluation of Student's Performance 4) Student's voluntary self-evaluation (not an instructor or course evalu- ation). 5) 6) 289 Would you recommend this student for Honors College or other honors organizations? Action taken: (circle one) Pass No Grade Instructor's signature Date 290 SAMPLE SAMPLE 1) 2) 3) 4) 5) 6) FORM FOR OBJECTIVE #1 - FOR DISCUSSION ONLY USE PEN OR TYPENRITER JUSTIN MORRILL COLLEGE - CONFIDENTIAL - COURSE WRITTEN EVALUATION Student (print) Student No. Class: (circle one) Fr. Soph. Jr. Sr. JMC Student? Yes No Course No. JMC 389A JMC Discipline IDC Section #_l__ Credits_4__ a- Instructor Josephs & Warden Term Spring 1973 Title and Brief Course Description: - The Experience of Hemingway & Faulkner This course examines the two American writers who dominated the period between the two world wars. We will read and criticize novels and short stories of each as a basis for understanding human relationships. The major style of the course will be discussion in groups of varying sizes, with student discussion leaders and with each student serving a turn as a process observer in relation to the dynamics of group interaction. Dr. Harden will emphasize interpersonal skills, Dr. Josephs will emphasize literary and creative skills. Each student will be asked to create an original skit illustrating the central interpersonal insight of the works we study. There will be no written examinations. Instructor's Evaluation Student's Voluntary Self-Evaluation Would you recommend this student for Honors College or other honors organizations? (Action taken: (circle one) Pass No Grade Instructor's signature APPENDIX 12 FALL 1972 PROPOSAL FOR MODIFYING THE JMC WRITTEN EVALUATION SYSTEM 29] Saar-18‘1”. ‘w V 292 APPENDIX 12 FALL 1972 PROPOSAL FOR MODIFYING THE JMC WRITTEN EVALUATION SYSTEM TO: Advisory Council Members FROM: Neil Re: Written evaluation The Steering Committee urges the following course of action by the JMC Advisory Council: l) At its ll/30/72 meeting, the Advisory Council should approve either the attached written evaluation form or an amended version of it as a substitute for the present student evalua- tion form. The A.C. should also reaffirm the 3 major objec- tives of the written evaluation system: a) to aid communication between teacher and learner as they evaluate the learner's performance in a particular course; b) to highlight specific course objectives and those college objectives stressed in a particular course. From each completed evaluation form a student learns if he has met or failed to meet the stated objectives: c) to provide a permanent record of a student's performance in each JMC course he takes so that if the student wishes, someone can summarize his demonstrated skills for review by: - Honors College - Prospective employers, graduate schools, etc. The college objectives offer an organizing scheme for the profile. Alternatively, at the student's request, the college can supply all his evaluation forms to prospective employers, etc. Collecting all the student's evaluation forms as a summary of his JMC work and/or writing a profile emphasize the notion that a student's education in JMC is more than a collection of unrelated courses. Our ability to achieve the above 3 objectives rests on the assumption that JMC has a set of college objectives which serve as one organizing principal for the JMC curriculum. To clarify this assumption, the S. C. urges the Advisory Council to endorse the following statement at its ll/30/72 meeting. 293 Unlike most colleges, JMC states two sets of curricular objectives, one concerned primarily with content and one highlighting learning skills. One can find JMC's content objectives in the majority of the college requirements--45 credits in the humanities, natural sciences and social sciences, a two-year competency in a foreign language, 40-45 credits in a field of concentration. In addition to endorsing the concept that students should gain exposure to cer- tain content areas, JMC faculty endorse the notion that students should hone certain learning skills during their undergraduate years. These skills are communicating effectively; acquiring, evaluating and synthesizing information; working independently; working in groups; and demonstrating creativity, intercultural awareness, an interest in self-understanding, and an appreciation for the rational and emotional realms in problem solving. Each course in JMC addresses itself to some of the stated process skills as well as dealing with a particular content area and each written- evaluation form used in the college includes a course description pinpointing content and process objectives as well as the bases for evaluation. The three A.C. actions recommended above will enable us to institute a modified form for use in Spring Term. Course descriptions written by mid-January would be placed on the form and the forms would be completed for use by the beginning of Spring Term classes. We need A.C. aCtion on written evaluation by ll/30/72 in order to modify the form this academic,year. 2) To enable the A.C. to act on written evaluation on ll/30/72, the S.C. recommends that: a) faculty take 5-lO minutes in at least one of their classes between now and ll/28/72 to obtain student response to the suggested modified form. A faculty member might tell the students how he would describe a Fall Term course in the suggested format and then ask the students: - Is such a form an improvement over the present one? How? - Would you urge the College to institute the change for Spring Term? (the faculty member would tell the students that the forms with the course descriptions, objectives and bases for evaluation would be available for discus- sion the first day of each Spring Term class.) - Do you see any drawbacks to the modified form? - Do you have any modifications to suggest? b) the Advisory Council meet at 4:00 p.m. Tuesday, ll/28/72, in the Trophy room, to discuss the suggested action by the {-7- 294 A.C. as well as student opinion obtained from in~class discussions. Final proposals can be prepared Wednesday for A.C. action on Thursday. Please discuss any of the various suggestions in this note with any of the S.C. members, Neil or Gordon. They would be happy to review any of the issues which were raised during their discussion last Wednesday. Faculty should pick up copies of the sample evaluation form for their class discussions from Mrs. Rhines. It is clear now that there is not enough agreement internally to use our present written evaluation system well. Personally, I feel that if the A.C. reaffirms the system with some enthusiasm and approves a modi- fied version of the form, JMC will have a system with a good chance of success. What if..... If the A.C. reaffirms the written evaluation system and approves a new form, we could implement it as follows: l) Between ll/30/72 and l/3/73 each faculty member could write a course description in the suggested format and give it to me. I could then select a sample of these to distribute on l/4/73 for all members of the A.C. to read. On l/5/73 the faculty seminar could meet to discuss the writing of course descriptions using the samples as a focus for discussion. 2) Faculty would get their Spring Term course descriptions to Chuck according to the schedule he distributed and give a copy to Mrs. Rhines. Before classes begin Spring Term, Mrs. Rhines will prepare enough evaluation forms for each class that a professor has. 3) Each faculty member can distribute the evaluation forms to his students on the first day of each of his classes. The class can then discuss the course as well as the aims of the written evaluation system. Unless each faculty member assumes this obligation the written-evaluation system will fall into dis- repair as it has in the past. 4) If the faculty wish, I will try to identify resources which will aid them as they write course descriptions in the sug- gested format. ll/20/72 dr APPENDIX 13 PROCEDURAL GUIDELINES FOR USING THE MODIFIED WRITTEN EVALUATION FORMS 295 296 APPENDIX 13 PROCEDURAL GUIDELINES FOR USING THE MODIFIED WRITTEN EVALUATION FORMS January 9, 1973 TO: All JMC faculty FROM: Neil . ' RE: Friday's seminar meeting on course descriptions f“ for the new forms. ‘ Those who were able to meet on Friday (Keven Bridge, Neil Cullen, * Eva Faulkner, Jim Goatley, Fred Graham, Tamara Harrod, Harold Johnson, Rosa Marti, Linda Minter, Milt Powell, Gordon Rohman, John Schroeder, Herm Struck, Tom Tamandl, Barbara Ward, Keith Williams, Glenn Wright, Don Weinshank, Al Welch) to discuss the sample course descriptions came to the following agreements on using the new forms: l) Several variations seem possible for writing the 2 course descriptions needed. The following are ranked in order of preference: a) Use the same course description, plus objectives, plus bases for evaluation for the evaluation form and for the "Course Descriptions" brochure compiled by Chuck Niles. b) Write an abbreviated version of the course description used in the "Course Descriptions" brochure for the evaluation form. Also include objectives plus bases for evaluation. c) Where the paragraph describing the course is lengthy, delete the itemized objectives and bases for evaluation in the “Course Descriptions" brochure. Follow b) above for the evaluation form. 2) The list of objectives for each course should include those college goals emphasized in a particular course plus major course objectives not covered by the college object1ves. 3) Many teachers assess whether or not a student has met the course objectives with several bases for evaluation. Therefore it is not necessary to pair a separate basis for evaluation with each 4) 5) 6) 7) 8) 297 objective. Where such pairing is appropriate and possible, however, it is desirable. If a teacher plans to use several bases for evaluation to determine the students' progress toward all course objectives, he should make this procedure clear on the evaluation form. When the ideas central to the course are included in the course description, it is appropriate to state as one of the course objectives: "demonstrated understanding of the ideas central to the course." The teacher can then refer the reader to the course description. The course descriptions, plus objectives, plus bases for evalu- ation should be clear yet brief. Ideally all three can be put on the front of the evaluation form and still leave room for the rest of the information (see item 1 above when such brevity in the course description proves impossible). In most cases faculty should limit their list of objectives to 3-5 major ones. A small list focuses the attention of the student and permits more specificity to emerge when the objectives are discussed in class. At the same time the list is kept to a reasonable length. Having the course objectives written on the evaluation form to discuss with the students in a class should make all concerned more conscious of a change in the direction of the course. When such a change occurs, the teacher and class can either decide they want to adhere more closely to the original objec- tives or modify the objectives according to need. In the latter case the new or modified objectives could be put on the evalua- tion form in lieu of the original ones. There seems no reason for people who prefer the grid design for listing objectives and evaluating performance to abandon that format. They might consider putting the grid on the front of the evaluation form below the course description and bases for evaluation. Teachers should get their course descriptions, plus objectives, plus bases for evaluation to Mrs. Rhines in 151 Snyder no later than the fourth week of classes during the term preceding the term the course is to be offered. Please get Spring Term l973 course descriptions to her by January 26. If faculty follow 1 a) above, they should give a copy of the description for the "Course Descriptions" brochure to Mrs. Rhines. Mrs. Rhines will complete each teacher's evaluation forms in time for the begin- ning of classes the subsequent term. 298 9) Teachers should use the evaluation forms at the beginning of each of their classes to review the purposes of the course and the written evaluation system. In t is manner, the form be- comes a vehicle to make course objectives clearer and to main- tain familiarity with the purposes of written evaluation. lO) I will attempt to discuss the written evaluation system with all faculty who are teaching in Justin Morrill for the first time. If you know of a colleague in your knowledge areas who is going to teach in JMC and is unfamiliar with our recent modifications in written evaluation, please discuss the changes with him or refer him to me. ll) During the meeting on Friday we also discussed the difference between stating course objectives and evaluating objectively. Stating objectives makes clear the teacher's expectations for student performance and the bases upon which he will judge student performance. The process does not necessarily lead to an objective evaluation. Rather it leads to an evaluation based on the objectives identified at the beginning of the course. If anyone has questions concerning use of the new form, please raise them with me. If necessary we can have another session to discuss the writing of course descriptions and objectives. If there are no objections to the suggestions made at last Friday's seminar, I hope all faculty will abide by them so that we can have some consistency in our use of the new form. APPENDIX 14 SUGGESTIONS FOR USING THE MODIFIED WRITTEN EVALUATION FORM 299 300 APPENDIX 14 SUGGESTIONS FOR USING THE MODIFIED WRITTEN EVALUATION FORM TO: All faculty teaching in JMC Spring Term l973 FROM: Neil Cullen, Assistant to the Dean RE: Written evaluation forms for Spring Term Attached are your written evaluation forms for each of your Spring Term courses. If there are any errors or if you have any questions, please contact me (3-4344, 149 Snyder). Remember: there will be no additional forms provided at the end of Spring Term. If you need more forms, request them from Mrs. Rhines in lSl Snyder. The Advisory Council approved the new forms last November as a means to improve JMC's written evaluation system. To facilitate such improve- ment, I urge that you use the forms in either the following or a similar manner: a) Distribute the forms to your students on the first day of each class so that they can complete the first section. b) Discuss your course objectives and bases for evaluation with the students. c) Review the objectives of written evaluation with your students and the manner in which you intend to use the system (I have attached a copy of the system's 3 major objectives). d) Collect the forms so that you can redistribute them when the students who wish to complete their self evaluation at the end of the term. e) Return the completed forms to Chuck Niles' office along with your grade cards as you have in the past. I realize that the procedure outlined above is simply common sense. However, I outlined it to emphasize that unless each faculty member follows a similar pattern in using the written evaluation forms, the value of the new forms will be severely diminished. Please remember that this is the first time students will be using this—particular form and that in every class you may have some students who have never studied under a written evaluation system before. You are the person whom stu- dents expect to explain the changes in the form and the systems objec- t ves. 301 If any faculty, especially those of you who are teaching in JMC on a part time basis, have any questions or suggestions, please contact me. There are bound to be a few rough spots as we try to improve writ— ten evaluation during the next few terms, and I would appreciate your comments and suggestions. 3/l9/73 dr APPENDIX 15 GUIDELINES FOR WRITING COURSE DESCRIPTIONS, OBJECTIVES AND BASES FOR EVALUATION--AN EXCERPT FROM A 4/25/73 MEMORANDUM BY NEIL H. CULLEN 302 303 APPENDIX 15 GUIDELINES FOR WRITING COURSE DESCRIPTIONS, OBJECTIVES AND BASES FOR EVALUATION—~AN EXCERPT FROM A 4/25/73 MEMORANDUM BY NEIL H. CULLEN Course Descriptions,,Objectives, etc.,gfor Fall Term 1973 Since you are now in the midst of writing course descriptions, etc., for Fall 1973, I thought it might be helpful to offer some comments on those written for use this Spring. While reading all the written evalu- ation forms which faculty developed for Spring Term, I tried to identify the characteristics of each part-~description, objectives, bases for evaluation-~which I would find helpful as a student. The following lists are obviously not definitive; I make them to encourage teachers to have some characteristics in mind as they write course descriptions, objectives, and bases for evaluation. a) The course description includes: l) The principal content of the course. 2) What the students and faculty will be doing, i.e., the activities of the course. Is it a seminar, a simulation or what? 3) The authors, readings and/or various media that the course will use. Perhaps how the teacher intends to use the materials. b) The objectives l) Should be limited in number. No more than 4-6 so that they clarify the thrust of the course rather than make the aim seem more complex. 2) The teacher should list both college objectives stressed and course objectives. The college objectives should have the terminology from the list approved by the College Advisory Council. As can be seen from the attached samples, one can use the terminology in a variety of ways. The course objectives may be specific aspects of college goals (e.g., "a discriminating listener of music") or may refer to instructor objectives unrelated to the college objectives (e.g., to develop an appreciation for Boroque Music . 3) Normally, both the college and course objectives that the instructor lists are broadly stated. They can be more fully explicated and analyzed as they are discussed in class. As can be seen in the form for Russian, however, some may be quite specific. 304 c) Bases for evaluation: l) Should where possible be linked with a specific course objective. 2; May be the same for all objectives. 3 Should be as specific as possible.