THE DEVELOPMENT OF A SET OF GUIDELINES AND CHECKLIST FOR USE IN THE DESIGN AND IMPLEMENTATION OF INSERVICE STAFF DEVELOPMENT PROGRAMS Dissertation for the Degree of Ph. D. MICHIGAN STATE UNIVERSITY RICHARD PAUL STIMPSON 1977 II INMMMMNI ~¥ LIBRARY Michigan Stats University A’#._ $1; This is to certify that the thesis entitled THE DEVELOPMENT OF A SET OF GUIDELINES AND CHECKLIST FOR USE IN THE DESIGN AND IMPLEMENTATION OF INSERVICE STAFF DEVELOPMENT PROGRAMS presented by Richard Paul Stimpson \ has been accepted towards fulfillment of the requirements for __Eh4_D_i__degree in Department of Admin- istration and Higher Education Major professor 0-7639 I t}; 42" Zr .rlflri't)”; $9933 “in U185 ABSTRACT THE DEVELOPMENT OF A SET OF GUIDELINES AND CHECKLIST FOR USE IN THE DESIGN AND IMPLEMENTATION OF INSERVICE STAFF DEVELOPMENT PROGRAMS BY Richard Paul Stimpson Student personnel professionals recognize the important role that inservice staff development should have within the profession. Such programs prepare staff members to effectively perform assigned responsibilities and promote professional growth and development. However, while various types of staff deve10pment activities are reported in student personnel literature, it is generally concluded that resources which examine the steps that should be followed to properly design and implement inservice staff development programs in accordance with educationally sound instructional procedures are limited. This lack of resources creates a problem for student personnel professionals who are interested in or who are responsible for inservice staff development. Therefore, to respond to this problem, it is the purpose of this study to deve10p such a resource by (1) investi- gating literature which examines the development of xVI‘ E! Richard Paul Stimpson instructional programs for use in formally recognized classroom settings or for use in staff deve10pment set- tings other than student personnel, (2) identifying the elements which selected authors have determined are important to the proper design and implementation of instructional programs, and (3) creating, by using the discussion of each element as a source, a set of guide- lines and checklist which student personnel professionals can follow in step-by-step fashion to insure that inservice staff development programs incorporate each of the elements which are identified and, therefore, are designed and implemented in accordance with educationally sound pro- cedures. Accordingly, it has been determined that there is significant agreement among selected authors concern- ing the elements which are important to the proper design and implementation of instructional programs. These elements are (l) assess needs, (2) develop objectives, (3) select instructional procedures, (4) evaluate, and (5) redesign. Drawing upon selected authors' views about each of these elements, a set of guidelines and a check- list to assist in designing and implementing inservice staff development programs have been formulated and are presented. While the guidelines give direction to the design process as it is taking place, it is intended Richard Paul Stimpson that, once the program is developed, the program designer will use the checklist as a tool to determine that he/she has taken the necessary steps to insure that each guide- line and, consequently, each element has been properly incorporated. The guidelines and checklist can only be utilized effectively if the designer has, through the careful study of source materials, developed a clear understanding of the contribution that each element makes to the success of the program and why the contributions made by all five of the elements must be accounted for as programs are designed and implemented. The guidelines and checklist are not presented as a substitute for such knowledge but rather as an aid to help the program designer insure that such knowledge is brought to bear in the design process. While designing and implementing inservice staff development programs in accordance with the steps included within the guidelines and checklist is a demand- ing task, doing so will positively contribute to the development of programs which, because they are based upon educationally sound instructional procedures, will enable participants to learn and, as a result, demon— strate mastery of desired knowledge and skills. *2: or " Good Ava-v. ctr- b- .— Tv‘ ‘0’! in pdr1 De THE DEVELOPMENT OF A SET OF GUIDELINES AND CHECKLIST FOR USE IN THE DESIGN AND IMPLEMENTATION OF INSERVICE STAFF DEVELOPMENT PROGRAMS BY Richard Paul Stimpson A DISSERTATION Submitted to Michigan State University in partial fulfillment of the requirements for the degree of DOCTOR OF PHILOSOPHY Department of Administration and Higher Education 1977 n. Tm Y. nn uv. nun 77 19 Copyright by RICHARD PAUL STIMPSON 1977 I: all the. 0 ‘ t Lie tamer: To all those at Michigan State University and at the University of Maryland who, by contributing to my own personal and professional development, have inspired this study. ii I wish ;;:=:.ce corn: 12ers of the 13:. John F Liar. I esp-e: Sgeial an 905 ;:ofessiona‘. 5:31;. He has I :s;iration. H= tiring and th' I: achieve the 4 higher educa‘ ~ Persistent e 3311-1 have been 3'39 been comple I would ’5- C. L. Wither Eswillingly of u detlee critic 9. ~45 '11 "WW of t) ACKNOWLEDGMENTS I wish to extend appreciation to my doctoral guidance committee for their patience and support. The members of the committee are Dr. Louis Stamatakos, chair— man; Dr. John Fuzak; Dr. James McKee; and Dr. Eldon Nonna- maker. I especially would like to acknowledge the very special and positive role which Dr. Stamatakos has had in my professional development and the completion of this study. He has been and remains a mentor of impact and inspiration. He has taught me the importance of critical thinking and the special rewards to be gained by striving to achieve the goals of the student personnel profession in higher education. Without his constructive criticism and persistent encouragement, my doctoral experience would have been diminished, and this study would not have been completed. I would also like to express my gratitude to Ms. C. L. Witherspoon. As a friend and assistant, she has willingly offered her skills and support. Her con- structive criticism and insightful editing have improved the quality of this manuscript. iii it. 2238 IS tar. :2 with the a O. ’Mv-‘ ‘ I‘ HVMIAr‘ete t Finally, I would like to recognize my mother and father's quiet pride in their son, my family's patience and understanding, and the encouragement which Dr. William L. Thomas, Jr., has offered. The intangible, yet most necessary, support from these individuals has provided me with the affective stimulation which I have needed to complete this task. iv 0-4 94 a '0 rl n *‘I (in (D "1 t? :J (T) '1 '1 (I) 'U't'Z'UMH '13 o C) r. 1:. was s Dy, «.«0 IDENT] DQVQ Ob We. Cor Te] TABLE OF CONTENTS Chapter I. INTRODUCTION . . . . . . . . . . . 1 Increased Accountability . . . . . . 2 Student Personnel Not Exempt . . . . . 6 Prepare To Be Held Accountable . . . . 8 Need for the Study . . . . . . . . 13 Purpose and Procedures . . . . . . . 19 Format of the Study . . . . . . . . 22 II. THE STATUS OF INSERVICE STAFF DEVELOPMENT WITHIN STUDENT PERSONNEL . . . . . . 25 The Importance of Inservice Staff Development Programs . . . . . 25 Suggested Inservice Staff Deve10pment Programs . . . . . . . . . . 31 Points To Consider in Design and Imple- mentation O O O O O O O O O O O 38 Conclusion . . . . . . . . . . . 42 III. IDENTIFY AND EXAMINE INSTRUCTIONAL PROGRAM ELEMENTS . . . . . .l . . . . . 44 Assess Needs . . . . . . . . . . 66 Developing Goals or Purposes from Assessed Needs . . . . . . . . 67 Determine that Instruction Is the Proper Course of Action . . . . . 81 Setting Priorities for Action . . . . 85 Deve10p Objectives . . . . . . . . 89 Objectives Defined. . . . . 89 Well-Written Objectives Are Important . 92 Components of Well-Written Objectives . 96 Terminal and Enabling Objectives Are Needed. . . . . . . . . . 113 ".:—.=' r.-. —.5 . "" H A. Red Emu; Chapter Select Instructional Procedures . . . Variables and General Principles . . Types of Learning and Conditions of Instruction . . . . . . . Positive Transfer of Learning . . . Select Methods and Materials . . . Evaluate . . . . . . . . . Types of Evaluation . . . . Improper Completion of Evaluation. . Factors for Inclusion in Evaluation Processes . . . . . . . . . Redesign . . . . . . . . . . The Purpose and Value of Redesign. . Interpret and Apply Information Gathered through Evaluation . . . Prototypes Prove Valuable . . A Variety of Redesign Actions May Be Necessary . . . . . . . . . IV. FORMULATE GUIDELINES AND CHECKLIST . . . Guidelines . . . . . . . . . . Sequence of Guidelines May Vary . . Checklist . . . . . . . . . . V. SUMMARY, FURTHER STUDY, AND IMPLICATIONS . Summary . . . . . . . Suggestions for Further Study. . . . Implications . . . . . . . . . SELECTED BIBLIOGRAPHY . . . . . . . . . vi 118 118 126 155 161 167 169 182 186 202 204 208 210 213 219 220 226 237 261 261 267 270 279 Figure 1. LIST OF FIGURES "An over-all structure of the design of an instructional system" . . . . . . "A time-line graph illustrating . . . the learning system design process" . . "An instructional system" . . . . . "A general problem-solving process" . . Presented in Igstructional Design by Jerrold E. Kemp . . . . . . . . "A general model of instruction" . . . "A goal-referenced instructional model" with "courses of action dictated by evaluation of results”. . . . . . Components or elements identified for inclusion in instructional programs . vii 49 51 56 57 59 59 61 63 Th1 traities i; fixation re 9- effOrt a: P. ':‘.::' w mucult have 108*; if: 99538! Cent Afte “‘9 were lib. Iii’dditiOnal 95051; unque: 3993Cie3 . T} EWeVer' as t “Tort, - the AEw.f0uDd endurin<3~~hig ‘h. "' genteel PO( CHAPTER I INTRODUCTION This is a demanding time for colleges and uni- versities in America. The Carnegie Commission on Higher Education reported in 1973 that " . . . sustained growth in effort and in attainment has given way to doubts and to difficulties . . ." (14:7). Colleges and universities have lost the favored position they enjoyed in the last quarter century. After World War II, institutions of higher learn- ing were liberated from what the Carnegie Commission calls "traditional genteel poverty" and, by the 19605, enjoyed almost unquestioned support from public and private agencies. The future seemed to hold limitless growth. However, as that same commission notes in its final report, " . . . a 'new depression' has quickly followed the new-found prosperity, and is likely to be more enduring-~higher education has moved from genteel poverty to genteel poverty in one generation . . ." (14:7). In t2 efle::s on 1 C? .,.1. were 5113'. the fed : the grow: atrestra: 19735 be: C: the w} student ¢ tutions < guarded < _ s 39E . doinc “991' One In the Journal of Higher Education, Witkowski reflects on the past few years: Those in higher education look nostalgically on the decade of the 19605 as the "Golden Age.” Golden certainly in the sense that vast amounts of money were showered on the higher education industry by the federal and state governments, but also because the growing enrollment of students led to an almost unrestrained optimism about the future. As the 19705 began, the gold turned somewhat, but educators on the whole still predicted modest increases in student enrollment. In 1973, however, many insti- tutions of higher learning found that even this guarded optimism was misplaced. . . . (102:48) Increased Accountability Along with questionable enrollment levels, support for higher education in both public and private circles has declined. Increasingly, Colleges and universities have been required to defend their programs and need for resources. T. R. McConnel reviews some of the reasons: Turmoil and disruption on the campuses; political action by students and faculty members; severe shrinkages in governmental, corporate, and indi- vidual income, coupled with rising taxes; and mounting distrust of higher education by the public are behind the increasing demand for colleges and universities to justify what they are doing and to disclose the effectiveness of their operations. . . . (55:446) In some instances, state governments have attempted to directly reduce institutional autonomy by strengthening the position of state boards of control or by instituting management programs such as the planned, programmed, budgeting system (PPBS) described by Peterson (72) and Thompson (91). In doing so, state ------ :ffieV and t9 ‘ u a on.- .A"EV""“"-1‘e‘; .1: j" -9 313-15: coup] :stitliions t reszurces and azzcmtable f9 :;'.‘ersities , 1:, have also :5 reduced re: 5:1:iation8. a: As coll nth enrollment eternal source at: programs K legislators and executives have sought to support only those programs which seem valuable from a public point of view and to insure that funds are spent wisely from the government's perspective. In other instances, public funding levels have been reduced or held constant. These actions, coupled with inflation, have forced public institutions to make choices about the best way to use resources and to hold departments and individuals accountable for their wise use. Private colleges and universities, while not directly subject to public fund- ing, have also felt the effect of enrollment fluctuations and reduced resources as funding from private donors, foundations, and the federal government has declined. As colleges and universities have been confronted with enrollment fluctuations, increased control from external sources, and reduced funding, those associated with programs within these institutions have seen an elimination of staff positions, have had less flexibility to develop new programs, have been forced to work with fewer supplies, and have encountered more scrutiny by campus administrators. It has become necessary to develop and defend program goals and objectives and to maintain performance standards which insure the highest quality at the lowest cost. Reactions to these developments have been varied. For example, some observers have forecasted an end to .'_,.§.n§9 ~~ “:--'db-d‘ a II V.- ‘9 o a ..S -4. ‘0": .L-atened “Hire 1 t treatened ‘~ "3 3C pi atlng their decisi here Comp-e: Vatchw u r T ‘ ‘9. ‘ (ft? »:m institutional autonomy and a loss of academic freedom when institutions seek to adjust their programs to main- tain funding and individual faculty members feel their security and flexibility challenged. Anderson warns that, as the press within academia continues to grow, "academic freedom is threatened; and, as academic freedom is threatened, the goals of the university--goals that require it to be creator and social critic--are threatened” (2:464). Witkowski maintains: Faculty members who once worked together in a spirit of community now occupy their time specu- lating on the effect of cuts on the institution, their department and themselves. . . . Hard decisions on firing and hiring produce an atmos- phere which is scholastically counter productive. Competition rather than co-operation is now the watchword. (102:48) However, others have seen these developments as a welcome change from the 19605 when money flowed and programs were added with little thought to their appropriateness or to their compatibility with insti- tutional goals and values. Winsted and Hobson point out that the development of ". . . specific objectives can provide focus for directing activities designed to achieve certain results . . . [and] to attain those goals which have received the institution's highest priority” (101:675). Those who have similar feelings believe that, rather than the continued existence of large numbers of programs, many of which are of marginal §-“ w .'--" a u..~AO"" O I 1;:‘95 ant: I n A - ‘ -.:.e...-c 9.... ‘ F 10:: *8: ‘ S acourse 0f renests th 1: presents are as ins Whi useersitie "L , ““9 15 110 I A . ivlaen ace FM... suture. quality, institutions must maintain a reduced number of programs which are of high quality and have clear objec- tives and outcomes. Striking a more moderate tone, Ikenberry main- tains that the expectations placed upon higher education ". . . require institutions to strike a better balance between the requirements for professional autonomy and academic freedom on the one hand, and the necessity for greater institutional accountability on the other" (39:34). Those supporting this point of view will strive to achieve a course of action which promotes reasonable response to requests that higher education account for itself while it presents adequate defense of fundamental principles such as institutional autonomy. While reactions have varied, few colleges and universities have escaped the impact caused by decreased support and the "new depression" within higher education. There is no evidence that great public favor and the ”golden age" will be regained within the foreseeable future; choices will have to be made in the distribution of scarce resources. Each program will have to be assessed in terms of its accomplishment of goals and objectives; that is, it will be subject to "increased accountability.” Off-campus agencies or campus admin- istrators responsible for holding programs accountable may conclude that programs which are not well organized I. v-“' The with: high gersmnel <5 5239 have a 52-329 those "the 19605 , :ized as su as“ ‘ ...ution 5““ ...tiny an or which are inconsistent with the goals and priorities of the institution should receive less support or be eliminated in order to balance the budget, redirect funds to programs with higher priority, or enable the develop- ment of new programs. Student Personnel Not Exempt The slide from "golden age” to ”new depression" within higher education has had an impact upon student personnel divisions as well as other programs. In fact, some have argued that student personnel programs are among those which were allowed to expand unchecked in the 19605, are not well organized, are not widely recog- nized as supporting the goals and priorities of the institution, and, therefore, should come under direct scrutiny and be among the first to be reduced or elimi- nated as budget cuts and program reductions occur. This attitude has not been ignored by many who are committed to the student personnel profession. For example, note the following comments by Shaffer, McIntyre, and Cross: [Student Affairs] . . . is now viewed, at the worst, as an expensive luxury with insatiable appetite for funds and staff, and at best, as a difficult- to-evaluate function which needs to clarify its roles, inputs, processes, and results. (82:391) Because of financial pressures, continued poor relations with the faculty members could work to the distinct disadvantage of the student personnel L1 '1 J H. I" v ‘ ‘ 1 O4: () f ) ' '\ ‘U (1‘ (I) (b profession. As money gets tighter, the temptation may be to cut back on programs that are not strictly educational in the traditional sense. (56:490) The first things to go in austerity budgets in which each line must be defended are the luxuries and the non essentials. Unfortunately, many faculty and some administrators see student personnel services as peripheral to the main business of education-- especially when it competes, as it must, with their own reason for existence. A tight budget induces competition, and only the strong survive. Objec- tively, student personnel administration is not a strong force. We are relatively few in number and we are weak in influence. We have no direct access to the seats of power of trustees and legislatures; we have no unions; we are ill-equipped to play the game of power politics. (20:20) Being under attack is not a new phenomenon for student personnel divisions, and some of its acknowledged leaders have offered suggestions about how the profession and its members should respond in order to strengthen its position. For example, O'Banion states: Purposes of student personnel work should be stated in specific behavioral goals. . . . Once specific behavioral goals are agreed upon, student personnel workers will know where they are going, what they are doing, and how to evaluate where they have been. (67:212) CIOSS states: The challenge, of course, lies in our ability to perform. The demands of that challenge are enormous. . . . In the first place, we must state goals that are widely recognized as important. Second, we must formulate specific programs that are designed to accomplish those goals. Third, we must demonstrate that we are making reasonable progress toward their fulfillment. . . . (20:30) Shaffer states: Before the student personnel subsystem of higher education can make the needed contribution to colleges and universities, it must clean its own house by clarifying its roles, reassessing its basic assumptions, and establishing an open relationship among its component parts. (82:388) Prepare To Be Held Accountable Suggestions made by the leaders of the profession are helpful in stimulating thought and shaping ideas. However, in order to effectively cope with the demands of accountability, each student personnel division must chart a course of action that is tailored to the charac- teristics of its campus and, at the same time, maintains professional integrity. It must be a course which main- tains divisional strength so that essential student per- sonnel programs are not reduced or eliminated. The chief student personnel administrator must be actively involved in institutional decision-making to insure that the philosophy and goals of the profession and division are represented and that the fundamental importance of the student personnel point of view is recognized as the institution responds to the "new depression.” Correlatively, and on behalf of the division, the chief student personnel administrator must be prepared to be held accountable for the accom- plishment of specific goals which clearly contribute to the mission of the institution. Therefore, the chief student personnel administrator must maintain a manage- ment system which insures that subordinates are prepared to and can be held accountable for contributing their :art to the ace: =22“ ;r:fessicnal tilitv means 135 O effectively . As Accountabi li ea:h member one for 6.01: plans and a; tngible per everyone wh: sun-ably to I it assumes 1 tributes to Which does r is intended, of every me: functional. In order 9°“? and able results' in a cc Arsomel admini Aedivision' mu .9 t L5 " n ,' bGuSestent w; part to the accomplishment of institutional, divisional, and professional goals. In the final analysis, accounta- bility means insuring that individuals are functioning effectively. As Lopez notes: Accountability refers to the process of expecting each member of the organization to answer to some- one for doing specific things according to specific plans and against certain timetables to accomplish tangible performance results. It assumes that everyone who joins an organization does so pre- sumably to help in the achievement of its purposes; it assumes that individual behavior which con- tributes to these purposes is functional and that which does not is dysfunctional. Accountability is intended, therefore, to insure that the behavior of every member of an organization is largely functional. (53:231) In order to insure that individuals are "func- tional” and able to ”accomplish tangible performance results” in a cost-effect manner, the chief student personnel administrator, as well as other leaders within the division, must accept responsibility for preparing staff members to complete their jobs in a fashion which is consistent with institutional, divisional, and pro- fessional goals. Clear job descriptions and objectives based on a sound philosophical foundation, well-developed programs, and management systems must be provided. How- ever, if individuals are to be held accountable, it is also the chief student personnel administrator's respon- sibility to insure the design and implementation of inservice staff development programs to prepare staff to achieve desired results. These programs must be 312295 CE IN Clari 10 designed to (1) make objectives clear as well as examine why they are important to the accomplishment of insti- tutional, divisional, or professional goals, (2) examine alternative methods, skills, and resources which could be used to assist in accomplishing objectives, (3) orient new staff to the characteristics and goals of the insti- tution and division, (4) reorient all staff as goals or expectations change, (5) prepare staff to use new tech- niques designed to improve divisional effectiveness, and (6) clarify the processes which exist to periodically assess progress toward the accomplishment of objectives. Various authors cite the importance of designing and implementing inservice staff development programs to maintain competent and talented professionals who will be able and willing to continually perform in accordance with institutional, divisional, and professional goals. Richardson notes: Higher education in the seventies faces challenges unique to the experiences of those who entered the field in the mid-fifties or later. Crucial to the successful administration of institutions in a new era . . . is a concept of change by substitution or contraction rather than by growth. If insti- tutions can no longer be changed primarily by the process of adding new personnel, then steps must be taken to help existing staff members adjust to the new demands being made of them. The process of improving staff capabilities for dealing effec- tively with new and continuing responsibilities is most commonly referred to as staff development. (76:303) o vnfwr F 2.... agvd‘ §aonhs . d on nyqvvq‘ A v- ». rue! .ue CU 9c":' an a: d -- O I uo'ugv'hi . S‘~:~, enemy, w] nL “93:1. 4. “8‘” 9109111 to it's». Tate 0V 736:9: sore ' f Pment ‘ ere he ‘ ‘1 tr. 11 From Richardson's point of view, inservice staff develop- ment programs must be designed and implemented in order to provide current employees with the opportunity to develop and maintain the skills necessary to meet the changing expectations placed upon them as the programs to which they are committed are phased out and replaced by new ones or as existing programs are eliminated and employees are absorbed by remaining programs. Beeler explains that: [while] . . . many institutions of higher learning are faced with financial exigencies . . ., the need for highly qualified and competent student personnel workers has never been more urgent. Services have subdivided into specialities, and been joined by new ones such as minority services. The depressed job market, however, has slowed the influx of new professionals into the student personnel field and forced staffs to seek among themselves for the new skills required to meet the needs of increasingly diverse student bodies. Normally, new staff members are looked to for new ideas and fresh insights on emerging issues and trends, and for adding profes- sional enthusiasm and vigor. Establishing or expanding in-service programs for present staff members can help replace this professional 1055. (6:38) Similarly, Williamson and Biggs note that, “With the ever- changing societal role of education, it is increasingly difficult to secure members of the staff who will remain adequate over the years until retirement" (99:169). Therefore, from their point of view, inservice staff development Opportunities must be provided in order to insure the ". . . continuous upgrading of the competence of the staff" (99:169) so they remain ". . . current £25 releva: :es respcn. :zrodsced, surge, 5:: :: ievelop have been 4 graiuate 0: W1". 55 P»; b~uCe St | I.-. 12 and relevant . . ." (99:169). As individuals take on new responsibilities, as new techniques or programs are introduced, or as the responsibilities of the division change, staff members need to be provided opportunities to develop those skills and abilities which they may not have been expected to master as part of their formal graduate or preservice training program. While this discussion of the importance of in- service staff development programs has centered around the need to respond to the demands for accountability, diminishing resources, and ever-changing programs, and while additional reasons that inservice staff development programs are important to the success of student person- nel divisions will be presented in Chapter II, the author wishes to stress the important role that these programs have in insuring, apart from external performance demands, personal growth and intrinsic satisfaction for individual staff members. As professionals who take pride in them- selves and their programs, student personnel staff members are committed to remaining well informed about develop- ments in the profession; mastering the skills and abili- ties necessary to continually improve their contributions to students, the institution, and the profession; and insuring their own continued growth and development. Inservice staff development programs which are well . . “xvi! 9" N .;.,.5‘ .Eu 5 r: assiSti: "‘ ~ o; *ocia'$c '6~:: ‘ ‘ 4 F v N uh‘.‘ cc‘i'e“ Pensibi] 9-39 Sgecifj 9-.1 ' “r:9~'3€1‘.t1r LillatiOn 1 filtt and .9115“ A. giOn Of 11C oliaro 13 conceived and implemented can have an important role in assisting staff members to fulfill this commitment. Need for the Study Student personnel administrators who accept an obligation for designing and implementing inservice staff development programs will find few resources to draw upon for assistance as they strive to meet this responsibility. Few publications within the profession are specifically devoted to the topic of designing and implementing inservice staff development programs. Two clear exceptions to this generalization are "Inservice Education for College Student Personnel” written by Truitt and Gross (93) and ”In-Service Development: A Function of Student Personnel" written by Stamatakos and Oliaro (87). As will be illustrated in the review of literature in Chapter II of this study, occasionally, articles devoted to other topics make reference to inservice staff development or an article is published which describes a specific program or technique that can be used for inservice staff development. However, the following illustrations support the general conclusion that neither graduate preparatory programs nor profes- sional literature available within the student personnel field provides many resources that can be consulted to iezernine u}: re} ZZEIECtEIISCIC 14 determine which steps ought to be followed or why certain characteristics ought to be included in the systematic design and implementation of inservice staff development programs. 1. Mueller's book, Student Personnel Work in Higher Education (1961), long used as a basic graduate text for many practicing professionals, makes no specific reference to inservice staff development as a function or responsibility which the members of the profession should be prepared to fulfill. Another basic text, William- son's Student Personnel Services in Colleges and Universities (1961), makes passing reference to the use of staff meetings (98:126) for inservice staff development, and a later text by Williamson and Biggs entitled Student Personnel Work, A Program of Developmental Relationships (1975), indicates the importance of allowing staff time to continuously upgrade their competence (99:169), but neither text devotes any attention to the importance of staff development in its own right or to the importance of careful design and imple- mentation to insure that inservice staff programs promote staff development. A similar omission occurred in the Fitzgerald, Johnson, and Norris book, College Student Personnel: Readings and ations' fessicn ment Ed 0f Stud entitle C011Ege Persona ADServi which m A IGVie 0 w 1% to Mar: to Win~ ASSOci. \ I an 111331 and Th 1965 t 15 Bibliographies (1970). While the need for and importance of inservice staff development is implicit throughout Miller and Prince's book, The Future of Student Affairs, A Guide to Student Development for Tomorrow's Higher Education (1976), specific attention is not given to the topic at any point. Neither the Council of Student Personnel Associ- ations' statement entitled "A Proposal for Pro- fessional Preparation of College Student Develop- ment Educators" (1971) nor the National Association of Student Personnel Administrators' Bulletin #1 entitled "A Guide to Programs of Training for College and University Student Services and Personnel Workers" (1966) makes reference to inservice staff development as a responsibility which members of the profession should be trained to accomplish. A review of all articles published in The Journal of College Student Personnel from October 1964 to March 1977, The NASPA Journal from Summer 1964 to Winter 1977, The Journal of the National Association of Women Deans, Administrators, and Counselors from Summer 1964 to Winter 1977, and The Journal of Higher Education from January 1965 to March 1977, indicated that, with the 8X 0. (I) :3 CO re CC ir ic 16 exception of the publication by Stamatakos and Oliaro (87), there are no articles specifically devoted to the tOpic of designing and implement- ing inservice staff development programs for college student personnel professionals. 4. A review of all annotations published by the Educational Resources Information Center (ERIC) since 1956 identified only one reference specifi- cally devoted to the topic of designing and implementing student personnel inservice staff development programs. The ERIC descriptors reviewed were on-the-job training, inservice courses, inservice education, inservice programs, inservice teaching, and staff orientation. The identified annotation (ED 022-203) referenced an article by Truitt entitled "Factors Underlying the Need for In-Service Development Programs in Student Personnel Work." Truitt and Gross reach a similar conclusion regarding the attention given to inservice staff develop- ment programs in the literature. They note: Establishment of comprehensive programs of inservice education represents a long neglected need of student personnel staff which can con- tribute effectively towards solving some of the problems facing higher education, and more par- ticularly, the student personnel area. (93:3) Critical examination of professional literature and a survey of existing programs reflects that insufficient attention has been given to the potential role of inservice education programs. (93:4) ‘1 . 5 ,- w F 319... ‘L y I.‘ - P “v ‘ U. - ‘90 the e iltereSt. '99 the j “Slated : timing E so». “.161 As Stude prOfe mOre SuCh ‘ 17 Almost ten years later, Miller concludes: With the exception of a statement of needs and recommendations entitled Inservice Education in Student Personnel Work (Truitt & Gross, 1966), a doctoraI dissertation by one of those authors (Gross, 1963) and a more recent needs and recom- mendations paper (Stamatakos & Oliaro, 1972), there have been few writings directly concerned with in-service development of student affairs professionals. (63:258) It would be an oversimplification to conclude that inservice staff development programs do not exist and that student personnel professionals, while committed to the education and development of others, are not interested in their own growth. For example, in report- ing the findings of a national study of student personnel- related staff development programs conducted by the Con- tinuing Education Committee of the American College Per- sonnel Association, Miller states that: . . . a vast majority of institutions release student affairs staff members to attend off-campus professional development activities. . . . nine out of ten institutions pay half or more of the cost of staff development activity if there is a probable benefit to the institution for such attendance . . . . . . staff members in three-fourths of the colleges and universities participate in approxi- mately four hours of on-campus inservice staff development activity per month as part of their normal staff workload. . . . two-thirds of the institutions allow staff release time of up to four hours per week to attend academic courses. . . . (63:262) A more realistic explanation of why many resources do not exist within the professional literature to assist those who wish to design and implement inservice staff ietelcp’:18r. t :as‘. 7‘: :99: ' . 233551 :9 c it met 12151 .. the c0116 t: problems a: of ‘165 allege or u ticing profs energy seems assi “‘99P deal wit 5&8e 9 “P011 sor 2‘53“ t! StUd6r 18 development programs is formulated by examining the basic nature of the student personnel field. It is a profession of practitioners who are called upon daily to meet individual student needs, improve the quality of the collegiate community whenever possible, and react to problems which arise without warning. The accomplish- ment of these roles provides valuable service to the college or university community and stimulation to prac- ticing professionals. Unfortunately, little time or energy seems to remain for such proactive efforts as the creation of resources for use by those who wish to design and implement inservice staff development programs. The need for such resources remains and is likely to grow if staff are to be prepared to demonstrate the achievement of specific goals. Recognizing that the lack of resources creates a problem for those who wish to design and implement inservice staff development programs and that more assistance is needed, it is the goal of this study to help deal with this problem by developing such a resource. The author believes that if student personnel administrators had adequate resources to assist them in design and implementation, inservice staff development programs would occur more frequently, they would be based upon sound instructional procedures, and, as a result, student personnel staff would be better prepared .p. 5“. 'v' .y. a-.;;e V‘ fiiielire it help g Staff dev. u '77 LviSiOfis 19 to achieve institutional, divisional, and professional goals and to demonstrate their achievements to those responsible for holding campus programs accountable. Purpose and Procedures The purpose of this study is to develop a set of guidelines and a checklist which can be used as a resource to help guide the design and implementation of inservice staff development programs within student personnel divisions. The author is hopeful that the resource developed will be of particular value to those student personnel administrators who are formulating inservice staff development programs because they accept the follow- ing proposition: Student personnel divisions will be called upon to demonstrate the accomplishment of specific outcomes which are consistent with and contribute to the achievement of institutional goals and priorities. As a result, student personnel administrators must hold members of their staff accountable for achieving spe- cific institutional, divisional, and professional goals. To do so effectively, they, themselves, must accept responsibility for the design and implementation of inservice staff development programs which will insure that their staff members are prepared to perform in a manner which will enable the division to demonstrate that desired outcomes have been accomplished and per- formance standards achieved. niel's le hit-:21 varic ‘w ':":.v< -~:-u-ESS its realiz; iE'FELQPEent ilCi'Jidual: 2999911118 1 533231 ins- Katie 0:3] . a ‘fi .e'V‘c. 4 niZQd Pr rt 96 Pt 20 Throughout the study, it is assumed that, since inservice staff development programs, like other instruc- tional programs, are designed to facilitate an indi- vidual's learning, they should incorporate those elements which various authors agree should be included in the design and implementation of all instructional programs, regardless of their setting. This study is based upon the realization that student personnel inservice staff development programs, while they may be directed at individuals who have had a good deal of formal training and while such programs may take place outside of the formal instructional setting, are basically designed to provide opportunities for individual learning so that performance goals can be achieved. Therefore, it is recognized that in order to maximize the impact of these programs, their design and implementation must be based upon educationally sound instructional procedures. Accordingly, the process followed to complete the study and to develop the guidelines and checklist is: (l) to investigate literature which examines the design and implementation of instructional programs for use in formally recognized class- room settings as well as training literature developed for settings other than student personnel; an;- ‘as 14) to as 21 (2) to abstract from this literature instructional program elements which, because there is agree- ment among various authors concerning their value, are determined to be appropriate for inclusion in the design and implementation of instructional programs; (3) to discuss the role of each element in the design and implementation of instructional programs; (4) to create, using the discussion of each element as a source, a set of guidelines which are organized into an orderly framework so they can be followed in a step-by-step fashion by those who are designing and implementing inservice staff development programs; and (5) to develop statements for each element which can be responded to in checklist fashion so that proper utilization of each element will be insured as inservice staff development programs are designed and implemented. It is not the intent of this study to design a set of instructions for use in the development of a specific program with a particular subject matter. Rather, the purpose is to develop a set of guidelines and a checklist which will be applicable in the design and implementation of any inservice staff development ~ .1 4‘ . . . :‘a .f‘ O ‘V' och .U luSu. :zec‘rlist c: .1 . . "" Qua r— .uovt ¢.A\. A. l— . 'V‘Q‘Je- u v- e :"~va\h-eb. sen'ice staf 2'li;:ent whi 22 program, regardless of the subject matter and setting, and to insure that those who use the guidelines and checklist create inservice staff development programs which incorporate educationally sound instructional procedures. Without such a resource, designers of in- service staff development programs must use their best judgment which may or may not result in educationally sound programs that achieve desired outcomes. Addi— tionally, it is the author's belief that those who review the entire study will understand how each element contributes to the accomplishment of instructional goals rather than routinely perform steps without an awareness of their educational significance or impact. Format of the Study The study is organized into five chapters. Chapter I has introduced the study by examining how trends within higher education have emphasized the importance of inservice staff development programs. After establishing the need for additional resources to assist in the design and implementation of these programs, the purpose of the study and procedures to be followed were presented. Chapter II reviews the literature within the student personnel profession that deals with inservice staff development. The review is organized to direct attention to three distinct themes which have appeared :2 the lit: sauce st; sen'ice st. said be I :f inservi- Cl“. 5:3: sourc lzterat'ore Priate f3;- 35 inservi atthors CC 23 in the literature. They are (1) the importance of in- service staff development programs, (2) suggested in- service staff development programs, and (3) points which should be considered in the design and implementation of inservice staff development programs. Chapter III identifies those elements extracted from sources other than college student personnel literature that are considered pertinent to and appro- priate for inclusion in the design and implementation of inservice staff development programs because various authors consistently present each as being essential to the proper design and implementation of instructional programs. Once identified, each element is discussed independently in order to clarify its role in the design and implementation of instructional programs. The effec- tiveness of the study is largely dependent upon the quality with which the decision to abstract each element and identify it as valuable for inclusion is defended and supported. Chapter IV presents guidelines to be followed in the design and implementation of inservice staff development programs. The guidelines are developed to insure that each element abstracted from the literature and presented in Chapter III will be accounted for as inservice staff development programs are created. A a. 3231151: for 1 Ear. incorpora' 3.3mm is als Chapte geezi as for t 0.3? +‘.‘ .v. Hte prefeq 24 checklist for use in assessing whether each element has been incorporated into an inservice staff development program is also presented. Chapter V summarizes key points and offers sug- gestions for the use of the guidelines and checklist. The implications which the author feels the study has for the profession are presented. 3:162:10: CHAPTER II THE STATUS OF INSERVICE STAFF DEVELOPMENT WITHIN STUDENT PERSONNEL Usually referred to as inservice training, attention to inservice staff development within student personnel literature is limited and widely scattered. However, references which do exist provide insight into the status of inservice staff development programs within the profession. The review presented here focuses on three distinct themes which have received attention in the literature. They are (l) the importance of inservice staff development programs (2) specific inservice staff development programs which are suggested for implemen- tation, and (3) points which should be considered as inservice staff development programs are designed and implemented. The Importance of Inservice Staff DevéIBpment Programs The design and implementation of inservice staff development programs is identified as a critically important responsibility of student personnel 25 26 administrators. Stamatakos and Oliaro contend that "in- service development" must be accepted as ". . . a function of an effective student personnel program . . . aimed at synthesizing and integrating the commitment, expertise, and efforts of the entire staff in the direction indi— cated by the philosophy of the student personnel division and the objectives of the institution” (87:271). The reasons presented by various authors to explain the importance of inservice staff development programs are varied. Swearingen (90) notes that a con- stantly evolving body of knowledge requires staff members to meet together to examine relevant research and the latest techniques available as well as for stimulation, sharing of ideas, and the development of mutual interests. Brunson maintains that ". . . if we allow ourselves to neglect the study of significant ideas, of research find- ings, and the like that are coming forth daily, we may find ourselves hopelessly out of date" (12:153). To remain up to date, Williamson encourages student person- nel professionals to continually examine ". . . new findings in adjacent areas of knowledge, particularly in the social sciences, psychiatry, education and psy— chology” (97:4). Truitt and Gross consider inservice staff develop- ment programs to be critically important if staff are to remain informed and responsive to change: 27 Inflexible staff and static programs will not suffice during the period of rapid transition and changing demands on higher education and the profession of personnel work. Demands made on individual students and colleges call for broader and more diversified approaches to student life programs. The need for staff upgrading is further emphasized by the great strides being made in man's knowledge, maturity, and problem-solving methodology. (93:16) Stamatakos and Oliaro believe that well-planned inservice staff development efforts which are integrated with other student personnel functions are important because they ". . . provide greater opportunities for preparing and developing programs and services to anticipate and meet student needs rather than have to react belatedly to current press from the college community" (87:272). Richardson maintains that inservice staff development programs play an important role in preparing faculty and staff to effectively interact ". . . on the 'front line' with dozens of students" (77:38). Similarly, Samler states: . . . that there is a constant on-going need for inservice training; that there is a need to explore existing problems, to establish pertinent principles and guidelines, and to create, by tapping successful experiences in many places, a fund of techniques . . . which can be adopted. . . . (78:16) Passons recognizes that an important outcome of inservice staff development programs is the increased understanding which staff from various offices gain of the similarities and differences of their roles in light of the overall goals of student personnel work (70:38). 28 Similarly, Stamatakos and Oliaro note that, among others, key outcomes which can be expected from inservice staff development programs are that: Each staff member would have a better overall understanding of the functioning of the student personnel division and could better articulate its objectives and philosophical direction to those members of the college community with whom he- comes in contact. Each staff member would have an opportunity to share his philosophy, expertise, and experiences as well as learn from the philosophy, expertise, and experiences of other staff members, and, as a result, better prepare himself for additional responsibilities. (87:272) Wanzek and Canon note that, as a result of staff develop- ment programs: . . . staff become more interested in the division as a whole, develop self-confidence, learn who they are and what they want, learn strategies to achieve their ends, and know how to work together as a total division. [There is also a] . . . breakdown in the isolation and self-interest of individual departments and greater cohesiveness and interest in the service to students. . . . (95:421) Williamson links the importance of inservice staff development to the maintenance of respect for the members of the profession within the academic community. He writes: Our work takes place in a societal institution devoted to the education of youth. We are not operating within a closed orbit of technical per- sonnel work, but within the context of an edu- cation that is broader and deeper than housing, counseling, and reading remediation. Staff development should stimulate continuous understand- ing by staff members of the various philosophies of higher education which compete for adoption by the faculty and central administration. (98:126) .v ' not! _" “-wov" I . neon. V“ uboe b {2‘ 29 Without such background, Williamson is fearful that stu- dent personnel professionals will be seen as "technology- bound interlopers on campuses properly devoted to higher learning" (98:126). Williamson and Biggs stress the importance of continuing study in order to maintain respect within the academic community. They explain: It cannot be stressed too much that the upgrading of staff members must also require some increased and current (perceptible) academic competence, not merely with respect to advanced academic degrees but also in reading outside of one's field of specialization and in conferences, informal and formal, with members of the faculty so that the faculty and the students come to respect the members of the staff as being intellectually competent. (99:170) Truitt and Gross note the importance of individual growth and development. They maintain that well-conceived inservice staff development programs should insure: . . . the stimulation and promotion of professional growth to enlarge one's vision, purpose, and moti- vation and enable each staff member to realize higher levels of responsibility and achievement; and the development of specific techniques and pro- cedures to assist in fulfilling more narrowly defined job responsibilities. (93:3) Williamson states that "in-service training" for indi- vidual staff members ”. . . is one of several means of up-grading and maintaining technical competence and effective functioning through continued professional growth" (97:4). Leventhal and Pumroy note that it is important to recognize that the need for inservice staff development extends to all staff and should not be viewed ”. . . simply as something to be provided to the 30 neophyte, but available in palatable form to those in whom society and professions have already made an invest- ment“ (49:296). The most comprehensive list of reasons for the design and implementation of inservice staff development programs for student personnel staff is found in the 1966 NASPA Bulletin by Truitt and Gross entitled "Inservice Education for College Student Personnel.” The reasons they list are: The inadequate or unrelated preservice education of many in student personnel work underscores the importance of inservice education. . . . Student personnel work is a dynamic and changing field of endeavor, and creative means of keeping abreast professionally must be continually sought. . . . Continued professional growth of its members is one of the distinguishing features of a profession and can be achieved through inservice education. Inservice education programs can effect change in the student personnel program. . . . Inservice education is one means by which policy change can be implemented and communicated. A structured inservice program is necessary to provide continuity for a specialized and constantly changing staff. . . . Inservice education programs can provide for the development of common objectives, a unifying frame of reference, a means of communi- cation, and the improvement of functional relation- ships of specialists that must work cooperatively. The fact that new staff members rarely assume their initial positions at their peak effectiveness requires the establishment of inservice education programs. Inservice education programs provide excellent opportunities for each staff member to contribute to the student personnel program. Inservice edu- cation should provide methods by which the creativity of individual staff members can be integrated into the developing program. An inservice education program provides oppor- tunity for staff members to . . . develop a common understanding of and approach to the objectives they wish to realize in student life programs. di99$ion 31 Properly planned inservice education programs enable the personnel staff to transcend the routine of daily personnel functions. . . . Inservice education activities that emphasize the broad foundations and implications of student personnel work enables a staff to think beyond the present problems and expand professional horizons, not only affecting what is accomplished today but in the future as well. An inservice education program assists in raising aspirations of staff members. . . . When the stu- dent personnel worker is able to put into practice the knowledge that he has acquired, this new exper- ience is usually accompanied by an attitude of satisfaction and accomplishment. (93:5) In a paper entitled "Factors Underlying the Need for In- Service Development Programs in Student Personnel Work," Truitt restates and, in some cases, rephrases and elaborates upon the points made in conjunction with Gross. Additionally, he observes that staff develOpment efforts should be designed to accomplish four major objectives. They are: 1. Teach the philosophy and objectives of the institution. 2. Teach each individual the responsibility of his position. 3. Teach each individual the skills and techniques for the effective fulfillment of his responsi- bility. 4. Stimulate each individual to assume higher standards of responsibility commensurate with his ability and experiences. (92:2) Suggested Inservice Staff DevelopmentHPrograms Some authors suggest a number of programs which they believe could be implemented by student personnel divisions that have recognized the importance of inservice Spec: SEVeI Staff Shoo} for E meeti SDOu] leaVe membe to f1 f0: 1 Sear: be me feSSi atior relat SUCh instr the 1 Staff 32 staff development. Truitt and Gross observe that inservice staff development programs generally fall within one of two areas of emphasis: . . . the first emphasis is general in nature and includes consideration of topics related to the overall objectives of the program and student per- sonnel work. Study areas such as "Institutional Goals and Student Personnel Work" and "Philosophy of Student Personnel Administration" are represen- tative of topics germane to the general emphasis of the program. . . . The second emphasis is more specific and includes topics related to the imme- diate concerns and special interests of partici- pants. These specialized areas of study are important at given times and for selected staff members. . . . (93:10) Recognizing that specific types of inservice staff development programs should be available to student per- sonnel professionals, Williamson suggests that: Special institutes, varying from a few days to several weeks, may be provided by and for the staff. Membership in professional organizations should be encouraged and budgetary provisions made for attendance on a rotational basis at professional meetings, regional and national. In addition, we should work to establish a program of sabbatical leaves equivalent to that provided for faculty members. Personnel workers, like teachers, need to free themselves from their work occasionally for refreshing growth experiences and an extended search for new ideas. Leadership provision should be made available for continued research and pro- fessional growth through publication and the prepar- ation of speeches. . . . Some kind of an inter- relationship of program and training is desirable, such as the use of professional staff for part-time instruction, close professional relationships, and the interchange of ideas between the training staff and the professional workers. (97:4) inservic (a) Penj who dent t0 5 ing: Stat COuI Spec difl avaj Can Worl dCt; Sil9'eme growth ‘ "IiSOry ( recent I Workshoi 33 Williamson and Biggs maintain that staff members should be encouraged to pursue various activities designed to upgrade their competence. For example, they note: ”This means allowing them [staff members] time on duty for professional reading, for periodic exchanges, for joint projects, for weekly staff seminars, case confer— ences, and visits to other institutions and to national meetings" (99:169). According to Burnett, the most common types of inservice staff development programs are: (a) case conferences based on actual student hap- penings; (b) working closely with those faculty who are already interested in doing more for stu- dents; (c) using various communication channels to stimulate faculty such as department staff meet- ings and discussion of individual cases, weekly staff meetings with the staff of counselors, campus councils or committees which meet regularly, and special seminars for new and old staff members from different parts of the college campus; (d) making available appropriate books and pamphlets which can be routed to the staff; and (e) conducting workshops, conferences, projects, and research activities. (13:131) Silverman (85) provides a similar list of "professional growth experiences" with suggestions ranging from super- visory conferences to making time available to review recent publications. Truitt and Gross suggest: ". . . workshops, case studies and conferences, research, tape recordings and films, staff seminars and retreats, directed readings and discussion, visiting lecturers, interschool visitations, panels, role-playing, individual evaluation and supervision, and attendance and heads 5! 5it’isio: for cri- each St tiv Whi con div are l) 2) 3) 4) 5) 34 participation at professional meetings" (93:3). Foxley suggests that inservice staff development activities be designed to ". . . provide experiences for: (a) effective communication and active listening, (b) work with people of different backgrounds, (c) case study exercises, and (d) self assessment" (26:204). Stamatakos and Oliaro suggest that a weekly staff conference where department heads share their plans and accomplishments helps to keep division members informed as well as provide opportunities for critical feedback. In addition, they suggest that each staff member should be: . . . responsible for the exploration of a substan- tive area outside his specific job responsibility which, in turn, would introduce fruitful areas of concern for the systematic examination by the division on a monthly or bi-weekly basis. These areas should include: 1) Current and developing issues in higher education 2) Professional activities, publications, workshops 3) Interdivisional communication within the college community 4) Current and developing societal issues as they affect or could affect the campus 5) Innovative student personnel programs on other campuses. (87:271) The literature contains few references to spe- cific inservice staff development programs which have been implemented. However, the programs which have been implemented and reported in the literature provide examples of other types of inservice staff development programs that could be implemented by student personnel divisions. 35 Many of the references to specific inservice staff development programs pertain to efforts designed to prepare residence hall staff to meet their responsi- bilities. Collins (18) suggests the assigned reading of books which examine counseling practices and procedures as a method of increasing residence hall staff competence and motivation. Operating with the belief that ". . . participation in staff planning increases professional growth," (69:523) Ohlson reports utilizing staff com- mittees to develop staff evaluation plans, an annotated bibliography for staff reference, and a residence hall manual. Sandeen (79) discusses the use of a required course which examined the objectives of higher education, characteristics of college students, and the role of the resident assistant. This course served as a formal supplement to on-going inservice training in each hall. Schroeder (80) suggests that adventure training provides a means of facilitating personal growth and group development for RAs. Jackson (40) developed a counseling- oriented program designed to prepare hall staff to meet the educational and developmental needs of resident stu- dents. Bellucci used Kagan's Interpersonal Process Recall Stimulus Films to sensitize ”. . . a group of resident assistants to several varieties of emotional confrontation" (7:108). Spurrier and Collins note that: "A program for in-service training of RAs should 36 emphasize the expectations and objectives for them . . . [and should] include an explanation of residence halls policies; a discussion of the 'when,‘ 'to whom,‘ and the 'how' of referrals; and a presentation on counseling techniques" (86:261). Schroeder, Hill, Gormally, and Anthony (81) report the success of a program in human relations training for resident assistants. Newton (66) examines a program in which communication skills were taught to resident assistants in order to improve the quality of their helping efforts. Shelton and Corazzini (83) developed a training program to increase residence hall paraprofessional knowledge of and to facilitate their referral of students to campus resource agencies. Miller explains the important role of inservice education in the ”. . . professional preparation and development of the residence educators who have been charged with the responsibility of implementing the student develop- ment function within the campus residence community" (62:165). Greenleaf discusses the various topics and goals which should be incorporated into the ". . . (1) spring orientation, (2) preschool orientation, and (3) on-the-job orientation" (37:189) of residence hall staff members. Powell suggests that inservice education for resident assistants should seek to incorporate four dimensions. They are: ". . . (1) education for orien- tation and information; (2) education in mutually mv-era“ ' H 'V" P:““ 5‘..V. say" I ling A uzct've r {lOI a: usat prov 37 cooperative relationships; (3) education in intra- and inter-personal dynamics and facilitation; and (4) edu- cation for contextual integration of students' develop- ment" (75:202). While very limited in number, various types of inservice staff develOpment programs which have been utilized in areas of student personnel other than resi- dence halls are presented in the literature. Bonar (9) (10) and Clark et al. (16) describe programs to train paraprofessionals as academic advisors while Westbrook and Smith (96) discuss training for paraprofessional Black peer counselors. Kirk (46) discusses the inte- gration of individual conferences, group meetings, and case studies as an effective way to provide for inservice counselor training. Harvey, Helzer, and Young consider the staff retreat to be a productive setting for in- service staff development because it enables the "use of small group methods to maximize departmental effec- tiveness . . . [and provides] a relatively non-threaten- ing and generally supportive climate . . ." (38:274) for the development of individual skills. Describing a “Mini-U” program, which consists of inservice staff development courses of approximately a month in duration, Beeler notes that the objectives of the program were: . . . (l) to provide staff with the opportunity to enhance, refresh, and learn skills and techniques usable in daily student personnel work, (2) to provide staff with the opportunity to study and C an +4 1? r1 a, (J ._ Coan ( attitu Latdic Implem 38 examine timely issues, trends, and concerns in stu- dent personnel work and (3) to provide staff with the opportunity to interact with fellow professionals from other student personnel offices and with other persons within the university-community. (6:40) Lane found that encounter groups enabled the individual staff member to: . . . gain increased insight into himself, particu- larly as others perceive him. He can also gain greater understanding of the dynamics of groups as they deal with the tasks at hand and of the indi- viduals within the groups. (47:185) Coan (17) describes a staff workshop designed to improve attitudes toward research and evaluation. Laudicina and Laudicina (48) point out that a carefully developed staff evaluation program which assesses abilities and weak- nesses contributes to staff development. Finally, Lewis maintains that national associations contribute to pro— fessional development. He contends that: Our Association (ACPA) is at once undertaking direct training and retraining efforts through workshops, conferences, and seminars. . . . Assistance in improving professional competency and skills among our members must continue to be a major concern. Our Association provides a unique opportunity for interaction with those whose primary concern is professional education. . . . Our unique general membership approach among student personnel associations can and does facilitate constant interaction between our educators, graduate students, and professional practitioners. (50:149) Points To_Consider in Design andiImplementation As consideration is given to the design and implementation of specific inservice staff development 8 NV "5: “:03“. .I. 6 '="1cu. VJ.» ‘ por Stud eVa] ties 0the Plan lndi need 39 programs which meet the needs of or are preferred by a particular student personnel division at a particular point in time, Kirk cautions that ". . . no one training technique is most effective. . . ." The primacy of the technique varies with the training, experience, and needs of each individual" (46:207). A review of the literature identifies a number of other points which authors recommend be considered by those who wish to design and implement an effective staff development pro- gram. A lengthy list is reported by Truitt and Gross in their 1966 NASPA Bulletin on inservice education: An inservice education program should be based upon objectives which give direction to the overall student personnel program and provide a basis for evaluation. Too often professional growth activi- ties lack overall purposes and are unrelated to other existing aspects of the inservice program. Each inservice education program must be planned, initiated, and perpetuated in view of individual staff and institutional goals and needs. . . . No one pattern or model is univer— sally applicable to all institutions. Inservice education programs should be geared to varying levels of professional preparation and experience of individual staff members. Inservice education programs should involve maximum participation of the total staff in the planning and ongoing activities. . . . Such a procedure allows the staff to express topics and activities of special interest and need while at the same time it builds mutual respect and support and fosters individual creativeness. Study t0pics and activities for inservice edu- cation programs should reflect both immediate and long-standing issues which face the staff, insti- tution, and student personnel work as a profession. Inservice education programs should utilize the knowledge and skill of the program participants as well as that of consultants and other resource personnel. A 6 Ha» of its 581’ one re; 0‘ 0115 the he 5. the di: re: s: In: eye] 40 Opportunity should be made to allow the appli- cation of new knowledge and increased understanding of theory and technique, which are gained through inservice education activities, to the program and services of the institution. Inservice education programs should be continu- ously planned, conducted, and maintained during a regularly designated time in the normal work schedule of the staff. Inservice education programs should be continu- ously evaluated with program participants playing a major role in the evaluation. . . . Applying the results of an inservice education program to the on-going, daily functioning of the institution and then evaluating the outcome are essential to the successful inservice education program. Responsibility of initiating, implementing, and directing an inservice education program should rest with one individual, preferably the chief student personnel officer. (93:7) Developing a similar list of points that should be incorporated in an effective inservice staff development program, Wilson states that: . . . inservice training must be continuous, . . . must be adapted to varying levels of professional readiness, . . . must be multi-disciplined, . . . should make broad use of the literature in the field, . . . must recognize personality needs of the staff, . . . should utilize community resources, . . . should be planned by the group, . . . [and] must be integrated and modified in terms of situational needs. (100:53) Federico suggests that the implementation of staff development activities should be guided by four questions: ". . . Where are we presently? . . . Where do we want to go? . . . How can we attain the goal? . . . [and] How will we know when we have attained the goal" (25:75)? While not providing generalized lists, other references elaborate upon aspects of an effective 41 inservice staff development program. The importance of support from top-level administrators is given particular attention. Stamatakos and Oliaro maintain that inservice staff development programs should reflect and be con- gruent with the chief student personnel administrator's ". . . analysis of the present situation within the division and the institution, and the current needs, competencies, and experiences of his staff . . ." (87:272). Silverman notes: "If top administration wants a profes- sional atmosphere and professional growth, this will materialize through the tone, the pace, and the example set by the director" (85:393). Swearingen (90) suggests that, in addition to providing time and resources, super- visors can be supportive of inservice staff development by being sensitive to and facilitating programs in response to areas of special staff need, encouraging cooperative staff efforts to develop inservice staff development programs, communicating the expectation that staff participate in growth experiences, expressing willingness to be of assistance, and fostering confidence by accepting individual staff efforts to grow and change. Harvey, Halzer, and Young (38) urge the develop- ment of a set of objectives as well as the identification of needed resources and the design of a plan of action to insure practical application and utilization of material learned during each inservice staff development aaaaaaa 42 program. While they are primarily interested in utili- zation of retreats for inservice staff development pro- grams, the points they make concerning efforts to inte- grate that which is learned into the daily work practices of the employee should be given careful consideration as part of the design and implementation of all inservice staff development programs. As Richardson states: Certainly, staff development requires exposure to new ideas and practices, but this represents only a starting point. Unless information gained through the exchange process is integrated into ongoing institutional experience, it loses most of its potential effectiveness. If our goal is response to changed circumstances, staff members must not only be aware of new ideas, they must understand how these relate to institutional processes and priorities; and they must be moti- vated to turn their information into action. (76:304) Conclusion The literature presented here discusses the importance of inservice staff development programs, specific programs which authors suggest for implemen- tation, and points for consideration as inservice staff development programs are designed and implemented. How- ever, the literature does not provide a set of guidelines for the design and implementation of inservice staff development programs which, if followed, will help insure that these programs include the elements which are incorporated in instructional programs in more tra- ditional learning settings. The next chapter presents 43 materials from sources other than the literature of student personnel in order to identify the elements to be incorporated into and the guidelines to direct the design and implementation of instructional programs for the purpose of inservice staff development. CHAPTER III IDENTIFY AND EXAMINE INSTRUCTIONAL PROGRAM ELEMENTS In preparing to properly design and implement an instructional program--whether it is for use in public school or college classrooms, industrial training, or student personnel inservice staff development--it is important to remember that the overall goal of the entire program is LEARNING and that, if it is successful, at its end students will be able to perform in accordance with stated objectives and thereby demonstrate that desired LEARNING has occurred. Supporting this con- tention, Banathy notes that the '. . . purpose of edu- cation is to ensure the attainment of specific knowledge, skills, and attitudes . . ." (4:24) and, therefore, LEARNING is the purpose for which instructional programs are designed. Logan defines learning as ". . . A RELA- TIVELY PERMANENT PROCESS RESULTING FROM PRACTICE AND REFLECTED IN A CHANGE IN PERFORMANCE“ (52:2). Similarly, Bass and Vaughan state, "Learning is a relatively perma- nent change in behavior that occurs as a result of 44 p:§“ He IE3913M: irstruc. Inst and 16a: tit} as 1 L131 the com; Stir Ver‘: fun: IE 8999161: its 19903 Circling 45 practice or experience" (5:8). Goldstein states that ”. . . training and education are defined as the sys- tematic acquisition of skills, rules, concepts, or attitudes that results in improved performance in another environment" (36:3). To increase the probability that desired learning will occur, it is necessary to design instructional programs which will assist participants in learning those skills required to improve their ability to per- form designated tasks or meet identified expectations. The careful design and implementation of instructional programs as a means for maximizing learning is widely recognized. For example, Gagné explains the function of instruction and its relationship to learning: Instruction may be thought of as the institution and arrangement of the external conditions of learning in ways which will opfimally interact with the internal capabilities of the learner, so as to bring about a change in these capabilities. Instruction thus deals with the manipulation of the conditions of the learning situation--with commanding attention, with presenting essential stimuli, and with the nature and sequence of verbal directions given to the learner. The function of instruction is the control of tHe external conditions of the learning situation. T313295) Similarly, Jahnke notes, "Instruction . . . is used in its most general sense to signify any environmental circumstances which establish the conditions of learning . . .” (41:181). Ericksen defines instruction as: . . . a multimedia implementing process between two anchoring points: the student and a body of a. :1. (D [1‘ [O' 91 '3 W) k‘ L; m f U) vi in 51‘ re. 46 knowledge. . . . Instruction is an idiosyncratic man-machine information-processing system in which the teaCher is the monitor. . . . Monitor: The teacher functions as the director of the Iearning process. Idiosyncratic: The enterprise of instruction requires expliciE recognition of and adaptation to the individuality of students--the prime responsi- bility of education in a democratic society. Man-machine: The student interacts with books, lecture notes, homework, audiovisual aids, com- puters--any and all kinds of people and devices used in instruction. Information-processing: By the various man- machine interactions, the student is presented with information, stores it, transforms it accord- ing to his needs, and retrieves it. His acqui- sition of knowledge, by reducing uncertainties, reinforces and motivates further learning. S stem: This word alludes to systems theory, whicH may be a passing fad. But in passing or until it passes, it provides a major contribution-- an integrating set of principles between the indi- vidual student and a succession of supersystems (the class, school, community, and society). (24:143) Gagné notes that when instructional programs are carefully designed, they ". . . create a learning situ- ation which in a sense 'captures control' of the nervous system of the individual so that he inevitably learns" (31:310). Similarly, Ausubel notes that teaching is ”. . . the deliberate manipulation of learning processes by some external agency for the purpose of enhancing learning outcomes" (3:212). Gagné points out that instruction is ". . . highly complex . . . [and that] it must be carefully planned and executed in order to accomplish its objective, which is to bring about learn- ing in another individual" (31:291). While learning is an internal, idiosyncratic event which cannot be 23.130118: H3) and inStruc he made. I prggra'ns ar grins are P the probabi quently, de. insure that organized t< 47 controlled directly, the careful planning of the instruc- tional process will increase the probability that desired learning will occur. Gagné notes: So far as is now known, instruction cannot directly control this internal event [learning]. The careful design of instruction can surely increase its proba- bility and, by so doing, can make the entire process of learning more sure, more predictable, and more efficient. (31:312) Having examined the relationship between learning and instruction, the following general observation can be made. Learning is the purpose for which instructional programs are designed. Assuming that instructional pro- grams are properly designed and competently implemented, the probability increases that learning and, conse- quently, desired student performance will result. To insure that environmental conditions are properly organized to maximize desired learning, instructional programs must be designed and implemented in an orderly fashion. Certain components or elements must be included. Davis et a1. explain: When we say that a learning system is an organized combination of elements, we mean two things: first, that there is an intentional arrangement of people, materials, and procedures (the elements of a system are not arranged haphazardly, but according to a specific plan); and second, that the elements of a system are interdependent (the people, materials, and procedures are part of a coherent whole where each contributes something to the others and every part is essential). Two of the essential charac- teristics of a learning system are a planned and interdependent arrangement of its component elements. CESSES have s“; bélietfe "v w 5‘~:.a:. 1R Ozzie 48 Banathy stresses that: It is the system as a whole--and not its parts separately--that must be planned, designed, developed, installed, and managed. What is really significant is not how the individual components function separately, but the way they interact and are integrated into the system for the purpose of achieving the goal of the system. (4:2) While they do not necessarily refer to the pro- cesses they recommend as "systems," a number of authors have suggested the components or elements which each believes should be incorporated into an instructional program and how they should be organized, as a whole, in order to insure that desired learning will occur. Banathy presents a six-part process to be fol- lowed in the "Design of an Instructional System" (4:28). She maintains that the development of an instructional program is a decision-making process that must be under- taken carefully if objectives are to be achieved. She explains that: ”Decisions have to be made about what should be learned, how, by whom, when, and where; how learning should be evaluated and improved, and what resources should be involved in preparing for, providing for, and evaluating learning" (4:28). Banathy believes that the structure she presents, which is illustrated in Figure 1, provides ". . . for the orderly development and change of the system" (4:29). She explains the six- step process as follows. (See page 50.) 49 A)! l I I III IV I FORMULATE ‘-€> ANALYZE -——§» DESIGN I OBJECTIVES LEARNING SYSTEM l TASKS I A I l I I I l l I II v I I DEVELOP ::;: IMPLEMENT (e—al I TEST AND TEST ' I OUTPUT . I I T I, ' , I I ' l 1 I VI ' I ' CHANGE | I : To IMPROVE I l I ' I I I ' I — — _. “ L — — — .- — _ _ _ - ~ 0—. _ _ J .. .. .. .. .. -feedback line Fig. 1. “An over-all structure of the design of an instructional system" (4:28). IV Cons VI . 50 I. The initial step is to formulate a statement that spells out what we expect the learner to do, know, and feel as a result of his learning experiences (Formulate Objectives). II. Develop a criterion test based on objectives and use it to test terminal proficiency (Develop Test). III. Find out what has to be learned by the student so that he can behave in the way described by the objectives' specifications. In the course of this analysis, the input capabilities of the learner must also be assessed--he does not have to learn whatever he already knows (Analysis of Learning Tasks). IV. Consider alternatives and identify what has to be done to ensure that the learner will master the tasks (Functions Analysis). Determine who or what has the best potential to accomplish these functions (Component Analysis). Decide when and where the functions are to be carried out (Design of the System). V. The designed system can now be tried out or tested, implemented, and installed. The per- formance of the learner, who is the product of the system, is to be evaluated in order to assess the degree to which he behaves in the way initially described (Implement and Test Output). VI. Findings of the evaluation are then fed back into the system to see what changes--if any-- are needed to improve the system (Change to Improve). (4:29) Davis et a1. present a "learning system design process" (21:19) to be followed as instructional programs are developed. As illustrated in Figure 2, this process is divided into three phases which are ". . . (1) analyz- ing system requirements; (2) designing the system; and (3) evaluating system effectiveness" (21:306). From the authors' point of view, the design process involves: ". . . the careful specification of requirements and objectives, the systematic analysis of these objectives / :1; .52.. It; 23.. Luz: urn: LELEOTEZ :RLTLC 7.qv— -r<-.—<>.- N zmu~usf~q~ \ p..N.....h \<2< V 51 .Amauamv gmmmooum smwmoo Ewumhm mcflsumoa wsu . . . mafiumuumsaaw ammum ocaalmfiflu s: .N .mwm NZHA HSHB aoumxm acouusu Obmuomoa mo>fluo0nno mugs: new o>fisoo coflum5~m>m 50wum5~m>m postcou CQHQ mxmmh. Obfluomoo ofioxoom cowmfl>om o>wuoombo tam mamas o~x~m=< newuosuumcH newuosuumcm acosoflasH :mflmoo zo~s<=q<>m zommmo mHm>qm~ FZW.ZA~OT~N>...~C QZAN CZHZHEH. .HZMHEMVTWVTWN 56 .Amaummv :Eoumhm HchAHUsHumcH ad: .m .mwm Hmmmcmua mumsHm>m H L+mswswmua mumsHm>m a 8L mcwcwmua Houwsozg H AL moosflmue ammumnm 1 _ mfluouwnu moam>oa_ AII mmdmm ZOHBdDA¢>m mewswmua uospcou L moaawocwum msflcummq can capo: mcwswmus uooamm L w mmdmm BZHZmOAm>WQ Q24 UZHZHde II 1; mo>wu0Onbo m>wumol4 mcomz ascowuosuunsH mmommd mmfiwm Bzmzmmmmmfl 57 .Awaumev wmmcoiuoowmm oocmsuomuom ocwesouoo «A e ammoooum msH>HOmIEmHnoum Hmumsom d: .v I. 22335 swoumsum campsfiom uuoaom .9 ‘4 IIHIL .6 enhanced mm omw>o Fm>flumchou~< newu3~ow us. A wanna—0:563. cowusaom ocwshouoa 9. av own-Hm Amtooz sou: season; suapcoea \7 L sq. IIIIIII tin. cat Kerr. AM). I] PIOCESS COR 1. Lie eac 2' EAL Stl be 3- Spe co: 4. Li: ob 5- De ba th- 6. Se ne tr ob 7. Co so to 9- EV ac re Pl BE 9180 St 9519118: I l- we 2- W‘r t< 3. IR he K: T (44:8). process 1. 2. 58 Revision of any or all previous steps (at any time in the process) to assure that the edu- cational system is responsive, effective, and efficient. (43:11) Kemp presents a process for "instructional design" Illustrated in Figure 5, Kemp explains that the consists of eight steps. They are: List topics, stating the general purpose of each one. Enumerate the important characteristics of the student group for which the instruction will be designed. Specify the learning objectives to be achieved in terms of measurable student behavioral out- comes. List the subject content that supports each objective. Develop pre-tests to determine the student's background and present level of knowledge about the topic. Select teaching/learning activities and the necessary instructional resources that will treat the subject content to accomplish the objectives. Coordinate such support needs as budget, per- sonnel, facilities, equipment, and schedules to carry out the instructional plan. Evaluate student learning in terms of the accomplishment of objectives, with a view to revising and reevaluating any phases of the plan that need improvement. (44:9) He also states that the plan enables the instructional designer to determine the answers to three questions: 1. 2. 3. What must be learned? What procedures and materials will work best to reach the desired learning skills? How will we know when the required learning has taken place? (44:9) Kibler et a1. present a "General Model of Instruction" (45:2). Illustrated in Figure 6, the authors note: 59 TOPICS AND GENERAL PURPOSES STUDENT CHARACTERISTICS IT'T’I LEARNING OBJECTIVES I r I I ‘— - - fi SUBJECT CONTENT I Ir L- -— - —) PRE-TEST I I l V p , r-->' TEACHING/LEARNING H SUPPORT 33 ‘ ACTIVITIES AND RESOURCES SERVICES 5 1 I; _ gy ° €- -- - --{ EVALUATION ] Fig. 5. Presented in Instructional Design by Jerrold E. Kemp (44:10). INSTRUCTIONAL___e,I p , INSTRUCTIONAL, OBJECTIVES PIKE-ASSESSMENT ? pROCEDURES ”a EVALUATION TIT—TI ET‘I III—T (FEEDBACK LOOP) Fig. 6. "A general model of instruction" (45:3). The I? model is the EfflC v—TTH s:e:I:Iec 4—7:?“— 1'er 1 uide in: me majo: instruct with whi process. The selection a: tives. Kibi ‘lIe secon tedetennin. . . , ( Objecti Should before instruc 60 The major philosophical premise underlying the model is that the goal of instruction is to maximize the efficiency with which all students achieve specified objectives. . . . ‘The two major functions of the model are (1) to guide instructionai designers and teachers through— the major steps in designing and carrying out instruction; and (2) to rovide an overall structure with which to view and study the instructional process. (45:2) The first step in this model provides for the selection and preparation of specific behavioral objec- tives. Kibler et al. explain that the completion of the second (pre-assessment) step enables the designer to determine: . . . (1) whether any student may omit any of the objectives in the unit: (2) whether any student should be required to master rere uisite skills before beginning the unit; ang (3) what specific instructional activities should be provided for specific students. (45:6) Explaining the third step, Kibler et al. note that: The design of the instructional procedures involves (1) selection of available instructional materials (e.g. books, films, or lesson plan); (2) preparing new instructional materials when necessary; and (3) developing a sequential plan which appears to be the most efficient for achieving the stated objectives. (45:7) The final step in the process provides for evaluation to determine ". . . whether the instruction was success— ful in achieving the unit's objectives" (45:13). The authors also note: "Changes in the objectives, the pre- instructional evaluation procedures, the instruction, or the post-instructional evaluation are to be made on the :asis of t 3.... on th PO ticnal Mod The author Litellectu g to a more Of a prose tires "3 :::: 61 basis of the evaluational results (note the feedback 1222 on the flow chart)" (45:14). Popham and Baker present a four-step "Instruc- tional Model" (73:13) which is illustrated in Figure 7. The authors explain that: "The model emphasizes the intellectual decision-making the teacher engages in REESE to and after instruction and, as such, is really more of a planning and assessment model than a 'teaching procedures' scheme" (73:13). IF OBJECTIVES ARE NOT ACHIEVED. REVISE SPECIFICATION OF ——e> PRE-ASSESSMENT=——e> INSTRUCTION ——€> EVALUATION OBJECTIVES T ] IF OBJECTIVES ARE ACHIEVED, AUCMENT Fig. 7. "A goal-referenced instructional model" (73:13) with "courses of action dictated by evaluation of results" (73:17). Popham and Baker explain the four-step process as follows: First, the objectives of instruction are specified in terms of learner behavior. Second, the student is pre-assessed as to his current status with respect to those instructional objectives. Third, instructional activities that should bring agout the intended objectives are designed. And fourth, the student's attainment of the objectives 18 evaluated. (73:13) data gathe: fiesigner s! to improve are mt ac} to male thI the instru< occurs if ( As the instru< that, Whilf 25" vary 5( agreement a "“011 shou tional prO Th authors 61' (N SEIQCt (5) RedESj is diSCus in the del grams. III the Views canOn of directiOr 62 Popham and Baker also note that, as a result of data gathered during evaluation, the instructional designer should be prepared to make revisions in order to improve the instructional procedures if objectives are not achieved. The designer should consider revisions to make the objectives more challenging and to design the instructional process so that additional learning occurs if objectives are achieved. As can be seen in Figure 8, a careful review of the instructional processes presented here illustrates that, while different terms may be used and the order may vary somewhat, there is a significant degree of agreement among various authors concerning the elements which should be incorporated into well-designed instruc- tional programs. The elements consistently recommended by these authors are (1) Assess Needs, (2) Develop Objectives, (3) Select Instructional Procedures, (4) Evaluate, and (5) Redesign. These five elements will be examined in the remaining sections of this chapter. Each element is discussed independently in order to clarify its role in the design and implementation of instructional pro- grams. The section entitled ”Assess Needs" will present the views of various authors concerning the identifi- cation of the relevant information necessary to provide direction to the instructional program design process III- IIIIIIIII .I.-I II b I .zfiiwbwfic' II I III I - EURCLLJ .vflmv 2C0 LO~>a2 .J .0 , .. l . L =_9~20.~L ~ 2 u - «a . I« Q .l yak? n0>~ubbfii \ .Au uCO k IUJLurCh «JUL—v9: u:.uLL-Q . .3 I m U mmbnvéx X... .-.u:0~v~ OCmI-UELC B~2~DEL0k \ II .I. I I\ - N! Ufiw >~F~I<7J~= .l\ M~><° III’ II. III I II I II- I III \ 19>w~ontfiLC ~2:O.. a 1:8 nUulah. v.0) m u U; “QC ~GL 329: \ $2.3. I Imclgmalblxk \ Z~MPPV44430 N REW<£€ 1.14.42 :ntr:< h: .Oau-.u..u‘031.n IU§L¢B§~ IbflflIuH:l_In../.. . I.II~4I_ZC..II.._Z.m nauseououm IMGCOwuuzhHmGH uuomom mo>wuoonao no~o>oa mpooz mummmd l zommaqqu «an omHaHezuoH EESmNezmzoazo. Ho mucocomsoo cm 05““ A nommsmuh _ ofiuxuom omw>om oua=~m>m _ commw>oa ucficqnuh Ilflcowun=~x>u newumsum>m ouna~w>m w “unocou t u . l mouw>som unwcwuuh :owuuouumcnm . useddam youmco: ucosoedEH m _ . u I mouusomom J m was .usu an M o>anE~ . 33232 32:2; 12.». on. H. 3 M acmchmoa unohIohm ” c "m a . oucmzu W \wcfisomoh . w E II. I wII l . mvco>wuuoumm amuoumuu .. “amuse _ utosmz< xuabvoom _ pmmbIoud aucasuomuom év>wum=~a>mv comum3~m>u m umoh can _ \om~>o :m—m _ . m onwauouoo no~o>oo “ uzoso_da_u . . mo>~uo-bow . L acoucou unwcmduh . . . Boun>m samuwsum>m camum3~w>m «cosomnsu oua3~a>m can mamuh uuombsm postcou o~x~ac< F :uwmon mousvououa smouauum mo~mwocwum a m xmmh I“ nomuuauumcH ”macaw mmwmwmmwuo :Owusfiom scmcwmmmwwu «souvMMw>oo .0Mwmwwwo m unmanned “ quuum:H . uuo~um uuomth . . . . Oaxaucd uo>wumchoum< madam" n w I mowuma can I . . mo>wuuo.bo.u accsmwomg gum—Hmong $33.35 mucus—033.com. mowwwwogo uuohouahwmfi 323 can Sawfly—on .0 m a unopsum cowuauom . on x awn—u o>Muon ocfiauouoo m. mo>wuu0nbo mo>m900nno monomusm vooz “snowy u0w>ngom laumxm mo Z283 «20:8 M3395 Iugumcu @333 38.39 «cinemas . mfiucovm I » oua~aauom munOMMMoo Iosuumcu ecu mowdoh mmomv< «wucovu obmuonoa zuxing whatever it is he is doing?" (59:28). He states: "I!egardless of how you choose to present it, your state- ‘nfiant of objective will define the behavior more sharply if it contains words describing the situation (givens, 99 allowances, restrictions) under which the student will be expected to show his achievement of the objective” (59:33). Turning to the criteria component of a well- written objective, Mager says: Now that you have described what it is you want the learner to be able to do, you can increase the ability of an objective to communicate by telling the learner HOW WELL you want him to be able to do it. You will accomplish this by describing the criterion of acceptable performance. If you can specify at least the minimum accep- table performance for each objective, you will have a performance standard against which to test your instructional programs; you will have a means for determining whether your programs are successful in achieving your instructional intent. What you must try to do, then, is indicate in your statement of objectives what the acceptable performance will be, by adding words that describe the criterion of success. (59:44) ldager suggests that "the criterion of acceptable per- :Eormance" can be stated within the objective by indi- <=ating, as appropriate, ". . . time limits . . ., . . . nminimum number of correct responses . . ., the number of Principles that must be applied . . . [or] identified - . . (59:49). . . . An alternative to indicating ErUUmber is to indicate percentage or proportion (59:50) ' . . [or an] acceptable deviation from some standard ' . ." (59:51). Having written an objective which indicates terminal behavior, conditions, and criteria, Mager s"31Ig’c3ests that clarity and completeness can be tested 100 by answering the following questions. Positive answers insure that each objective is well written and possesses the components which Mager believes are important. 1. 2. contain 1. 2. Does the statement describe what the learner will be doing when he is demonstrating that he has reached the objective? Does the statement describe the important con- ditions (givens or restrictions, or both) under which the learner will be expected to demonstrate his competence? Does the statement indicate how the learner will be evaluated? Does it describe at least the lower limit of acceptable performance? (59:52) Kibler et al. indicate that objectives should "five elements": Who is to perform the desired behavior (e.g., 1"the student” or "the learner") The actual behavior to be employed in demon- strating mastery Of the objective (e.g., "to write," or "to speak") The result (i.e., the product or performance) of the Behavior, which will be evaluated to determine whether the objective is mastered (e.g., ”an essay," or "the speech") The relevant conditions under which the behavior is to be performedi(e.g., "in a one-hour quiz," or "in front of the class") The standard which will be used to evaluate the success of the product or performance (e.g., "90 percent correct," or "four out of five correct"). (45:33) To illustrate how each of the above components is accounted for in an actual objective, Kibler et al. present an example preceded by a note of explanation: Each of these components is identified in the objec- tive shown below. The number of each component is identified above the appropriate portion of the planning objectiVe. Those words and phrases that are not a part of a specific component have no identification appearing above them. 101 4 /During the one hour mid-term examination,//the sthdent//will be able//to spellzcorrectly//45 out of 50/éwords//randomly selected from the 200 words listed i? Units onezand two of the Spelling book. //The studenE//will write//the words//as they are presented orally by the teacheg.//In order to be correct the spelling of each word must match the spelling in the spelling book/. (45:34) Elaborating upon each component, Kibler et al. note that determining the first component is easy: "You usually want the student to demonstrate the behavior" (45:32). The second component is that portion of the objective which specifically states the observable act (or behavior that the student is expected to perform in (order to demonstrate mastery of the objective and, there- :Eore, achievement of the desired outcomes of the instruc- tional program. Kibler et al. stress that "By behavior, we mean aetions and movements which people can be observed (seen, heard, or felt) making" (45:32). Since ". . . objectives mllst identify the action a person must perform, all behavioral objectives require a psychomotor component" (45:32) . Kibler et al. endorse the use of ". . . 'hard,‘ 'Cllear,' action verbs to classify the behavior to be performed" (45:34) . They cite a number of action verbs and their definitions. The list was originally developed by Henry H. Walbesser and was taken from Constructing EEEbavior Objectives. 102 Identifying. The individual selects (by point— ihg, touching, or picking up) the correct object of a class name. For example: Upon being asked, "Which animal is the frog?" when presented with a set of small animals, the child is expected to reSpond by picking up or clearly pointing to or touching the frog; if the child is asked to "pick up the red triangle" when presented with a set of paper cutouts representing different shapes, he is expected to pick up the red tri- angle. This class of performances also includes identifying object properties (such as rough, smooth, straight, curved) and, in addition, kinds of changes such as an increase or decrease in size. Distinguishing. Identifying objectives or events- which'are potentially confusable (square, rec- tangle), or when two contrasting identifications (such as right, left) are involved. Constructin . Generating a construction or draw- ing which igentifies a designated object or set of conditions. Example: Beginning with a line segment, the request is made, "Complete this figure so that it represents a triangle." Namin . Supplying the correct name (orally or in written form, for a class of objects or events. Example: "What is this three-dimensional object called?" Response: "A cone." Ordering. Arranging two or more objects or events in proper order in accordance with a stated category. For example: "Arrange these moving objects in order of their speeds." Describing. Generating and naming all of the necessary categories of objects, object proper- ties, or event prOperties that are relevant to the description of a designated situation. Example: "Describe this object," and the observer does not limit the categories which may be generated by mentioning them, as in the question "Describe the color and shape of this object." The child's description is considered sufficiently complete when there is a probability that any other individual is able to use the description to identify the object or event. Stating a Rule. Makes a verbal statement (not necessarily’in technical terms) which conveys a rule or principle, including the names of the proper classes of objects or events in their correct order. Example: "What is the test for determining whether this surface is flat?" The 103 acceptable response requires the mention of the application of a straightedge, in various directions, to determine touching all along the edge for each position. 8. Applying a Rule. Using a learned principle or rule to derive an answer to a question. The answer may be correct identification, the supply- ing of a name, or some other kind of response. The question is stated in such a way that the individual must employ a rational process to arrive at the answer. Such a process may be simple, as "Property A is true, prOperty B is true, therefore property C must be true." 9. Demonstrating. Performing the operations neces- sary to the application of a rule or principle. Example: "Show how you would tell whether this surface is flat." The answer requires that the individual use a straightedge to determine touch- ing of the edge to the surface at all points, and in various directions. 10. Inter retin . The child should be able to identify ohjects and/or events in terms of their consequences. There will be a set of rules or principles always connected with this behavior. (45:35) The third component is designed to identify ". . . the product, the performance, [or] the 'what' the Student is to do . . . [and is] the result of the behavior” (45:36) . The authors note: "To determine time result you can expect from students in an educational Setting, you must first decide what you wait them to do as a result of instruction and then write planning objec- ti‘Ves which identify what behavior is to be performed" ( 4 5:37) . Kibler et al. offer suggestions on how to deter- mine "THE RELEVANT CONDITIONS UNDER WHICH THE BEHAVIOR IS TO BE PERFORMED" (45:37) in order to accomplish the ftDurth component of a well-written objective. They suggest: 104 l. Specify the information, tools, equipment, source materials and anything else which will be available to the student to help him perform the terminal behavior required in the objective. 2. Specify the information, tools, equipment, source materials and anything else which the student cannot use when demonstrating the ter- minal behavior. 3. List as many of the actual Conditions as possible under which the student must be expected to demonstrate the terminal behavior in a real-life setting, and try to include as many of them in the objective as possible. (45:37) Having suggested ways to state conditions under which the terminal behavior is to be performed, the authors discuss the fifth component of a well-written objective. Component five states ". . . how effectively a student must perform to demonstrate adequate mastery of a prescribed behavior" (45:38). The standard of or criterion for performance stated within the objective provides a basis for evaluating the successful accomplish- ment of the objective. Kibler et al. suggest four "classes of performance standards" which can be incorporated into an objective to be sure that mastery of the desired terminal behavior can be assessed. The four classes and examples of each are listed below: Minimum Number . . . must list four steps. . . ." . . . write all ten —words presented accurately. ". . . distinguish three main ideas. . . ." Percent or Proportion “I . write (spell) accurately lOijercent of the 10 words presented. . . . . . list 80 percent of the verbs appearing in a 200 word—message. . . ." 105 Limitation of Departure from Fixed Standard 7—H. . . must—be correct to the nearest percent. ". . . must be within five decibels of. . . . Distinguishing Features of Successful Performance W. . . the radio plays within a one-day period. ". . . all balls on the paper are colored red. . . ." (45:39) In addition to determining how satisfactorily a person must perform to successfully demonstrate mastery of the desired terminal behavior, the authors state that the standards component of the objective should also describe ". . . how accuracy will be determined (e.g., how accurate responses can be defined operationally . . .)" (45:41). For example, "Accuracy will be determined by matching a student's responses on the written list to the correctly spelled words in the textbook (pp. x and y)" (45:41). Kemp says, "Each objective must be an unambiguous statement. It must mean exactly the same thing . . . to all students who will use it" (44:23). Kemp believes that written objectives should consist of three "essential parts." He explains that the instructional program designer should: 1. Start with an action verb that describes a specific behaviSr or activity by the learner. 2. Follow the action verb with the content reference that describes the subject being treated. . . . 3. End with the performance standard that indicates the minimum acceptable accomplishment in measurable terms. . . . (44:23) 106 Kemp explains that some writers add a fourth part which includes the ". . . criteria and conditions under which the learning must take place . . ." (44:24). Kaufman notes that "Mission objectives are per- formance objectives that specify outcomes in measurable terms. They require the same degree of specificity as any other performance or behavior objectives . . ." (43:54). Therefore, from his perspective, an objective must be written to answer the following questions: 1. What is to be done to demonstrate completion? 2. By whom it is to be done; that is, who will display the outcome(s)? 3. Under what conditions is the outcome to be demonstrated? 4. What criteria will be used to determine if the outcome has been achieved? (43:54) Performance requirements is the term used by Kaufman to label the criteria referred to in point four. They are designed to provide the specification by which the accomplishment, or lack thereof, is assessed, and they include: 1. Specifications stating the criteria by which the terminal success of the mission objective may be measured--what the product will look like or actually do. 2. Specifications stating the context or "ground rules" under which the product is to be pro- duced, such as environment, costs, personnel, or other "givens." (43:56) Payne lists a number of "desirable characteristics" which he believes objectives should possess: A. Objectives should be stated in the form of expected pupil changes. . . . If we are to validly measure and evaluate, we must know 107 precisely what it is we expect students to be able to do at the end of a course or unit of instruction. B. Objectives should be stated in behavioral terms. Words should be used which have the same meaning for student and instructor. . . . In general, we are concerned with a broad class of words called "action verbs." Objectives must be stated operationally if we are to adequately evaluate them. C. Objectives should be stated singly. Compound objectives, either in terms of content or expected behavior, are likely to lead to incon- sistent measurement. . . . D. Objectives should be parsimonious. Obviously statements of instructional goals are easier to work with when trimmed of excess verbage. E. The objectives should be grouped logically, so they make sense in determining units of instruction and evaluation. . . . F. The conditions under which the expected pupil behavior will be observed should be specified. G. If possible, the objective should contain a statement indicating the criteria of acceptable performance which indicate that the objective has been met. Criteria might be involved with time limits or a minimum number of correct responses. (71:22) Davis et al. maintain that "A learning objective consists of three components: (1) terminal behavior; (2) test conditions; and (3) standards" (21:33). They describe terminal behavior as: . . . the intended outcome of instruction. It describes what the student will be able to do in order to demonstrate that he has achieved the objective. It is the behavior that will be accepted as evidence that the student has learned. (21:33) Davis et al. stress the importance of using "action verbs." These include words like ”discriminate . . ., Choose (or select), assemble, adjust, identify . . ., Solve, apply, align, [and] list . . ." (21:35) and do 108 not include ambiguous, nonbehavioral words like "know, understand . . ., determine, appreciate . . ., grasp . . ., [and] become familiar with" (21:35). When writing the terminal behavior component of an objective, an instructor should answer questions such as: o What do I expect a student to be able to do? 0 In what way should the student demonstrate that he has learned? 0 What student performance will I accept as evidence that he has learned? (21:35) Discussing the conditions component, the authors state: The conditions component of a learning objective describes the situation in which the student will be required to demonstrate the terminal behavior. It is the component that describes the test con- ditions. (21:37) Davis et al. cite three types of conditions that affect performance. They are ". . . the aids or tools that the student will be permitted to use . . ., the kinds of restrictions that will be placed on a student . . ., [and] how information will be presented . . ." (21:37) by the student during performance periods. Examining the standards component, the authors state: The third component of a learning objective is a statement of the standards by which the learner's performance will be judged. A standard describes the minimal level of performance that will be accepted as evidence that the learner has achieved the objective. (21:38) The authors explain that there may be times when a teacher feels that a student's performance of a specific 109 terminal behavior may be due to chance. If so, it is necessary to ". . . decide what proportion of successes will convince him otherwise" (21:41). In these cases, the objective should include a statement of ”. . . the number of opportunities the student will be given on the test and the number of times he must succeed" (21:41). Davis et al. refer to this sub-section of the standards component as the statement of performance stability or stability criterion. The authors explain that: . . . the standards component describes how well the student must perform the terminal behavior. The stability criterion states how many times he must perform the terminal behavior at or above the standard. (21:41) Davis et al. carefully examine another factor which should be incorporated in the process of developing objectives that are designed to achieve desired outcomes. They point out that the ". . . knowledge and skills stu- dents learn will be useful to them at some later time, that is, in some situation external to the course” (21:56). The location or situation in which the terminal behaviors mastered in the course will be useful is known as the referent situation. If the designer of the instructional program fails to identify the referent situation prior to the formulation of each objective, he/she runs the .risk that the terminal behavior achieved will not be as useful to the student as anticipated. Additionally, the 110 designer ". . . runs the risk of students not perceiving the relevance of what he is trying to teach" (21:56). To be sure these situations are avoided, as a designer begins the process of writing specific objec- tives, he/she should ask the following questions: Where will the student use the behavior mastered through the successful accomplishment of these objectives? What behavior will be required of the student in the referent situation? What are the conditions and standards for performance in the referent situation? Davis et al. suggest that to gain insight into the actual referent situation and to be sure that the objectives which are developed are relevant to it, the designer may want to consult with individuals who have practical experience within it. Keeping the referent situation in mind, the designer should: 0 Write a learning objective that approximates the referent situation as closely as possible. . . . 0 Include in the learning objective, all aids and restrictions that influence performance of the task in the referent situation. . . . 0 Write the lower limit of performance using the same type of standard (time, error, etc.) as the referent situation. . . . (21:72) Additionally, o The performance stability limit should be the same ratio of success to opportunities as the student is likely to have in the referent situation. (21:72) 0 The instructor should write objectives that deviate as little as possible from the referent situation. . . . 111 o If classroom constraints require deviations, the instructor should select conditions, behaviors, and standards for the objectives that result in maximum transfer to the referent situation. (21:63) Similarly, Goldstein discusses the importance of designing instructional programs to be sure that, upon their completion, students will be prepared to use what they have learned at a future time in another setting. Using the term "transfer setting" where Davis et al. use "referent situation,” he points out that, properly written, "The objectives provide direct input for the design of the training program and help specify the criterion measures that will be used to evaluate the performance of the trainee at the end of the training program and in the transfer setting (on the job, in the next program, and so on)" (36:45). Goldstein points out that the instructional pro- gram can be designed to have positive impact upon par- ticipant performance in the appropriate transfer setting by increasing the degree of similarity between the "tasks" completed in the instructional setting and those which will be required in the transfer setting and/or by insuring participant mastery of key principles which the student can draw upon when accomplishing tasks in the transfer setting. He notes that: . . . if the tasks are identical in training and transfer, trainees are simply practicing the final task during the training program and there should be high positive transfer. (36:108) 112 . . . the principles theory suggests that train- ing should focus on the general principles necessary to learn a task so that the learner may apply them to solve problems in the transfer task. (36:109) Regardless of whether circumstances call for instructional programs which rely on identical tasks or principles to insure the desired terminal behavior, the specific objectives developed must be designed in terms of desired performance in that setting if the instruc- tional program is to be successful. The objectives must be designed to insure that there is a relationship between the instructional and transfer settings and that the learning which occurs within the instructional setting and the behaviors which result from the instruc- tional program are those desirable for use in-the transfer setting. Similarly, in "The Use of Objectives in Instruc- tion,” Ammerman and Melching discuss the importance of designing objectives in light of the intended "work situation." They state: . . . the work performance situations of interest must be identified. The purpose of this is to place appropriate constraints upon instruction, limiting and defining what is to be considered relevant. As used here, the term ”work situation" refers to the performance situation for which the student is to be prepared, and in which he will be expected to perform effectively after instruction. (1:74) Ammerman and Melching note that a variety of points might have to be considered in designing objectives which, if mastered, will prepare individuals to perform 113 in a particular "work situation." These include ". . . unusual environmental conditions . . . as well as perti— nentgeographical or cultural conditions . . ., organi- zational or administrative conditions . . ., the level of responsibility and the degree of autonomy permitted . . ., [and] particular types of equipment . . . involved" (1:75). The performance and standards must also be con- sidered. The authors explain: The definition of the performance situation is a very critical step in the derivation of valid objectives. It is the definition that establishes the basis for identifying the important performance conditions and standards. Additionally, it identifies and limits the scope of the performance situation for which objectives must be determined. (1:75) Terminal and Enabling Objectives Are Needtd Developing objectives which incorporate the com- ponents discussed above will insure that the terminal behavior which the student should be able to perform upon completion of an instructional program has been clearly stated. In many instances, this terminal behavior is very complex and a variety of intermediate skills or abilities must be mastered before the terminal behavior can be performed properly. For example, flying an airplane may be the desired terminal behavior. To properly perform this behavior, such intermediate skills or ability as starting the plane, instrument reading, take-off, and landing must be mastered. When the 114 terminal behavior is complex, it is necessary to complete an analysis of the terminal behavior to identify and to insure that the instructional program is designed to teach all of the skills and abilities that are necessary to achieve its mastery. Having identified the desired terminal behavior which must be mastered and demonstrated at the conclusion of the instructional program, Goldstein observes that a task analysis should be undertaken. He states: . . . the task analysis consists of several compo- nents, each of which further delineates the per- formance required to succeed at the task. Thus, the analysis begins with a task description, followed by a detailed specification of behaviors necessary to perform each task. (36:31) He explains that the completion of a task description will insure that the ". . . activities per- formed on the job and the conditions under which the job is performed" (36:34) have all been properly identified. The specification of behaviors which are needed to per- form the desired task (terminal behavior) is completed to insure that the ". . . actual steps necessary to perform each important task" (36:36) are clear. Simi- larly, Davis et al. discuss the value of task description as a useful tool for ". . . describing how a task is performed" (21:130). To assist in gathering information to complete the task analysis, Goldstein points out that "Previous task analysis and documentary materials provide useful 115 introductions to tasks being investigated" (36:42). He suggests that information can also be gathered through questionnaires or interviews given to individuals who are familiar with the tasks, direct observation of the tasks being completed, and/or a conference of experts who can provide clarification of the behaviors a person needs to master and the sequence in which they must be mastered in order to perform the task. Davis et al. note that information about the task can be obtained by con- sulting practitioners and/or expert sources, individual and group interviews, direct observation, and review of technical manuals. The realization that, to insure effective instructional programs, complex terminal behaviors must be analyzed to identify the intermediate skills and abilities which must be mastered before it can be per- formed and that objectives possessing the components presented above must be written for each, has led some authorities to distinguish between "terminal" and "enabling" objectives. Kemp explains: . . . objectives can be specified on two levels. The first level delineates the terminal objectives, which state what the student will do in terms of a continuum of experiences during his study of the topic. For example, under the topic of weather maps, a terminal objective may be "to interpret a weat er pattern shown on a weather map.” When we treat points or concepts along this continuum with more precisely stated objectives, we have the second level, called interim or enablin objectives. Each of these objectives represents a Single activity or learning step. To specify the terminal 116 objective for weather maps, we must indicate that the student will learn "to relate symbols used on a map to actual weather conditions," "to recognize air mass types and their characteristics as inferred on a map,” and so forth. (44:21) Ammerman and Melching describe the distinction as follows: Terminal objectives . . . represent the perfor- mance that is to be attained through instruction. . . . They establish meaningful and measurable goals for the instruction, upon which all other aspects of the program must be based. (1:77) [Enabling objectives] . . ., in general, con- sist of the component actions, knowledges, skills, and so forth, the student must learn if he is to attain the terminal objectives. These bridge the gap between where the student is at the beginning of instruction and where he should be upon com- pletion of instruction. (1:75) Similarly, Davis et al. state: . . . the objectives for a given instructional unit are called terminal objectives; the objectives for the subunits of instructiOn are called enabling ob'ectives. Enhhling objectives may be thought of as pre- requisites for terminal objectives. Enabling objectives describe the knowledge or skills that contribute to or facilitate achieving the terminal objective. . . . . . . the unit consists of several component skills which must be taught before the terminal objective can be achieved. An objective can be stated for each one and, as a group, they comprise the enabling objectives. (21:43) Accomplishment of the terminal objective of the instructional program reflects mastery of the terminal behavior itself. Accomplishment of the enabling objec- tives insure mastery of the intermediate skills or abili- ties necessary to effectively perform the terminal behavior. Consider the observation of Ammerman and Melching: 117 . . . when statements of the desired outcome of instruction refer to actions that occur on the job, and such actions have job value in and of themselves, then these statements should properly fall into the class of terminal objectives. And, in contrast, statements that refer to actions which serve only to facilitate or assist the student's attainment of the desired job performance should be classed as enabling objectives. (1:76) Davis et al. explain their view of the process which is likely to occur as the instructional program designer seeks to insure that the necessary well-written terminal and enabling objectives have been developed. They explain: As a general rule, it seems to us that the process resembles a problem-solving activity. The designer of a learning system begins with some sort of problem. He tries to define as carefully as he can what the problem is and, in the process, gathers as much information as possible about it. Slowly he hammers out a terminal objective which, on inspec- tion, reveals specific tasks. He then sets out to describe these tasks. From the task description, new enabling objectives emerge which may, in turn, suggest new tasks. Furthermore, in the process of doing these various things, the designer may have rewritten the original terminal objective and described the initial task several times. In other words, the process is iterative, by which we mean the designer goes back over his work and recreates the steps as new dimensions and aspects emerge. (21:152) Having developed a well-written terminal objective and having identified and written the necessary enabling objectives to insure that intermediate skills necessary to perform the desired terminal behavior will be mastered, the instructional program designer is prepared to select instructional procedures. 118 Select Instructional Procedures With needs assessed and objectives written, the designer is prepared to select the procedures which will be used in the instructional program. To complete this element of the instructional program, the designer identifies and selects those procedures which will establish the conditions within the instructional environment that must exist if the student is to learn the skills necessary to perform in accordance with the behavioral expectations stated in each objective. The element is discussed in this section and is labeled "Select Instructional Procedures." In order to examine what the literature reveals concerning the selection of these procedures, the views of selected authors about (1) variables and general principles, (2) types of learning and conditions of instruction, (3) transfer of learning, and (4) selection of methods and materials are considered here. Variables and General Principles In selecting procedures which will guide student learning and result in the student's achievement of established objectives, it is important to resist the temptation to incorporate a particular procedure in the instructional program simply because it is available, is new and unusual, or is familiar. Unless the designer determines that a particular procedure will make a 119 positive contribution to the student's attempts to learn the skills necessary to perform as required, it should not be used. To effectively contribute to the development of those conditions within the learning environment which positively impact the achievement of desired outcomes, the selection of procedures to be implemented should be made in response to the types of variables encountered in each instructional setting and the type of performance stated in the objective. Expressing a theme which will be reemphasized by many authors throughout this section, Mager and Beach note that, as a designer reviews the objectives which must be achieved, "There are several different kinds of performance, and different procedures and materials are appropriate for teaching each" (60:45). Siegel and Siegel contend that, in order to determine the most effective procedures for implementation in a particular instructional environment, the designer should consider variables ". . . related to learning environments, instructors, learners, and courses" (84: 265). They explain: The learning environment is defined by the . physical setting and characteristics of the class- room or other instructional site and by certain events transpiring in the physical environment. (84:265) , Some of the specific variables entering into the composition of the environment include (1) class size; (2) physical characteristics of the classroom; (3) the physical presence or absence of an "authority figure" maintaining discipline, 120 taking attendance, and the like; (4) the methods by and extent to which audiovisual devices of various kinds are utilized; and (5) the extent and level of participation by students in class activities. Whereas learning environments describe the physical setting and structure provided for the course, the instructor variables describe the unique contribution to a given learning environment by the teacher. The teacher's operations that have been selected as particularly pertinent to the general conceptual scheme for the instructional gestalt are . . . l. The instructional objective manifested to his students—by his behavior in class andtby his examini Tprocedures . . . e amount and quality of personal contact between teather and students . . 3. Effie intellectual climate developed by the instructor. . . . (84 :266) The students exposed to any combination of classroom environment and instructor variables are heterogeneous with respect to a large number of learner variables. The ones selected as particu- larly pertinent to the paradigm are the following: 1. A constellation of characteristics variously designated as intelligence, academic ability, scholastic aptitude, and the like. 2. Knowledge about the subject matter prior to enrollment in the course. . . . 3. Motivation with respect to the specific course content. . . . 4. The students' set toward education. The extreme poles on the continuum of set may involve a predisposition, on the one hand, to accumulate isolated or specific facts and, on the other, to attempt generalization by learning fact clusters, developing concepts, and discovering principles. 5. Creativity in organizing his perceptual field and in solving problems. , Certain kinds of courses lend themselves more readily than others to particular kinds of structures (learning environments) and instructor behaviors. . . . Hence, at least three features of the course are important to the paradigm: (l) the subject- matter area, (2) the level of presentation (elemen- tary or advanced), (3) whether the course is required or elected by the students. (84:268) As the designer considers the impact of these variables upon a particular instructional program, Siegel 121 and Siegel indicate the importance of recognizing that interactions occur among the variables. The authors believe that interactions must be carefully studied and that instructional procedures should be selected which will result in a combination of variables which will have a positive impact upon the learning which occurs. They explain: As a generalization, the effects of various kinds of instruction within a given course can be con- ceptualized and empirically studied in relation to variations in learning environments, learner characteristics, and the relevant activities of the instructor. The burden of investigation pro- ceeding from this view is to discover combinations of learner, instructor, environmental, and course variables optimizing desired educational outcomes. (84:269) A number of authors believe that the probability of selecting instructional procedures which will have a positive impact upon the achievement of desired outcomes will be improved if the designer carefully considers and applies, as appropriate, general principles which they believe are relevant to the proper design of instructional programs. While noting that they all will ". . . not apply automatically to all students and all subject areas" (44:7), Kemp cites ten principles which should be considered as instructional procedures are selected. They are: l. Pre-Learning Preparation. Learners must master the prerequisite behaviors to succeed in new learning experiences. 122 2. Motivation. Students are more efficient if they want to learn what is being taught. . . . 3. Providing a Model of Terminal Performance (Mastery). When possible, learners shofild be SHOWn examples of what they are to produce or to do at the end of a learning experience. . . . 4. Active Respondigg. At the outset of training, learners can prOfit from watching or listening to someone else perform the acts to be learned, but most learners will become proficient only if they perform the acts themselves. . . . 5. Guidance. When attempting to demonstrate new Behaviors to be learned, instructors should guide and prompt the students. . . . Such prompts should be eliminated gradually until the learner is able to perform the task with- out them. . . . 6. Practice. Opportunities should be provided for learners to use newly learned behaviors repeatedly. Practice will be effective to the extent that the behaviors practiced are similar to behaviors to be performed in the future (the terminal objective). . . . 7. Knowledge of Results. A learner should have prompt and frequent—knowledge of the success of his responses. He must find his success rewarding in order for the behavior to be rein- forced. . . . 8. Graduated Sequence. Subject matter should be organized in a hierarchical form from the simple to the complex--from the familiar to the unfamiliar. The steps should be paced so that the learner succeeds in each step but does not become bored. . . . 9. Individual Differences. PeOple learn at dif- ferent speeds; therefhre, learning experiences should be designed so that each student may proceed at his own pace. . . . 10. Classroom Teaching Performance. Skills in stimu- lating interest, explaining, guiding, identify- ing and administering reinforcers, and managing classroom behavior can make an enormous dif- ference in instructional effectiveness. . . . (44:8) Bass and Vaughan offer seven principles and maintain that a particular instructional procedure Should be ". . . judged adequate . . ." (5:86) if it accounts for them. The seven principles are: 123 1. Provide for the learner's active participation. 2. Provide the trainee with knowledge of results about his attempts to improve. 3. Promote by means of good organization a meaning- ful integration of learning experiences that the trainee can transfer from training to the job. 4. Provide some means for the trainee to be rein- forced for appropriate behavior. . Provide for practice and repetition when needed. 6. Motivate the trainee to improve his own per- formance. 7. Assist the trainee in his willingness to change. (5:86) Popham and Baker suggest principles which they believe should be considered by the instructional program designer. They maintain that, "By incorporating these principles in the plan of instruction, . . . [the designer] is more likely to bring about the desired terminal behaviors" (73:77). They present the follow- ing five principles: . . . the first principle to follow in setting up an instructional sequence is that the teacher should communicate to the learners the nature of the behavior changes he has in mind for them. . . . Objectives should be communicated to the learners in language that they will understand. (73:78) A principle strongly related to the act of revealing objectives is that of perceived purpose. According to this principle, the learner should he shown the relevance of what he is studying. In the first place, an effective description of what it is the learner is supposed to accomplish is most helpful. If, in addition, the learner can be shown ghy these objectives are worthwhile, it is far more likely that he will achieve the desired goals. (73:80) Of the many principles from the field of instructional psychology that a teacher can utilize, perhaps the most potent is that of eppropriate practice. According to this principle, the teacher must provide opportunities during an instructional sequence for the learner to behave in a fashion consistent with the instructional objectives. (73:82) 124 Closely related to the principle of appropriate practice is the principle of knowledge of results. According to this principle, provisions should’be made to enable the learner to determine the adequacy of his responses shortly after he makes them. (73:85) Another principle of great utility in planning instructional sequences is to differentiate instruc- tion for the learners. There are at least two methods of differentiating instruction. One of these is to modify the objectives for different students so that particular learners are given dif- ferent objectives. . . . A second scheme for dif- ferentiating instruction involves the use of dif- ferent means to accomplish identical instructional ends. (73:87) Davis et al. present principles which they believe ". . . can be applied in any learning system regardless of the age of the learner, the subject matter, or the type of learning” (21:198). If the procedures implemented during the instructional program are selected in response to the following principles, the probability that the student will develop the skills necessary to perform in accordance with the criteria presented in a particular objective will be increased. The nine principles presented by Davis et al. are: PRINCIPLE l--MEANINGFULNESS: A student is likely to be motivated to learn things that are meaningful to him. (21:198) PRINCIPLE 2--PREREQUISITES: A student is likely to learn something new if he has all the prerequisites. (21:201) PRINCIPLE 3--MODELING: The student is more likely to acquire new behavior if he is presented with a model performance to watch and imitate. (21:202) PRINCIPLE 4--OPEN COMMUNICATION: The student is more likely to learn if the presen- tation is structured so that the instructor's messages are open to the students' inspection. (21:203) 125 PRINCIPLE 5--NOVELTY: A student is more likely to learn if his attention is attracted by relatively novel presentations. (21:204) PRINCIPLE 6--ACTIVE APPROPRIATE PRACTICE: The student is more likely to learn if he takes an active part in practice geared to reach an instruc- tional objective. (21:205) PRINCIPLE 7--DISTRIBUTED PRACTICE: A student is more likely to learn if his practice is scheduled in short periods distributed over time. (21:207) PRINCIPLE 8--FADING: A student is more likely to learn if instructional prompts are withdrawn gradually. (21:207) PRINCIPLE 9--PLEASANT CONDITIONS AND CONSEQUENCES: A student is more likely to continue learning if instructional conditions are made pleasant. (21:208) Gagné presents a ". . . set of instructional events . . ." (29:303) which reflect principles such as those presented above and which he believes should be incorporated into the instructional process in order to insure that students learn to perform as desired. He describes the role of these events in the instructional process and then lists them: Instruction may be seen to comprise a set of separate events, each of which has a distinct effect upon the learner. They engage his attention, they provide him with information and feedback, they present the essential stimulus for learning, they stimulate his recall, they insure that he gets practice in what he has learned. As a total set of events, they usually begin a few minutes before the time of actual learning and come to an end some time afterward. Their general function is to insure that the timing and sequencing of events internal to the learner is proper for the occurrence of learning, and also for retention and transfera- bility of what is learned. The specific functions of these different events that are components of instruction may be described briefly as follows: 126 l. Gaining and controlling attention. An external stimulus arouses the appropriate attentional set. 2. Informing the learner of expected outcomes. Communication, usually verbal, tells the learner about the kind of performance he will be able to do after he has learned. 3. Stimulating recall of relevant prerequisite capabilities. The learner is reminded of the rele- vant inte1IECtual skills, and also verbal knowledge, he has previously learned. 4. Presentingthe stimuli inherent to the learning task. The particular stimuli to Which the newly learned performance will be directed are displayed. 5. Offering guidance for learning. Usually by verbal communicatiOns the'learner‘s thinking is directed by prompts or hints until the essential performance is achieved. 6. Providing feedback. The learner is informed of the correctness of his newly attained performance. 7. Appraising performance. Opportunity is pro- vided for theilearner to verify his achievement in one or more situations. 8. Making provisions for transferability. Additional examples are used to establish increased generalization of the newly acquired capability. 9. Insuring retention. Provisions are also made for practice and use of the new capability so that it will be remembered. (29:304) Types of Learning and Con- ditions 0 Instruction Many instructional program designers would agree that (l) a careful alignment of the variables identified by Siegel and Siegel (84:265) and (2) an application of the various principles which have been presented here would result in the selection and implementation of those procedures within the instructional process necessary to insure that the student would be able to perform as expected at the conclusion of the program. 127 Gagné states that many believe the processes involved in bringing about human change are the same regardless of what is being learned: . . . there has been a guiding assumption that the nature of the change called learning must be in some fundamental sense the same, regardless of what is being learned. Accordingly, for a great many years, theories about the optimal conditions for learning have been dominated by concern with the variables of contiguity, reinforcement, and.fre- uenc . Investigators—have searCHed for certain general laws relating these obviously important variables to learning outcomes, independently of "what is being learned," that is, of the nature of the change in capability being studied. (31:296) However, while a designer should realize the value of applying the general principles presented above, he/she should not apply these principles and determine instructional procedures until he/she has carefully analyzed each objective which has been identified for inclusion in the instructional program to determine the specific type(s) of learning which must occur to insure mastery of the objective. Many authors believe that this will insure the selection and implementation of the most effective instructional procedures--those which will provide the greatest probability that desired learning will occur as a result of the student's par- ticipation in an instructional program. Once the type(s) of learning has been identified, the designer can deter- mine those specific conditions of instruction which will optimize the probability that required learning 128 will occur and select procedures which will insure that these conditions are accounted for within the instruc- tional program. According to these authors, the procedures most likely to create those conditions of instruction which will facilitate desired learning vary depending upon the type of learning which must occur in order to insure mastery of the objective. lGoldstein explains: A productive training environment is created by careful examination of the training objectives to determine the type of learning necessary for acquiring essential behaviors. . . . [The designer] . . . analyzes his objectives and determines the required behaviors. Then, the behavior is matched to the most appropriate learning environment and instructional media. Learning environment refers to the dynamics of the instructional setting, with particular emphasis on learning variables--for example, knowledge of results or massed and spaced practice. Instructional media refers to particular devices and techniques, like simulators, programmed instruction, films and lectures. . . . [It] . . . is important for the learning environment and instructional media to be determined by the objec- tives and the form of performance required. However, inappropriate techniques are often used because they are readily available. Unfor- tunately, the design of the learning environment and the selection of the appropriate instructional variables have not been treated with the same degree of awareness. Often a training designer insists that knowledge of results or feedback is necessary, without first determining what kinds of behavior are desired and whether feedback is appropriate for learning those particular behaviors. The approach emphasized . . . [by Goldstein] stresses the determination of objectives through need assessment and the analysis of those objectives to determine the behaviors required. After that has been accomplished, the proper learning environ- ment, with appropriate learning variables, media, and techniques can be selected. (36:91) 129 Gagné notes: The major implication of this approach to the problem of instruction is surely that the instructor cannot be guided by a simple set of rules that apply to all cases. The important aspects of the instructor's behavior do not lie in the fact that he uses a general principle to control learning (such as rein- orcement or contiguity), but rather in the fact that he employs different techniques for different kinds of learning. (3l:304l' Rather than treat all learning processes the same regardless of what is being learned, Gagné strongly supports an affirmative answer to the question, "Is it in fact possible to divide objectives into categories which differ in their implications for learning?” (10:38). From Gagné's point of view, The form taken by instruction needs to be tailored to the particular objective which represents the kind of performance change to be brought about. Instruction is used to establish the necessary conditions for learning, and instruction differs in accordance with what is to be learned. (31:304) Similarly, Jahnke notes that: . . . to be effective, formal instruction may require a preliminary analysis of the behaviors required by the task. Such systematic analysis may disclose that whereas one task requires that responses already in the learner's repertoire come under the control of new stimuli, another requires that a skilled response to the correct cue be made at the correct time. Thus, different educational goals may require different instructional tech- niques. Instruction which is primarily verbal and does not require an overt response from the student may suffice for the attainment of certain educational objectives. On the other hand, verbal instruction accompanied by active, overt responding in a variety of training situations may be necessary for the attainment of other objectives. . . . (41:203) 130 Glaser offers a similar perspective. He explains that: From the point of view of instruction, component repertoire analysis identifies the kind of behavior involved so that the learner can be provided with instructional conditions optimal for that kind of behavior. The underlying assumption here is that the learning of various kinds of component reper- toire requires different kinds of teaching pro- cedures. The important research task is to identify the learning process and appropriate instructional procedures associated with different component repertoires. (34:435) In another statement, Glaser notes: One ramification, then, of the analysis of behavior upon instructional design is the necessity to dis- tinguish between subject matter content and component repertoires. The designing of optimal instruction may be a matter of choosing the proper tactics for categories of behavior implied by the component repertoire characteristics of instructional objec- tives. (35:775) The discussion to follow includes (1) identifi- cation of the various types of learning which may be involved as a student seeks mastery of a particular objective and (2) an explanation of the need to select particular conditions of instruction to insure that the identified types of learning do occur. It is important to understand that the designer must, in part, determine his instructional procedures in light of the general principles or "events” discussed earlier. In addition, however, to insure that the instructional program will be most effective, the designer must also identify, through analysis of each objective, the particular types of learning which must occur and incorporate into his/her procedures, those specific conditions 131 of instruction necessary to insure that the desired type of learning takes place and mastery of the objective is achieved. Recognizing that the effective accomplishment of each learning objective requires, as Gagné notes, the use of ". . . different techniques for different kinds of learning" (31:304), the instructional program designer must analyze each objective and assign it to a category which represents a type of learning or behavior and which has particular conditions for instruction associated with it. These conditions must be provided within the instructional environment as procedures are selected and implemented if the student is to master the desired behavior stated in the objective. This process is referred to as task analysis. DeCecco explains: . . . once the teacher has made adequate statements of the instructional objectives (or task description), he must analyze these objectives by fitting them into various classes of behavior. The chief purpose of task analysis is to help the teacher determine the optimum learning conditions for the various tasks the student must learn to perform. (22:45) . . . in a proper task analysis the teacher would correctly classify the behavior involved in each objective and establish the best learning con- ditions. (22:46) Davis et al. offer a similar explanation: ". . . the task analysis involves a careful examination of the enabling objectives or task descriptions in an effort to identify those factors which will influence the final design of the learning system" (21:182). 132 Gagné's work has received wide recognition as a substantive examination of the task analysis process. Many authors (Banathy [4:50], Briggs [11:75], Davis et a1. [21:166], DeCecco [22:45], Glaser [34:435], Goldstein [36:131], Kemp [44:25], and Popham and Baker [73:65]) recognize the value of Gagné's work and paraphrase or quote extensively from his writings. This is not to say that methods of completing the task analysis process are completely developed or universally accepted. Gagné himself notes: . . . the technique of describing instructional objectives is fairly well agreed upon. But the next step, which is called task analysis, has neither been so fully developed nor so precisely specified. (28:34) Goldstein states that there is debate over Gagné‘s work. However, he recognizes its value: ". . . the system is important because it suggests a procedure that organizes behaviors into learning types and then relates the cate- gories to conditions for learning and instructional media" (36:134). However, due to its depth and the recognition it has received, Gagné's work will provide the basis for the following discussion of the task analysis pro- cess. Asserting that the performance required by each objective can be identified, Gagné's hypothesis is that: . . . the identification of these different kinds of performance, together with the different kinds of capabilities they imply, suggests that there may 1133 be at least as many different kinds of learning. And if this is so, it may be supposed that there exist an equal number of conditions of effective learning to correspond with each variety. A théory of instruction, then, cannot be maximally useful if it concerns itself with only those conditions that are general to all classes of learning. Instead, such a theory must concern itself in an individual manner with each of the types of learn- ing. (31:300) In another statement, Gagné notes that what is involved is: . . . the identification of classes of behavior which differ in respect to the conditions most effective for their learning. The optimal strategy for the attainment of a generalization, for example, is presumably not the same as the optimal strategy for the establishment of a multiple discrimination. (28:35) As noted earlier, the task analysis process begins with an assessment of each objective and its assignment to a distinct category representing a particular type of learning. Gagné has identified eight types of learning, each of which includes a separate and distinct behavior or performance which can be observed when learning within that category has occurred. The eight categories or types of learning are ordered with simple types of learning first and complex types last. Gagné presents a detailed and lengthy description of each type of learn- ing in his book, The Conditions of Learning (29). While the reader should consult this text to broaden his/her understanding of Gagné's work, DeCecco presents a brief but informative description of each type of learning and an example of each. His statement is presented here in its entirety: 134 SIGNAL LEARNING. In this type of learning (often also called classical conditioning) the animal or individual acquires a conditioned response to a given signal. Pavlov studied such learning in great detail. In it the responses are diffuse and emotional and the learning is involuntary. Examples are the withdrawal of the hand upon sight of a hot object, the salivation of a dog upon hearing food poured into his metal feeding dish, and the tearing of the eyes upon sight of an onion. The signals are the sight of the hot object, the sound of food being poured in the dish, and the sight of the onion. The con- ditioned responses are withdrawal of the hand, salivation, and tearing of the eyes. STIMULUS-RESPONSE LEARNING. In this kind of learning, exemplified by animal training, the animal makes precise movements of the skeletal muscles in response to specific stimuli. At first this training usually requires the use of a leash and a choke chain. As the dog learns par- ticular responses for particular jerks of the leash and chain, his master rewards him with pats and praise. Later the master does not have to use the leash and chain; the animal sits, stays, or lies down upon hearing the simple verbal command. Whereas the responses in signal learning are diffuse and emotional, the responses in stimulus-response learn- ing (often called operant conditioning) are fairly precise physical movements. CHAINING. In this type of learning (frequently called skill learnin ), the person links together two or more units 0 stimulus-response learning. Gagné reserves this category for nonverbal sequences. The chief condition for the learning of a chain is the reinstatement of the stimulus-response units in the proper order. In the elementary school, the child acquires many chains. He learns to button, fasten, tie, use a pencil, erase, and cut. Later he learns the chains of physical acts involved in printing and writing. He also learns a number of recreational skills, like catching, throwing, and kicking balls of various sizes and shapes. VERBAL ASSOCIATION. This learning is a type of chaining, but the links are verbal units. The simplest verbal association is the activity of naming an object, which involves a chain of two links: An observing response enables the child to identify properly the object he sees; and an internal stimulus enables the child to say the proper name. When the child can not only name an object "ball" but also say "the red ball," he has 135 learned a verbal association of three links. Gagné calls another common verbal association translation responses; in these, for example, the individual giVes the German or French equiva- lent of an English word, or one nonsense syllable or English word in response to another syllable or word. The learner frequently acquires verbal associations by verbal mediation--an internal link which helps him associate, for example, the French word and its English equivalent. If the student were learning hand in response to the French word main, the English word manual would provide verbal mediation. MULTIPLE DISCRIMINATION. In this type of learn- ing the student must learn different responses for stimuli which might be confused. The student learns to distinguish between motor and verbal chains he has already acquired. In studying French, for example, the student must associate faim with hun er and femme with woman. When American Boys undertafie to identify all the new models of automobiles pro- duced in this country in a particular year, they are engaging in multiple-discrimination learning. They must associate each individual model, with its distinctive appearance, with the correct model name and with no other name. When there is only one model to consider, linking the correct name with the right model illustrates verbal association. When there are several models and names, linking that name with the same model and no other model illustrates multiple-discrimination learning. Teachers, Gagné suggests, engage in multiple- discrimination learning when they devise means for calling each student by his correct name. CONCEPT LEARNING. In learning a concept we respond to stimuli in terms of abstract character- istics like color, shape, position, and number as opposed to concrete physical properties like specific wavelengths or particular intensities. Gagné uses this example. A child may learn to call a small cube a "block,” and also to call similar objects which vary in size and shape "blocks.” Later he may learn the concept cube and discover that cubes can be made of wood, lass, wire, and other materials, can vary in color, and can be of any size. Or, if a student is given a series of numbers, he is able to select those which belong to the class odd as opposed to those in the class even. In conEEEt learning, the student's behavior is not under the control of particular physical stimuli but of the abstract properties of each stimulus. Concepts have concrete references even though they are learned with the use of language. 136 PRINCIPLE LEARNING. In learning a principle we relate two or more concepts. According to Gagné, the simplest principle may be depicted in the form: "If g, then Y," as in the example, "If a feminine noun, then the-article 13.” Or, ”If the temperature of the water is above 212°F, then the water boils.” Principles are, in effect, chains of concepts. We may represent knowledge as a hierarchy of principles, in which we must learn two or more principles before learning a higher-order principle which embraces them. If the student has learned the component concepts and principles, the teacher can use verbal instruction alone in leading the student to put the principles together. PROBLEM SOLVING. In the set of events called problem solving, individuals use principles to achieve some goal. When the goal is reached, how- ever, the student has learned something more and is then capable of new performances using his new knowledge. What is learned, according to Gagné, is a higher-order principle, the combined product of two or more lower-order principles. Thus problem solving requires those internal events usually called thinking. Gagné suggests these examples. A driver who maps his route through traffic rather than being swept along by it is solving a problem. In replanning his luncheon schedule to accommodate a new appointment, the individual is solving a problem. When the housewife shops selectively for particular items on the basis of price variations, she is solving a problem. Without knowledge of the prerequisite principles, the problems cannot be solved. (22:47) Having written objectives in accordance with the guidelines discussed in the previous section of this chapter, the instructional program designer is in a position to assign each objective to one of the eight categories of learning identified here. The category to which each objective is assigned is determined by the type of performance stated in the objective as the expected outcome of the instructional process. Consider the following examples presented by DeCecco: 137 1. When given examples of various types of geometric figures, the student selects only (and all) those which are triangles. 2. When given a list of French words, the student gives all the correct English equivalents. (22:45) Each expected outcome represents a different type of behavior and, from Gagné's point of view, a different type of learning is involved. DeCecco explains that, "The first involves concept learning; the second, verbal learning” (22:45). Before discussing the conditions of instruction which must exist for each type of learning in order to insure that desired performance is achieved, it seems prudent to digress from the discussion of Gagné's work to be sure that the importance of well-written objectives and their valuable role in this element of the instruc- tional design process are clearly understood. As noted previously, various elements of the instructional program design process are interdependent. In this instance, if the designer has not written objec- tives carefully, he/she cannot properly complete the task analysis and should not proceed until they are develOped. Additionally, if well-written objectives have been developed, they should not be ignored. They should be carefully consulted and information gathered to determine enabling as well as terminal objectives should be utilized as the process continues. 138 As many authors have indicated, the critical component of a well-written objective is the statement, in specific behavioral terms, of the skill the student will be expected to demonstrate or perform upon completion of the instructional program. Gagné notes that one of the most valuable roles which well-written objectives have in the design of instructional programs is that they make it possible for the designer to draw ". . . distinctions among the different classes of behavior to be established, as a basis for inferences concerning how modification of preexisting behavior can be undertaken" (28:25). He states that "Unambiguous and complete state- ments of tasks to be performed when instruction is finished make possible the identification of certain categories of behavior to be learned" (28:60). Further: Actually, this may turn out to be the most important reason for describing objectives, although it has not always been stated clearly. What is intended is nothing less than the definition of certain classes of terminal behavior (such as discrimi- nation, chains, etc.) each of which, regardless of its specific content, carries a particular set of implications for the conditions of learning required for its establishment. For example, if it is known that the learner must be able to dig: criminate among 10 printed foreign words when ifistruCtion has been completed, this has a certain implication for the conditions of learning as they are built into an instructional sequence. Further- more, it is quite a different implication than is the case for the establishment of a capability to reproduce orally a particular chain or sequence of 10 foreign words. (28: 25) If the expected behavior stated in the objective is in ‘Vague terms or is nonexistent, it will be difficult to 139 categorize the objective in terms of the type of learn- ing involved. Because expected behavior is important in identifying the type of learning during the task analysis process, if it is not clearly stated, the objective will have to be rewritten. The importance of well-written objectives to the success of the instructional program is further emphasized when one recalls the distinction between enabling and terminal objectives which was discussed in the preceding section of this chapter. As noted, enabling objectives often must be mastered so a student will have the skills necessary to be able to perform a terminal objective. As DeCecco points out, the task analysis must be com- pleted for all objectives whether they are "subtasks” (22:46), that is, enabling objectives, or the "task proper" (22:46), that is, the terminal objective. He explains that "This is important because the subtasks may fall into different classes of behavior and require, therefore, different learning conditions" (22:46). For example, while the terminal objective may be assigned to one type of learning, the various enabling objectives associated with the successful performance of that objective may be categorized within other types of learning. This process cannot be completed prOperly if both terminal and enabling objectives are not well written. 140 Once the enabling objectives have been assigned to one of the learning categories identified above, they must be arranged in hierarchical order to insure their mastery. That is, the sequence of instruction must be arranged so that enabling objectives assigned to lower level learning categories must be mastered prior to those assigned to higher order categories in order to insure that the student will have properly learned the skills necessary to achieve the terminal objective. Briggs notes the impact the hierarchical order has upon the sequencing of instruction: . . . if there is a hierarchical structure . . . to the subordinate competencies of an objective, this structure suggests, at least in part, the appropriate sequencing of instruction. In this form of structure, sequencing implies that transfer of training occurs from the bottom to the top of the hierarchy in an upward direction. . . . (11:73) The same sequencing of instruction would be needed if the instructional program included the mastery of more than one terminal objective, each of which was assigned to a different learning category. Returning to the discussion of Gagné's work, once the objective has been assigned to one of the eight categories of learning, the designer is ready to select instructional procedures in accordance with the conditions of instruction which Gagné believes are necessary to insure the mastery of the behavior stated 141 in the objective. In The Conditions of Learning, Gagné provides extensive discussion of the conditions of instruction associated with each type of learning. It is these conditions, selected in response to each type of learning and incorporated into the instructional process, that help insure desired learning occurs. Gagné believes that the conditions of instruction are affected by two distinct types of variables. They are "Variables within the Learner" (31:292) and ”Variables in the Learning Situation" (31:293). They are often referred to as variables ". . . internal to the learner . . . [and] external to the learner . . .' (29:23). Each of these two types of variables will be examined separately. Internal Variables-~From Gagné's point of view, the internal variables which must be accounted for in the design process are the student's “. . . initial capa- bilities and motivation” (31:292). Examining the initial capabilities variable, Gagné states that there may be capabilities which are necessary for effective performance of a particular skill and ". . . if they are not [present], instruction specifically designed to teach task X will not work“ (31:293). These capabilities may be absent due to mental or physical deficiency or because capa- bilities ". . . dependent upon previous learning” (31: 293) may not have been mastered. In order to assist the instructional program designer in clarifying the initial 142 capabilities which the student must possess in order to effectively participate in a program designed to insure mastery of a particular objective, Gagné has described the prerequisite conditions which must exist within the learner for each of the eight types of learning he has identified. Once the objective which must be mastered has been identified with a type of learning, the pre- requisite initial capabilities which must exist if the learning program is to be effective can be specified. Gagné's point is illustrated by looking at the conditions which he believes must exist within the learner before that individual can effectively master concept learning and rule [principle] learning. Concept Learning Conditions within the Learner. Prerequisites to the—learning oficoncepts are capabilities that have previously been established by multiple discrimination. A set of verbal (or other) chains must have previously been acquired to representative stimulus situations that exhibit the chiractetistics of the class that describes the concept, and that distinguish these stimuli from others not included in the class. (29:180) Rule Learning Conditions within the Learner. The prerequisite for acquiring the EHains oficoncepts that constitute rules is knowing the concepts. (29:200) From a careful review of Gagné's discussion of initial capabilities, it is clear that the ordering of the eight types of learning from the least complex to the most complex provides helpful guidance in determin- ing prerequisite capabilities which must be learned in 143 order to successfully master more complex types of learn- ing. The mastery of lower level types of learning is essential to the accomplishment of higher level types of learning. Gagné notes: . . . the acquisition of a more complex capa- bility requires the previous existence of a simpler one, whereas the possession of a simpler capability does not imply that the individual can exhibit a more complex one. For example, the performance known as rule using (or principle using) implies that the indi- vidual can also classify the terms which make up the rule; otherwise, he would not have been able to learn the rule. On the contrary, the individual who can classify the terms contained in a rule does not necessarily know the rule; that is, he cannot necessarily show that he can use it. (31:298) Kemp interprets Gagné's point by noting that: A student begins a learning sequence by learn- ing facts. He then uses the facts to identify concepts. Finally, he builds relationships among concepts, which enables him to identify principles and develop problem-solving capabilities. Saying this another way, any higher order ability is dependent upon the learner having mastered all of the lower-level abilities. (44:26) These abilities are hierarchical in nature, so that successful experience in problem-solving requires the pre-learning of principles, which requires the pre-learning of conce ts, which requires the pre-learning of discriminations, and so on. (44:27) Recognizing that the need to master behaviors or develop skills which are assigned to lower categories in the learning hierarchy before more complex skills can be learned has an impact upon the development of appropriate procedures for a particular instructional program. Gagné states: 144 This has two important implications for instruc- tional design. First, it means that the sequence of instruction, to be most effective, must proceed from associations to discriminations to concepts to principles, and not vice versa. Second, it implies that the learner's previously acquired capabilities are of critical importance to the effectiveness of instruction and must surely be known if the instruc- tional program is to "take hold." (28:60) In discussing what Gagné refers to as the initial abilities of the student, Davis et al. note that: One of the ways learners differ is in their entry behaviors. Entry behaviors are the skills and learning sets which students bring with them to the learning situation. By learning sets, we mean prior learning available to a student for use in the new situation. (21:185) If the student does not possess the entry behaviors or initial capabilities which are prerequisites to the successful accomplishment of the types of learning necessary to master the objective in question, then the instructional designer must formulate enabling objectives and select the instructional procedures which will enable the student to successfully complete the less complex types of learning necessary to provide him/her with the skills and abilities needed to proceed. As will be dis- cussed in the next section of this chapter, a major role of evaluation in the successful design and implementation of an instructional program is the assessment of a stu- dent's initial or entry capabilities to determine if the prerequisites necessary for effective participation have been mastered or which capabilities must be acquired before an individual can proceed with the program. 145 Motivation is the second variable which Gagné believes the designer must account for when dealing with the learner's internal characteristics as the conditions of instruction are developed. He states: Motivation is another kind of state internal to the learner and prerequisite to effective instruc- tion. . . . However established, it seems fairly clear that a certain kind of motivational state must be present as a precondition for learning (Gagné and Bolles, 1959). It is probably a mistake to think of the necessary state as "motivation for learning." Instead, the essential motivation is something more like "willingness to enter into the learning situation." Obviously, if an individual is determined not to respond to a learning situ- ation, but to escape from it physically or other- wise, instruction cannot be effective. (31:293) Many authors identify the factors which they feel have an impact upon motivation and which they believe the instructional designer must take into con- sideration as he/she accounts for the internal variables related to the conditions of instruction. Kibler et al. believe that individual student motivation can be: . . . promoted by convincing learners of the value of mastering the subject matter and by offering rewards (e.g., social approval or grades) for accomplishing learning objectives. Selecting subject matter that interests students and permits them to participate in planning their educational activities can increase their incentive to learn. Shaping favorable attitudes toward the subject matter, the instructor, learning, and education in general can have positive long-range conse- quences for student motivation. (45:8) From Mager's point of view, the instructional program must be designed to motivate students to have 146 . strong approach tendencies toward a subject . . . (58:25). He notes that such individuals: . . . keep coming back for more experiences with the subject. They seek out experiences with the subject in preference to other desirable experiences. The more strongly they are attracted to a subject, the more obstacles they will overcome to come into contact with it and stay in contact with it. (58:25) A person's native abilities influence the kinds of activities he will engage in and the kinds of objects and events he will tend to approach. But though he may be partial toward those activities at which he is particularly adept, tendencies are influenced primarily by events in the world around him. They are shaped mostly by the attitudes of the people he encounters, by objects and experiences, and by the consequences of his own actions. (58:32) How can we improve our chances of strengthening approach? 0 By making sure there are as few aversive conditions present as possible while the student is in the presence of the subject we are teaching him. 0 By making sure that the student's contact with the subject is followed by positive, rather than aversive, consequences. 0 By modeling the very kind of behavior we would like to see exhibited by our students. (58:97) Katz examines six factors which have an impact upon motivation. They are: (l) conformity to legal norms or rule compliance; (2) instrumental system rewards; (3) instrumental individual rewards; (4) intrinsic satisfaction from role performance; (5) internalization of organizational goals and values; and (6) involve- ment in primary-group relations. (42:134) Lindquist examines factors which affect faculty motivation to change. He presents the following grid (51:334) to help illustrate his contention that the strongest motivation occurs when there is potential impact upon an individual's survival. 147 Survival Status/Esteem Formal Goals Holding my Tenure, pro- Scholarship, . . job motion, repu- Teaching Indiv1dual tation effective- ness Maintaining National vis- Producing _ our depart- ibility. majors and Sub group ment FTEs Graduate research programs discoveries Keeping Higher edu- Educating enrollment cational and students. . . up public Advancing Organization acclaim knowledge. Serving cultural needs Levels and Kinds of Needs. He notes: . . . the strongest motivation to change or to resist it [change] may lie in the upper left corner. . . . If institutional concerns come after individual or subgroup interests, and if the pursuit of truth, beauty, and justice comes after needs for physical security and social esteem, the weakest motivation to change occurs in the lower right corner. . . . (51:335) Gaff maintains: When environmental conditions are supportive and non-threatening, an individual's native growth tendencies are elicited. A person who is defensive and threatened will not reveal his true feelings and cannot take the risks necessary for change, because his energy must go into maintaining and supporting his own position. . . Faculty members will change when: (a) they have knowledge about alternative ways of behaving, such as information about alternative teaching- learning practices; (b) they have the belief that change is desirable; (c) they believe that they can change in the desired ways; (d) they receive 148 nonthreatening feedback about their own behavior; (e) they are praised, recognized, and rewarded for effectiveness and for improvement. . . . (27:17) Banathy points out that a student's motivation will be adversely affected if he/she is being asked to complete tasks for which he/she has not mastered the necessary prerequisites or if he/she is being asked to relearn material he/she has already mastered. She says: If we do not pay attention to individual differences in input capabilities, we invite trouble. The learner who has not acquired the capabilities we believe he should have will be frustrated and will probably fail. On the other hand, the student who is scheduled to learn something he already knows if going to be bored and will probably lose interest. (4:48) Goldstein notes: . . . motivational variables are more effective if they are: (a) viewed as instrumental for future activities; (b) intrinsic; (c) positive rather than aversive stimuli when extrinsic motivators are used; (d) set in terms of clear and concise goals. (38:119) Bass and Vaughn maintain that: the Whenever possible, the trainer should offer goals to the learner that have intrinsic, rather than extrinsic, reward value. A goal is intrinsically reinforcing if its achievement is naturally or inevitably reinforcing. . . . A goal is extrinsi- cally reinforcing if its achievement is arbitratily or artificially related to the task. . . . (5:56) DeCecco summarizes the role of motivation in learning process as follows: Motivation refers to factors which invigorate student behavior. For invigoration you may read increased effort. Motivation is cIOSely related to enteringbehavior: When students are motivated, 149 they make the strongest responses they have learned. No amount of motivation can trigger or direct responses the students have not learned. The factors which produce motivation are grouped into four classes: arousal, expectancy, incentives, and punishment. These factors, of course, are interrelated and interdependent. Arousal is the general level of alertness of the individual. The optimum levels of arousal are related to stimulus seeking, anxiety, and frus- tration. Expectancies are momentary beliefs about the probability of the occurrence of particular results. Expectancies are often products of dis- crepancies, and they are closely related to valences--what the individual wants as well as what he expects. Incentives are goal objects which become associated with certain stimuli and responses; they trigger anticipations. Punishment involves the suppression of an undesired response-- the individual learns what not to do. . . . From this model we derive the four moti- vational functions of the teacher. The arousal function requires the teacher to engage the student in learning. The expectancy function requires him to describe concretely what the student will be able to do at the conclusion of instruction. The incentive function requires the rewarding of present achievement in such a way as to encourage future achievement. Finally, the disciplinary function requires the teacher to use a combination of rewards and punishment in controlling student behavior. (22:180) To conclude this discussion on the internal con- ditions which must exist within the learner in order for effective instruction to occur, it is important to note that the designer may have little control over those variables when the student enters the instructional environment initially. However, as instructional pro- cedures are formulated, the designer must include pro- cedures which establish the existence of or provide exercises to deve10p the necessary prerequisites which 150 must be mastered before more complex types of learning can take place. Additionally, the designer must develop procedures which will contribute to the student's moti- vation to learn. External Variables--The second type of variables that Gagné believes should be accounted for is the external variables. These are the conditions external to the student over which the designer has maximum con- trol. As instructional procedures are selected, the designer can affect and is responsible for establishing those conditions within the instructional environment which will help insure that desired learning occurs. The designer seeks to arrange the instructional environ- ment in such a manner that the desired behavior will be mastered. As Davis et al. note, ". . . we are concerned with the structure and organization of the environment to bring about learning in an Optimal way" (21:162). To meet this goal, the designer must insure that the conditions of and the instructional activities presented within the environment are responsive to the specific type of learning required to occur in order to insure mastery of each objective. For each of the eight types of learning presented earlier, Gagné has identified specific external conditions which must be incorporated into the instructional environ- ment in order to insure that the desired learning will 151 occur. His point is illustrated by looking at the con- ditions which he believes must exist within the learning situation in order for an individual to master concept learning and rule learning: Concept Learning Conditions with the Situation. . . . the situational conditions for learning concepts are largely embodied in a set of verbal instructions. . . . l. The specific stimulus Objects, to which chains that include a common final link have been pre- viously learned, are presented simultaneously, or in close time succession. Instructions are used to stimulate the learner to recall and reinstate these chains. . . . 2. Instructions go on to elicit the same common link to a stimulus situation belonging to the proper class but to which the learner has not previously responded. . . . 3. Once these events have occurred, the new capa- bility may be verified by asking for the identification of several additional instances of the class, again using stimuli to which the learner has not acquired specific verbal chains. 4. The condition of reinforcement is present in the concept-learnifig situation. The learner's response . . . to the new set of stimuli must be confirmed more or less immediately if the concept is to be learned. Conti uit is also of importance, as emphasized in condition 1. The specific instances need to be presented contiguously, and their names recalled just prior to the presentation of the "new" instance. . . . (29:181) Rule Learning Condition in the Learning Situation. The major external conditidns of rule learning are embodied in yerbal instructions. . . . 1. The conditibns of rule learning often begin with a statement of the general nature of the pggf formance to be expected when learning is com- plete. . . . It provides the learner with a means for obtaining immediate reinforcement when he has reached the terminal act. Having this statement for a model, he will be able to know when he has finished learning and, in many cases, when he has acquired the correct rule. . . . 152 2. Verbal instructions continue by invoking recall of the component concepts. . . . 3. Verbal cues are next given for the rule as a whole. . . . 4. Finally, a verbal question asks the student to demonstrate the rule. . . . The exact form is not of great importance as long as it truly requires the student to demonstrate the rule in its full sense. . . . 5. The presence of some familiar learning conditions may be recognized in rule learning. Contiguity appears to be an important condition appliCable to time interval between the recall of component concepts (step 2) and the verbal cuing of the rule with these parts properly sequenced (step 3). Reinforcement is provided when the rule is exhibited in its complete form. . . . (29:201) Omitting the first type of learning, signal learning, Gagné presents, in table form, a "Summary of Conditions Considered Necessary for Seven Kinds of Learning" (31:301) which provides an overview of the conditions, both internal and external, which he believes are necessary to maximize learning within each learning * category. *In his book, Conditions of Learning (29), Gagné uses the terms stimulus-response learning rather than specific responding, verbal association rather than verbal, discrimination-learning rather than multiple discrimination, concept learning rather than classifying, and rule learning rather than rule using. However, the ideas communicated are intended to be the same. In the presentation of Gagné's learning categories reported by DeCecco and presented earlier, the term stimulus-response learning is used rather than specific responding, verbal association is used rather than verbal, concept learning is used rather than classification, and principle learning is used rather than rule learning. Once again, the ideas communicated are intended to be the same. 153 Performance established by learning Internal (learner) conditions External conditions Specific responding Chaining: Motor Verbal Multiple discrimi- nation Classifying Rule using Problem Solving Certain learned and innate capa- bilities Previously learned indi- vidual con- nections Previously learned indi- vidual connec- tions, including implicit "coding" connections Previously learned chains, motor or verbal Previously learned multiple discriminations Previously learned concepts Previously learned rules Presentation of stimulus under conditions command- ing attention; occurrence of a response contiguous in time; reinforcement Presenting a se uence of external cues, effecting a sequence of specific responses contiguous in time; repetition to achieve selection of response-produced stimuli Presenting a seguence of external verba cues, effecting a sequence of verbal responses 3223 tiguous in time Practice providing con- trast of correct an incorrect stimuli Reinstating discriminated response chain contigu- ously with a variety of stimuli diffeting in appearance, but belong- ing to a single class Using external cues (usually verbal), effect- ing the recall of pre- viously learned concepts contiguously in a suit- able sequence; specific applications of the rule Self-arousal and selec- tion of previously learned rules to effect a novel combination 154 Gagné summarizes the role of these variables in the process of instruction: There are, then, two broad classes of variables that influence learning: those within the learner, and those in the learning situation. These sets of variables undoubtedly have interactive effects upon learning, as many writers have emphasized. The external variables cannot exert their effects with- out the presence in the learner of certain states derived from motivation and prior learning and development. Nor can the internal capabilities of themselves generate learning without the stimulation provided by external events. . . . The learning problem is one of finding the necessary relation- ships which must be obtained among internal and external variables in order for a change in capa- bility to take place. . . . Instruction may be thought of as the institution and arrangement of the external con- ditions of learning in ways which will optimally interact with the internal capabilities of the learner, so as to bring about a change in these capabilities. (31:295) In his book, Conditions of Learning (29), Gagné provides an extensive discussion of the conditions which, depending upon the type of learning involved, must exist in the learning situation. DeCecco provides a similar discussion in his book, The Psychology of Learning and Instruction (22). The instructional program designer would be well advised to consult these texts as he/she determines those procedures to be implemented as part of the instructional program in order to establish those conditions within the instructional environment necessary to insure that desired outcomes are achieved. 155 Positive Transfer of Learning As the instructional program designer identifies and incorporates the appropriate conditions of instruction necessary to assure mastery of the objective, it is important to insure that the instructional program which is developed and implemented makes "Provision for Trans- ferability" (29:318). Gagné encourages instructors to remember that: . . . learning is managed and instituted for broader purposes than simply the modification of particular human performances. It is brought about in order to establish capabilities that will be of lasting and general usefulness to the individual. (29:334) He identifies two ways in which skills and capa- bilities learned in the instructional setting can be useful in other settings One is in making it possible for the individual to execute some performances that are not directly learned but are in some sense similar to those that are learned. . . . The capabilities specifically learned in school should enable the student to per- form some acts of practical value to him, whether in his everyday life or in connection with an occupation. This transferability of what has been learned may be calledtlateral’transfer, since it refers to a kind of generalizing that spreads over a broad set of situations at roughly the same ”level of complexity.” The second kind of use for learned capabilities lies in making it possible for the individual to learn additional, "advanced," or more complex things. . . . The subordinate capabilities transfer to the higher-order learning and facilitate its occurrence. This may be called vertical transfer, since it refers to the effects that learned capa- bilities at one level have on the learning of addi- tional ones at higher levels. . . . (29:335) 156 From Gagné's point of view, insuring transfer- ability has clear implications for the type of procedures which should be incorporated into the instructional program. For example, he maintains that lateral transfer of a particular learned capability can be increased ". . . if it is practiced in as wide a variety of situ- ations as possible" (29:336). Gagné stresses: "The implication for the management of instruction is, there- fore, quite clear: provision needs to be made for encouraging the learner to apply his knowledge broadly and in as great a variety of new situations as can be devised" (29:337). Gagné maintains that insuring the vertical trans- ferability of a learned capability requires the ”. . . mastery of the subordinate capabilities" (29:337). He believes that vertical transfer is improbable: . . . unless the relevant lower-order chains, con- cepts, or rules have been learned. Learning of these subordinate capabilities must recede transfer. . . . The most important prescription for managing the conditions of vertical transfer can be stated as follows: insure that relevant subordinate capabilities have been thoroughly learned before calling on vertical transfer to aid the learning of "advanced" capabilities. (29:337) Goldstein examines the conditions which need to exist in the instructional setting in order to maximize the transfer of behavior which is learned to other settings where it can be used. He suggests the degree 157 of transfer is affected by ". . . the relationship between the learning and transfer setting . . . [and] the degree of learning in the training environment" (36:112). As discussed in the previous section of this chapter, he points out that the degree of transfer increases if the tasks to be mastered and performed in the training setting and the transfer setting are identical and/or if the student has learned the princi- ples necessary to perform the task and can ". . . apply them to solve problems in the transfer task" (36:109). Similarly, Bass and Vaughan indicate that: The identical-elements theory states that positive transfer will occur only if identical elements are present in the old and new situations. . . . The more similar the learning situation is to the job situation, the higher the degree of positive trans- fer we can expect. (5:39) Bass and Vaughan (5:38), Goldstein (36:107), Ellis (23:3), and DeCecco (22:440) stress that transfer is not necessarily positive. Positive transfer will occur if skills and behaviors mastered in the learning setting enhance performance in the transfer setting. Negative transfer will occur if skills or behaviors mastered in the learning setting disrupts performance in the transfer setting. Zero transfer will occur if skills or behaviors mastered in the learning setting have no effect upon performance in the transfer setting. Referring to work done by Ellis (23:70), Goldstein cites 158 a series of procedures which, if incorporated into the instructional program, will increase positive transfer. They are: l. Maximize the similarity between the teaching and the ultimate testing situation. 2. Provide adequate experience with the original task. Most research shows that adequate prac- tice in training is essential for positive transfer. This is especially true for new skills and concepts for which thorough training must be given early in the learning process. 3. Provide for a variety of stimulus situations so that the student may generalize his knowledge. One way of overcoming the constraints of a training setting is to provide a variety of stimulus situations so that the learner can begin to generalize his concepts to the many situations in which transfer must occur. 4. Label or identify important features of the task. Labeling helps distinguish the signifi- cant characteristics of the task. Thus, the learner is able to use the necessary cues to determine when transfer behavior is appropriate or inapprOpriate. 5. Make sure that general principles are understood. This can be accomplished by presenting a variety of situations and asking the learner to apply the general principle. If the program is based on the learning of principles and the trainee does not thoroughly understand them, he has gained little from the training program that will be useful in the transfer setting. (36:111) Kibler et al. explain that providing students with clearly stated objectives increases positive transfer. They note: Research concerned with the transfer of learning indicates that students generally do not apply learned skills or knowledge to practical situations unless the teacher specifically demonstrates the application. The popular phrase in academic circles which reflects this body of research is "teach for transfer." The teacher attempting to implement this strategy makes desired behaviors explicit and specifies the variety of conditions under which 159 the behaviors or skills may be applied after they have been adequately learned. When behavioral objectives are given directly to students, the exact behaviors desired and the conditions under which the behaviors are to be exhibited are specified. By being given behavioral objectives, students do not have to guess what is expected of them in the learning setting. (45:106) Recognizing the importance of transferability, Davis et al. note that ". . . a basic assumption under- lying all training is that it will transfer from one situation . . . to some other situation . . ." (21:193). They recognize that often the learning situation or setting will not be exactly like the transfer setting where the task will be performed and note that ”. . . frequently, a task . . . must be performed under a wide variety of conditions which only more or less approximate the original learning condition” (21:193). However, Davis et al. suggest that, as a general guideline, efforts should be made to increase the similarity between the instructional setting and the transfer setting. They explain: . . . try to include as many elements from the real- life situation in the training conditions as pos- sible. At the start, the designer will want to abstract away a good deal of the complexity so that the learner does not have to cope with too many variables at once, but gradually he will have to prepare the learner to deal with the task outside of the classroom where it will often be extremely complex. This means that sooner or later the practice situation must resemble as nearly as possible the conditions under which terminal behaviors are to be performed. (21:193) 160 O'Connor also recognizes that the degree of transfer will be increased if there is similarity between the stimulus provided and response required in the instructional setting and the transfer situation. She explains that: Complex skills and much knowledge are derived by combining, in a new pattern, already acquired knowledge or skills. One important way of insuring positive transfer in learning is to give many oppor- tunities for combining past elements in new ways. (68:105) Jahnke stresses that: ”Instructors must con- tinually review the content of their courses with an eye to increasing the fit between what is taught and what is to be demanded of the student when he has left the classroom" (41:203). In addition to the development and implementation of conditions within the instructional setting which will maximize the probability of positive transfer, Mosel points out that the learner must be ”. . . motivated to change his job behavior to reflect what he has been taught in training" (65:57). The transfer of what is learned from the instructional setting to the work setting is much more apt to occur if it is ". . . gratifying and rewarding to do so” (65:57). The designer must work closely with supervisors to be sure there are ". . . rewards and punishments, incentives and detriments in the job situation . . ." (65:57) which will reinforce 161 positive transfer. Additionally, Mosel believes instruc- tional programs should ". . . include training on how to overcome the problems encountered in applying the train- ing" (65:63). Select Methods and Materials Having determined the procedures which should be incorporated into the instructional program in order to (1) apply those general principles of learning which are appropriate, (2) incorporate the conditions of instruction in light of the type of learning that must occur to insure mastery of the objective, and (3) insure positive transfer of what is learned, the instructional program designer is in the position to select the specific methods and/or materials to be used during the instructional program. These become the visible representatives of the procedures the designer has decided to implement within the instruc- tional environment in order to provide the student with the experience necessary to master a given objective. .Methods include on-the-job training, lecture, programmed instruction, field trips, computer-assisted instruction, independent study, laboratory training, simulation (e.g., case study, role-playing), discussion, demonstration, cIuestion-answer, modeling, projects and reports, and homework. Materials include books, television, motion [rictures, chalkboards, charts, pictures, natural objects, 162 overhead transparencies, photographs, instruments, tape recorders, and slide programs. While the list of methods and/or materials could be expanded, the important point is not what this list might include but how the items on it are selected for use in a particular instructional program. As presented earlier, depending upon the objec- tive involved, particular types of learning must occur to insure mastery of the desired skills and behaviors; particular conditions of instruction must exist to insure mastery of the objective; and the instructional program must be designed to prepare students to perform mastered skills and behaviors in the transfer setting. Methods and materials should be selected because they contribute to establishing the appropriate conditions of instruction necessary to insure that desired learning occurs and because they contribute to positive transfer. As Briggs notes, methods and materials ". . . should be chosen to facilitate demonstration of the desired types of behavior or response" (11:100) which the student will be expected to perform at the end of the instructional program. Briggs explains that methods and materials enable the student to perform in accordance with ”. . . the do what and 32 or with what . . ." (11:100) conditions stated in the objective. 163 The important concept to keep in mind is that decisions related to the design of instructional pro- grams should be made to facilitate the student's success- ful performance of the tasks stated in the objective. Goldstein cautions, "This is a delicate process that requires a blend of learning principles and media selection, based on the tasks that the trainee is even- tually expected to perform” (36:21). When considering the selection of methods and materials, it is important that decisions be made on the basis of desired performance, not on the basis of which methods or materials might be available or easy to use. Citing a statement by Gilbert, Goldstein cautions that, if the designer begins to con- sider the development of the instructional program with a survey of methods and materials available, there is a high temptation to use these techniques or devices regardless of whether or not they properly facilitate the mastery of identified objectives. Gilbert says, "If you begin with a device of any kind, you will try to develop the teaching program to fit that device" (36:21). The instructional program must not be built around available methods or materials. Rather, methods and materials should be selected based upon the con- ditions of instruction and the transfer setting identi- fied as critical to the successful accomplishment of an identified objective. As Mager and Beach note: 164 Some teachers decide which procedures and materials to use on the basis of what they are most comfortable with. . . . Though it may be acceptable for amateurs to select their teaching procedures this way, the professional wants a more rational basis for making his decisions. After all, the professional instructor is an individual who can make his selections in a manner that results in efficient instruction. (60:44) Gagné suggests that the conditions which must be provided within the instructional environment in order to insure that desired learning occurs should guide the selection of instructional methods and materials. He offers these examples: If multiple discriminations are being established among varieties of rocks, care must be taken to select the samples so that the differential features will be emphasized. . . . If the concept of magnetic attraction is to be learned, a suitable variety of magnets and pole orientations must be employed. The kind of learning that is contemplated therefore has considerable significance for the choice of objects selected as media for instruction. (29:351) The desire to insure a high degree of transfer from the instructional setting to the work performance setting affects the selection of methods and materials. Since positive transfer is more apt to occur when there is a high degree of similarity between the instructional setting and transfer setting, methods and materials selected for use in the instructional setting should be as similar as possible to those which will be encountered when the student is asked to perform what he/she has learned in the transfer setting. For example, when the transfer of principles is involved, the 165 selection of methods and materials which enable students to practice the principle in a variety of ways within the instructional setting will help insure that students will be able to apply the principle in the transfer set- ting. From Gagne's point of View, the selection of methods and materials should be designed to increase: . . . the variety of stimulus situations over which the student is encouraged to generalize his knowl- edge. The more varied these can be made, the more useful will the learned capability become. At lower educational levels, this variety may be achieved by deliberate use of a whole range of natural objects and events in the classroom or on field trips. At higher levels, the function of providing contextual variety can be largely per- formed by verbal communication, of the sort that may take place in a ”discussion group,” for example. (29:339) Mager and Beach summarize: Hopefully, the point has been made that the selection of appropriate teaching procedures begins with determining precisely what the performance objec- tives are. With the type of performance identified for each part of each task, it is possible to go on to identifying the general class of procedure, or combination of procedures, appropriate for reaching each objective. For example: If one objective is, "Given two pairs of engine sounds, the student must be able to identif the one most representative of a smooth-running engine," then some form of audio instruction is appropriate. If one step in learning to perform a task is learning to recognize when a table has been properly set, then some form of visual technique is appropriate; a drawing, or slide, or photograph, or film might be used. A tape recording or a lecture would be less appropriate in reaching the objective. If one of the objectives calls for the student to be able to actually set a table, then a different technique is called for. First, he should be taught to discriminate between a properly set and improperly set table; then he should be given actual practice. For this objec- tive, a table and utensils are more appropriate than a discussion or a filmstrip. (60:54) 166 Mager and Beach offer three guidelines to follow in selecting an instructional technique which will contribute to the accomplishment of the desired objective: 1. Choose the technique that most closely approxi- mates the performance conditions called for by the objective. . . . 2. Choose the technique that causes the student to perform in a manner most closely approximating the performance called for on the job. . . . 3. Choose the technique that will allow the student to make the largest number of relevant responses per unit time. . . . (60:56) Mager and Beach stress the importance of select- ing those techniques that are the ”. . . most practical from among those that are appropriate” (60:58). Simi- larly, Kemp notes that, in addition to selecting methods and/or materials which best facilitate the mastery of specific objectives, there are a number of practical questions which must be considered. These include: Does the needed material already exist in suitable form and quality? What are the anticipated purchase or preparation costs? . What are the reproduction or duplicating costs, if any? How much time will be required to locate or pre- pare each item? What are the requirements for equipment, facilities, technical skills, or services in preparation? Is one medium more suitable than the others with respect to ease of viewing or student handling? Will there be problems regarding equipment, facili- ties, supervision, and scheduling? (44:68) The value of those methods and materials selected for use in the instructional program should be judged both in terms of their practicality and the degree to which 167 they facilitate a student learning those skills necessary to perform in accordance with the criteria presented in the objective. Having selected those principles of learning and conditions of instruction necessary to insure that desired performance and transfer will result, and having determined the procedures which he/she believes should be implemented as well as having selected appropriate methods and materials necessary to operationalize identified principles and conditions, the designer is prepared to implement the instructional program and, as will be discussed in the next section of this chapter, evaluate its effectiveness. At this point, the instruc- tional process has been designed to accomplish the trans- action outlined by Glaser when he states: "If entering behavior is considered state A and a subsequent per- formance objective is state B, then the instructional process is designed to arrange the student's environment to get him, or, if one prefers, have him get himself from state A to state B" (34:440). Evaluate As instructional programs are designed and imple- mented, it is necessary to develop processes to evaluate the program's effectiveness at appropriate stages in order to determine whether objectives are being achieved 168 and whether selected procedures are having the desired impact upon student learning. Evaluation will be examined in this section. The placement of this element near the end of this chapter should not be interpreted to mean that the evaluation process should be planned late in the design process or that evaluation only occurs after objectives are written and instructional procedures are determined and implemented. On the contrary, the design and imple- mentation of evaluation processes occur throughout the instructional program. Evaluation begins with the assessment of needs and continues through program redesign; the element discussed in the next and final section of this chapter. As Goldstein notes: The criteria and methods for evaluating programs cannot be conveniently added onto the end of the project without disrupting the training program. In addition, some of the data must be collected before and during the training program, as well as some time after the student has completed training. The evaluation design is an integral part of the entire program. . . . (36:48) The important role that evaluation processes should have throughout the instructional program can be illustrated by recalling material presented in previous sections of this chapter. For example, the value of carefully assessing where instructional pro- grams are needed, the requirement that well—written enabling and terminal objectives be developed so student progress in achieving desired outcomes can be 169 evaluated, and the need to assess the student's mastery of prerequisites in order to insure that he/she possesses the entry skills necessary to effectively participate in the instructional program, are indicative of the on- going inclusion of evaluation processes throughout the instructional program. Recognizing that evaluation processes must occur throughout, this section of the chapter will focus on evaluation as an element which must be included in the design and implementation of instructional programs. The views of selected authors about (1) the type of evaluation that should occur, (3) the frequently improper completion of evaluation, and (3) the factors which should be considered as evalu- ation processes are designed and implemented are con- sidered here. Types of Evaluation Evaluation has been defined by Stufflebeam as ". . . the science of providing information for decision- making” (88:6). Similarly, Cronbach states ". . . to draw attention to its full range of functions, we may define evaluation broadly as the collection and use of information to make decisions about an educational pro- gram” (19:672). Stressing the important contribution which evaluation makes to the decision-making process related to the design and implementation of instructional programs, Goldstein states: 170 Evaluation consists of procedures designed to sys- tematically collect the descriptive and judgmental information necessary to make effective training and educational decisions. Decision makers are concerned with questions related to the selection, adoption, and value of various training activities. The objectives of instructional programs reflect many different types of goals, ranging from student progress to organizational goals; thus, the evalu- ation must examine the total complexity of the program. (36:49) Cronbach identifies three types of decisions which must be made. Evaluation processes are needed in order to gather the information required to make these types of decisions. He explains: We may separate three types of decisions for which evaluation is used: 1. Course improvement: deciding what instructional materials and methods are satisfactory and where change is needed. 2. Decisions about individuals: identifying the needs of the pupil for the sake of planning his instruction, acquainting the pupil with his own progress and deficiencies. 3. Administrative regulation: judging how good the school system is, how good individual teachers are, etc. (19:673) After defining evaluation as ". . . a continuous process of collecting and interpreting information in order to assess decisions made in designing a learning system" (21:81), Davis et al. explain its impact upon the design of an instructional program: First, evaluation is an on-going process, not some- thing you do only at the end of a course. It is a process that starts even before instruction begins and continues until the end of instruction. Second, the evaluation process is not haphazard but instead, is directed toward a specific goal. It is directed at finding answers about how to improve instruction. Third, evaluation requires using accurate and 171 appropriate measuring instruments to collect infor- mation needed for decision making. The evaluation process involves collecting information to enable you to decide how your instruction is progressing, how it turned out in the end, and how to do better next time. (21:81) To fulfill the role discussed here, evaluation must take place when the most valid and reliable infor- mation can be gathered and when such information will be most useful to the decision-making processes involved, whether they are related to the development of the best possible instructional program or to assessing the achieve- ment of the program's desired outcomes. Therefore, various types of evaluation need to be utilized at various points in the design and implementation of an instructional program. The following presentation illustrates the types of evaluation that various authors believe should be incorporated into an instructional program. Catalanello and Kirkpatrick suggest four types of evaluation which should be incorporated into an instructional program. They are: l. Reaction--How well did the trainee like the program? 2. Learnin --To what extent did the trainees learn the facts, principles, and approaches that were included in the classroom training? 3. Behavior--To what extent did their job behavior change Because of the program? 4. Results--What final results were achieved? (Reduction in costs, reduction in turnover, improvement in production, etc.). (15:2) 172 Davis et al. discuss three types of evaluation for which they believe ". . . critical evaluation questions" (21:82) must be formulated and answered. The first type they discuss is evaluation of initial con- ditions. They explain that: Because of the limited amount of instructional time available to achieve objectives, a teacher must make certain decisions about the ability level of the students when they enter the system. It would be a waste of time to cover material students already know. On the other hand, if we assume stu- dents know more than they actually do, they will have great difficulty learning. Students must have attained a particular level of entry skills and knowledge so that they will be prepared for instruc- tion. If the students do not have the necessary skills, the instructional procedures you have designed cannot be effectively implemented. (21:83) Davis et al. also note that the determination of the entry skills of each student provides one of the key pieces of information needed to assess the effectiveness of a particular instructional program. Without knowledge of the student's entry skills, the instructor will be unable to compare entry and exit skills and, therefore, will have no basis upon which to judge the amount of progress made by the student during the course. As Davis et al. note: By gathering information at the beginning of a unit, the teacher obtains information about the student's prior learning. By measuring learning at the end of the unit, the teacher can see what students have learned from the instruction. The effectiveness of the learning system is determined by comparing these two types of information. (21:94) 173 Davis et a1. further explain that knowledge of the stu- dent's entry skills, coupled with the knowledge of the objectives which must be mastered at the end of the instructional program, helps to focus the student's attention on what must be learned and is apt to increase motivation as well as give direction to the learning pro- cess. The second type of assessment which Davis et al. believe should occur is evaluation of instructional pro- cedures. Referred to as ”CONTINUOUS EVALUATING” (21:99), this type of assessment, through continuous monitoring of student progress, enables the designer to make decisions regarding the quality of the instructional program as it is put into effect. The evaluation processes associated with this type of assessment must be developed so that, as the authors note, the data gathered as the instruc- tional program is being put into effect can be utilized ". . . to provide feedback to students about their learn- ing progress and to identify and correct instructional problems as they arise" (21:99). The third type of evaluation for which Davis et al. believe questions should be formulated and answered is designed to assess whether ”TERMINAL OBJEC- TIVES" (21:83) or "INSTRUCTIONAL OUTCOMES" (21:105) were achieved in the most effective, efficient manner possible. They explain: 174 Terminal objectives describe the instructional out- comes of a learning system, and student achievement of these objectives represents one of the most important criteria for evaluating system design. However, a learning system is evaluated by other criteria as well. In designing the system, decisions are also made about how available resources can be used most efficiently. In some cases, the outcomes from two different learning systems, such as tele- vised yg. regular classroom instruction, may be evaluated and compared. (21:84) Stufflebeam has identified four types of evaluation which he believes should be incorporated into the design and implementation of a project or program. He identifies and explains each as follows: Context evaluation would be used when a project is first being planned. The major objective of context evaluation is to define the environment where change is to occur, the environment's unmet needs, problems underlying those needs, and oppor- tunities for change. . . . The objective of input evaluation is to identify and assess relevant capabilities of the proposing agency, strategies which may be appropriate for meeting program goals and designs which may be appropriate for achieving objectives associated with each program goal. The end product of input evaluation is an analysis of alternative procedural designs in terms of potential costs and benefits. Once a designed course of action has been approved and implementation of the design has begun, process evaluation is needed to provide periodic feedback to project managers and others responsible for continuous control and refinement of plans and procedures. The objective of process evaluation is to detect or predict, during the implementation stages, defects in the procedural design or its implementation. . . . Product evaluation is used to determine the effectiveness of the project after it has run full cycle. Its objective is to relate outcomes to objectives and to context, input, and process, i.e., to measure and interpret outcomes. (88:7) 175 Bloom et al. discuss three types of evaluation which they maintain should be carefully designed and incorporated into instructional programs developed to achieve desired student learning. They note that: . . . diagnostic evaluation . . . involves a valuing, determination, description, and classifi- cation of some aspect of student behavior. How- ever, the two purposes of diagnosis-—either to place the student properly at the outset of instruction or to discover the underlying causes of deficiencies in student learning as instruction unfolds--distinguish it from other forms of evaluation. Diagnostic evaluation performed prior to instruction has placement as its primary function; that is, it attempts to focus instruction by locating the proper starting point. Diagnosis for this purpose may take several forms. First, it may seek to determine whether or not a student possesses certain entry behaviors or skills judged to be pre- requisite to the attainment of the objectives of the planned unit. Second, it may attempt to estab- lish whether the student already has mastery over the objectives of a given unit or course, thereby allowing him to enroll in a more advanced program. Finally, it may aim to classify students according to certain characteristics, such as interest, per- sonality, background, aptitude, skill, and prior instructional history, hypothesized or known to be related to a particular teaching strategy or instructional method. Diagnostic evaluation performed while instruction is underway has as its primary function determining the underlying circumstances or causes of repeated deficiencies in a student's learning that have not responded to the usual form of remedial instruction. The causes for a student's failure in a formative unit may be unrelated to the instructional methods and materials per se, but may instead be physical, emotional, cultural, or environmental in nature. Diagnosis tries to pinpoint the reasons for observed symptoms of learning disorder so that, when possible, remedial action can be taken to correct or remove these blocks to progress. (8:87) Formative evaluation is for us the use of sys— tematic evaluation in the process of curriculum construction, teaching, and learning for the pur- pose of improving any of these three processes. 176 Since formative evaluation takes place during the formative stage, every effort should be made to use it to improve the process. This means that in formative evaluation one must strive to develop the kinds of evidence that will be most useful in the process, seek the most useful method of report- ing the evidence, and search for ways of reducing the negative effect associated with evaluation-— perhaps by reducing the judgmental aspects of evalu- ation or, at the least, by having the users of the formative evaluation (teachers, students, curriculum makers) make the judgments. The hope is that the users of the formative evaluation will find ways of relating the results of the evaluation to the learning and instructional goals they regard as important and worthwhile. (8:117) On the other hand, summative evaluation is directed toward a much more general assessment of the degree to which the larger outcomes have been attained over the entire course or some substantial part of it. (8:61) We have chosen the term "summative evaluation" to indicate the type of evaluation used at the end of a term, course, or program for purposes of grading, certification, evaluation of progress, or research on the effectiveness of a curriculum, course of study, or educational plan. . . . Per- haps the essential characteristic of summative evaluation is that a judgment is made about the student, teacher, or curriculum with regard to the effectiveness of learning or instruction, after the learning or instruction has taken place. (8:117) Kaufman identifies three types of evaluation. In addition to the evaluative processes associated with "Determining Educational Needs" (43:28), Kaufman sug- gests that summative and formative evaluation should be incorporated into instructional programs. He explains that: Summative evaluation is the determination of the degree to which we have accomplished our ends. . . . (43:140) [Formative evaluation] . . . provides infor- mation relative to whether we are "on target,” 177 and if we are not, it gives information for mid— course corrections to assure that we will even- tually be successful. (43:141) Gagné and Briggs also recognize the need for formative evaluation and summative evaluation. In dis- cussing the purpose of formative evaluation the authors note: One purpose for which evidence of an instructional program's worth is sought is for use in making decisions about how to revise the program while it is being developed. In other words, the evi- dence collected and interpreted during the phase of development is used to form the instructional program itself. If one discovers by means of an evaluation effort that a lesson is not feasible, or that the newly designed topic falls short of meet- ing its objectives, this information is used to revise the lesson, or to replace portions of the t0pic, in an attempt to overcome the defects which have been revealed. (33:232) To explain summative evaluation, Gagné and Briggs note: Summative evaluation is usually undertaken when development of an instructional entity is in some sense completed, rather than ongoing. Its purpose is to permit conclusions to be drawn about how well the instruction has worked. . . . In general, summative evaluation concerns itself with the effectiveness of an instructional system, course, or topic. . . . The evaluation is called summative because it is intended to obtain evidence about the summed effects of a set of lessons making up a larger unit of instruction. (33:236) Goldstein discusses the value of formative and summative evaluation. He notes that: . . . formative evaluation is utilized to determine if the program is operating as originally planned or if improvements are necessary before the program is implemented. The major concern of summative evaluation is the evaluation of the final product with the major emphasis being program appraisal. 178 Thus, formative evaluation stresses tryout and revision processes, primarily using process cri- teria, while summative evaluation uses outcome criteria to appraise the instructional program. (36:69) DeCecco discusses two types of assessment which he believes need to occur in order to evaluate a stu- dent's performance during, as well as at the end of the instructional program. He maintains that the student's . . . auxiliary and terminal performances . . ." (22:610) need to be measured: Auxiliary performances are behaviors which must be acquired at the lower levels of a learning structure before the terminal performances are acquired at the higher levels. . . . Terminal performances . . . refer to the end products of instruction. . . . The emphasis on the measurement of both auxiliary and terminal performances means that you should not think of performance assessment as occurring only at the end of a unit or a course. The assessment can occur whenever the teacher or the student needs information about the adequacy of the student's present learning for subsequent instruction. (22:610) DeCecco suggests that the information gathered from the assessment of auxiliary and terminal performances can be used for three purposes. They are: First, you use the feedback to determine how well the student has achieved the instructional objective (or terminal performance). If the student has reached the standard of acceptability the instruc- tional objective states, the instruction has been successful and you can proceed to new instruction. Second, you use the feedback to determine the adequacy of entering behavior. This use is appro- priate either during the course of instruction, when you are measuring auxiliary behavior, or at the end of instruction, when you are determining why the instruction was unsuccessful. Third, you use the feedback to determine the adequacy of your 179 instructional procedures, particularly when you compare the effectiveness of one with another procedure. (22:610) Mager and Beach suggest that the instructional program designer should prepare two types of tests so that the student's entry skills as well as performance level at the end of the program are evaluated. The "Prerequisites Test" (60:39) is necessary to determine whether the student has the entry level skills and abili- ties necessary to effectively participate in the instruc- tional program. Mager and Beach explain that ". . . the criterion exam, or 'post-test,’ is constructed solely from the course objectives. The object is to determine how well the student's performance at the end of instruc- tion coincides with performance called for in the objec- tive" (60:40). The authors also stress the importance of assessing the efficiency and effectiveness of the instructional program: The course is efficient to the degree it does what it sets out to do. It is effective to the degree it sets out to do those things most related to the job or vocation to be taught. As we have seen, efficiency is checked by comparing actual student performance with the objectives. Effec- tiveness, on the other hand, is checked by com- paring the objectives with the actual job or vocation. The effective vocational course is one that selects the appr0priate objectives ... . and causes each student to reach them. (60:71) Kemp discusses three types of evaluation which should be incorporated into an instructional program. 180 The first form of evaluation, pre-test, should be designed to answer two questions. They are: "1. Are the students prepared to study the topic or unit? 2. Have the students already achieved some of the stated objectives?" (44:47). Kemp explains that a second type of evaluation, "Measuring Student Learning" (44:38), is designed to assess ". . . the degree to which each student has mastered the objectives" (44:38) of the instructional program. Kemp notes that the measurement of student learning need not only occur at the conclusion of the program. He states: Evaluation does not have to be confined to an examination given at the conclusion of the study of a topic or unit. Testing can be on-going throughout the unit, as the student moves along in the learning sequence. Brief "check-points" or exercises can be used as intermediate evalu- ation devices so that the student, as well as the teacher, has an opportunity to know how successful his learning is as he proceeds. (44:40) Kemp notes that the purpose of the third type of evaluation, "Evaluating the Instructional Plan" (44:40): . . . is to determine if there are any weaknesses in the instructional plan in order to enable the teacher to improve it. In this case, the student serves as the primary resource for evaluating the program. An analysis of his results of tests or other evaluation measures, as well as direct observations made while the student works, can indicate deficiencies in the learning sequence and the need for corrections. (44:40) 181 Kemp also maintains that the evaluation of the instruc- tional plan must include an answer to the question, "How effective and efficient has the program been in achieving the desired objectives for the student group?" (44:96). Kemp explains that to assess effectiveness, . . . you should determine how many students accomplished the stated objectives within the time set. Or, to be more specific, determine the per- centage of the students who reached an acceptable level of achievement for each objective. In evaluating efficienpy, there are two aspects that require attentiOn. One is a measurement of student performance. . . . This measurement is the ratio of the number of objectives a student achieved to the time he took to achieve them. . . . The other aspect of efficiency is that of cost. . . . This cost structure should consist of two parts: (1) developmental costs of planning and pilot try-outs, and (2) gperational costs incurred during actual implementation. '(44397) At this point it can be concluded that, while authors use different terms to label the various types of evaluation they believe should be incorporated into an instructional program, a number of types consistently receive attention. They are: Pre-assessment to deter- mine where instruction is needed and to determine whether a student possesses the necessary entry skills to effec- tively participate in the program; Formative evaluation to determine (1) the student's mastery of appropriate skills (enabling objectives) at particular points during the instructional process and (2) the quality of the contribution to student learning being made by particular instructional procedures incorporated into the program; 182 and Summative evaluation to determine (1) the student's ability to perform as desired (terminal objectives) at the end of the instructional program and (2) the quality of the contribution to student learning made by the entire instructional program. Improper Completion of Evaluation While there is wide recognition of the various types of evaluation which ought to be incorporated into an instructional program, several authors point out that evaluation is often not carried out at all or, if it is, it is not designed and implemented prOperly. Bass and Vaughan note that, ”Generally, training programs are designed with little or no thought as to how they will be evaluated” (5:139). Goldstein makes similar comments a number of times in his text. He notes: Unfortunately, few programs are evaluated. Indeed the word evaluation raises all sorts of emotional defense reactions. In many cases, the difficulties seem related to a failure to under- stand that instructional programs are research efforts that must be massaged and treated until the required results are achieved. (36:23) The complexities of evaluation should not be underestimated; however, the most serious problem has been the failure to even consider examining the instructional methods. (36:51) Few investigations have bothered to measure learning, behavior, or results, and those that have done so rarely stress proper evaluation procedures (for example, control groups). It is probably not unreasonable to suggest that these investigations also do not consider criterion relevance and reliability. (36:61) Unfortunately, the feedback process that could result from effectively designed evaluations has 183 been likely to lead to emotional reactions rather than to decisions to use the information to improve the program. (36:213) Catalanello and Kirkpatrick also note the lack of proper evaluation. They state: The evaluation "state of the art" is still in its infancy. . . . As we consider the more important and difficult steps in the evaluation process (i.e., learning, behavior, and results), we find less and less Being done, and many of these efforts are superficial and subjective. (15:9) Goldstein draws upon the views of various researchers to examine the reasons that proper evaluation has not occurred. He notes that: 1. There has been considerable difficulty in finding acceptable criteria (MacKinney, 1975). This problem becomes more serious as researchers attempt to measure the achievement of organi- zational objectives. However, as we shall see, the measurement of behavior in any setting is difficult. 2. There is a serious lack of personnel trained in the methodology of evaluation. Guba (1969, p. 37) quotes a director of a research and development center. We are having trouble finding people . . . with sufficient sophistication so that they can help with technical problems. We need an evaluator interested in measur- ing change, who is statistically competent and has all the characteristics of a stereo- type methodologist in evaluation but who has a willingness to look at new kinds of problems. 3. Wallace and Twichell (1953) have lamented the difficulties in establishing meaningful relation- ships in industrial settings. Guba (1969) noted that school evaluation studies are fre- quently incapable of securing any significant information. Studies of different alternatives most often find no statistically significant differences, and, even when differences are established, researchers are uncertain about the variables that determine the effect. 184 4. The personnel responsible for training and edu- cational research are often not responsive to the need for evaluation or are fearful of the entire process. In some cases, management is reluctant to expend effort to evaluate a program that it considers to be more than adequate (Wallace & Twichell, 1953). In other cases, the training or educational director is afraid of evaluation because, if his program were found to need modification, it might jeopardize the continuance of the program as well as his position as director (Howell & Goldstein, 1971). The latter view assumes that training programs that are not immediately successful will be dissolved; the theory that training programs should be continually evaluated in order to modify and improve the product is not recognized. (36:50) Sullivan explains that evaluation efforts often fail to provide useful assessment of the instructional program's quality because the results achieved from the evaluation of factors other than the accomplishment of desired ". . . post-instructional learner outcomes . . ." (89:66) are used to judge its effectiveness. In many cases, the correct use of a particular technique (method or material) or the level of student enjoyment will be evaluated and, if positive results are reported, there is a tendency to conclude that the program has achieved desired performance outcomes. While it is appropriate to evaluate the value of particular techniques or to assess student enjoyment, Sullivan stresses the inappro- priateness of concluding that positive results in these areas means that desired learning has occurred. He explains: 185 One common error in the evaluation of school programs results from the tendency of many educators to treat the content of the program as the most important criterion for evaluation. The content of an educational program is simply the materials and methods employed by the teacher. . . . Although it should be apparent that the content of a program is an educational means, not a goal in itself, it is not always treated as such. From time to time, certain methods and materials, often classified as innovative, become cherished in educational circles. . . . The popularity of these programs and program components is often based more upon some sort of intrinsic appeal or other elusive factors than upon empirical evidence of their effectiveness. Never- theless, teachers and educational programs are often evaluated on the basis of whether or not they employ certain favored methods and types of materials, and little attempt is made to determine the effective- ness of this content in improving learner performance. The presence or absence of discovery procedures, individualized instruction, multi-media materials, or a multi-sensory approach clearly is not an appropriate criterion for evaluating instruction. A second inappropriate basis for evaluation involves desired learner behaviors that occur under certain classroom conditions but do not represent actual behavior changes in the learner. . . . It is so reinforcing to see students happily involved in a classroom activity that it is tempting to conclude that the instruction must be highly suc- cessful. Yet, with the many desirable instructional outcomes essential to the learner's "happy involve- ment" and success in later life, it seems shallow to consider as important sources of evaluation in- class behavior or conditions which do not represent new learning for the child. The student who spends a happy, involved, self-expressive educational career in the classroom but fails to acquire basic reading and mathematics skills . . . is a sad product of an educational system. (89:66) In short, Sullivan maintains that evaluation efforts often are designed to assess one aspect of the instruc- tional process but the results acquired are inappropri— ately used to make decisions regarding the program's effectiveness in achieving desired instructional outcomes. 186 Bass and Vaughan note that because of the poor design of evaluation processes or confusion over what gathered information really means, ". . . we never know what a particular program accomplished" (5:140). They explain why they believe this is true: Usually, the criterion of accomplishment is a statement by the trainees indicating whether they think they learned something; less often, the criterion is based on whether the trainees' supervisors think they learned something and seldom on how much trainees actuall learned. In industry we employ many unevaluated techniques, some of which, like job rotation, are remarkably costly; yet we spend relatively little time and money in actual research on their training effec- tiveness in comparison with less costly approaches. (5:140) Factors for Inclusion in Evaluation Processes While it may be reasonable to conclude that proper evaluation of instructional programs seldom occurs and, consequently, the designer often is not able to determine what has been accomplished, many authors suggest a number of factors which they believe should be incorporated into evaluation processes in order to insure that they are effective. A designer's commitment to evaluation and his/her efforts to incor- porate these factors into the design and implementation of evaluation processes will help insure that the instructional program will not suffer from the defi- ciencies noted above. 187 Stufflebeam provides an overall "Structure of Evaluation Design" (88:9). Introducing the structure, he explains that once the program designer has selected the type of evaluation (". . . context, input, process, or product . . ." [88:91) which must be undertaken: . . . he must next select or develop a design to implement his evaluation. This is a difficult task since few generalized evaluation designs exist which are adequate to meet emergent needs for evaluation. . . . What follows is an attempt to provide a general guide for developing evalu- ation designs. . . . First, one must identify the objectives to be achieved through implementation of the design. . . . Second, . . . one should identify or define the decision situations in the procedure for achieving the evaluation objective. . . . Third, for each identified decision situation the evaluator needs to make a choice among the available alterna- tives. Thus, the completed education design would contain a set of decisions as to how the evaluation is to be conducted and what instruments will be used. (88:9) In the outline presented here, Stufflebeam presents a list of decisions which he believes must be made as the evaluation design is developed. These decisions are broken down into six parts. They are ". . . (l) focusing the evaluation, (2) information collection, (3) infor- mation organization, (4) information analysis, (5) infor- mation reporting, and (6) the administration of the evaluation" (88:9). Developing Evaluation Designs The logical structure of evaluation design is the same for all types of evaluation, whether con— text, input, process or product evaluation. The parts, briefly, are as follows: 188 Focusing the Evaluation 1. 2. 4. Identify the major level(s) of decision- making to be served, e.g., local, state, or national. For each level of decision-making, project the decision situations to be served and describe each one in terms of its locus, focus, timing, and composition of alterna- tives. Define criteria for each decision situation by Specifying variables for measurement and standards for use in the judgment of alterna- tives. Define policies within which the evaluation must operate. Collection of Information 1. 2. 3. 4. Specify the source of the information to be collected. Specify the instruments and methods for collecting the needed information. Specify the sampling procedure to be employed. Specify the conditions and schedule for information collection. Organization of Information 1. 2. Specify a format for the information which is to be collected. Specify a means for coding, organizing, storing, and retrieving information. Analysis of Information 1. 2. Specify the analytical procedures to be employed. Specify a means for performing the analysis. Reporting of Information 1. 2. 3. 4. Define the audiences for the evaluation reports. Specify means for providing information to the audiences. Specify the format for evaluation reports and/or reporting sessions. Schedule the reporting of information. Administration of the Evaluation 1. 2. 3. 4. Summarize the evaluation schedule. Define staff and resource requirements and plans for meeting these requirements. Specify means for meeting policy require- ments for conduct of the evaluation. Evaluate the potential of the evaluation design for providing information which is valid, reliable, credible, timely, and pervasive. 189 5. Specify and schedule means for periodic updating of the evaluation design. 6. Provide a budget for the total evaluation program. (88:10) Stufflebeam maintains that if an approach such as the one he outlines is followed, evaluation processes will be properly developed and will include the components necessary to insure their effectiveness. Bass and Vaughan suggest that the following three ”. . . General Principles of Evaluation" (5:144) should guide the development of evaluation processes: 1. Evaluation should be planned at the same time as the training program and should constitute an ihtegral part of’the total program from gaginning to end. . . . (53l44) 2. EvaluatiOn should follow the most rigorous experimental design possiBle. . . . (5il45) 3. Evaluation should he carried out at several levels and at several times. . . . (5:147Y Several authors recognize that a critical factor to be incorporated into properly developed evaluation processes is the assessment of whether each participant has learned the skills and abilities necessary to perform in accordance with the criteria stated in the instruc- tional program's objectives. As Popham and Baker note, ". . . the ultimate test of whether an instructional sequence works is exclusively a function of the learner's final attainment of goals" (73:77). As one prepares to evaluate a student's ability to perform as expected at the end of the instructional program, the value of well- written objectives again becomes apparent. As Gagné states: 190 An instructional program has the aim of establish- ing the capability for certain kinds of behavior; the learner must be able to do something after com- pleting the instruction that he could not do before- hand. To know whether a program has fulfilled such an aim, it must be possible to observe, or in a more refined sense to measure, this post-learning behavior. Here also, then, is a reason why the objectives of the instruction must be specified in terms which imply reliable observability. Whatever capability of the learner cannot be specific in such terms cannot be measured. (28:24) Davis et al. note: Objectives play a critical role in instructional evaluation. They lay a foundation for the evalu- ation and provide the primary criteria used to judge both student achievement and the success of the instructor. (21:12) Similarly, Sullivan states: Two widely claimed advantages for the statement and use of behavioral objectives in curriculum and instruction are: (1) they enable the teacher to know exactly what behaviors the learner should be able to perform as a result of instruction, and consequently facilitate the selection of materials and activities to develOp these behaviors; and (2) they permit valid assessment of whether or not students have acquired desired post-instructional behavior, and thereby also indicate the effective- ness of the instruction. (89:69) As discussed in the third section of this chapter, a well-written objective can effectively contribute to the evaluation process when it possesses the criteria or standard of performance which the student must be able to achieve in order to demonstrate mastery of the desired skills and abilities which he/she is expected to learn during the instructional program. In addition, the cri— teria for success describes the conditions under which performance is to occur and the lowest limit of performance 191 acceptable in order to demonstrate mastery of the objec- tive. The existence of clearly stated performance cri- teria in each objective is a critical factor in determin- ing the designer's ability to evaluate student mastery of desired skills and, in turn, the effectiveness of the instructional program. The evaluation plan outlines how the instructional designer intends to arrange particular methods and materials in order to assess a student's ability to perform in accordance with the criteria stated in the objective. Mager and Beach offer two guidelines for the preparation of an evaluation test to determine if the student is able to perform in accordance with stated criteria: 1. Use the objectives as your guide. Prepare as many items as necessary to find out how well the student meets each objective. In some cases, only one item is appropriate. . . . In other cases, you may feel that several items are needed to make an assessment. 2. Create items that call for the same kind of behavior specified in the objective. If an objective calls for a student to use a certain tool, then create test items that cause him to use the tool. In such a case, it would not be appropriate to ask him to write an essay about the use of the tool or to answer multiple-choice questions about the use of the tool. If an objective calls for an ability to repair some- thing, then the appropriate test item is one that asks the student to repair. Again, multiple-choice items are not appropriate. If an objective asks the student to be able to talk about something rather than do it, then an oral item or essay item is appropriate. (60:40) 192 In order to insure that proper criteria are selected for measuring a student's achievement of desired objectives and, consequently, the success of the instructional program, a number of other important factors should be considered. Goldstein states that one of the factors is "CRITERION RELEVANCY" (36:53). He notes: Relevancy can . . . be thought of as a relationship between the operational measures (criteria) and the true values that will hopefully be represented. The true values are sometimes referred to as ultimate criteria, because they represent a com- plete array of the aspects that determine success. In a sense, they are the final goals of a training or educational program. (36:53) From Goldstein's point of View, the ultimate criteria for the success of an instructional program is the degree of positive correlation between a participant's ability to perform as desired at the end of the program and the contribution such performance makes to the accomplishment of desired organizational goals. The actual criteria are the measures of success stated in each program objective. Goldstein explains that "The degree of over- lap between the actual and ultimate criteria determines the degree of relevance" (36:53). That is, criteria selected for inclusion in the objective are increasingly relevant as they approximate the criteria for positive contribution in meeting desired organizational goals. Actual criteria can be made more relevant by reducing ". . . criterion deficiency and criterion contamination" (36:53). Goldstein states: 193 Criterion deficiency is the degree to which there are components in the ultimate criteria that are not present in the actual criteria. The more learned through need assessment about the components that determine ultimate success, the easier it will be to identify more immediate criteria that provide for the measurement of all the required behaviors. (36:54) [Criterion contamination] . . . pertains to extraneous elements present in the actual criteria that are not part of the ultimate criteria. The existence of these elements that contaminate the criteria can lead to incorrect conclusions regarding the validity of the training program. (36:55) MacKinney explains relevance as follows: . . . criterion is a measure of proficiency on the job, a measure that tells us whether the performance of an individual or group is effective. . . . "Relevance” is roughly equivalent to quality. A relevant criterion is a good criterion, one that accurately reflects the contribution of the group or individual to the organization and does not contain any extraneous factors. (57:75) MacKinney notes that criteria which assess the contribution that successful performance of the desired skill will make in achieving the organization's goals is most rele- vant whereas opinions about the training experience itself are apt to be least relevant. Validity, a factor referred to in the second guideline cited above by Mager and Beach, is also a vital characteristic of effective evaluation processes. Evaluation tests must be valid if the information gathered through them is to be used to determine that a student has mastered skills in accordance with per- formance criteria presented in the objective. Davis et al. explain that "A test is valid when it requires 194 the learner to perform the same behavior under the same conditions specified in a learning objective" (21:86). They note: A learning objective states the behavior the learner is expected to demonstrate as a result of instruction and the conditions under which he will perform that behavior. In effect, an objec- tive describes the achievement test. Therefore, in order to obtain a valid measure of achievement, the test must be designed so that it presents the conditions and requires the behavior specified in the objective. (21:86) Briggs explains that: A test is valid if it measures what it is supposed to measure--in the present context--if it measures the objective for which it is intended. It is par- ticularly important to examine the verb in an objective to see if the kind of behavior or kind of student performance identified in the does what part 3f the objectitetis actually measured By the test. If the objective says, ”Without references, the student will correctly describe two of the three standard processes for the commercial production of steel," then it is clear that the only relevant kind of test would be an oral or a written essa test requiring him to correctly describe two 0 the three processes taught. No true-false nor multiple-choice test would measure the performance, "describe." Nor would it be valid to have the learner check which of six printed descriptions of processes are the correct ones, nor to have him set up a laboratory demonstration of the processes. It would also be an invalid test to ask the learner to describe how to manufacture brass objects or silver nitrate. Both the verb (descrihel and the dbiect (steél) must be correctly used in the test or It to be valid. (11:48) Another factor which must be included in a quality evaluation is "CRITERION RELIABILITY" (36:57). Goldstein explains: Reliability refers to the consistency of the cri- teria measures. If the criteria are ratings of performance, and there is little agreement between 195 two raters, then there is low reliability. Cor- respondingly, consistently different performance scores by the same individual at different times also reflects low consistency and thus low relia- bility. (36:57) Briggs notes that "A reliable test gives a stable, ade- quate measure of whatever it does measure. . . . [There- fore, it is] . . . important to test the student thoroughly enough to be satisfied that resulting scores accurately reflect his true ability to perform on the objective" (11:58). Davis et al. indicate that "A reliable test provides a consistent measure of a learner's ability to demonstrate achievement of an objective" (21:89). They explain that: A student demonstrates achievement of an objective by performing the behavior according to the standards specified. If the instructor provides several opportunities for the student to do this and the student performs at the minimum standard most of the time, then he can be reasonably con- fident that the student could perform at that level from that time on; thus, the test would pro- vide a consistent measure of the student's ability to perform the objective. (21:89) Davis et al. explain that there are many other factors which can affect the student's performance during a particular testing situation other than his ability to meet the criteria presented in the objective. They note that: Some of these, like fatigue, tension, or his love life, are beyond your control; however, the effect of other factors which are under your control should be reduced if you are to obtain a stable [reliable] measure of performance. Some of these factors are: 196 Unclear instructions Ambiguous test items Test conditions that are different from those stated in your objective 0 Use of jargon words, the meaning of which has not been taught o A new kind of test that the student has never experienced 0 Extreme levels of temperature and humidity in the testing room 0 Raising student anxiety needlessly, e.g., "I don't expect many of you to pass this test!" If such factors are allowed to operate, the test will not provide a reliable measure of student achievement. (21:90) The designers must be sure to develop an evaluation plan which takes factors like these into consideration so tests or testing situations do not have an adverse effect upon a student's ability to demonstrate what he/she has learned. In addition to being relevant, valid, and reliable, Briggs suggests that the evaluation plan should provide for efficient and practical testing. He explains: Efficient use of evaluation time is important in obtaining the most valid and reliable evaluation data per student per unit of time for test admin- istration and scoring. One aspect of efficiency is to design as short a test as possible which yields validity and reliability. . . . Practicality is determined by considering all factors relating to time, effort, space, or equipment needed to administer, score, and interpret tests. (11:60) Another factor which the designer should take into consideration when develOping evaluation processes is the degree of rigor necessary to assess the effective- ness of the instructional program. While the student's SC V96 (Ir nua' tic imp POS: of a stei affe fOr', ShOuJ of C r190 of i_ the mete 197 ability to respond to evaluation tests in accordance with the performance criteria stated in the objective is a critical measure of the program's success, the validity of such a conclusion can be strengthened by increasing the rigor of the evaluation design. Rigor is a term used to refer to the degree of control (rigidity) exer- cised through the design of evaluation processes in order to minimize threats to the validity of the results so that the evaluator can reasonably conclude that they were produced by the treatment (instructional procedure or program), not some other factor(s), e.g., student maturation or an event occurring outside the instruc- tional environment. The importance, as noted by Bass and Vaughan, of implementing the ". . . most rigorous experimental design possible . . ." (5:145) when undertaking the evaluation of an instructional program is stressed by others. Gold- stein notes that since ". . . the rigor of the design affects the quality and quantity of information available for evaluation" (36:24), the evaluation planner ". . . should employ the design that has the greatest degree of control over threats to validity" (36:78). The more rigorous the design, the greater the quality and quantity of information collected for use in judging the effec— tiveness of the instructional program. Goldstein further notes that: than c with 1 many e Proce Cont] dent: 198 There are situations in which it is not possible to use the most rigorous design because of cost or because of the particular setting. In these cases, it is important to use the best design available and to recognize those factors that affect the validity of the information. (36:25) It is clear that some designs are more rigorous than others. Noting that the degree of rigor varies with the type of design, Bass and Vaughan state that many evaluation designs . . . fall in the class of "after-only? evalu- ations-~that is, the trainee's béhavior is observed or measured only after his exposure to training. A slight improvement over this single ”after" measure of the training group is a comparable measure of a matched control group. A significantly higher level of scientific rigor is reached when evaluation includes "before- after" measures of the experimental (training) group and a matched control group. The use of two or more control groups permits even more SOphistication in design. . . . (5:145) Similarly, POpham and Baker (73:152) and Goldstein (36:78) present various types of evaluation designs and indicate that the degree of rigor and, therefore, the quality of the information produced will increase if the design does not rely on a simple measurement of the performance of program participants at the end of the instructional process and instead incorporates pre/post-testing, control groups, and randomized selection of those stu- dents to be evaluated. Goldstein states: Certainly, it is possible to avoid choosing a use- less design. In many cases, the main difficulty has been the failure to plan for evaluation before the [instructional] program was implemented. In which train ,\23 rrra-fi-crasuo rurac+'£lr+:> Notin expla I'M hl gmfim ”Fr ~- A); 199 these instances, the utilization of a few procedures-- for example, pre/post-testing and control groups-- could dramatically improve the quality of information. (36:78) In discussing various types of evaluation designs which could be used to assess the effectiveness of a training program, MacKinney maintains that: At the top of our hypothetical scale is the con- trolled experimental study. This is a desigh_in which two groups are used, the one to receive training, and the other to act as a control group. The procedure is as follows: (1) a "before" pro- ficiency measure is taken for both groups; (2) one group is trained while the other is left on the job; (3) an "after" proficiency measure is taken for both groups; (4) the "before" is subtracted from the "after" for both groups to measure the gain in proficiency. If the training did any good, the trained group should have gained significantly more in job proficiency than the untrained group. (57:74) Noting that other designs are less effective, MacKinney explains that: At the second level in this classification sys- tem is the evaluation of training by means of the trained group only. This is not an acceptable experimental design. It must be admitted, however, that this type of evaluation is widely used and is perhaps better than no evaluation at all. . . . The evaluation design is a simple one. "Before" and "after” measures are compared and any gain is attributed to the training. . . . At the lowest level, we have the evaluation of the trained group only, as above, with the criterion measure taken after training, but not before. . . . All that can be done is look at the information gathered after training and make some sort of guess as to whether it is "good enough." (57:75) Davis et al. explain that one instructional pro- gram can be compared with another in order to determine which is most effective and efficient. They explain that: 200 In evaluating two learning systems: 1. The learning objectives should be the same for both. In this way, the same criteria will be used for comparing the effectiveness of the two systems. 2. The average entry skill level of the students in both systems should be equivalent. This can ,be achieved by giving the terminal test to all students before instruction begins. Students with equivalent scores should then be paired and different members of each pair assigned to each of the systems. 3. At the end of instruction the systems should be compared with regard to all of the following: student achievement, efficient utilization of student and instructor time, cost of designing and operating each system, and student attitudes. (21:115) The instructional program designer must be pre- pared, within the limits of cost or the characteristics of a particular environment, to select the most rigorous design possible so that the information gathered can be used to determine if the instructional program achieved the desired results and whether these results were attributable to the program. As an instructional program designer strives to develop evaluation processes which are valid, reliable, practical, and rigorous, he/she must keep in mind that such processes are not developed for use solely at the end of the program to evaluate student achievement of desired terminal objectives. While this may be the single most important assessment which must take place, since the program is undertaken to insure mastery of those performance criteria stated in the objective, evalua tional needs plishI ment « incre. achie not bi desir tiona impor be in will decis inst: ing t atioj inst that tion inst 201 evaluation plays a critical role throughout the instruc- tional program. In fact, if such forms of evaluation as needs assessment, determination of entry skills, accom- plishment of enabling objectives, and formative assess- ment of instructional procedures do not occur, the odds increase that the instructional program will never achieve its goals and, therefore, positive results would not be achieved when evaluation of student mastery of desired terminal objectives did occur. When instruc- tional programs are designed and implemented, it is important that prOperly developed evaluation processes be incorporated wherever and whenever their presence will help collect information which will assist in making decisions related to the proper development of the instructional program and whether the program is achiev- ing the desired results. As has been noted in this section, often evalu- ation processes are not prOperly incorporated into instructional programs. Bass and Vaughan point out that this is much less apt to happen when the instruc- tional program itself is positively received within the institutional setting which it is designed to serve. In general, good training conditions produce good conditions for making an evaluation. That is, when training objectives have been clearly defined and related to company goals and when management is actively committed to the program, then training may be carried out under very favorable conditions. And likewise, the evaluation of training can proceed 202 in a clear and unclouded atmosphere, free of secret strategy, and the information needed for evaluation can be collected much more freely. The opposite is true when training objectives are not clear or when management does not support the training pro- gram. (5:144) Just as he/she must take specific steps to write clear objectives and select prOper instructional procedures, the designer must seek and secure the support of top level decision makers within the organization or insti- tution which the instructional program is designed to serve. Their support helps insure that the designer will have the resources necessary to design the best possible instructional program, that its implementation will move forward as planned, and that evaluation pro- cesses can be carried out so information is available to make necessary modifications in the program's design as well as to assess the achievement of desired outcomes. Redesign The instructional program would be totally successful if, upon completion of the various types of evaluation discussed in the previous section, assessment indicated that each participant performed in accordance with criteria presented in the objectives. It, there- fore, could be concluded that the program had been properly designed and implemented. While instructional programs are expected to strive for and to reach this level of excellence, as programs are being developed 203 and, once develOped, as variables in the instructional environment change, it is likely that it will be neces- sary to redesign various elements of the program in order for it to achieve and maintain desired excellence. For example, if the needs of the organization which the instructional program is being designed to serve change, performance criteria of the objectives must be redesigned if the program is to continue to effectively meet the organization's needs. In turn, this change may require redesign of the evaluation processes develOped to test student achievement of the objectives. From another point of view, if a designer is attempting to develop an evaluation process to assess participant achievement of desired objectives, and he/she is unable to determine the level of performance which must be achieved in order to make such a determination because the desired per- formance criteria is not stated clearly in the objective, the designer must redesign the objective so that such criteria is included and, therefore, evaluation can take place. Having implemented evaluation processes, the instructional program designer is prepared to interpret the information gathered, identify program design and implementation problems, and determine if and where modification (redesign) of various elements is necessary in order to improve or update the instructional program 204 so that desired results are achieved. To examine the redesign element of the instructional program, the views of selected authors about (1) the purpose and value of redesign, (3) the need to interpret and apply data, (3) the positive role of prototype programs, and (4) the variety of redesign actions which may be necessary are considered here. The Puppose and Value of Redesign Various authors label the redesign element of the instructional program differently. However, whether labeled redesign (21:322), revision (43:135), review and revision (5:84), improvement phase (60:6), or quality control (4:79), the authors recognize the purpose and accept the value of this element of the instructional program design process. Kaufman explains: Based on the performance of the system as indicated by the performance data, any or all previous system steps may be modified and a system redesign job accomplished if necessary. This self-correctional feature of a system approach assures constant rele- vance and practicality. An educational system is never considered to be complete, for it must be constantly evaluated in terms of: 1. Its ability to meet the needs and requirements it sets out to respond to. 2. The continued appropriateness of its original needs and requirements. Thus we must have not only internal consistency and performance, but constant checking of needs and requirements to assure external validity as well. (43:23) Relating redesign to the type of assessment referred to as formative evaluation by various authors 205 cited in the previous section of this chapter, Kaufman notes that ". . . any time the interim or in-process objectives are not being met, necessary revisions may be made" (43:135). He further explains that revision is ". . . continuous and ongoing . . ." (43:135) and that "This self-correctional feature is the element which assures that the needs will be eventually met" (43:135). Kaufman notes: The critical tool involved in revision is the requirement that the process information be systematically and periodically reported to the decision maker so that necessary corrective action may be taken. It is also important to recall that the revise-as-required step is to take place throughout system planning and imple- mentation. Using this "formative“ evaluation, there is a constant check on utility and an attempt to make the system responsive. Thus the approach is not a rigid, unyielding experi- ment, but rather a "people-centered,” flexible process for meeting human needs. (43:135) Kaufman encourages the designer to recognize that ". . revision is not just redoing for its own sake . . ." (43:141) but that the revision element is an ongoing attempt to make, when necessary, ". . . logical and planned . . .” (43:141) changes in the instructional program so that the program will eventually achieve its desired results. He believes that, when changes are necessary, they should not be interpreted as a failure in the design of the program but as a clear recognition that, particularly during formative periods, such mod- ifications must be a critical part of program 206 development if success is to be achieved. Kaufman indi- cates that the inclusion of a revision element in the design process ". . . provides educators with the right to fail and the ultimate obligation to succeed . . ." (43:135). Goldstein stresses that, in order to insure that a designer will use relevant data gathered during evalu- ation processes to redesign and, therefore, improve the quality of the instructional program, it is necessary to think of the program as a ". . . closed-100p system . . ." (36:25). He explains: A training program should be a closed-loop sys- tem in which the evaluation process provides for continual modification of the program. An open- loop system, in contrast, either does not have any feedback or is not responsive to such information. In order to deve10p training programs that achieve their purpose, it is necessary to obtain the evaluative information and to use this information for program modifications. . . . Even in those instances in which the training program achieves its stated objectives, there are continual developments that can affect the program including the addition of new media techniques and changes in the characteristics of trainees. These changes often cause previous objectives to become obsolete. The development of training programs must be viewed as a continually evolving process. (36:25) Evaluation must be treated as an information- gathering process that cannot possibly result in decisions that categorize programs as all good or all poor. These evaluation attitudes will be established only if it is clear that instructional programs are never complete but, instead, are designed to be revised on the basis of information obtained from evaluations. . . . (36:213) betwel of vi gram n—un rQ-n S‘ H.u.n.u.|.-J -4 D tha ach at 207 Cronbach stresses the important relationship between evaluation and program revision. From his point of view, revision is particularly valuable when the pro- gram is in its developmental stages. He notes: The greatest service evaluation can perform is to identify aspects of the course where revision is desirable. . . . To be influential in course improvement, evidence must become available midway in curriculum develOpment, not in the home stretch, when the developer is naturally reluctant to tear open a supposedly finished body of materials and techniques. Evaluation, used to improve the course while it is fluid, contributes more to improvement of education than evaluation used to appraise a product already placed on the market. (19:675) Banathy explains that: The purpose of evaluation and quality control is to ensure that the objectives of the system are being met or, if not, that adjustments will be introduced in order to correct the system so that objectives can be eventually attained. This phase of systems development is comprised of several strategies with specific purposes of their own, such as system monitoring, which is used to evaluate continuously the effectiveness of the system, and performance testing, which is a means of measuring the progressive achievement and terminal proficiency of the learner. The continued accomplishment of these two strate— gies provides us with information we can use to carry out appropriate adjustments in order to improve the terminal performance of the learner and to optimize the effectiveness and economy of the system. (4:79) Davis et al. maintain that, in order to insure that appropriate adjustments are made so the program achieves success, the designer must maintain a flexible attitude toward the modification of design plans. They explain: 208 Inflexibility is one distinguishing characteristic of the inexperienced teacher. After preparing a lesson, the novice often goes ahead despite indi- cations that the plans he has prepared are not working. He may not even be aware of the fact that a problem exists. On the other hand, the skillful teacher tries alternative approaches when his original plans flounder. (21:96) Bass and Vaughan maintain that, if a training program is to remain relevant and valuable, it is neces- sary to gather information for the purpose of continu- ously updating the program. They note: "The very essence of organizational life is change; thus a training program must be reviewed constantly and revised in the light of changes in a company's resources, its objectives, its internal organizational climate, and the total environment within which it operates" (5:84). Similarly, Mager and Beach note: Vocations change, new teaching techniques and devices become available, and the average char- acteristics of the incoming student may gradually shift. It is appropriate, therefore, to set in motion a process guaranteeing that the course will always be fresh and up-to-date as this morning's newspaper. (60:6) lpterpret and Apply Information Gathered through EvaluatiOn While it may be clear from comments made thus far in this section, it is important to note that, to be useful in determining action which should be taken to redesign a particular aspect of an instructional program, the information gathered during evaluation processes must be interpreted and applied. Gagné and Briggs note that: 209 . . . various kinds of evidence, collected by means of observational records, questionnaires, and tests, are now employed to draw conclusions as to whether a lesson needs to be kept as it is, revised, reformulated, or dis- carded" (33:235). Similarly, Davis et al. explain: "Once having collected information about student performance, it becomes necessary to interpret that information and decide what steps should be taken to improve the effec- tiveness and efficiency of the instructional procedures used in the course" (21:111). For example, if a student or group of students do poorly in a course, that is, they do not perform in accordance with criteria presented in the objective, Davis et al. suggest that the designer interpret information gathered during evaluation processes in order to seek answers to questions related to: ORGANIZATION OF SUBJECT MATTER Was the objective clear to the students? Were the students tested to determine whether they understood the objective? Were any enabling objectives omitted? Was subject matter sequenced so that prerequisite enabling objectives were achieved before later ones? INSTRUCTIONAL PROCEDURES Review the methods of presenting material to the students. Was adequate direction and context provided? Were a sufficient number of examples provided? Did students have sufficient opportunity to practice? Did students receive feedback so that they could tell how they were progressing? STUDENTS WERE NOT ADEQUATELY PREPARED Some stu- dents may have lacked the necessary entry skills needed to achieve the objective. To check this possibility, the teacher should compare the entry test scores of students who achieve the objective with those students who do not achieve it. This comparison should readily indicate whether or not a lack of entry skill accounts for the failure of some students to achieve the objective. (21:112) The des Dav inf use ind not) inf. rev; def: in 1 nece the Prot ____ deve fini he/s form desfl imp dif “Po red warl inf 210 The answers to questions like these help direct the designer's revisions to the instructional program. As Davis et al. note, “Merely gathering and interpreting information is insufficient. The information must be used to modify instructional procedures when a change is indicated" (21:118). Using questions such as the ones noted here to guide the analysis and interpretation of information gathered through evaluation will insure that revisions which are made will help eliminate problems or deficiencies within the instructional program that result in the participants' inability to learn those skills necessary to meet the performance criteria stated in the objective. Protopypes Prove Valuable Since, as noted earlier by Cronbach, ". . . the develOper is naturally reluctant to tear open a supposedly finished body of materials and techniques" (19:675), if he/she recognizes that instructional programs are in formative stages of development during their early design periods or during the first few times they are implemented, the designer is more apt to find it less difficult to interpret evaluative information and, based upon decisions reached, apply changes through program redesign. While changes must be made whenever they are warranted to maintain program effectiveness and efficiency, initially treating a program as a prototype makes it easi rede deve and that Simi 211 easier to accept the value and appropriateness of redesign and to do so at a time in the program's development when changes are apt to be most frequent and necessary. Because there is a distinct possibility that redesign will be necessary, Kemp suggests that: . . . one or more "try-out" or ”dry run" phrases of instruction should be made before a program is actually used. A program should be tested on a representative sample of the student group under what will be normal conditions. The information obtained (called feedback) from the evaluation of this pilot test may ihdicate that one or more revisions in the plan should be made before it is used with an entire student group. The procedure of trial testing and revision (and possibly retesting and further revision if necessary) is important to the success of a plan. It should relate not only to the suitability of objectives, subject content, learning methods, and materials, but also to the roles of personnel, the use of facilities and equipment, schedules, and other factors that all together affect the optimum performance for achievement of the objectives. (44:40) Similarly, Davis et al. explain: Continuous evaluation procedures are also used to test a new learning system while it is being developed. By testing a prototype of the new system at an early stage of development, the designer may identify and correct deficiencies as they arise. The prototype should be evaluated using a representative group of students. Both achievement tests and measures of student attitudes should be incorporated into the evaluation plan. (21:103) Gagné and Briggs note that instructional pro- grams are field tested during their ". . . formative stage . . .' (33:225) so that information gathered (formative evaluation) can be used to improve the prog wide this Fie] part prOg envi prot that grar Sim 212 program before it is ". . . installed for regular or widespread use" (33:225). They explain that during this formative stage: . . . often small portions of the new system are tried out with a few individual learners in a "one-to-one" situation. The designer observes the student at work, records the questions he asks or the comments he makes, and analyzes test responses in order to spot weaknesses in the program. These data are used to revise the materials or procedures, or both. Following the individual tryouts, small group tryouts of the system are held and further revisions are made. Then "field tests" of parts or all of the system are made in the environment for which the system is intended; that is, these tryouts are made with normal-sized groups under "actual" con- ditions. (33:225) Field testing continues and revisions are made until the participants perform as expected. At this point, the program is ready for use in the intended instructional environment. The value of prototype testing of instructional programs becomes clearer when one considers the impact that redesign within one element has on the entire pro- gram. In explaining this impact, Kaufman notes: Whenever a revision is made, it should be carefully verified that all the interactions (or consequences) of the change have also been accounted for. . . . Remember that any change in a dynamic system has implications for all the other parts. (43:141) Similarly, Davis et a1. state that: . . . no system component or procedure can be modified without having an effect on other compo- nents or procedures. Consequently, during the redesign process, when the designer considers modifying one aspect of the system, he must analyze 213 the possible effect of the proposed modification and determine other changes in the system that are required. . . . Many efforts to modify and improve educational systems have floundered because they were directed at changing one component of the system without considering the effects of that change on the other components. (21:323) Since redesign decisions can have a broad impact upon all the elements of the instructional program design process, completing as many revisions as possible during the period of prototype testing will reduce the number of students affected by these changes and increase the quality of instruction in the ultimate instructional setting. Additionally, the degree of impact which a redesign decision has upon the various elements of the instructional program is apt to be less discouraging to the designer since he/she is prepared for such change during these initial develOpment periods. A Variety of Redesign Actions May Be Necessary Having examined the purpose and value of redesign as well as the valuable contribution prototypes can make in facilitating the completion of redesign processes, the following examples will illustrate the variety of redesign actions that an instructional program designer might need to consider as he/she strives to improve the quality of the program's impact upon the participants' ability to perform in accordance with the criteria out- lined in the objectives and as he/she seeks to modify 214 the program in response to changing conditions in the instructional environment. Briggs suggests: If you seek the simplest, least expensive way to improve an ongoing course, without abandoning present materials, without redefining the role of the teacher, and without a need to retrain teachers, you may find some fairly simple changes can often improve a course, though large improvements may require a more complete overhaul of present pro- cedures. For minimum improvement efforts, the following suggestions are offered: 1. Be sure your objectives are clear to you and to the students. Tell or show the students what you want them to be able to do and tell them that that is the basis for evaluating their achievement. 2. Reexamine your present materials and methods for relevance to the activities described in your objectives. Ask, "Am I teaching what I described, or something else?" 3. Reevaluate your testing, grading, and evaluation procedures. Are you testing for your stated objectives? Or, are you stating a problem- solving type of objective, teaching simple concepts, and testing rote memory of facts? (11:178) Citing the desirability of revising objectives until they are written in clear behavioral terms, McAshan states that ". . . it is a real challenge for any teacher, program planner, or project director to revise and re- state behavioral objectives to the point that there is no doubt concerning their appropriateness, clarity, or specificity" (54:65). Elaborating on a similar perspec- tive, Tyler states: . . . as you work with objectives and with your efforts to teach them you frequently have a basis for the re-definition of your objectives. As you see what really is possible, you may see more clearly the kinds of things the pupils need in addition to those that you thought of in your original planning. The process of clarifying 215 goals, then working toward them, then appraising progress, then re-examining the goals, modifying them and clarifying them in the light of the experience and the data is a never-ending pro- cedure. (94:96) Kemp notes instances when objectives may have to be changed: You should be aware that in any program there may be times when objectives need modifications as the unit or course proceeds. You may have misjudged student preparation and readiness for pursuing an objective, or you may discover a new area of importance during discussion or student study that should be investigated. In either case, be flexible. Revise an objective or add a new one as student needs indicate. (44:32) Mager and Beach cite a variety of reasons why objectives may have to be revised. Jobs change, and sometimes they change rapidly. Computer programming, for example, is a course that needs revision almost monthly if it is to keep up with the world. New tools become available, new techniques are introduced, new information must be mastered, and new environments appear. The vocational educator, probably more than anyone else, is painfully aware of the ways in which jobs change. And for just this reason, he needs to make periodic checks on the relevance of his course objectives. (60:71) The instructional program designer might also have to consider redesign action due to changes in the referent situation. Davis et a1. explain that: The referent situation should be examined to see if either the performance requirements or the conditions have changed sufficiently to require a modification in learning objectives. One method of determining whether or not objec- tives should be modified is to collect information about how well students who have taken the course succeed in the referent system. An analysis of the difficulties they encounter, the extent their 216 success is related to their achievement in the course, and the kinds of knowledge and skills they are required to learn on the job, will provide some indication of the direction in which the objectives of the learning system should be changed. (21:118) Sullivan explains that, rather than changes in the objectives, the designer may wish to make other revisions when students do not perform as expected: The most important purpose of formative evaluation in a carefully planned instructional program is to indicate the desired outcomes that learners do not acquire at an acceptable performance level. The teacher or curriculum developer . . . is then able to design, implement, and evaluate potential improvements in the instruction related to these objectives. (89:82) Mager and Beach suggest that other redesign actions will have to be taken based upon student mastery of prerequisite skills. They explain: When students come into a course lacking various prerequisite skills or knowledge, the normal response is to provide remedial instruction. But students also enter a course knowing more than the prerequisites assume. In this case, remedial action should be applied to the course itself so students will not be bored by being taught what they already know. (60:39) Popham and Baker offer a list of redesign actions which might be implemented. They note that if the instructor . . . has produced the intended change in the learner, at a level approaching what he had previously determined as a performance standard, he should seek ways to make himself even more efficient. He might raise the criterion levels that he originally set, either by requiring more of individual students, or by expecting more stu- dents in the class to meet the minimal level, or by doing both. He may wish to add more objectives mi th 217 to his course or to substitute more complex objec- tives for simple ones. At any rate, his task is pleasant and he can be somewhat at peace. The teacher who has poor results on tests has bigger problems. First, he must resist the desire to place the bulk of the responsibility for the unhappy outcome on the students. The fact that unsatisfactory performance occurs is more often an indictment of the teacher's poor planning or imple- mentation of instruction (or both) than it is the student's fault. Once a teacher accepts responsi- bility for his students' performance, he can begin to find ways to improve. Among the problems that might have adversely affected pupil learning are the following; hypothetical actions that may help alleviate these problems are given for each: Problem 1: Prerequisite behaviors not measured. Action: Modify pretest to include prerequisite behaviors; give remedial instruction where necessary. Problem 2: En route behaviors not achieved. Action: Have more frequent learner-performance assessments and reteaching cycles. Problem 3: Sequence incorrectly analyzed. Relevant prerequisites and en route behaviors omitted. Order of instruction inappropriate. Action: Reanalyze objectives. Vary the compo- nent behaviors practiced or the order in which instruction occurs or both. Use pupil data, post-tests, and quizzes on en route behaviors for clues. Problem 4: Insufficient appropriate practice. Action: Plan for more appropriate practice in general or allow greater proportion of time for equivalent practice. Problem 5: Student attention to task poor. Action: Include more activities designed to reinforce attending behaviors. Problem 6: Test items not representative. Action: Obtain colleagues' opinions of test items; revise the test. Inspect pattern of student responses to determine if certain incorrect responses recur. Check instruction on this point as well. (73:143) Regardless of the type of redesign action which might be called for and the number of elements within the instructional program affected by a particular red des it of mus int ati men dec pet and em: mee PIC 10 218 redesign decision, the important point is that the designer should be prepared to take such actions. Whether it be during the use of a prototype or the implementation of a fully developed instructional program, the designer must make those necessary changes identified through interpretation of information collected during evalu- ation processes. Providing for the proper redesign of various ele- ments of the instructional program insures that those decisions will be made which are necessary to (1) improve a student's ability to master skills required to meet the performance criteria stated in the program's objectives and (2) remain responsive to changes in the instructional environment. To the extent that an instructional program meets these redesign goals, it is reasonable to conclude that the design and improvement of the instructional program will not come to a close but will continue as long as students are enrolled. CHAPTER IV FORMULATE GUIDELINES AND CHECKLIST The preceding chapter dealt with the identifi- cation of those elements which various authors believe should be incorporated into the design and implementation of instructional programs in order to maximize the proba- bility that desired learning will occur and, in turn, that students will be able to demonstrate mastery of the specific behaviors stated in the program's objectives. As illustrated in Figure 8 of Chapter III, there is a significant degree of agreement among those authors regarding the elements which should be incorporated into well-designed instructional programs. These elements are (l) assess needs, (2) deve10p objectives, (3) select instructional procedures, (4) evaluate, and (5) redesign. The discussion presented in Chapter III considers the views of selected authors to describe the role of each element in the design and implementation of an instructional program. Each element makes a distinct contribution to the program's success. To insure that desired outcomes are achieved, it is important that 219 220 program designers understand each element's role in the deve10pment of successful instructional programs and the steps which should be taken to properly incorporate all the elements into the design process. Accordingly, this chapter provides student personnel professionals who are interested in or who are charged with responsi- bility for inservice staff develOpment with a resource tool (the lack of which was established in Chapter I) which has been developed to assist in program design and implementation and to help insure that each element identified in Chapter III has been properly incorporated. Guidelines The discussion of each element identified and presented in Chapter III has been carefully studied and, based upon the views of the various authors presented, a set of guidelines to be followed during the design and implementation of inservice staff development programs has been formulated. It is intended that the program designer will use the guidelines to help insure that each of the five elements recognized as essential to the proper design and implementation of instructional programs is incorporated into the inservice staff development program and, therefore, maximize the proba- bility that the program's objectives will be achieved. eleme ASSE 221 The guidelines are listed below according to the element from which they are drawn: ASSESS NEEDS A. Identify the unit (division, department, staff group) which the instructional program will be designed to serve. Accept the value of and decide to design and implement instructional programs in a carefully planned manner. Insure that support exists or, if necessary, develop support for the implementation of well-designed and relevant instructional programs. Complete organizational and job analyses to determine desired employee responsibilities and performance levels. Assess the employees' current performance level in completing assigned responsibilities. Determine if/where employees' current performance of assigned responsibilities does not meet desired performance levels in order to identify specific performance problems. Determine which performance problems are due to a skill or knowledge deficiency and determine whether employee participation in an instructional program is the most effective and efficient course of action to eliminate these problems. When resources are limited and all problems identified for resolution through the implementation of an instructional program cannot be responded to, develop a priority-setting criteria and select priorities for action. Determine the characteristics of the employees who will be members of the target population which the instructional program will be designed to serve. In light of the target population, for those per- formance problems given priority for resolution formulate general goal statements which give direction to the instructional program planning and design pro- cess and state, in broad terms, the desired outcome of the program. 222 DEVELOP OBJECTIVES A. Recognize and accept the reasons that well-written objectives are important to the development and implementation of an instructional program. Utilizing performance problems given priority for resolution and general goal statements as a guide, clarify and state the desired behavior which the students will be expected to perform at the end of their participation in the instructional program. To insure that achieving desired behavior will be useful and relevant, identify the transfer setting in which the students will be expected to utilize the skills and knowledge acquired through partici- pation in the instructional program; what behavior will be required, under what conditions will behavior be required, and what standard of performance will be expected. Using expected performance in the transfer setting as a guide, write parsimonious, unambiguous terminal objectives to insure that the desired behavior which the students will be expected to master by the end of the instructional program has been carefully defined and can be precisely communicated. Since the desired behavior which the students will be expected to master by the end of the instructional program is often complex, analyze it to determine which intermediate skills and knowledge (behaviors) must be mastered before the desired terminal objec- tive can be achieved. For each intermediate behavior which must be mastered in order to achieve the terminal objective, write a parsimonious, unambiguous enabling objective. SELECT INSTRUCTIONAL PROCEDURES A. So that the procedures utilized during a particular instructional program maximize the probability that mastery of the desired behaviors will result, precede their selection with an analysis of each objective (terminal and enabling) to identify the type of learning which must occur in order for it to be achieved. 223 Once the type of learning has been identified for each objective, arrange the objectives in hierarchical order according to the type of learning so that those requiring a lower order type of learning are placed before those requiring a higher order type of learning. Select procedures for use in the instructional pro- gram which will establish the conditions of instruction within the instructional environment necessary to maximize the possibility that the desired type of learning occurs so the students will be able to per- form in accordance with the behavior stated in each objective. Select instructional procedures which are responsive to the variables found in a particular instructional environment. Select instructional procedures which apply general principles of instruction to the type of learning which must occur in order to achieve mastery of the desired behavior and to the variables found in the instructional environment. Select instructional procedures which will, within the conditions and constraints of the instructional environment, maximize the degree of positive transfer (lateral or vertical) of desired behavior to the transfer setting. Sequence instructional procedures so that objectives which require that a lower order type of learning occur will be mastered first. Select methods and materials which are practical, establish the appropriate conditions of instruction and apply the appropriate general principles of instruction necessary to enable the student to master and demonstrate performance of the desired behavior stated in the objective, and facilitate the positive transfer of desired behavior ot the referent situation. Implement the instructional program in accordance with selected procedures. EVALUATE A. Insure the existence of and, if necessary, develop an environment that is supportive of evaluation. 224 Plan to include evaluation procedures at appropriate points during the design process so they are an integral part of the instructional program from beginning to end and prepare to collect, interpret, and use evaluative data when it will be most valid and useful to the decision-making processes associated with the program. Develop evaluative procedures to assess where per- formance problems and, therefore, the need for an instructional program potentially exists. Develop evaluative procedures to determine the resources available for use in the design and imple- mentation of the instructional program as well as the conditions and constraints in the instructional environment. Develop evaluative procedures to assess whether the students possess the initial or prerequisite capa- bilities necessary to effectively participate in the instructional program. Develop evaluative procedures which will, whenever needed at intermediate points during the program's implementation, provide the students, instructor, and designer with the data necessary to assess student accomplishment of the instructional program's enabling objectives, whether the program is proceeding as planned, and the effectiveness of the instructional procedures. Develop evaluative procedures to provide the students, instructor, and designer with the data necessary to assess student accomplishment of the instructional program's terminal objective and the effectiveness of the instructional program itself. Develop evaluative procedures to provide the students, instructor, and designer with the data necessary to assess whether the students meet performance expec- tations in the transfer setting, the effectiveness of the instructional program itself, and whether the design and implementation of an instructional program is the correct solution to meet the particular per- formance problem. Implement evaluation procedures at appropriate points during the implementation of the instructional program. 225 REDESIGN A. When objectives have not been achieved, resist the temptation to place the responsibility for this failure on the students, accept the fact that there may be defects in the design and implementation of the instructional program, and take responsibility for identifying and taking appropriate action whenever and wherever it is necessary to improve, through redesign, the ability of the program to meet its objectives. When changes are made to improve the instructional program, analyze the impact which redesign within one element has on other elements and make appropriate changes in those elements which are affected. Although redesign must occur whenever and wherever necessary to maintain the efficiency and effectiveness of the instructional program, since changes (redesign) are apt to be more frequent and necessary when a new program is in early design stages or is being imple- mented the first few times, whenever practical, plan to design and implement a prototype program to identify problems and correct deficiencies in the program. If a single type of learning cannot be identified for each objective, evaluate its quality and rewrite the objective as appropriate. When students do not possess the initial or pre- requisite capabilities necessary to successfully participate in and benefit from the instructional program, develop enabling objectives and select instructional procedures which will enable the stu- dents to attain these capabilities. When students do possess initial or prerequisite capabilities which exceed the minimum required to successfully participate in and benefit from the instructional program, advance the starting point for the instructional program. When evaluative procedures cannot be developed because the objective does not clearly state the desired behavior or what performance conditions and criteria are to form the basis for assessing student achievement, evaluate the objective's quality and rewrite it as appropriate. 226 H. When the interpretation of evaluative data indicates that the failure to achieve a particular enabling objective is due to an inadequacy in the instructional program, take corrective action which will redesign the elements of the instructional program at appro- priate points in order to remove problems and improve the program so that defects are overcome and objectives are achieved. I. When the interpretation of evaluative data indicates that the failure to achieve the terminal objective is due to an inadequacy in the instructional program, take corrective action which will redesign the ele- ments of the instructional program at appropriate points in order to remove problems and improve the program so that defects are overcome and objectives are achieved. J. When the interpretation of evaluative data gathered in the transfer setting indicates that the failure of the students to perform as expected is due to an inadequacy in the instructional program, take cor- rective action which will redesign the elements of the instructional program at appropriate points in order to remove problems and improve the program so that defects are overcome and objectives are achieved. K. If terminal objectives are achieved but evaluative data indicate that the needs, expectations, and/or characteristics of the organization the program is designed to serve have changed since the instruc- tional program was originally designed, redesign the instructional program so that it remains up-to- date and reflective of the real needs of the organi- zation if and when it is reimplemented. L. If terminal objectives are achieved, consider redesign actions to make the instructional program even more effective and efficient by raising the performance standards or introducing more complex objectives. Sequence of Guidelines May Vary While the guidelines have been presented accord- ing to the elements from which they were develoPed, the designer is not required to follow the sequence offered 227 above. In fact, there is no single sequence for the arrangement of the guidelines within an element that can be uniformly applied to the design and implementation of all instructional programs. Neither is it necessary to draw upon all of the guidelines associated with one element before applying guidelines associated with other elements. As noted in Chapter III, Davis et al. state: Whenever one approaches the problem of designing a complex system, there is always the temptation to assume that a fixed sequence of steps will invariably produce the one best solution to the problem. Unfor- tunately, this is seldom true. There may be an idealized or model solution to particular design problems; but in practice, the optimal approach generally involves deviating from the model in numerous ways. (21:3) The designer must be prepared to rearrange the guidelines presented here and sequence them in the order which most accurately reflects the actual process that he/she will follow in designing and implementing the specific program. Since, in practice, the various ele- ments of the instructional program design process inter— act and are interdependent, the guidelines should be ordered and applied in a sequence which reflects this interplay. To illustrate this point, the guidelines are presented a second time and are arranged in a sequence which reflects the integration of the guidelines assigned to each element. So that the reader is able to visualize the interaction between and interdependence of the five 228 elements, each guideline is followed by the corresponding element's title and the letter of the alphabet assigned to the guideline when it was originally presented in order by element on pages 221 through 226. 1. Identify the unit (division, department, staff group) which the instructional program will be designed to serve. ASSESS NEEDS/A 2. Accept the value of and decide to design and implement instructional programs in a carefully planned manner. ASSESS NEEDS/B 3. Insure that support exists or, if necessary, develop support for the implementation of well-designed and relevant instructional programs. ASSESS NEEDS/C 4. Although redesign must occur when- ever and wherever necessary to main- tain the efficiency and effectiveness of the instructional program, since changes (redesign) are apt to be more frequent and necessary when a new program is in early design stages or is being implemented the first few times, whenever practical, plan to design and implement a prototype program to identify problems and correct deficiencies in the program. REDESIGN/C 5. Insure the existence of and, if neces- sary, develop an environment that is supportive of evaluation. EVALUATE/A 6. Plan to include evaluation procedures at appropriate points during the design process so they are an integral part of the instructional program from beginning to end and prepare to collect, interpret, and use evaluative data when it will be most valid and useful to the decision-making processes associated with the program. EVALUATE/B 10. 11. 12. 13. 14. 229 Develop evaluative procedures to assess where performance problems and, therefore, the need for an instructional program potentially exists. Complete organizational and job analyses to determine desired employee responsibilities and per- formance levels. Assess the employees' current per- formance level in completing assigned responsibilities. Determine if/where employees' current performance of assigned responsibilities does not meet desired performance levels in order to identify specific per- formance problems. Determine which performance problems are due to a skill or knowledge deficiency and determine whether employee participation in an instructional program is the most effective and efficient course of action to eliminate these problems. Develop evaluative procedures to determine the resources available for use in the design and imple- mentation of the instructional program as well as the conditions and constraints in the instruc- tional environment. When resources are limited and all problems identified for resolution through the implementation of an instructional program cannot be responded to, develop a priority- setting criteria and select pri- orities for action. In light of the target population, for those performance problems given priority for resolution, formulate general goal statements which give direction to the EVALUATE/C ASSESS NEEDS/D ASSESS NEEDS/E ASSESS NEEDS/F ASSESS NEEDS/G EVALUATE/D ASSESS NEEDS/H 15. 16. 17. 18. 19. 230 instructional program planning and design process and state, in broad terms, the desired outcome of the program. ASSESS NEEDS/J Determine the characteristics of the employees who will be members of the target population which the instructional program will be designed to serve. ASSESS NEEDS/I Recognize and accept the reasons that well—written objectives are important to the development and implementation of an instructional program. DEVELOP OBJECTIVES/A Utilizing performance problems given priority for resolution and general goal statements as a guide, clarify and state the desired behavior which the stu— dents will be expected to perform at the end of their partici- pation in the instructional program. DEVELOP OBJECTIVES/B To insure that achieving desired behavior will be useful and relevant, identify the transfer setting in which the students will be expected to utilize the skills and knowledge acquired through participation in the instructional program; what behavior will be required, under what conditions will behavior be required, and what standard of performance will be expected. DEVELOP OBJECTIVES/C Using expected performance in the transfer setting as a guide, write parsimonious, unambiguous terminal objectives to insure that the desired behavior which the students will be expected to master by the end of the instruc- tional program has been care- fully defined and can be pre- cisely communicated. DEVELOP OBJECTIVES/D 20. 21. 22. 23. 24. 25. 231 DevelOp evaluative procedures to provide the students, instructor, and designer with the data neces- sary to assess student accomplish- ment of the instructional program's terminal objective and the effec- tiveness of the instructional pro- gram itself. EVALUATE/G Deve10p evaluative procedures to pro- vide the students, instructor, and designer with the data necessary to assess whether the students meet per— formance expectations in the transfer setting, the effectiveness of the instructional program itself, and whether the design and implementation of an instructional program is the correct solution to meet the par- ticular performance problem. EVALUATE/H When evaluative procedures cannot be developed because the objective does not clearly state the desired behavior or what performance conditions and cri- teria are to form the basis for assess- ing student achievement, evaluate the objective's quality and rewrite it as appropriate. ‘ REDESIGN/G Since the desired behavior which the students will be expected to master by the end of the instructional program is often complex, analyze it to determine which intermediate skills and knowledge (behaviors) must be mastered before the desired terminal objective can be achieved. DEVELOP OBJECTIVES/E For each intermediate behavior which must be mastered in order to achieve the terminal objective, write a parsimonious, unambiguous enabling objective. DEVELOP OBJECTIVES/F Develop evaluative procedures which will, whenever needed at inter- mediate points during the pro- gram's implementation, provide the students, instructor, and designer with the data necessary 26. 27. 28. 29. 30. 232 to assess student accomplishment of the instructional program's enabling objectives, whether the program is proceeding as planned, and the effectiveness of the instructional procedures. EVALUATE/F So that the procedures utilized during a particular instructional program maximize the probability that mastery of the desired behaviors will result, precede their selection with an analysis of each objective (terminal and enabling) to identify the type of learning which must occur in order for it to be aChieved. SELECT PROCEDURES/A If a single type of learning can- not be identified for each objec- tive, evaluate its quality and rewrite the objective as appropriate. REDESIGN/D Once the type of learning has been identified for each objective, arrange the objectives in hierarchi- cal order according to the type of learning so that those requiring a lower order type of learning are placed before those requiring a higher order type of learning. SELECT PROCEDURES/B Select procedures for use in the instructional program which will establish the conditions of instruction within the instruc- tional environment necessary to maximize the possibility that the desired type of learning occurs so the students will be able to perform in accordance with the behavior stated in each objective. SELECT PROCEDURES/C Select instructional procedures which are responsive to the variables found in a particular instructional environment. SELECT PROCEDURES/D 31. 32. 33. 34. 35. 36. 233 Select instructional procedures which apply general principles of instruction to the type of learning which must occur in order to achieve mastery of the desired behavior and to the variables found in the instruc- tional environment. SELECT PROCEDURES/E Select instructional procedures which will, within the conditions and constraints of the instruc- tional environment, maximize the degree of positive transfer (lateral or vertical) of desired behavior to the transfer setting. SELECT PROCEDURES/F Sequence instructional pro- cedures so that objectives which require that a lower order type of learning occur will be mas- tered first. SELECT PROCEDURES/G Select methods and materials which are practical, establish the appropriate conditions of instruction and apply the appropriate general principles of instruction necessary to enable the student to master and demonstrate performance of the desired behavior stated in the objective, and facilitate the positive transfer of desired behavior to the referent situation. SELECT PROCEDURES/H Develop evaluation procedures to evaluate whether the stu- dents possess the initial or prerequisite capabilities necessary to effectively participate in the instruc- tional program. EVALUATE/E Implement the instructional program in accordance with selected procedures. SELECT PROCEDURES/I 37. 38. 39. 40. 41. 42. 234 Implement evaluation procedures at appropriate points during the imple- mentation of the instructional program. When students do not possess the initial or prerequisite capabilities necessary to successfully participate in and benefit from the instructional program, develop enabling objectives and select instructional procedures which will enable the students to attain these capabilities. When students do possess initial or prerequisite capabilities which exceed the minimum required to successfully participate in and benefit from the instructional program, advance the starting point for the instructional program. When objectives have not been achieved, resist the temptation to place the responsibility for this failure on the students, accept the fact that there may be defects in the design and imple- mentation of the instructional program, and take responsibility for identifying and taking appropriate action whenever and wherever it is necessary to improve, through redesign, the ability of the program to meet its objectives. When changes are made to improve the instructional program, analyze the impact which redesign within one element has on other elements and make appropriate changes in those elements which are affected. When the interpretation of evaluative data indicates that the failure to achieve a particular enabling objec- tive is due to an inadequacy in the instructional program, take corrective action which will redesign the ele- ments of the instructional program at appropriate points in order to remove problems and improve the program so that defects are overcome and objec- tives are achieved. EVALUATE/I REDESIGN/E REDESIGN/F REDESIGN/A REDESIGN/B REDESIGN/H 235 43. When the interpretation of evaluative data indicates that the failure to achieve the terminal objective is due to an inadequacy in the instructional program, take corrective action which will redesign the elements of the instructional program at appropriate points in order to remove problems and improve the program so that defects are overcome and objectives are achieved. REDESIGN/I 44. When the interpretation of evaluative data gathered in the transfer setting indicates that the failure of the stu- dents to perform as expected is due to an inadequacy in the instructional program, take corrective action which will redesign the elements of the instructional program at appropriate points in order to remove problems and improve the program so that defects are overcome and objectives are achieved. REDESIGN/J 45. If terminal objectives are achieved but evaluative data indicate that the needs, expectations, and/or character- istics of the organization the program is designed to serve have changed since the instructional program was originally designed, redesign the instructional program so that it remains up-to-date and reflective of the real needs of the organization if and when it is reimple- mented. REDESIGN/K 46. If terminal objectives are achieved, consider redesign actions to make the instructional program even more effective and efficient by raising the performance standards or introducing more complex objectives. REDESIGN/L Arranging the guidelines in a sequence as pre- sented here illustrates that, while a guideline may be formally assigned to one of the elements given comprehen- sive consideration in Chapter III, in practice, the 236 process of actually designing and implementing a program will necessitate ordering the guidelines so they are responsive to the interaction and interdependence of the five elements. For example, while formally a part of the fifth element (Redesign), the fourth guideline above illustrates that the designer must determine early in the design process whether a prototype program should be used to facilitate necessary redesign actions. Similarly, as illustrated by the placement of the sixth guideline, formally a part of the fourth element (Evaluate), the designer must plan early in the design process to include evaluation procedures at appropriate points throughout the design and implementation of the instructional pro- gram. The interaction and interdependence of various elements is further illustrated by noting the placement of the twelfth guideline and recognizing how a response to this guideline at this point in the instructional pro- gram design process is important to the designer's ability to properly respond to guideline 13. Similarly, note the interplay between the elements of the design process by examining guidelines 20, 21, and 22, and/or those numbered 26 and 27. The designer must be prepared to arrange the guidelines in a sequence which accurately reflects the processes which he/she will follow in designing and 237 implementing a specific program. The guidelines have been developed for use as a tool to assist in insuring that each element is incorporated into the instructional program and that, as a result, the program's objectives will be achieved. Therefore, the designer must be pre— pared to use this tool in a manner which will be most useful in achieving this goal and not seek to rigidly apply the guidelines in the same sequence regardless of the circumstances associated with the design and imple- mentation of a specific program. Checklist The guidelines presented above give direction to the design and implementation of an instructional program and help insure that each element has been properly incorporated. Having formulated these guidelines, the discussion of each element presented in Chapter III has been studied further and, based upon the views of the various authors presented, a checklist has been developed for use in assessing whether each element has been incorporated into the instructional program as it is designed. It is intended that, once the instructional program has been developed, the program designer will use the checklist to determine that he/she has taken the steps necessary during the design and implementation of the instructional program to insure that each guideline and, consequently, each element has been properly 238 incorporated. While the guidelines give direction to the design process as it is about to take place or is taking place, the checklist provides a tool for deter- mining that the proper steps have occurred during the design and implementation of the instructional program. The items in the checklist are listed below in numerical order by element. The guidelines previously presented in this chapter and the checklist items identified below are both derived from the elements. Therefore, a relationship can be observed between the guideline and checklist items. So the reader is able to identify the pertinent guideline for a specific checklist item, the number of each checklist item is followed by the letter of the alphabet assigned to the associated guideline as previously presented on pages 221 through 226. An affirmative response to the checklist item; that is a check in the space provided for YES following each of the items of the checklist, insures that each of the guidelines has been responded to and, in turn, that the elements have been properly incorporated in the design and implementation of the instructional pro— gram. ASSESS NEEDS l/A The unit (division, department, staff group) which the instructional program will serve has been identified. D YES [:3 NO 2/B 3/C 4/D 5/D 6/D 7/E 8/F 239 A commitment has been made to design and implement the instructional pro- gram in a carefully planned manner. Support for the implementation of well-designed and relevant instruc- tional programs exists. Through organizational analysis, the characteristics, philosophical assumptions, responsibilities, and goals and objectives of the unit have been identified. Through organizational analysis, the characteristics, philosophical assumptions, responsibilities, and goals and objectives of the larger organization or environment within which the unit functions and the contribution the unit must make to help meet the needs of the larger organization or environment have been identified. The tasks which employees within the unit must perform to meet assigned responsibilities and the skills, knowledge, and attitudes which they must possess to do so successfully have been identified. The current performance level with which employees (potential students) complete assigned responsibilities has been assessed. The current performance of assigned responsibilities has been compared with desired performance levels and those employees who are unable to perform as desired and those responsibilities which are being inadequately met have been identified. D YES [:| NO E] YES [:l NO 1:] YES [3 NO E] YES C] NO [3 YES E] NO E] YES C] NO C] YES [3 NO 9/F 10/F ll/G 12/G 13/H 14/H 240 Those responsibilities which are inadequately met have been identified in terms of Specific desired behaviors which are not being exhibited by par- ticular employees. D YES D NO Desired behaviors which are not being exhibited by particular employees have been identified as performance problems which may demonstrate the need for the design of and employee participation in an instructional program in order to improve performance to acceptable levels. [3 YES [3 NO The performance problems which are due to Skill or knowledge deficiency have been identified, and it has been concluded that implementation of an instructional program will positively contribute to solving the problem, will meet the identified need, and, therefore, is the most cost-effective and productive course of action for accomplishing desired outcomes. D YES E] NO It has been determined that another course of action (e.g., new equip- ment, salary increases, changes in selection criteria, transfer of employees to new responsibilities) would not be more effective in solving particular performance problems since the problem is due to a skill or knowledge deficiency. [3 YES [3 NO The human and physical resources available for use in the design and implementation of an instructional program and the degree of support for the program from leaders within the unit and/or the larger organi- zation or environment have been identified. C] YES [3 NO The constraints (policies, pro- cedures) and pressures within the unit and/or the larger organization or environment which affect the allocation of resources to accomplish program goals and objectives have been identified. D YES [3 NO 15/H 16/H 17/H 18/I 19/I 20/J 241 Relevant and concerned members of the organization or population the instructional program is to be designed to serve (e.g., potential students, supervisors, community members, educational Specialists) have been involved in the selection and application of priority-setting criteria. Conflicts have been resolved between relevant and concerned members of the organization or population the instructional program is to be designed to serve regarding the selection of priority-setting cri- teria and/or the priority given to the resolution of particular per- formance problems. Priority-setting criteria (e.g., limited instruction time and resources, cost of taking no action, importance of a particular skill to successful job performance, policy constraints) have been established and applied in order to select the most important problems for resolution. The characteristics of potential students (e.g., skills and abilities which students already possess, edu- cational level and experience, interests and aspirations, physical limitations, and attitudes and prejudices) have been identified. The number of potential students to be placed in the instructional environment and the degree of heterogeneity of the student group have been determined. General goal statements for the instructional program have been written. [:j YES [3 NO D YES [3 NO [3 YES E] NO [3 YES [3 NO [3 YES [3 NO 242 DEVELOP OBJECTIVES 1/A 2/A The importance of well-written objectives has been recognized because: a. they clarify the desired behavior which the instructional program is being designed to enable the students to achieve and therefore give direction to the design process; they enable the designer to identify the type of learning which must occur in order for students to achieve the desired behavior and, in turn, facili— tate the selection of the proper con- ditions of instruction necessary to promote its mastery; they enable the judicious selection of instructional methods and materia which will promote mastery of desired behavior; they provide students with a clear understanding of what they will be expected to do at the end of the instructional program, the conditions/ standards under which performance will be expected to occur, and enable the students to self-evaluate their pro- gress in achieving desired behaviors; they enable the instructor to pretest student mastery of desired behavior; they provide observable standards for use in measuring student achievement of desired behavior; they enable the designer/instructor to evaluate the effectiveness of instructional procedures and materials. D YES [3 NO The importance of insuring that each well-written objective is properly utilized throughout the design and implementation of the instructional program has been recognized. D YES [3 NO 243 3/B The Specific behaviors which the students will be expected to perform at the end of the instructional program in order to demonstrate that they have learned desired skills have been identified. [:]YES [:jNO 4/C The location or Situation (transfer setting) where, at a future date, the students will be expected to perform behaviors mastered during the instruc- tional program has been identified. [3 YES [3 NO 5/C The behaviors which the students will be expected to perform or tasks which they will be expected to accomplish in the transfer setting have been identified. [3 YES [j NO 6/C Through direct observations or dis- cussions with employees, pertinent environmental conditions, organi- zational conditions and constraints, types of equipment used, and aids and restrictions which exist in the transfer setting and affect per- formance of desired behavior have been identified. D YES D NO 7/C Pertinent performance conditions and standards which will be used to evaluate student success in the transfer setting have been identified. [3 YES [:3 NO 8/D Terminal objectives for the instruc- tional program have been written. [:1 YES D NO 9/D The terminal objectives include words such as to write, to select, to list, or to sdlve and d6 not include words such as to know, to a reciate, or to understand to-' descrihe the overt actibns_fihich will be accepted as evidence that the student has mastered the skills or knowledge necessary to achieve desired behavior. D YES [:1 NO lO/D 11/D 12/D 13/D 14/D 15/E 16/E 244 The terminal objectives include con- ditions statements which describe what will be provided and what will be denied the students when they are demonstrating that they have mastered desired behavior as well as the char- acteristics of the particular setting within which the behavior will be expected to occur. The terminal objectives include phrases (e.g., minimum number of correct responses, time limits, speed and level of accuracy, number of chances to perform, proportion of correct reSponses required) which Specify the minimum standards that must be achieved in order to demon- strate mastery of the desired behavior. The components of the terminal objec- tives are written so that the behavior which the students will be expected to demonstrate at the end of the instructional program represents as closely as possible that required in the transfer setting with devi- ations limited to those dictated by limits or constraints existing within the instructional environment. The terminal objectives have been trimmed of excess words and concepts. The terminal objectives do not include procedures, content of the instructional program, methods, or materials to be used during instruction. Through an analysis of the desired behavior stated in the terminal objectives, intermediate behaviors which must be mastered before they can be achieved have been Specified. In order to identify intermediate behaviors which must be mastered to achieve a particular terminal objec- tive, information gathered from individuals familiar with the E] YES D NO E] YES [:1 NO |:| YES C] NO C] YES D NO [:1 YES [3 NO [3 YES El NO 17/E 18/F 245 performance of the desired behavior, direct observation of the desired behavior while it is being per— formed, and/or discussions with supervisors has been analyzed and interpreted. D YES D NO Through an analysis of the desired behavior stated in the terminal objective, the order (steps or sequence) in which intermediate behaviors must be completed in order for it to be properly per- formed has been clarified. [3 YES [Z] NO Enabling objectives which possess the same characteristics as those included in terminal objectives (see checklist items 9 through 14 above) have been written for each intermediate behavior which must be mastered in order to achieve the desired (terminal) behavior. [:3 YES D NO SELECT INSTRUCTIONAL PROCEDURES l/A 2/A Each objective (terminal or enabling) has been analyzed and the type of learning (signal learning, stimulus-response learning, chaining, verbal association, multiple discrimi- nation, concept learning, principle learning, problem solving) which must occur in order to insure mastery of it and the desired behavior stated with- in it has been determined. E] YES D NO The importance of selecting instruc- tional procedures in response to the type of learning which must occur in order to insure mastery of desired behavior in a particular objective and not uniformly applying the same set of procedures regard- less of the type of learning required has been accepted. E] YES [3 NO 3/B 4/B 5/C 6/C 246 The proposition has been recognized and accepted that transfer of learning occurs from less complex to more com- plex types of learning; that is, from Signal learning to stimulus-response learning stimulus-response learning to chaining chaining to verbal association verbal association to multiple discrimination multiple discrimination to con- cept learning concept learning to principle learning principle learning to problem solving. [3 YES E] NO When a number of enabling objectives must be mastered in order to insure performance of a terminal objective and/or a number of terminal objec- tives have been ordered so those requiring a lower order type of learning and whose mastery is a necessary prerequisite to the accomplishment of objectives requir- ing a higher order type of learning, are mastered first. D YES D NO The work of Robert M. Gagné and/or authors who have interpreted and applied his work has been consulted to identify and select for use in the instructional program the spe- cific conditions of instruction which must be present in the instruc- tional environment in order to maximize the probability that the type of learning necessary to master the desired behavior stated in each objective will occur. E] YES ['3 NO The preposition that the conditions of instruction which must exist in the instructional environment in order to maximize mastery of desired behavior are different for each type of learning involved has been accepted. D YES D NO 7/C 8/C 9/C 10/C ll/C 12/D 247 Instructional procedures have been selected in response to the con- ditions of instruction which must be present in the instructional environment in order to maximize the probability that the type of learning necessary to master the desired behavior stated in each objective will occur. The initial or prerequisite capa- bilities which the student must possess or lower order types of learning which must have pre- viously occurred in order to effectively participate in the instructional program have been identified. Instructional procedures have been selected and sequenced in order to insure mastery of objectives requiring a lower order type of learning before those requiring a higher order type of learning. As a prerequisite to effective instruction, procedures have been selected to encourage student moti- vation to learn. The procedures to be implemented have been selected and arranged so that the conditions within the instructional environment maximize the probability that the desired type of learning occurs and, in turn, the desired behavior, as stated in the objective, will be demonstrated by the student at the end of the instructional pro- gram. AS procedures are selected to maxi- mize the probability that a desired type of learning occurs, steps have been taken to insure that they are compatible with the unique variables which will be encountered in the instructional environment when implementation of the instructional program occurs. D YES D NO D YES D NO D YES D NO D YES D NO D YES D NO D YES D NO 13/E 14/F 248 Whenever appropriate, in order to maximize the probability that a desired type of learning occurs, general principles of instruction have been applied and, accordingly, the procedures implemented during the instructional program have insured that the students have been a. provided with knowledge of rele- vant entry Skills; b. advised of the objective which must be achieved; c. advised of the reasons that developing the desired behavior is valuable; d. provided an example of desired terminal performance; e. stimulated to recognize/recall necessary prerequisite capa- bilities; f. provided relevant, orderly guidance; 9. provided active, reasonably dis- tributed practice; h. appraised of results; 1. provided with an opportunity to perform with less and less guidance and support from the instructional environment. D YES D NO Whenever possible within the limits and constraints of the instructional environment, instructional pro- cedures have been selected to create conditions as Similar as possible to those of the transfer setting in order to maximize the degree of posi- tive transfer of desired behaviors which are mastered during the instructional program. D YES D NO 15/F 16/F 17/F 18/G 19/H 20/H 249 When the limits and constraints of the instructional environment make the creation of conditions similar to those of the transfer setting impos- sible, students have been provided the opportunity to master and apply the principles necessary to perform successfully in the transfer setting. Whenever possible within the limits and constraints of the instructional environment, students have been pro- vided opportunities to apply and practice desired behavior in as wide a variety of situations as possible so that the number of transfer set- tings where the behavior might be useful is increased as much as possible. Whenever relevant and possible, the instructor has worked closely with the students' supervisors to insure that there are incentives within the transfer (job) setting which will reinforce positive transfer of skills learned during the instructional program. Instructional procedures have been sequenced so that objectives which require that a lower order type of learning occur will be mastered first. Methods and materials have been selected because they contribute to the implementation of selected instructional procedures and, in turn, establish the conditions of instruction within the instruc- tional environment and apply general principles of instruction necessary to insure that the desired type of learning occurs. Methods and materials have been selected because they positively contribute to the accomplishment of instructional objectives and have not been selected for use in D YES D NO D YES D NO D YES D NO D YES D NO D YES D NO 21/H 22/H 23/H 24/I 250 the instructional program simply because they are available and familiar or new and unusual. The methods and materials have been selected because they enable the student to perform in accordance with the conditions stated in the objective. The methods and materials selected for use in the instructional program are as Similar as possible to those which the students will encounter when they will be required to per- form the desired behavior in the transfer setting so positive transfer will occur. The methods and materials have been selected because they can be utilized within the limits, constraints, and other variables existing in the instructional environment and because they are, among the alternatives available, the most efficient and practical. The instructional program has been implemented in accordance with selected procedures. EVALUATE 1/A 2/A 3/B A supportive environment within which to complete the selected types of evaluation exists. Students, instructors, and designers understand the important role of evaluation in insuring that the instructional program is successful; that is, desired behaviors are mastered. The information needed and, therefore, the type of evaluation which must occur and when it must occur, in order to make appropriate decisions during the design and implementation of the instructional program, has been determined. D YES D YES D YES D YES D YES D YES D YES D YES DNO DNo DNo DNO DNo DNo 4/B 5/B 6/B 7/B 8/B 9/B 10/B 251 For each evaluation procedure, the objective to be achieved and who will be served has been determined, the decisions which the evaluative data collected will be used to make have been clarified, and the policies within which evaluation procedures must operate have been identified. Within practical limits, the evalu— ation formulated utilize the most rigorous experimental design possibld. From the alternatives available, the evaluation procedures formulated are as efficient and practical as pos- sible and still remain valid and reliable. The instrument used to collect evaluative data possesses criterion relevance, and criteria deficiency andtcriteria contamination have been reduced as much as possible. The instrument used to collect evaluative data possesses items which are valid and, therefore, it requires the same behavior, under the same conditions and standards, stated in the objective. The instrument used to collect evaluative data possesses a suf- ficient number of items to insure reliability. The evaluation procedures formu- lated account for and/or eliminate factors (e.g., unclear instructions, excessive noise in the evaluation setting) which negatively affect students' ability to perform skills which have been learned and, therefore, adversely affect the reliability. D YES D NO D YES D NO D YES D NO D YES D NO D YES D NO D YES D NO 11/B 12/C 13/D 14/E 15/F 16/F 252 How the evaluation procedures will be administered has been determined, the methods and materials necessary to implement them have been selected and organized, and how the data collected will be organized, interpreted, and reported has been determined. Evaluative procedures (checklist items 3 through 11 above) have been developed to assess where the need for an instructional program exists, to verify the continued relevance of assessed needs, and to identify new needs. Evaluative procedures (checklist items 3 through 11 above) have been developed to determine the resources available for use in the design and implementation of the instructional program as well as the conditions and constraints of the instructional environment. Evaluative procedures (checklist items 3 through 11 above) have been developed to determine the existence of those initial or prerequisite capabilities which the student must possess or lower order types of learning which must have occurred previously. Evaluative procedures (checklist items 3 through 11 above) to assess accomplishment of each enabling objective have been designed in response to the desired behavior and performance conditions and cri- teria stated in each objective. When the data gathered through evaluation indicates that a particu- lar enabling objective has been achieved, the students, instructor, and designer recognized this achieve- ment and proceeded with the implemen- tation of the instructional program in accordance with selected pro- cedures. D YES D NO D YES D NO D YES D NO D YES D NO 17/F 18/G 19/G 20/G 21/G 253 When the data gathered through evalu- ation indicates that a particular enabling objective has not been achieved, steps have been taken to interpret the data and determine if the failure is due to inadequacy in the instructional program. The evaluative procedures (checklist items 3 through 11 above) to assess accomplishment of the terminal objective have been designed in response to the desired behavior and performance conditions and cri- teria stated in the objective. When the data gathered through evalu- ation indicates that the terminal objective has been achieved in a reasonable length of time and at reasonable costs, the students, instructor, and designer recognized this achievement. When the data gathered through evalu- ation indicates that the terminal objective has not been achieved, steps have been taken to interpret the data and determine if the failure is due to inadequacy in the instruc- tional program. When failure to achieve the terminal objective is due to inadequacies in the instructional program, evaluative data has been interpreted to determine if a. the need for or goal of instruction was properly identified; b. terminal objectives were properly written and were consistent with the identified goal; c. all necessary enabling objectives were identified, properly written, and properly sequenced; d. students possessed necessary entry skills; D YES D YES D YES D YES DNo 22/H 23/H 24/H 254 e. the types of learning and conditions of instruction selected were con- sistent with the behavior stated in the objective; f. the instructional procedures and methods and materials selected were implemented as planned or Should be changed; g. important conditions of instruction or procedures were omitted; h. the evaluation of student achievement was not designed to reflect the behavior, conditions, and standards stated in the objective; i. environmental constraints or con- ditions adversely affected the per— formance of the objective. D YES D NO Evaluative procedures (checklist items 3 through 11 above) have been develOped to assess the students' performance in the transfer setting and are designed to determine if the students are able to perform as required in order to meet the organization's needs. D YES D NO When the data gathered through evalu- ation indicates that the students meet performance expectations in the transfer setting, the students, instructor, and designer recognized this achievement. D YES D NO When evaluative data gathered in the transfer setting indicates that a dif- ference between desired performance and the students' actual performance continues to exist, steps have been taken to interpret the data and determine if the performance dis- crepancy is due to inadequacies in the instructional program. D YES D NO 255 25/H When a performance discrepancy is due to inadequacies in the instruc- tional program, evaluative data have been interpreted to determine if a. the correct performance problem was identified; b. the desired behavior or performance conditions and standards have changed; c. instruction was the correct solution for the identified problem; d. the desired behavior and performance conditions and criteria set down in the objective reflected those expected in the transfer setting; e. the performance required in the instructional setting reflected as closely as possible that required in the transfer setting; f. the evaluation of student achieve- ment in the instructional environ- ment possessed criterion relevance, validity, and reliability. D YES D NO 26/I Evaluation procedures have been implemented D YES D NO REDESIGN l/A The fact that there may be defects in the design and implementation of the instructional program and that redesign actions will be necessary to improve the instruc- tional program in order to meet objectives has been recognized and accepted. D YES D NO 2/A To insure that a self-corrective redesign feature exists, the instructional program has been developed as a "closed-loop system" so evaluative data are interpreted and applied in order to decide upon courses of action to change various 3/B 4/C S/C 6/C 7/D 8/E 256 elements of the program, thereby improv- ing the effectiveness and efficiency with which the program achieves desired objectives and remains relevant to the organization it is designed to serve. D YES D NO The impact which redesign within one element of the instructional program has on other elements has been analyzed and appropriate changes in those ele- ments which are affected have been made. D YES D NO A prototype has been used whenever it is advantageous to increase the designer's willingness to redesign various elements, to reduce the number of students affected by redesign decisions, and to insure that the quality of the instructional program which is finally implemented in the instructional environment is as high as possible. D YES D NO The prototype has been designed and implemented under the same con- ditions which will be encountered in the final instructional environment. D YES D NO In using the prototype, evaluative data have been gathered and inter- preted and, based upon conclusions reached, appropriate revisions have been made. D YES D NO If, when an objective has been analyzed, a single type of learning cannot be identified, action has been taken to rewrite it. D YES D NO When students have not possessed the initial or prerequisite capabilities necessary to successfully partici- pate in and benefit from the instruc- tional program, enabling objectives have been developed and instruc- tional procedures have been selected in order to enable the students to master necessary prerequisite skills and knowledge. D YES D NO 9/F 10/G ll/H 12/H 13/I 257 When students have possessed initial or prerequisite capabilities which exceed the minimum required to suc- cessfully participate in and benefit from the instructional program, the starting point for the instructional program has been advanced. When it has been unclear what behavior and what performance con— ditions and criteria are to form the basis for evaluation of student achievement of desired behavior because the objective is not properly written, action has been taken to rewrite it. When the interpretation of evaluative data has indicated that the failure to achieve a particular enabling objective is due to an inadequacy in the instructional program, correc- tive action which redesigns the ele- ments of the instructional program at appropriate points in order to remove problems and improve the program so that defects are over- come has been taken. When the interpretation of evaluative data has indicated that the failure to achieve a particular enabling objective is due to possible physical and/or emotional factors having an impact upon the instructional environment which are not directly related to the procedures selected and implemented as part of the pro- gram, whenever practical, actions have been taken to remove these factors. When the interpretation of evaluative data has indicated that the failure to achieve the terminal objective is due to an inadequacy in the instruc- tional program, corrective action which redesigns the elements of the instructional program at appropriate points in order to remove problems and improve the program so that defects are overcome has been taken. D YES D NO D YES D NO D YES D NO D YES D NO D YES D NO 14/I 15/J 16/J 17/J 258 When the interpretation of evaluative data has indicated that the failure to achieve a particular terminal objective is due to possible physical and/or emotional factors having an impact upon the instructional environ- ment which are not directly related to the procedures selected and imple- mented as part of the program, when- ever practical, actions have been taken to remove these factors. D YES D NO When the interpretation of evaluative data gathered in the transfer setting has indicated that the failure of the student to perform as expected is due to an inadequacy in the instructional program, corrective action which re- designs the elements of the instruc- tional program at appropriate points in order to remove problems and improve the program so that defects are overcome has been taken. D YES D NO When the interpretation of evalua- tive data gathered in the transfer setting has indicated that the reason for the students' inability to meet performance expectations in the transfer setting are not related to the effectiveness of the instruc- tional program, other solutions to the performance problem have been pursued or changes in the transfer setting have been made so that more positive transfer of learned Skills from the instructional setting to the transfer setting occurs. D YES D NO When evaluative data have indicated that a performance discrepancy is not due to inadequacy in the instruc- tional program, further analysis has been undertaken to determine if inadequate performance is due to other factors such as a. lack of reward or even punishment for performing desired behavior in the transfer setting; 259 b. rewards for performing other than in the manner taught in the instructional program; c. changes in the transfer setting that make desired behavior no longer useful or that have changed per- formance conditions and standards; d. a lack of equipment or poor environmental conditions. D YES D NO l8/K Prior to reimplementing the instruc- tional program, careful assessment has been undertaken to determine if a. the needs and/or conditions of the transfer setting have changed; b. the characteristics of students have changed; c. some other change has occurred which will require revision of the instructional program so it will continue to meet organi- zational needs. D YES D NO l9/K If the interpretation of evaluative data has indicated that changes have occurred since the program was origi- nally designed, appropriate redesign actions have been taken prior to the program's reimplementation so it remains relevant. D YES D NO 20/L When the terminal objectives have been achieved, appropriate redesign actions have been taken to make the instructional program more effective and efficient. D YES D NO As those responsible for the design and implemen- tation of staff development programs prepare to use the guidelines and checklist in this chapter, it is stressed that they can only be utilized effectively if the designer has, through the careful study of resource mate that tior five and 11m for gra con Acc Che con eve to 260 materials, develOped an understanding of the contribution that each element makes to the success of the instruc- tional program and why the contributions made by all five of the elements must be accounted for as the design and implementation of the program proceeds. The guide- lines and checklist are not presented as a substitute for such knowledge, but rather as an aid to help the pro- gram designer insure that such knowledge and the elements' contributions are brought to bear in the design process. Accordingly, the designer is encouraged to consult Chapter III and/or other resources, particularly in complex topical areas (e.g., conditions of instruction, evaluation), in order to secure basic understanding prior to using the guidelines and checklist. CHAPTER V SUMMARY, FURTHER STUDY, AND IMPLICATIONS Drawing upon the views of selected authors, a set of guidelines and a checklist for use in the design and implementation of inservice staff development pro- grams were presented in the previous chapter. This portion of the study (1) summarizes the dissertation, (2) makes suggestions for further study, and (3) dis- cusses the implications which use of the guidelines and checklist has for student personnel. Summary As established at the beginning of this disser- tation, a careful review of student personnel literature demonstrates that, while many recognize the important role that inservice staff development programs should have within the profession, those who are interested in or who are responsible for the design and implementation of these programs are provided with few resources upon which to draw for assistance. While a few authors within 261 262 the student personnel profession (most notably, Truitt and Gross in their article, "Inservice Education for College Student Personnel”) have sought to provide "general principles" to help guide the development and administration of inservice staff development programs within student personnel divisions, resources of this type are very limited. Recognizing that limitation, it has been the goal of this study to develop a set of guidelines and a checklist which can be used to help guide the design and implementation of student personnel inservice staff development programs. The author believes that, as a result of making such a resource available to assist student personnel professionals with design and implementation, inservice staff development programs will occur more frequently, their design and implementation will more likely be based upon educationally sound instructional procedures, and, therefore, student personnel staff will be better prepared to achieve institutional, divisional, and pro- fessional goals and to demonstrate their achievements to those responsible for holding campus programs accountable. Throughout the study, it has been assumed that, since inservice staff development programs, like other instructional programs, are designed to facilitate an individual's learning, they should incorporate those 263 elements which selected authors agree are important and ought to be part of the design and implementation of any instructional program, regardless of its setting. There- fore, in order to identify the appropriate elements of an instructional program and, in turn, to insure that the guidelines and checklist developed for use in the design and implementation of inservice staff development pro- grams are based upon educationally sound instructional procedures, the process followed to complete the study and to formulate the guidelines and checklist has been: (1) to investigate literature which examines the design and implementation of instructional programs for use in formally recognized class- room settings as well as training literature developed for settings other than student per- sonnel; (2) to abstract from this literature instructional program elements which, because there is agree- ment among various authors concerning their value, are determined to be appropriate for inclusion in the design and implementation of instructional programs; (3) to discuss the role of each element in the design and implementation of instructional programs; 264 (4) to create, using the discussion of each element as a source, a set of guidelines which are organized into an orderly framework so they can be followed in a step-by-step fashion by those who are designing and implementing inservice staff development programs; and (5) to develop statements for each element which can be responded to in checklist fashion so that proper utilization of each element will be insured as inservice staff development programs are designed and implemented. Following these procedures, it has been deter- mined that, while different terms may be used and the order may vary somewhat, there is a significant degree of agreement among selected authors concerning the ele- ments which should be incorporated into well-designed instructional programs in order to optimize the proba- bility that desired learning will occur and students will demonstrate mastery of the behaviors stated in the objectives. These elements are (l) assess needs, (2) develop objectives, (3) select instructional pro- cedures, (4) evaluate, and (5) redesign. Drawing upon the views of selected authors, the role of each of these elements in the design.and implementation of instruc- tional programs is specifically described in separate sections of Chapter III. eac prc Che vie The SCI sta me: in po: mu li He 11' be 265 Having examined the distinct contribution which each element makes to the success of an instructional program, the discussion of each element presented in Chapter III was carefully studied and, based upon the views of the authors presented, a set of guidelines and a checklist were developed and are presented in Chapter IV. The resulting resource is designed to assist student per- sonnel professionals to design and implement inservice staff develOpment programs and to insure that each ele- ment which was determined to be appropriate for inclusion in educationally sound instructional programs is incor- porated. In utilizing this resource, the program designer must avoid rigidly or uniformly applying all the guide- lines and checklist items in the same sequence each time a program is designed and implemented. It is neither necessary nor advantageous to draw upon all of the guide- lines or checklist items associated with one element before applying any of those associated with another element. While each of the five elements must be incorporated into each program, because they interact and are interdependent, the guidelines and corresponding checklist items should be ordered and applied in a sequence which reflects this interplay among the elements and which is in response to the specific aspects of a particular program. 266 As the designer of an inservice staff development program uses the guidelines and checklist developed in this study, it must be remembered that the effective use of this resource can only be achieved if the designer has completed a thorough examination of literature such as that presented in Chapter III, and has developed a clear understanding of the instructional process, the contribution which each element makes to the success of the entire instructional program, and the reason that all five of the elements must be accounted for as the design and implementation of the program proceeds. Understanding is particularly important in such complex areas as task analysis, conditions of instruction, and evaluation. The guidelines and checklist are not presented as a substitute for such knowledge. Some designers may be tempted to try to routinely follow the guidelines or use the checklist prior to such understanding. This will not result in a successful program since many of the ideas and concepts presented must be understood in order for them to be effectively used to design and implement instructional programs which will achieve desired objec- tives. For those who must develop such understanding, the bibliography at the end of this dissertation cites source materials which the reader can consult for elabor- ation upon the material presented in Chapter III. 267 Suggestions for Further Study As stated above, student personnel staff who are interested in or who are responsible for the design and implementation of inservice staff development programs must study relevant literature to insure understanding of the instructional process and the contributions which each element makes to the success of a program. How- ever, as understanding is achieved, the designer must not conclude that further study is unnecessary. Extensive literature is available in such fields of study as instructional system design and industrial or management training. New developments occur and are reported regu- larly. Therefore, an on-going commitment to continue to study available material and to become informed of new developments is essential so that new knowledge is gained and applied to improve the quality of program design and implementation. The program designer must study the literature and think carefully about its con- tent to continually improve his/her understanding of the instructional process. He/she must also take steps to incorporate the knowledge gained into his/her instructional procedures whenever such action will improve the program's impact upon staff development. The inservice staff development program designer is encouraged to utilize the knowledge gained through such study to determine whether changes in the guidelines 268 and checklist developed in this study are warranted. It is expected that new knowledge concerning the process of human learning and the instructional procedures required to impact it will be continuously developed. Therefore, it will be necessary and advantageous to modify the guidelines and checklist in order to incor- porate new knowledge so their quality as a resource to assist in the design and implementation of inservice staff develOpment programs will be increased. Each program designer who elects to use this resource is encouraged to take responsibility for remaining informed about developments in the study of human learning, determining those modifications in the guidelines and checklist which should occur, understanding how these changes will improve the quality of this resource, and making selected changes in the guidelines and checklist items. These actions will insure that the guidelines and checklist are subjected to continuous assessment and that relevant improvements will be made. Efforts should also be made to evaluate and analyze the effectiveness of specific inservice staff development programs which have been designed and imple— mented in accordance with the guidelines and checklist. Where it is determined that desired objectives were not achieved, efforts Should be made to determine if this 269 outcome was due to shortcomings in the way the guidelines and checklist are formulated or sequenced. If so, they should be modified. AS noted earlier, this study has been completed to provide student personnel professionals with a resource to assist in the design and implementation of inservice staff development programs. The profession would benefit from Similar studies designed to develop other resources which would elaborate upon particular aspects of the instructional design process. Such studies are likely to complement the resource provided in this dissertation and would be very useful to those designing and imple- menting inservice staff development programs. For example, following an approach similar to the one taken in this study, the views of selected authors could be studied in order to develop needs assessment procedures and processes specifically designed to identify those student personnel staff members who would benefit from participation in inservice staff development programs and which topic areas would be most relevant. Addi- tionally, procedures and processes could be designed to apply Gagné's work to the design and implementation of inservice staff development programs and therefore improve the selection of the conditions of instruction necessary to insure that the desired type of learning occurs. Finally, procedures and processes could be 270 designed to evaluate inservice staff development programs to determine the quality of instructional procedures selected for use in a particular program and to assess participants' mastery of desired objectives in the instructional as well as the transfer settings. Efforts such as those mentioned here will increase the resources available which specifically apply the knowledge and techniques available in other fields of study to meet the needs of the student personnel profession. Implications As discussed in Chapter I and in the first section of Chapter II, there are many important reasons why inservice staff development programs should be incorporated within student personnel divisions. These reasons range from the need to prepare staff members to perform and be held accountable for the accomplishment of specific tasks to providing staff members with an oppor- tunity to remain informed about developments in the profession and insure their own continued growth and development. The author believes that the guidelines and checklist developed in this study are a useful resource to assist in the design and implementation of student personnel staff development programs, regardless of the subject matter or the setting in which the program takes place. For example, this resource will be useful when 271 (l) the chief student personnel administrator and his/her immediate staff are preparing to respond to legislative developments, (2) resident assistants are being trained to improve their listening skills, (3) the financial aides staff members are learning how to comply with the procedures associated with a new federal loan program, (4) the student union receptionists are developing an improved ability to dispense resource information in a clear and personable manner, (5) interested staff are learning new Skills so they might apply new techniques as they complete assigned responsibilities or pursue new employment opportunities, (6) the staff of a par- ticular department within the student personnel division are meeting to discuss recent professional literature, conference presentations, or other professional develop- ments, (7) a specific staff member is completing indepen- dent activities to respond to his/her supervisor's evalu- ation that he/she develop particular skills and abilities in order to effectively meet assigned job responsibili- ties, or (8) for any other reason, particular staff members pursue inservice staff development. Regardless of the topic, motivation for participation, status level of the participants within the division, or tenure of the participants, inservice staff development programs should be based upon educationally sound instructional procedures. The guidelines and checklist designed in 272 this study and presented in Chapter IV provide student personnel staff with a resource to assist them in achieving this goal. Few would argue with the wisdom of designing and implementing staff development programs which incorporate those instructional principles necessary to insure that desired learning occurs and, as a result, staff members are able to demonstrate mastery of the skills necessary to perform as stated in the objectives of the program. However, when electing to use the guidelines and check- list to achieve desired instructional objectives, student personnel professionals must recognize the implications of this decision. They are discussed here. First, designing and implementing inservice staff develOpment programs in accordance with the processes and procedures presented in this dissertation is a demanding task. It should not be undertaken unless the user is prepared to recognize this task for what it is-- hard work. However, since learning and the achievement of stated objectives ought to be the desired outcome when the decision is made to institute inservice staff develop- ment programs, those interested in or responsible for the success of these programs must be prepared to com- plete the tasks necessary to insure that they are achieved. 273 Second, in order to effectively design and imple- ment inservice staff development programs and respond to the demanding task of doing so in accordance with edu- cationally sound instructional procedures, it is essential that the chief student personnel administrator, as well as other leaders of the division, provide visible and sufficient support to inservice staff development efforts. These individuals must accept responsibility for and pro- vide leadership to the design and implementation of in- service staff development programs. They must have a thorough understanding of and commitment to inservice staff development and actively participate in the develop- ment of objectives for these programs. They must demon- strate their recognition of the importance of inservice staff development by actively participating in these programs themselves. Clearly, the resources (e.g., funds, facilities, equipment) necessary to the success of particular pro- grams must be provided. Beyond that, however, divisional leaders must set a tone and promote a climate which encourages staff members to participate in staff develop- ment and recognize that doing so will improve and/or maintain individual and divisional efforts to meet responsibilities to students, the institution, and the profession. Staff members must recognize that the sig- nificant leaders within the division support and clearly 274 expect participation in, and encourage staff members to take time for staff develOpment as a regular part of each person's job responsibilities. In addition, staff members who have participated in inservice staff development programs must be encouraged in and be rewarded for applying skills, abilities, or concepts which have been mastered during the inservice staff development program as they carry out their responsibilities in their work situation. Failure to encourage participants in inservice staff development programs to apply what has been learned in the instruc- tional environment to their daily efforts to provide programs and services will result in their having little impact upon the achievement of institutional, divisional, and professional goals. Divisions which are particularly committed to educationally sound inservice staff development programs may provide further support by hiring one or more staff members who are specifically Skilled in the design and implementation of instructional programs. Such an indi- vidual(s) could provide overall direction to staff development efforts within the division and insure that programs are formulated in accordance with sound instruc- tional processes and procedures. Third, those committed to the design and imple- mentation of programs which are based upon educationally 275 sound instructional procedures must resist the temptation to develop a program around a particular technique (e.g., role playing), experience (e.g., staff retreat), or t0pic (e.g., Management by Objectives) Simply because it is familiar, enjoyable, or currently in vogue. On the contrary, program topics must be carefully formulated in response to specifically identified needs, interests, or performance problems. Particular techniques and exper- iences must only be selected when they are best suited to providing the particular conditions of instruction necessary to insure participants' mastery of the pro- gram's objectives and, in turn, ability to achieve institutional, divisional, and professional goals. Those who design staff development programs around a particular technique, experience, or topic rather than designing and implementing such programs in accordance with educationally sound instructional procedures run the risk of investing time and resources with little or no assurance that desired outcomes will be identified or achieved. Such risks become increasingly intolerable when resources available to fulfill the goals of the division are limited. A review of the literature presented in Chapter II indicates that overcoming this temptation may be one of the most difficult challenges for student personnel staff members who are interested in or responsible for 276 the effective design and implementation of inservice staff development programs. Many of the programs reported in the literature describe the use of a particular tech- nique, experience, or topic with little or no discussion of whether the program is designed to meet specifically identified needs or to provide the conditions of instruction necessary to insure that desired program objectives are achieved. Fourth, to assist them in the design and imple- mentation of educationally sound staff development pro- grams, designers are encouraged to seek out and draw upon the experience and knowledge of campus staff and faculty who have specific expertise (e.g., needs assess- ment, writing instructional objectives, media selection, program evaluation) in the development of instructional programs. Most student personnel professionals are not expected to thoroughly understand or be skilled in human instruction and learning. Therefore, seeking out those on campus who do possess expertise in these areas to help design and implement inservice staff development programs will strengthen their quality and impact upon staff development. In addition to calling upon individuals outside of the division, steps should be taken to involve staff members from within the division. Their involvement will increase staff understanding of the value of these 277 programs and increase their support for continuing such programs. To insure staff involvement, a task group might be formulated to assist the program designer in assessing needs, selecting staff development topics, managing staff development resources, and coordinating logistics for implementation of staff development pro— grams. Finally, regardless of how committed a division may be to inservice staff development, to maximize the effectiveness of these programs, each inservice staff development program must be designed and implemented in context with the other activities and programs of the division. They must not exist in isolation from the goals and objectives of the division. Rather, inservice staff development programs should be designed and imple- mented to assist staff members to master the skills and knowledge necessary to meet the performance expectations articulated in the job description for each position. Inservice staff development programs must not exist as ends in themselves. The designer must guard against the design and implementation of programs which are irrelevant to the achievement of the personal or pro- fessional goals of individual staff members or the goals of the institution, division, or profession. Programs must facilitate the efforts of the division in general and individual staff members in particular to achieve 278 recognized goals. As a result, programs must be tailored to the particular campus and division which they are designed and implemented to serve. Having elected to use the guidelines and checklist developed in this study and carefully considered the implications presented here, student personnel profes- sionals are prepared to take those steps necessary to design and implement inservice staff development programs which maximize the probability that individual learning will occur. While instruction cannot directly control the internal human event referred to as learning, Gagné notes that: The careful design of instruction can surely increase its probability, and by so doing, can make the entire process of learning more sure, more pre- dictable, and more efficient. (31:312) To have taken the steps necessary to arrange the instruc- tional process in order to effectively contribute to an individual staff member's learning or development makes the effort and commitment necessary to properly design and implement inservice staff development programs a worthwhile investment. Doing so helps insure that stu- dent personnel staff will be and remain prepared to achieve institutional, divisional, and professional goals as well as insure their own continued growth and development. SELECTED BIBLIOGRAPHY SELECTED BIBLIOGRAPHY Ammerman, Harry L., and Melching, William H. "The Use of Objectives in Instruction." In Instruc- tional Design: Readings, pp. 72-81. Edited By M. David Merrill. Engléwood Cliffs, N.J.: Prentice-Hall, Inc., 1971. Anderson, G. Lester. "Comment." Journal of Higher Education 42 (July 1971): 463-66} Ausubel, David P. "A Cognitive-Structure Theory of School Learning." In Instruction: Some Con- temporary Viewpoints, pp. 207- 57'. Edited by Laurence Siegel. San Francisco: Chandler Pub- lishing Company, 1967. Banathy, Bela H. Instructional Systems. Belmont, Calif.: Fearon Publishers, 1968. Bass, Bernard M., and Vaughan, James A. Training_in Industry__ The Management of Learning. Belmont, Calif.: WadSworth PubliShing Company, Inc., 1966. Beeler, Kent D. ”Mini-U: A Promising Model for Student Affairs Staff Development." NASPA Journal 14 (Winter 1977): 38-43. Bellucci, JoAnn E. "Affective Simulation in a Resident Assistant Training Program. Journal of the National Association for Women Deans, Administrators, and Counselorsd9(Spring—I976): l07-l0. Bloom, Benjamin 8.; Hastings, J. Thomas; and Madaus, George F. Handbook on Formative and Summative Evaluation of Student Learning. New York: McGraw-Hill Book Company, l97l. Bonar, John R. "Developing and Implementing a System-Design Training Program for Academic Advisors." Journal of College Student Personnel 17 (May 1976li’l90498. 279 10. 11. 12. 13. 14. 15. 16. 17. 18. 19. 20. 21. 280 Bonar, John R. "Review of an Operating Program Model Using Paraprofessionals." Journal of Colle e Student Personnel 17 (Septefiher 1976): 400-64. Briggs, Leslie J. Handbook of Procedures for the Dgsign of InstructiOn. Washington, D.C}: AmeriEan Institute for Research, 1970. Brunson, Mary A. "Professional Development in a Time of Change." Journal of the National Association of Women Deans and Counselors 30 (Simmer 1967): 151-53. Burnett, Collins W. "Selection and Training of School and College Personnel Workers.” Review ggEducational Research 24 (February 1954): 1-330 "Carnegie Commission's Final Report." The gpronicle of Higher Education 8 (October 9, 1973): 7-17. Catalanello, Ralph F., and Kirkpatrick, Donald L. "Evaluating Training Programs-~The State of the Art." Training and Development Journal 22 (May 1968): 2-9. Clark, R. James; LaFave, Francis; and Clarkson, James. "The Development of the Undergraduate Paraprofessional: Selection, Training and Supervision.” Research in Education 10 (October 1975):—18} Coan, Donald L. "Effects of a Workshop on Per- ceptions About Evaluation." Journal of College Student Personnel 17 (May 1976): l86489. Collins, Duane. "An In-service Training Program for Residence Counselors." Educational and PS cho- logical Measurement 7 (Autumn 1947): 647. Cronbach, Lee J. "Course Improvement Through Evalu- ation." Teachers College Record 64 (May 1963): 672-83. Cross, K. Patricia. "New Roles for Deans and Coun- selors." Journal of the National Association of Women Deans and Counselors 36 (Fall 1972): I§:§6. Davis, Robert H.; Alexander, Lawrence T.; and Yelon, Stephen L. Lgarniug System Design. New York: McGraw-Hill Book Company,l974. 22. 23. 24. 25. 26. 27. 28. 29. 30. 31. 32. 33. 281 DeCecco, John P. The Psychology of Learning and Instruction: EdngationaliPsychology. Engléwood Cliffs, N.J.: Prentice-Hall, Inc., 1968. Ellis, Henry C. The Trnnsfer of Learning. New York: The Macmillan Company, 1965. Ericksen, Stanford C. "The Zigzag Curve of Learning." In Instruction: Some Contemporary Viewpoints, pp.—l4l-79. Edited by Laurence Siegel. San Francisco: Chandler Publishing Company, 1967. Federico, Joseph J. "A Staff Development Model for Student Personnel Services.” Research in Edu- cation 10 (March 1975): 75. Foxley, Cecelia H. "A Workshop for the Support Staff.” Personnel and Guidance Journal 51 (September l972): 203-05. Gaff, Jerry G. Toward Facult Renewal. San Fran- cisco: Jossey—Bass Pdhlishers, I975. Gagné, Robert M. "The Analysis of Instructional Objectives for the Design of Instruction.” In Teaching Machines and Programmed Learning, II, pp. 2ll651 Edited by Robert Glaser. Washington, D.C.: National Education Association, 1965. . The Conditions of Learning. New York: Holt, Rinehart and Winston, Inc., 1965. . "The Implications of Instructional Objec- tives for Learning." In Defining Educational Objectives, pp. 37-46. Edited By C. M. Lindvall. Pittsburgh, Penn.: University of Pittsburgh Press, 1964. . ”Instruction and the Conditions of Learning." In Instruction: Some Contemporary Viewpoinng, pp. 29l-3l3. Edited by Laurence Siegel. San Francisco: Chandler Publishing Company, 1967. . "The Reasons for Specifying Objectives." In Instructipnal Design: Reagings, pp. 81-85. Editedthy M. DavidtMerrill. Englewood Cliffs, N.J.: Prentice-Hall, Inc., 1971. , and Briggs, Leslie J. Principlestof Instructional Design. New York: Holt, Rinehart and Winston, Inc., I974. 34. 35. 36. 37. 38. 39. 40. 41. 42. 43. 44. 282 Glaser, Robert. "Psychological Bases for Instruc- tional Design." A V Communication Review 14 (Winter 1966): 433Ld9. . "Toward a Behavioral Science Base for Instructional Design.” In Teaching Machines and Programmed Learning, II, pp. 771-809. Edited by Robert Glaser. Washington, D.C.: National Edu- cation Association, 1965. Goldstein, Irwin L. Training: Program Development and Evaluation. Monterey, Calif.: Brooks/Cole Publishing Company, 1974. Greenleaf, Elizabeth A. "The Role of Student Staff Members." In Student Development and Education in College Residence Halli, pp. l81-94. Edited By DaVid A. DeCoster and Phyllis Mable. Washing- ton, D.C.: American College Personnel Associ- ation, 1974. Harvey, Virginia P.; Helzer, Timme A.; and Young, Jerry W. "The Retreat: Keystone to Staff Development.” NASPA Journal 9 (April 1972): 274-78. Ikenberry, Stanley 0. "The Organizational Dilemma." gournal of Higher Education 43 (January 1972): 23434. Jackson, Ronald. "Development of Dormitory Staff as Sub-Professional Counselors." Research in Education 3 (February 1968): 37. Jahnke, John C. "A Behavioristic Analysis of Instruction." In Instruction: Some Contempprary Viewpointg, pp. l8l-206. Edited—by Laurence Siegel. San Francisco: Chandler Publishing Company, 1967. Katz, Daniel. "The Motivational Basis of Organi- zational Behavior.” Behavioral Science 9 (April 1964): 131-46. Kaufman, Roger A. Educntiongl System Planning. Englewood Cliffs, N.J.: Prentice-Hall, Inc., 1972. Kemp, Jerrold E. Instructional Design. Belmont, Calif.: Fearon Puhlishers, 1971. 45. 46. 47. 48. 49. 50. 51. 52. 53. 54. 55. 56. 57. 283 Kibler, Robert J.; Barker, Larry L.; and Miles, David T. Behavioral Objectives and Instruction. Boston: Allyn and Bacon, Inc., 1970. Kirk, Barbara. "Techniques of In-service Counselor Training." Personnel and Guidance Journal 34 (December 1955): 204-07. Lane, Elsa C. "The Use of Leaderless Encounter Groups in Staff Training.” NASPA Journal 8 (January 1971): 185-89. Laudicina, Robert, and Laudicina, Eleanor. "Staff Evaluation and the Administrative Process." NASPA Journal 10 (October 1972): 114-17. Leventhal, Allan M., and Pumroy, Donlad K. "Training in Behavior Therapy: A Case Study.” Journal of Colle e Student Personnel 10 (September 1969): 296-361. Lewis, Charles L. "ACPA President's Assessment and Proposals." Journal of College Student Personnel 10 (May 1969)?‘II7:§l. Lindquist, Jack. "Political Linkage--The Academic Motivation Process." Journal of Higher Education 45 (May 1974): 323-43. Logan, Frank A. Fundamentals of Learning_and Moti- vation. Dubuque, Iowa: Wm. C. Brown Company Publishers, 1970. Lopez, Felix M. "Accountability in Education." Kappan 52 (December 1970): 231-35. McAshan, H. H. Writing Behavioral Objectives: A Newnpproach. New York: Harper and Row, I970. McConnel, T. R. "Accountability and Autonomy." Journal of Higher Education 42 (June 1971): McIntrye, James P. "The Management of Student Per- sonnel Programs." Journal of College Student Personnel 15 (Novemberl974lk 487-91. MacKinney, A. C. ”Progressive Levels in the Evalu- ation of Training Programs." Egrsonnel 34 (November/December 1957): 72-77. 58. 59. 60. 61. 62. 63. 64. 65. 66. 67. 68. 69. 284 Mager, Robert F. Developing Attitudgs Toward Learn- ing. Palo Alto, Calif.: Fearon delishers, 1968. . Preparing Instructional Objectives. Palo Alto, Calif.: Fearon Publishers, 1962i , and Beach, Kenneth M., Jr. Developing Vocational Instruction. Belmont, Calif.: Fearon Publishers, 1967. , and Pipe, Peter. Analyzing Performance Prdblems. Belmont, Calif.: Fearon PuhliShers, I970. Miller, Theodore K. "Professional Preparation and Development of Residence Educators." In Student Development and Education in Cplle e Residence Halls, pp. 164-80} Edited by Davi A. DeCoster and Phyllis Mable. Washington, D.C.: American College Personnel Association, 1974. . "Staff Development Activities in Student Affa1rs Programs." Journai_of College Student Personnel 16 (July 1975): 258-64. , and Prince, Judith S. The Future of Student Affairsy A Guige to Student Deyglopment fdr TomorrowTs Higher Education. San FranCiSco: Jossey-Bass PuhliShers, 19761 Mosel, James N. "Why Training Programs Fail to Carry Over." Personnel 34 (November/December 1957): 56-64. Newton, Fred. "The Effect of Systematic Communi- cation Skills Training on Residence Hall Para- professionals." Journal of Colle e Student Personnel 15 (September 1974): 36 469. O'Banion, Terry. "Purposes of College and University Student Personnel Work.” NASPAAgournal 8 (January 1971): 206-12. O'Connor, Kathleen. Learning: nn Introduction. London: Scott, Foresman and Company, 1968. Ohlsen, Merle M. "An In-service Training Program for Dormitory Counselors." Occupations 29 (1951): 531-34. 70. 71. 72. 73. 74. 75. 76. 77. 78. 79. 80. 285 Passons, William R. "In-Service Training for Stu- dent Personnel Staff: A Pilot Project." Journal of the National Association of Women Deans and Counselors 33 (Fall 1969): 34-38. Payne, David A. The Specification and Measurement of Learning Outcomes. Waltham, Mass.: Blaisdell Publishing Company, 1968. Peterson, Marvin W. "The Potential Impact of PPBS on Colleges and Universities." Journal of Higher Education 42 (January 1971): 1-20. Popham, W. James, and Baker, Eva L. Systematic Instruction. Englewood Cliffs, N.J.: Prentice- Hall, Inc., 1970. . ”Objectives and Instruction.” In Instructional Opjectives, pp. 32-52. Chicago: Rand McNally and Company, 1969. Powell, John R. "Inservice Education for Student Staff." In Student Development and Education in College Residence Halls, pp. 195-208. Edited By David A. Déddster and Phyllis Mable. Washing- ton, D.C.: American College Personnel Associ- ation, 1974. Richardson, Richard C., Jr. "Staff Development--A Conceptual Framework." Journal of Higher Edu- cation 46 (May 1975): 301-11. Richardson, Thomas E. "What Do You Teach Here: Staff Development." NASPA Journal 13 (Fall 1975): 36-38. Samler, Joseph. "Professional Training: End Goal or Kick-Off Point?” Personnel and Guidance Journal 31 (October 1952): 15-19. Sandeen, Arthur C. "A Required Course for Under- graduate Resident Hall Assistants.” Journal of Ens National Asgociation of Women Deans ahd Counselors 31 (Fall 1967): 42-45. Schroeder, Charles C. "Adventure Training for Per- sonal Growth and Group Development of RAS." Journal of College Student Personnel 17 (January 1976): 11-l5. 81. 82. 83. 84. 85. 86. 87. 88. 89. 90. 286 Schroeder, Karla; Hill, Clara E.; Gormally, James; and Anthony, William A. ”Systematic Human Relations Training for Resident Assistants." Journal of College Student Personnel 14 (July 1973): 313-16. Shaffer, Robert. "An Emerging Role of Student Personnel--Contributing to Organizational Effec- tiveness." Journal of College Student Personnel 14 (September 1973): 386-91. Shelton, John L., and Corazzini, John G. "The Referral Process in the College Community: Some Guidelines for Residence Hall Paraprofessionals.” Journal of the National Association for Women Deans, Administrators, and Counselors 39 (Shfing I976): I02-06. Siegel, Laurence, and Siegel, Lila Corkland. "The Instructional Gestalt." In Instruction: Some Contenporar View oints, pp. 261-90. (Edited By Laurence Siegel. San Francisco: Chandler Pub— lishing Company, 1967. Silverman, Harold. "Agency Directors and Profes- sional Growth of Personnel.” Personnel and Guidance Journal 35 (February I957): 39I-93. Spurrier, Jack L., and Collins, Pamella C. "Develop- ing a Resident Assistant Training Program." NASPA Journal 10 (January 1973): 259-62. Stamatakos, Louis C., and Oliaro, Paul M. "In- service Development: A Function of Student Personnel." NASPA Journal 9 (April 1972): 269-73. Stufflebeam, Daniel L. ”Toward a Science of Edu- cational Evaluation." Educational Technology 8 (July 30, 1968): 5-12. Sullivan, Howard J. "Objectives, Evaluation, and Improved Learner Achievement.” In Instructional Obaectives, pp. 65-90. Chicago: Rand McNally an Company, 1969. Swearingen, Mildred. "Identifying Needs for In- service Growth." Educational Leadership 17 (1960): 322-35. 91. 92. 93. 94. 95. 96. 97. 98. 99. 100. 101. 102. 287 Thompson, Donald L. "PPBS: The Need for Experience.” Journal of Higher Education 42 (November 1971): 678-91. Truitt, John W. "Factors Underlying the Need for In-service Development Programs in Student Per- sonnel Work." Research in Education 4 (February 1969): 26. “Inservice Education NASPA Bulletin, , and Gross, R. A. for College Student Personnel." No. 1 (June 1966). Tyler, Ralph W. ”Some Persistent Questions on the Defining of Objectives.” In Instructional Design: Rendings, pp. 89-96. Edited by M. David Merrill. Englewood Cliffs, N.J.: Prentice-Hall, Inc., 1971. Wanzek, Robert P., and Canon, Harry. "Professional Growth in Student Affairs.” Journal of Colle e Student Personnel 16 (September I975): 4l8-2I. Westbrook, Franklin D., and Smith, Joel B. "Assist- ing Black Resident Students at a Predominantly White Institution: A Paraprofessional Approach." Journal of College Student Personnel 17 (May 1'67 6W 205-l0 . ”Professional Preparation of School and Sociepy Williamson, E. G. Student Personnel Workers." 86 (January 1958): 3-5. . Student Personnel Services in Colle es and UniverSities. New York: McGraw-Hill, I961. , and Biggs, Donald A. Student Personnel WOrk, A Pro ram of DevelopnentaiiRelationShi s. New Yotk: Jo n Wiley and Sons, Inc., 1975. Wilson, Frances M. "What Makes an Effective Inservice Training Program?” Journal of the National Associ- ation of Women Deans and Counselors’l6’(January Winstead, Philip C., and Hobson, Edward N. "Insti- tutional Goals: Where to From Here?” Journal of Higher Education 42 (November 1971): 669-77. Witkowski, Edward H. "The Economy and the University: Economic Aspects of Declining Enrollments." Journal of Higher Education 45 (January 1974): HICHI lllllllll