INFORMATION TO USERS This manuscript has been reproduced from the microfilm master. UMI films the text directly from the original or copy submitted. Thus, some thesis and dissertation copies are in typewriter face, while others may be from any type o f computer printer. T he quality o f this rep ro d u ctio n is dependent upon th e quality of the copy subm itted. Broken or indistinct print, colored or poor quality illustrations and photographs, print bleedthrough, substandard margins, and improper alignment can adversely affect reproduction. In the unlikely event that the author did not send UMI a complete manuscript and there are missing pages, these will be noted. Also, if unauthorized copyright material had to be removed, a note will indicate the deletion. Oversize materials (e.g., maps, drawings, charts) are reproduced by sectioning the original, beginning at the upper left-hand comer and continuing from left to right in equal sections with small overlaps. Each original is also photographed in one exposure and is included in reduced form at the back o f the book. Photographs included in the original manuscript have been reproduced xerographically in this copy. Higher quality 6” x 9” black and white photographic prints are available for any photographs or illustrations appearing in this copy for an additional charge. Contact UMI directly to order. UMI A Bell & Howell Information Company 300 North Zeeb Road, Arm Arbor MI 48106-1346 USA 313/761-4700 800/521-0600 AN ANALYSIS OF THE MICHIGAN EDUCATION ALTERNATIVE TRAINING PROGRAM USING SELECTED CONCEPTS OF PROGRAM PLANNING By John Gregory Zappala A DISSERTATION Submitted to Michigan S tate University in partial fulfillment of the requirements for the d egree of DOCTOR OF PHILOSOPHY Department of Educational Administration 1996 UMI Number: 9631367 UMI Microform 9631367 Copyright 1996, by UMI Company. All rights reserved. This microform edition is protected against unauthorized copying under Title 17, United States Code. UMI 300 North Zeeb Road Ann Arbor, MI 48103 ABSTRACT AN ANALYSIS OF THE MICHIGAN EDUCATION ALTERNATIVE TRAINING PROGRAM USING SELECTED CONCEPTS OF PROGRAM PLANNING By John Gregory Zappala This investigation sought to determine whether Adult Education Alternative Training (AEAT) providers implemented and used systematic planning for developing training or retraining programs, and what relationship that might have had on meeting program planners’ satisfaction with outcomes and actual program outcomes. The researcher determined the extent to which AEAT retraining program planners used program planning concepts, the extent to which those concepts were perceived to b e important and effective, and the relationship of planning to program outcomes. The study also described and explained mitigating variables affecting the planning process. The data were gathered through a survey of the 1992-93 AEAT program planners, on-site interviews with those program planners, two focus group meetings, and reports, and Department of Education docum ents and reports. The survey included information about the planning process that used 50 separate concepts and planners' perceptions about the importance and effectiveness of those concepts. John Gregory Zappala The interviews provided an opportunity to obtain perspectives of the political and practical considerations in program planning. The focus group meetings suggested a context within which the planning w as developed. The researcher found that the majority of planners w ere satisfied with their program outcom es and that more than 90% of those surveyed used 35 or more (of 50) concepts. In addition, planners perceived developing administrative and instructional plans to be most important, w hereas they perceived developing an administrative plan to be most effective. Developing a marketing plan w as perceived to be least important and effective. Program planners using 45 or more concepts had an average placement rate of 95%, a s compared to program planners using 41 or fewer concepts, who had an average placement rate of 72.42%. Use or nonuse of concepts appeared to have no significant relationship to planners’ satisfaction with outcomes. The interviews and focus group meetings suggested that program planners w ere most successful when they combined technical planning skills with political savvy. Two considerations emerged from the study. First, constructing or planning a program is one role that planners play. Second, program planning is not a panacea. Although this is a critical first step, the complexity of program planning and outcom es requires that serious consideration be given to internal and external forces. ACKNOWLEDGMENTS Kirsch, again, for cheerleading; Pam, Jason, and Jo z for extraordinary support; John and Shirlee for valuing and promoting education; Cas, Rudy, Chuck, Fred, and Marylee for the procedures; Marsha for more patience than anyone I know; Gene and Ron for making it work; The Michigan Jobs Team for their endorsements; Deni for the attitude; The Michigan Customized Trainers for their instincts in program planning and delivery; Zubair and S u e for their professional abilities; Marty for dancing a Ph.D. program by me; Craig and his friends in the dissertation writing group Phil and Suzanne for the Saturday mornings; Jo for the roadmap; And Tom and Leon, gone now, but their friendship and encouragem ent still linger. TABLE OF CONTENTS LIST OF T A B L E S ..............................................................................................................viii LIST OF F I G U R E S ............................................................................................................. ix Chapter I. BACKGROUND OF THE S T U D Y ........................................................... 1 Training and Retraining in Michigan ...................................................... 7 The Problem ................................................................................................ 9 Purpose of the S t u d y ................................................................................ 10 Research Questions ................................................................................ 11 M e th o d o lo g y .............................................................................................. 12 Significance of the S t u d y ......................................................................... 13 Definition of Terms .................................................................................. 14 Limitations of the S t u d y ........................................................................... 17 Summary and Organization of the Study ...........................................17 II. REVIEW OF THE LITERATURE...........................................................19 Introduction ................................................................................................ 19 Theoretical Foundations of Adult Program P la n n in g ........................19 Analyze Planning Context and Client System ...............................26 Justify and Focus Planning ............................................................... 27 Develop Objectives ............................................................................. 28 Formulate Instructional Plan ............................................................. 29 Formulate Administrative P l a n ...........................................................29 Develop Evaluation Plan .................................................................... 30 Develop a Marketing Plan ..................................................................31 Program D e liv e ry .................................................................................. 32 Practical and Political Character of P l a n n in g ................................. 33 Federal and State-Supported Training/Retraining Programs . . . 35 Manpower Development and Training Act ( 1 9 6 2 ) ..................... 37 Vocational Education Act of 1963 ................................................. 39 Economic Opportunity Act: Job Corps ( 1 9 6 4 ) ............................ 41 v New C areers Program (1965) ......................................................... 47 Emergency Employment Act (1971) ...............................................48 Com prehensive Employment and Training Act (1 9 7 3 ) .............. 48 Trade Act of 1974 ................................................................................. 53 Job Training Partnership Act (1982) .............................................. 55 Economic and Worker Assistance Act (1988) ............................. 61 State Retraining Programs ....................................................................62 Quik Start (1981-1 991 )...................................................................... 63 Job Training/Retraining Investment Fund (1983-84) .................. 67 Adult Education Alternative Training Program (1992) ............... 68 Michigan Department of Education ................................................. 70 S u m m a r y .....................................................................................................73 METHODS AND PROCEDURES ........................................................ 74 Introduction................................................................................................ 74 R esearch Q u e s t i o n s ................................................................................75 Description of Research M e t h o d s ........................................................ 76 R esearch D e s ig n .......................................................................................78 Sample Selection .................................................................................... 79 Instrument D e v e lo p m e n t.........................................................................79 Ethnography, Visits, and Interviews ....................................................80 Program Planners Focus Groups ........................................................ 81 Data Collection ......................................................................................... 82 Data Analysis ........................................................................................... 83 Strengths and W eak nesses of the M ethodology...............................84 S u m m a r y .....................................................................................................85 ANALYSIS OF THE DATA .................................................................... 87 Introduction................................................................................................ 87 1992-93 AEAT Program Outcomes ................................................. 89 Program Planners’ Satisfaction With O u t c o m e s ...............................90 Satisfaction With AEAT Activity ........................................................ 91 Dissatisfaction With AEAT A ctivity....................................................93 Use of Planning C o n c e p ts ...................................................................... 97 Important and Effective Concepts That Work: Recom ­ mendations From Planners ........................................................... 119 Relationship Between Planning/Satisfaction/Outcomes .............. 123 Mitigating Variables (Political and Practical Considerations) . . 135 Focus Group: Effective Program Planning ...............................136 Focus Group: Program Modification ............................................. 140 S u m m a r y ...................................................................................................145 V. SUMMARY, CONCLUSIONS, AND RECOMMENDATIONS . . 1 4 7 Introdu ction ..............................................................................................147 Summary of Findings ........................................................................... 148 1992-93 AEAT Program O u t c o m e s ............................................ 148 C o n c lu sio n s..............................................................................................151 Recom mendations for Future R e s e a r c h ...........................................157 S u m m a r y ...................................................................................................159 Reflections ........................................................................................... 160 APPENDICES A. Letter of Approval From the University Committee on Research Involving Human Subjects ............................................... 164 B. Michigan Jo b s Team Letter of S u p p o r t .............................................. 165 C. Mailing to Program P l a n n e r s .................................................................166 D. 1992-93 Michigan Education Alternative Training Program Planning Survey .....................................................................................167 E. Assessing Program Planning in AEAT Activities Com ments . . . 177 F. Individual Interview Consent Form ..................................................... 187 G. Interview Q u e s tio n s ................................................................................. 188 H. Focus Group Meeting N o t e s ................................................................. 190 I. Summary of Planner Satisfaction, Outcomes, and C orrelation................................................................................................ 200 BIBLIOGRAPHY.............................................................................................................. 204 LIST OF TABLES 1. AEAT Planners’ Satisfaction With Program Outcomes ................................90 2. Program Planning Concepts U s e d ..................................................................... 98 3. Frequency of Program Planning Concepts U s e d ............................................99 4. Frequency of Concept use by Concept A r e a .................................................102 5. Importance of Program Planning C o n c e p t s ................................................... 106 6. Importance of Planning Concept Areas by Mean Scores 7. Effectiveness of Program Planning Concepts .............................................. 110 8. Planning Concept Areas and Effectiveness Mean S c o r e s ......................... 114 9. Correlation: Importance and Effectiveness by Rank Order C o rre la tio n ............................................................................................................ 115 10. Program Satisfaction/Percentage Placed in Job ..........................................124 11. Satisfaction/Placement M e a n s .......................................................................... 127 12. Satisfaction and Placement Outcome C l u s t e r .............................................. 128 13. Summary of Concept Areas and Effectiveness Means by Program . . 130 14. Sum m ary of Placement Outcomes and Mean of Mean Effectiveness S co res for All C o n c e p t s .................................................................................. 133 viii ......................... 109 LIST OF FIGURES 1. Problems Related to Company Training Commitment/Responsibility . 138 2. Problems Related to Intercollege Considerations .......................................138 3. Problems Related to A ssessm ent and E v a lu a tio n .......................................138 4. Monetary/Resource Lim itations........................................................................139 5. Problems Related to Developing Objectives ................................................ 139 6. Communication Problems With C o m p a n y .....................................................142 7. Communication Problems With the State .....................................................142 8. Internal Communication P r o b le m s ................................................................... 142 9. Lack of R e s o u r c e s ............................................................................................... 143 10. Business C l im a te ................................................................................................. 143 11. Company Commitment to Training .................................................................144 12. Political Considerations ......................................................................................144 CHAPTER I BACKGROUND OF THE STUDY For more than 25 years, federal and state governments have financially supported retraining programs to e a s e the labor market adjustments required of workers directly or indirectly displaced from their jobs- or underemployed in their existing jobs. The U.S. Department of Commerce publication, Statistical Abstract of the United S tates (1994), indicated the following: 1. National unemployment went from 6,528,000 in 1990 to 9,384,000 in 2. Nearly two million jobs were lost between 1985 and 1990. 3. Manufacturing jobs were lost b ecause of company closings or specific 1993. positions being abolished, or there were production slowdowns. According to Secretary of Labor, Robert Reich (1993), "the need to seek reemployment or higher paying employment in a new occupation or industry requires that displaced and under-employed workers acquire the vocational skills needed in expanding industries, and may also require the enhancem ent of long-forgotten job search skills" (p. 21). Retraining is broadly defined to include both. 1 2 According to Leigh (1990), the principal goals or purposes for publicly sponsored retraining programs are twofold: (a) to reduce the private and social costs associated with u nnecessary delays in the reemployment process, and (b) to assist in the replacement of specific human capital lost when a perm anent layoff takes place. The extent to which th ese two purposes are met may determine whether the retraining program worked. In Losing Ground. Murray (1984) drew on a technical body of social science data and reported that "job training and retraining programs were expected to be a sure bet. They deal with individuals, not institutions, and teaching a person to learn is something we know how to do" (p. 37). But starting with the first evaluation reports in the mid-1960s and continuing to the present day, the data have failed to show the hoped-for results, or anything close to them. "The programs were seldom disasters; they simply failed to help many people get and hold jobs that they would not have gotten and held anyway" (Murray, 1984, p. 37). Murray further stated that history has encouraged government to believe that educated and trained workers are abundant, but demographics continue to play a key role in dictating the priorities of business and industry. For example: 1. Most of the new job entrants will be women, minorities, and immigrants. Many of th ese individuals will not have the skills for new jobs. 2. The overall size of the workforce will decline at the entry level, a s baby boom ers move into older age. 3 3. The rate of the labor force growth will be slower than during the past 12 years. Auletta (1982), in The U nderclass, raised similar suspicions about governm ent’s overall effectiveness in helping individuals in job training or retraining programs. "The programs have to be intensive, and we have to be willing to experience a certain failure rate. S u c c ess is in the ey es of the beholder. It’s how you define su c c e ss and in the end, it has to do with values" (p. 316). Auletta’s studies with the Manpower Demonstration R esearch Corporation (MDRC) offer ample evidence of how difficult it is to reach those who need to be trained or retrained. "There is no pink pill in this business. Progress is not m easured by breathtaking touchdown p a sse s, but by grinding out two, three and four yards at a time" (p. 316). Similarly, Jen ck s (1993), in Rethinking Social Policy, argued that such government-sponsored programs were "not just ineffective, but positively harmful. The problem w as that th ese programs cost a great deal of money and that they hurt the very people they intended to help” (p. 70). Jencks cited social, cultural, and moral indicators and suggested that th ese problems have steadily gotten worse, even with the intervention of the government in job training programs. He reported that intercity crime and illiteracy have not decreased, w hereas te e n a g e pregnancies and welfare recipients have increased. The long-term joblessness of 2 5 -to 50-yearold men continues to rise, while drug u se continues to be a persistent problem. "If we want to reduce poverty, joblessness, illiteracy, violence or despair, we will surely 4 need to change our institutions and attitudes in hundreds of small ways, not in one big way" (Jencks, 1993, p. 203). Like Jencks, Wilson (1987) suggested that government-sponsored training programs will continue to be ineffective, unless far more comprehensive economic and social reform is included in these initiatives (p. 139). Wilson’s major em phasis ap p ears to explain the increases in job lessn ess in black urban communities. He recognized that many factors are at work, but his argument has three points: 1. Jo b le ssn e ss has increased am ong black men b ecau se there are fewer unskilled and semi-skilled jobs. 2. Thetwo-parent blackfamily isdisappearing b ecause male job lessn ess has m ade marriage less attractive. 3. Single parenthood and male jo b lessness have increased b e c a u se the black middle class is moving out of the ghetto. Consequently, job se ek e rs have fewer employed neighbors to help them find jobs. As with various economic development initiatives, the findings of Murray, Auletta, Jen ck s and Wilson vary in detail, but not in pattern. Each sugg ested that efforts to solve the problems of the displaced or underemployed worker are going to be expensive, difficult, and only partly successful. For example, Murray (1984) stated, "People who w ere doing the helping did not su cceed nearly a s often a s they deserved. Why, when their help w as so obviously needed and competently provided, w as it so often futile? In the instances when the help succeeded, what were the conditions that permitted or precluded success?" (p. 10). 5 Wilson (1987), in The Truly Disadvantaged, advocated that public training programs must be designed and administered in close conjunction with a nationally oriented labor market strategy to avoid becoming "enm eshed in local political patronage and being attacked as costly, inefficient, or corrupt" (p. 151). Jen ck s (1993) summarized his argum ent a s follows: "It is not just that we administered good programs improperly, or that sound concepts w ere som etim es converted to operations incorrectly. The error w as strategic" (p. 71). Cervero and Wilson (1994), Sims (1993), Mitchell (1993), Sork and Buskey (1986) and others have suggested that th ese conditions may be grounded in the u se or n onuse of effective and responsible program planning concepts, such a s those advanced by Boyle (1981). For example, Sims (1993) observed that federal or state funded training or retraining programs fail due to "inadequate planning or design" (p. 595) Sim s added that "poor training programs also produce anxiety, resentment, budget reductions, and efforts to sabotage the program" (p. 595). Similarly, Mitchell (1993), in The Trainer’s Handbook: The AMA Guide to Effective Training, detailed the steps in the process of planning and preparing for training: preparing a needs analysis, evaluating effectiveness, researching subject matter, using aids for training, and marketing the training function. Once again, th ese concepts of successful program planning are similar to those of Boyle (1981), a s stated in Planning Better Programs (pp. 44-60). 6 Sork and Buskey (1986, p. 89) reviewed a variety of prominent approaches to planning adult programs. They advanced a generic planning model, similar to that of Boyle and Mitchell, which w as composed of the following steps: 1. Analysis of the planning context and client system to be served 2. A ssessm en t of client system needs. 3. Development of objectives. 4. Selection and ordering of content. 5. Selection, design, and ordering of instructional process. 6. Selection of instructional resources. 7. Formulation of budget and administrative plan. 8. Design of a plan for assuring participation. 9. Design of a plan for evaluating a program. Sork and Buskey (1986) noted that successful program planning includes all of th e s e elements and must be treated by the program planner. This position is expanded upon by Cervero and Wilson (1994), who stated that successful program planners must know how to act responsibly within relationships of power. Similarly, they suggested that "power relationships structure the terrain on which programs are planned and on which planners must act" (p. 12). The training/retraining process goes nowhere if the plan is not adequate (Boyle, 1981; Nadler, 1977). New em p h a se s by accrediting agencies, such a s the North Central Accreditation Association and the American Society for Training and Development, have recognized the need for better program planning and the necessary linkage between 7 the planning process and new m easu res of institutional or Adult Education Alternative Training effectiveness. The basis of any plan of operation, according to Boyle (1981), is the organization’s raison d ’etre. Training and Retraining in Michigan Eighty percent of the Michiganians who will be working in the year 2000 are working now (Nespoli, 1991, p. 18). At the rates experienced in a recent five-year period, 1 in every 12 current workers faces the risk of losing his or her existing job b e ca u se of changing technology or intensifying global competition (American Association of Community and Junior Colleges [AACJC], 1990, p. v). Thus, increases in the productivity of the current Michigan workforce must be a top priority. While this restructuring of the labor market is one paramount attribute of society undergoing economic transformation, the preparation of a retrained labor force is fundamental. The American Society for Training and Development (ASTD) reinforced this concept by confirming that there is a strong link between workforce training and the competitive life cycle of any new strategy, technology, product, or service (Carnevale, 1990, p. 48). Deficiencies in training/retraining programs undermine this cycle, and c ause delays, defects, and customer rejections. Moreover, the research of Hall and Miller (1975), Brecher (1972), and Friedman and Yarbrough (1985) further indicated that the most effective methodology to provide retraining in the workplace is the applied approach, which links actual learning outcom es directly to job performance. 8 Although educational/training institutions are directly related to the communities they serve, the retraining role dem ands a different type of relationship with public and private sector agencies. It includes intricate linkages am ong and between agencies with similar but not always compatible interests. T h ese linkages, such a s the analysis of the planning context and client system to be served; a ss e s s m e n t of client/system needs; development of appropriate objectives; selection, design, and ordering of an instructional process; selection of instructional resources; formulation of an administrative plan; and the design of a plan that e n su res participation and evaluation are critical to the s u c c e ss of the training mechanism (Sork & Buskey, 1986, p. 89). The training/retraining programs previously investigated-CETA, Manpower, theEconomicOpportunityAct.theJTPA, and a host of state and federally funded assistance programs—have shown that th ese linkages may or may not have occurred and "the programs worked in som e places and not in others" (p. 247). Recognizing the importance of skilled and adaptable workers to high-value economic development, state of Michigan policymakers looked increasingly to their educational and training institutions to help in implementing new directions in state economic policy. T hese institutions are variously equipped to upgrade the skills of the workforce through education and training. The Adult Education Alternative Training (AEAT) program w as created by the Michigan State Legislature in 1992 specifically to enhance or better tap the resources of universities, community colleges, intermediate and local school districts, and employment consortiums for 9 economic development. "This program is designed to create jobs and keep Michigan’s economy strong," said Doug Rothwell, CEO of the Michigan Jo b s Commission (Partners. 1995, p. 1) (The Adult Education Alternative, or AEAT program in greater detail in Chapter III.) The Problem The recommendation that em erges m ost strongly from the empirical evidence analyzed by Auletta (1982), Murray (1984), Jencks (1993), and Sims (1993) is that government-funded training, retraining, or similar helping programs should be carefully developed among the program planner, the employer, the participant, and related stakeholders if the activity is to accomplish its objectives. This investigator sought to determine whether Adult Education Alternative Training (AEAT) providers implemented and used a systematic plan for developing a training or retraining program, and what relationship that might have had on meeting program planners’ satisfaction with outcom es and actual program outcomes. The literature suggests that program developers may improve the value of the training/retraining when they implement successful program planning concepts and examine program outcom es (Sims, 1993, p. 592). Without a systematic plan, a lack of em phasis on the determination of the worth of a program can m ean danger in training efforts in the long run. That is, failure to systematically plan training activities leaves open the potential for growth in training without accountability. This may lead to the continuation or even proliferation of ineffective programs, or in times of budget cutbacks, the perception by top administrators that training/retraining programs are 10 superfluous and should be cut. If program planners are to eliminate this uncertain approach to agency support for training, systematic planning and outcome a sse ssm e n t must becom e a part of every program, whether or not key agency stakeholders require it. Purpose of the Study The researcher’s major purpose in this study w as to determine whether and the extent to which Michigan Education Alternative Training planners used selected concepts of program planning to achieve specific objectives. The writer determined the extent to which AEAT retraining program planners used concepts outlined in Cervero and Wilson’s (1994) Planning Responsibly for Adult Education. Mitchell’s (1993) The Trainer’s Handbook. Sork and Buskey’s (1986) "Descriptive and Evaluative Analysis of Program Planning Literature, 1950-83," and Boyle’s (1981) Planning Better Program s. Adapting th e se concepts of successful programs, this investigator identified the extent to which they are perceived to be incorporated into the planning process and their potential relationship to the achievement of the planner’s goals and objectives. These concepts include: 1. Analyzing the planning context and client system served. 2. Justifying and focusing of planning. 3. Developing objectives. 4. Formulating an administrative plan. 5. Formulating an instructional plan. 6. Developing a marketing plan. 7. Developing an evaluation plan. 11 In using th ese concepts of program planning, practical realities that underlie the planning process were addressed, and a context w as provided for the interpretation of a program’s results. A second purpose in this study w as to determine whether the implementation of a planning process supports or e n h an c es the program planner’s satisfaction with outcomes, and actual program outcomes, a s identified by participants’ placement in jobs. The effect of mitigating variables on program outcomes, or those variables not directly related to planning, also w as investigated. In summary, this writer examined the planner’s satisfaction with the program outcomes; which planning components were perceived to be incorporated into the planning design; the perceived frequency, importance, and effectiveness of their inclusion; the possible relationship of th ese linkages to realizing the goals of the program planner; and mitigating variables that may affect program outcomes. R esearch Questions The following research questions were examined in this study: 1. To what extent are Adult Education Alternative Training planners satisfied with the outcom es of specific 1992-93 programs? 2. To what extent do Adult Education Alternative Training program planners u se planning concepts as outlined by Boyle, Mitchell, Buskey, Cervero and Wilson, and others? Are som e program planning concepts used more frequently than others? If so, what are they? 12 3. Do program planners consider som e program planning concepts to be more important than others? Is so, what are they? 4. Do program planners consider som e program planning concepts to be more effective than others? If so, what are they? 5. Do associations exist between using planning concepts and planners’ satisfaction with completing stated objectives and actual placement outcom es? 6. What other mitigating variables may influence the outcom es of the various training activities? Methodology The answers to these and other related issues were pursued through (a) the review of related documents, (b) the review and analysis of actual AEAT program plans (proposals) and outcomes, (c) the completion of an AEAT survey instrument, (d) interviews with program planners, and (e) two focus group meetings of program planners. The survey instrument (see Appendix D) w as developed b a sed on concepts of successful program planning, the extent to which th ese concepts of successful program planning are used, and their relationship, if any, to completing planner objectives. Them es were developed from the program development conceptual frameworks advanced by Boyle, Mitchell, and Buskey. The survey questions were constructed after testing the pilot interview questions and format with five program planners. The interview questions were refined after th em es or issues had em erged from the survey results. The focus group meetings of program planners, coordinated by the Michigan Jo b s team, use a failure m ode and effects- 13 analysis technique. This technique further refines and prioritizes mitigating variables that influence individual program outcomes. R esearch findings of previous government-funded training or retraining programs such a s the Comprehensive Employment and Training Act (CETA), the Job Training Partnership Act (JTPA), and the Job Training and Retraining Investment Fund (JTRIF) were analyzed to compare and contrast the u se or nonuse of reported programming concepts. Significance of the Study In this study the researcher investigated the extent to which concepts of successful program planning were perceived to be included in the AEAT program, and their relationship to the completion of program objectives. This study is significant b e c a u se AEAT planners were asked what program planning concepts were used and what role the implementation (or nonimplementation) of those concepts played in program m ers’ satisfaction with outcomes. The study may raise questions about why som e programs’ objectives were completed w hereas others were not. This accountability study is the first in Michigan to document the u se of program planning concepts in the AEAT initiative. Results may have future policy implications that will strengthen programs. In this study, the researcher will recommend specific planning strategies that may be used by practitioners and planners of customized training and retraining programs. The implicit belief motivating this work is that adult program planners must examine their past 14 experience to determine "patterns and trends before they can adequately consider future programs and policy options" (Jacobs, 1992, p. 1). To date, no local, state, or national data have been available on the degree to which AEAT programs offered to business and industry clients have achieved specific goals related to funding requirements or program design. This study is significant in that it responds to the work of Auletta (1982), Murray (1984), Levitan and Gallo (1988), Leigh (1990), Sims (1993), and other researchers who have argued for more careful and/or critical program development. It is anticipated that this study will make a positive contribution to the training process, recommending program guidelines for future AEAT program planners and trainers. The results, implications, and recommendations of this study are relevant to Michigan community colleges, adult high schools, intermediate school districts, and employment consortiums whose training/retraining programs should not be generalized to similar programs in other training or educational institutions. Definition of Terms The purpose of the Adult Education Alternative Training (AEAT or 107A) program is "to expand educational opportunities for Michigan’s workforce and encourage the establishm ent of programs that will enable the creation of new jobs, retain existing workers for the changing workplace, and will strengthen the sta te ’s economic b a s e ” (Michigan Jo b s Team, 1993, p. 1). A sse ssm e n t is the process of descriptively evaluating the product of the institution in an objective manner. Certain basic decisions must be m ade at the 15 outset for the program effectiveness process to be undertaken. Who is to do it? How is it to be done? What data are to be presented, when, and how? In what m anner are conclusions to be drawn? Implied within this process is mission relevance with its attendant goals, objective strategic planning, and functions of program research (McLeod & Atwell, 1992, p. 32). Community college refers to one of 29 state-supported institutions of higher education that typically provide two-year associate degrees, transfer courses to other institutions, or customized training/retraining programs to area residents (Michigan Department of Education, 1988). Customized training or retraining programs are those activities designed to contribute to the economic growth of a local, regional, or state a rea by preparing, upgrading, and retraining individuals for participation in the workforce of a specific business or industry (Iowa Impact Study, 1991). Economic development is the process of creating new jobs and retaining existing jobs by mobilizing resources to attract new b u sin e sse s while helping other on es prosper (Goetsch, 1988, p. 48). Effectiveness is an a ss e s s m e n t process that determ ines how well an institution su c ce ed s in accomplishing its mission. Objective verification can be of two kinds: tho se objectives that are either accomplished or not, and those objectives that are accomplished in som e degree (McLeod & Atwell, 1992). Failure mode and effects analysis (FMEA) is an analytical technique that identifies potential product-or outcome-related process failure m odes, a s s e s s e s the 16 effects of the failures, identifies the potential process c au se s, and identifies significant process variables to focus controls for the prevention or detection of the failure conditions. This process is used by the Ford Motor Company and is referred to as 8-D problem solving. A fishbone chart is som etim es used in the FMEA process and diagrams "cause and effect" relationships. A mission defines the fundamental, unique purpose that differentiates one organization from another. The more explicit that is, the better it will be understood by the entire organization (North Central Accreditation Association, 1993). Planning concepts are those specific items that provide the basis or foundation for a variety of decisions for all p h a se s of the program planning effort. In som e c a s e s in this study, the words "concept" and "component" are used synonymously. Planning concept a re as are those planning concepts/com ponents that are grouped or clustered into one of eight areas, such a s planning analysis and justification, objectives, administration, instruction, marketing, evaluation, and program delivery. Program planning is defined a s "a deliberate series of actions and decisions through which problems or situations can be changed or improved" (Boyle, 1981, p. 5). According to Crosby (1989), "Quality" is defined a s conformance to requirements, not goodness. It’s achieved through prevention, not appraisal. The quality performance standard is zero defects, not acceptable quality levels. 17 Successful programs were those 1992-93 AEAT activities identified by Michigan Department of Education officials "who completed program objectives" and "did what they said they would do" (MDE, 1995). T hese specific programs were recom m ended by the MDE to be surveyed. Limitations of th e Study This study w as limited by factors inherent in the u se of the survey, interview, and focus group instruments. The validity of the study w as affected by the honesty and accuracy with which participants responded. This study w a s conducted with the following specific limitations: 1. The study w as limited to designated (i.e., nam ed a s a contact person for specific projects) program planners who designed, implemented, and evaluated AEAT training/retraining programs in 1992-93. 2. The study did not include input or resp o n ses from educational or training agency representatives who were not directly responsible for program planning or development. The study did not include input or resp o n ses from the b u sin e sse s or industries participating in the Michigan Education Alternative Training program. Therefore, results, implications, or recommendations may not be appropriate for generalizations beyond participating practitioners. Summary and Organization of the Study In Chapter I, an overview of the need for AEAT program planning and project objectives a s se ss m e n t w as provided. The problem statem ent and purpose of the 18 study were presented, a s well as research questions and the study methodology. The significance and limitations of the study and definitions of key term s also were discussed. Chapter II contains a review of the literature, focusing on models of effective and responsible program planning, and previous government-funded training or retraining programs. The procedures used in this study and the individuals being surveyed/interviewed are explained in Chapter III. A description of the instrument and the procedures used in collecting and analyzing the data are also presented in this chapter. The research findings are presented in Chapter IV. Conclusions of the study and recom mendations for future activity are to be found in Chapter V. CHAPTER II REVIEW OF THE LITERATURE Introduction Chapter II contains a review of the related literature and is divided into two topic areas: (a) the theoretical foundations of adult program planning and (b) previous federal or state supported training/retraining programs. Theoretical Foundations of Adult Program Planning The purpose of this section is to review the concepts, models, and principles that form the theoretical foundations of educational planning and to suggest the types of research that will strengthen and expand those foundations. Educational planning is a decision-making process that produces the outcome and the design specifications for a systematic instructional activity that is expected to change human activity in som e respect (Sork, 1990, p. 74). Planning and plans are tools to increase the amount of control exercised over events and outcom es of events. A decision to plan is a decision to control. Conventional wisdom in adult and continuing education (Boone, 1985; Boyle, 1981; Mitchell, 1993) sug gests that planning should be a highly participatory activity involving representatives of the client group, and content and process experts who are familiar with both educational planning principles and relevant learning theory. 19 20 More recent literature, such a s that advanced by Cervero and Wilson (1994), h as su g g ested that although technical knowledge and skills are n eeded to plan well, they are not enough. Cervero and Wilson argued that ethical vision coupled with political knowledge and skill are n ecessary to plan responsibly for the education of adults. Most traditional program planning m odels discussed in the literature ap p ear to be b ased on the logic of system atic planning and are "linear in design" (Murk & Wells, 1988, p. 45). In system atic planning, the process focuses first on clarifying or defining goals or objectives, then on selecting resources and strategies to achieve the objectives. According to Sork (1990), system atic planning is b ased on four assum ptions: 1. The context in which planning occurs m ust be relatively stable economically, philosophically, politically, and socially. If stability d o es not exist, then system atic planning h as limited utility b e c a u se the ends and m ean s are constantly changing. 2. Clarifying ends logically p reced es identifying m eans. 3. The b est plans are developed when rational choices are m ade about which m ean s are m ost likely to produce the desired ends. 4. In order for plans to be effective, there must be m odest agreem ent am ong stakeholders on w hat is to be achieved and how it will be achieved. The work of Tyler (1949) is cited a s providing the structure for educational planning. (Knowles indicated that, before that time, educators of adults had no 21 theory to support their practice, so they relied on intuition.) According to Sork and Buskey, Tyler contributed two fundam ental ideas. First, he em phasized that educational activities should b e based on clear objectives that describe what the learner is expected to know or be able to do. Second, he proposed that evaluation should b e b a sed on the d eg ree to which objectives are achieved. Many educators of adults have noted that the logic of Tyler’s framework has also becom e the classical viewpoint in program planning (Apps, 1979; Brookfield, 1986). Houle (1972) placed primary em phasis on the planner’s ability to m ake judgm ents in a specific context and to justify them . He proposed a two-part system of program design. He believed that the decision points in program planning are not a se t of logical steps, but a complex of interacting elem ents that are dealt with at various points throughout the planning process (Cervero & Wilson, 1994). Thus, according to Houle, the quality of any particular program "depends in large m easure upon th e wisdom and com petence of the person making the choices" (Cervero & Wilson, 1994, p. 223). Pennington and G reen (1976) su g g ested that educational planners employ m odels in only a general se n se and alter planning when confronted with unanticipated constraints or opportunities. This may explain why much of the literature regarding program planning is referred to a s "planning theory" consisting of how planning should be done, not how it actually is done. Still, planning m odels provide a framework for the application of theories that are relevant to educational design. 22 Apps (1979) viewed program planners a s problem solvers who should rigorously apply five principles in their practice. T hese principles include (a) a sse ss in g learners’ needs, (b) defining objectives based on th ese needs, (c) identifying learning experiences, (d) organizing those learning experiences, and (e) evaluating the program in term s of the objectives accom plished. According to C ervero and Wilson (1994), the planning literature h as repeated this structure for so long that theorists s e e it a s the sine qua non of good program planning. Knowles (1980) expanded on the work of Tyler and asked what educators of adults do. His data suggested that adult educators help learners diagnose their n e ed s for learning, plan a seq u en ce of experiences that produce desired results, create conditions that will c a u se learners to want to learn, select effective m ethods and techniques, provide hum an and material resources to produce desired learnings, and help learners m easure the outcom es of the learning experiences. In Planning Better Program s. Boyle (1981) defined program planning a s "the art of designing and implementing a course of action to achieve an effective educational program" (p. 42). This simple definition implies that the program planner is involved in reaching decisions through th e implementation of a rational planning or developm ental model. Like Pennington and Green (1976), Boyle recognized that a completely rational model is rarely, if ever, achieved. This concept se em s to be supported by his statem ent that the "beliefs, attitudes and values of the program m er are very important in developing a conceptual framework for program development" 23 (p. 19). They provide th e basis or foundation for a variety of decisions for all p h ases of the total programming effort. Boyle (1981) further stated that program planning is a complex decision­ making process with m any variations. To simplify and bring order to this complex process, he suggested a program planning model that can b e used to represent the salient characteristics of planning. Following is a brief review of 15 concepts considered important for program planning or development. 1. Establishing a philosophical b ase for programming: The program planner should clearly identify his or her beliefs about th e program, the learner, and the planning process. 2. Situational analysis of problem s or need s of clients: This com ponent em p h asizes the study, analysis, interpretations, and judgm ents about thecomm unity and clientele. 3. Involvement of potential clientele: This m eans including participants in the process and connecting them to the process. 4. Levels of intellectual and social developm ent of clientele: The program m er m ust understand and provide for differences in the social and educational background of the clientele. 5. S ources to investigate and analyze in determining objectives: No single source of information is ad eq u ate to provide a basis for com prehensive decisions about educational objectives. 24 6. Recognition of institutional and individual constraints: Som e of th e se constraints include organizational philosophies, resources available, beliefs, and assum ptions. 7. Criteria for establishing priorities: Priority setting is a continuous process that tak es place during all p h a se s of programming, including delineating needs, specifying goals, identifying target audiences, defining resources, and determining necessary actions. 8. D egree of rigidity or flexibility: The program must be allowed to develop in order to m eet the specific n e ed s and to u se the m ost appropriate resources. 9. Legitimation and support with formal and informal power structures: The planner, the planning agency, and th e program itself may need support in order to be successful. 10. Selecting and organizing experiences: The program m er m ust focus on the learner and what the learner will experience. 11. Identifying instructional design: This involves the selection of the m ethod, the techniques, and the devices needed to bring about appropriate results. 12. Using effective promotional priorities: All successful promotional efforts m ust start with an organized and inclusive plan that considers objectives, audience, media characteristics, and deadlines. 25 13. Obtaining resources necessary to support program: The planner m ust facilitate the organization’s efforts to obtain continuity and ad eq u ate financial resources. 14. Determining the effectiveness, results, and impact: A concept of evaluation should be developed to m eet the n e ed s of th e participants. 15. m akers: Communicating the value of th e program to appropriate decision It is essential that the individuals involved in making decisions about funding program s obtain a clear understanding of the value and limitations of the program. In summary, Boyle (1981) defined program planning or developm ent a s "designing a course of action to achieve a quality program" (p. 51). Boyle believed that the 15 concepts discussed are the e sse n c e of ideal planning. He recom m ended that the planner u se them a s guidelines in implementing a process in a given programming situation. Boyle stated further that the ideal philosophical framework allows the planner to system atize considerations while retaining flexibility forchange. In addition, the philosophy m ust be constantly revised and adapted to m eet new situations in programming. Sork and Buskey’s (1986) evaluation of the program planning literature from 1950 to 1983 indicated that 93 planning m odels w ere advanced during that time period. After studying th e se program planning models, Sork and Buskey developed a generic model of sev en specific steps used in completing planning tasks. T hese step s served a s the theoretical framework for this study. This model departs from 26 m any in th e literature by beginning with an analysis of the planning context and client system . It is an interactive model in the se n s e that decisions m ade at any step can influence decisions m ade at other steps. It is also a linear model that is b a sed on the logic of system atic planning in which certain elem ents logically precede or follow other elem ents. The generic model includes the following: 1. Analyze planning context and client system 2. Justify and focus planning. 3. Develop objectives. 4. Formulate instructional plan. 5. Formulate administrative plan. 6. Develop evaluation plan. 7. Develop marketing plan. Analyze Planning Context and Client System In this discussion, "client" is used to designate those who are eligible for the attention of the planner b e ca u se they are included within the m andate of the organization in which the planner works. Analysis of the planning context involves developing a detailed understanding of the milieu in which planning occurs (Murk, 1990, p. 77). The organization in which a planner works has a structure, leadership, policies, and procedures that may have important implications for later planning. The planning m ust also consider other stakeholders like professional associations, governm ental agencies, special interest groups, and com peting organizations. 27 Two fundam ental com ponents constitute analysis of the client system . First, the boundaries of the system are determ ined. Second, relevant client characteristics are identified. Sork (1991) stated that a relevant characteristic is one that should be taken into account to improve the plan. If a relevant characteristic is not considered, then th e plan may be incomplete or faulty. Hanson (1989) believed that research findings about various clientele characteristics, such a s surveys, should be incorporated into educational planning. Mitchell (1987) believed that three questions need to be asked of the trainees before training begins: 1. Are trainees ready to learn the m aterial? 2. Have sufficient opportunities been provided for th e trainees to su c ce ed ? 3. Is there sufficient opportunity to practice what h as been learned? Although this process can be costly and time consum ing, the knowledge gained can be directly applied to the planning process. O ne purpose of the present study w as to determ ine w hether an analysis had been conducted within th e planning context and with the clients to be served. Justify and Focus Planning Several techniques can be used to justify and focus the planning effort. Houle (1972) su g g ested that program ideas can em erge from a wide range of sources and situations. Houle further stated that this p ro cess includes m ore than an informationgathering process b e ca u se identifying desired capabilities and setting priorities involve making value judgm ents. O ther so u rces for justifying and focusing planning 28 include interest and dem and inventories, practice audits, m arket tests, trends, problem s, and situational analyses (Knox, 1986; Levine & Cordes, 1984; Sork & Fielding, 1987). Insights from th e se a sse ss m e n t tools improve a practitioner’s understanding of how planning decisions are m ade and what c o n seq u en ces can be expected from allocating limited educational resources. This research er determ ined the extent to which various com ponents had been used to justify and focus the planning. Develop Objectives "Objectives are detailed descriptions of expected program outcom es" (Sork, 1990, p. 79). Objectives describe expected behaviors of the learner following the program. Understanding how objectives are developed and by whom, how they are related to n e ed s or other sources of program ideas, how they are used by planners and instructors, and to what degree they are used would be important contributions to educational theory. Mitchell (1987) stated that the developm ent of objectives is "when the planner h as pulled together all the thinking to set down exactly what the trainees will achieve so a s to fulfill the goals and bring about the chan g es m andated by the n e ed s analysis" (p. 161). The present researcher determ ined the extent to which planners developed objectives in planning AEAT program s, and the relationship this m ay have had to program outcom es. 29 Form ulate Instructional Plan All those activities that are considered "necessary and sufficient to bring about th e desired learning" (Sork, 1990, p. 80) m ake up the instructional plan. Learning styles, motivation, instructional technique, conditions of learning, instructional design, media, and adult developm ental sta g e s are used to develop the educative structure of the program. Tracey (1992) su g g ested that th ese variables need to be strongly considered so that teaching points and learning activities can be arranged in the best seq u en ce for learning. According to Murk and Wells (1988), selecting and sequencing instructional activities and specifying the requisite instructional resources are the essential tasks at this sta g e of planning. This investigator determ ined the extent to which an instructional plan w as used and determ ined its perceived importance and its effectiveness in accomplishing program goals. Form ulate Administrative Plan Boyle (1981) and Sork and Buskey (1986) stated that this step of planning involves consideration of the financial dim ensions of the program, the strategy for assuring participation of the client group, and the administrative tasks required to implement the plan. Financial dimensions include estimating costs of the resources to be u sed in th e program, determining how th e se costs will be recovered, and setting program fees. Break-even points and direct, indirect, and overhead costs are needed to understand the calculations and considerations involved in program finance. 30 Ensuring participation in the program is also a challenge when the clients are not compelled to participate. Sork (1990) recom m ended that adult educators review the principles of marketing re s e a rc h -th a t is, those concepts that can com m unicate the character of a program in such a way that is attractive and inviting to th o se who learn about it. Murk and Wells (1988) stated that administrative details, such a s establishing adequate, reliable administrative personnel to maintain enrollment, registration and bookkeeping functions are important to program design. Colgan (1993) referred to this com ponent of planning a s "positioning." In this phase, sh e su g g ested that this highly analytic model em phasizes econom ic and political considerations. For exam ple, the planner m ust consider conditions that may be unique to the training institution, political issues, and other anecdotal information. This research er determ ined which of th e se administrative com ponents had been used and the extent to which they w ere believed to be important or effective in completing AEAT program goals. The planning information obtained from staff, advisory councils, and program participants n eed s to be heard —and addressed. Develop Evaluation Plan B ecause the intention of m ost training or retraining program s is to improve perform ance, evaluation m odels that focus on determining ch an g e in practice are m ost relevant. Key com ponents of this portion of the planning include formative and sum m ative evaluation procedures. According to Murk and Wells (1988), formative evaluation m easures the program ’s effectiveness at each p h a se of the planning 31 process relative to the overall goals and objectives of the program. Formative evaluation procedures are also vital in monitoring the overall process or identifying potential chan g es that m ay occur. This type of evaluation often g e n erates constructive criticism that is both n e cessary and useful to the planners and the su c c e ss of th e program. Mitchell (1987) stated further that formative evaluation is also a vital function for the trainees, a s it provides a source of constant feedback to develop the motivation to continue. To determ ine w hether the program w as effective and w hether the participants actually achieved their desired learning outcom es, a sum m ative evaluation is conducted near the end of the program activities, to sum up what went well and what did not. According to Simms (1993), planners "must dem onstrate that their program s get results, improve job perform ance, m ake efficient u se of resources, and bring satisfactory returns on training dollars invested” (p. 592). This research er determ ined w hether AEAT planners had developed evaluation com ponents so that th e se types of claims could be substantiated, and w hether using th e se com ponents w as perceived to be important and effective in completing goals. Develop a Marketing Plan Boyle (1981), Sork (1990), and Mitchell (1993) all detailed the need for a strategic marketing plan in developing any program. It is essential that all stakeholders involved-including those individuals making decisions a b o u tfu n d in g obtain a clear understanding of the value and limitations of the program. Mitchell (1987) su ggested that several steps are essential to the marketing effort. T hese 32 step s include (a) researching and defining the target populations to be served, (b) clarifying the exact nature of the service to be delivered to all stakeholders, and (c) choosing the m ost effective channel to com m unicate th e program to th o se involved. The different m ethods available for u se in advancing the program vary from the informal, face-to-face contact to the m ore formal research report. The program planner will need to develop this plan using the m ost appropriate content and medium for the various constituencies that need to have specific communication. This resea rch e r determ ined w hether any of th e se marketing elem ents had been used and the extent to which they w ere believed to be important and effective in reaching program goals. In their com prehensive review of the literature on planning program s for adults, Sork and Caffarella (1989) concluded that there are shortcom ings in the planning literature. O thers involved in adult education have voiced similar concerns about th e incom pleteness of program planning literature (Miller, 1989; U sher & Bryant, 1989). It is for th ese reasons that two additional com ponents to successful program planning w ere studied: (a) actual program delivery and (b) the practical and political character of planning. Program Delivery Building a theory that takes into account the exigencies of day-to-day responsibilities of practitioners must be undertaken if planning theories are to be taken seriously (Sork & Caffarella, 1989). Similarly, Murk and Wells (1988) docum ented the im portance of actual program implementation. The equation for a 33 successful program includes continual coordinating and monitoring during the delivery of the program. Actually reaching the goals and objectives of the program m ay hinge on the trainer’s ability to provide the n e cessary instructional m aterials, equipm ent and supplies. The facilities, th e instructional content, and the p ace of instruction should b e monitored. Programming or scheduling adjustm ents need to be m ade to ensure satisfaction and optimum learning am ong the participants. Implementation involves accom m odating the special n e e d s of program participants and granting them appropriate credit at the conclusion of th e activity. It is for th ese rea so n s that this study included questions regarding th e implementation/delivery of a program, and asked program planners to com m ent about its overall effectiveness and im portance in m eeting established objectives. Practical and Political C haracter of Planning For program planners, three kinds of knowledge and skill for constructing program s are n ecessary (Forester, 1989; H aberm as, 1971). They need technical knowledge and skill in how to construct program s effectively. This can include designing survey and evaluation instruments, organizing learning activities, writing budgets, selecting and training staff, and publicizing program s. Planners also need political knowledge and skill in order to be able to get things done with the people in the social and organizational contexts in which they work. For instance, they need to develop trust and respect, understand the formal and informal power structure of the organization, and know which strategies will and will not work in a given situation. Program planners also need ethical knowledge in term s of both the 34 educational program s that are constructed and the sociopolitical relationships that are reconstructed. They m ust focus knowledge and skill on the im portance of nurturing a dem ocratic planning process in the face of a power structure that either th reaten s or supports the vision of responsible planning. In addition, Cervero and Wilson (1994) argued that, to improve practices, the program developer’s attention m ust also be on the practical and political character of programming. Their central thesis w as that pragm atic planners m ust be able to read organizational power relationships in order to anticipate conflict and provide support in carrying out a vision of planning that is "substantively democratic" (p. 115). As program planners may know, the context for program planning is not always m arked by co n sen su s and cooperation in political relationships. Planners m ust negotiate betw een conflicting interests in an arena w here power relationships are asym metrical and complex. This consideration w as further supported by F orester (1989), who stated, "Planning, while ignoring the opportunities and dangers of an organizational setting is like walking acro ss a busy intersection with o n e ’s ey es closed" (p. 7). According to Cervero and Wilson, planning a program is not simply a m atter of individual m astery and intuition. It is also a social activity in which people such a s planners, teachers, learners, and institutional leaders se e k to construct a program together. This "social construction form s the core of planning by giving meaningful form to a program that is recognizable, coherent and attainable by a variety of 35 interested parties" (p. 155). It is for th e se reaso n s that this researcher included interviews and focus groups regarding the politics of program planning, a s well a s the source of th e planners’ influence a s evidenced in their ability to m anage and negotiate the interests of those involved. Federal and State-Supported Training/Retraining Program s The fact that United S tates business and industry face a workforce crisis has been accepted (Lane. 1992). Emerging from a period of irreversible econom ic change in the 1980s, the United S tates now confronts a new global reality. For exam ple, several forces are reshaping the workforce and the nature of work, including increasing work diversity, competing dem ands of work and family, global competition, the growing importance of strategic hum an resource planning, the need to reeducate em ployees for new technologies and more dem anding jobs, and renew ed interest in ethics and social responsibility (Coates, Jarratt, & Mahffie, 1990). Urgent attention m ust be given to ad d re ss th e se workforce needs. A sn ap sh o t of the workforce in m any American communities su g g e sts countless a re a s w here further training and retraining are imperative (Waddell, 1991). Kantor (1992) stated that the successful com pany m ust be designed so that "the workforce is multiply-skilled and multiply-trained and can be redeployed quickly a s issu es change" (p. iv). According to Saul (1990), "the workplace will be a different place everywhere" (p. 52). Work will require higher quality perform ance by workers, a s well a s m ore interpersonal relations am ong workers. M anagem ent strategies will require higher thinking and analysis skills and collaborative work patterns from all 36 levels of workers. New skills and values in the a re a s of interpersonal relations, collaborative work styles, and critical thinking will be n e ce ssa ry for persons at all levels of the organization. The training question, then, is how can th ese concerns be adequately ad d re ssed in the shortest tim e? W ork-based training would m eet th e challenges of increasing the skills and knowledge of em ployees and bring with it increased productivity and a respectable position in the global econom y. The leaner, sm aller workforce will heighten the im portance of training and adaptability. If present trends continue, the "gap betw een need and capability will expand and prevent the U.S. from increasing its competitive ed g e in som e industries and retaining it in others" (Lane, 1992, p. 4). Anthony C am evale (1990), of the American Society for Training and Development, reported that the investment in learning on the job has contributed m ore than half of all in creases in the nation’s productive capacity in the last 40 years. That is alm ost three times greater than the investm ent in m achine capital has produced (Lane, 1992). The most globally competitive com panies are already making the em ployee training/retraining investm ent. Through their investm ent in hum an capital and strategic development, th e se com panies have been able to build a workforce that can m ake m ore effective u se of technology, develop collaborative and efficient m an ag ers and em ployees, and be m ore readily able to solve problem s through creative solutions that capture the imagination of th e m arketplace. This em phasis in training and retraining has served to usher in a new activism and a new focus in federal and state-funded econom ic developm ent program s 37 (Committee for Economic Development, 1986; Fosler, 1988; Osborner, 1988). Although the federal governm ent has prom oted the welfare of the citizenry since the earliest y ears of the republic, sustained employment and training efforts focusing on the displaced or underem ployed em erged only a quarter century ago (Levitan & Gallo, 1988). Since 1962, the federal governm ent and individual state governm ents have provided the following retraining program s to e a s e the labor market adjustm ents required of workers directly or indirectly displaced from their jobs by a m ass layoff or plant closure (Leigh, 1990). M anpower Developm ent and Training Act (19621 The federal governm ent’s first com prehensive attem pt to provide adjustm ent a ssista n c e to displaced workers w as the Manpower Developm ent and Training Act (MDTA) (Leigh, 1990). P a sse d in 1962, the MDTA represented the response of C ongress to a rising unem ploym ent rate coupled with growing concern over the effects of technological change on the employment options of m id-career adult workers. According to Murray (1984), the Kennedy Administration saw them selves a s "hardnosed idealists who would be able to get results w here the social workers had failed. Their promise: the able-bodied will be on their way to perm anent selfsufficiency" (p. 23). The primary objective of this initiative w as to provide retraining for workers w hose skills had been m ade obsolete by new technology. By the mid-1960s, an improved labor market and lessened concern over autom ation led to a shift in 38 interest and funding aw ay from the reem ploym ent problem s of displaced workers and toward the employability of disadvantaged young people and welfare recipients. Taken a s a whole, the M anpower Developm ent and Training Act w as not claimed a s one of K ennedy’s achievem ents. It is a fact, however, that "social welfare spending under his administration rose less rapidly than it had under E isenhow er’s ” (Murray, 1984, p. 23). A num ber of evaluation studies of MDTA appeared in the late 1960s and 1970s, but early attem pts at evaluation w ere generally ham pered by the lack of a com parison or control group, a s well a s th e a b sen c e of good information on earnings. Boyle (1981) stated that, in the evaluation of such a program, a determination must be m ade about w hat learners have achieved through program participation. The fundam ental problem in program evaluation is developing a reliable methodology for a sse ssin g w hat would have happened to participants had they not enrolled in the program (Leigh, 1990). Without a com parison or control group, analysts interested in obtaining net impact estim ates are basically limited to participants a s their own control group by comparing post-program labor market outcom es, like earnings, with the level of participants’ own pre-program earnings. Taggart (1981) found that participants enrolled in the MDTA program increased their earnings betw een $250 and $300 in the first year after termination, w hereas Murray (1984) reported that this figure actually d e c re a se d by half after five years. In sum m ary, a panel study of the effects of this vocational training concluded that w ag es of program participants increased in som e c a s e s (Kiefer, 1974). The 39 review of the literature does not include any reference to the role of program planning and its influence, if any, on th e s e training outcom es. The relationship betw een program planning and program outcom es w as analyzed in this study of the Adult Education Alternative Training program. Vocational Education Act of 1963 Before 1963, little attention w as given to satisfying individual learning n e ed s through participation in the appropriate p h a se s of two or m ore occupational fields (Lamar, 1978). Efforts to improve the quality of instruction were largely confined to each occupational field. This pattern w as changed with the p a ssa g e of this 1963 Act. The em phasis shifted from occupational fields to serving vocational education n eed s. Increased em phasis w as also placed on improving the quality of instruction by the u se of supporting services such a s administrative supervision, vocational guidance, research and curriculum developm ent, and program evaluation. A greater level of im portance w as placed on developing com prehensive vocational educational program s "through cooperation and coordination in the planning process" (Lamar, 1978, p. 18). Lam ar (1978) stated that the Vocational Education Act changed the planning p ro cess for vocational education. Instead of using vertical planning, or "top down" planning dictated at the state level to the local school districts, planning m oved horizontally and upward to include the federal agencies involved in vocational education. This new pattern occurred b e c a u se vocational education had becom e a national concern and n eeded support at th e national level. According to Lamar, 40 the pro cess tended to strengthen vocational education, especially the planning process, b e c a u se it: 1. Provided additional resources from federal agencies. 2. Brought about a stronger national, state, and local linkage in dealing with vocational education and thereby provided increased support to the total planning effort. 3. Extended the planning process acro ss all facets and a re a s of responsibility (including the involvement of individuals, agencies, and advisory groups that have a vested interest in the planning process and the ultimate outcom es of vocational education). The pream ble to the Vocational Education A m endm ents of 1968 reads, "the C ongress finds it n e ce ssa ry to reduce the continuing seriously high level of youth unem ploym ent by developing a m eans fo r. . . better job preparation forthose young people who end their education at or before the completion of high school" (Lamar, 1978, p. 245). That position statem ent set the tone for placing further importance on state planning, including the requirement to develop long-range program plans for vocational education. Advisory councils w ere established to assist boards in developing plans that w ere responsive to labor m arket needs. In sum m ary, the Vocational Education Act of 1963 and 1968 A m endm ents greatly expanded the federal role in vocational education and im posed new planning requirem ents on sta te s in order to be eligible for federal funding. The Act also enabled sta te s to broaden th e scope of occupational a re a s to be supported by 41 federal funding. Federal expenditures for vocational education jum ped from $55 million in 1964 to $234 million in 1966. "While the 1963 Act presented the states with vast opportunities to serve more people and to prepare them for m any more occupational areas, it presented them with planning responsibilities for which they had little preparation" (Lamar, 1978, p. 78). Labor-market information n ecessary for sound planning w as not then available to state planners, although the Act required vocational education ag en cies to develop cooperative arrangem ents with their respective employm ent agencies. This research er investigated the role of program planning, such a s that advanced by the p a ssa g e of the Vocational Education Act of 1963, and its relationship to planner satisfaction of program outcom es. Economic Opportunity Act: Jo b Corps (1964) Several new m anpow er program s w ere added by the Economic Opportunity Act of 1964, which w ere designed to "declare w ar on poverty" (Levitan & Gallo, 1988, p. 5). The most ambitious com ponent of the Economic Opportunity Act w as th e Jo b Corps. It aimed to interrupt the "vicious cycle of poverty" (Taggart, 1981, p. 13) by providing a structured residential environm ent for learning and developm ent w here poor youths aged 14 to 21 could e sc a p e from deprivation and realize their full potential. The Job Corps statutory goal w as "to assist young individuals who need and can benefit from an unusually intensive program, operated in a group setting, to becom e m ore responsible, em ployable and productive citizens" (Levitan & Gallo, 1988, p. 123). 42 Work experience w as also used to help needy adults, including public a ssista n ce recipients. The Economic Opportunity Act initiated job creation program s for the hard-to-em ploy who w ere left behind despite the econom ic growth of the mid 1960s. The act, in theory, w as designed to create jobs for older workers and welfare recipients on the assum ption that work w as preferable to dependency. Vet, according to Levitan and Gallo (1988), the unemploym ent rate rem ained "stuck" betw een 5% and 6% throughout the duration of the Job Corps program. Although the intention of the legislation w as to promote "maximum feasible participation" (Levitan & Gallo, 1988, p. 5), the institutional result w as the em ergence of community action agencies and com m unity-based organizations a s advocates for the poor and deliverers of the services. A prominent programming concept advanced by Knowles (1980), Boyle (1981), and Sork and Caffarella (1989) is the n eed to involve clients in the program developm ent process. Such a process should en co u rag e the involvement of individuals in relation to groups. There is no evidence that th e Jo b Corps facilitated this involvement. Boyle (1981) added that it is essential that the purpose or reaso n s for involvement are congruent with the m ethods and resources used. The major source of program instability has been widely fluctuating funding support and attem pts by Presidents Nixon and R eagan to abolish the Corps, resulting in capacity enrollment ranging from 25,000 to 40,000. In inflation-adjusted 1986 dollars, Job Corps funding reached m ore than $1 billion in 1966, but dropped to $300 million in the mid-1970s (Levitan & Gallo, 1988). Tyler (1949) and Boyle 43 (1981) stated that identification and acquisition of resources n ecessary to implement the learning activities is important to program su ccess. Auletta (1982), Murray (1984) and Jen ck s (1992) indicated that the Job Corps did not have ad eq u ate reso u rces to support program success. The Nixon and R eagan Administrations’ efforts to eliminate or scale back the Jo b C orps diminished the program ’s cost effectiveness (Levitan & Gallo, 1988). The program ’s utilization rate, a m easurem ent of average center enrollment com pared to capacity, declined more than 99% in 1983, increasing costs by about $600 per Jo b C orps m em ber each year. Job Corps director Peter Rell testified before a congressional comm ittee that the efforts to end the program "were the major reason" behind recruitm ent difficulties, b ecause young people w ere wary of enrolling in a program which might imminently close" (Levitan & Gallo, 1988). Labor Departm ent staff reductions further impaired federal administration. From 1980 to 1987, federal Job Corps personnel diminished by over a third, from 294 to 190. Job Corps business representatives also noted that the Labor D epartm ent’s annual program reviews, designed to improve program operations, had becom e "more cursory" (Levitan & Gallo, 1988, p. 127). T hese criticisms w ere substantiated by a Departm ent of Labor m em o that concluded, "It se e m s clear from all indications that w e are not doing a fully ad eq u ate job of monitoring." Sim m s (1993) su g g ested that the ability "to modify th e training program based on feedback is critical" to su c c e ss (p. 595). 44 A further criticism of the Job Corps program centers on an apparent lack of effort to determ ine w hether applicants could be better served by alternative program s. In 1979, the U.S. General Accounting Office concluded that the program ’s screening w as so lax that nearly any disadvantaged youth can qualify" (Levitan & Gallo, 1988, p. 131). Boyle (1981) contended that consideration of "the level of developm ent and the n e ed s of th e student" (p. 11) have an important effect on the outcom es of a program. Ensuring that a s m any new entrants a s possible com plete the program is critical to the su c c e ss of Job Corps. The Job Corps w as a voluntary program, and participants could leave a s they wished. The average stay for Job Corps participants w as 7.2 m onths, but a third of the participants left within three months, half of th e s e within the first month. Only a third of the enrollees completed the program in 1985 (Levitan & Gallo, I988). Corps m em bers have indicated the following principal reaso n s for early departures: 1. H om esickness. 2. An inability to adjust to the Jo b C orps’ structure 3. Insufficient pay. 4. Poor screening by recruiters. 5. Enrollees’ inability to m ake decisions about their interests. and rules. The Job Corps’ placement-reporting practices have raised troubling questions also. Until the early 1980s, the program only reported outcom es for Corps m em bers whom it w as able to locate. It had been assu m ed that unlocated participants had the 45 sam e rate of placem ent su c c e ss a s the recorded group of individuals who receive no assista n ce from placem ent agencies. However, this assum ption is questionable b e ca u se perform ance standards discourage placem ent agencies from submitting records for individuals not placed. In addition, placem ent audits have not been verified. For the year ending in June 1986, using the C orps’ estimation procedure, 74% of term inees w ere successfully placed. Based on th e se estim ations, the Job Corps conducted a self-study and found that their efforts "convincingly dem onstrate the program ’s worth in improving enrollees’ employment prospects" (Levitan &Gallo, 1988, p. 154). According to the study, former participants had significantly greater em ploym ent and earnings, more education, better health, and less serious criminal records than the com parison group. O ne way to further a s s e s s the m agnitude of this training program impact is to com pare the present dollar value of estim ated benefits with the costs incurred to produce th e s e results. The ratio of benefits to costs is an indicator of the rate of return on the investm ent in hum an resources. The Jo b Corps is one of the program s that h as been subjected to careful benefit-cost analysis that provides a framework for the a sse ssm e n t of other training activities. To begin with, benefits and costs can be m easured from a social perspective-w hich includes the gains and losses for participants a s well a s nonparticipants. From the social perspective, costs include all operating expenses, plus the output that is foregone during the period the enrollee is in training rather than available for work. The benefits include in-program and increased post­ 46 program output; any administrative cost savings; and reductions in criminal justice, corrections, and victimization costs to th e extent crime is reduced a s a result of participation. Taggart (1981, p. 61) found that 1977 participants’ individual costs w ere $5,070, or $2,271 less than the estim ated $7,343 value of benefits. "The effects from this m agnitude w ere far from the results that had been anticipated when the program began" (Murray, 1984, p. 38). The ratio of benefits to costs w as 1.45. From the taxpayers’ perspective, the benefit-cost ratio for Job Corps com es closer to a break-even ratio, or .96. This ratio is computed differently b e c a u se costs include all program operating and administrative expenses, plus allow ances and other expenditures. T he m agnitude of other benefits from classroom training is speculative b e c a u se there are no control-group studies to m ake the necessary careful estim ates of im pacts on criminal activity, reduced drug and alcohol use, and reliance on other transfer and training program s. D espite this evidence, the National Advisory Council on Economic Opportunity, which w as established by the Economic Opportunity Act of 1964, in its thirteenth report (1981), "warned that it w as a myth that poverty had been abolished within th e past ten years" (Auletta, 1982, p. 255) a s som e social scientists and politicians associated with the program had claimed. In sum m ary, research on the Jo b Corps program outcom es h a s varied. Levitan and Gallo docum ented that the screening of participants, or Boyle’s concept of defining th e clientele to be served, w as unstructured. Sork (1990) stated that 47 "conventional wisdom in adult and continuing education sug g ests that planning should be a highly participatory activity involving, at minimum, representatives of the client group" (p. 74). It ap p ears this involvement w as lacking in th ese projects. This writer investigated the role of defining and involving clientele in the AEAT program, and its relationship, if any, to program outcom es. New C areers Program (1965) In 1965, the New C areers Program w as introduced, which sought to restructure professional jobs in the public and nonprofit sectors, to train the disadvantaged to perform a s paraprofessionals, and to subsidize their on-the-job learning. New C areers trained the poor and undereducated for paraprofessional jobs and employed older rural residents at conservation tasks. According to Levitan and Gallo (1988), th e program failed b e ca u se the training required a long-term com m itm ent and b e ca u se of resistance by professionals protective of their jobs and status. H ence the program may have lacked the legitimation and support within the formal and informal power situation. Boyle (1981) and Sork (1990) stated that legitimation needs to be applied at many different tim es in the program developm ent process. For exam ple, program m ers m ay have to establish them selves with the client group, and the program itself m ay need support in order to be successful. R eactions from legitimizers range from a flat refusal to go along with program ideas to wanting to becom e the center of the activity. Boyle added that if legitimation is not obtained, successful programming will not be achieved. Colgan (1993) further 48 stated that planners m ust consider strategy making a s a political process, promoting coalitions to prom ote change. In sum m ary, the New C areers Program may have failed b e ca u se of resistance from essential stakeholders. This study of the AEAT initiative investigated the effect, if any, of legitimation and support on program outcom es. Em ergency Employment Act_M971) The Nixon Administration cam e to power with this commitment in the training and employm ent field: to consolidate and at th e sam e time decentralize the diverse program s that had em erged during the 1960s. The recession of 1970-71, coupled with the approaching presidential election, generated sufficient political pressure to induce President Nixon to sign the 1971 Em ergency Employment Act authorizing the public employm ent program. In summary, a $2.25 billion appropriation allowed state and local governments and nonprofit organizations to hire som e 150,000 unemployed persons. It is not known how clients w ere identified, what instruction occurred, or specific evaluation criteria. This research er determined the extent to which th e se program -developm ent concepts w ere incorporated into the 1992-93 AEAT initiative and their relationship, if any, to program outcom es. C om prehensive Employment and Training Act (1973) Nixon’s support of public employment may have lasted until 1973, when, amid a period of disarray in the Executive Branch, the Labor Departm ent negotiated 49 directly with C ongress to create the next major federal training initiative-the p a ssa g e of the C om prehensive Employment and Training Act (CETA), which consolidated nine earlier program s including MDTA. The CETA com prom ise called for locally m anaged but federally funded training and public job-sector job-creation program s. This permitted local governm ents broad discretion to tailor job training program s to community needs. Program services under the CET A w ere directed toward workers unem ployed for both structural and cyclical reasons, and program participants typically received incom e m aintenance stipends. The range of services provided included: classroom training, on-the-job training, and work experience (subsidized public-sector jobs em phasizing work habits and basic skill developm ent designed for individuals with essentially no prior labor m arket experience). Under CETA, annual first-time enrollments ranged betw een 1.9 million and 4.0 million individuals. At its height, CETA had an annual budget of $10.6 billion. This new em ploym ent and training program w as overwhelmed by unemploym ent, which climbed from a 5% rate at the beginning of 1974 to more than 7% by D ecem ber. By 1974, charges of careless m anagem ent and enrollment of ineligible applicants led to a change in focus of CETA funding from training toward job creation. Meanwhile, unemploym ent continued to rise until it peaked at 9% in 1975, and averaged 7.7% in the 1976 election year. Although CETA program s were not limited to training a ssistan ce orto serving displaced workers, the CETA evaluations are important for two reasons. First, they 50 provide baseline quantitative estim ates to which the effects of later program s and dem onstration projects can be com pared. Second, a discussion of th e CETA evaluations represents an opportunity to introduce som e of the main methodological issu e s involved in program evaluation. An important feature of CETA w as that, for the first time, the U.S. Departm ent of Labor (USDOL) funded the developm ent of a d a tab ase specifically designed for program evaluation. Term ed the Continuous Longitudinal M anpower Survey (CLMS), this d a ta b ase included three com ponents: 1. Data for random sam ples of CETA enrollees collected quarterly beginning in 1975. 2. Data from com parison groups drawn from March Current Population Survey (CPS) files. 3. Social security earnings records for each CETA enrollee and each m em ber of the C PS com parison groups (Leigh, 1990, p. 10). Thus, the methodological approach to program evaluation permitted by CLMS data involves the u se of an externally selected comparison group. A general problem in this methodology is that differences betw een the treatm ent and com parison groups will exist b e ca u se they are not drawn from the sam e population. The two groups are not statistically equivalent. An advantage of the CETA evaluation format is that it allowed random assignm ent of program eligible workers to treatm ent and control groups. Leigh (1990) stated that this breaks the link betw een program participation and 51 unobservable determ inants of earnings, and m ay allow unbiased net program effects to be obtained. In defending the value of nonexperimental m ethods of program evaluation, however, Heckman, Hotz, and Dabos (1987) em phasized the costs and practical difficulties of conducting social experim ents and, in their view, the limited value of experim ental data. They noted, in particular, that participation in a training or retraining program entails a multistage process of application, selection, continuation in the program until completion, and job placem ent. An experimental a sse ssm e n t of th e effect of training is conditional on completing each stag e of the process and requires random assignm ent of each sta g e-so m e th in g that is rarely done in social experim ents. Hence, a c ase can be m ade that nonexperimental m ethods have a role to play in realistic plans of program evaluation. Bamow (1987) provided a survey of 11 major CETA evaluations. He essentially sum m arized the net-impact estim ates that used data for adult workers and that provided som e breakdown in the results by gender, race, and type of program service. Finifter (1987) also evaluated CETA. The estim ates m easure the im pact of CETA on first-year post-program earnings for participants enrolled in 1975 and/or 1976 net of earnings of the Current Population Study com parison group. T hree conclusions appear to be w arranted from the work of Barnow (1987) and Finifter (1987): 1. Most of the estim ates shown for wom en w ere larger than th o se for m en. This is consistent with Bloom and McLaughlin (1987), who indicated that the 52 main effect of CETA training w as to facilitate labor m arket entry. Thus, persons out of the labor m arket—primarily w om en-enjoyed a larger program impact than those with extensive but unsuccessful labor m arket experience—primarily men. If Bloom and McLaughlin’s suggestion is correct, the net impact estim ates for wom en will be upwardly biased to the extent that fem ale labor force entrants are not a random sampling of all women. 2. classroom On-the-job training (OJT) w as found to be more effective than training, particularly for minority enrollees. T hese data were substantiated by Harlan and Hecker (1984), who indicated that employm ent after participation in OJT CETA program s w as 4.0 tim es greater for white wom en, 3.0 tim es g reater for white m en, 2.4 tim es greater for black women, and 1.7 times g reater for black m en than those in classroom training. Boyle stated that identifying the m ost logical seq u en c e for learning experiences and providing the m ost appropriate m ethods and techniques, such a s OJT, is important to program su ccess. The larger impact for on-the-job training than classroom training m ay have occurred b e c a u se the m ost job-ready of enrollees are those who are likely to be selected by em ployers for OJT slots. On-the-job training m ay have a larger impact on earnings in the short run, b ecau se job retention w as usually assured for a short time after a subsidy ended. 3. The estim ates for CETA’s impact is "uncomfortably wide" (Leigh, 1990, p. 12). Studies using the sam e data to estim ate the sam e treatm ent effect arrived at different estim ates. For example, Dickinson (1987) found that CETA participants 53 had a significant negative effect on m en’s earnings and hours worked. However, the sam e experience had a positive effect on w om en’s earnings and hours worked. The basic problem, according to Leigh (1990), is that the a b se n c e of a classical experim ent in which m em bers are randomly assigned to either the treatm ent group or the control group requires Longitudinal Manpower Survey users to m ake a num ber of critical decisions. Most of th ese decisions involve (a) controlling for differences betw een m em bers of the treatm ent and com parison groups and (b) coping with the selection-bias problem. A se p ara te finding w as advanced by the General Accounting Office (1984). Their report investigated "fraud and a b u se in CETA, that often occurred b e c a u se of w e a k n e sse s in internal controls, particularly in accounting and reporting at the service delivery level" (p. 3). Boyle (1981) reminded successful program planners to "provide effective communication so that everyone with a role or responsibility clearly understands what is happening and when" (p. 56). In sum m ary, the multiplicity of such CETAfindings m akes it difficult to a s s e s s the extent to which differences in a sse ssm e n t approaches account for the wide range of net impact estim ates. This investigator determ ined the extent to which clear program evaluation w as used in the AEAT projects and its relationship, if any, to program outcom es. T rade Act of 1974 Under the Trade Act of 1974, workers w hose employm ent is adversely affected by increased imports may apply for Trade Adjustment A ssistance (TAA). 54 This a ssista n ce is available to workers who lose their jobs or w hose hours of work and w ag es are reduced a s a result of increased imports. TAA includes a variety of benefits and reem ploym ent services to help unem ployed workers prepare for and obtain suitable employment. According to the Michigan Employment Security Commission, workers m ay be eligible for training, a job search allowance, a relocation allowance, and other reem ploym ent services. Additionally, weekly trade readjustm ent allow ances (TRA) may be payable to eligible workers following their exhaustion of unem ploym ent benefits. Usually, benefits will be paid only if an individual is enrolled in an approved training program. Approved training m ay include on-the-job training, vocational or technical training, and rem edial education. Program participants m ay receive training benefits while in training, provided they continue through all training activities. To qualify for TRA, the participant must: 1. Be covered by a certificate. 2. Be totally separated from employment. 3. Have worked at least 26 w eeks at w ages of $30 or m ore a week in adversely affected employment. 4. Have been entitled to and exhausted all rights to unem ploym ent benefits. 5. Be enrolled in, or have com pleted, an approved training program. This program is administered by the Employment and Training Administration of the U.S. D epartm ent of Labor. No details w ere found regarding the planning 55 activities of this program. This writer investigated another governm ent-funded training program, the AETA program. Job Training Partnership Act (1982) The CETA program expired in 1982 (along with 300,000 CETA-funded jobs) with the national econom y mired in one of th e d e e p e st recessions since the 1930s (Leigh, 1990). Rather than renew CETA program s, with their political unpopularity, President R eagan and the C ongress developed the Job Training Partnership Act (JTPA) to train and place workers in private-sector jobs. No new youth- unem ploym ent initiatives w ere begun by the R eagan Administration. "Government w as getting out of the way, consigning disadvantaged youths to the m ercies of a private job market that in years past either feared or deem ed them superfluous" (Auletta, 1982, p. 245). With the jobless rate approaching 9%, President R eagan’s budget, introduced in 1982, recom m ended a $2.4 billion budget for job training, or a fourth of the am ount appropriated before his election. This rising unem ploym ent and increasing plant closures led the U.S. D epartm ent of Labor to begin funding a series of dem onstration projects intended to test the effectiveness of alternative reem ploym ent services in placing displaced workers in private-sector jobs. D espite the general em phasis on reduced spending, the addition of a new program for retraining dislocated workers w as not controversial. The problem of dislocated workers w as viewed a s increasingly acute during the early 1980s b e cau se of increased foreign econom ic competition, the continued relative decline 56 in m anufacturing employment, and the deepening recession (Levitan & Gallo, 1988, p. 13). Although m any dislocated workers had previously p o sse sse d good jobs, the difficulty they experienced in regaining em ploym ent w as thought to justify federal intervention. Relative to CETA, this new legislation gives increased responsibility to state governm ents for planning and implementing displaced worker program s (National Alliance of B usiness, 1982). More still, it defines a m ore active role for the business community in program developm ent through the establishm ent of Private Industry Councils (PICs). JTPA also differs from CETA in its concentration of resources on training, and its requirem ent that numerical perform ance standards be used in a sse ssin g local program su ccess, such a s job placem ent rates, participants’ earnings, and training costs (Leigh, 1990, p. 17). JTPA instituted m andatory national targets and established m onetary aw ards for successful program s and sanctions against localities that performed poorly. C ongress supported increased coordination betw een job training and related social program s, such a s employm ent and welfare program s. The law vested principal responsibility for coordination with the governor’s office and allocated funds directly to governors for coordination activities under JTPA ’s principal training program. Two other significant administrative provisions designed to avoid problems that had plagued CETA included the authorization of JTPA a s a perm anent program and eliminated quadrennial reauthorization debates. Second, to provide localities 57 with a d eq u a te lead time to plan the coming y ear's expenditures, JTPA ’s operating year w as scheduled to begin in the July following the start of the federal governm ent’s fiscal year in October. As its title connotes, the Job Training Partnership Act is designed to create a working partnership am ong the three levels of governm ent and the private sector, yet it retains federal responsibility for financing, monitoring state and local com pliance with the law, supplying technical assistance, a sse ssin g the program, and ensuring fiscal accountability. Virtually all observers of the JTPA a g ree that the Labor D epartm ent abjured leadership of the program (Levitan & Gallo, 1988). Once again, Boyle (1981) stated that legitimation and support from the formal and informal power structure is a key elem ent to program su ccess. The D epartm ent of Labor’s "hands off' policy d o e s not ap p ea r consistent with garnering support from the power structure. Misguided personnel actions com pounded the departm ent’s policy of distancing itself from the administration of JTPA. For example, JTPA had only 1,700 em ployees, 300 below the level authorized by Congress. Lost expertise left JTPA in a poor m anagem ent position. R epeated reorganizations resulted in more than 200 dem otions, affecting m orale and efficiency. According to the U.S. G eneral Accounting Office (1985, HRD 85-61), about 80% of the JTPA staff "had no prior training or experience for their jobs" (p. 3). According to Boyle, the program -developm ent process m ust provide for the "legitimation and supportive actions that will facilitate the organization’s efforts to obtain continuity and resources to support the program" (p. 50). 58 Levitan and Gallo (1988) also reported that "federal assistance, data collection and research and monitoring of sta te s and localities rem ained inadequate" (p. 19). Little evidence se em s to exist that supports Pennington and G reen’s (1976) or Boyle’s (1981) framework of evaluation of the program, in term s of determining what the learners actually achieved. According to Simms (1993), "unless public sector trainers are committed to evaluating the effectiveness of their program s, accountability, efficiency and credibility will not be improved" (p. 593). The Labor Departm ent is required to submit to C ongress an annual a sse ssm e n t of JTPA that incorporates research and evaluation findings. Until 1987, th e departm ent ignored this statutory requirem ent, and there is no record that C ongress ever prompted the departm ent to fulfill its responsibility. B ecause of inadequate funding a s well a s inefficient allocation of the available research money, major gaps exist in our knowledge of JTPA outcom es. Two major field studies exam ined JTPA, but neither exam ined the role and activities of subcontractors who provide the training, or th e individuals who receive it. Simms (1993) reported that "failure to work out the details of the program results in pitfalls and error" (p. 595). B ecause the administrative agencies infrequently provide services directly to enrollees, the failure to exam ine service providers is a deficiency in JTPA overall asse ssm e n t. Consequently, little is known about the providers of training, their quality, the criteria used to accep t or reject applicants, and the factors responsible for su c c e ss or failure (Levitan & Gallo, 1988, p. 29). With such knowledge lacking, it is difficult to g auge the JTPA ’s su c c e ss or improve the 59 program. This writer attem pted to answ er questions about training providers and determ ine the relationship, if any, that it m ay have to program outcom es. A review of the JTPA-related literature indicated that the following inferences can be m ade about the JTPA outcom es: 1. That little change occurred in th e area of coordinating activities from CETA to JTPA. Specifically, a num ber of important issu es w ere raised, such a s "a decline in service and ab sen ce of progress in implementing customized training courses" (Bailis, 1987). 2. Jo b search assistance services have the intended effects on a variety of labor market outcom es. T hese include earnings, placem ent, and employment rates. Given the relatively low costs per worker, the evidence su g g ests that Job Search a ssista n ce services are cost effective (Leigh, 1990). 3. Evidence gathered for JT P A has indicated that classroom training fails to have a sizable incremental effect on earnings and employment. It does not ap p ear to be the c a se that the additional effect of classroom training is large enough to com pensate for the higher cost of classroom training services. Authors of the major evaluation cited by Leigh, such a s Corson et al. (1989), Bloom and Kulick (1986), and Corson, Maynard and Wichita (I984), offered a num ber of caveats for their findings, including the difficulty of drawing reliable inferences from small sam ple sizes, the problem that program participants undergoing skill training have relatively little time left to receive placem ent assistance, the scarcity of training providers capable of putting together high quality, short-duration training courses on short 60 notice, and the possibility that the classroom training provided is either not saleable in the local labor market or not of particular interest to either client population. 4. On-the-job training has not been found to consistently have a positive effect on em ploym ent rates (Committee on Labor and Human R esources, 1984). 5. JTPA participation had a small positive impact on w ages for reem ployed claim ants. There is no evidence that program services perm anently increased labor productivity (Leigh, 1990). 6. Classroom training curricula may not match th e backgrounds and perceived n e ed s of client workers (Bailis, 1984). 7. Skill training program s administered to JTPA trainees have not been found to have a significant incremental effect in improving reem ploym ent prospects (Butler, 1988). In summary, Levitan and Gallo (1988) concluded that, in the a b sen c e of careful oversight, JTPA contractors "cut corners ontraining quality to increase profits or in resp o n se to federal or local p ressu res to reduce costs" (p. 174). The quality of remedial education and occupational training can be improved by providing localities with funds to hire better quality instructors, purchase n ecessary equipm ent and operate program s of sufficient length. Unless enrollees acquire skills that are valued in the m arketplace, JTPA is unlikely to achieve more than fleeting gains in enhancing participants’ employability. Similarly, Reich (1983), in The Next American Frontier. argued that this training initiative "may have been m ore successful if it had been strategically planned--that included preparation for jobs in the private sector" (p. 61 209). Reich stated that training "had been disconnected from the process of industrial change in America" (p. 210). This underscores Boyle’s (1981) and Boone’s (1985) observation that obtaining resources n ecessary to support program s is an important consideration in program developm ent. Similarly, a s "the selection and organization of learning experiences" are identified, it should be determ ined that the planned learning opportunities are related to program outcom e, such a s m eeting employm ent stan d ard s (Boyle, 1981, p. 56). This research er exam ined th e se concepts, identified the extent to which they were included in program planning, and determ ined what relationship, if any, they had to program outcom es. Econom ic and Worker A ssistance Act (1988) This act am ended JTPA and sharply increased the level of federal funds to be u sed by the state in establishing program s to m eet the adjustm ent a ssista n ce n e ed s of displaced workers. The increased level of funding w as used to "support services such a s outreach and orientation, job and career counseling, testing and a s s e s s m e n t of labor market information, jobs clubs, job developm ent, child care and commuting assistance" (Leigh, 1990, p. 4). The act also specified that funds w ere not to be spent on public service em ploym ent (PSE) program s, but that needs-related paym ents m ay be provided to an eligible displaced worker who "does not qualify or h as c e a se d to qualify for unem ploym ent com pensation in order that he or sh e may participate in training or education programs" (Leigh, 1990, p. 4). 62 Little evidence se em s to exist about the overall evaluation of this program, or the inclusion of Boyle’s (1981), Sork and Buskey’s (1986), and Mitchell’s (1993) concepts of successful program planning in its design. This study, however, investigated th e se roles and their relationships, if any, to program outcom es for the AEAT projects. State Retraining Program s Before reviewing the details of particular state program s, it is useful to draw attention to two important features that distinguish state-funded initiatives in general from th o se provided by state governm ent. First, w hereas only unem ployed workers are currently eligible for JTPA services, state program s typically are offered in addition to employed workers at the risk of being perm anently laid off if their skills are not upgraded. S econd, m any states have a d d re ssed the critical issue of what to retrain displaced workers to do by tailoring training program s to m eet the n e ed s of individual em ployers. This m eans that state program s have the econom ic developm ent objective of creating new jobs, in addition to the tradiiional view of retraining a s a hum an capital investm ent intended to raise the level of w orkers’ skills to enable them to qualify for existing jobs. Federal program s, in contrast, generally are designed to qualify program graduates for jobs in what are anticipated to be high-dem and occupations. In the c a se of the federally funded JTPA, for exam ple, program planners m ade "the explicit decision not to tailor training program s to m eet firmspecific labor d em an d s” (Leigh, 1990, p. 51). 63 T h ese two c h an g es from federally funded training or retraining program s enable program planners to define target clientele, and their levels of intellectual and social developm ent. This will help planners identify specific content a re a s to be taught, identify more customized instructional approaches, and evaluate the program b ased on what participants have achieved, and how the results can be applied in future program m ing (Boyle, 1981, pp. 54-56). Quik Start (1981-1991) Through the Quik Start program, the Michigan Departm ent of Education (MDE) h as been working to advance econom ic developm ent by providing custom ized training in a variety of industries throughout the state (Quik Start is Working. 1989, p. 3). According to the MDE, the effort has been possible through the developm ent of partnerships am ong business, industry, education, labor, governm ent, and other community organizations. T h ese specific agencies include the Private Industry Councils, the Michigan Employment Security Commission, Michigan Rehabilitation Services, econom ic developm ent agencies, labor organizations, and the Departm ents of Com m erce and Labor. Quik Start h a s been adm inistered by the MDE Vocational and Technical Services. This departm ent funds secondary and postsecondary institutions to custom design training program s that m eet the job-specific n e ed s of business and industry to train, retrain, or upgrade workers. Quik Start funds are used by the educational agency to "design, develop, and operate training program s which ultimately prom ote local, regional, and sta te econom ic development" (Quik Start Is 64 Working, p. 3). Only econom ic-based com panies expanding or upgrading their workforce participated in this program. In 1987, after six years of operation, the Quik Start program w as evaluated by the Instructional Developm ent Evaluation A ssociates, Inc. (IDEA) to evaluate the su c c e ss of the program. Information w as sought to determ ine the following: 1. Program effectiveness in term s of com pany satisfaction with training- related services and programs; 2. A ssessm ent of the influence Quik Start funding had on com pany decisions to locate, expand, or stay in Michigan; 3. Identification of the overall benefits of the program to th e com panies served. The results of that study include: 1. Seventy-nine percent of the participating em ployers believed that the training m aterials and presentations w ere highly relevant (Quik S tart, p 4). 2. Eighty-three percent of the em ployers believed that the training "very much" m atched their expectations. 3. Seventy-nine percent of the em ployers stated that available funding had influenced their decision to expand in the state. 4. Ninety percent of the em ployers said that the program increased productivity, increased efficiency, and improved the quality of their products. 5. Seventy-five percent of the employers surveyed had promoted workers who took part in the training. 65 According to the Quik Start Final Report, over the period from July 1981 to S eptem ber 30,1991,621 grants w ere aw arded to 84 educational institutions serving 443 com panies. Funding in the am ount of $13.6 million w as used to train or retrain m ore than 44,182 em ployees. The a v erag e cost of training per trainee w as $308. Of the 44,182 people who w ere trained, nearly a third w ere current workers who needed upgraded skills to perform their jobs. According to the 1990 Quik Start Jo b Training Program statem ent (August 1990), "it ap p ears that once a particular institution has applied for and received a Quik Start grant, the process becom es familiar, less intimidating, and many institutions return for more funding" (p. 2). A further benefit of involvement in Quik Start projects "seem ed to be the enhancem ent of institutional expertise" (p. 2). The MDE Executive Sum m ary (1986) indicated that th e benefits of the Quik Start Training program to the company, a s perceived by the em ployers, included increased production rates, increased quality of work performed, increased production quality, and enhanced com petitiveness. B ecause th e se findings w ere based on em ployers’ perceptions, no quantifiable data w ere used to determ ine the extent to which the outcom es noted above w ere increased or enhanced. The MDE Executive Sum m ary added that "the institutions' ability to provide custom ized training which m eets the n e e d s of the em ployee and com pany w as one factor which contributed to overall high ratings for the quality of training" (p. 5). O ther factors included highly relevant m aterials and presentations, highly effective instructors, especially in their ability to com m unicate with the adult learner, and an 66 instructor who provided sufficient communication and w as receptive to com pany needs. The Vocational-Technical Education Services D epartm ent adm inistered a telephone survey in 1986 to program participants. R esults of the survey indicated that th e "strengths of the program w ere characterized by good programming, bureaucratic efficiency, cooperation and commitment" (p. 2). W eak n esses of the program that acted a s hindering factors could not be identified by ag en ts of m ore than half of the projects. Lack of cooperation and guideline limitations w ere m ost frequently cited a s hindering factors. It w as further recom m ended that (a) a need for a ssista n ce be offered to educational institutions in establishing a fram ew orkforcustom ized training program s a s well a s coordinating program s statew ide, (b) institutions be encouraged to advertise services available to com panies, (c) guidelines be established for instructors and individuals be encouraged to pursue skilled trad es training a s a career, and (d) m anagem ent/supervisor training be stre sse d for at least two years. In sum m ary, the MDE’s final Quik Start Report suggested that this program has "spread financial resources through local communities to create and upgrade individual positions in com panies, thereby maintaining employm ent in Michigan and attracting/expanding business in this State" (Quik Start Is Working. 1985, p. 5). This study focused on outcom es associated with the Michigan AEAT program (1992-93), and determ ined the extent to which various program planning concepts, a s identified 67 by Boyle (1981), Sork and Buskey (1986), and others w ere perceived to have been included in the program design. Jo b Training/Retraining Investment Fund (1983-841 Public Act 263 of 1982 introduced a new concept in the funding of Michigan’s 29 community colleges with JTRIF. This investm ent in the community colleges w as predicated on the belief that th e se institutions would be significant contributors to the econom ic rehabilitation and developm ent of Michigan. In D ecem ber 1983, each of the 29 community colleges prepared a proposal outlining its intention to participate in th e Investm ent Fund program. The colleges qualified for participation in this $3.28 million program. This appropriation b ecam e an investm ent by the S tate of Michigan a s it encouraged significant additional funding from both internal college sources and external sources a tth e federal levels. The Investm ent Fund allowed the expansion of college staff while providing needed hum an services and econom ic developm ent to their people and comm unities (Investm ent Fund Projects: Impact Statem ent. 1984, p. 7). The colleges reported that more than 31,000 students received direct benefits from the activities initiated by the Investment Fund program. In addition, the Impact S tatem ent portrayed colleges a s "becoming an important resource in the econom ic developm ent of their community" (p. 12). The report continued by stating that "communication and cooperation betw een business, industry and education are crucial to maintaining a healthy economy. Community colleges have recognized this and have m ade assisting business and industry a major part of their mission" (p. 12). 68 In summary, the 1982-83 Impact S tatem ent reported that with th e initiation of JTRIF, community colleges have dem onstrated that they have th e expertise, credibility, and creativity to respond to the n e ed s of Michigan’s businesses, industries, and labor force (p. 12). Community colleges have maximized the financial impact of the Investment Fund program by using m onies to leverage other internal and external funds to the extent that the job training and retraining investm ent effort more than "doubled in one year" (p. 11). The two m ost significant funding n e ed s that have em erged in implementing this program are (a) for start-up costs of state-of-the-art equipm ent and new personnel and (b) for m aintenance and supplies that compliment high technical equipm ent. Like the JTRIF Impact S tatem en t, this study attem pted to explain the role of appropriate personnel and equipm ent in the program developm ent of the AEAT funding. This study is different from th e Impact Statem ent information p resented here in that itfocused on the AEAT program, 1992-93. Adult Education Alternative Training Program (1992) The most recent strategy at the S tate of Michigan level w as to create the Adult Education Alternative Training (AEAT) program to help institutions of secondary and post-secondary education becom e m ore successful in m eeting its mission of "economic and workforce development through lifelong learning programs and custom ized training" (Wismer, 1993, p. iii). This project m ade it possible for state policy m akers, such a s the Michigan Jo b s Commission, to earm ark special m onies for specific econom ic developm ent purposes. (Twenty-five million dollars 69 w as budgeted for this purpose in FY 1993, w hereas $40 million w as se t aside in 1994. Only California allocates m ore dollars for state-sponsored training or retraining program s). This funding also enabled public- and private-sector educators and trainers to play a m ore direct role in state econom ic developm ent while enhancing the quality of their educational program s. Simply stated, this funding initiative w as used to develop hum an capital or people potential. For the people potential needed to fuel the S tate of Michigan’s economy, there are only two sources: new entrants to the workplace and the current workforce. The Michigan School Aid Act 148 of 1992, Section 107A, authorized $25 million in funds for the new AEAT Grant Program. The Act specified that th e $25 million in training funds w ere to be allocated through a competitive application process. As specified in Subsection l(E )o fth e 1992 Act, the S tate Board of Education w as to award grants only to applicants included in the list of recom m ended grant recipients. The S tate Board aw arded grants at three se p ara te m eetings: 1. D ecem ber I992--47 grants awarded. 2. January I993--42 grants aw arded. 3. March I993--63 grants aw arded. A total of 330 applications w ere processed to an Interagency Committee (com prised of D epartm ent of Labor, C om m erce, Employment Security Commission, and Education officials). Of th o se 330 applications, 156 ultimately w ere approved by the S tate Board. It should be noted, however, that before MDE review and 70 ab sen t MDE concurrence, the Interagency Committee and the D epartm ent of M anagem ent and Budget developed and mailed to the field the grant application forms. Further, th ese grant forms, according to the MDE "Final Report" (1994, p. 3): 1. Contained language which provided that the Interagency com m ittee had authority over responsibilities delegated by the legislature to the MDE. 2. Conditioned paym ents to g ran tees upon perform ance, even though ultimate control over perform ance rested with the com panies. 3. Disallowed career guidance and placem ent costs, which deterred ag en cies from providing new job training. 4. Omitted instructions for completing evaluation procedures. Michigan D epartm ent of Education Frequently, the MDE has been ask ed to provide written approvals for program modifications. T hese modifications typically related to changes in the program design or delivery due to changes in the sponsoring em ployer’s circum stances. According to the "Final Report of Activities" (1994, p. 3), the m ost frequent requests for modification were substitution of coursework, substitution of em ployers, expansion of training, changes in the num ber of participants, and budget line item revisions. All requests for revisions w ere required to be m ade in writing and were confirmed in writing by the Michigan D epartm ent of Education. Interagency Committee. As specified in Subsection 1(D) of the Act, priority in the Interagency Committee’s recom m endations to the S tate Board for the award of grants w as b ased on the following criteria: dem onstrated need, cost per pupil, 71 strength of commitment to guaranteed job placem ent, collaboration with appropriate community and business organizations, inclusion of an evaluation plan, and other criteria a s determ ined by the Interagency Committee. In developing the grant application, the Interagency Committee failed to provide instructions for completing the narrative portions of the grant application. In addition, the reviewer’s criteria/score sh e et did not follow the content of the grant application. According to MDE officials, this created significant problem s in attem pting to provide a competitive review process. Eligible applicants. Subsection 1 (A) specified that an eligible applicant may be a school district, intermediate school district, community college, public university that aw ards asso ciate degrees, nonprofit organizations, or proprietary schools. Participant outcom es. Subsection 3 specified that not later than 90 days after completion of the program, grant recipients w ere to provide an evaluation report to the D epartm ent of Education on the educational and em ploym ent outcom es of the trainees. For reaso n s described above, the form and m anner of reporting participant educational and employm ent outcom es did not follow th e original guidelines a s proposed by MDE. “The lack of coordination am ong the agencies would ultimately limit the MDE’s ability to conduct a meaningful evaluation of the program due to the many exceptions to reporting procedures that w ere m ade n ecessary by the contradictions, om issions, lack of clarity and specific instructions in the application package" (Final Report, 1994, p. 6). In defining evaluation procedures (and again without MDE 72 concurrence), the Interagency Committee required that follow-up forms had to be com pleted and signed by each of the 36,696 proposed trainees. T hese forms were issued without definition of instructions, and resulted in unreliable data on the participants, and no quantifiable data on the achievem ent of em ployer objectives for training. The MDE also requested that g ran tees provide a final narrative report of program outcom es. Paym ent m echanism s. According to Section 5 of th e Act, the MDE w as to m ake th ree paym ents to the g ran te es a s follows: PAYMENT PAYMENT TIMING PAYMENT AMOUNT First Within 30 days of approval 25% Second At training midpoint 25% Third Conclusion of grant period 50% Extension of training period. Provisions w ere m ade for grant recipients to continue training activities and carry over Section 107A funds beyond the fiscal year, which ended Septem ber 30, 1993. To qualify for the extension, grantees w ere requested to com plete a "request for extension of training" and return it to the MDE for approval. Under this agreem ent, 78 projects were approved for extension. The following list sum m arizes the reaso n s for extension: 1. Late delivery of training m aterials/supplies. 2. Implementation problem s due to turnover in industry. 3. Late start in training due to late notification. 4. Installation of new equipm ent took longer than planned. 73 5. The need to continue production precluded som e training times; 6. Extensive d em an d s on em ployees cau sed delays and interruptions. 7. C hange in training m anagers delayed implementation. Audit responsibility. Subsection 7 of the Act specified that a recipient of a grant shall "allow a c c e s s for the Departm ent to audit all records related to the grant for all entities that receive money" (Final Report, 1994, p. 10). Grant recipients w ere to reim burse the state for all disallow ances found in an audit. B ased on th e reviews conducted, the actual $25 million allocation w as significantly less in the total aw ards actually m ade. This sum m arizes the information m ade available through the MDE’s final report on the Adult Education Alternative Training program. Sum m ary In C hapter II. the literature related to establishing a philosophical framework for program planning w as reviewed. The literature related to state and federally funded training/retraining program s also w as reviewed. It show ed that previously funded job training/retraining d o es not exist in a vacuum, but may be influenced by the sophistication and com pleteness ofthe program plan. The more recent literature called for an expanded definition of planners' roles so that they negotiate in an ethically sensitive, politically astute, and technically sound m anner. CHAPTER ill METHODS AND PROCEDURES Introduction In C hapter II, the literature relative to successful program planning m odels and previous governm ent funded training program s w as reviewed. The literature included topics relating to federal and state-supported activities. It suggested that a s s e s s m e n t of outcom es in those program s w as not always clear and that the implementation of planning com ponents w as not carefully studied. The purpose of this study w as to determ ine w hether and the extent to which Michigan Education Alternative Training planners used selected concepts of program planning to achieve specific objectives. A second purpose w a sto determ ine w hether the implementation of a formalized planning process supported or enhanced program planners’ satisfaction with outcom es. The m ethodology for the study is described in this chapter. This includes the research questions, study design, selection of the population and sam ple, data so u rces and methodology, the u se of ethnography, and the survey analysis that w as used. The concept of training/retraining program planning and outcom es a s s e s s m e n t h as received w idespread recognition a s beneficial, but the practice of 74 75 system atic program developm ent and its a sse ssm e n t has lagged behind (Bell & Kerr, 1987). Few reports o fth e relationship, if any, betw een system atic program planning and program outcom es have been published; com pared to the num ber of program s, few a sse ssm e n ts have been m ade. Lack of formalized training program planning and outcom e a sse ssm e n ts is even m ore evident in the public sector, and is possibly the least developed aspect o fth e training process in public agencies. Yet the training process is not com plete until and unless effective program planning and a sse ss m e n t h a s taken place, for it is th e se p ro ce sses that inform th e training practitioner and give it m eaning. R egardless o fth e need, the m ethod, or the purpose, the program planner must carry out a system atic identification and organization of important factors relative to planning and outcom es. To dem onstrate training or retraining’s importance, program planners and trainers m ust prove that their program s get results, improved job perform ance, m ore efficient u se of resources, and satisfactory returns on the dollars invested. A review o fth e literature indicated that planners and trainers increase the value of training when they system atically plan a program that incorporates selected concepts of program planning and implement program a sse ss m e n ts of outcom es. Failure to system atically plan and a s s e s s training leaves open a potential for training/retraining without accountability. R esearch Q uestions T he following general research questions w ere ad d ressed in this study: 76 1. To w hat extent are Adult Education Alternative Training planners satisfied with the outcom es of specific 1992-93 program s? 2. To w hat extent do Adult Education Alternative Training program planners u se planning concepts a s outlined by Boyle, Mitchell, Buskey, Cervero and Wilson, and others? Are som e program planning concepts used more frequently than others? If so, w hat are they? 3. Do program planners consider som e program planning concepts to be m ore im portant than others? If so, what are they? 4. Do program planners consider som e program planning concepts to be more effective than others? If so, what are they? 5. Do associations exist betw een using planning concepts and planners’ satisfaction with completing stated objectives and actual placem ent outcom es? 6. W hat other mitigating variables may influence the outcom es of the various training activities? W hereas one of the broad research questions focused on the planning concepts of the AEAT initiatives, another significant question concerned the relationship betw een program planning and the planner’s satisfaction with completing ex p ressed program objectives and program outcom es. Description of R esearch Methods This study com bined qualitative and quantitative data. It w as designed to provide a b a se for understanding the perceived use, importance, and effectiveness of program planning com ponents. Data w ere gathered and evaluated through the: 77 1. Inspection o fth e records, planning docum ents, and final reports of approxim ately 75 Adult Education Alternative Training (AEAT) planners/deliverers m aintained by th e Michigan Departm ent of Education. 2. D evelopm ent of a survey instrument based on selected concepts of successful program planning. 3. Distribution of survey instrum ents to program planners in the AEAT activities and collection of the com pleted instruments. 4. Personal interviews of 10 program planners. 5. Tw ofocus group m eetings of AEAT planners, using a failure-mode and effects-analysis technique, to identify and prioritize problem s associated with the AEAT program. 6. Analysis of data. 7. Synthesis and implications. 8. Conclusions. This investigation started in March 1993, v/hen the literature review began and a preliminary research proposal w as developed. In the spring and early sum m er of 1995, the selected AEAT program planners w ere surveyed and asked their opinions on several statem ents taken from the literature on successful program planning concepts and outcom es. This information w as compiled and served a s a basis for th e developm ent o fth e interview instrument (see Appendix G), which w as adm inistered to 10 program planners in July 1995. Two focus groups of th ese program planners w ere then conducted by the Michigan Jo b s Team . The focus 78 group utilized a failure m ode and effects analysis technique that further refined and prioritized mitigating variables or factors that influenced specific program outcom es. R esearch Design Review of the literature on program planning m odels and state or federally funded training/retraining program s provided a framework to this study and helped give direction to it. An inquiry w as m ade of the Michigan D epartm ent of Education, Adult Extended Learning Services, in Spring 1994. R epresentatives of this departm ent w ere ask ed to identify training agencies that participated in the 1992-93 AEAT program s and com pleted the required final report. That information, with institutional contacts, w as provided to the research er so that questionnaires could be distributed and interviews could be conducted. The research population for this study consisted of the 71 program planners from community colleges, intermediate school districts, K-12 school districts, and em ploym ent consortium s that com pleted the 1992-93 AEAT training and retraining program s. Only those agencies that filed a final report with the Michigan Departm ent of Education were provided a questionnaire. The information w as analyzed in -aggregate resp o n ses and by individual agencies to provide the respective training agency a m easurem ent of program effectiveness. 79 Sam ple Selection Ten randomly selected interviews (five from the group of individuals returning a survey, five from the group who did not return a survey) w ere conducted with the AEAP planners, after the survey w as com pleted. T hese interviews focused on patterns that had em erged from the surveys. Instrument Development The purpose of the questionnaire w as to a s s e s s the program planners’ perceptions about the planning events that happened during a particular AEAT activity (se e Appendix D). It served a s a vehicle to explore differences in attitude toward the use, importance, and effectiveness of program planning com ponents and the relationship, if any, they had to meeting the objectives o fth e program. Developing the questionnaire w as a six-step process: 1. A review of the literature for appropriate them es, concerns and phrases. 2. A first draft organized around a career planning a sse ssm e n t model developed by C as Heilman (1992). 3. A review ofthe draft by a group of instructors, administrators, and state officials (D epartm ent of Education and the Michigan Jo b s Commission) who had participated in the AEAT activity. 4. A second draft tested with doctoral candidates in Education. 5. A third draft reviewed by the dissertation committee. 6. A fourth draft submitted to receive study approval. 80 Survey questions w ere clustered into one of eight concept a re a s (planning analysis, planning justification, objectives, administration, instruction, marketing, evaluation, and program delivery). T hese a re a s and related questions w ere developed through: 1. A review of program planning literature. 2. Recom m endations from the doctoral com m ittee and D epartm ent of Education officials. The m echanics of administering the survey were planned with the main institutional contact (of each institution completing an AEAT program), for one specific AEAT project, a s noted in the final reports filed with the Michigan D epartm ent of Education. Surveys w ere mailed to that contact at the sam e time. Ethnography. Visits, and Interviews After processing the surveys, visits and interviews w ere scheduled with ten program planners. The interview included a structured format, with som e openended questions (se e Appendix G). Q uestions included two types of open-ended questions. O ne se t w as descriptive o fth e events that took place. The other w as causal or explanatory questions. T hese interviews of approximately 30 minutes explored topics that related to the planners’ satisfaction with the program ’s outcom es and the conditions that took place (or did not take place) that might have contributed to the completion of the stated objectives. Gordon (1980) suggested that follow-up interviews such a s th e se are important in learning people’s beliefs, attitudes, values, knowledge, and any other 81 subjective orientation or mental content. This field research helped to expand the understanding that others had about the planning p ro cess and its outcom es. Firestone and Dawson (cited in Fetterman, 1988) su g g ested that subjective understanding can be fully used a s a source of data, a s a m ean s to generate new hypotheses, and a s a way of helping the reader develop a fuller appreciation ofth e phenom enon of interest. Pugh (1988) reiterated that an ethnographic approach can give a critical perspective that em pow ers others. The use of interviews and the focus group to gather data in "the subjects’ own words helped the researcher develop insights on how subjects interpret som e piece of the world” w as further supported by Bogdan and Biklen (1982, p. 135). Program Planners Focus Groups In July 1995, two focus group meetings for planning and improving the AEAT pro cess w ere conducted by the Michigan Jo b s Commission and the Michigan Customized Training Association. The purpose of the focus group, consisting of 50 program planners representing 21 training institutions, w as to identify and prioritize mitigating variables or factors that influenced the outcom es o fth e 1992-95 AEAT activities. The process w as developed on a fact-based problem-solving technique, using a failure mode and effects analysis. This specific technique, som etim es referred to a s Team Oriented Problem Solving (TOPS), w as developed by the Ford Motor Com pany to identify problems and suggest solutions. The entire process contained th e se specific components: 82 1. Use of a team approach (AEAT program planners). 2. Identification of problems associated with successfully completing AEAT program s. 3. Division of specific concerns associated with completing AEAT program s into an affinity chart that clustered problem s into related areas. 4. The transfer of the affinity charts into a fishbone diagram (see Appendix H). 5. Prioritization of individual clusters of concerns, or mitigating variables. 6. Listing of all concerns related to one cluster area. 7. Discussion of possible solutions to address priority concerns. Data Collection A variety of research strategies w as used to gather d ata-questionnaires, interviews, focus groups, and an examination and analysis of docum ents. This u se of multiple m ethods applied to the sam e phenom ena of program planning w as n ecessary for a num ber of reasons. The various m ethods produced different information that supplem ented each other. However, in this investigation, program planner questionnaires, interviews and a focus group w ere the primary strategies used. The instrum ents provided information on which selected program planning concepts w ere perceived to be used, important, and effective in the AEAT projects. The techniques also provided specific information on mitigating variables or factors that influenced th e outcom es of the activities. 83 D ocum ents w ere also exam ined and analyzed. T hese included AEAT final reports, compiled data on AEAT program s, docum ents relating to the program, and related publications. Data Analysis Preliminary analysis began a s soon a s this study w as undertaken. The analysis of data w as an ongoing process of reviewing the related literature and existing data to discover patterns and linkages. To be able to describe successful or unsuccessful training and retraining program s, patterns in the data m ust be discovered b e ca u se "carefully designed program s and evaluations are keys to better agency training" (Sims, 1993, p. 612). Discovering patterns and linkages w as achieved by the process of coding the questions and the data. Fieldnotes, questionnaire answ ers, interview resp o n se s and focus group data w ere read and reread to determ ine words and p h rases that represented linkages and patterns. Certain coding characteristics w ere su g g ested by the research questions. Departm ent of Education and Michigan Jo b s Commission officials also aided in the p ro cess of analyzing th ese data and reporting the findings. This study focused on a sam ple of all agencies who participated in the 199293 AEAT projects, and who finished and filed their final program report. The purpose o fth e survey, interview, and focus groups w as to add depth and detail to the description of planning in the AEAT initiatives. Most ofth e data are descriptions or attem pts to show correlations. 84 Using a 486 IBM computer, the researcher entered the data into a d a tab ase sp read sh eet. S P S S w as used for statistical analysis. Descriptive data for th e background information entered in the AEAT final reports w ere printed in total and disaggregated by institution type. Descriptive data ofthe kinds of program planning com ponents used w ere printed in tables in total and disaggregated by institution type. Basic descriptive statistics including m ean, median, mode, variance, and standard deviation w ere tabulated on several categories of questions. R esp o n ses were printed by institution and by overall results. The groups of questions included the following: Planning Analysis (1.1-1.5), Planning Justification (2.1-2.5), Objectives (3.1-3.4), Administration (4.1-4.7), Instruction (5.1-5.5), Marketing (6.16.8), Evaluation (7.1-7.7), and Program Delivery (8.1-8.9). R esp o n ses for th ese questions used a five-point Likert-type scale ranging from "Not Important" to "Essential" and "Did Not Use" to "Very Effectively Used." Relationships suggested by the data w ere explored through the t-test, using the .05 level of significance. Results are reported in C hapter IV. Strengths and W eaknesses of the Methodology In sum m ary, th e strength of the methodology is that judgm ents about the influence and legitimacy of the AEAT w ere gathered from program planners in a cost-effective, timely way. Every planning and training participant had the opportunity to com m ent on every qualitative argum ent or a sse ssm e n t. Thus, it seem ed m uch easier to determ ine the uncertainties that several responsible persons 85 had about the issu es under study than a single survey would provide. The primary and overriding objective of the research should be to collect data that serve a s a valid basis for improving the AEAT training system and maintaining quality control over its program-planning com ponents. The w eakness o fth e interview/survey/focus group m ethod, a s reported by Linetones and Turoff (1975), is that it lacks a completely sound theoretical basis. The questionnaire and interview experience derive alm ost wholly either from studies carried out without proper experim ental controls or from controlled experim ents in which participants are used a s surrogate experts. Other limitations associated with using th ese m ethods include problem s of questionnaire construction and the difficulty of getting each round com pleted without delay. Finally, care had to be taken when interpreting the results b e cau se no way had been found to sort out the effects of exogenous influences on the final results. For exam ple, did shifts in opinions result from the participants’ deliberations, or did the participants reread the questions and understand them better? To be effective, this research strategy must be carefully planned. Failure to work out the details ofthe research technique, or failure to include appropriate datacollection instruments, can limit the su c c e ss o fth e study. It is critical that specific procedures be followed throughout th e investigation. Sum m ary The study design w as described in this chapter. The m ethodology combined a review of docum ents, survey, interview, and focus group techniques. This chapter 86 described the nature of the docum entation used, the construction of the survey instrument, the justification of interviews, and the developm ent o fth e focus groups. It included the nature of the study’s questions and the portrayal of results in statistical and anecdotal treatm ents. The findings from the surveys, interviews, and focus group m eetings are presented in C hapter IV. CHAPTER IV ANALYSIS OF THE DATA Introduction In C hapter III the methodology for th e study w as reviewed. Program planning concepts w ere identified as th e unit of analysis. The process for choosing both the population and the sam ple w as described. The methodology ofthe study contained three main com ponents: surveys, interviews, and focus group meetings. C hapter IV is divided into four parts. The first part presents a sum m ary of planner satisfaction of program outcom es. Part two describes the planning concepts and concept a re a s used and the perceived im portance and effectiveness of each concept. The third part explores the relationship betw een use of selected concepts of program planning, satisfaction with outcom es, and actual placem ent outcom es. The fourth part reviews information about the practical and political considerations, or mitigating variables, affecting the AEAT program. The purpose of this study w as to determ ine w hether and the extent to which Michigan AEAT program planners used selected concepts of program planning to achieve specific objectives. A second purpose w as to determ ine w hether the implementation of a planning process supported or enhanced the program planners’ 87 88 satisfaction with outcom es and/or actual outcom es (placem ent). The following research questions are answ ered in this chapter: 1. To what extent are Adult Education Alternative Training planners satisfied with the outcom es of specific 1992-93 program s? 2. To what extent do Adult Education Alternative Training program planners u se planning concepts a s outlined by Boyle, Mitchell, Buskey, Cervero and Wilson, and others? Are som e program planning concepts used m ore frequently than others? If so, what are they? 3. Do program planners consider som e program planning concepts to be m ore important than others? If so, w hat are they? 4. Do program planners consider som e program planning concepts to be m ore effective than others? If so, what are they? 5. Do associations exist betw een using planning concepts and planners’ satisfaction with completing stated objectives, and actual placem ent outcom es? 6. W hat other mitigating variables may influence the outcom es of the various training activities? Three major step s w ere used to answ er th ese research questions. T hese ste p s included the u se of a 50-item questionnaire, 10 interviews, and two focus group m eetings. The survey w as sent to 71 potential respondents (program planners) by the research er through direct mailings. Forty-five program planners, or 63% of the 89 respondents, returned the survey. In addition, com m ents were submitted by the respondents (see Appendix E). Ten face-to-face interviews w ere com pleted with program planners. A tap e recording and a written transcript of each interview w ere completed. Field notes of the focus group m eetings w ere taken, and a final draft o fth e proceedings of those m eetings is included in Appendix H. 1992-93 AEAT Program O utcom es Although th ese outcom es are not part o fth e study findings, this information d o es serv e to provide a context for the scope of the 1992-93 AEAT program. According to the Michigan Departm ent of Education, the administrative and evaluative agency o fth e 1992-93 AEAT program, 1. One hundred fifty-six training or retraining grants/projects were funded, for a total budget of $24,990,000. 2. In all, 36,696 individuals w ere served (32,040 individuals were retrained for additional skill acquisition; 4,656 individuals were trained for new positions); 3. Training costs per individual averaged $681.00. The following account is a presentation of the study findings, organized around the six research questions. 90 Program P lanners’ Satisfaction With O utcom es R esearch Q uestion 1: To w hat extent are Adult Education Alternative Training providers satisfied with the outcom es of specific 1992-93 program s? This particular research question is a d d ressed in two parts. The first part provides data taken directly from the survey’s first question, "To what extent w ere you satisfied with the outcom e of this particular training activity?" (S ee Table 1.) It should be noted that th o se individuals who com pleted m ore than one project w ere directed to respond to their m ost successful program, a s identified by MDE criteria. The MDE criteria for a "successful" program are those projects accomplishing stated objectives, such a s placing participants in jobs and utilizing allocated training dollars. T h ese particular projects w ere recom m ended for study by the MDE. Table 1 show s AEAT planners’ satisfaction with outcom es, based on individual planner’s m ost successful program s. Table 1: AEAT planners’ satisfaction with program outcom es (based on their m ost successful program). (N = 45) R esp o n se No resp o n se Frequency Percent 1 2.2 Very satisfied 20 44.4 Satisfied 18 40.0 Neutral 3 6.7 Dissatisfied 2 4.4 Very dissatisfied 1 2.2 45 100.0 TOTAL 91 R e su lts: Eighty-four percent of the respondents indicated that they w ere "satisfied" or "very satisfied" with their program outcom es. The second part of R esearch Question 1 is a d d ressed by summarizing com m ents m ade during the interview p ro cess with the 10 planners and 50 focus group participants regarding planner satisfaction and dissatisfaction. q uotes from participants are listed. Selected T hese quotes w ere representative of the participants’ reactions to the questions: 1. "In general, how satisfied w ere you with the outcom es of your training program ?" 2. "Did you feel you accom plished the objectives of the program ? Which o n e s? Why or why not?" 3. "Did the training do what you said it would?" Satisfaction With-AEAT Activity The following are th e findings from the interview and the focus groups. The resp o n se s w ere clustered around th ese them es: 1. Personal satisfaction. 2. C onsistency with institutional mission/philosophy of service to community. 3. Opportunity for interaction with other program planners and professional colleagues. Personal satisfaction. Program planners w ere asked to sort through their experiences, feelings, knowledge and learning p ro cesses to m ake s e n s e out of the 92 AEAT program. In this process, they identified th ese items a s being personally satisfying in providing the training or retraining activity: It’s very gratifying to m e personally to h ear representatives from a com pany sa y they are getting better and m ore competitive b e ca u se of my efforts in making the program work. S eeing what the training h as done to change em ployees’ lives for the better is very satisfying. There is a direct and positive result b e ca u se of our efforts. My job h as changed b e ca u se of the AEAT program. Our office is doing more training than ever before. More people are getting jobs and upgrading their skills b e cau se of what we are doing. C onsistency with m ission. The following com m ents w ere m ade regarding the relationship of the AEAT program s to the training institutions’ missions: The AEAT program helps us realize and acknowledge that workforce training for local business and industry is one of our major missions. The AEAT program clearly fits into the mission of what our departm ent and institution are about. Our goal is to be thought of a s the preferred trainer for our area. The program is win, win, win for our college, the com pany trained and our community. The AEAT funding helps us to be an arm in delivering training. It’s fun to se e the com pany com e back again and again for training. The AEAT program provides training to com panies who may not otherwise have it. It has a positive impact on that company, making it m ore productive. The training helps stimulate business, upgrade the labor force, and gets team s working together. The training program gives local com panies a good kick start. The training can be focused and intense. It won’t be spread out like it may have been. Completing a program successfully m akes m oney for our institution. That puts a smile on everyone’s face. Opportunity for professional interaction. Partnerships with business and community leaders w as an elem ent of satisfaction expressed by the planners. 93 Individual com m ents also reflected that the AEAT program stimulated collegial interaction. The following com m ents w ere made: T he training program helped us develop a relationship with com panies that allowed for dialogue of problems. W e like the program b e ca u se it keeps us and our program s current with workforce needs. The training spurs curriculum development. It gets faculty into the business and industry environm ent and vitalizes their instruction. W e w ere able to share concerns about program s with other practitionersw hether we needed to find appropriate instructors or figure out state guidelines. Working with groups like the Michigan Customized Trainers helped us articulate problems, concerns and better ways of conducting program s. Developing a positive and professional relationship with the Michigan Jo b s Commission helped us to answ er training questions and eligibility requirem ents. In short, they have becom e a friend in our efforts. And they keep referring new clients to our institution. Sum m ary. The com m ents from the interview and focus groups provided a wider array of data and were consistent with the survey findings. This qualitative data related to personal satisfaction of the AEAT program reinforces individual, institutional and em ployer confidence in the AEAT activity. The com m ents also show ed that th e training w as consistent with th e community college mission for community econom ic development. The com m ents also reflected the importance of ongoing practitioner interaction with colleagues and other program stakeholders. Dissatisfaction With AEAT Activity Although 84% of those surveyed w ere satisfied with the AEAT program, the interviews and focus groups revealed som e a sp e c ts of the program that resulted in program m er dissatisfactions with and/or w ays to improve the program. T h ese 94 program planners com m ented on the AEAT activity and discussed reaso n s why dissatisfaction occurred with a particular program. Again, certain them es or issu es em erged from th e resp o n ses. The resp o n ses w ere clustered around the following areas: 1. Institutional concerns. 2. C oncerns with com pany receiving training or retraining. 3. C oncerns with local agencies. 4. C oncerns with state policies/procedures. Institutional concerns. Institutional politics w as expressed a s a concern of program planners. Suggestions m ade by the respondents in the interview and focus groups indicated that institutional support for program s w as unrealistic, such as: Our office frequently had pressure from the Dean or even the President for having program quantity, not always program quality. This pressure som etim es resulted in a scram bled effort to get a proposal submitted and funded. W e would get to the point where we couldn't do any m ore projects, unless we could hire additional staff. That w as not allowed to happen. O ther program planners described a lack of clarity of what departm ent would be responsible and/or accountable for duties related to completing the project, such as: Som etim es it w as unclear about what office w as to plan and execute a particular program. At our institution, three se p ara te offices each wrote training proposals. This put different offices and personnel in a competitive environm ent. It w as frustrating getting other offices to com plete their responsibilities. For instance, we experienced problems with our business office b e ca u se som e sta te billing procedures w ere ignored. This resulted in delays of paym ents. 95 Som e planners reported problems associated with forming collaborations with local agencies: You need to work with the local Service Delivery Area (SDA), the interm ediate and local school districts, adult education program s, and the Michigan Employment and Security Commission. Not all of th e se groups played by the sa m e rules, so it w as hard to know who the taskm aster was, and w here your loyalty should lie. Our trouble with local agencies w as the existence of two or more ag en d a s to negotiate. The sta te w ants the local players to collaborate and get along nicely, yet th o se players w ere constantly changing. It seem ed like we w ere being socially punished for past sins that w ere committed by som eone else. Com pany co n cern s. Although one elem ent of dissatisfaction w as expressed for the institution conducting th e training and the local agencies involved, m ost of the com m ents m ade w ere focused around problem s with the com pany to be trained. We sink if we don’t get complete and responsible commitment from the com pany. Unless we get that commitment for follow through, things will go w ro n g -an d then get worse. You need to deal, negotiate and side-bar agreem ents with union m em bers, nonunion m em bers, m anagem ent and corporate headquarters. Som etim es our training team ended up in the middle of m anagem ent and labor disagreem ent over training-w hat it should be, when it should be held, and who should be involved. We took a lot of hits. The political fallout w as enorm ous. Com pany officials-m anagem ent, union, and corporate headquarters--each had their own ag en d a s and were frequently pulling in different directions. We had som e com panies who said, "Get us the money, w e’ll figure out a training plan." In other instances, a few com panies saw the AEAT program a s a way to get free m oney and subsidize training program s. 96 Concerns witfa_state.policies/procedures. Concerns with state policies and procedures w ere ex p ressed in the interviews and focus groups. For exam ple, the focus group identified "meeting established criteria requirements" and "making program modifications" a s especially frustrating. Related com m ents included the following: It se e m s harder and harder to b e successful b e c a u se the state guidelines and priorities for the (AEAT) program seem to be changing monthly. I som etim es feel like the state and legislators are tying our h an d s and shackling our ankles, expecting us to do a quality job. The state really n eed s to stabilize its rules and guidelines. The am ount of paperwork and paper chasing is staggering. And th ere ’s no one else that can b e assigned to do that but me. The guidelines have changed so that it is becoming harder for the small b u sin e sse s (the mom and pop com panies) to receive this grant-funded training. This is especially true b e ca u se the state requires a significant inkind com pany match. I’m concerned with legislators trying to m icromanage our projects. After all, th ey’ve drafted th e language forthe program guidelines. They’ve allowed the program to becom e a political zing-zing activity. That is ham pering our su ccess. Making a program modification w as cum bersom e, and slowed the training program considerably. My program needed modification and it took seven m onths for it to be approved. By that time, the training w as no longer needed. Much of our planning involved countless hours of firefighting. The constant solving of problem s w as often the result of unclear directions from the state regrading program modification. O ther com m ents w ere more general, but expressed overall dissatisfaction aim ed at state agencies: 97 I'm frustrated with the entire program b e ca u se the state requirem ents are extrem ely burdensom e. W e’re thinking of getting out of the (AEAT) business altogether. T he state and its analysts don’t seem to understand the realities associated with training and retraining. They don’t understand production problem s or schedules, com pany shutdowns, or even the effects of a hunting se aso n on th e training. Maybe the state n e ed s to completely rethink the AEAT program and th e types of grants that should be funded. Som etim es when you do things for the right reasons, it doesn’t work. You alw ays have to consider the personalities and characteristics of the people involved. Program planning is a piece of cake when you com pare it to keeping the state--and everyone else--happy. Sum m ary. Again, the interview and focus group com m ents provided additional qualitative data regarding planners’ dissatisfaction with the AEAT program . T hese com m ents reflected the political and practical implications of program planning. Specific concerns w ere expressed about the lack of clear comm unication betw een state offices, the com pany to be served, community ag en cies, and the training institution. Problem s were also associated with the difficulty in meeting stated objectives, particularly when the com pany to be trained w as not committed to the program, or when the state changed program rules and guidelines. U se of Planning C oncepts R esearch Q uestion 2 : To what extent do Adult Education Alternative Training program planners u se planning concepts a s outlined by Boyle, Mitchell, Buskey, Cervero and Wilson, and others? Are som e program planning concepts used more frequently than others? If so, what are they? 98 Forty-five program planners responded to a 50-item survey regarding the u se of specific planning concepts. A planning concept provides the basis or foundation for a variety of decisions for all p h ases of the total programming effort. An exam ple of a planning concept is "establishing a planning or advisory committee." Data gathered from this survey instrument are displayed in Table 2. Table 2: Program planning concepts used. C oncepts Used 50 Frequency Percent 0 0.0 45-49 25 55.6 35-44 17 37.7 34 or fewer 3 6.7 Did not respond 0 0.0 R esults. 1. The data indicated that more than one-half or nearly 56% of th e program planners used 45 or more program planning concepts. 2. The data also indicated that approximately 93% of the program planners used 35 or m ore planning concepts. Table 3 illustrates frequency of program planning concepts used; th ese are ranked in descending order. Each concept is also identified in a concept area, or a grouping of related concepts. There are eight concept areas, including Planning 99 Analysis, Planning Justification, Objectives, Administration, Instruction, Marketing, Evaluation, and Program Delivery. Table 3: Frequency of program planning concepts used. (N = 45) No. P rogram P lanning C oncept C o n cep t A rea Freq. % 8.6 P articipants applied c o n c e p ts and skills P rogram Delivery 45 100 1.5 C onsideration w a s provided for social/educational b ac k g ro u n d s of learner P lanning Analysis 44 98 2.4 Identified program priorities b a s e d upon em ployer n e e d s Planning Justification 44 98 4.2 A dm inistrator w a s in c h a rg e of program Adm inistration 44 98 4.4 S e le c te d instructional staff with d o cu m en ted e x p e rtise in co n ten t a re a Adm inistration 44 98 4.7 D eveloped a b u d g et covering all anticipated re so u rc e co sts Adm inistration 44 98 5.1 D eveloped instructional plan Instruction 44 98 7.1 D eveloped evaluation plan Evaluation 44 98 8.7 P articipants could apply skills on real work m aterial P rogram Delivery 44 98 1.4 P roblem a re a /n e e d s stu d ied an d analyzed P lanning A nalysis 43 96 3.2 O bjectives w e re e sta b lish e d prior to program im plem entation O bjectives 43 96 4.3 D eterm ined criteria for selecting instructional staff A dm inistration 43 96 5.2 D eterm ined m eth o d s of p re sen tatio n ap p ro p riate to job perfo rm an ce objectives Instruction 43 96 5.3 S e le c te d instructional m aterials appropriate to job perform ance objectives Instruction 43 96 5 .5 Utilized m eth o d s o f interactive learning in instructional activities Instruction 43 96 100 Table 3: No. Continued. P rogram P lanning C oncept C o n cep t A rea F req. % 6.5 O riented adm inistrative an d instructional staff to all a s p e c ts of program M arketing 43 96 8.4 P articip an ts w e re provided opportunity to a sk q u e stio n s or s e e k clarification in training s e s s io n P rogram Delivery 43 96 8.5 Instructors w e re p re p are d for c la s s e s P rogram Delivery 43 96 8.8 Instructors provided for participant u n d erstanding P rogram Delivery 43 96 1.3 Identified targ e t population for program Planning A nalysis 42 93 2.3 Identified program priorities b a s e d on em ployer needs Planning Justification 42 93 4.6 Clearly defined adm inistrative roles/ responsibilities Adm inistration 42 93 6.6 Inform ed prospective participants of program M arketing 42 93 7.2 D eveloped evaluation instrum ents for effectiv en ess Evaluation 42 93 8.2 A tten d an c e w a s m onitored P rogram Delivery 42 93 3.3 O bjectives related to co m p an y ’s job O bjectives 41 91 p erform ance 2.2 S e le c te d subject a re a s related to p erform ance deficits Planning Justification 40 89 3.4 O bjectives related to in te re sts/n e e d s of tra in e e s O bjectives 40 89 6.2 C om m unicated value of program to ap p ro p riate em ployer M arketing 40 89 7.6 A nalyzed/reported results of evaluation Evaluation 40 89 2.1 Identified specific job deficits to b e a d d re s s e d Planning Justification 39 87 4.1 An adm inistrative plan w a s d ev e lo p e d Adm inistration 39 87 101 Table 3: No. Continued. P rogram P lanning C oncept C o n cep t A rea Freq. % 4 .5 A rranged ap propriate facilities an d serv ic es A dm inistration 39 87 7.3 A dm inistered evaluative instrum ents b a s e d on criteria Evaluation 39 87 7.4 O b se rv e d participants during instruction Evaluation 39 87 2.5 Identified program priorities b a s e d on train ee needs Planning Justification 38 84 3.1 S e le c te d objectives th at implied c h a n g e s in b eh av io r O bjectives 38 84 6.3 E m ployer pow er stru ctu re su p p o rted training M arketing 38 84 7 .5 E v alu ated o b jectives/m ethods constantly Evaluation 38 84 6.8 O riented participants to program M arketing 37 82 1.2 All sta k e h o ld e rs w e re re p re se n te d on th e com m ittee Planning A nalysis 36 80 5.4 D ev elo p ed pre-training activities helping le a rn e r to anticipate id eas and c o n c e p ts to be p re s e n te d Instruction 36 80 1.1 E stab lish ed planning/advisory com m ittee P lanning A nalysis 34 76 7.7 D ev elo p ed post-training and appropriate follow-up activities Evaluation 33 73 6.7 R ecruited participants for program M arketing 32 71 8.1 P ro g ram planner participated in delivery of p rogram P rogram Delivery 29 64 6.1 D ev elo p ed m arketing plan M arketing 27 60 6.4 M arketing effort w as organized and inclusive M arketing 26 58 8.3 S u p p o rt c la s s e s w ere also offered to help s tu d e n ts learn P rogram Delivery 23 51 8.9 A p lacem en t com ponent w a s im plem ented a s p art of th e training program P rogram Delivery 19 42 102 R esults. 1. All program planners used th e concept "participants applied concepts and skills" (concept num ber 8 6). 2. C oncepts w ere used in various levels of intensity, with use rates from a low of 42% to a high of 100%. Table 4 indicates frequency of concepts used. T h ese are listed by concept areas. Table 4: Frequency of concept use by concept area. (N = 45) C oncept Area 1. Planning: Analysis 2. Planning: Justification No. Program Planning Concept Freq. 1.5 Consideration w as provided for social/educational backgrounds of learner 44 1.4 Problem a re a s/n e e d s studied and analyzed 43 1.3 Identified target population for the program 42 1.2 All stakeholders w ere represented on the committee 36 1.1 Established planning/advisory comm ittee 34 2.4 Identified program priorities b a sed upon employer n eed s 44 2.3 Identified program priorities b a sed upon trainees’ interests and needs 42 2.2 Selected subject a re a s related to identified perform ance deficits 40 2.1 Identified specific job perform ance deficits to be ad d ressed 39 2.5 Identified program priorities b a sed upon provider’s institution’s preparedness/availability 38 103 Table 4: Continued. C oncept Area 3. Objectives 4. Administra­ tion 5. Instruction No. Program Planning Concept Freq. 3.2 Objectives w ere established prior to program implementation 43 3.3 Objectives w ere related to the com pany’s job perform ance deficits 41 3.4 Objectives w ere related to the interests/needs of trainees 40 3.1 Selected objectives that implied ch an g es in behavior of individuals 38 4.2 Administrator w as in charge of program 44 4.4 Selected instructional staff with docum ented expertise in content area 44 4.7 Developed a budget covering all anticipated resource costs 44 4.3 Determined criteria for selecting instructional staff 43 4.6 Clearly defined administration roles and responsibilities 42 4.1 An administrative plan w as developed 39 4.5 Arranged appropriate institutional facilities and services 39 5.1 Developed instructional plan 44 5.2 Determined m ethods of presentation appropriate to job perform ance objectives 43 5.3 Selected instructional m aterials appropriate to job perform ance objectives 43 5.5 Utilized m ethods of interactive learning in instructional activities 43 5.4 Developed pre-training activities helping learner to anticipate ideas and concepts to be presented 36 104 Table 4: Continued. C oncept Area 6. Marketing 7. Evaluation 8. Program Delivery No. Program Planning C oncept Freq. 6.5 Oriented administrative and instructional staff to all a sp ec ts of program 43 6.6 Informed prospective participants of program 42 6.2 Com m unicated value of program to appropriate em ployer decision m akers 40 6.3 The power structure within participating employer supported training program 38 6.8 Oriented participants to program 37 6.7 Recruited participants for program 32 6.1 Developed marketing plan 27 6.4 Marketing effort w as organized and inclusive 26 7.1 Developed evaluation plan 44 7.2 Developed evaluation instrum ents to m easure program effectiveness 42 7.6 Analyzed and reported results of evaluation 40 7.3 Administered evaluative instrum ents based upon established criteria 39 7.4 observed participants during instructional activities 39 7.5 Evaluated objectives and m ethods constantly 38 7.7 Developed post-training and appropriate follow-up activities 33 8.6 Participants applied skills and concepts 45 8.7 Participants could apply skills on real work material 44 8.4 Participants w ere provided opportunity to ask questions or se ek clarification in training session 43 105 Table 4: Continued. C oncept Area 8. Program Delivery (cont’d) No. Program Planning Concept Freq. 8.5 Instructors w ere prepared for classes 43 8.8 Instructors provided for participant understanding 43 8.2 Attendance w as monitored 42 8.1 Program planner participated in delivery of program 29 8.3 Support c la sse s w ere also offered to help students learn 23 8.9 A placem ent com ponent w as implemented a s part of the training program 19 R esults. Although all concept a re a s w ere used, certain concepts within those a re a s w ere used more frequently than others. Concept a re a s may have had both low and high use. The following concepts w ere used by 98% or more of those planners surveyed: 8.6: Participants applied skills and concepts (Program Delivery) 2.4: Identified program priorities b ased upon employer n e ed s (Planning Justification) 1.5: Consideration provided for social/educational backgrounds of the learner (Planning Analysis) 4.2: Administrator in charge of program (Administration) 4.4: Selected instructional staff with documented expertise (Administration). 4.7: Developed budget covering anticipated resource costs (Administration) 5.1: Developed instructional plan (Instruction) 106 7.1: Developed evaluation plan (Evaluation) 8.7: Participants could apply skills on real work material (Program Delivery). R esearch Question 3 : Do program planners consider som e planning concepts to be m ore important than others? If so, what are they? Data w ere gathered from the 50-item survey that specifically asked program planners to respond to the question, "How important w as this planning concept to an effective program?" Table 5 displays the specific concepts considered to be important in program planning. C oncepts are listed in rank order of importance. Table 5: No. Importance of program planning concepts. (1 = not important; 5 = essential) P rogram P lanning C o n cep t Function M ean 8.7 P articipants could apply skills on real work m aterials P rogram Delivery 4.62 8.6 P articipants applied c o n c e p ts and skills P ro g ram Delivery 4.62 8.5 Instructors w e re p re p are d for c la s s e s P rogram Delivery 4.60 4.7 D eveloped a b u d g et covering all anticipated re so u rc e co sts Adm inistration 4.47 8.4 P articipants w e re provided opportunity to a s k q u estio n s or s e e k clarification in training se s sio n P rogram Delivery 4.47 4.4 S e le c te d instructional staff with d o cu m en te d ex p e rtise in co n ten t a re a Adm inistration 4.44 8.8 Instructors provided for participant u n d erstanding Program Delivery 4.44 5.1 D eveloped instructional plan Instruction 4.40 2.4 Identified program priorities b a se d upon em ployer needs Planning Justification 4 .3 8 1.3 Identified targ e t population for th e program P lanning A nalysis 4 .3 3 3.2 O bjectives w e re estab lish ed prior to program im plem entation O bjectives 4 .2 9 8.2 A tten d an ce w a s m onitored P rogram Delivery 4.24 107 Table 5: No. Continued. P ro g ram P lanning C o n cep t Function M ean 3.3 O bjectives related to co m p an y 's job perform ance deficits O bjectives 4.18 5.3 S e le c te d instructional m aterials appropriate to job p erfo rm an ce objectives Instruction 4.18 1.4 P roblem a r e a s /n e e d s studied and analyzed P lanning A nalysis 4.13 4.2 A dm inistrator w a s in c h a rg e of program Adm inistration 4.02 4.3 D eterm ined criteria for selecting instructional staff Adm inistration 4.00 5.2 D eterm ined m eth o d s of presen tatio n appropriate to jo b p erfo rm an ce objectives Instruction 3.98 7.2 D eveloped evaluation instrum ents to m e a s u re program effectiv en ess Evaluation 3.98 7.1 D eveloped evaluation plan Evaluation 3.96 7.6 A nalyzed/reported re su lts of evaluation Evaluation 3.89 6.6 Inform ed pro sp ectiv e participants of program M arketing 3.87 4.6 C learly defined adm inistrative roles/responsibilities Adm inistration 3.80 6.2 C om m unicated value of program to appropriate em ployer decision m ak ers M arketing 3.78 6.3 P o w er structure within participating em ployer su p p o rted training program M arketing 3.78 5.5 Utilized m eth o d s of interactive learning in instructional activities Instruction 3.76 6.5 O riented adm inistrative/instructional staff to all a s p e c ts of program M arketing 3.73 4 .5 A rranged ap p ro p riate institutional facilities/services Adm inistration 3.71 1.5 C onsideration provided for social/educational b ack g ro u n d s of learner P lanning A nalysis 3.69 2.2 S e le c te d su b ject a r e a s related to identified p erfo rm an ce deficits P lanning Justification 3.69 7.3 A dm inistered evaluative instrum ents b a sE d upon esta b lish e d criteria Evaluation 3.64 108 Table 5: No. Continued. P rogram P lanning C o n cep t Function M ean 12 All sta k e h o ld e rs w e re re p re se n te d on com m ittee P lanning A nalysis 3.60 5.4 D eveloped pre-training activities helping learn er anticipate id e a s/c o n c e p ts to b e p re se n te d Instruction 3.60 6.8 O riented participants to program M arketing 3.58 7.4 O b serv ed participants during instructional activities Evaluation 3.51 7.5 E valuated ob jectiv es/m eth o d s constantly Evaluation 3.51 4.1 An adm inistrative plan w a s dev elo p ed A dm inistration 3.49 2.3 Identified program priorities b a se d upon tra in e e s' in te re sts/n e e d s P lanning Justification 3.40 3.4 O bjectives related to in te re sts/n e e d s of tra in e e s O bjective 3.47 2.1 Identified job perform ance deficits Planning Justification 3.47 1.1 E stab lish ed planning/advisory council P lanning A nalysis 3.47 7.7 D eveloped post-training follow-up activities Evaluation 3.40 6.7 R ecruited participants for program M arketing 3.33 3.1 S electe d objectives th at implied c h a n g e s in individual behavior O bjective 3.31 8.1 P lan n er participated in delivery of program P rogram Delivery 2.96 2.5 Identified program priorities b a s e d on institution's preparedness/availability P lanning Justification 2.96 8.3 S upport c la s s e s offered to help stu d e n ts learn P rogram Delivery 2.76 6.1 D eveloped m arketing plan M arketing 2.51 6.4 M arketing effort organized/inclusive M arketing 2.47 8.9 P la cem e n t co m p o n en t im plem ented P rogram Delivery 2.42 109 R esults. Of the 17 concepts having a m ean score of 4.00 or higher, six were in th e Program Delivery area, four w ere in the Administration area, two w ere in Planning Analysis, two w ere in Instruction, and two w ere in Objectives. Of the six concept a re a s having a m ean score of 3.00 or less, three w ere in Program Delivery, two w ere in Marketing, and one w as in Planning Justification. The data indicated that certain concepts within a concept area w ere perceived to be m ore important than other concepts within the sam e concept area. Table 6 illustrates program planning concept a re a s and im portance m ean scores. Again, a concept area w as a grouping of related planning concepts. There w ere eight concept areas. Table 6: Importance of planning concept a re a s by m ean scores. (1 = not important; 5 = essential) C oncept Area No. of Items Mean of M eans Std. Dev. Planning; Analysis 5 3.84 .37 Planning: Justification 5 3.58 .52 Objectives 4 3.81 .49 Administration Plan 7 3.99 .36 Instructional Plan 5 3.98 .32 Marketing Plan 8 3.38 .57 Evaluation 7 3.70 .24 Program Delivery 9 3.90 .91 TOTAL 50 110 R esults. 1. The data indicated that developing an administrative (3.99) and instructional plan (3.98) had the highest m ean sc o res when comparing concept a re a s ’ perceived importance. 2. Developing a marketing plan (3.38) had the lowest concept area m ean score. The next section, R esearch Question 4, exam ined the perceived effectiveness of using selected planning concepts within an AEAT program. R esearch Question 4 : Do program planners consider som e concepts more effective than others? If so, w hat are they? Data w ere gathered through the 50-item survey in which planners w ere asked to respond to the question, "To what degree w ere you effective in using this in your program?" A sum m ary of their resp o n ses is provided in Table 7. The table show s specific program planning concepts that w ere perceived to be m ore effective than others. C oncepts are listed in rank order. Table 7: No. Effectiveness of program planning concepts. (1 = did not use, 2 = not effectively used, 3 = som ew hat effectively used, 4 = effectively used, 5 = very effectively used) P rogram Planning C oncept Function M ean 8 .5 Instructors w e re p re p are d for class P rogram Delivery 4.38 4.4 S e le c te d instructors with d o cu m en te d ex p ertise in co n ten t a re a Adm inistration 4 .3 6 2.4 Identified program priorities b a se d upon em ployer needs Planning Justification 4.31 111 Table 7: No. Continued. P ro g ram P lanning C o n cep t Function M ean 8.4 P articipants could a s k q u e stio n s/se e k clarification P rogram Delivery 4.31 8.8 Instructors provided for participant understanding P ro g ram Delivery 4.20 8.7 P articipants could apply skills on real work m aterial P rogram Delivery 4.20 8.6 P articipants applied c o n c e p ts and skills P rogram Delivery 4.18 8.2 A tten d an c e w a s m onitored P rogram Delivery 4.16 1.3 Identified targ e t population for program P lanning A nalysis 4.13 4.7 D eveloped bu d g et covering anticipated co sts Adm inistration 4.07 5.1 D eveloped instructional plan Instruction 4.04 4.2 A dm inistrator in c h a rg e of program Adm inistration 4.02 3.2 O bjectives e sta b lish e d prior to program im plem entation O bjective 4.00 4.3 D eterm ined criteria for selecting instructional staff A dm inistration 4.00 5.3 S electe d instructional m aterials appropriate to job p erform ance objectives Instruction 3.96 1.4 P roblem a r e a s /n e e d s studied/analyzed P lanning A nalysis 3.82 3.3 O bjectives related to co m p an y ’s job perform ance deficits O bjective 3.78 6.6 Inform ed pro sp ectiv e participants of program M arketing 3.73 5.2 D eterm ined m eth o d s of p resentation ap propriate to job p erfo rm an ce objectives Instruction 3.71 2.2 S e le c te d su b ject a r e a s related to identified p erform ance deficits Planning Justification 3.69 5.5 Utilized m eth o d s of interactive learning in instructional activities Instruction 3.69 7.1 D eveloped evaluation plan Evaluation 3.69 112 Table 7: No. Continued. P rogram P lanning C o n cep t Function M ean 1.5 C onsideration provided for social/educational b ack g ro u n d s of learn ers P lanning A nalysis 3.67 4 .5 A rranged appropriate institutional facilities/services Adm inistration 3.67 4.6 C learly defined adm inistrative roles/responsibilities Adm inistration 3.67 7.2 D eveloped evaluation instrum ents b a s e d upon e sta b lish e d criteria Evaluation 3.64 6.2 C om m unicated value of program to ap p ro p riate em ployer decision m ak ers M arketing 3.58 6 .5 O riented adm inistrative/instructional staff to all a s p e c ts of program M arketing 3.56 7.6 A nalyzed/reported resu lts of evaluation Evaluation 3.56 2.3 Identified program priorities b a se d upon tra in e e s' in te re sts/n e e d s Planning Justification 3.44 3.4 O bjectives related to in te re sts /n e e d s of tra in e e s O bjective 3.40 6.8 O riented participants to program M arketing 3.40 2.1 Identified specific jo b p erfo rm an ce deficits to be a d d re s s e d P lanning Justification 3.38 2.5 Identified program priorities b a s e d upon provider’s institution's p reparedness/availability P lanning Justification 3.38 4.1 Adm inistrative plan dev elo p ed Adm inistration 3.36 7.3 A dm inistered evaluative instrum ents b a s e d upon e sta b lish e d criteria Evaluation 3.33 7.4 O b serv ed participants during instructional activities Evaluation 3.27 3.1 S e le c te d objectives that implied c h a n g e s in behavior of individuals O bjective 3.24 5.4 D eveloped pre-training activities Instruction 3.18 6.3 P ow er stru ctu re within participating em ployer su p p o rted training program M arketing 3.16 113 Table 7: Continued. P rogram P lanning C o n cep t No. Function M ean 1.2 All sta k e h o ld e rs re p re s e n te d on advisory com m ittee P lanning A nalysis 3.13 6.7 R ecruited participants for program M arketing 3.09 1.1 E stab lish ed planning/advisory com m ittee P lanning A nalysis 3.04 7.5 E valuated ob jectiv es/m eth o d s co nstantly Evaluation 3.02 8.1 P la n n e r participated in delivery of activities Program Delivery 2.84 7.7 D eveloped post-training/follow -up activities Evaluation 2.78 6.1 D eveloped m arketing plan M arketing 2.44 8.3 S upport c la s s e s offered to help s tu d e n ts learn P rogram Delivery 2.31 6.4 M arketing effort w a s organized/inclusive M arketing 2.22 8.9 P la c e m e n t co m p o n en t im plem ented in training program P rogram Delivery 2.07 Results. 1. Of the 14 concepts having a m ean score of 4.00 or higher, six w ere in the Program Delivery a re a and four w ere in Administration. 2. Of the six planning concepts with a m ean score of 3.00 or less, three occurred in Program Delivery and two occurred in Marketing. A sum m ary of planning concept a re a s and effectiveness m ean sc o res are p resented in Table 8. 114 Table 8: Planning concept a re a s and effectiveness m ean scores. (1 = did not use, 2 = not effectively used, 3 = som ew hat effectively used, 4 = effectively used, 5 = very effectively used) Concept Area No. of Items Mean of M eans Std. Dev. Planning: Analysis 5 3.56 .46 Planning: Justification 5 3.64 .40 Objectives 4 3.61 .35 Administration Plan 7 3.88 .33 Instructional Plan 5 3.72 .34 Marketing Plan 8 3.15 .55 Evaluation 7 3.33 .34 Program Delivery 9 3.63 .94 TOTAL 50 Results. 1. The data indicated that Administration (3.88) has the highest m ean sco re when com paring concept a re a s’ perceived effectiveness. 2. Developing a marketing plan (3.15) had the lowest concept area m ean score. Information will now be presented on the correlation betw een each concept’s im portance and effectiveness. Although this information does not a d d re ss a particular research question, it does provide th e reader with additional data regarding th e association betw een what is perceived to be both important and effective. 115 Using the data from the survey instrument, a statistical analysis w as com pleted to determ ine the correlation betw een perceived importance and effectiveness of planning concepts. It should b e noted that a high correlation does not always m ean that a planning concept is important or effective. A high correlation d o es illustrate that a relationship does exist. For exam ple, "marketing" w as rather consistently ranked low for im portance and effectiveness. However, the correlation for this concept area w as ranked high. Table 9 illustrates planning concepts that w ere m ost highly correlated betw een perceived im portance and effectiveness to the .001 level of significance. Table 9: No. Correlation: Importance and effectiveness by rank order correlation. P rogram P lanning C o n cep t C o n cep t A rea Corr. 4.5 A rranged ap p ro p riate institutional facilities and serv ic e A dm inistration .9265** 8.5 Instructors w e re p re p are d for c la s s P rogram Delivery .9129** 2.2 S e le c te d su b ject a r e a s related to identified p erfo rm an ce deficits P lanning Justification .9006** 6.7 R ecruited participants for program M arketing .8965** 6.3 P o w er stru ctu re within participating em ployer su p p o rted training program M arketing .8821** 6.8 O riented participants to program M arketing .8810** 6.2 C om m unicated v alu e of program to ap p ro p riate em ployer decision m ak ers M arketing .8637** 4.4 S e le c te d instructional staff with d o cu m en te d e x p e rtise in co n ten t a re a Adm inistration .8609** 116 Table 9: Continued. No. P rogram P lanning C o n cep t 3.1 S e le c te d objectives th at implied c h a n g e s in behavior o f individuals O bjectives .8560** 8.8 Instructors provided for participant un d erstan d in g P rogram Delivery .8549** 8.4 P articipants provided opportunity to a s k q u estio n s or s e e k clarification P rogram Delivery .8571** 6.1 D eveloped m arketing plan M arketing .8520** 8.9 P la cem e n t co m p o n en t im plem ented a s part of training program P rogram Delivery .8485** 5.4 D eveloped pre-training activities helping learn er to anticipate id e a s/c o n c e p ts p re s e n te d Instruction .8392** 6.4 M arketing effort o rganized and inclusive M arketing .8347** 1.2 All sta k e h o ld e rs w e re re p re se n te d on th e com m ittee P lanning A nalysis .8310** 2.3 Identified program priorities b a se d upon tra in e e s ’ in te re sts/n e e d s Planning Justification .8138** 8.3 S upport c la s s e s offered to help s tu d e n ts learn P rogram Delivery .8122** 8.1 P rogram plan n er participated in delivery of program P rogram Delivery .8106** 6.6 Inform ed pro sp ectiv e participants of program M arketing .8079** 2 .5 Identified program priorities b a se d upon provider’s institution's prep ared n ess/av ailab ility Planning Justification .8066** 1.1 E stab lish ed planning/advisory com m ittee P lanning A nalysis .7983** 4.3 D eterm ined criteria for selecting instructional staff Adm inistration .7845** 5.5 Utilized m eth o d s of interactive learning in instructional activities Instruction .7780** 7.4 O b serv ed participants during instructional activities Evaluation .7634** 4.1 Adm inistrative plan w a s d ev eloped Adm inistration .7616** 2.1 Identified specific jo b perfo rm an ce deficits to b e a d d re s s e d P lanning Justification .7573** C o n cep t A rea Corr. 117 Table 9: No. Continued. P rogram P lanning C o n cep t C o n cep t A rea Corr. 1.3 Identified targ et population for program P lanning Analysis .7549** 4 .7 D ev elo p ed b u d g et covering all anticipated re so u rc e c o s ts Adm inistration .7529** 3.3 O bjectives related to co m p an y ’s jo b perfo rm an ce deficits O bjectives .7512** 2.4 Identified program priorities b a s e d upon em ployer needs P lanning Justification .7356** 3.4 O bjectives related to in te re sts/n e e d s of tra in e e s O bjectives .7001** 8.2 A tten d an c e w a s m onitored P rogram Delivery .6989** 6 .5 O riented adm inistrative/instructional staff to all a s p e c ts of program M arketing .6823** 1.5 C onsideration provided for social/educational b ack g ro u n d s Planning A nalysis .6679** 8.7 P articipants could apply skills on real w ork m aterial P rogram Delivery .6641** 1.4 P roblem a r e a s /n e e d s studied an d an aly zed Planning A nalysis .6 6 3 4 ’* 7.3 A dm inistered evaluative instrum ents b a s e d on e sta b lish e d criteria P rogram Delivery .6610** 4 .2 A dm inistrator w a s in ch a rg e of program A dm inistration .6574** 7.7 D eveloped post-training and ap p ro p riate follow-up activities P rogram Delivery .6565** 4.6 C learly defined adm inistrative roles/responsibilities A dm inistration .6536** 3.2 O bjectives estab lish ed prior to program im plem entation O bjectives .6494** 7 .5 E valuated o b jectiv es/m eth o d s constantly Evaluation .6323** 5.3 S e le c te d instructional m aterials ap p ro p riate to job p erfo rm an ce objectives Instruction .5001 7.6 A nalyzed/reported resu lts of evaluation Evaluation .4889 118 Table 9: Continued. No. P rogram Planning C o n cep t C o n cep t A rea Corr. 5.2 D eterm ined m eth o d s of p re sen tatio n ap p ro p riate to job p erfo rm an ce objectives Instruction .4862 5.2 D eveloped instructional plan Instruction .4701 7.2 D ev elo p ed evaluation instrum ents Evaluation .4405 7.1 D ev elo p ed evaluation plan Evaluation .4178 8.6 P articip an ts applied co n c ep ts an d skills P ro g ram Delivery .3060 R esults. The perceived im portance and effectiveness of arranging appropriate facilities and services, having instructors prepared for class, and selecting subject a re a s related to perform ance deficits had correlations higher than .9000. T he following concepts had m ean im portance and effectiveness sco res of over 4.0, and are ranked by correlation: 1. Instructors w ere prepared for class (Program Delivery). 2. S elected instructional staff with docum ented expertise in the content area (Administration). 3. Instructors provided for participant understanding (Program Delivery). 4. Participants provided opportunity to ask questions o rse e k clarification (Program Delivery). 5. Determ ined criteria for selecting instructional staff (Administration). 6. Identified target population for program (Planning Analysis). 119 7. Objectives related to com pany’s job perform ance deficits (Objectives). 8. Identified program priorities based upon com pany n e ed s (Planning Justification). 9. A ttendance w as monitored (Program Delivery). 10. Participants could apply skills on real work material (Program Delivery). 11. Administrator w as in charge of program (Administration). 12. Selected instructional m aterials appropriate to job perform ance objectives (Instruction). 13. Participants applied concepts and skills (Program Delivery). Program planners also recom m ended specific strategies that they found to be important and effective. They are reviewed in the next several pages. Important and Effective C oncepts That Work: R ecom m endations From Planners The data gathered in the interviews and focus group m eetings indicate that the 1992-93 program planners actually utilized additional planning concepts that were not included in th e questionnaire’s 50 concepts. Q uestions such a s "Are there som e things you might have done differently in this project?” and "Is there any advice you would give to future program planners wanting to implement a training/retraining program?" w ere asked. T h ese questions w ere suggested by the Michigan Departm ent of Education and the Michigan Jo b s Commission to determ ine specific program planning recom m endations that actual planners perceive to be important and effective. The resp o n se s are grouped in on e of the eight concept 120 are similar to those existing concepts with that area. A sum m ary of program planners suggestions to enhance program outcom es is presented: 1. CONCEPT AREA: PLANNING ANALYSIS Approach AEAT program s from th e b usiness point of view. Make sure everyone who n eed s to be involved in the planning is involved. Ask questions and get firm answ ers before beginning training. Do an a sse ssm e n t of participants’ basic skills before training. Provide appropriate classes/activities to prequalify participants before training. 2. CONCEPT AREA: PLANNING-JUSTIFICATION Determ ine that the AEAT is compatible with th e com pany’s real training needs. Make certain that training requested is legitimate, fundable, and m eets state eligibility requirem ents. Validate/verify that the training is n eeded at the plant level (don’t a ssu m e that corporate headquarters can dictate blanket training at all plants). Training m ust be consistent with training institution mission. Determ ine that trainers understand what the com pany e x p e c ts -a n d w hat the anticipated results should be. Provide an orientation to participants before starting program. E nhance two-way communication betw een com pany to be trained and state officials. (State n e ed s to better understand training and production problems; com pany n e ed s to better understand strict conditions and procedures asso ciated with the AEAT program.) 121 3. CONCEPT AREA: OBJECTIVES Have com panies start with a wish list of training they would like to have done. Then convert that list, with com pany officials, into a reality list of training that can actually be provided. Make sure that the com pany and th e college are comfortable with realistic objectives. The com pany n eed s to accept them; th e college m ust be able to deliver them. After the objectives are drafted, provide opportunity for com pany to review and modify, if necessary. Then, have colleagues from your institution (including business office) review for accuracy and do-ability. 4. CONCEPT AREA: ADMINISTRATION Involve everyone who n eed s to be involved at the earliest possible occasion. Make sure the com pany to be trained is familiar with the stringent state requirem ents and procedures for training. Delineate specific responsibilities in writing. Develop a trusting and open relationship with other area agencies: Michigan Employment and Security Commission, the Service Delivery Area, and local schools. Sidebar agreem ents with com panies to recover unallowable expenses. Charging the com pany for services ten d s to m ake th e service worth m ore (in th e ey es of the company). Be honest in all accounting principles; otherwise this will haunt you. Establish and maintain a system atic pattern of evidence that dem onstrates w hat your program accom plishes. 5. CONCEPT AREA: INSTRUCTION Hand pick your instructors/trainers. Make sure they know their content and adult learning theory. Have instructors get on the com pany site to m eet designated com pany representatives. This will enhance their understanding of what is expected of them . 122 6 CONCEPT AREA: MARKETING Develop and maintain an ongoing professional relationship with the Michigan Jo b s Team. Attend their sem inars, focus group m eetings, and other functions. Call them routinely. Involve the account representative from the Commission at every possible occasion. Maintain and nurture ongoing com pany contact before, during, and after the training. Learn from other program planners. sh are resources. Participate in training institutes and R ecognize and praise instructors, program advocates, and participants who advance the training initiatives. Inform your legislators and other area leaders about the training conducted and its effect. Develop a prospect file for potential com panies to be served. 7. CONCEPT AREA: EVALUATION Provide opportunity for informal a sse ss m e n t and evaluation by participants. Program participants, instructors, and com pany officials need to have ongoing opportunity to a s s e s s and evaluate the training. Do everything possible to have com pany commit to training and to providing the n ecessary follow-up. 8. MONITORING PROGRAM DELIVERY Start program delivery a s soon a s training proposal is approved. Deal with instructional/training problem s openly, honestly, and when they occur. Try to never say "no" to com pany requests for training. developing a training program. Be creative in 123 R esults. The preceding practical suggestions w ere recom m ended to improve program outcom es by the AEAT planners. T hese suggestions, although similar to th o se planning concepts surveyed, offer the AEAT planner additional information in developing a program plan. The next research question explored the relationship betw een planning program m ers’ satisfaction and actual outcom es. Relationship Between Planning/Satisfaction/Outcom es R esearch Question 5 : Do associations exist betw een using planning concepts and planners’ satisfaction with completing stated objectives, and actual placem ent outcom es? Data w ere gathered from the survey instrument (extent of satisfaction with program) and frequency of planning concepts used. This information w as m erged with data from the final program reports provided to the Departm ent of Education (placem ent p ercentages into actual jobs). Som e training institutions trained and/or placed m ore participants than w as indicated in the program objectives. For this reason, their placem ent p ercentages m ay be higher than 100%. A sum m ary of this data is p resented in Table 10. This table indicates the num ber of program planning concepts used by each planner. It also show s the level of satisfaction (1 = very satisfied; 5 = very dissatisfied) and the job placem ent percentage of program participants. A com plete sum m ary table of concepts used, im portance and effectiveness of concepts, and correlations is included in Appendix I. 124 Table 10: Program satisfaction/percentage placed in job. (1 = very satisfied, 2 = satisfied, 3 = neutral, 4 = dissatisfied, 5 = very dissatisfied) Satisfaction R ate Training Institution Planning C oncepts Used Placem ent % 1 G arden City 49 73.08 1 LMCC 49 183.33 1 Lansing P 49 100.00 1 COOR 48 200.00 1 Allegan 47 100.00 1 So. Kent 47 87.18 1 SCCC 46 0.00 1 SWM 46 116.27 1 OCC 46 109.75 1 W ashtenaw 46 66.67 1 Davenport 44 100.00 1 Kellogg 43 106.45 1 Eaton 42 74.78 1 Ottawa ISD 42 76.96 1 Glen O aks 42 116.25 1 W ashtenaw 41 89.11 1 Davenport 39 100.00 1 Detroit P 36 0.00 1 Monroe 36 87.10 1 Ferris 35 48.00 1 AVERAGE 4 3 .6 5 9 1 .7 5 125 Table 10: Continued. Satisfaction R ate Training Institution Planning C oncepts Used Placem ent % 2 SW Mich 49 116.27 2 Newaygo 49 108.33 2 Saginaw P 49 103.95 2 Alpena 49 127.20 2 Coldwater 49 91.11 2 Montcalm 48 118.18 2 W ayne CC 48 0.00 2 H Ford 47 90.00 2 Tuscola ISD 47 16.35 2 KVCC 47 77.68 2 No. Central 46 174.77 2 H Ford 46 90.00 2 H Ford 46 130.77 2 Kent ISD 43 33.33 2 CS Mott 41 0.00 2 Macomb 41 100.00 2 Montcalm 41 20.00 2 Jackson 40 93.33 2 AVERAGE 4 5 .8 9 8 2 .8 5 126 Table 10: Continued. Satisfaction R ate Training Institution Planning C oncepts Used Placem ent % 3 Wyoming 48 97.81 3 Macomb 33 80.00 3 Service Industries 20 100.00 3 AVERAGE 3 3 .6 7 9 2 .6 0 4 White Pigeon 44 24.00 4 Bay 43 100.00 4 AVERAGE 4 3 .5 6 2 .0 0 5 MCCC 46 0.00 R esults. Table 10 show s that th e planners used 20 to 49 program planning concepts, ranged in the satisfaction resp o n se from "very dissatisfied" to "very satisfied," and had placem ent rates ranging from 0% to 200%. 1. T hose planners with a satisfaction rate of "1" (very satisfied) used an av erag e of 43.65 concepts, with a m ean placem ent score of 91.75% . 2. T hose planners with a satisfaction score of "2" (satisfied) utilized an av erag e of 45.89 concepts and had a placem ent m ean of 82.85%. 3. Three planners had a satisfaction score of "3" (neutral), used 33.67 concepts and had a m ean placem ent of 92.6%. 4. Two planners had a satisfaction score of "4” (dissatisfied), utilized an av erag e of 43.5 concepts and had a placem ent rate of 62%. 127 5. O ne planner had a satisfaction score of "5" (very dissatisfied), used 46 concepts, and had 0% placem ent. Table 11 illustrates m ore com posite data of satisfaction sc o re s and placem ent p ercen tag es, a s com pared to actual concepts used. Table 11: Satisfaction/placem ent m eans. No. of C o n c e p ts U sed No. of P la n n e rs Using C o n cep ts P lan n er S atisfaction With P rogram --M eana P la cem e n t O u tc o m e s -M e a n 49 8 1.63 112.91 7 3 .0 8 - 183.33 48 4 2.00 104.00 0.00 - 2 0 0 .0 0 47 5 1.60 74.24 1 6 .3 5 - 100.00 46 8 1.88 44 2 2.50 62.00 2 4 .0 0 - 2 0 0 .0 0 43 3 2.33 79.93 3 3 .0 0 - 106.45 42 3 1.00 89.33 7 4 .7 8 - 116.25 41 4 1.75 52.28 0.00 - 100.00 40 1 2.00 93.33 93.33 39 1 1.00 100.00 100.00 36 2 1.00 43.55 0 .0 0 - 87.10 35 1 1.00 48.00 4 8 .0 0 33 2 1.50 90.00 80.00 - 100.00 20 1 3.00 100.00 100.00 a 1 = high, 5 = low. 86.03 P la c e m e n t R a n g e Low High 0 .0 0 - 174.77 128 Results- 1. Planners using 48 or 49 concepts had the highest placem ent outcom e m ean (104% and 112.91%, respectively). 2. Planners using 48 or 49 concepts had a placem ent range of 0% to 3. Planners using 46 to 47 concepts had a placem ent range of 0% to 200 %. 174.77%. 4. Planners using 40 to 44 concepts had a placem ent range of 0% to 5. Planners using 20 to 39 concepts had a placem ent range of 0% to 6. No pattern seem ed to em erge regarding 200 %. 100 %. num ber of concepts used and satisfaction rates and/or placem ent outcom es. Table 12 indicates the num ber of planners using 46 or m ore com ponents and th o se using 45 or less. Table 12: Satisfaction and placem ent outcom e cluster. No. of C oncepts Used Freq. Ave. Satisfaction S core3 A verage P lacem ent % 12 1.75 109.94 46-47 13 1.77 81.50 41-44 12 1.83 70.08 8 1.50 76.05 48-49 40 or few er a 1 = high, 5 = low. 129 Results- 1. Planners using 48 to 49 concepts had the highest placem ent p ercen tages (109.94%). 2. Planners using 46 to 47 concepts had the second highest placem ent m ean, or 81.5 percent. 3. Planners using 40 or fewer concepts had th e third highest placem ent m ean, or 76.05%. 4. Planners using 41 to 44 concepts had the lowest placem ent m ean or 70.08%. 5. Although planners using 4 0 o r fewer concepts had the highest averag e satisfaction scores (1.5), there w as a tendency for th o se planners using m ore concepts to have a higher satisfaction score. Table 13 contains a sum m ary of each program ’s satisfaction rates, placem ent p ercen tages and perceived effectiveness m eans of each concept area. R esults. Of th e 20 planners who w ere very satisfied (satisfaction score = 1) with a specific training program, 10 had placem ent rates of m ore than 100%, and another 7 had placem ent rates higher than 50%. Table 14 contains a sum m ary of placem ent outcom es and planners’ m ean sco res for all concepts. Table 13: Summary of concept areas and effectiveness means by program (N = 45). (1 = very satisfied, 2 = satisfied, 3 = neutral, 4 = dissatisfied, 5 = very dissatisfied) Planning Concept Areas Place­ ment % Satis­ faction 200 Planning Analysis Planning Just. Objec­ tive Adminis­ tration Instruc­ tion Market­ ing Evalua­ tion Program Delivery 1 4.8 4.2 2.5 4.4 4.2 4.1 4.3 4.7 183 1 4.6 5.0 5.0 5.0 5.0 5.0 5.0 5.0 175 2 3.2 3.2 3.0 3.9 4.0 2.8 3.9 2.6 131 2 2.8 2.8 3.5 3.9 3.2 3.3 2.9 3.4 127 2 3.0 3.6 3.5 2.7 3.4 4.1 3.6 3.3 118 2 4.6 4.6 5.0 4.4 5.0 4.1 4.7 3.8 116 2 4.0 3.0 4.5 4.1 3.4 3.6 3.9 3.4 116 1 4.0 3.0 4.3 3.9 4.0 3.9 4.0 3.6 116 1 4.2 3.4 4.3 3.9 4.4 4.5 3.5 2.8 110 1 3.4 3.6 4.0 4.0 4.0 3.0 3.1 3.4 108 2 3.8 4.4 3.8 3.1 4.6 3.5 4.4 4.4 106 1 3.2 2.8 4.3 4.6 3.8 2.3 3.4 3.7 104 2 4.4 4.0 4.0 4.3 4.4 3.5 2.9 4.3 100 1 4.4 4.6 4.5 4.6 4.8 4.4 3.9 4.3 100 2 3.6 4.4 3.0 3.7 4.0 2.1 2.7 3.7 Tabie 13: Continued. Planning Concept Areas Place­ ment % Satis­ faction 100 Planning Analysis Planning Just. Objec­ tive Adminis­ tration Instruc­ tion Market­ ing Evalua­ tion Program Delivery NA 2.6 4.5 4 3.7 4.3 4 3.4 4.7 100 3 3.5 NA 4.7 NA 3.8 NA 3.0 4.0 100 1 3.6 3.8 4.0 3.9 4.0 4.0 2.7 3.4 100 4 2.6 3.2 2.3 3.4 3.0 1.3 2.9 2.9 100 1 2.4 4.0 3.3 4.0 3.4 2.4 3.4 4.1 100 1 2.2 3.6 3.5 4.0 3.2 3.4 3.1 3.1 98 3 3.0 3.2 2.8 3.7 3.2 3.5 1.7 4.3 93 2 2.4 4.4 4.3 3.4 2.0 2.1 3.9 3.6 91 2 4.8 5.0 5.0 5.0 5.0 5.0 5.0 4.4 90 2 4.6 5.0 4.8 4.4 4.0 3.5 4.9 3.8 90 2 3.8 3.8 4.0 4.1 3.6 3.6 3.9 3.8 89 1 2.4 3.8 3.5 4.3 3.6 2.9 3.5 3.8 87 1 3.6 4.4 3.3 3.4 3.0 1.0 1.1 3.3 87 1 4.4 4.2 4.8 4.4 4.4 4.5 3.0 4.1 80 3 2.8 3.4 0.0 3.6 2.8 3.1 NA 28 78 2 3.8 3.4 3.0 4.3 4.2 3.2 4.3 4.1 Table 13: Continued. Planning Concept Areas Place­ ment % Satis­ faction 77 Planning Analysis Planning Just. Objec­ tive Adminis­ tration Instruc­ tion Market­ ing Evalua­ tion Program Delivery 1 4.4 4.6 4.8 4.6 4.6 3.3 4.2 3.8 75 1 2.5 4.0 2.0 2.7 3.0 3.1 2.0 2.1 73 1 3.6 4.4 5.0 5.0 4.6 4.8 4.3 4.9 67 1 4.2 2.4 4.3 4.3 3.8 3.0 3.6 4.0 48 1 3.8 4.0 4.0 3.6 3.3 3.4 NA 2.3 33 2 4.0 2.6 2.8 2.9 3.6 4.0 3.1 2.9 24 4 2.6 2.6 2.8 3.6 3.0 3.1 3.0 3.4 20 2 3.8 4.0 2.0 4.7 3.7 3.8 3.8 4.3 16 2 3.6 3.6 3.3 4.1 3.2 3.1 3.1 3.4 0 2 2.5 4.0 4.0 3.4 4.6 2.4 3.7 4.8 0 1 4.0 2.0 3.8 2.7 3.8 4.2 2.3 3.6 0 1 4.6 4.0 4.7 4.9 4.8 4.7 4.4 4.7 0 2 3.0 4.0 3.0 3.6 4.0 2.7 3.0 34 0 5 2.6 3.2 3.0 3.3 3.2 2.6 3.3 3.4 133 Table 14: Sum m ary of placem ent outcom es and m ean of m ean effectiveness sco res for all concepts. Placem ent P ercen tag e Mean of C oncept A reas 200 4.15 183 4.95 175 3.33 131 3.23 127 3.40 118 4.53 116 3.74 116 3.84 116 3.88 110 3.56 108 4.00 106 3.51 104 3.93 100 3.68 100 2.70 100 3.50 100 3.44 100 4.44 100 3.40 100 3.90 100 3.80 98 3.18 93 3.26 134 Table 14: Continued. Placem ent P ercentage Mean of C oncept A reas 91 4.90 90 4.38 90 3.83 89 3.33 87 2.89 87 4.23 80 2.64 78 3.79 77 4.29 75 2.67 73 4.58 67 3.70 48 3.49 33 3.24 24 3.01 20 3.76 16 3.81 0 3.64 0 3.34 0 4.60 0 3.34 0 3.08 135 R esults. Eighty percent of those planners with a 0% placem ent had concept m ean sc o res of 3.64 or less. Seventy percent of those planners with a placem ent p ercen tage of 116% or higher had a m ean concept score of 3.64 or higher. The mitigating variables, or those political and practical considerations, that may influence the outcom es of various training activities, are discussed in the next section. Mitigating Variables (Political and Practical Considerations^ R esearch Question 6 : What other mitigating variables may influence the outcom es of the various training activities? To answ er this question, data w ere gathered from the survey com m ents and interview questions (such a s "Were there any mitigating variables that affected the outcom es of the project?" and "Were th ere political, econom ic or institutional policy factors?"). Information w as also extrapolated from the focus group m eetings. The first focus group w as com prised of 18 AEAT program planners. The second focus group com prised 32 planners. The Michigan Jo b s Commission convened th ese m eetings to discuss perceived planner concerns or limitations in successfully completing the AEAT program. The purpose of the focus group m eetings w as to explore the possible relationship betw een mitigating variables, such a s political and practical considerations, in completing an AEAT program. The focus groups used a fact-based problem-solving technique that included a failure m ode and effects analysis (see C hapter III). The two groups then identified and ranked two specific 136 concerns, or mitigating variables, associated with the completion of program objectives: 1. Developing an effective program plan. 2. Making the n e ce ssa ry program modifications with the Michigan Jo b s Commission after the plan had been previously subm itted and approved (see Appendix H). This information is significant in that it illustrates the types and sources of factors that affect th e outcom es of the AEAT program. This information illustrates the role of practical and political considerations in program planning. In addition, this information supports the work of Cervero and Wilson (1994), who believed that program planning is only th e first step toward successful program outcom es. Focus Group: Effective Program Planning The first program planning focus group identified them es or clusters a s problematic in developing an effective program plan. The th em es are listed in the fishbone diagram, a s se e n in Appendix H. Specific problem s associated with program planning w ere ranked in this order: 1. The apparent lack of com pany commitment to or responsibility for the training or retraining program. 2. Intercollege or training environm ent considerations. 3. Lack of skills a sse ssm e n t/evaluation regarding the readiness skills of participants. 137 4. M onetary (dollars) and other resource limitations for completing the training activity. 5. Defining clear training objectives. This focus group then ranked the problems, or mitigating variables. The group identified "the training commitment and responsibility of the com panies being served" a s the m ost problem atic variable related to effective program planning. The first focus group re sp o n se s a re included in Figures 1 through 5. The resp o n ses w ere consolidated and modified to prom ote clarity for the reader. Figure 1 displays the group-developed affinity chart (or cluster of them es) asso ciated with com pany training commitment/responsibility. The focus group’s multiple voting score for "com pany training commitment/responsibility w as 18/36 possible points. Figure 2 displays the group-developed affinity chart for a second a re a of concern asso ciated with effective program planning: intercollege considerations. The focus group multiple voting score for "Inter-College considerations" w as 10/36 possible points. Figure 3 displays th e group-developed affinity chart for the third concern asso ciated with effective program planning: a sse ssm e n t and evaluation of the training program . The multiple voting score w as 4/36 possible points. Figure 4 displays the group-developed affinity chart for another concern a sso ciated with effective program planning: m onetary and resource limitations. The multiple voting score for "Dollars" w as 3/36 points. 138 * * * * * * Lack of quality input from com pany m anagem ent Scheduled training conflicts with production activities Lack of com plete and concise information from the com pany Having the com pany do w hat they said they would do Lack of understanding of time n eed ed to com plete training C om pany’s lack of understanding of how m any em ployees to be trained * Determining the real agenda, interest, and commitment of com pany to be served * Conflicts betw een com pany at local level and corporate headquarters of the type of training needed * Com pany’s choice of steering com m ittee to guide the training process Figure 1: Problem s related to com pany training commitment/ responsibility. * * * * * Lack of trainer resources to com plete training Scheduling and matching appropriate faculty The identification of best institutional team to com plete training Lack of time to com plete training in satisfactory fashion Political involvement of community agencies and groups at institutional level * Determining appropriate objectives and training content * Decisions regarding credit v ersu s noncredit for training activities * Interoffice cooperation, especially for reporting purposes Figure 2: Problem s related to intercollege considerations. * Com pany defined evaluative criteria without consulting with training provider * Com pany som etim es refused to a s s e s s em ployee’s skill levels * A ssessm en t plan not developed Figure 3: Problem s related to a sse ssm e n t and evaluation. 139 * Som e com panies said, "Just get us the money, w e’ll determ ine how to spend it" * Lack of administrative personnel (and funding) to com plete the training * Budget constraints-especially in a re a s not funded * Unrealistic com pany expectations for what budget could deliver Figure 4: M onetary/resource limitations. Figure 5 displays th e group-developed affinity chart for another concern asso ciated with effective program planning: developing objectives. The focus group multiple voting score for "Objectives" w as 1/36 points. * Getting com pany to assist in a sse ssm e n t process * Lack of written objectives in program plan * Receiving com pany agreem ent of training n e ed s and associated objectives * Lack of understanding of objectives * M anagem ent views of objectives not consistent with em ployees’ Figure 5: Problem s related to developing objectives. R esults. Five affinity charts, or clusters of mitigating variables, associated with effective program planning w ere identified by one focus group. The five main mitigating variables to program planning included: 1. The lack of com pany com m itm ent to training. Exam ples exp ressed in this are a included the a b se n c e of m anagem ent input, conflict with production activities, and conflicts betw een the com pany and the corporate headquarters. 140 2. Intercollege considerations. Specific problem s included scheduling conflicts, identifying appropriate training team s, time constraints, community involvement, and an a b se n c e of cooperation. 3. Lack of participant skill assessm ent/evaluation. For example, an a sse ss m e n t plan w a s not developed or used appropriately. 4. M onetary and other resource limitations. Specific concerns included unrealistic expectations for training and m isunderstandings of how the grant w as to be used. 5. The need for clear training objectives. Exam ples in this area included the a b se n c e of written objectives, and differences betw een m anagem ent’s and em ployees’ interpretations of the objectives. Focus Group: Program Modification The second focus group a d d ressed problem s associated with modification of the accepted training proposal at the com pany and state levels. The fishbone diagram displaying problem s a sso ciated with the modification of the training program can be found in Appendix H. Specific problems associated with modification include: 1. Communication problem s with the com pany to be trained. 2. Communication problem s with appropriate state office. 3. Community college or training provider’s internal communications. 4. Lack of a d eq u a te resources (hum an and other) to effectively complete the training or retraining activity. 141 5. The role of the business climate in which the training w as being conducted. 6. The level of com pany commitment to training. 7. Political considerations at local, company, and state levels. This focus group then ranked the problems, or mitigating variables, asso ciated with program modification. The group identified "communication problem s with the com pany to be trained" and "communication problem s with the state" a s the two m ost important considerations in making program modifications. T he second focus group’s resp o n se s are included in Figures 6 through 12. The re sp o n se s w ere consolidated and modified to prom ote clarity for the reader. This focus group did not u se th e sa m e multiple voting scoring system a s the first group, and w ere able to vote on only one problem, not two (as before). Communication problem s with the state, company, and at the college level had a single multiple voting score of 10/32 points. Although "Planning” had a score of 6/32, it w a s not discussed b ecau se this variable w as dealt with by the first group. Figure 6 displaysthe group-developed affinity chart ofthem es associated with "communication problem s with the company." Figure 7 displays the group-developed affinity chart for another primary mitigating variable affecting program modification: communication with the state (Michigan Jo b s Team). Figure 8 displays the group-developed affinity chart for concerns associated with program modification: internal communication problems. 142 * Lack of understanding betw een com pany and the sta te offices about the modification process * Lack of communication betw een sta te and applicant organization * S ta te contact with com panies not coordinated with training institution * Misinterpretation from com pany thinking grant m oney w as theirs to u se in any fashion * Misinterpretation of requirem ents and procedures by the com pany Figure 6: Communication Problem s With Company. * * * * * * * * Changing state requirem ents Conflicting communications from sta te offices and personnel Misinformation from account representatives or field m anagers Failure to clarify levels of a ccep tan ce for program modification Poor instructions in grant-writing procedure V ague definitions in grant applications Delays in the approval process Grant deadline requirem ents som etim e unrealistic Figure 7: Communication problem s with the state. * * * * * * * * * Poor or insufficient resources College c an ’t anticipate ch an g es in timely fashion AEAT application is incomplete Staff chan g es within training institution Staff overextended to com plete program s Limited availability of trainers and/or instructors Poor program administration Overall lack of internal communication Failure to understand the application process Figure 8: Internal communication problems. 143 Figure 9 displays the group-developed affinity chart for concerns associated with program modification: training institution's lack of resources to com plete training activity. The multiple voting score for "Lack of R esources" w as 5/32 points. * * * * Staff overextended Internal personnel changes Availability of trainers Program s not funded at levels requested Figure 9: Lack of resources. Figure 10 displays the group-developed affinity chart for concerns associated with program modification: the role of the business climate. The multiple voting score for "B usiness Climate" w as 5/32 points. * * * * * * Employee reductions and turnovers Com pany w ants plan changed after being subm itted and approved C hanges in production schedules C hanges in plant contact person C hanges in training focus from corporate office C hanges in technology used at plant Figure 10: B usiness climate. Figure 11 displays the group-developed affinity chart for concerns associated with program modification: com pany commitment to training. The multiple voting score for "Com pany Commitment" w as 3/32. 144 * * * * Lack of cooperation from com pany Em ployees don’t always complete training Employers don’t insist on participant attendance Com pany cancels scheduled training activities Figure 11: Company commitment to training. Figure 12 displays the group-developed affinity chart for concerns associated with program modification: politics associated with training. The multiple voting score for "Politics" w as 3/32 points. * D isputes betw een union m em bers and m anagem ent over nature of training, participants to be involved, and scheduling * Som e plants refuse funds b ecau se they don’t w ant training responsibility Figure 12: Political considerations. R esults: T he second focus group discussed problem s, or mitigating variables, affecting th e AEAT program-modification process. Several a re a s or clusters w ere included in making program modifications. T h ese variables, listed in order of perceived importance, include: 1. Communication problems at th e company, state, and college levels. Exam ples ex p ressed in this area included misinterpretations of AEAT rules and regulations, failures to clarify acceptance levels, and changing state regulations. 145 2. Lack of ad eq u ate resources to complete training. Specific concerns w ere d iscu ssed regarding staff and m onetary limitations. 3. An unpredictable business climate. Plant reductions, turnovers, and production schedules w ere cited a s exam ples in this area. 4. A perceived lack of com pany commitment to training. Specific problem s included an a b se n c e of com pany cooperation and follow-through. 5. Politics betw een the com pany, state, and the college. Specific concerns included internal disputes and refusal of funds. The focus group m eetings discu ssed and presented several political and practical concerns related to the Adult Education Alternative Training program. During th o se focus group m eetings, Michigan Jo b s Commission personnel advised th e planners to accom m odate th e se mitigating variables that m ay affect the training program. Sum mary C hapter IV w as divided into four parts. AEAT program outcom es and p lan n ers’ satisfaction with those outcom es w ere described in the first part. Qualitative and quantitative data directed toward R esearch Q uestions 2 through 4 w ere p resen ted in the second part. The actual program delivery techniques used to implement the AEAT activities w ere discussed, and the frequency of concepts u sed, and their perceived im portance and effectiveness, w ere explored. R esearch Q uestion 5 w as answ ered in the third part. 146 In th e fourth part, the role of other mitigating variables (R esearch Question 6) related to program outcom es w as explored. This information w as drawn primarily from interviews, written com m ents taken from the surveys, and focus group m eetings. C hapter V contains a sum m ary of the major findings, conclusions drawn from the findings, and recom m endations for further research. CHAPTER V SUMMARY, CONCLUSIONS, AND RECOMMENDATIONS Introduction C hapter IV p resented the data gathered during the study. It discussed actual AEAT outcom es and planner satisfaction of those outcom es. In addition, planning concepts used by AEAT program planners and the frequency of concepts used w ere reviewed. The preceding chapter also sum m arized the data related to the perceived im portance and effectiveness of the planning com ponents and the relationship betw een the u se of planning concepts and planners’ satisfaction with program outcom es. Finally, th e data collected through the interviews supported the work of Cervero and Wilson (1994), who suggested that program planning is a s much a sociopolitical process a s it is a technical one. One program planner sum m ed up the pro cess by saying, "You can plan for all the right reaso n s and your program can still fail." This study, using both quantitative and qualitative data, presented exploratory research that described the nature of planning in educational organizations. The literature on governm ent funded training and retraining program s helped in interpreting the planning experiences at those institutions. 147 148 Sixty-three percent (45) of the individuals completing 107 A activities returned a questionnaire. Ten m ore participated in an interview, and 50 participated in a focus group. The real effect of their planning efforts is just beginning. For the rea d er’s convenience, a consolidation of much of the data can be found in Appendix I. C hapter 5 contains a sum m ary of major findings, conclusions, and recom m endations for further research. Sum m ary of Findings 1992-93 AEAT Program O utcom es According to th e Michigan Departm ent of Education, the 1992-93 AEAT program resulted in the following outcom es: 156 projects w ere funded: $24,990,000 w as budgeted: 36,696 individuals w ere trained or retrained; and the training costs per individual averaged $681. Although the Michigan Departm ent of Education w as required to provide outcom e information on the trainees, m ore relevant evaluation of the job training program based on company-specific outcom es related to the training n e e d s of th e em ployer w as lacking. Therefore, local program sources were u sed to an sw er the following research questions. R esearch Q uestion 1: To what extent are Adult Education Alternative Training planners satisfied with the outcom es of specific 1992-93 program s? According to the survey, 84% of the program planners w ere very satisfied or satisfied with their program outcom es. Another 6.6% of the respondents were dissatisfied or very dissatisfied with th e program outcom es. While com m ents taken 149 from the program planner interviews and focus groups w ere consistent with the survey findings, the qualitative data illustrated the types of planner concerns with the AEAT program and reflected the political and practical a sp e c ts of program planning (se e Table 1, p. 90). R esearch Question 2 : Do Adult Education Alternative Training program planners use program planning concepts a s outlined by Boyle, Mitchell, Buskey, Cervero and Wilson, and others? Are som e program planning concepts used more frequently than others? If so, what are they Of a total of 50 specific program planning concepts, 55.6% (25) of the resp o n d ents used 45 or m ore in planning the AEAT activity. Another 37.7% (17) planners used 35 to 44 concepts. Only three planners (6.7%) used 34 or few er planning concepts. The m ost frequently used concept area w as in program delivery (see Table 2, p. 98). R esearch Question 3: Do program planners consider som e planning concepts to be more important than others? If so, w hat are they? Program planners recognized the im portance of using program planning concepts and appeared to be at different sta g e s in the extent to which they w ere used, and believed to be important. C oncepts in developing an administration plan had th e highest m ean score of perceived im portance (3.99). C oncepts in developing an instructional plan had the second highest m ean score (3.98). It ap p ears that the developm ent of a marketing plan w as perceived to be least important, a s it had a m ean sco re of 3.38. 150 R esearch Q uestion 4 : Do program planners consider som e concepts m ore effective than others? If so, what are they? Program planners recognized the effectiveness of using certain program planning concepts. Planners appeared to be at different sta g e s in the extent to which they w ere believed to be effective. Administration had the highest m ean score (3.88). It m ay b e inferred that this particular concept area w as perceived to b e m ost effective. However, program delivery had six of the highest m ean scores, which could indicate that som e of those concepts w ere also perceived to be important. Marketing, with a m ean of 3.15, w as perceived to be least effective (see Table 8, p. 114). R esearch Question 5 : Do associations exist betw een using planning concepts and planners’ satisfaction with completing stated objectives and actual placem ent outcom es? U se of m ore program planning concepts by the planners increased their selfreported goal-completion rates. Planners using 48 or more concepts had an a v erag e placem ent ratio of 104.94%, w h ereas planners using 44 or fewer concepts had an av erag e placem ent of 72.42%. For th o se 25 planners using 46 or m ore concepts, their average placem ent rate, or program outcom e, w as 95.15%. Their average satisfaction score w as 1.76 (1 = very satisfied; 5 = very dissatisfied). For th o se 12 planners using 41 to 44 concepts, their average placem ent rate w as 70.08%. Their average satisfaction score w as 1.83, or lower than the group using m ore concepts. The eight planners using 40 or few er concepts had an average placem ent score of 76.05%, and had the highest a v erag e satisfaction scores (see T ables 10 through 14. pp. 124-134). 151 R esearch Question 6 : What other mitigating variables may influence the outcom es of th e various training activities? Planning m ay be a social practice and not always a scientific one. The interviews and focus group m eetings tended to reveal that training or retraining program s for adults em erged from the personal and organizational interests of the people involved in the planning. (The focus group m eetings identified "commitment to and responsibility for the training process by the com pany involved" and "modifying the program plan" a s the two m ost important mitigating variables that affected outcom es--see Figures 1 through 12, pp. 138-144.) Conclusions N um erous conceptual insights cam e to light a s a result of this study of program planning concepts. This study led the research er to draw the following major conclusions: 1. AEAT program Planners who used a structured and system atic program planning format were more satisfied with program outcom es and experienced a higher degree of participant placem ent in jo b s. D iscussion. Using program planning concepts provides a basic opportunity to involve planners in a collaborative way and to se t the tone for the nature of the training activity. The greater and m ore effective u se of program planning concepts a p p ea rs to increase the likelihood for program su c c e ss and to increase the planners’ satisfaction with the program. The data show ed that planners using the m ost planning concepts had the highest placem ent rates. 152 It m ay also be argued that so m any relationships exist betw een program planning, satisfaction rates and outcom es that it is impossible to draw a truly definitive conclusion. However, it ap p ears that general program planning concepts can b e identified that are likely to contribute to th e program ’s effect and the planner’s overall satisfaction. 2. The planning context is only th e first step in successful AEAT program s; political and practical considerations play a key role in actual o utcom es. D iscussion. Several program planning authors have urged planners to analyze the planning context a s a first step in constructing program s (Boyle, 1981; Buskey, 1987; Caffarella, 1988; Knox, 1979). Although this is clearly one role that planners play, it sells the complexity of their practice far short. Planning a program is not simply a m atter of implementing a theoretical planning framework. It is also a social activity in which planners, learners, em ployers, trainers, and other stakeholders se ek to construct a program together. Som e vision m ust guide the planners in accomplishing this task. The vision that se e m s to em erge is the need to u se substantial dem ocratic planning, w here all stakeholders should be involved in th e decisions of what is important about the program. This association gives su b sta n c e to a program that is responsive, organized, and meaningful. Planners may need to sacrifice som e of the principles that underlie classical planning m odels and respond to situations that arise in their everyday environm ents. Program planners need th e technical, political, and ethical skills and knowledge to further en su re the completion of stated objectives. It is incum bent upon planners to 153 develop an ongoing aw are n ess that their program s m ay have been created in political situations, open to modification a s a result of political decisions, and dep en d en t on a positive political climate (Cervero & Wilson, 1994). An important factor is that program planning m erges both th e negotiation and networking knowledge to work with the process required for completion of goals and the skills n eed ed to nurture a dem ocratic planning process. In general, program planners ap p ea r to implement program planning concepts, and recognize their relative im portance and effectiveness in meeting objectives. However, program planners r : cognize their precarious role in planning AEAT activities and tend to plan in the context of interests, influence, and negotiation, rather than applying researchbased principles. Program planners also tend to organize custom ized planning concepts to overcom e other perceived problems, such a s institutional concerns, resource limitations, th e business climate of th e com pany trained, and working relationships with local and state agencies. In short, planning responsibly m ean s planning politically. 3. The proactive commitment ofthe stakeholder’s leadership and support is a requirem ent to develop a meaningful training or retraining program . D iscussion. According to the focus groups, interviews, and surveys, the stakeholder’s commitment, responsibility, and appropriate follow-through are absolutely critical to th e program ’s outcom es. This one elem ent is a required asp ect of program su c ce ss, regardless o fth e num ber of planning concepts used. When AEAT program s are planned with a busin ess or industry, th e people in those 154 supervisory roles have interests about which planners m ust negotiate in constructing th e program . T hese individuals have expressed and real interests that influence their exercise of power in the planning process and program outcom es. The central form of action for the program planner to follow is to be able to negotiate those interests. The planner m ust m ake practical judgm ents in eac h of th e se situations w here th e com pany’s leadership and support hinges on personal values, environm ental constraints, available resource alternatives, and other factors (Pennington & Green, 1976, p. 22). Part of the planner’s skill in developing successful program outcom es is knowing h ow to read leadership commitment and using it to build a better planning process. Along with this, planners m ust have the knowledge and skills n ecessary to tak e calculated risks that are politically astute and technically sound. This politically s a w y knowledge used by program planners may not alw ays be an explicit part of their planning repertoire, yet it certainly must be cultivated and included in the program planning framework. 4. AEAT program planners em phasized the need for several effective and im portant concepts at both the organizational and activity levels of program m ing. D iscussion. Program planners recognized and im plem ented concepts that underscored the im portance of prepared instructors who involved participants in the learning process, relevant objectives and program priorities that w ere b a sed on com pany need s, and learner a ssista n ce and support used in meaningful fashions. B ecau se resources are always central considerations in program planning and 155 delivery, planners would be well-advised to incorporate th e se planning concepts into every program plan. T he program planning literature te n d s to be a highly prescriptive planning framework, with little attention being given to how program developm ent actually occurs in real-world settings. Consequently, few alternative m ean s of completing various planning tasks have been recom m ended to adult planners. However, in this study, AEAT reflective practitioners have custom ized and advanced planning concepts that work in this particular context. U se of this updated model may en h an ce AEAT program outcom e su c ce ss. Further examination of th e se planning concepts is recom m ended. 5. AEAT program planners, although not required to follow prescribed planning directions by state officials, adopted or revised planning principles advanced by Bovle (19811. Buskev (1987T Cervero a nd Wilson f1994). and oth ers. D iscussion. Effective practice is b a sed on being able to fully understand o n e’s own planning framework, knowing how to a s s e s s it, and being able to change it w hen necessary. This study indicated that effective practice will not be attained by blindly following another planner’s framework. Individual beliefs about learning and training are too diverse and institutional issu es too complex to permit any single fram ework to be universally appropriate. However, it a p p ea rs that textbook fram ew orks can be useful a s long a s program planners are aw are of their limitations. In addition, a theoretical framework provides a way to influence the planning p ro cess at the individual, departm ental, or institutional level. By using the 156 conceptual tools of good planning, making better decisions in a specific context m ay be improved. For those connected with the reality of routinely planning program s, th e se a re important considerations that can be achieved w hen the planner decides to develop and u se a system atic plan. This m ay m ean risking change in a previously established w ay of delivering instruction. It may m ean that the negotiation of personal and organizational interests could be a central activity in the planning practice. Planning is done by real people in real organizations that have institutional m em ories, relationships of power, and hum an n e ed s and interests. O nce again, all of the institutions studied se em e d to u se som e framework of program planning (more than half of the respondents used 45 or more concepts). Still, th e knowledge u sed in adopting th o se principles m ay not be derived from the literature, w here the prescriptive planning fram eworks can be found, but from the p lanners’ repertoire of exam ples, im ages, experiences, and observations. The practical planning recom m endations advanced by the planners also su g g est that planning professionals develop their own theory of practice under real-life conditions. It may follow that the literature's prescriptive fram eworks are useful to the extentthey help practitioners becom e practical theorists. 6. AEAT program planners are effective practitioners. D iscussion. A portion o fth e literature on program planning implied that, by using a particular planning framework, one can becom e a com petent and effective planner. T h ese fram eworks are found in a variety of graduate program s and in other forms of professional preparation, such a s conferences, sem inars, and inservices. 157 However, the findings in this study suggested that AEAT planners w ere successful in their own right, without following a theoretical framework to the letter. This study supports the contention that successful planners actually develop a pragm atic framework that works for them in a particular training context. The vast majority of the planners w ere able to work with others, develop trust, locate resistance and support, be sensitive to timing, know the ropes, and still train nearly 37,000 Michigan residents. Michigan governm ental officials associated with the AEAT program can take great pride and satisfaction in knowing that planners are making the upgrading of em p lo y ees’ skills a priority within their institutions. Program planners are playing a central role in the sta te ’s econom ic developm ent strategy. R ecom m endations for Future R esearch As indicated in C hapter I, the purpose of this study w as to determ ine w hether AEAT training providers implemented and used a system atic plan for developing training or retraining program s, and its relationship to planner satisfaction and m eeting program objectives. The nonlinear nature of program planning raises many issu es that require additional research: 1. Further study is needed into the nature o fth e collaboration betw een stakeholders involved in AEAT activities. Collaboration betw een training providers, their clients, and the appropriate local and state agencies, from planning sta g es through program evaluation, is essential in order to maintain course relevance and quality (Kantor, 1994, p. 104). Additional study could explore this relationship. 158 2. The effect of context, ethos, politics, and culture on program planning. By a sse ssin g the cultural climate of an organization, training providers can gather essential baseline information about an organization and h ence provide a guide for m ore skills-specific a sse ssm e n t, curriculum developm ent, delivery, and evaluation. Educational program s may be largely determ ined by structural forces, namely, the dom inant ideologies and interests of the "social, cultural, and political institution" (Cervero & Wilson, 1994, p. 27). The issu e s need exploration. 3. The relationship betw een a training/retraining provider’s proposed program and its actual capacity to deliver th e established objectives. Further study m ay su g g est that "training providers can ’t be all things to all people." A study of this nature m ay help identify rea so n s why a program had poor outcom es. 4. The relationship of financial resources to the nature o fth e training or retraining program. The im portance of reso u rces to implement successful program planning concepts cannot be discounted. All AEAT providers w ere funded for the proposed training projects. Several respondents raised the concern about being funded to the extent they requested. A future study n eed s to explore w hether and to w hat extent the financial support m akes a difference. 5. To ensure that job-training program s are of high quality, relevant perform ance standards and accountability m easu res need to be investigated-and established. A follow-up study for program participants and their em ployers is recom m ended. The study should determ ine what happened a s a result of the training/retraining interventions. 159 6. The im portance of up-to-date information on local econom ic n e ed s a s well a s national and global trends is recom m ended. Training and retraining program s m ust be developed, promoted, and evaluated on their ability to m eet client n eed s. P o ssessin g a clear knowledge of labor market n e ed s is critical to ensuring that w orkers can receive training in industries that will be strong in the future. Further research is suggested to determ ine the extent to which training providers collect and u se that information. 7. The difference betw een recruiting program participants directly from industry and recruiting anyone from the local community m ay be significant. Exploration of one group’s job readiness and motivation skills could expand our knowledge in this area. 8. The job training system in Michigan’s future m ust ensure that the skills workers acquire through training are portable from program to program. Further research into avenues for planner collaboration and professional developm ent activities is recom m ended. Summary Studying the AEAT program planning process provides interesting insight into the nature of how institutions approach program developm ent. The program planning process a p p ea rs to incorporate several research findings that help to contribute to the completion of stated program objectives. Although program planning offers no magic solution for training or retraining providers, it a p p ears to begin a p ro cess that provides for an improvement in program outcom es. It d o es this 160 in several ways: by changing the nature o fth e program planner to becom e m ore system atic, by focusing on a model for an improved planning process, by merging staff developm ent resources on the need for program planning, by encouraging m ore collaboration and involvement am ong and betw een program planners and program stakeholders, and by providing a basis for connecting various program planning com ponents into a single, more unified one. Of equal importance, th e context always m atters a s attem pts to plan are m ade. A truly practical theory ap p ears to offer planners not only a set of practical procedures, but also a way to understand the organizational contexts in which the p rocedures are to be carried out. judgem ents: In short, planners m ust m ake practical "a judgem ent of what to do, or what is to be done, a judgem ent respecting the future termination of an incomplete and so far indeterm inate situation" (Dewey, 1915, p. 514). T h reats to and opportunities in Am erica’s econom ic developm ent are coming from m ore than one direction: technology change, work force dem ographics, and global competition that requires training and retraining (Kantor, 1994, p. 84). American adult program providers m ust focus on preparing our citizens for a global m arketplace. Planning is future making, which is why it m atters and why planners should c are about doing it better. Reflections Reflecting on the results of the study, I am struck with several im pressions that extend beyond th e findings and conclusions, and need to be shared with the 161 reader. First, the question of why an educational institution, particularly the community college, would wish to procure AEAT funding n e ed s to be addressed. O ne obvious and com m on resp o n se is the self-serving one of increased enrollments and revenues. However, I am convinced that there are d e e p e r motivations that stem from the core beliefs o fth e program planners on behalf of the institutions they serve so well. The desire to advance and better the community through meaningful collaboration with busin ess and industry, area service agencies, and state departm ents is one exam ple of that stronger motivation. Although enrollment and revenue issu e s m ay continue to drive decisions to participate in community econom ic developm ent activities, the S tate of Michigan can be encouraged that workforce training for local business and industry is a major mission of the community college and a logical extension of its traditional career preparation function. Second, the study reaffirmed my strong belief that educating and training a world-class workforce is key to Michigan’s econom ic growth and ability to com pete in a global m arketplace. Michigan’s AEAT providers are well-positioned to provide quality, cost-effective workforce training and retraining. Still, it is incum bent upon program planners and leaders to apply the political and practical a sp e c ts of planning. They need to continue to be proactive in working with federal, state, and local governm ent to initiate policies and incentives for b u sin e sse s to invest in workforce training and en su re appropriate funding/resources for the training institutions. 162 In addition, those sam e planners and leaders need to work even more closely with b u sin ess and industry m anagem ent and em ployees to a s s e s s training needs. This m ea n s that the traditional instructional system s may need to b e abandoned and replaced with a custom ized instructional design and delivery that works. Once again, the m ost effective and important planning concept cited by the practitioners w as "making training relevant." It is clear that those training or retraining activities that are taught at the shop-floor level, using more applied technology techniques, will help en sure program su ccess. Third, program planning is not a p anacea; it is not a miracle cure for ensuring that all program objectives are met. Planning requires significant time and effort. It requires a practical melding of process, personalities, and implementation that a p p e a rs to be missing in institutional professional developm ent program s-including Michigan S tate University. At a time when b usiness and industry are d esp erate for relevant and practical training that will maintain com pany com petitiveness and profitability, Michigan’s adult program planners require similar "just-in-time" upgrading and professional developm ent. W hereas planners currently look to each other for improving their skills a s practitioners, and affiliate with professional associations, such a s the Michigan Custom ized Trainers and the American Society for Training and Developm ent, they need additional training sources, including the sta te ’s universities. Michigan S tate University, particularly the College of Education, n e ed s 163 to take a hard look at its course offerings and a s s e s s its professional relationships with adult program planners. T h ese reflections would be incom plete without making som e observations about the professionals involved in the study. Having engaged my colleagues in various interactions, I remain im pressed about their passion and enthusiasm for what they do. Through the interviews and focus groups, the program planners seem genuinely committed to making a positive effect on their communities. Each planner had a story to tell that allowed for the diversity of on e’s institution, yet sh ared a collective vision of service, professionalism , and devotion to others. I have a renew ed s e n s e of pride for my colleagues and friends who are "future making." In reviewing the dissertation process, I am pleased with what h a s been accom plished. I have increased my understanding ofthe characteristics and effects of planning and the outcom es that result. Best of all, I believe I have discovered practical information that can be useful to other program planners. APPENDICES APPENDIX A LETTER OF APPROVAL FROM THE UNIVERSITY COMMITTEE ON RESEARCH INVOLVING HUMAN SUBJECTS 164 M ICHIG AN STATE U N I V E R S I T Y March 2, 1995 TO: John G. Zappala Mid MI. Community College 5805 E. Pickard Mt. Pleasant, MI 48858 RE: IRB#: TITLE: REVISION REQUESTED: CATEGORY: APPROVAL DATE: 94-573 AN ANLYSIS OF THE MICHIGAN EDUCATION ALTERNATIVE TRAINING PROGRAM USING SELECTED CONCEPTS OF SUCCESSFUL PROGRAM PLANNING 02/01/95 1 -A .C ,E 01/19/95 The University Committee on P.eEeirch Involving Kunan Subjects’(UCP.IH5) review of this project is complete. I am pleased to advise that the rightB and welfare of the human subjects appear to be adequately protected and methods to obtain informed consent are appropriate. Therefore, the UCRIHS approved this project including any revision listed above. RENEWAL: UCRIHS approval is valid for one calendar year, beginning with the approval date shown above. Investigators planning to continue a project beyond one year must use the green renewal form (enclosed with the original approval letter or when a project is renewed) to seek updated certification. There is a maximum of four such expedited renewals possible. Investigators wishing to continue a project beyond that time need to submit it again ror complete review. REVISIONS: UCRIHS must review any changes in procedures involving human subjects, prior to initiation of the change. If this is done at the time of renewal, please use the green renewal form. To revise an approved protocol at any other time during the year, send your written request to the UCRIHS Chair, requesting revised approval and referencing the project's IRB / and title. Include in your request a description of the change and any revised instruments, consent forms or advertisements that are applicable. PROBLEMS/ CHANGES: O FFIC E or RESEARCH AND GRADUATE STUDIES Should either of the following arise during the course of the work, investigators must notify UCRIHS promptly: (1) problems (unexpected side effects, complaints, etc.) involving human subjects or (2) changes in the research environment or new information indicating greater risk to the human subjects than existed when the protocol was previously reviewed and approved. p l- s a r - s If we Can on any Cuvurc at (517)355-2180 or FAX (517)336-1171. University Committee on Research involving Human Subjects (UCRIHS) gn Suir G i A j.T iriS ’.rjfiCT Bu- C :rg Eas:L3TS.I-5 48=:J-'!0J< 5 5i:.355-:i£D FA.\ ' J 3 2 - 1 * 7* oTvid E. Wright, Ph.D.\ /ll/'BTUC /'Vt • { v / I /UCRIHS Chair DEW:pjm cc: Rodolfo Garcia d r- v APPENDIX B MICHIGAN JO BS TEAM LETTER OF SU PPORT 165 •dS&b «rfL- MICHIGAN JOBS COMMISSION Office of W orkforce D evelopm ent MICHIGAN jfcSSi' JOBS i j j j j j f ; T E A M iS B ? May 24, 1995 O o u g R o th w e ll Chief Executive Officer J o h n Engler. G overnor John Zappala Dean o f Liberal Arts and Continuing Education Mid Michigan Community College 5805 East Pickard Street Mt. Pleasant, Michigan 48858 Dear John: Thank you for inviting the cooperation o f the Michigan Jobs Commission in your study o f the local planning processes used to develop training projects for the 1992-1993 Adult Education Alternative Training Program. This state grant program has been revised each year since then in response to various types o f statewide program information and analysis. However, I am not aware o f other formal studies o f local training project implementation. I hope the report o f your study will be useful to the many different educational institutions who are continuing to improve the effectiveness o f services to Michigan's employers. Therefore, I agree to provide some information and other limited collaboration to assist your study. As we have agreed, I reserve the right to review and comment on any materials related to this study prior to publication. Please feel free to share this description of the Michigan Jobs Commission's role in this study with those who receive your survey. As any additional issues arise on this survey, please feel free to contact Barb Chubb or myself. Sincerely, •fy^John S. Palmer, Jr. W orkforce Development Executive Economic Development Job Training Program 201 N. W ashington S q u a re • Victor Office C en ter • 3rd floor ♦ Lansing, Ml 48913 ♦ Til {517) 373-6508 ♦ Fax (517) 373-6179 M ichigan Relay C en ter V800-649-3777 {Voice and TOO) APPENDIX C MAILING TO PROGRAM PLANNERS 166 MMCC* Mid M ichiganC om m unityC a lle g e 5 B 0 5 E a s t P ic k a r d _____________________________M t. P l e a s a n t . M ic h ig a n 4 8 8 5 8 ____________________T e l e p h o n e 5 1 7 /7 7 3 -M M C C . D ate Name I n s t i t u t i o n D ear As in P ro a r e c o l ___________________, a p ro M ic h i g ram in a le a g u g ra m p la n n e r o f one o f th e f i r s t e d u c a tio n g a n t o d e v e l o p a 107A A d u lt E d u c a t i o n A l t e r f o r P r o i e c t Name ( P r o j e c t N u m b e r ______________) u n iq u e p o s i t i o n to b e a b le to s h a r e y o u r e e s i n t e r e s t e d in p ro g ram im p ro v em en t. a l i n s t i t u t i o n s n a t i v e T r a in in g in 1 9 9 2 -9 3 , you x p e r ie n c e s w ith Y our h e lp in c o m p le tin g t h i s s tu d y w i l l ad d to th e in f o r m a tio n a b o u t c o n c e p ts o f s u c c e s s f u l p ro g ra m p la n n in g . T h is s tu d y s e e k s to d e te rm in e w h e th e r p la n n in g co m p o n en ts a r e p e r c e iv e d to h a v e b een in c lu d e d in th e A d u lt E d u c a tio n A l t e r n a t i v e T r a in in g p ro g ra m , and th e r e l a t i o n s h i p o f th e s e co m p o n en ts, i f any, to c o m p le tin g p ro g ram o b j e c t i v e s . As you an sw er th e q u e s tio n s on t h i s s u rv e y , p le a s e c o n s id e r o n ly th o s e p la n n in g co m p o n en ts t h a t you u t i l i z e d in th e p r o j e c t n o te d a b o v e . NOTE: T h is s tu d y is n o t an e v a l u a t i o n o f th e 107A p ro g ra m . T h is s tu d y in v o lv e s th e s e d a ta g a th e r in g te c h n iq u e s : 1. A s u rv e y o f th e h u n d re d -p lu s p la n n e r s in v o lv e d in a c t i v i t y . 2. S e le c te d on s i t e o r te le p h o n e in te r v ie w s . As D ean o f L i b e r a C om m unity C o lle g e , I h a v e a tte n d e d p r e s e n te d p ro g ra m o f E d u c a tio n in A th e 107A l A rts and C o n tin u in g E d u c a tio n a t M id M ic h ig a n I h a v e e x te n s iv e e x p e r ie n c e in p ro g ra m p la n n in g . s e v e r a l p ro g ram p la n n in g t r a i n i n g s e s s i o n s and p la n n in g in f o r m a tio n fo r th e M ic h ig a n D e p artm e n t u g u st, 1994. The d e t a i l s of t h i s r e s e a r c h s tu d y a r e e x p la in e d in th e sum m ary e n c lo s e d in t h i s l e t t e r . The s tu d y w i l l be u s e d by me in a d o c to r a l d i s s e r t a t i o n a t M ich ig an S t a t e U n iv e r s ity . A ll p ro g ram p la n n e r s who p a r t i c i p a t e d in M ic h ig a n 's 107A a c t i v i t i e s w i l l be c o n ta c te d . I w i l l p ro v id e a l l p a r t i c i p a n t s w ith r e s u l t s o f th e s tu d y . Y our c o o p e r a tio n I f you h a v e 7 7 3 -1 7 5 0 o r in t h i s im p o rta n t r e s e a r c h any q u e s tio n s , f e e l f r e e w ork (517) 7 7 3 -6 6 2 2 . to is c a l l g r e a t l y me S in c e r e ly , Jo h n Z a p p a la a t a p p r e c i a te d . home (517) APPENDIX D 1992-93 MICHIGAN EDUCATION ALTERNATIVE TRAINING PROGRAM PLANNING SURVEY 167 This survey is designed to obtain a structured appraisal of the utilization of program planning components in the 1992-93 Michigan Education Alternative Training activities. The data you provide will be used in a doctoral study at Michigan State University. To preserve confidentiality, your name is not requested, and your views will be consolidated with those of others. Please read the questions carefully, and follow instructions where they are provided. Th*nk You PLEASE AND ANSWER RETURN IN ALL THE QUESTIONS ON SELF-ADDRESSED THE QUESTIO N N A IRE STAMPED ENVELOPE TO: John Zappala Mid Michigan Community College 5805 East Pickard Mt. Pleasant, MI 48858 Preliminary Information 1. Project Number ___________ 2. To what extent were ycu satisfied with the outcome of this particular training activity? _____ Very Satisfied ________ S a t i s f i e d ______ Neutral _____ Dissatisfied _____ Very Dissatisfied Training Provided To____________________ (Company Name) ASSESSING PROGRAM PLANNING IN 107A ACTIVITIES DIRECTIONS: Please answer ea c h q u e s t i o n b y circ l i n g the a p propriate scale response. If y o u w i s h to change an answer, pleaBe m a r k an X ove r the original answer an d circle a n e w response. » Column I, H o w important is this c omponent to an effec t i v e program? * Column II, To w h a t degr e e w e r e y o u effective in u s i n g this component in y o u r program? C O L U M N II COLUMN I 1. NOT I M P O R T A N T 1. DID 2. 3. 4. SOMEWHAT I M P O R T A N T IMPORTANT VERY I M P O RT AN T 2. 3. 4. NOT E F F E C T I V E L Y US E D S OMEWHAT E F F E C T I V E L Y E F F E C T IV E L Y USED 5. ESSENTIAL 5. VERY P ROGRAM PLANNING COMPONENTS A N D ELEMENTS COLUMN I PLANNING: ANALYSIS 1.2 1.3 USED C O L U M N III EFFECT I V E N E S S How i m p o r t a n t was t h i s com ponent to an e ffe c tiv e program ? 1.1 EFFECTIVELY To w h a t d e g r e e w ere you e f f e c tiv e in using th is in y our program ? E stab lish ed p la n n in g /ad v iso ry com m ittee 1 2 3 4 5 1 2 3 4 5 A ll sta k e h o ld e rs w ere re p re s e n te d on th e co m m ittee 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 Id e n tifie d ta rg e t p opulation fo r the program 1.4 Problem a r e a s / needs s tu d ie d and analyzed 1.5 C on sideration provided for was so c ia l/e d u c a tio n a l backgrounds of learn er USED C O L U M N II I M PORTANCE 1. NOT USE COMMENTS (I f you w ish, you m ight i n d i c a t e why you d id n o t u s e a p a r tic u la r com ponent). ASSESSING PROGRAM PLANNING IN 107A ACTIVITIES COLUMN I 1. C O L U M N II 2 . NOT IMPORTANT SOMEWHAT I M P O R T A N T 3. 4. 5. IMPORTANT VERY IMPORTANT ESSENTIAL PROGRAM PLANNING COMPO N E N T S A N D ELEMENTS 2. COLUMN I How im portant was th is component e ffe c tiv e program ? 3. 4. D I D NOT U SE NOT E F F E C T I V E L Y USED SOMEWHAT E F F E C T I V E L Y E F F E C T I V E L Y USED 5. VERY EFFECT I V E N E S S to an 2.3 I d e n t i f i e d p r io r itie s tra in e e s' needs 2.4 Id e n tifie d upon in te re s ts w ish, you d id e ffe c tiv e in u sin g th is in y o u r program ? com ponent). 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 and program p r io r itie s based em ployer needs upon 2.5 I d e n t i f i e d program p r io r itie s based upon p ro v id e r's in s titu tio n 's p re p a re d n e ss/a v a ila ­ b il i t y COMME N T S m ig h t i n d i c a t e why you not use a p a rtic u la r program based USED To w h at d e g r e e w ere you 2.2 S e l e c t e d su b je c t areas re la te d to id e n tifie d perform ance d e f ic its USED C O L U M N III 2.1 I d e n t i f i e d s p e c ific job perform ance d e f ic its to be addressed EFFECTIVELY C O L U M N II IMPORTANCE PLANNING: JUSTIFICATION 1. 2. (If you ASSESSING PROGRAM PLANNING IN 107A ACTIVITIES COLUMN I 1. 5. 1. NOT IMPORTANT SOMEWHAT I M P O R T A N T IMPORTANT 2. 3. VERY IMPORTANT ESSENTIAL 4. 5. 2. 3 . 4. C O L U M N II PROGRAM PLANNING COMPO NENTS AN D ELEMENTS Column I How to an program ? 3.1 3.2 3.3 3.4 VERY USED C o l u m n III COMMENTS To w h a t d e g r e e w ere you you you e ffe c tiv e in u sin g th is in your program ? p a r tic u la r S elected o b je c tiv e s th a t im plied changes in b e h av io r of in d iv id u a ls 1 2 3 4 5 1 2 3 4 5 O b je c tiv e s w ere e s ta b lis h e d p r i o r to program im plem entation 1 2 3 4 5 1 2 3 4 5 O b je c tiv e s w ere r e la te d th e com pany' s j ob perform ance d e f ic its to 1 2 3 4 5 1 2 3 4 5 O b je c tiv e s w ere r e la te d the in te re s ts/n e e d s of to 1 2 3 4 5 1 2 3 4 5 tra in e e s USED USED EFFECTIVELY EF F E C T I V E N E S S im portant was t h i s component e ffe c tiv e EFFECTIVELY C o l u m n II IMPORTANCE 3. OBJECTIVES D I D NOT U SE NOT E F FE C T IV E L Y USED S O MEWHAT E F F E C T I V E L Y (If you m ight in d ic a te d id n ot use a w ish, why com ponent). ASSESSING PROGRAM PLANNING IN 107A ACTIVITIES COLUMN I C O L U M N II 1. NOT 2. S OMEWHAT I M P O R T A N T IMPORTANT VERY IMPORTANT ESSENTIAL 3. 4 . 5. PROGRAM PLANNING COMPONENTS A N D ELEMENTS 4. 1. IMPORTANT 3. IMPORTANCE plan of in EFFECTI V E N E S S e f f e c tiv e in u sin g th is in your program ? 4.3 D e t e r m i n e d s e le c tin g 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 charge program c r ite r ia fo r in s tru c tio n a l s ta f f 4.4 S e l e c t e d s ta f f in s tru c tio n a l w ith docum ented e x p e rtise 4.5 A r r a n g e d in 4.6 C le arly ro le s area ap p ro p ria te in s titu tio n a l serv ice s | content f a c i l i t i e s defin ed and and a d m in istra tiv e re s p o n s ib ilitie s 4.7 D e v e l o p e d a ll a budget covering a n tic ip a te d resource c o sts USED USED C O L U M N III was developed 4.2 A d m i n i s t r a t o r w a s EFFECTIVELY To w h at d e g re e w ere you an e ffe c tiv e program ? 4.1 A n a d m i n i s t r a t i v e VERY C O L U M N II How i m p o r t a n t was t h i s com ponent to USED SOMEWHAT E F F E C T I V E L Y E F F E C T IV E L Y USED 4. 5. COLUMN I AD M INISTRATION D I D NOT U S E NOT E F F E C T I V E L Y 2. COMMENTS (If you you m ight in d ic a te you d id n ot use a p a r tic u la r com ponent). w ish, why ASSESSING PROGRAM PLANNING IN 107A ACTIVITIES COLUMN I C O L U M N II 3. 4 . 5. PROGRAM PLANNING COMPONENTS A N D ELEMENTS 5. 1. 2. 3 . 4. 5. NOT IMPORTANT SOMEWHAT I M P O R T A N T IMPORTANT VERY IMPORTANT ESSENTIAL 1 . 2 . COLU M N I How To D eveloped in s tru c tio n a l plan 5.2 D eterm ined m ethods o f se n ta tio n ap p ro p ria te job 5.3 5.4 perform ance p re ­ to perform ance D eveloped 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 o b je c tiv e s S elected in s tru c tio n a l m a te ria ls a p p ro p ria te to job degree w ere you e f f e c tiv e in u sin g th is in y o u r program ? to an e ffe c tiv e program ? 5.1 what o b je c tiv es p re -tra in in g a c tiv itie s (e g ., o rie n ta tio n , m aterials) h e lp in g le a rn e r to a n tic ip a te ideas and concepts p resented 5.5 to be U t i l i z e d m ethods of in te ra c tiv e le arn in g in s tru c tio n a l in a c tiv itie s USED C O L U M N III EFFECT I V E N E S S im portant was t h i s component E F F E C T IV E L Y USED VERY E F F E C T I V E L Y C O L U M N II IMPORTANCE INSTRUCTION D ID NOT USE NOT E F F E C T I V E L Y USED S OMEWHAT E F F E C T I V E L Y U S E D COMMENTS (If you w ish, you m ig h t i n d i c a t e why you d id not use a p a rtic u la r com ponent). ASSESSING PROGRAM PLANNING IN 107A ACTIVITIES COLUMN I C O L U MN I I 1. 1 NOT I M P O R TA N T SOMEWHAT I M P O R T A N T IMPORTANT VERY IMPORTANT 2. 3. 4. 5. 6. C O L U MN I 6.1 D eveloped 6.2 C om m unicated m arketing value plan of 6.5 M arketing e f f o r t and in c lu s iv e of Inform ed R ecruited O riented program program ? your program ? 3 4 5 1 2 3 4 5 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 2 2 was organized to a ll p ro sp ectiv e of 1 program p a rtic ip a n ts p a rtic ip a n ts 2 C O MME N T S you you for to (If you m ight in d ic a te d id not use a p a r tic u la r program program 6.8 degree program p a rtic ip a n ts 6.7 what w ere you e f f e c t i v e in u sin g t h is in 1 O riented a d m in istra tiv e and in s tru c tio n a l s ta f f asp ects 6.6 tra in in g To program The power s t r u c t u r e w ith in p a r t i c i p a t i n g em ployer supported C O L U MN I I I EFFECTIVENESS im portant was t h i s component to an e f f e c tiv e 1 to a p p r o p r ia te em ployer d e c is io n m akers 6.4 C O L U MN I I IMPORTANCE MARKETING How 6.3 4. 5. ESSENTIAL PROGRAM P L A N N I N G C O M P O N E N T S AND E L EM EN TS D I D NOT USE NOT E F F E C T I V E L Y USE D SOMEWHAT E F F E C T I V E L Y U S E D E F F E C T IV E L Y USED VERY E F F E C T I V E L Y U S E D . 2. 3 . w ish, why com ponent). ASSESSING PROGRAM PLANNING IN 107A ACTIVITIES C O L U MN I C O L U MN I I 1 . 2. NOT IMPORTANT SOMEWHAT I M P O R T A N T 2. D I D NOT USE NOT E F F E C T I V E L Y 3 . 4. 5. IMPORTANT VERY IMPORTANT 3. 4. SOMEWHAT E F F E C T I V E L Y E F F E C T IV E L Y USED ESSENTIAL 5. VERY 1. EFFECTIVELY USED USED USED P R O G RA M P L A N N I N G C O M P O N E N T S AND E L EM E N T S 7. C O LUMN I EVALUATION C O L U MN I I IMPORTANCE How i m p o r t a n t EFFECTIVENESS To w hat d e g re e w ere you was t h i s component to an e ffe c tiv e program ? 7.1 D eveloped 7.2 D eveloped e v a lu a tio n in stru m e n ts to m easure e v alu atio n plan program C O L U MN I I I e f f e c tiv e in u sin g th is in your program ? 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 e ffe c tiv e n e ss 7.3 A dm inistered e v a lu a tiv e in stru m en ts based upon e s ta b ­ lish e d 7.4 7.5 c r ite r ia O bserved p a r tic ip a n ts d u rin g in s tru c tio n a l a c tiv itie s E valuated m ethods 7.6 A nalyzed of 7.7 o b je c tiv e s and c o n sta n tly and rep o rted re s u lts ev alu atio n D eveloped p o s t-tra in in g a p p ro p ria te a c tiv itie s follow -up and C O MME N T S you m ight ( I f you w ish, i n d i c a t e why you d id n o t use a p a r tic u la r com ponent). ASSESSING PROGRAM PLANNING IN 107A ACTIVITIES SECTION I I This section examines the Program Planner's opinions regarding the delivery of the actual program. Assess each criteria based on the following: * Column I, How important is this component to an effective program? * Column II, To what degree were you effective in using this component in your program? COLUMN I COLUMN II 1 . NOT 1. DID 2. 3 . 4 . SOMEWHAT I M P O R T A N T IMPORTANT VERY IMPORTANT 2. 3. 4. NOT E F F E C T I V E L Y U S E D SOMEWHAT E F F E C T I V E L Y U S E D E F FE C T IV E L Y USED 5. ESSENTIAL 5. VERY IMPORTANT PROGRAM PLANNING COMPONENTS AND ELEMENTS 8. 8.1 Program p lanner 8.3 A ttendance was of m onitored Support c la sse s w ere o ffered to students help EFFECTIVENESS How i m p o r t a n t was t h i s com ponent to an To w hat d e g re e w ere you e f f e c tiv e in e ffe c tiv e program ? using th is in your program ? 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 a lso learn 8.4 P a r tic ip a n ts w ere vided o p p o rtu n ity p ro ­ to ask q u estio n s c l a r i ­ fic a tio n sessio n 8.5 or in In s tru c to rs for cla sses seek tra in in g w ere prepared USED COLUMN III p a r t i c i ­ d e liv e ry EFFECTIVELY COLUMN II IMPORTANCE PROGRAM DELIVERY p ated in program 8.2 COLUMN I NOT USE COMMENTS (If you w ish, m ig h t i n d i c a t e why yo u not use a p a rtic u la r com ponent). you d id ASSESSING PROGRAM PLANNING IN 107A ACTIVITIES COLUMN I 1 2 3 4 . . . . 5. C O L U M N II SOMEWHAT I M P O R T A N T IMPORTANT VERY IMPORTANT 1. 2. 3 . 4. ESSENTIAL 5. NOT IMPORTANT PROGRAM PLANNING COMPONENTS A N D ELEMENTS e. 8.6 8.7 8.8 8.9 COLUMN I C O L U M N II C O L U M N III IMPORTANCE EFFECTIVENESS COMMENTS (If How i m p o r t a n t was t h i s com ponent to an To w h at d e g re e w ere you e f f e c tiv e in you in d ic a te e ffe c tiv e program ? u sin g t h i s in y our program ? PROGRAM DELIVERY P a rtic ip a n ts concepts and D I D NOT U SE NOT E F F E C T I V E L Y USED S OMEWHAT E F F E C T I V E L Y U S E D E F FE C T IV E L Y USED VERY E F F E C T I V E L Y US E D applied s k ills P a rtic ip a n ts could apply s k i l l s on r e a l w ork m a terial 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 In s tru c to rs p a rtic ip a n t provided fo r understanding 1 2 3 4 5 1 2 3 4 5 A placem ent im plem ented com ponent was as p a rt of 1 2 3 4 5 1 2 3 4 5 the tra in in g m ight d id not use com ponent). a you w ish, why you p a rtic u la r program Thank you for completing this survey. your earliest convenience. Please us e the self-addressed, stamped envelope and return at APPENDIX E ASSESSING PROGRAM PLANNING IN AEAT ACTIVITIES COMMENTS 177 A S S E S S IN G PROGRAM PLANNING IN 107A ACTIVITIES COMMENTS (COLUMN III) Component 1 1.1 S om e com panies were more open to planning than others. W astes valuable time; encourages com prom ises; kills leadership. Training n e ed s w ere well established by com pany prior to making grant application. Time! Very difficult to maintain stability and consistency of group m em bership with small m anufacturers. Used local Printers Guild a s planning com m ittee-did not have to reinvent the wheel. Com mittee w as preestablished by company. Most com panies committed to the formation of a steering com m ittee to initiate com pany-specific training. Several small com panies involved only m anagers in planning. Planning based on successful program at T exas S tate Technical CollegeW aco. Rushing for time frame on grant and com pany time schedule. 1.2 Especially important in unionized settings to have all stakeholders represented. T here w ere no dentists on the comm ittee in person, but the Jo b Placem ent Supervisor and instructor gathered input from a num ber of professionals. Very small num ber of trainees/courses. Em ployees should be represented. Key stakeholders trained at Waco. 178 1.3 Participants identified by company. Grant creates delays in the delivery of program objectives. 1.4 Accomplished daily by those responsible for determining needs. Absolute support of Dean w as essential. 1.5 W e had to have people who could read and write. delivered training at all skill levels. Coordinated effort Built into W aco design. Program cannot be used without absolute and unconditional commitment to the learner. Component 2 2.1 Overall rise in self-esteem and, therefore, ability to deal with deficits w as dramatic. Important, however, often basic skills deficiencies becom e the focus of training. Deficits might not be a s accurate a word a s "improvements." Unemployed. 2.2 Plan focused on math, physics, comm unications, and com puters-all "applied." Dislike the term "deficit" since training is concentrated on expansion of skills rather than a lack of skill in present job. Again, a lot of time is spent bringing people up to sp e ed in basic skills areas. Unemployed. 2.3 N eeds to em erge once c la sse s begin and trust level is established. Program design took generic problem s into account. S tudents cam e to a planned program. Unable to totally rely on interests a s personal interests might have nothing to do with work requirem ents. "Needs" stressed over "interests." 179 2.4 W aco design jointly done with Head of T exas JTPA and Cord. Validated by su b seq u en t College DACUM studies. 2.5 Too hard to match grant "schedule" and be flexible for com pany needs. If the college, particularly the Dean, had not been prepared, the program would have failed. Brought in appropriate instructional team w hether they w ere college faculty or not. As long a s you can pull together the resources, you do not have to have them already. This is an a re a our college has: faculty, experience, educational tools. Did not approach from this angle. The priority is with th e com pany need. If w e could not provide, w e found another local college or training provider who could. W hen not available or prepared, then w e develop. Component 3 3.1 The corporation desired specific m aterials to be covered for each individual. S hared-m anagem ent work team s. W e saw m assive changes. Taught som e TQM principles to two of th e four com panies only. Hard to m easu re behavior changes. Too short and too few in this attempt. 3.2 O thers established over the year with em ployee input. Although, m ust be open to changing or revising during if necessary. 3.3 Visits to com pany to observe com pany’s n e ed s . . . and implement. Employers are seeking to enlarge em ployee work responsibilities, not correct shortcom ings in individual job tasks. 180 3.4 N eeds first, interests after "needs" met. Extremely important. Did a hands-on interview. Modifications took place b ased on n e e d s of students, but key objectives w ere se t early in planning and m aintained. Component 4 4.1 Important for grant but too abstract to plan for unknown. Hard to maintain plan w hen sta te ch an g es rules constantly. Plan grew out of previous system ized projects. A structure w a s in place along with others—faculty, company. I’m not sure w hat you m ean by "administrative plan." B usiness a s usual. 4.2 Committed to program. Trained in program. Supervisor directly involved under direction of administration. 4.3 Experience in teaching adults m ore important than personality. All W aco trained. All certified and qualified personnel utilized. 4.4 None 4.5 Had to go off-site for awhile--not good. Improvised som e equipment. Bought PT lab for second round. In-plant instruction. Used com pany facility. 4.6 B usiness a s usual. This w as difficult seeing a s how administration w as not funded for this grant! 4.7 Only applied for $8,000. 181 Component 5 5.1 ICC = individualize com pressed curriculums w ere developed C hanges m ade a s needed. U nforeseen changes. W aco plan u se d . 5.3 Had to have state objectives. Should have adjusted materials in resp o n se to trainee feedback. CORD and AIT applied materials used a s designed to be used. 5.4 Key rule of W aco training for staff. Word of mouth still best so may start small, but if it’s good and schedule and product can be worked o u t-th ey will come. 5.5 Little experience in th ese projects to support answ er. Pilot for Applied Learning her at college. Now we are #5. Component 6 6.1 W e achieved racial and sexual equal opportunity. Did not u se for g ran t-d id n ’t need to m arket program. Com pany ID’d trainees. Employers continuously cam e to us. No n e ed —one com pany training not optional. More work needed to be done in this area. Not n e cessary for grant. 6.2 Fed into JTPA u se of program in ’94 and '95. Com pany knew it w as very important. C om pany decided we provided-w e did not sell. Had to work 12 w eeks after program. Employers believed their em ployees could learn all within a 4-5 w eek prep, time. 6.3 They have to "walk their talk." Fed into JTPA use of program in ’94 and ’95. I don’t understand this statem ent. Non-unionized environment, which ad d s another dimension. Varied at locations. 6.4 W e achieved racial and sexual equal opportunity. Did not use. Internal marketing to em ployees w as done. To the extent possible. I don’t know what you m ean by "inclusive." 6.5 Which staff—em ployee or trainer? Plant visits w ere scheduled and required-useful for planning, etc. 6.6 Som e com panies were more aggressive with this activity than others. This w as handled by employers. Designed for select existing em ployers. Done by company. A ssessm ents, etc.-individual sessio n s. 6.7 Word of mouth still best, so may start small, but if it’s good and schedule and product can be worked out--they will com e. This w as also done by em ployers. D esigned for select existing em ployers. Within Mercury. Not n e ce ssa ry —they w ere assigned. Done by company. Com pany assigned. Recruited corporations. 6.8 D esigned for select existing em ployers. Done by company. First class session, m ost often. Very essential and n ecessary for som e participants. Component 7 7.1 Required by state. State-m andated. I left institution prior to completion of second part. Do not know w hether evaluation w as done-left project before it w as com pleted. 7.2 More to be done! Required by state. Pre and Post testing. (?) Working with new equipm ent w as the evaluation, (production) U sed TABE and later developed. Short a ssessm en t. 184 7.3 Gaining m easurable perform ance benchm arks from em ployers requires considerable coordination. 7.4 Very com pany-specific training with foreign vendors providing training. W ho observed? Instructors/yes. Administrator/no. Only instructor did. 7.5 After every class day, w e met a s an instructional group to discuss the day’s sessio n s. Half-way point of each class; at end. Not constantly. Periodic follow-up by staff with trainers and com pany representatives. 7.6 Reported a com posite, not individual results. For all sections. Report to whom? For som e, not all, a s I had left institution. R eported yes. Analyzed/no. T here w as not enough time. 7.7 O pen-door/open entry upon grant completion for further and brush up. Required for 12 weeks. Most com panies resist. C om pany may have done this. For som e. Person instrumental in program left company. Employers did this. Not in program design for us. Em ployer-based. Post-training follow-up represented an excellent opportunity for dialogue with com pany representatives. 185 Component 8 8.1 If you m ean administrator or planning com m ittee-no. If you m ean instructor who planned program--yes. Faculty involved in planning. Not used. No. 8.2 This is a requirem ent of the grant. M andated or we wouldn’t have. 8.3 Not applicable to us. Yes, if size warrants. Program w as support. Follow-up used support cla sses. Prior a sse ss m e n t w as done to insure students had appropriate prerequisite skills. Basic math and reading. Not structured classes. O ne-on-one tutoring w as offered—additional supporting m aterials available. Participants were hand-picked by company. Support c la sse s not needed. Offered tutoring. This w as a "support" class. All education opportunities in area presented to all participants. N eeded b e c a u se of welfare situation. Literacy council. 8.4 Built-in. 8.5 Training. A team of pros. 186 8.6 Design. A m ust in workforce training. 8.7 Now the whole curriculum is work driven. A m ust in workforce training. Job-related training. 8.8 None. 8.9 Used College Adm issions & Placem ent. Training limited to th o se already employed. Not all a sp ec ts of program s required assessm en t. Not applicable. In this project, all trainees w ere already employed. Existing em ployees. All w ere currently employed. Not applicable to project. Em ployees at this com pany very stable—no force reduction—this w asn ’t needed. Not applicable. Probably, but I’m not sure I understand the question. Required. Achieved 100%. Already employed. APPENDIX F INDIVIDUAL INTERVIEW CONSENT FORM 187 Individual Interview Consent Form Program Planning Study T o w hom i t m ay c o n c e r n : I _______________________________________________________________ c o n s e n t t o t h e r e s e a r c h f o r A n A n a l y s i s o f t h e M i c h i g a n T r a i n i n g I P r o g r a m u n d e r s t a n d r e p o r t , f r o m w i t h f o l l o w i n g t h e r e p o r t s U s i n g t h a t r u l e s r e s e a r c h e r o f r e s e a r c h s p e c i f i c S e l e c t e d t h e D a te o f w i l l s t r i c t e s t k n o w f i n d i n g s r e s p o n s e s C o n c e p t s i n f o r m a t i o n o r I n o t JAM 1: a n d m u st b e r e n e w e d within n m o n t h s tc c o n tin u e . m a y i d e n t i t y p e r m i t P l a n n i n g b e o f t h e u s e d N o s t u d y . i n o n e s u b j e c t a s s o c i a t i n g t h e a s i d e a n d s u b j e c t s f i n d i n g s . S i g n a t u r e UCF.iHS A PPR O VAL FOR THIS project EXPIRES: P r o g r a m s h a r e c o n f i d e n t i a l i t y . t h e w i l l o f b e i n g i n t e r v i e w e d f o r E d u c a t i o n A l t e r n a t i v e o f p e r s o n i n t e r v i e w e d APPENDIX G INTERVIEW QUESTIONS 188 IN T E R V IE W Q U ESTIO N S B a c k g r o u n d P o s i t i o n , a s s i g n m e n t , l e n g t h i n p o s i t i o n , r o l e i n 1 07 A p r o g r a m D i s c u s s o t h e r y o u r D i s c u s s t h e E d u c a t i o n P r o g r a m m a n a g e m e n t a d m i n i s t r a t o r s p r o g r a m s t y l e . a t p l a n n i n g A l t e r n a t i v e I s y o u r i t c o n s i s t e n t w i t h t h a t o f i n s t i t u t i o n ? p r o c e s s T r a i n i n g i n y o u u s e d f o r t h e A d u l t 1 9 9 2 - 9 3 . S a t i s f a c t i o n I n g e n e r a l , t r a i n i n g Do y o u W h ic h h o w s a t i s f i e d f e e l y o u a c c o m p l i s h e d o n e s ? W hy o r D id t h e t r a i n i n g D id y o u l o s e T o w h a t y o u W h a t t h e w h y d o m o n e y e x t e n t i n t e r e s t s A r e w e r e y o u w i t h t h e o u tc o m e s o f y o u r p r o j e c t ? o f w a s t h e w h a t o n c o n d i t i o n s o b j e c t i v e s o f t h e p r o j e c t ? s a i d i t w o u ld ? E x p l a i n . p r o g r a m ? p r o g r a m u s e d t o f u r t h e r t h e e c o n o m ic i n s t i t u t i o n ? c o m f o r t a b l e c o m p l e t i o n y o u t h e t h e t h e n o t ? w i t h t o o k o f y o u r p l a c e t h e p l a c e m e n t t h a t s t a t e d y o u r a t e s ? b e l i e v e c o n t r i b u t e d to o b j e c t i v e s ? I n v o l v e m e n t How 1 07A How m a n y p e o p l e w e r e i n v o l v e d i n t h e r e a l p l a n n i n g o f t h e p r o g r a m ? w a s d e c i d e ( A d u lt t h i s t h e p r o g r a m p u r p o s e s , p a r t i c i p a n t s , How w e r e W as i t How d i d t h e s e t h e D e s c r i b e p l a n n i n g y o u sa m e g e t r o l e s c o n c e p t u a l i z e d ? c o n t e x t a n d i n s t r u c t o r s , W h ic h f o r m a t o f p e o p l e t h e l e a d e r s h i p , g o t t o p r o g r a m ? p l a n n e r s ) n e g o t i a t e d ? g r o u p o t h e r s w ho a l w a y s w o rk s o n e x t r a p r o j e c t s ? i n v o l v e d ? h o w t h e p r o g r a m o f t h e p r o g r a m . p a r t i c i p a n t s w e r e i n v o l v e d i n t h e 189 D o e s t h e p l a n n i n g o r g a n i z a t i o n ? s t a f f W h at d i d f e e l y o u t h a t d o 107A t o b e l o n g s a c h i e v e w i t h y o u r c o n d u c t t h i s t h i s ? R e s o u r c e s D id y o u h a v e e n o u g h t i m e D id y o u h a v e a d e q u a t e a d e q u a t e t o p l a n t h i s p e r s o n n e l t o p r o g r a m ? p l a n a n d p r o g r a m ? D id y o u h a v e D id y o u h a v e e q u i p m e n t M i t i a a t i n a a d e q u a t e f o r t h i s f i n a n c i a l r e s o u r c e s i n s t r u c t i o n a l f o r m a t e r i a l s , t h i s p r o g r a m ? s u p p l i e s , a n d p r o g r a m ? V a r i a b l e s W e re t h e r e o u tc o m e s W ere a n y o f t h e y m i t i g a t i n g t h e p r o j e c t ? v a r i a b l e s C an y o u t h a t a f f e c t e d d e s c r i b e p o l i t i c a l , e c o n o m ic , o r so m e y o u h a v e t h e th e m ? i n s t i t u t i o n a l p o l i c y f a c t o r s ? A r e t h e r e p r o j e c t ? I s t h e r e a n y p l a n n e r s i f O t h e r w a s a t a d v i c e w a n t i n g r e t r a i n i n g How t h i n g s m i g h t d o n e d i f f e r e n t l y i n t h i s E x p l a i n . t h e t o t h a t y o u im p le m e n t w o u ld a g i v e t o g o v e r n m e n t f u t u r e f u n d e d p r o g r a m t r a i n i n g o r p r o g r a m ? t r a i n i n g p r o c e s s im p r o v e d a f t e r t h i s e x p e r i e n c e , a l l ? C o m m e n ts A r e t h e r e a n y o t h e r c o m m e n ts y o u w is h t o m a k e a b o u t t h e p r o g r a m ? T h e s e q u e s t i o n s b a s e d o n t h e c o v e r p e r s o n t h e b e i n g i n t e r v i e w c o m p o n e n t. i n t e r v i e w e d . P h r a s i n g m ay v a r y APPENDIX H FOCUS GROUP MEETING NOTES 190 107A P R O C E S S FLOW CHART Em ployer C o n tac ts C ollege D eterm ine Im pact on C om m unity Apply for Final P ay m en t A ssess Eligibility C o m p lete and S ubm it Application R eceive Final P ay m en t Explain P ro c e s s R eceiv e Approval Identify and ReviewTraining R e so u rc e s R eceive Initial Paym ent D eterm ine C om pany's N eed s P re p a re and S ubm it M odifications C reate Training Plan Apply for M id-point P aym ent C ontact M ESC/SDA/M JC R eceive M id-point P ay m e n t 191 Creation o f Training Plan 192 Scoring o f Problems Problem Scores Average Score 1. Commitment o f company to training 18,12,10,23,13 15.2 2. Steering Committee 8,6,11,5,13 8.6 3. Accurate company information 19,0,10,10,15 10.8 4. Political considerations 14,18,4,6,5 9.4 5. Training new to company 13,23,7,10,12 13 CAUSE and EFFECT DIAGRAM for PROBLEMS WfTH TRAINING PLANS Multi. Voting S core: Mull. Voting S co re: 2 1(T ) OBJECTAfES 1 2 DOLLAR8 No plan G etting com pany to clarify actual training n ee d s C oncept u a la llo n M anagem ent andunlon_ d s a g re e m e n ts F ree training money of program Getting com pany to articulate w hat they w ant to accomplish b e c a u se of training Getting com pany to a s s e s r s o f train ees p e r class Receiving Q uaRy Input from Co. M anagem ent Physical taking of O n e p erso n s e ts Co. wV not p re ­ a s s e s s em ployees PROBLEMS WITH TRAINING PLANS <£> W tn l n h g plan data C om pany re sp o n se to Identification o f b e s t Instructional te am I e. C o le g e & private trainers Ju st get m e the J . I determ ine w hat I n eed Lack o f administrative reso u rces N eeds and view s o f em ployees and m anagem ent differ Matching th e trainer that w orld b e s t At the training requirem ents C om pany esp eetatlo n s of training length unrealistic. Budgetary considerations Determine the objectives / content of training req u est for Information Training Is new to com pany Co. lack o f skfls a s s e s sm e n t Evaluating Employee skM level Changing polUcal M anagem ent d o e s not ag ree Lack o f reso u rces to getting production floor Input Into the plan on the c o le g e 's part considerations No time to do a good_ Involvement o f com munty groups & agency In planning trsln h g plan p ro c e ss Definition of_ training plan Training commitment and resp o n se Co. mission Interpretation No n eed for plan train for skills only W hen to schedule training S chedulng Training not In conflkt with production C om pany choice of Steering Comm ittee co m p o st Ion Decision o f pro p er credit hours lo m atch em ployee tk fl level C om pany defined evaluation criteria Utilization / avaDabmty o f resources 222 11 222 A8SE98MENT Mull. Voting S core: 111112 1112 Co. defined evaluation crlerta Joint fo to w if) on evaluation r e s e ts G etting com plete concise Information from the Co. INTER-COLLEQE CONSIDERATIONS Multi. Voting Score: i evaluation Multi. Voting Score: 2 12 (T ) D eterm ine current skfl level of Iratre e I em ployee 194 Methods to Improve Commitment o f Company to Training Results o f Pad Storming Explain the process Present data on successful program (show benefits to company Company pays if they fail to do their part - contract with college Company commitment - employee attitude change and involvement demonstrated by projects financed by company Company commitment - communicate benefits derived by other participating companies State requires written commitment Inform management o f value o f training to companies success Communicate to company the purpose o f training grant Joint site visits with MJC/CoUege Clear up front instructions to company Management education Company commitment (as a state requires - large cash match up front - Jobs Commission writes this in guidelines Incentives from MJC/CC for higher percentage completion Press conference Recognition through newspapers Raise training ratio for vendor training N o training plan - no grant proposal - make training plan a pre-requisite requirement Case studies o f successful company initiatives Company Commitment - conduct workshop for potential participating companies on plan impact if involved and responsibilities. Impact: emphasizing up front requirements o f grant for receiving funds Commitment o f funds for uncompleted training Develop a marketing plan for employers that show the benefits o f training 195 Concerns With Modifications After completing our Pad Storming session the group divided the responses into 3 large categories: Internal (College), External (State), and External (Employer). Then the responses were further lumped into smaller Tike groupings' under each larger category as follows COLLEGE Lack o f Resources Planning Communications Late/ slow start Poor planning & anticipation of Coflege problems communicating requirem ents to head off modifications with the state Applicant d o e sn t complete requirem ents in a timely manner College problems communicating with the employer Failure to re ad approval application S tate contacts with Company Trainer availability Staff overextended not coordinated with college New provider required Staff change Lack of resources Poor administration STATE Grant Form Approval Delay Staff Communication Grant Deadline Requirements Staff overextended Failure to clarify acceptance levels Unclear legislation Time for grant approval Staff change State requirements changed Modification form Changing State Requirements Rules changed Application form Contingency for grant approval Vague definitions in grant application Poor instructions Mis-irrformation from account m anager Conflicting State communications Lack o f communication bertw een State and applicant 198 EMPLOYER Politics Company Commitment Communications Business Climate Co. up fo r sale P o tties b etw e en G etting Motfrflcation form inform ation C om pany m tsnnterpretation com pany & S ta te from th e com pany th a t th e m oney is theirs Co. em ployees Em ployers not insisting on a tte n d a n c e M is-M erprtatlon o f th e niles C o. R eo rg an izes C om pany not understanding C o c a n cels c la s s e s d otrt w ant m oney Union dispute Lack of cooperation from th e com p an y th e modification p ro c e s s O u tsid e ag e n d a E m ployees not Com pleting training C o. C hanging training plan AJ P o p e Em ployee motivation C u sto m e r co n tact change C om pany canceling c la s s e s C h an g e in fo cu s from co rp o rate office Ex p ectatio n s o f Co. W anting training now S ta ff C h an g e s C h an g e in H.R. p erso n E m ployee reduction N ew Technology N ew em p lo y ees P roduction change CAUSE and EFFECT DIAGRAM : PROBLEMS WITH MODIFICATIONS External: State External: State A pproval Delay StafT External: Company External: Company C om pany Com m km tant External: Company B u s in e s s Climate P o ttie s Co up for sale Staff Changes Co. Reorganizes Change In H R. person Getting Modficatlon form information from thecom apny PoBlcs between company end State Grant Deadline Employers not Insisting on attendance Requirements Co. cancels classes Company em ployees do not want money StafT change Employee reduction Co Changing training plan New Technology Lack of co o p eratio n ____ Time for p a n t _ Customer contact Union dbpute from the company approval New employees change Changing State Employees not Requirements completing training ____ Outside agenda Production change Change In focus Item corprate office Employee mot h a lion Contlgency for grant approval Expectations of Co. Company cancelling Wanting training now classes Unclear Legislation Falure to clarify acceptance levels Modification Stale requirements changed Appication Lack of communclatlon between S tate & applicant Poor planning t C olege problems Company mK-Interpretation that the $ Is theirs communicating- anticipation of with the state reqttrm ents to head off motiflcatiors C olege problems communicating w th the employer form Vague deflnkions In grant appication Misinterpretation Poor Instructions - of rules Applcant doesn't complete requkem TsIn a timely manner Late/ slow start - S taff overextended New provider required S tate co n tad s with Comp, not coordhated F a k a e to read _ approval appication Co. understandkig mod process Staff change — with co leg e Poor ackninfstratlon Confiding S tale __ communications Rules changed C om m unications Exlemal: State C om m unications External: Company C om m unications Internal: Colege P lanning Internal: College CO CO Trainer avalability- Lack of resources Mls-lnformatlon from _ account m anager PROBLEMS WITH MODIFICA­ TIONS Lack of Resources Internal: College APPENDIX I SUMMARY OF PLANNER SATISFACTION, OUTCOMES, AND CORRELATION SUMMARY OF SURVEY DATA PLANNING CONCEPT 1.1 1.2 1.3 1.4 1.5 FREQUENCY IMPORTANCE 34 36 42 43 44 3.47 3.60 4.33 4.13 3.69 39.8 3.84 39 40 42 44 38 3.47 3.69 3.40 4.38 2.96 40.6 3.58 SD EFFECTIVENESS SD 3.04 3.13 4.13 3.82 3.67 CORRELATION .7983** .8310** .7549** .6634** .6779** 1 AVERAGE/ MEAN OF MEANS 3.56 .46 3.38 3.69 3.44 4.31 3.38 .7573** .9006** .8138** .7356** .8066** 2 AVERAGE/ MEAN OF MEANS .52 3.64 .40 200 2.1 2.2 2.3 2.4 2.5 .37 SUMMARY OF SURVEY DATA (CONTINUED) 3 AVERAGE/ MEAN OF MEANS 4 AVERAGE/ MEAN OF MEANS 38 43 41 40 3.31 4.29 4.18 3.47 40.5 3.81 39 44 43 44 39 42 44 3.49 4.02 4.00 4.44 3.71 3.80 4.47 42.1 3.99 SD EFFECTIVENESS SD 3.24 4.00 3.78 3.40 .49 3.61 .8560** .6494** .7512** .35 3.36 4.02 4.00 4.36 3.67 3.67 4.07 .36 3.88 CORRELATION .7616** .6574** .7845** .8609** .9255** .6536** .7529** .33 201 4.1 4.2 4.3 4.4 4.5 4.6 4.7 IMPORTANCE • * 3.1 3.2 3.3 3.4 FREQUENCY -sj o o PLANNING CONCEPT SUMMARY OF SURVEY DATA (CONTINUED) PLANNING CONCEPT 5.1 5.2 5.3 5.4 5.5 5 AVERAGE/ MEAN OF MEANS FREQUENCY IMPORTANCE 4.40 3.98 4.18 3.60 3.76 41.8 3.98 27 40 38 26 43 42 32 37 2.51 3.78 3.78 2.47 3.73 3.87 3.33 3.58 35.6 3.38 EFFECTIVENESS SD 4.04 3.71 3.96 3.18 3.69 .32 3.72 CORRELATION .4701** .4862** .5001** .8392** .7780** .34 202 44 43 43 36 43 SD 6.1 6.2 6.3 6.4 6.5 6.6 6.7 6.8 6 AVERAGE/ MEAN OF MEANS 2.44 3.58 3.16 2.22 3.56 3.73 3.09 3.40 .57 3.15 .8520** .8637** .8821** .8347** .6823** .8079** .8965** .8810** .55 SUMMARY OF SURVEY DATA (CONTINUED) PLANNING CONCEPT 7.1 7.2 7.3 7.4 7.5 7.6 7.7 8.1 8.2 8.3 8.4 8.5 8.6 8.7 8.8 8.9 44 42 39 39 38 40 33 IMPORTANCE SD 3.96 3.98 3.64 3.51 3.51 3.89 3.40 EFFECTIVENESS SD 3.69 3.64 3.33 3.27 3.02 3.56 2.78 .4178* .4405* .6610** .7634** .6323** .4889** .6565** 39.3 3.70 29 42 23 43 43 45 44 43 19 .24 3.33 .34 2.84 4.16 2.31 4.31 4.38 4.18 4.20 4.20 2.07 2.96 4.24 2.76 4.47 4.60 4.62 4.62 4.44 2.42 .8106** .6989** .8122** .8571** .9129** .3060 .6641** .8549** .8485** 36.8 8 AVERAGE/ MEAN OF MEANS CORRELATION 3.90 .91 3.63 203 7 AVERAGE/ MEAN OF MEANS FREQUENCY .94 BIBLIOGRAPHY BIBLIOGRAPHY Alfred, R. L. (1991). Evaluating results of econom ic and workforce developm ent program s. New Directions for Community Colleges. Z5, 85-99. Alfred, R. L.. & Linder, V. P. (1990). Rhetoric to reality: Effectiveness in community colleges (Community College Consortium Report). Ann Arbor: University of Michigan. American Association of Community and Junior Colleges. (1990). Executive sum m ary and reports 1 and 2. W ashington, DC: Author. Apps, J. W. (1979). Problem s in continuing education. San Francisco: Jo sse y B ass. Auletta, K. (1982). The underclass. New York: Random House. Bailis, L. N. (1987). The m ore things change. . . . A study of the status of FY 85 JTPA coordination. W ashington, DC: National Commission for Employment Policy. Barnow, B. S. (1987). The impact of CETA program s on earnings: A review of th e literature. Journal of Human R eso u rces. 22(2). 157-193. Blong, J. T., & Schultz, R. M. (1990). The dislocated worker: When training is not enough. Community. Technical and Junior College Journal. £0(4), 2832. Bloom, H. S. (1987). W hat works for whom ? CETA impacts for adult participants. Evaluation Review. 11(4), 510-527. Boone, E. J. (1985). Developing program s in adult education. Englewood Cliffs, NJ: Prentice-Hall. Boyle, P. G. (1981). Planning better program s. New York: McGraw-Hill. 204 205 Bratman, M. E. (1987). Intention, plans, and rea so n . Cambridge, MA: Harvard University Press. Brecher, C. (1972). Upgrading blue-collar and service w orkers. Baltimore, MD. Jo h n s Hopkins Press. Brint, S., & Karabel, J. (1989). The diverted dream ; Community colleges and th e prom ise of educational opportunity in America. New York: Oxford University Press. Brookfield, S. D. (1986). Understanding and facilitating adult learning: A com prehensive analysis of principles and effective practice. San Francisco: Jossey-B ass. Brooks, A. (1990). South Carolina’s special schools: A dollars and se n se incentive. Community. Technical and Junior College Journal. 61(4), 38-41. Bryant, D. W. (1988). The process of planning for community college effectiveness by m anaging change. Community College Review. 16(3), 21-25. Bryant, E. C., & Rupp, K. (1987). Evaluating the impact of CETA on participant earnings. Evaluation Review. 11(4), 473-492. Buskey, J. H. (1987). Utilizing a system atic program planning process. In Q. H. G e ssh er (Ed.), Handbook on continuing education. New York: American Council on Education & Macmillan. Butler, E. P. (1988). The search for the bottom line: W hat lessons are learned in th e JT P Act? Policy options for vocational education. W ashington, DC: Vocational Education. Caffarella, R. S. (1988). Program developm ent and evaluation resource book for trainers. New York: Wiley. Calhoun, C. C., & Finch, A. V. (1976). Vocational and career education: C oncepts and operations. Belmont, CA: W adsworth. Cantor, J. A. (1985). A local industry solves its training needs: A cooperative training venture that works. Charleston, SC: National Conference on Technical Education. C arnegie Foundation. (1991). The learning industry: Education for adult w orkers. Laurenceville: Princeton University Press. 206 C am evale, A. P., Gainer, L. J., & Holland, S. L. (1990). Training partnerships: I inking em ployers and providers. Alexandria, VA: American Society for Training & Development. C am evale, A. P., Gainer, L. J., & Villet, J. (1990. Training in America: The organization and strategic role of training. S an Francisco: Jossey-B ass. Carr, C. (1992). Sm art training. New York: McGraw-Hill. C asner, Lotto, J., and A ssociates (Eds.). (1989). Successful training strategies. San Francisco: Jossey-B ass. Cervero, R. M. (1989). Becoming more effective in everyday practice. In B. A. Quigley, Fulfilling the prom ise of adult and continuing education. San Francisco: Jossey-B ass. Cervero. R. M., & Wilson, A. L. (1994). Planning responsibly for adult education: A guide to negotiating power and influence. San Francisco: Jossey-B ass. C oates, J., Jarratt, J., & Mahaffie, J. B. (1990). Future work: Seven critical forces reshaping work and the workforce in North Am erica. San Francisco: Jossey-B ass. Cohen, A. M. (1989). W hat next for community colleges? Community College Review, 1Z(2), 53-58. Colgan, A. H. (1993, March/April). Models for program developm ent. Adult Learning, pp. 7-9. Cosgrove, J. (1990). The continuing education and community services m ission. R eferences in A ssessing institutional effectiveness in community colleges. Laguna Hills, CA: League for Innovation in the Community College. Coughlin, D. C., & Bielen, R. P. (1981). Benchmarking: Issues in the design and implementation of a benchmarking system for employm ent and training program s for young people. Braintree, MA: Nellum & A ssociates. Cowart, S. C. (1988). A survey on using student outcom es m ea su re s to a s s e s s institutional effectiveness. New York: American College Testing Services. Crawford, F. W., & Webley, S. (1992). Continuing education and training. London: Needham Printers. 207 Crosby. P. B. (1989). Let’s talk quality: 96 questions you always w anted to a sk . New York: McGraw-Hill. Dewey, J. (1915). The logic of judgem ents in practice. Journal of Philosophy. Psychology and Scientific M ethods. 12, 505-523. Dickinson, K. P. (1987). The impact of CETA program s on com ponents of participants’ earnings. Industrial and Labor Relations Review. 4Q(3), 430441. Dominick. J. (1990). How adult educators m ake program developm ent decisions in practice. Unpublished doctoral dissertation, University of Georgia. Fifield, M. L. (1990). W orkers for the world: Occupational program s in a global econom y. Community. Technical and Junior College Journal. 61. 15-19. Finifter, D. H. (1987). An approach to estimating net earnings impact of federally subsidized employm ent and training program s. Evaluation Review. 11(4), 528-547. Fleishm an, E. A., & Mumford, M. D. (1989). Individual attributes and training perform ance. In I. L. Goldstein and Associates, Training in developm ent in organizations. S an Francisco: Jossey-B ass. Forester, J. (1989). Planning in the face of power. Berkeley: University of California Press. Forester, J. (1993). Critical theory, public policy, and planning practice. Albany: S tate University of New York Press. Friedman, P. G., & Yarbrough, E. A. (1985). Training strategies from start to finish. Englewood Cliffs, NJ: Prentice-Hall. G eneral Accounting Office. (1984). Strong internal controls at service delivery level will help prevent CETA-type fraud and abuse in JTPA program s. W ashington, DC: Author. Goldstein, I. L. (1990). Training in the year 2000. American Psychologist. 15, 134-143. Governor’s Office for Job Training. (1988). Investing in people (Vol. 4). Lansing, Ml: Author. 208 G rede, J. F. (1992). Toward a national job training program . W ausau, Wl: National Council for Occupational Education. (ERIC Document Reproduction Service No. ED 364 262) H aberm as, J. (1971). Knowledge and hum an interests. Portsmouth, NJ: H einem ann Educational Books. Hall, K., & Miller, I. (1974). Retraining and tradition. London: George Allen. Ham mersley, M. (1993). R ese arch d e sig n : Philosophy, politics and practice. Newbury Park, CA: Sage. Harlan, S., & Hackett, E. J. (1984). Federal job training program s and em ploym ent outcom es (Working P aper No. 129). Wellesley College/ National Science Foundation. Hart, M. U. (1992). Adult education and the future of work. New York: Routledge & Kegan Paul. Heckman, J. J. (1987). Do w e need experimental data to evaluate the impact of m anpow er training on earnings? Evaluation Review. 11(4), 395-427. H erschbach, D. R. (1988). Linking with employment: Training from the perspective of em ployers. St. Louis, MO: American Vocational Association Convention. Hirshberg, D. (1991). S ources and information: Community colleges and econom ic developm ent. New Directions for Community Colleges. 75. 101107. Hlavna, D. P. (1992). Economic development: Human capital theory and the community college. Community College Review. 19(4), 47-51. Hockaday, J., & Friga, J. J. (1989). A ssessm ent of institutional effectiveness: A practical model for small colleges. Community College Review. 1Z(3), 2833. Houle, C. O. (1961). The inquiring mind. Madison, Wl: University Press. Houle, C. O. (1972). The design of education. San Francisco: Jossey-B ass. Iowa D epartm ent of Education. (1991). A study of the impact of Iowa community college continuing education program s. Des Moines: Author. 209 Jaco b s, J. (1992). Custom ized training in Michigan: A n ecessary priority for community colleges. Mt Clem ens, Mt: Macomb Press. Jen ck s, C. (1993). Rethinking social policy: Race, poverty and the underclas. New York: Harper Perennial. Jurm o, P. (1993). Rethinking how to plan and evaluate workplace education program s. Innovations in New York S ta te . Albany, NY: Education Departm ent. Kantor, S. L. (1994). A practical guide to conducting custom ized work force training. S an Francisco: Jossey-B ass. Katsinas, S. G., & Lacey, V. A. (1990). Community colleges and econom ic developm ent. Community Services Catalyst. 2Q, 6-14. Kelly, J. S. (1989). Keeping the client: JTPA retention m ade easier. W ashington, DC: National Association of Counties. Kiefer, N. M. (1979, October). Population heterogeneity and inference from panel data on the effects of vocational education. Journal of Political E conom y. 6Z, 213-226. Knowles, M. S. (1950). Informal adult education. New York: Association P ress. Knowles, M. (1979). W here does training fit into the adult education field? Training and Developm ent Journal. 33(12), 40-42. Knowles, M. S. (1989). Everything you w anted to know from Malcolm Knowles. Training. 26(8), 45-50. Knox, A. B. (1980). Developing, administering and evaluating adult education. S an Francisco: Jossey-B ass. Kopecek, R. J. (1991). Assuming a leadership role in community econom ic developm ent. New Directions for Community C olleges. Z5, 41-47. LaLonde, R., & Maynard, R. (1987). How precise are evaluations of employm ent and training programs: Evidence from a field experiment. Evaluation Review. 11(4), 428-451. Lamar, C. S. (1978). Com prehensive planning for vocational education. Arlington, VA: American Vocational Association. 210 Lawther, W. C., & Gromelski, R. A. (1984). An evaluation model for m anpow er training program m anagers. Evaluation Review. S(6), 843-859. League for Innovation in the Community Colleges. (1990). A ssessing institutional effectiveness in community colleges. Laguna Hills. CA: Author. Leigh, D. E. (1989). Assisting displaced workers^. Do.the states..haye.a batter idea? Kalamazoo, Ml: Upjohn. Leigh, D. E. (1990). Does training work for displaced workers? Kalamazoo, Ml: Upjohn Institute for Employment R esearch. Levitan, S. A., & Gallo, F. (1988). A second chance: Training for jo b s. Kalamazoo, Ml: Upjohn Institute for Employment R esearch. Lorenzo, A. L. (1991). Anticipating our future purpose. Community. Junior and Technical College Journal. 4, 42-45. Lyons, T. M., & Knight, G. A. (1985). The Chrysler-W eldtech approach to retraining displaced workers (Final Technical Reportt. Detroit: Chrysler Learning. M agaziner, I., & Reich, R. B. (1982). Minding America's b u sin ess. New York: Harcourt, Brace, Jovanovich. Matoon, J. S. (1992). Evaluating training and educational program s: A review of the literature. Brooks AFB, TX: Armstrong Lab. McLeod, M. W., & Atwell, C. A. (1992). A mini dictionary on institutional effectiveness term s. Community College Review. 2jQ, 30-39. Melville, J. G., & Chmura, T. J. (1991). Strategic alignment of community colleges and state econom ic policy. New Directions for Community Colleges. 75. 7-15. Michigan Community College Association. (1993). New directions commission: Sum m ary of em erging issu es and opportunities. P aper presented at the MCCA m eeting at Henry Ford Community College, Dearborn, Ml. Michigan Community Colleges Job Training Network. (1984). Michigan community colleges job training and retraining investm ent fund: Impact statem en t. Ann Arbor: Michigan Departm ent of Education. 211 Michigan D epartm ent of Education. (1984). Michigan’s community colleges JTRIF: Impact statem ent. 1983-84. Lansing: Author. Millar, C. (1989). Educating the educators of adults: Two cheers for curriculum Mitchell, G. (1993). The trainer’s handbook. The AMA guide to effective training. New York: American M anagem ent Association. Mora, P. L., & Giovannini, E. V. (1989). Focus on the 1990’s: Clarifying the role of community colleges in econom ic development. Community Services C atalyst. 20, 8-14. Murk, P. J., & Wells, J. H. (1988). A practical guide to program planning. Training and D evelopm ent Journal. 42(10), 45-47. Murray, C. (1984). Losing ground. New York: Basic Books. Nadler, L. (1982). Designing training program s: The critical events m odel. Reading, MA: Addison-W esley. National C enter for R esearch in Vocational Education. (1992). Betwixt and between: Education, skills, and employment in subbaccalaureate labor m arkets. Berkeley, CA: Author. National Council on Employment Policy. (1983). Back to basics under JTPA: A policy statem en t. W ashington, DC. (ERIC Document Reproduction Service No. ED 294 047) Nedwek, B. P. (1987). Political socialization and policy evaluation: The c a se of youth employment and training program. Evaluation and Program Planning. 10(1), 35-42. Nespoli, L. A. (1991). Investing in hum an capital: S tate strategies for econom ic developm ent. New Directions for Community C olleges. 75. 17-30. Nichols, J. O. (1989). Institutional effectiveness and outcom es a s s e s s m e n t. New York: Agathon Press. Pace, C. R. (1979). M easuring outcom es of college. San Francisco: Jo ssey B ass. 212 Palmer, J. (1990). How do community colleges serve business and industry? A review of issu es discussed in the literature. W ashington, DC: American Association of Community and Junior Colleges. P arees, B., & Bartok, L. A. (1989). Jo b s for econom ic growth: An Allegheny Joint Initiative Project. Community Services C atalyst. 12(4), 27-28. Parnell, D. (1993). Logolearning: Search for m eaning in education. W aco, TX: C enter for Occupational R esearch and Development. Patterson, T. J. (1993). Paradigm s for program planning: Complex problem s require new approaches. Journal of Extension. 31, 21-22. Pennington, F., & Green, J. (1976). Com parative analysis of program developm ent in six professions. Adult Education. 2Z, 13-23. Peters, T. J., & W aterm an, R. H. (1982). In search of excellence: L essons from Am erica’s best-run com panies. New York: Harper & Row. Putrow, M. L. (1986). Adult education. . . . Is there a planning m odel? O ptions. 6 ( 11 ), 10- 12 . Quinley, J. (1990). Institutional effectiveness planning. Charlotte: Central Piedm ont Community College. Rainbird, H. (1990). Training m atters. Oxford. England: Basic Blackwell. Ramey, L. (1984). Re-employment training: Evaluation of the Oakland University RECAP/JETS program . Rochester, Ml: Oakland University. Ree, M. J., & Earles, J. A. (1991). Predicting training success: Not much more than G. Personnel Psychology. 44, 321-332. Reich, R. B. (1986). Tales of a new Am erica. New York: Times Books. Reich, R. B. (1991). The work of nations: Preparing for 21st century capitalism . New York: Knopf. Reich, R. B. (1993). Strategies for a changing workforce. Educational R ecord. 74(4). 21-23. Reich, R. B., & Judis, J. B. (1994). Can training save Am erica’s living standard s Training. 31(4), 37-41. 213 Rossi, P. H., & Freem an, H. E. (1993). Evaluation: A system atic approach. Newbury Park, CA: S age. Rosow, J. M., & Zager, R. (1988). T raining-the competitive e d g e . San Francisco: Jossey-B ass. S araso n , S. B. (1972). The creation of settings and the future societies. San Francisco: Jossey-B ass. Saul, J. R. (1990). Can community colleges provide the training for educators of adults? Community College Review. 18(3), 51-57. Schon, D. A. (1991). The reflective turn: C ase studies in and on educational practice. New York: T eachers College Press. Schuette, G., & Giles, W. E. (1988). Institutional effectiveness via outcom es a s s e s s m e n t. New York: American College Testing Program. Simerly, R. G. (1989). Handbook of marketing for continuing education. San Francisco: Jossey-B ass. Simms, M. C. (1989). The effectiveness of governm ent training program s (Paper No. 10). W ashington, DC: Departm ent of Labor. Sims, R. R. (1993). Evaluating public sector training program s. Public Personnel M anagem ent. 22(4), 591-614. Smith, E. J. (1983). Manpower training coordinators project (Final Report). Columbia: South Carolina Departm ent of Education. Solomon, J. S. (1985). Re-emplovment for dislocated workers: JTPA . Maryland: D epartm ent of Education. Sork, T. J. (1991). Mistakes m ade and lessons learned: O vercoming obstacles to successful program planning. S an Francisco: Jossey-B ass. Sork, T. J., & Buskey, J. H. (1986). A descriptive and evaluative analysis of program planning literature, 1950-1983. Adult Education Quarterly. 38(2), 86-95. Sork, T. J., & Caffarella, R. S. (1989). Planning program s for adults. In S. B. Merriam & P. M. Cunningham, Handbook of adult and continuing education. S an Francisco: Jossey-B ass. 214 Strom sdorfer, E. W. (1987). Economic evaluation of CETA: An overview of recent findings and ad v an ces in evaluation m ethods. Evaluation Review. 11(4), 387-394. Taggart, R. (1981). A fisherm an’s guide: An a sse ssm e n t of training and remediation strateg ies. Kalam azoo, Ml: Upjohn Institute for Employment R esearch. T exas Departm ent of Com m erce. (1993). Sm art jobs training plan. Austin, TX. (ERIC Document Reproduction Service No. ED 369 927) Tyler, R. W. (1949). Basic principles of curriculum instruction. Chicago: University of Chicago Press. Ullrich, R. A., & Araskog, R. V. (1991). The American workforce: Labor and em ploym ent in the 80’s . Ann Arbor, Ml: University P ress. United S tates Departm ent of C om m erce. (1994). Statistical abstract of the United S ta te s. W ashington, DC: U.S. Departm ent of C ensus. United S tates D epartm ent of Education. (1986). T rends in adult education. 19698 4 . W ashington, DC: Author. United S tates Distance Learning Association. (1992). Retraining 50 million Am ericans (USDLA Publication, Vol. 6, no. 1). S an Ramon: Applied B usiness Telecom m unications. Updike, K. M. (1991). A com parative study of contract training in select community colleges. Phoenix: Maricopa Community College. Usher, R. (1991). Theory and m etatheory in the adult education curriculum. International Journal of Lifelong Education. 112(4), 300-315. Usher, R.. & Bryant, I. (1989). Adult education a s theory, practice, and research: The captive triangle. New York: Routledge & Kegan Paul. Waddell, G. (1990). Tips for training a world class workforce. CommunityTechnical and Junior College Journal. 60. 21-27. Waddell, G. (Ed.). (1991). Econom ic and work force developm ent. New Directions for Community C olleges. Z5, 3-5. 215 Warford, L. J. (1989). A study of custom ized training program s at selected community colleges. Unpublished doctoral dissertation, University of Oregon. W armbrod, C. P., & Faddis, C. R. (1983). Retraining and upgrading workers: A guide for postsecondary educators (R esearch & Developm ent S eries No. 235). Columbus: Ohio S tate University. Welker, W. F., & Morgan, S. D. (1991). O ne step beyond w hat the literature sa y s on institutional effectiveness of community, junior, and technical colleges. Community College Review. 12(2), 25-31. Wilson, W. J. (1987). The truly disadvantaged. Chicago: University of Chicago P ress. Wismer, J. N., & Zappala, J. G. (1993, October). Michigan community college workforce training program s. (Available from the Michigan Community College Association, Lansing, Ml) W uertele, P. (1990). The business and industry liaison a s consultant. Community. Technical and Junior College Journal. 22(4), 26-27. Zeiss, T. (1990). Employee retention: A challenge of the 90's. Community. Technical and Junior College Journal. 22(4), 34-37. Zenger-Miler, Inc. (1992). B usiness education partnerships in the 9Q’s: Collaboration nets big w ins. S an Jo se: Author.