t . \.x.s.7 1|. :3. ..L 1: sisox 5: l... . a. 1.1..) ... . ‘7‘. 1 trit- :. . .1». 3» fi. , . 41.9.7.3... I); .I!\itp ... rrnr .. . , . Are... tn... M... NIT -~ ‘lllllllll This is to certify that the thesis entitled TRANSFER OF ENVIRONMENTAL TRAINING INFORMATION presented by AMY ELIZABETH HERMODSON has been accepted towards fulfillment of the requirements for M.A. COMMUNICATION degree in M / D Major profes r Date M— 0-7639 MS U is an Affirmative Action/Equal Opportunity Institution LIBRARY 1 Michigan State Unlverslty PLACE IN RETURN BOX to tomato thb checkout from your record. TO AVOID FINES rotum on or bdoro data duo. DATE DUE DATE DUE DATE DUE 4 WWW » MSU IsAn Affirmative Action/Equal Opportunity lnotttmton TRANSFER OF ENVIRONMENTAL TRAINING INFORMATION BY Amy Elizabeth Hermodson A THESIS Submitted to Michigan State University in partial fulfillment of the requirements for the degree of MASTER OF ARTS Department of Communication 1993 ABSTRACT TRANSFER OF ENVIRONMENTAL TRAINING INFORMATION BY Amy Elizabeth Hermodson What happens to the information presented in training workshops once the workshops end? After attendees return to their organization do they use that information on the job? What factors are associated with the attendee using or not using the information? It has been estimated that not more than 10 percent of all training information is transferred to the job. While many studies have examined this transfer problem, few studies have resulted in empirical solutions or even a better understanding of the failures. The current thesis concerns the use of information that is presented in environmental training workshops. An investigation of the factors which influence the use of the information on the job was conducted. Eighty-five participants were contacted by telephone survey from one to four years after their initial training experience. Results suggest that the percentage of use of the training workshop information and materials was very high (up to 80 percent for some measures of use), compared with an average of 10 percent use for other training studies. The investigation of factors that influence use of training was inconclusive. Conclusions of the findings and recommendations for further study are discussed. This thesis is dedicated with much love to my parents Mark and Sue and to my sister Gail and to my favorite person who knows who he is iv ACKNOWLEDGEMENTS I thank my committee members Jim Dearing, Cynthia Fridgen, and Alicia Marshall for all their help. Cynthia and Alicia's suggestions and guidance proved to be invaluable. I especially want to thank Jim for making this whole thesis possible -- from its conception, to its funding, to its development and through to its completion. I felt very honored to be a participant in the overall research project of which this thesis was a part. Most of all, I appreciate all of the many hours he took out of his busy schedule to help and guide me through this project. His patience with me is to be commended. I am also very grateful to Gary Meyer. His insight and critical point of view were of great help to me. His humor and wit helped me get through many marathon meetings, as well. Thanks also to Julie, Eric, Carolyn and Tony whose contributions to this project were significant. Without their help with the surveys and data analysis this project would not have been done. Special thanks go to the Educational Design Team at the [naiversity of Michigan, specifically to Paul Nowak and Michael Berger, for taking the time to acquaint me with their training program and for getting me all the information I needed for the project. Also, thanks to Andy Duncan for sending me all the data lists. Finally, thanks to my family who have been a great support to me through this endeavor. And, as always, love to my favorite person who let me go 625 miles away to get my degree and who married me when it was all over. vi LIST OF Chapter I. II. III. IV. TABLE OF CONTENTS FIGURES O O O O O O O O INTRODUCTION . . . . . . Transfer of Training . . The Training Workshop. . Multimedia Training Programs . Short-term Process Evaluation. The Current Study. . . . REVIEW OF LITERATURE . . Statement of the Problem Organizational Barriers. Trainee Barriers . . . . Training Barriers. . . . Organizational Facilitators Trainee Facilitators . . Training Facilitators. . Research Objectives. . . METHODS O O O O O O O O O Subjects . . . . . . . . Instrumentation of Data. Collection of Data . . . Treatment of Data. . . . RESULTS O O O O O O O O O Results of Hypothesis 1. Results of Hypothesis 2. Results of Hypothesis 3. Results of Hypothesis 4. Results of Hypothesis 5. Results of Hypothesis 6. vii Page ix p QVU'l-bt‘ 10 11 17 21 25 30 34 38 41 41 42 45 48 55 55 61 65 67 69 71 V. CONCLUSIONS & DISCUSSION . . . . . . . . . . . . . 74 Summary of Procedures. . . . . . . . . . . . . . . 74 Summary of Findings. . . . . . . . . . . . . . . . 75 Conclusions. . . . . . . . . . . . . . . . . . . . 81 Discussion and Recommendations . . . . . . . . . . 82 APPENDIX A: Final Survey . . . . . . . . . . . . . . . . 89 APPENDIX B: Survey Codebook. . . . . . . . . . . . . . . 95 REPMCES O O O O O O O O O O O O O O O O O O O O O O O O 1 o 1 viii LIST OF FIGURES FIGURE 1 Baldwin and Ford's Model of the Transfer Process. . . 2 Summary of Organizational Barriers. . . . . . . . . . 3 Summary of Trainee Barriers . . . . . . . . . . . . . 4 Summary of Training Barriers. . . . . . . . . . . . . 5 Summary of Organizational Facilitators. . . . . . . . 6 Summary of Trainee Facilitators . . . . . . . . . . . 7 Summary of Training Facilitators. . . . . . . . . . . 8 Codes for the Survey's Open-Ended Responses . . . . . 9 Results of the Frequency Tests Run on the Variables Of use O O O O O O O O O O O O O O O O O O O O O O O O 10 Summary of Significant Mean Score Comparisons for Use Variables and Use Variable Correlations with Organizational Factors. . . . . . . . . . . . . . . . 11 Summary of Significant Mean Score Comparisons for Use Variables and Use Variable Correlations with Trainee Characteristics . . . . . . . . . . . . . . . . . . . 12 Summary of Significant Mean Score Comparisons for Use Variables and Use Variable Correlations with Workshop Experiences . . . . . . . . . . . . . . . . . . . . . 13 Summary of Significant Mean Score Comparisons for Use Variables and Use Variable Correlations with Practicality of Format. . . . . . . . . . . . . . . . 14 Summary of Significant Mean Score Comparisons for Use Variables and Use Variable Correlations with Relevance of Content. . . . . . . . . . . . . . . . . ix PAGE 16 20 24 29 33 37 50 57 62 66 68 7O 73 CHAPTER I INTRODUCTION What happens to the information presented in training workshops once the workshops end? After attendees return to their organization do they use that information on the job? What factors convince the attendee to use the information or not use the information? These are questions that have only recently begun to be answered through training research. The present study investigates answers to these questions. It is interested in the percent of attendees which used the information presented to them in an environmental training workshop. More specifically, an investigation of the factors which influenced the use of the information on the job was conducted. T E E T . . Transfer of training is the degree to which trainees effectively apply the knowledge, skills and attitudes gained in a training context to the job (Newstrom, 1984: Wexley & Latham, 1981). It is generally recognized that the conditions of transfer include both the application of learned material to the job and the maintenance of trained skills over a period of time on the job (Baldwin & Ford, 1988). Thus, the transfer of training is the communication of information which is useful for decision-making and the continued use of that knowledge. One model for understanding what transfer of training is and for identifying what factors influence the transfer process was developed by Baldwin 8 Ford (1988). Figure 1 is a copy of this model. The model shows us that training input factors and training outcomes have direct and indirect effects on the conditions of transfer. There are six links in this model which are critical to the transfer process. Link 6 shows a direct link between learning and retention and the conditions for transfer. This means that training materials must be learned and retained in order for the skills to be transferred. Links 4 8 5 also have a direct link to transfer. Examples given by Baldwin 8 Ford of this link is that if lack of motivation on the trainees' part or supervisory support is not adequate, well-learned skills may not be retained. Links 1, 2, and 3 directly affect training outcomes and have an indirect effect on transfer from their direct link on training outcomes (Baldwin & Ford, 1988). The present study is interested in the percent of cases in which generalization occurred for the workshops studied and which training inputs influenced this condition of transfer. Figure 1 Baldwin and Ford’s Model of the Transfer Process Trainee Characteristics - ability 4 - personality - motivation 2 Training Design it - Principles of Learning Learning Generalization - Sequencing. ‘--— l -——) a ‘—-- 6 —-§ 1. - Training Content Retention Maintenance 1‘ qt 3 Work Environment - Support - Opportunity to Use 5 1;;1n1ng_1nng§g : Trainee Characteristics, Training Design. a Work Environment Igginjng_gugpn1§ = Amount of learning that occurred in training a retention after training angltjgn§_jgg_1£gn§£gx = Generalization of training to the job and maintenance over time Igijngg_§n‘£.glg£1§jjgfi = Ability or skill, personality factors, a motivation 1131n1n3_ng§13n = incorporation of learning principles. sequencing of training materials, I job relevance of training content W = Support from supervisor and/or peers and the opportunities to perform learned behaviors on the job T] T . . ll 1 1 In 1986, the Educational Design Team in the School of Natural Resources at the University of Michigan was contacted by the Association of State and Territorial Solid Waste Management Officials (ASTSWOMO) for assistance with ASTSWOMO's United States Environmental Protection Agency training programs. Because of high turnover rates in the staff at the United States Environmental Protection Agency and at the state level, and a lack of resources to train incoming staff, a new program was needed to train new staff quickly and efficiently. With the cooperation of the Educational Design Team, ASTSWOMO, and the EPA, three basic training programs (inspector training, permit writing, and RCRA orientation) were developed and completed by 1988. The training programs were multimedia in format, consisting of a group workshop with speakers and facilitators, a video, and a written manual. The topics covered in the programs included, but were not exclusive to: 1) An introduction to the Federal Resource Conservation and Recovery Act (RCRA), 2) hazardous waste safety issues, 3) inspection procedures, and 4) permit writing procedures. The programs were designed as train-the-trainer materials, meaning that those attending the workshop could go back to their agency and conduct the same program for their staff. The training programs were first pilot-tested with people from the Educational Design Team, ASTSWOMO, and other specialists prior to being refined for presentation at the workshops. The subjects attending the main workshops were primarily from state environmental agencies across the United States, but others who could benefit from the information were also present. After the workshops, subjects were to take the materials back to their organizations to lead workshops of their own. The Educational Design Team also sent the workshop materials to various regional centers of the EPA. Additional copies of the workshop materials were publicized and could be ordered from the Educational Design Team. It is not known to the Educational Design Team how many people have used the workshop information to date and what factors influenced the use or non-use of the training information. MW Video is a medium which offers distinct advantages in staff training (Shaddock, 1989). In most instances where video was used as a training technique, results show that people acquired more knowledge following the workshop than when they received no video treatment at all (Cassileth, Heiberger, March, 8 Sutton-Smith, 1982; Kaufman 8 Kaufman, 1983; Maguire, Goldberg, Hobson, Margison, Moss, 8 O'Dowd, 11984; Moldofsky, Broder, Davies, 8 Leznoff, 1979; Nielsen 8 Sheppard, 1988; Pratt, Schmall, Wilson, Benthin, 1991; Soukup 8 Albrecht, 1983; 8 Uzark, Rosenthall, Behrendt, 8 Becket, 1985). In some cases video-treated groups performed as well, but not better than control groups who received no video treatment (Bradlyn, Christoff, Sikora, O'Dell, 8 Harris, 1986; Raphel 8 Wagner, 1974; 8 Tietge, Bender, 8 Scutchfield, 1987). When combined with other mediums such as written material, audio tapes, lectures and other workshop activities, the video becomes a highly effective training tool. However, special attention should be paid to what combination of media is used. Many studies have found that certain combinations of media work better than others, such as written material with the video alone or a lecture that is enhanced by the video (Flanagan, Adams, 8 Forehand, 1979; Kaufman 8 Kaufman, 1983; O'Dell, Mahony, Horton, 8 Turner, 1979: Sox, Marton, Higgins, 8 Hickam, 1984). Training program developers should also be cautioned against using too many media elements. Trainees may feel overwhelmed with too much information presented to them in too many ways (Bell, Prewitt-Diaz, Molina, 8 Mendez, 1985). Even with these issues in mind, multimedia training programs are still as good and are often more effective than more traditional single media training programs in increasing short-term knowledge. Of course, creating multimedia programs tends to be relatively expensive and require better planning and coordination than single media training programs. This literature suggests that the outcome of the multimedia training program conducted by the Educational Design Team should be positive. People should be successfully trained and new knowledge should be applied to the job. However, some evidence suggests that multimedia training is not so effective in improving long term retention and use of the training program information (Gagliano, 1988). WWW Training programs, in general, have not been effectively evaluated in the past. In very few instances have the developers of training programs incorporated methods by which the persistence and the actual use of information can be measured in the organization. Few developers or researchers have bothered to find out what happens to the information gained in the workshop over a period of years. Most researchers assume that the information will be lost at some point. Estimates suggest that only 10 percent of all training information will be utilized after training (Baldwin 8 Ford, 1988). In their efforts to make a great training program, most program developers have ignored post-intervention evaluation. Program developers often make sure that the program meets the needs of the trainees and their organizations, that materials are pretested and modified if necessary, and that the presentation of the program is done in the best way possible. ‘But when evaluation is conducted after training programs end, it is often in the form of a brief survey that asks the trainees to judge the extent to which they are satisfied with the program. The problem with these surveys, as Jones (1990) cited in (Petrini, 1990) points out, is that end-of-training questionnaires generate statistics with a pristine, truthful look, but their validity and usefulness is questionable. Jones (1990) cites 26 reasons for not using these surveys to evaluate training programs. Perhaps the most important of Jones’ reasons is that the ratings generated by these surveys do not correlate with the transfer of training or, more importantly, with the actual use of the information which was originally provided in the workshop. W313 The current study addresses problems associated with the communication of, and evaluation of, training program information. This study focuses on the percentage of cases in which transfer and use took place after an environmental training workshop. It will also examine which factors influenced this use of the information. Chapter 2 examines the literature about transfer of training. The review concentrates on the organizational, trainee, and training barriers that affect the use of training information. Research objectives, derived from this literature are stated at the end of Chapter 2. Chapter 3 reviews the methods of the present study. The subjects and how they were selected to participate in the survey are detailed. The development, distribution, and evaluation of the present telephone survey are explained. Chapter 4 reveals the results of the survey, and Chapter 5 discusses conclusions and implications. CHAPTER II REVIEW OF LITERATURE SLELBMQDL_QL_§DQ_EIQDIQM It has been estimated that American industries annually spend $50 to $100 billion on direct and indirect costs for training every year (Baldwin 8 Ford, 1988). Not more than 10 percent of those expenditures actually result in the training being used on the job (Baldwin 8 Ford, 1988). While many studies have examined this transfer problem, few have resulted in empirical solutions or even a better understanding of the failures. There is still much work to be done in this area of study. The chances for transfer and generalization of training to the job need to be improved. The following literature will be framed around the model of transfer of training presented in Chapter 1. A review of known and perceived organizational input barriers which influence training outcomes and conditions of transfer will be discussed. Trainee characteristics and training program input barriers will follow. Then a discussion of organizational, trainee, and training program input facilitators which influence training outcomes and use, will follow. A summary 10 11 of the literature, directions for the current study, and research questions end the chapter. : . !' J E . Organizational barriers are factors within the workplace which inhibit the use of training information or materials. Several studies have made speculations on some organizational barriers to the transfer of training, based on the observations and experiences of human resource development and training professionals, such as Anderson 8 Wexley (1983), Kelly (1982), Parry (1990), and Robinson 8 Robinson (1985). Anderson 8 Wexley (1983) developed an approach that promotes the application of managerial training based on five observations they had made in their years of training. Two of these observations suggest barriers to transfer by the organization. They observed that sometimes the perception of managerial training as a break from work reduces the amount of importance the organization puts on training. The second observation is that top management in an organization is often hesitant to put a high priority on training because they rarely see clear and short-term results reflected in bottom line business terms. These two observations demonstrate a lack of organizational support for facilitating transfer. Kelly (1982) also makes two observations which block the facilitation of transfer by the organization. The first deals *with.the problem that, in most organizations, education is 12 strategically isolated. Therefore, as Kelly (1982) points out, management which views anyone paid to do a peripheral job as a peripheral person will not bring that person's ideas into the work place. So the chances for effective transfer are drastically reduced. The second observation is that many times organizations neither implicitly nor explicitly require that trainers or management education consultants know how to build transfer into their training programs -- that management just doesn't look for people who have demonstrated the capability to design or select courses for transfer of training. Parry (1990) and Robinson 8 Robinson (1985) list the following general barriers that organizations erect against training transfer. Parry found that the climate (norms, culture, and expectations of fellow employees and managers), time and timing (time to do things that were taught in training and opportunities to apply new learning), and the degree of fit (procedures, forms and equipment in agreement with training) often become barriers to transfer. Robinson 8 Robinson (1985) write that it is the boss who does not coach trainees on how to use skills, who does not provide reinforcement, and who does not support use of skills who creates barriers. They also mention that if skill use is punished or given a low priority, that if no feedback is given to trainees after training, and that if time constraints, or a lack of authority is present, the chances of transfer is also diminished. 13 These observations by training experts only give a general idea of some barriers that may or may not occur as a result of an organization. There are a few researchers that are attempting to add more solid evidence to these observations through experiments. Two such studies were reported by Broad (1992). In a survey of top executives by Kotter (1988), Broad (1992) says four major factors that frequently inhibited the success of training were found. Seventy-one percent of the executives identified the lack of involvement by top personnel as a barrier to transfer, 51 percent said efforts to improve (productivity) through training were too centered at the top which resulted in little acceptance from the lower levels, 21 percent report that improvement attempts were too staff centered with insufficient participation by the direct users, and 17 percent believed that expectations of training were too unrealistic -- that managers expected too much too soon. Another study reported by Broad was done by Newstrom (1986). First, a group of 24 trainers identified the major barriers to transfer in their organizations. Responses were grouped into nine categories. From this categorization, a second questionnaire was constructed and administered to 31 trainers from a diverse set of organizations. Their responses ‘were tabulated, averaged, and used to create a rank-ordered list of the most serious barriers to transfer. Resulting barriers, in order of most significant barriers to least, are: 1) Lack of reinforcement by the organization on the job, 2) 14. interference from the work environment, 3) non-supportive organizational culture, and 4) pressure from peers to resist change. Using a similar research method as Newstrom (1986), Broad (1982) found some organizational barriers as well. A select group of Human Resource Development professionals reviewed a list of 74 actions management can take to support transfer of training and were asked to identify and rate the importance of each, as seen in an organization. The actions were grouped into five categories. Differences in the rating of important actions and actual observance of the actions between categories were determined. It was found that job linkage (helping trainees make the connection of training skills to on-the-job situations) and follow-up (making sure skills are being used and assisting in the skill usage, if necessary) aren't used very often in organizations. This is unfortunate because these factors are believed to be most important to the condition of transfer. Huczynski 8 Lewis (1980) reported one major organizational barrier in their comparison of two groups of training course members. Both these groups had attended courses to teach them network analysis and to use this technique in their work. The first group consisted of 17 people and the second group consisted of 32 people. The purpose of the study was to identify differences between the two groups and the differences from those who applied and did not apply training in the two groups. They found that 15 trainees who fail to transfer training did so largely because they did not perceive any organizational support. This study empirically supports this commonly held assumption in the training literature. A final study of the organization's effect on transfer involved students from a graduate program in public administration (Eddy, Glad, 8 Wilkins, 1967). The method consisted of four stages. First, student groups were formed to define and describe important training areas for evaluation. Second, a content analysis of these groups' efforts was done. Third, a questionnaire was developed and administered to learn student attitudes. Finally, an analysis of the student’s organizational climate was made based on the attitude questionnaire. Results revealed that the agencies to which the trainees belonged did not encourage their participation in the training workshop, the degree of reward from improved job performance after training was not made apparent to the trainees, the trainees received resistance from their peers, and that others reported increased organizational dissatisfaction after training (Eddy, Glad, 8 Wilkins, 1967). In their extensive review of the transfer literature, Baldwin 8 Ford (1988) insist that while the above literature tends to stress that positive transfer is highly contingent on factors in the trainees work environment, much empirical work still needs to be done. So few studies do not constitute a complete understanding of the often complex workings of 16 Figure 2 Summary of Organizational Barriers STUDY Anderson 8 Wexley (1983) Kelly (1982) Parry (1990) Robinson 8 Robinson (1985) Rotter (1988) Newstrom (1986) Broad (1982) Huczynski 8 Lewis (1980) Eddy, Glad, 8 Wilkins (1967) BARRIERS FOUND lack of management support no acceptance of educator’s ideas no expectations for trainers negative work climate no time to transfer no degree of fit lack of management support time constraints lack of management involvement improvements centered at the top improvements too staff centered unrealistic expectations lack of reinforcement interference from work environment work environment non-supportive peer pressure to resist change lack of job linkage no follow-up from organization no organizational support participation in training discouraged no reward for use of training skills resistance from peers received organizational dissatisfaction 17 organizational barriers. A summary of these findings is found in Figure 2. W A trainee barrier is some aspect of the trainee’s personality that acts as an inhibitor to the use of training information and materials. As with the barriers from the organization, the studies written about trainee barriers have mostly been suggestive and have not always been based on empirical evidence other than the experience of the training or human resource development professional. Here, the suggestions from these professionals will be reviewed, followed by empirically based studies. Parry (1990) suggests four barriers that the trainee can create or that are inherent in their personality that block transfer of training. He first suggests that motivation plays a role in training transfer. If the trainee already knows the subject matter (or believes that she or he already does), or is unsatisfied with work and the job, then transfer will be hindered. A second factor is ability, meaning if the trainee does not have the ability to learn the material then it will be difficult to apply it in the work setting. Third, the trainee must be attentive. If the person cannot concentrate or has more important matters to think about, then the training itself might not be worth doing. The final factor is relevance. The trainee must see the training as relevant to 18 the job and her or his personal needs in order for the training to be effective. Robinson 8 Robinson (1985) and Spice 8 Kopperl (1984) both suggest two trainee barriers to transfer. Robinson 8 Robinson (1985) find that if trainees see that training teaches things that are contrary to their personal values they may reject it altogether. They also found that if the trainee does not feel confident enough to use skills learned in training, those skills are lost at the point the training is over. Spice 8 Kopperl supports this last point by stating that self-image is the prime generator of performance of skills learned in training. They also add that if the trainee is unwilling to transfer their training skills that this presents a challenge. Some training research has begun to support these ideas by training professionals. In Newstrom's (1986) investigation of transfer barriers, trainees discomfort with change and effort was found to be one of the most significant barriers as perceived by 31 trainers from a diverse range of organizations. The questionnaire of which these trainers were given, also revealed that to some extent the fear of failure, resistance to change, lack of authority to implement change, mandated attendance when the trainee did not want to attend training, and training incompatibility with beliefs and values also played roles as barriers to transfer. This supports the beliefs of Parry (1990), Robinson 8 Robinson (1985), and Spice 8 Kopperl (1984). 19 When developing an approach that promotes application of managerial training, Anderson 8 Wexley (1983) came across an observation that showed an example of a trainee barrier to transfer. They found that even when training programs are well designed and conceived, trainees rarely accept or transfer everything they have learned. Perhaps this can be explained by Watson (1974). He derived data from a survey he conducted after a management training course was completed and the trainees had time to apply their skills. He suggests that application of the training theory was made difficult by several trainee characteristics. The trainees did not accept or transfer what they learned because they did not know where to apply concepts, they found it difficult to identify the key concepts of the course, they had a lack of drive, understanding or willingness, had negative attitudes toward the course or transfer, and the transfer process had discouraged them. Again, Baldwin 8 Ford (1988) would suggest that this review of trainee characteristics is representative of the kinds of studies that have been done -- a wide variety of trainee characteristics are thought to affect transfer but little empirical evidence of ability, personality and motivational effects on transfer has been collected. A summary of trainee barriers can be found in Figure 3. 20 Figure 3 Summary of Trainee Barriers STUDY Parry (1990) Robinson 8 Robinson (1985) Spice 8 Kopperl (1984) Newstrom (1986) Anderson 8 Wexley (1983) Watson (1974) BARRIERS FOUND - lack of motivation - lack of ability - lack of attentiveness - can’t see training relevance - training contrary to values - lack Of confidence - lack of self-image - unwilling to transfer - discomfort with change - fear of failure - resistance to change - lack of authority to change - resistance to training — training incompatible with beliefs - doesn’t accept training - doesn’t know where to apply concepts - couldn’t ID key concepts - lack of drive - lack of willingness - negative attitude - transfer too difficult 21 T'iE' Training barriers are most often recognized as characteristics of the training program, its design, its content of information, or the trainer her or himself which inhibits transfer. The majority of transfer of training literature has concentrated on this input factor in the transfer process (see Figure 1, page 3). Many studies have concentrated on training design facilitators (to be discussed later), so empirical evidence for the training barriers themselves is not so strong. First, a distinction between types of training should be made. Technical training includes teaching someone to type or to operate a mechanical tool. Technical training is almost always successful because the training can be directly related to the workplace and demonstrated on the job. Soft skills training teaches subjects like management, leadership, interpersonal communication, problem solving, and sales techniques. These subjects are very difficult to transfer to the actual job. Most studies on transfer of training concentrate on soft skills training. Georges (1988) offers some suggestions from his experience as a training consultant as to why soft skills training doesn’t "take". The major problem he sees is that skills really aren't what is being taught. What is actually being delivered is information, and that it is not easy to transfer information because there is nothing tangible to 22 bring back to work. He also sees a problem in the way this training is conducted. Trainers often try to teach people soft skills by using steps. Essentially meaning: First you do this, then you do that -- like what would be done for technical training. But teaching soft skills this way is a problem because applying the steps is awkward and rarely works. Real life situations do not necessarily follow orderly steps. Georges (1988) has also found that role- playing exercises do not transfer soft skills to the job. Again, it is not possible to predict how a situation will work on the job, therefore, unless the role playing situation is like the workplace situation, it will be difficult to apply training. Mironoff (1988) supports Georges (1988) ideas on training. Mironoff has also found that too often trainers teach things that do not exist in the trainee's organization. The programs are also too vague. They rarely teach specific skills that can be applied to the trainee's particular situation. They almost never teach that there are right and wrong ways of doing things. Parry (1990) gives a general framework for the most commonly found barriers of training. They fall under four categories. First, there are course design barriers. These include inappropriate methods and media, inadequate facilities and equipment, and less than desirable length of workshop and its objectives. Second is the choice of appropriate emphasis for the course. If, for instance, the base of the course was centered on theory when practice sessions would have been more 23 appropriate for the audience, transfer becomes complicated. Third is the instructor. If the instructor is not perceived to be credible or effective, then often times the trainee's potential for transfer is lost at the workshop. Finally, if follow-up or feedback is not provided, the trainee may give up on transferring information sooner than someone who received assistance. The study by Newstrom (1986), as reported by Broad (1992), supports some of these points and offers additional training barriers. It was found that trainee's perception of impractical training programs or irrelevant training content inhibits transfer. The trainee's perception of the instructor was also significant. Transfer was made difficult when the trainee perceived that the instructor had poorly designed the workshop or just didn't deliver it properly. Finally, it was found that trainees had a hard time applying what they learned in class when they were no longer under the supervision and guidance of the training instructor. Eddy, Glad, and Wilkins (1967) reported two training barriers as a result of an attitudes questionnaire administered to students from a graduate program in public administration. The students felt that the generalist nature of the course they attended was a problem for the job specialists in the class because it was too simple and a problem for the clerical students in the class because the materials weren’t geared for them, either. In addition to this finding, 61 percent of the respondents felt there should 24 Figure 4 Summary of Training Barriers STUDY Georges (1988) Mironoff (1988) Parry (1990) Newstrom (1986) Eddy, Glad, 8 Wilkins (1967) Anderson 8 Wexley (1983) Napier 8 Deller (1985) BARRIERS FOUND - nature of training - training conducted poorly - taught things not found in Job - program too vague - doesn’t teach applicable skills - course design - no appropriate emphasis - instructor - no feed-back - impractical/irrelevant training - instructor - couldn’t apply training — too general - emphasis not appropriate - emphasis not appropriate - program not conducted properly 25 be greater integration of academic and realistic working problems, suggesting that the emphasis of the course needed to be re-worked. Anderson 8 Wexley (1983) also support this point when they found that training activities or exercises may often be too theoretical and inflexible to realistically apply to the job. More emphasis on training a person for realistic work situations they may encounter is needed. A final training barrier was suggested by Napier 8 Deller (1985). They developed two training programs for bank teller supervisors. Each program was divided into three treatment groups: 1) An extensive training group, 2) a moderate training group, and 3) a control group. Results suggest that the supervisors who received extensive training received better scores than the other two groups (a result that was expected). However, the control or no treatment group faired better than the group which received moderate training. The authors concluded that this meant that if the training program is not conducted properly, there might be instances where no training is better than inadequate training. A summary of these training barriers can be found in Figure 4. : ill JE'l'!! Baldwin and Ford (1988) identified two major organizational facilitators that were necessary for the condition of training transfer. These two training input 26 factors were support and opportunity for use (see Figure 1, page 3). Parry (1990) found that a work climate which supported (support being from both the supervisor and peers) newly learned behaviors, gave the trainee time to apply new learning, and whose daily procedures "fit" with the training facilitates the condition of transfer. Many other studies have found evidence of these organizational facilitators, as well. Georgenson (1982) states that while many elements influence skill usage, none has as much impact on the trainee as support from her or his manager. He has found that the following actions demonstrate this support to trainees: 1) Managers who have pre-course discussions of the important material the trainee will be exposed to in training and who discuss how it relates to success on the job, 2) managers who provide feedback to the trainees after the course has been completed and when the trainee is back on the job, 3) managers who work together with the trainees to collect performance data that evaluates how well the trainee is applying learned skills, 4) managers who become the link between training and the trainee's job to make the transition phase easier, 5) managers who help incorporate what is learned in training into the organization, 6) managers who work together with the trainees to change policies so that the use of training can be facilitated. Other authors have also found these factors to be evidence of management support. They have also drawn their 27 own conclusions as to what constitutes managerial support. Berke (1984) makes the following suggestions: 1) Support is evidenced by management that gets actively involved in selecting training programs that fit the specific needs of the trainees, 2) support is evidenced by managers who meet with the trainer to talk about the trainees specific needs, to discuss how the course will be taught, to review the materials that will be used, and to make sure participative learning techniques will be utilized, 3) support is evidenced by management that "sells" others on the value of the upcoming training, and 4) support is evidenced by management that continues to be a model after training is done by encouraging use of learned skills and working with the trainees toward the desired actions. Calvin-Clark (1986) adds that management that makes specific follow-up assignments for the trainees actually helps them to apply and demonstrate their newly acquired training skills. Finally, Mironoff (1988) suggests that training can only elaborate on that which exists in the organization and cannot create new behavior for an organization that won't support training. Part of supporting training is for management to make clear what the organization's mission, values and standards are so people can understand and perform them correctly. One study that backs up these ideas with empirical evidence was conducted by Huczynski 8 Lewis (1980). From a series of questionnaires and interviews of two groups of management course members, the research team discovered the 28 following results: 1) 48 percent of the trainees who had attempted to transfer their training had a pre-course discussion with their supervisor (evidence of managerial support) and, 2) 70 percent of the trainees who had acquired beneficial results from training occurred when their efforts were supported by their managers. The authors of the study concluded that boss's attitudes and support is critical to transfer. Baumgartel 8 Jeanpierre's (1972) study examined the transfer efforts of 240 Indian managers and determined the relationship of these efforts to various facilitating variables including the organizational setting. From the analysis of their surveys they found six organizational climate variables that significantly correlated with the transfer efforts of the trainees. These were: 1) Freedom to set personal performance goals, 2)the higher degree to which higher management is considerate of feelings of lower management, 3) the higher degree to which the organization stimulates and approves of innovation and experimentation, 4) the higher degree to which the organization is anxious for executives to make use of knowledge gained in management courses, 5) the higher degree of free and open communication among the management group, and 6) willingness of top management to spend money for training. At the conclusion of their study, they reported further observations and implications. They found that the organizational climate is the single most important factor affecting efforts to apply 29 Figure 5 Summary of Organizational Facilitators STUDY Baldwin 8 Ford (1988) Parry (1990) Georgenson (1982) Berke (1984) Colvin-Clark (1986) Mironoff (1988) Huczynski 8 Lewis (1980) Baumgartel 8 Jeanpierre (1972) FACILITATORS FOUND support opportunity for use positive work climate time given to apply training job fits with training manager support manager support manager support organizational support pre-course discussion with boss manager support freedom to set goals consideration of lower staff approval of experimentation encourages use of training free and open communication willingness to spend money organizational climate 30 new knowledge. Therefore, training pays off for those companies whose climates are favorable to the changes brought about by training. They suggest that companies who concentrate efforts on managers who are in positions of freedom and responsibility to engage in implementation to facilitate transfer are those which experience better training results.’ A summary of these findings can be found in Figure 5. T l E .1.! ! Just as Baldwin and Ford (1988) have outlined the major facilitating factors the organization has on the condition of transfer of training, so have they also outlined the major trainee facilitating factors to a positive training transfer. There are three such training inputs -- ability, personality, and motivation. Parry (1990) adds attention of the trainee and relevance of training to the trainee's needs. Again, these represent the major groups of trainee facilitators to the transfer of training. The following studies elaborate on these major categories of facilitators. Ehrenberg (1983) found that trainees who consistently and productively communicated what they had learned to other people were those that had enhanced the transfer process. Spice 8 Kopperl (1984) found that those trainees who were able to recognize their unwillingness to transfer training and tried to rectify that situation were those who had better 31 training success. Noe (1986) constructed a model of trainee attributes and attitudes which he believed played a role in training effectiveness. These included: 1) The locus of control of the trainee (that is, does the trainee feel that she or he is in control of the world around her or him or that the world controls the events in their lives), 2) the expectancies of the trainee to the training, 3) the career and job attitudes of the trainee, 4) the reactions of the trainee to skill assessment that is conducted by the trainers, 5) the motivation of the trainee to learn, 6) the motivation of the trainee to transfer, and 7) the trainees' perceptions of the favorability of the work environment to training use. Noe 8 Schmitt (1986) studied this model using 60 educators who had participated in an assessment program but who had not been promoted to principalship yet. The educators, as well as their supervisors, two teachers, and two support staff, completed assessment measures for the educator both before and after the educator's training. The results indicated five correlations. First, trainee reaction to skill assessment was an important antecedent to the satisfaction of the training. Second, trainees who reacted positively to needs assessment measures were more likely to be satisfied with the training content than those who reacted unfavorably. Third, involvement in jobs was an important antecedent of learning and behavior change. Fourth, career planning was found to be important for improvement of job behavior after training. Finally, career decisions may directly affect 32 attainment of success and satisfaction in work. While the results indicate that this model is accurate, the authors suggest strongly that the generalizability of the model may be limited. Tziner, Haccoun, 8 Kadish (1991), while testing a relapse prevention model for the training of 81 Israeli military instructors, discovered two personal characteristics that the instructors possessed that facilitated use of training. They found that the instructors who were rated as internal locus of control (people who feel they are in control of their environment) individuals and who were able to perceive a supportive work environment were judged by their supervisors to display more transfer strategies. The Huczynski 8 Lewis (1980) study described earlier found two significant trainee characteristics that had a positive effect on transfer. First, 35 percent of the trainees in their study were those who attended the training on their own. This showed commitment on their part to the training. They also found that all the trainees who attempted to transfer training continued to believe that training would be of use to them on the job and in the future. In other words, they were able to perceive the benefits of training. Eddy, Glad, 8 Wilkins (1967) also found that their trainees who saw the long-term benefits of training and got personal satisfaction for training were more likely to transfer their skills. 33 Figure 6 Summary of Trainee Facilitators STUDY Baldwin 8 Ford (1988) Parry (1990) Ehrenberg (1983) Spice 8 Kopperl (1984) Noe (1986) Noe 8 Schmitt (1986) Tziner, Haccoun, 8 Radish (1991) Huczynski 8 Lewis (1980) Eddy, Glad, 8 Wilkins (1967) Baumgartel 8 Jeanpierre (1972) FACILITATORS FOUND ability personality motivation attention ability trainee communication recognition of unwillingness locus of control expectancies of trainee career attitudes reaction to skill assessment motivation to learn motivation to transfer perceives favorable work environment positive reaction to skill assessment positive reaction to needs assessment involvement in job career planning satisfaction in job internal locus of control perceive work support attend training on own believes training is useful perceives long term benefits income had opinions of training felt skills had been learned 34 Baumgartel 8 Jeanpierre (1972) found the following characteristics of trainees which resulted in an effort to tappay training. The one variable that showed a statistically significant relationship with the self reported effort to ‘transfer was income. Those with a higher income reported more often that they had made an effort to apply training. However, other analyses revealed other trainee factors to transfer. Results from an analysis of within group variance inferred that those trainees who had had an opinion about a particular aspect of the training were also those who more often reported a transfer effort. Individual reactions to particular training programs were shown to have high and positive relations with scores on the scope of innovation (attempts to transfer) variable. In other words, trainees who felt that they had learned skills relevant to their job and had gained self confidence/raised personal goals as a result of this training, reported more attempts to apply training. The summary of the trainee facilitators is found in figure 6. W The training facilitators that were found to be essential to the condition of transfer in Baldwin 8 Ford's (1988) model were the incorporation of learning principles in training, the proper sequencing of training material, and the job relevance of the training content. 35 Parry (1990) found that the course design can also be an important facilitator to the successful transfer of training. In the definition of course design he included the tappropriateness of the training methods and media, the use of lappropriate facilities and equipment, and the appropriate length of objectives. Cureton 8 Cureton (1983) also found that the appropriateness of the program for the trainees was essential to successful transfer. They write that programs that focus on the actual needs of the trainee verses the perceived needs of the trainer were the programs that had more trainees successfully transfer the taught skills. Feldman (1981) also finds that response to actual needs is an essential facilitator of transfer. A second training facilitator that was cited by Parry (1990) is the emphasis of the course. He found that issues such as theory vs practice, knowledge verses skills, and talking verses doing were important to transfer. Much has been written about this facilitating training factor. However, there is not one single emphasis that seems to be the best for all training situations. As a result, there are studies which have discovered that training courses with a relapse prevention model as the emphasis of the course was highly successful in transferring training (Marx, 1982: 1986; Tziner, Haccoun, 8 Kadish, 1991), training courses with post- training strategies as the emphasis of the course as being highly successful (Wexley 8 Baldwin, 1986: Gist, Bavetta, 8 Stevens, 1990: Kruger 8 May, 1985), training courses with 36 interim project training design emphases as highly successful (Harmon, 1974), and training courses with modeling strategies as their main emphasis as being highly successful (Byham, Adams, and Kiggins, 1976). The main point of all this being that the emphasis of the course is a very important facilitator to transfer and special care should be taken to make sure the emphasis of the course meets the need of the trainee. A third training facilitator described by Parry (1990) was the instructor of the course. Instructors who are seen as credible and effective have a great impact on how the course is received and eventually transferred. Kelly, Orgel, 8 Baer (1985) and Lefton 8 Buzzotta (1980) support this idea, as well. Two studies demonstrated the impact an instructor can have on her or his trainees. Eden 8 Shani (1982) and Eden 8 Ravid (1982) tested the Pygmalion effect, or enhanced learning as a result of the high expectations of the training instructor. Eden 8 Ravid first tested this approach on 61 personnel in the Israeli Defense Forces. Trainees were split into three conditions randomly: 1) High expectancy, 2) control instructor expectancy, and 3) insufficient information. Reported self-expectancy was measured before, during and after the course. Results confirmed the Pygmalion effect that inducing high self expectations enhances trainee ‘performance. Eden 8 Shani (1982) confirmed the Pygmalion effect in a second study as well. They found that trainees whom instructors had induced to expect better performance 37 Figure 7 Summary of Training Facilitators STUDY Baldwin 8 Ford (1988) Parry (1990) Cureton 8 Cureton (1983) Feldman (1981) Marx (1982 8 1986) Tziner, Haccoun, 8 Kadish (1991) Wexley 8 Baldwin (1986) Gist, Bavetta 8 Stevens (1990) Kruger 8 May (1985) Harmon (1974) Byham, Adams, Kiggins (1976) Kelly, Orgel, 8 Baer (1985) Lefton 8 Buzzotta (1980) Eden 8 Shani (1982) Eden 8 Ravid (1982) FACILITATORS FOUND incorporation of learning principles sequencing of training materials job relevance of training content course design emphasis of course instructor follow-up appropriateness of program response to actual needs course emphasis course emphasis course emphasis course emphasis -course emphasis course emphasis course emphasis instructor instructor instructor instructor 38 scored significantly higher on tests, exhibited more positive attitudes, and displayed more positive leadership behavior. A final training facilitator that was described by Parry (1990) was follow-up. Parry found that trainers who got feedback on trainee’s performance after training and took appropriate actions, with the trainee or the course design could significantly improve the chances for training transfer. The summary of the training facilitators can be found in Figure 7. E l :1. !' There are a few areas of interest to the current study that can be derived from the literature. From these areas of interest, the hypotheses for the study have been derived. The first issue to be examined is that of the use of the training information to the job. This is a variable which is not often measured in the literature. More often, what is measured is how well the attendee liked the program or their intent to use the information. Actual use on the job is not often measured. Perhaps this is due in part to the statistics which tell us that the training fails to transfer 10 percent of the time. In light of this estimate the first hypothesis is: Hypothesis_1: Use of the training information and materials will be low. Fewer than 10 percent of the subjects will report using the information. 39 Much of the literature on the organization focused on management's support or non-support for the use of training. The conclusions of both the organizational barriers and facilitators is that any support management can give to their trainees is critical to the successful transfer of training. Other organizational factors such as time, money, peer support, and personal freedom can also assist transfer. The second hypothesis, drawn from these particular studies, is: Hypothesi§_2: Use of the training information and materials occurs more frequently for those who reported the highest number of organizational facilitating factors. The literature on the trainee's influence on transfer revealed a few characteristics that researchers found that describe trainees who were more likely to succeed in the process of transfer. Essentially, factors that were related to motivation, willingness, ability, fulfilled expectancies, and commitment promoted transfer. The current study hypotheses that: nypgtng51§_1: Use of the training information and materials occurs more frequently for those who report more facilitating personal characteristics. The fourth hypothesis focuses on the training experience. The literature suggested that a positive training experience could help the trainee transfer the information back to the workplace. The training experience includes factors such as 40 the instructor, the interactions with other trainees, or the setting in which the training took place. Hypothg§i§_4: Use of the training information and materials occurs more frequently for those who reported more positive training experiences. Another feature of training that was found to be critical to the transfer process was the practicality of the training. This was defined as meeting the needs of the trainees and their organization, setting standards that were reasonable for the trainee to meet, and demonstrating the applicability of the training to the workplace. If the training is not seen as practical then, at best only some of the training will transfer and at worst, none of it will transfer. Therefore: Hypothg§1§_§: Use of the training information and materials occurs more frequently for those who reported more practicalities of the training format. A final feature of training that is important to the current study was the relevance of the training content. This applied more to what was taught than how it was taught. For instance, if the materials presented during training were too technical for the audience, then the resulting lack of understanding would cause problems for transfer. The final hypothesis for the study is: fiypgthgsis_§: Use of the training information and materials occurs more frequently for those who reported more relevancies of the training content. These hypotheses guide the rest of the study. CHAPTER III METHODS Subjects A list of 271 people was provided to the Michigan State University research team by the Educational Design Team at the University of Michigan. The list consisted of people who had taken one or more of the Educational Design Team's workshops, had been to a workshop where a member of the Educational Design Team had been a speaker, and/or had received materials from the Educational Design Team. Records of workshop participants collected by the Educational Design Team indicated the affiliation of participants, (E= EPA staff member, I= American Indian participant, O= other or unidentified people, and S= people affiliated with other state offices) and what agency they had worked for, but these were not criteria for subject selection. Instead, attempts were made to contact all 271 people on the list. Three members of the Michigan State research team made these calls to each name on the list. When necessary, several calls were made to locate people who had moved or whose numbers had changed. These initial calls were made to explain the current study and to request a time and date in which to do an interview. A 41 42 total of 154 people from the Educational Design Team’s list were located and interviews were set up with them. After the interviews had been set up, the Michigan State research team was informed that the people on the Educational Design Team’s list had attended workshops of two separate emphases. One was conducted entirely by the Educational Design Team, using their materials exclusively, and had a train-the-trainer emphasis. The other type of workshop had sponsors other than the Educational Design Team, only used certain components of the Educational Design Team's materials, and had no train-the-trainer emphasis. It was not clear from the Educational Design Team's records which participants had been to which workshop emphasis. The emphasis that the current study was interested in was the one in which the train-the-trainer approach was utilized and where the participants had received the Educational Design Team’s materials. A series of questions at the beginning of the survey (to be detailed in the next section) were designed to single out those participants who had been to the workshops of this study's interest. From this procedure a total of 85 participants became the subjects for this study. W The survey was designed to obtain information on the percent of the subjects who made use of the training information and materials they had received during the 43 Educational Design Team’s workshops. It also sought to determine which factors influenced the use of training information. Because the generalization (or use) of training information is a complex process, several measures of use of training were developed for the survey. Three main questions probing 1) the use of training to share with others, 2) the use of training in decision-making, and 3) the use of training to train other people, asked the subject to respond affirmatively or negatively to each of the questions. A series of sub-questions (nine in all) stemming from positive responses to the main questions, probed further 1) the use other people made of training information and materials, and the use other people made to train others, to be recorded on an affirmative to negative scale, 2) the extent of use of the training video in decision-making, the extent of use of the training manual in decision-making, and the extent of use of supplemental material in decision-making, to be recorded on a seven-point Likert-type scale ranging from no use to extensive use, and 3) the extent of use of the training manual to train others, the extent of use of the training video to train others, the extent of use of the training format to train others, and the extent of use of supplemental materials to train others, to be recorded on a seven-point Likert-type scale ranging from no use to extensive use. The percentage of the subjects who made use of the training information and materials was measured separately for each of these questions. 44 To measure which factors had the greatest impact on the use of the training information, correlations between the use questions and the responses to an open-ended question were run. The open-ended question, located at the end of the survey, was designed to extract as many comments that the ,subjects had about the factors which they believed most influenced their use of the training information. The following set of procedures resulted in the completed survey. First, the Michigan State research team met to discuss the possible questions that would appear on the survey. Also discussed was the definitions of use of the training information and the variables that may affect the use of training information. It was then decided that the team would split up into two groups, each draft a survey, then come together to combine these efforts. The second meeting of the Michigan State research team concentrated on making changes to the draft survey. This included changing the wording of questions for clarity and changing the flow of the survey. Questions were also eliminated or condensed as a result of the group discussion. A test run of the second draft was conducted on six subjects. Four of the subjects were Ph.D. students in the Department of Communication at Michigan State. One professor from the same department also participated. The final participant was a professor in the Department of Resource Development at Michigan State, who was quite familiar with the type of training and subjects the team was studying for this 45 project. The purpose for conducting this test run was to further improve the clarity and flow of the instrument. A few changes were made as a result of this test run and the final survey that was used to interview the subjects of this study can be found in Appendix A. The first paragraph of the survey indicated who made the call to the subject as well as who set up the interview with the subject. The second two paragraphs, which were read to each subject, indicated the purpose of the study to the respondents as well as gave the subject the guidelines for their confidentiality. The subjects were also given the opportunity to receive a copy of the results as well as have any of their questions answered. Because it was not known which workshop experience the subject had attended, a series of questions asked the respondents to verify the location, the topic, the materials they had received, and if they had been to a train-the-trainer type workshop. A series of questions that were not used for analysis but were used to update the profile of the subjects for another study, were also added to the survey (question 4- 10' 14' 20' 8 21). Wt: Interviewers for the current study were members of the Michigan State research team. They included two Ph.D. students, three Masters students and one professor, all from 46 the Department of Communication at Michigan State. All interviewers committed themselves to participating in data collection for one week of the scheduled interviews. Two participants continued interviewing up to two weeks after the initial week, to cover those subjects whose schedules did not fit into the initial week. Participation was voluntary. All of the interviewers were familiar with the content of the surveys and the intent of the study, as they had helped develop the project and co-write the survey. Therefore, they did not need to be briefed in these areas. In other ways the interviewers were trained. Interviewer training was done in a half-day session. First, all the interviewers read the survey together, question by question, to make sure that everyone was clear on how to score each answer and how to follow the questioning route properly. Then the interviewers were instructed on the procedure for recording the responses to open-ended question at the end of the survey. Interviewers were encouraged to acquire as many reasons as the subjects could identify for the use or non-use of the training information. The interviewers were allowed to use generic probes, if necessary, such as "can you expand on that point?" or "was there any other organizational factors that influenced your use of the training information?" Interviewers were instructed to not lead respondents to an expected answer. After this briefing, the interviewers were given their interview schedules and the surveys. 47 All the interviews were conducted on the telephone at a time and day previously arranged with each respondent. If the subjects were not available at the designated time, the interviewers made attempts to call them at a later time. In some instances the subjects were not ever located, even after three weeks of attempts. That total was small -- only two subjects who had previously agreed to participate in the study could not be re-contacted. In all other cases the interviewer identified her or himself, mentioned the person who set up the interview, and briefed the respondent about confidentiality. For requests for the survey, the interviewer noted the subject's address. Questions about the study were then answered by the interviewer. The next step in the survey procedure was to determine whether, or not, the respondent should be questioned and become a part of the study. If the subject indicated that they had been 1) to a workshop with a train-the-trainer emphasis, and 2) had received a manual and or a video on one of three subjects (inspector training, permit writing, or RCRA orientation), then they were asked to continue with the survey by answering questions. Subjects who did not meet both the above criteria were dropped from the study. Through this procedure, 67 of the 152 scheduled interviewees were dropped from the study, leaving a total of 85 interviewees. The procedure for recording answers to the survey questions varied according to the question. Some questions 48 required the interviewer to circle one of a set of possible responses, others required that numbers be filled into blanks, and Likert-type scales were also used. Three sets of questions were asked only when a certain response had preceded them. Answers to the open-ended question at the end of the survey were written out by each interviewer. After the questioning was completed, further questions were answered by the interviewer where applicable. IIBRIEEDL_Q£_Q§LB The variable relationships that were analyzed in this study stem from the model of the transfer process (Baldwin 8 Ford, 1998) shown in Figure 1, Page 3. The model shows that three main training inputs -- trainee characteristics, training design, and work environment -- have both direct and indirect links to the conditions of transfer (the use of training materials and information). Several trainee characteristics, training design characteristics, and work environment characteristics found to have direct and indirect links to use of training were covered in Chapter 2. The present study examined how workshop experiences, trainee characteristics, human and non-human organizational factors, practicality of the program format, and the relevance of training content, influenced the generalization (or use) of t:he Educational Design Team's training information and 49 materials. Also examined was the percent of the subjects who made various uses of the training information and materials. A number of coding procedures took place when the surveys were completed. The measures of use (as represented by questions 11, 12, 13, 15, 16, 17, 18, 19, 22, 23, 24, and 25 on the survey in Appendix A) were coded as they appeared on the survey. "Yes" answers always had the number 1 in front of it, "no" answers always had the number 2 in front of it, and "other/don't" know always had the number 3 in front of it. Hence, if an answer of "no" was given, the code of 2 was recorded. For the measures of use that had utilized a seven- point Likert-type scale of response, the number which was indicated by the subject was the number that was recorded for analysis. Open-ended responses represented the training input variables examined in this study. The coding of these open- ended question responses required that a four-part procedure take place. All the responses were recorded on separate index cards. These cards were placed into groups of similar responses. Based on the literature about training input variables, seven categories of responses were derived. These were labeled 1) workshops experiences, 2) trainee characteristics, 3) human organizational factors 4) non-human organizational factors, 5) practicality of program format, 6) relevance of training content, and 7) other factors. Then, the categories were each split in two, representing both 50 Figure 8 Categories for the Open-Ended Responses ___gCate .orx Edemjam. Positive Workshop Experiences A Negative Workshop Experiences B Positive trainee Characteristics C Negative Trainee Characteristics D Positive Human Organizational Factors E Negative Human Organizational Factors F Positive Non-Human organizational Factors G Negative Non-Human Organizational Factors H Positive Practicality of Program Format I Negative Practicality of Program Format . J Positive Relevance of Content K Negative Relevance of Content L Positive Other M Negative Other N 51 positive and negative responses. Thus, fourteen categories were established in all (see Figure 8). For every survey there were some categories which had no responses. In these cases a zero was noted for that category. A final procedure for coding the open-ended responses was to reverse code those categories that were labeled negative in front of them (code names B, D, F, H, J, L , and N). This was in keeping with the way the hypotheses were stated for this study. Hypothesis 1 concerns the percent of subjects who made use of the Educational Design Team's training information and materials. It stated: Hypothesis 1: Use of training will be low. Fewer than 10 percent of the subjects will report using the information and materials. This hypothesis was analyzed through a series of frequency measures. Frequencies were run on the following variables of use: 1) Question 11 (from the survey, see Appendix A) -- subject's use of training to share with others, 2) Question 15 -- subject's use of training for decision- making, 3) Question 19 -- subject's use of training to train others, 4) Question 12 -- other's use of training for decision-making, 5) Question 13 -- other's use of training to train others, 6) Question 16 -- subject's use of video for decision-making, 7) Question 17 -- subject's use of manual for decision-making, 8) Question 18 -- subject's use of 52 supplemental materials for decision-making, 9) Question 22 -- subject's use of manual to train others, 10) Question 23 -- subject's use of video to train others, 11) Question 24 -- subject’s use of format to train others, and 12) Question 25 - - subject's use of supplemental information to train others. From these frequencies the percent of the total sample who had made various uses of the training information and materials were derived. For those sub-questions in which answers were only given in response after a positive response was acquired for a main question, the percent of the selected group was given in addition to the percent of the overall sample. Again, the use of training is too complicated a process to be measured by just one variable, hence a number of percentages were derived from this treatment and each was treated as separate measures. To determine which input factors influenced the use of the training information and materials, five additional hypotheses were constructed. These hypotheses and the means by which they were tested are listed below. Hypothesis 2: Use of the training information and materials occurs more frequently for those who reported the highest number of organizational facilitating factors. First, the subjects were sorted into 1) use, and 2) no use, based on their responses to the variables subject's use of training to share with others, subject's use of training for decision-making, and subject's use of training to train others. T-tests were then run for the two groups for the 53 input variables positive human organizational factors, positive non-human organizational factors, negative human organizational factors, and negative non-human organizational factors. Twelve T-tests were calculated through this procedure. Then, the subjects who had indicated use of training for decision-making and use of training to train others were isolated for further study. The variable subject's use of training for decision-making had three sub-variables connected with it. These sub-variables were extent of subject's use of video for decision-making, subject's use of manual for decision-making, and subject's use of supplemental materials for decision-making. The variable subject's use of training to train others had four sub-variables connected with it. These sub-variables were extent of subject's use of manual to train others, subject's use of video to train others, subject’s use of format to train others, and subject's use of supplemental material to train others. Correlations were run between these sub-variables and the input variables, positive human and non-human organizational factors, and negative human and non-human organizational factors, for the isolated group. Through this procedure twenty-eight significance levels were calculated. Hypothesis 3: Use of the training information and materials occurs more frequently for those who report more facilitating personal characteristics. 54 The same procedure for a t-test and correlation, presented in Hypothesis 2, was used for Hypothesis 3. The procedural difference made was to change the input variables that were tested. Instead of organizational factors, positive and negative trainee characteristics were tested. Hypothesis 4: Use of the training information and materials occurs more frequently for those who reported more positive training experiences. The same procedure for a t-test and correlation, presented in Hypothesis 2, was used for Hypothesis 4. Changed were the input factors from organizational factors to positive and negative training experiences. Hypothesis 5: Use of the training information and materials occurs more frequently for those who reported more positive practicalities of training format. The same procedure for a t-test and correlation, presented in Hypothesis 2, was used for Hypothesis 5. Changed were the input factors, from organizational factors, to positive and negative practicalities of the training format. Hypothesis 6: Use of the training information and materials occur more frequently for those who reported more positive relevance of the training content. The same procedure for a t-test and correlation, presented in Hypothesis 2, was used for Hypothesis 6. The input variables were changed to positive and negative relevance of training content. CHAPTER 4 RESULTS This study focused on the percentage of cases in which transfer and or use took place years after an environmental training workshop had been conducted. It also examined several variables which are believed to influence the use of training information and materials. This chapter reports the findings for each of the six hypotheses of study. W To determine what percentage of the subjects had made use of the training information and materials, a series of frequency measures were taken on twelve variables of use derived from questions 11, 15, 19, 12, 13, 16, 17, 18, 22, 23, 24, and 25 of the survey (see Appendix A). To review these variables were: 1) Subject's use of training to share with others, 2) subject's use of training for decision-making, 3) subject's use of training to train others, 4) other's use of training for decision-making, 5) other's use of training to train others, 6) subject's use of video for decision-making, 7) subject’s use of manual for decision-making, 8) subject's use of supplemental materials for decision-making, 9) 55 56 subject's use of manual to train others, 10) subject’s use of video to train others, 11) subject's use of format to train others, and 12) subject’s use of supplemental materials to train others. Results of the frequency measures run on the use variables and the percentages of the total sample for each variable is reported in Figure 9. The variable subject’s use of training to share with others resulted in a high frequency of use versus no use. Seventy-five or 88.2 percent of the subjects reported use for this variable. Ten or 11.8 percent of the subjects reported no use. The variable subject’s use of training for decision- making scored much lower in frequency of use. Thirty-nine or 45.9 percent of the subjects reported use for this variable whereas 44 or 51.8 percent of the subjects reported no use. The frequency of use went up again for the variable subject's use of training to train others. Sixty-three or 74.1 percent of the subjects reported use and 22 or 25.9 percent of the subjects reported no use. Figure 9 also reports the frequencies of the sub-use variables other's use of training for decision-making, and other's use of training to train others. This was done for the group of subjects who indicated use for the main use variable, subject's use of training to share with others. Two percent-of-sample columns are shown in this figure. The first indicates the percent of the total sample. The second indicates the percent of the group who indicated use for the variable, subject's use of training to share with others. 57 Figure 9 Results of the Frequency Tests Run on the Variables of Use Subject's use of training to share with others miahle Use: 75 No Use: 10 Subject's use Use: 39 NO Use: 44 Subject's use Use: 63 No Use: 22 Others use of Use: 43 No Use: 12 Others use of Use: 30 No Use: 31 Subject's use No Use Extensive Use Subject's use No Use Extensive Use Percent of Sample: Percent of Sample: of training for decision-making Percent of Sample: Percent of Sample: others of Sample: of Sample: of training to train Percent Percent training for decision-making Percent of Sample: Percent of Sample: training to train others Percent of Sample: Percent of Sample: of video for decision-making 1) 16 Percent of Sample: 2) 11 3) 4) 5) 6) 7) HOGUIN of manual for decision-making 1) Percent of Sample: 2) 3) 4) 5) 6) 7) thtDSOQw 88.2 11.8 45.9 51.8 74.1 25.9 50.6% 14.1% 35.3% 36.5% 18.8% 12.9% 2.4% 5.9% 7.1% 1.2% 3.5% 8.2% 10.6% 10.6% 7.1% 4.7% 3.5% \\ \\\\\\\ \\\\\\\ 7.7% 17.9% 23.1% 23.1% 15.4% 10.3% 7.7% yariable Subject's No Use Subject’s No Use Extensive Subject’s NO Use Extensive Subject's No Use Extensive Subject's No Use Extensive use use Use use Use use Use use Use 58' Figure 9 -- continued supplemented materials for decision-making 5 5.9% / 3 3.5% / 4 4.7% / 5 5.9% / 6 7.1% / 8 9.4% / 10 11.8% / manual to train others 3 3.5% / 2 2.4% / 10 11.8% / 9 10.6% / 15 17.6% / 12 14.1% / 12 14.1% / video to train others 14 16.5% / 3 3.5% / 5 5.9% / 6 7.1% / 11 12.9% / 11 12.9% / 13 15.3% / format to train others 11 12.9% / 8 9.4% / 10 11.8% / 6 7.1% / 11 12.9% / 12 14.1% / 5 5.9% / supplemented materials to train others 3 3.5% / 4 4.7% / 4 4.7% / 16 18.8% / 12 14.1% / 9 10.6% /1 15 17.6% / 12.8% 7.7% 10.3% 12.8% 15.4% 20.5% 25.6% 4.8% 3.2% 15.9% 14.3% 23.8% 19.0% 19.0% 22.2% 4.8% 7.9% 9.5% 17.5% 17.5% 20.6% 17.5% 12.7% 15.9% 9.5% 17.5% 19.0% 7.9% 4.8% 6. 3% 6. 3% 25. 3% 19. 0% 23. 8% 59' Results show that 43 of the subjects were aware of others use training for decision making. That is equal to 50.6 percent of the total sample and 57.3 percent of the selected sample. Twelve of the subjects said that others did not use the training information and materials for decision making. That is equal to 14.1 percent and 16 percent of the two total samples respectively. Slightly more subjects reported no use for the variable, other's use of training to train others than the variable, other’s use of training to train others. Thirty-one reported no use and 30 reported use. The resulting percentages were 36.5/41.3 percent versus 35.3/40.0 percent. Of those subjects indicating use of training for decision-making, frequencies were run for the sub-use variables 1) subject's use of video for decision-making, 2) subject’s use of manual for decision-making, and 3) subject's use of supplemental materials for decision-making. These sub- use variables were measured on a seven-point scale ranging from 1) no use to 7) extensive use. The frequency column shows the number of respondents who answered for each number 1-7. The results of the sub-use variable, subject’s use of video for decision-making, shows that the majority of the subjects reported 1 and 2 on the seven-point scale. No one reported a 6, and only one person reported a 7 on the seven- point scale. The results of the sub-use variable, subject's use of manual for decision-making demonstrated that most subject's responses fell in the 3,4, and 5 range and slightly 60 fewer subjects reported responses falling in the 1 and 2, and 6 and 7 range. Overall, this variable was fairly evenly distributed. The results of the third sub-use variable, subject's use of supplemented materials for decision-making indicated that more of the responses were found at the top of the scale (6 and 7) than at the bottom (1 and 2), and that the middle range of the scale (3,4,5) contained slightly fewer scores than the top, but more than the bottom. Subjects that indicated use of training to train others also answered sub-use questions subject's use of manual to train others, subject's use of video to train others, subject's use of format to train others, and subject's use of supplemented materials to train others. The variable, subject's use of manual to train others resulted in roughly equally high frequencies on numbers 3-7 of the seven- point scale. Few responded with a 1 or 2. Results for the variable, subject's use of video to train others shows a high concentration of responses at 1 and the top range of the scale, 5,6,and 7. The frequency of responses for the variable, subject's use of format to train others were spread rather evenly along the seven-point scale. Finally, responses to the variable, subject's use of supplemented materials to train others resulted in 4-7 most frequently reported and 1-3 not so frequently reported. 61 W To determine what, if any, influence organizational factors have on the use variables, both t-tests and correlations were run for this hypothesis. Results of the t- tests are reported first. T-tests were run for the input variables (positive and negative organizational human factors, and positive and negative organization non-human factors) and the main use variables (subject's use of training to share with others, subject's use of training for decision-making, and subject's use of training to train others). The results of these t- tests are shown in Figure 10. The direction of the difference between sample means was predicted beforehand. It was expected that the positive input variables of the first sample reflect a population whose mean is greater than the mean of the second population (mul > mu2). The opposite possibility (mul < mu2) was expected for the negative input variables. The variables of which this was not the case were noted with (~). To reject the null hypothesis, the value of the t-score had to be greater than the value of the table t (approximately 1.665). In only three cases was the null hypothesis rejected. The rejection of the null hypothesis is noted by a single asterisk (*). These cases were the positive non-human organizational factors for the variable, subject's use of training to share with others, negative human organizational factors for the variable, 62 Figure 10 Summary of Significant Mean Score Comparisons for Use Variables and Use Variable Correlations with Organizational Factors yariable Hean_Score. thained.t Subject's use of training to share with others - human organizational factors a) .280 ~ -l.4l7 b) .100 ~ + non-human organizational factors a) .307 4.652 b) .000 Subject's use of training for decision-making - human organizational factors a) .256 ~ -.081 b) .244 ~ + non-human organizational factors a) .179 ~ -1.54 b) .356 ~ Subject's use of training to train others + human organizational factors a) .270 ~ -.461 b) .364 ~ - human organizational factors a) .143 2.196 b) .591 + non-human organizational factors a) .349 3.455 b) .045 63 Figure 10 -- continued fl . l] : J l' 2 EE' . l 5' 'E' I J Subject's use of video for decision-making - human organizational .0859 ~ .302 factor Subject’s use of manual for decision-making + human organizational .387 .007* factor Subject’s use of supplemental materials for decision-making + human organizational -.0221 ~ .447 factor - human organizational .0512 ~ .378 factor + non-human organizational -.1559 ~ .172 factor - non-human organizational .1940 ~ .118 factor Subject's use of manual to train others - human organizational .1097 ~ .196 factor - non-human organizational -.2129 .047 * factor Subject's use of video to train others - human organizational .0161 ~ .450 factor Subject’s use of format to train others ‘ + human organizational -.0946 ~ .230 factor - human organizational .1803 ~ .079 factor - non-human organizational -.0473 ~ .356 factor Subject’s use of supplemental materials to train others + human organizational -.0436 ~ .367 factor + non-human organizational -.2579 ~ .021 * factor NOTE: ~ reversal of expected correlation * = significant relationship 64 subject’s use of training to train others, and positive non- human factors for the variable, subject's use of training to train others. Correlations were run for the portion of subjects who had indicated use for the main variables, subject's use of training for decision-making and subject's use of training to train others. The correlations were run for the sub-use variables (subject’s use of video for decision- making, subject's use of manual for decision-making, subject's use of supplemental materials for decision-making, subject's use of manual to train others, subject's use of video to train others, subject’s use of format to train others, and subject's use of supplemental materials to train others) and the input variables (positive and negative human organizational factors, and positive and negative non-human organizational factors). As hypothesized, the resulting correlation coefficients should be positive where the input variables are positive, and negative where the input variables are negative. However, the opposite was found in several cases. These are indicated with (~). The subject's use of manual for decision- making was the only set of correlation coefficients that correlated in the directions that were predicted. The significance levels show that only three correlations proved to be significant (p <.05). These correlations were indicated by a single asterisk (*). The significant correlations in detail were, positive human organizational factors and the subject's use of manual for decision making, negative non-human organizational factors 65 and the subject's use of manual to train others, and positive non-human organizational factors and the subject's use of supplemental material to train others. 8W T-tests and correlations were run to determine what, if any, influence trainee characteristics had on the use variables. These tests were run in the same fashion as presented in the results of Hypothesis 2, only the input variables were changed to positive and negative trainee characteristics. The t-test results for Hypothesis 3 are shown in Figure 11. As in Hypothesis 2, the mean scores for sample (a) should be greater than the mean scores for sample (b) when the input variable is positive, and visa versa when the input variable is negative. This was not the case for positive trainee characteristics for the variable, subject's use of training to share with others. This, however, was the only instance in which that result occurred. Two of the six null hypotheses were rejected. These were for negative trainee characteristics for the variable, subject's use of training to share with others, and negative trainee characteristics for the variable, subject's use of training to train others. The correlation results for Hypothesis 3 are also shown in Figure 11. 66 Figure 11 Summary of Significant Mean Score Comparisons of Use Variables and Use Variable Correlations with Trainee Characteristics yariabla Haan_Saora thainad_t Subject’s use of training to share with others + trainee characteristics a) .093 ~ -5.632 b) .200 ~ — trainee characteristics a) .133 1.702 * b) .800 Subject's use of training to train others -trainee characteristics a) .111 2.26 * b) .500 Variable Correlation_soaffiaient Significanaa_Laxal Subject's use of video for decision-making + trainee characteristics -.2841 ~ .040 * Subject's use of manual for decision-making + trainee characteristics -.2005 ~ .111 Subject's use of supplemental materials for decision-making + trainee characteristics -.1364 ~ .204 - trainee characteristics .1658 ~ .157 Subject's use of manual to train others + trainee characteristics -.1320 ~ .151 Subject's use of video to train others + trainee characteristics -.2146 ~ .046 * Subject's use of supplemental materials to train others - trainee characteristics -.2885 .011 * NOTE: ~ * reversal of expected correlation significant relationship 67 On the whole, the correlation coefficient results were the opposite of what was hypothesized: that is, the subject's use of format to train others, and the subject’s use of supplemental materials to train others were the only variables that correlated completely as predicted. Three correlations proved to be significant in this hypothesis. These were positive trainee characteristics and the subject's use of video for decision making, positive trainee characteristics and the subject's use of video to train others, and negative trainee characteristic and the subject's use of supplemental materials to train others. W After the input variables were changed to positive and negative workshop experiences, t-test and correlations were again run to determine what, if any, of these two variables had on the use variables. The t-test results, show in Figure 12, revealed that the mean scores for sample (a) were less than sample (b) for positive workshop experiences for the variable, subject’s use of training to train others. The opposite should have been true to be consistent with the hypothesis. Out of the six null hypotheses tested, only two were rejected. These were, the positive workshop experiences variable for the variable subject's use of training to share with others, and positive 68 Figure 12 Summary of Significant Mean Score Comparisons of Use Variables and Use Variable Correlations with Workshop Experiences lariabla HaaLSaare Obtained; Subject's use of training to share with others + workshop experiences a) .680 1.969 * . b) .300 Subject's use of training for decision-making + workshop experiences a) .846 1.723 * b) .467 Subject’s use of training to train others + workshop experiences a) .603 ~ -.464 b) .727 ~ lanabla W W Subject’s use of video for decision-making + workshop experiences -.0275 ~ .434 Subject's use of manual for decision-making + workshop experiences -.0786 ~ .317 - workshop experiences .0824 ~ .309 Subject’s use of manual to train others - workshop experiences .3509 ~ .002 * Subject's use of video to train others - workshop experiences .2716 ~ .016 * Subject's use of format to train others - workshop experiences .1711 ~ .090 Subject's use of supplemental materials to train others + workshop experiences .2951 .009 * - workshop experiences .2287 ~ .036 * NOTE: ~ reversal of expected correlation * = significant relationship Vi 69 workshop experiences for the variable subject’s use of training for decision-making. The correlation results, shown in Figure 12, showed once again that the correlation coefficient results, on the whole, were opposite of what they should have been. Only the subject's use of supplemental material for decision-making had coefficients consistent with what was predicted. Out of the fourteen correlations run, four were significant. The negative workshop experiences and the subject’s use of manual to train others, the negative workshop experiences and the subject’s use of video to train others, the positive workshop experiences and the subject's use of supplemental materials to train others, and the negative workshop experiences and the subject's use of supplemental materials to train others, were the four significant correlations. W T-tests and correlations were run to determine what, if any, influence the practicality of the training format had on the use variables. The T-test results for Hypothesis 5 are shown in Figure 13. Again, the mean scores for sample (a) should be greater than the mean scores for sample (b) when the input variable is positive, and visa versa when the input variable is negative. This was not the case for practicality of format for the variable, subject's use of training for decision-making, and 70 Figure 13 Summary of Significant Mean Score Comparisons of Use Variables and Use Variable Correlations with Practicality of Format Iariable Haan_s_car_e Obtained; Subject’s use of training for decision-making + practicality of format a) .513 ~ -.339 b) .578 ~ Subject's use of training to train others + practicality of format a) .635 1.76 * b) .318 - practicality of format a) .190 ~ -.934 b) .091 Tatiana WWW Subject's use of video for decision-making - practicality of format .0549 ~ .370 Subject's use of manual for decision-making - practicality of format .0295 ~ .429 Subject's use of manual to train others - practicality of format .0071 ~ .478 Subject's use of format to train others + practicality of format -.0658 ~ .304 - practicality of format .1596 ~ .106 Subject's use of supplemental materials to train - practicality of format .2466 ~ .026 * NOTE: ~ = reversal of expected correlation * = significant relationship 71 for practicality of format for the variable, subject’s use of training to train others. One of the six null hypotheses was rejected. This was for the variable positive practicality of format for the variable subject's use of training to train others. The correlation results for Hypothesis 5 are shown in Figure 13. On the whole, the correlation coefficient results were still opposite what they should have been. The variable subject's use of supplemental material for decision-making, and the variable subject's use of video to train others were the only variables that had correlation coefficients that correlated completely as predicted. Only one correlation proved to be significant for this hypothesis. This was the negative practicality of format and the variable subject's use of supplemental material to train others. RESHLLS_Q£_H¥DQ§hs§i§_§ T-tests and correlations were run to determine what, if any, influence the two variables, positive relevance of content and negative relevance of content, had on the use variables. The T-test results, shown in Figure 14, revealed that the mean scores for sample (b) were less than sample (a) for relevance of content for the variable, subject's use of training to train others, a result that was opposite that 72 which was predicted. Out of the six null hypotheses tested, none were rejected. The correlation results, found in Figure 14, showed that the correlation coefficients were opposite what they should have been for the positive relevance of content and the variable, subject's use of supplemental materials for decision-making, the negative relevance of content and the variable, subject's use of format to train others, and the subject's use of supplemental material to train others. Out of the fourteen correlations run, only one was significant. This was the negative relevance of content and the variable, subject's use of supplemental material to train others. I‘C l-(z (I) 73 Figure 14 Summary of Significant Mean Score Comparisons of Use Variables and Use Variable Correlations with Relevance of Content Zariable Hean_s_o_ore Obtained; Subject's use of training to train others - relevance of content a) 1.048 ~ -2.63 b) .409 ~ Ramble W SW1 Subject’s use of supplemental materials for decision-making + relevance of content -.0801 ~ .314 Subject's use of format to train others - relevance of content .0014 ~ .496 Subject's use of supplemental materials to train others + relevance of content -.0832 ~ .258 - relevance of content .2592 .020 * NOTE: ~ = reversal of expected correlation * = significant relationship CHAPTER V CONCLUSIONS AND DISCUSSION It has been estimated that not more than 10 percent of all training information is transferred to the job. While many studies have examined this transfer problem, few have resulted in empirical solutions or even a better understanding of the failures. This thesis focused on the percentage of trainees which used the information and materials presented to them in an environmental training workshop. More specifically, an investigation of the factors which influenced the use of the training information and materials on the job was conducted. This chapter reports briefly the summary of the procedures and findings of the study, states the conclusions of the findings, and makes recommendations for further study. $2flflfl2¥.9fi.£l§£§dfllfi§ Eighty-five participants of workshops which were developed by the Educational Design Team at the University of Michigan responded to a telephone survey. They were chosen on the basis that they had: 1) Attended a workshop conducted entirely by the Educational Design Team, 2) received a manual 74 75 and/or video on the same subject that was taught in the workshop they had attended, and 3) attended a workshop with a train-the-trainer emphasis. The survey was designed to learn the percentage of subjects who had made use of the training information and materials and determine which factors influenced the use of the training information. Use of training information and materials was probed through a series of close-ended questions. The factors believed to influence measures of use were derived from responses to an open-ended question. SmarLeLEindines Results of all the frequency tests showed that more than 10 percent of the subjects had made use of the training information and materials. Percentages on the main use variables ranged from a low percentage of 45.9 percent of the total sample to a high percentage of 88.2 percent of the total sample. Percentages for the variables other's use of training for decision-making and other's use of training to train others, ranged from 35.3 percent to 50.6 percent of the total sample. Percentages of the sub-use variables were recorded along a seven-point scale, but in adding scores 2-7 together (little use to extensive use) resulting percentages of use were well above 10 percent. These were significant findings. They suggested that use of the Educational Design Team's training information and 76 materials was high. In many cases, much higher than the expected 10 percent. That more people used the information and materials to share with other people rather than for their own use or to train other people implies a few things for the Educational Design Team. First, evidence from the open-ended question at the end of the survey showed in many cases that the subjects who had attended the workshop and did not train other people, shared the information and materials with someone who could do the training or with someone for whom the information and materials would be useful. Eventhough the Educational Design Team had expected the workshop attendees to use the information and materials to train others, it appears that the materials and information can easily be passed to other people for training and personal use. Second, the lower usage of the materials on the job verses to share indicates that the Educational Design Team's materials are not as well suited for personal reference. Finally, because many people shared the information and materials with others, the Educational Design Team’s information and materials went far beyond just the people who attended their workshops. Other frequency results also proved to be interesting. Little to no use of the video and moderate use of the manual was reported for individual decision-making. High use of supplementary materials was reported for individual decision- making. What this means, again, is that the Educational Design Team's materials are not well suited for personal 77 reference on environmental topics, and that other resources are preferred by most subjects. The extent of use of the materials to train varied. The manual was used quite frequently to train. Feedback on the Educational Design Team's manual was usually quite good. Most criticisms of the manual were that it was too elementary, and as a result couldn’t be used very often. However, the Educational Design Team had designed the manual for trainees with less than 6 months of experience in their field. It was not unexpected, then, that extensive use of supplementary materials was also reported for those cases where trainees had more experience. The use of the video to train received either extensive or no use ratings by the subjects. It appears that the subjects had strong opinions about how the Educational Design Team's video should be used -- either not at all or quite frequently. The frequency of responses for the format to train were spread rather evenly along the seven-point response scale. The subjects apparently found a variety of ways to use the format to train. This was consistent with the Educational Design Team’s stated purpose to make the format flexible to meet a variety of training needs. Results of all the t-tests revealed that very few null hypotheses could be rejected, meaning that few differences between the use and no use groups were found. Apparently the training input variables examined in this study (see Figure 8) 78 do not strongly influence the subject's decision to use or not use the Educational Design Team's training information and materials, individually. It is possible that combinations of the training input variables may have influenced use. However, this was not examined in this study. The following null hypotheses were rejected for this study. Under Hypothesis 2 three null hypotheses were rejected. These hypotheses were rejected for the input variables positive non-human organizational factors and the variable, subject's use of training to share with others, for the input variable negative human organizational factors and the variable, subject's use of training to train others, and the input variable positive non-human organizational factors and the variable, subject's use of training to train others. These results were consistent with what was predicted. Two null hypotheses were rejected for Hypothesis 3. These hypotheses were rejected for the input variables negative trainee characteristics for both the variables, subject's use of training to share with others and subject's use of training to train others. These were strange results as the literature suggested that the more negative characteristics the trainee had the less that person would use training information and materials. It is interesting that the subjects chose to pass the Educational Design Team information and materials along and also used it to train. It cannot be determined why this is the case. 79 The null hypotheses rejected for Hypothesis 4 rejected the input variable positive workshop experience for both variables, subject's use of training to share with others and subject's use of training for decision-making. These are not unexpected results as the literature suggested that a positive workshop experience is a significant factor that influences the use of training. Only one null hypothesis was rejected for Hypothesis 5. The input variable positive practicality of format and the variable, subject's use of training to train others was rejected, suggesting that the Educational Design Team's format was satisfactory to the subject, and it helped facilitate the use of the training materials. No null hypotheses were rejected for Hypothesis 6. Correlations run on the hypotheses of this study resulted in few significant correlations. For Hypothesis 2 a positive correlation for positive human organizational factors and the variable, subject's use of manual for decision-making, a positive correlation for negative human organizational and the variable, subject's use of manual to train others, and a negative correlation for positive non-human organizational factors and the variable, subject's use of supplemental materials to train others, were found. The correlation of the latter warrants further discussion. The correlation coefficient indicated a relationship opposite that which was predicted. This was not a surprising finding. It was established earlier that the need to supplement the 80 Educational Design Team's materials was high. It is quite possible that even with non-human organizational factors supporting the use to train, that it was not enough to override the need to supplement. Hypothesis 3 also had three significant correlations. These were: 1) A negative correlation of positive trainee characteristics and the variable, subject’s use of video for decision-making, 2) a negative correlation of positive trainee characteristics and the variable, subject's use of video to train others, and 3) a positive correlation of trainee characteristics and the variable, subject's use of supplemental material to train others. These findings do not follow what was predicted for this study, but it is possible that for those people with the most positive characteristics, the need for the Educational Design Team's materials would not be high. It could be that the people displaying more positive characteristics are too "advanced" for the Educational Design Team’s materials. Correlations run on Hypothesis 4 revealed the highest number of significant correlations, for a total of four. A negative correlation for negative workshop experiences and the variable, subject's use of manual to train others was found. A negative correlation for negative workshop experiences and the variable, subject's use of video to train others was also found. The last two correlations were positive for the positive workshop experience and the variable, subject's use of supplemental materials to train others, and for the mega of 5 5L8? vor] rec: ta 5 a pra sup rel su; not SUI of “Y Fer Sit 81 negative workshop experiences and the variable, subject's use of supplemental materials to train others. These findings suggest that those subjects who reported more negative workshop experiences were not turned off to the materials they received from the Educational Design Team. This could mean that the Educational Design Team produces materials that are satisfactory enough to overcome a not so satisfactory workshop experience. One significant correlation was found for both Hypothesis 5 and 6. Negative correlations were found for negative practicality of format and the variable, subject's use of supplemental material to train others, and found for negative relevance of content and the variable, subject’s use of supplemental material to train others, respectively, results not predicted for this study. Conclusions The findings of the frequencies run for Hypothesis 1 suggested that high use was consistently the result for most of the variables of use. In all cases, use was higher than the 10 percent of the sample predicted by the hypothesis. Hypothesis 1 was rejected. The results of the t-test and correlations run for Hypotheses 2, 3, 4, 5, and 6 proved to be rather inconclusive. Few null hypotheses were rejected and few correlations were significant. Many of the rejected null hypotheses and siq of Oil (3 ’4 n re pr 82 significant correlations did not follow the same relationships of variables set forth by Hypotheses 2, 3, 4, 5, and 6. Based on this information Hypotheses 2, 3, 4, 5, and 6 were rejected. E' . i E l l' The summary of the findings for this study seems to indicate great success for the Educational Design Team and their training program. However, evidence from the open-ended question responses on the survey, as well as preliminary results from a follow-up study (Dearing and others, in progress) has yielded contradictory findings that indicate great drawbacks to the Educational Design Team training. Members of the Educational Design Team knew via anecdotal evidence from surveys they had conducted, that they had accomplished some success with their training programs. This study seems to confirm and, indeed, exceeds this belief, based on the frequency of use results. However, it is important to note that the methodologically different follow-up study to this thesis found that some of the percentages of use may have been inflated by the subjects (Dearing and others, in progress). Baldwin and Ford (1988) have found that self- report surveys often have accuracy problems. Future studies should examine other means to measure use, via multiple measures . 83 The responses from the open-ended question on the survey, as well as the follow-up study in progress, also suggest that the usage of the Educational Design Team's training was not always what the Educational Design Team intended. For instance, there were several cases where the Educational Design Team’s materials were used to teach community groups (examples found: a Kiwanis club, an elementary school class, and a tribal council group). The subjects within these community groups are not even remotely close to the group the Educational Design Team training was intended for. There is also evidence that suggests that when the Educational Design Team's materials were used by the targeted subjects, they often did not use them as specified by the Educational Design Team. It was found in several cases that the materials were just placed on shelves within organizations -- that use in these cases meant it was at the disposal of the staff. There is also strong evidence that suggests that subjects who indicated that they had used the materials to share with others or to train others, mainly just handed the materials over to those people with little explanation. It is clear that the Educational Design Team's train-the-trainer workshops resulted in few subjects who actually did quality training with the materials, as the Educational Design Team had intended. These unintended responses to the Educational Design Team training should be closely examined. If, indeed, a great proportion of the subjects were making use of the materials 84 but in ways not intended by the Educational Design Team, than it is clear that the Educational Design Team is not meeting the needs of the subjects. Feldman (1981) suggests that response to the actual needs of the subjects attending training workshops must be met in order for that training to transfer successfully and in the way it was intended. Cureton and Cureton (1983) further suggest that training that focuses on the perceived needs of the trainer (as seems to be the case of the Educational Design Team) instead of the actual needs of the trainee will not be successful. It is also clear from these unintended results that the Educational Design team should more closely examine what the feedback to their programs really means. More specifically, they should be making adjustments to their program based on the feedback they receive. Parry (1990) found that trainers who received feedback on trainee's performance after training and took appropriate actions with the course design could significantly improve the quality of training transfer. The results for the use of the manual, video, format, and supplemental materials are also problematic. The manual, on the whole, received positive feedback and, indeed, seemed to be a good resource for new staff with less than six months of experience in their field. The problem with the manual is, again, that it doesn't meet the needs of the subjects who attended the Educational Design Team training workshops. A large number of the subjects who attended these workshops had a great deal of experience in their field so they personally 85 found that the manual was too elementary. This is an expected result because the subjects attending the workshops were supposed to train people who needed a simple introduction to their field, ideally, people with less than six months of experience. In reality, the responses to the open-ended question and the responses to the follow-up study in progress suggest that the experience of the trainees (and, surprisingly, the intended trainers) differed vastly, resulting in a significant number of people within a given organization who would not or should not be trained with these materials. Eddy, Glad, and Wilkins (1967) found that the generalist nature of courses like those of the Educational Design Team are a problem for both the expert who finds it too elementary and the non-expert who cannot figure out the program. To compound the matter it was also found that the regulations the subjects follow vary from state to state. Hence, a high percentage of the subjects had to supplement heavily the manual with their own materials (a result consistent with the tests run on the hypotheses for this study). The results of the use of the training format seemed to indicate that the format was flexible in meeting the needs of a variety of training situations. The other evidence also found this to be the case. One to one hundred people could be trained with the materials and the materials were used in a variety of orders and settings. However, use of the format also included those subjects who had just handed the materials 86 to new trainees to review on their own. Truly effective training requires more guidance on the part of the trainer. The most problematic results for the Educational Design Team have to do with the use of their video. The highest frequencies of NON-use were reported for the video. The two most frequently reported reasons for non-use were that the videos were too long/elementary/boring and that, after one viewing, they could not be used again. The Educational Design team should re-examine this issue. A large proportion of their budget goes to the production of the videos. If use of these videos is indeed limited, they should carefully consider who receives the videos. If more videos are made in the future, more "test-marketing" should be done to meet the needs of the subjects, or at least these videos should be distributed to those who are really interested. What the Educational Design Team should now do with their entire program is to carry out an extensive evaluation of the actual needs of their clients and make changes to their program accordingly. It appears that, up until this point, the Educational Design Team had just been producers of materials and haven't really made an attempt to customize their programs for their clients. It is a well-known fact that training programs which take a hit-or-miss approach to training, essentially throwing a program out and seeing if it works, will, most likely, not produce good results. Perhaps the Educational Design Team should focus in on what it really knows how to do: teaching people skills for training. An 87 overwhelming majority of the subjects who attend their workshops go to learn the method of training and not the content of the Educational Design Team materials. Many of the subjects of this study had never been given lessons in the art of giving a presentation or teaching someone how to learn their job. What the subjects of this study apparently want and need are more of these lessons. These results should be of great interest to the main supporter of the Educational Design Team -- the Environmental Protection Agency. Their funding of the Educational Design Team would be better utilized if the training programs were re-evaluated and changed to meet the actual needs of the subjects. In particular, they should be most critical of the need for the videos, which use up a large portion of the Educational Design Team’s budget but produces little return. As for this study, there are a few comments that need to be made. First, while the results of the t-tests and correlations run for Hypotheses 2, 3, 4, 5, and 6 proved to be inconclusive, it still shed a few bits of light onto some factors that influenced use in this study. That statistical analysis was performed on the survey, was a step in the right direction for transfer research. Many studies have been based, at best, on anecdotal evidence and, at worst, on merely the opinions of "training professionals." Empirical studies on factors that influence training use is still sparse, so there are many directions in which future studies can go. 88 Further studies done on the Educational Design Team's workshops and materials should examine, in detail, the correlations between input factors and training use that this study found significant. Isolating a few variables for study may yield better results than this study, which looked at a very broad range of variables, and relationships between those variables. Other suggestions for future research noted by Baldwin 8 Ford (1988), that are in keeping with the interest of this study are: 1) To test the operationalizations of the training input variables that have been posited as having an impact on use, 2) to develop a framework for conducting research on the effects of the training input variables on use, and 3) to concentrate on research with more relevant criterion measures of generalization and maintenance versus the short-term, single-source studies that dominate the transfer of training literature. APPENDICES APPENDIX A Final Survey 89 APPENDIX A Final Survey Date Time Is this ? I am following up on a call made to you by , to conduct a 15-minute interview. My name is . I’m a graduate student at Michigan State University. Thanks for agreeing to participate. Our purpose is to learn what happened regarding the possible implementation and use of U.S. EPA workshop information and materials, developed by Paul Nowak at the University of Michigan. The information concerned inspector training, permit writing, and RCRA orientations. This information was presented in the form of a workshop, a video, and a manual. This survey is being conducted with the cooperation of the University of Michigan, and is funded by the U.S. EPA. You are guaranteed confidentiality. Neither your name or affiliation will be identified to anyone except within our 6- person research team at this university. You are not being tape-recorded, and you may stop answering questions at any time. You indicate your voluntary agreement by answering the questions on the questionnaire. Our results will appear as a part of a paper which we will submit for publication to an academic journal. We will also report the results to the EPA. WOULD YOU LIKE TO RECEIVE A COPY OF THESE RESULTS? [If yes, ....] WHAT IS YOUR.MAILING ADDRESS? DO YOU HAVE ANY QUESTIONS AT THIS TIME? 90 Just to verify our records: 1) WHAT WAS THE LOCATION OF THE WORKSHOP YOU ATTENDED? 2) WHAT TOPIC DID THE WORKSHOP FOCUS ON? 1) Inspector Training 2) Permit Writing 3) RCRA Orientation 4) Other 2a) DID THAT WORKSHOP HAVE A TRAIN-THE-TRAINER EMPHASIS? 1) Y 2) N 3) DID YOU RECEIVE VIDEOS AND MANUALS FOR THAT TOPIC? 1) Video only 2) Manual only 3) Video and manual [Interviewer: Continue the survey if 1) the respondent had been to one of the three topics listed, 2) the workshop had a train-the-trainer emphasis, and 3) the respondent had received a manual and/or a video for the topic of the workshop they had attended. 4) WERE YOU REQUIRED TO ATTEND THE WORKSHOP BY YOUR EMPLOYER? l)Y 2)N 5) DID YOU WANT TO ATTEND THIS WORKSHOP? 1) Y 2) N 6) IN , HOW'MANY YEARS HAD YOU WORKED FOR ? NUMBER OF YEARS = 7) WOULD YOU DESCRIBE THIS EMPLOYER.AS A.FEDERAL AGENCY, A REGIONAL AGENCY, A.STATE AGENCY, A LOCAL AGENCY, A PRIVKTE ORGANIZATION, OR OTHER? 1) FED 2) REGIONAL 3) STATE 4) LOCAL 5) PRIVATE 6) OTHER 91 8) MORE SPECIFICALLY, AT THE TIME OF THE WORKSHOP, HOW'MANY YEARS HAD YOU WORKED IN THE SAME DEPARTMENT? NUMBER OF YEARS = 9) AT THE TIME OF THE WORKSHOP, WERE YOU A.SUPERVISOR? 1) Y 2) N 10) THE WORKSHOP WAS STRUCTURED TO COMMUNICATE TWO THINGS. FIRST, YOU WERE TAUGHT SPECIFIC ENVIRONMENTAL REGULATORY KNOWLEDGE. SECOND, YOU WERE TAUGHT A METHOD FOR TRAINING OTHERS -- A TRAIN-THEdTRAINER APPROACH. DID YOU PRIMARILY ATTEND THE WORKSHOP TO LEARN THE ENVIRONMENTAL CONTENT, THE TRAINING METHOD, OR WERE BOTH EOUALLY IMPORTANT TOIYOU? 1) CONTENT 2) METHOD 3) BOTH EQUALLY 4) OTHER 11) TO ASSIST OTHER PEOPLE IN YOUR.DEPARTMENT, DID YOU EVER SHARE THE INFORMATION OR MATERIALS PRESENTED IN THE WORKSHOP? 1) Y 2) N -- go to question 15 [If yes, ... 12) TO YOUR KNOWLEDGE DID THEY EVER USE THE INFORMATION OR MATERIALS YOU SHARED WITH THEM FOR DECISION-MAKING UNRELATED TO TRAINING OTHER PEOPLE? 1) Y 2) N 3) DON’T KNOW 13) TO YOUR KNOWLEDGE DID THEY EVER USE THE INFORMATION OR MATERIALS YOU SHARED WITH THEM TO TRAIN OTHER PEOPLE? 1) Y 2) N 3)DON'T KNOW [If yes to 12 or 13, go to question 14 ... 14) HOW MANY PEOPLE, IN YOUR BEST ESTIMATE, DID THEY TRAIN? NUMBER OF PEOPLE = Next, I will ask you questions of two types. First, I will ask a series of closed-ended questions about the extent to which you, yourself, used the workshop information and materials. Second, I will ask an open-ended question about the reasons why the workshop information and materials were or were not used, by yourself or by others in your department. 92 15) HAVE YOU EVER USED THE WORKSHOP INFORMATION OR MATERIALS IN YOUR.JOB, FOR DECISION-MAKING UNRELATED TO TRAINING OTHER PEOPLE? 1) Y 2) N -- If no, go to question 19 On a seven-point scale, ranging from 1) no use, to 7) extensive use, please answer the following questions: 16) TO WHAT EXTENT HAVE YOU USE! THE WORKSHOP VIDEO IN YOUR OWN JOB, FOR DECISION-MAKING UNRELATED TO TRAINING OTHER PEOPLE? 1 2 3 4 5 6 7 17) ON THE SAME SCALE, TO WHAT EXTENT HAVE YOU USED THE WORKSHOP MANUAL IN YOUR.OWN JOB, FOR DECISION‘MAKING UNRELATED TO TRAINING OTHER PEOPLE? 1 2 3 4 S 6 7 18) ON THE SAME SCALE, TO WHAT EXTENT DID YOU FEEL THAT IT WAS NECESSARY TO USE SUPPLEMENTARY MATERIALS, SUCH AS OTHER REFERENCE GUIDES, ALONG WITH THE WORKSHOP MATERIALS, FOR DECISION-MAKING UNRELATED TO TRAINING OTHER PEOPLE? 1 2 3 4 5 6 7 19) ON RETURNING BACK TO YOUR ORGANIZATION FROM THE WORKSHOP, WERE YOU RESPONSIBLE FOR TRAINING OTHERS IN YOUR ORGANIZATION? 1) Y 2) N -- if no, go to question 17 [If yes, ... 20) FOR HOW MANY YEARS WERE YOU RESPONSIBLE FOR.TRAINING OTHERS? NUMBER OF YEARS = 21) APPROXIMATELY HOW MANY PEOPLE DID YOU TRAIN USING THESE MATERIALS? NUMBER OF PEOPLE = 93 22) Again, with 1 representing "no use" and 7 representing “extenSive use," TO WHAT EXTENT DID YOU USE THE MANUAL WHICH YOU RECEIVED IN THE WORKSHOP TO TRAIN OTHERS? 1 2 3 4 5 6 7 23) ON THE SAME SCALE, TO WHAT EXTENT DID YOU USE THE VIDEO TAPE WHICH YOU RECEIVED IN THE WORKSHOP TO TRAIN OTHERS? 1 2 3 4 5 6 7 24) ON THE SAME SCALE, TO WHAT EXTENT DID YOU MAKE USE OF A WORKSHOP FORMAT TO TRAIN OTHERS? 1 2 3 4 5 6 7 25) ON THE SAME SCALE, TO WHAT EXTENT DID YOU FEEL THAT IT WAS NECESSARY TO USE SUPPLEMENTARY MATERIALS, SUCH AS STATE- LEVEL MATERIALS, FOR TRAINING PURPOSES? 1 2 3 4 5 6 7 [Last Question] Now here's your last question, but it’s the most important question for our study. I'd like you to tell me, as an explanation or a story, why specific aspects of your department or organization or fellow employees, or aspects of the workshop information and materials, helped or hindered implementation and use. In your own words, we want to know WHERE SHOULD THE CREDIT AND THE BLAME GO - TO THE WORK ENVIRONNENT, OR TO THE WORKSHOP INFORMATION AND MATERIALS? AND WHY? [Interviewer: write out responses] Thank you very much for your time and participation. You may fax any questions you have about our study to Professor James Dearing, at 517 336-1192. APPENDIX B Survey Codebook APPENDIX B SURVEY CODEBOOK Column I: 1&2 ID This is the identification number of each survey 1-85 Column #: 3 This is a blank column indicating a page turn on the survey Column #: 4&5 WORSHP Indicates which workshop was taken O1=Austin, TX Jan 88 02=Ann Arbor, MI June 91 03=Atlanta, GA Jan 89 04=Ann Arbor, MI Apr 89 05=Philadelphia, PA Sep 89 06=Philadelphia, PA Feb 90 07=Or1ando, FL Apr 90 08=Niagara Falls, NY May 90 09=Albuquerque, NM May 90 10=Ann Arbor, MI Sep 89 11=Washington DC Nov 90 12=Washington DC Jan 91 13=Ann Arbor, MI 88 Column 3: 6 TOPIC Indicates which topic was taken 1=Inspector Training 2=Permit Writing 3=RCRA Orientation 4=l&2 5=2&3 6=1&3 7=1,2,&3 8=Other 94 95 Column I: 7 MEDIA Indicates which media the person had received 1=Workshop 2=Video 3=Nanual 4=WV 5=WN 6=VN 7=WVH 8=Other Column i: 8 Question_11__1 REQUIRED Were you required to attend the workshop by your employer? 1=Yes 2=No 3=Other column i: 9 Question_£;__§ WANT Did you want to attend this workshop? 1=Yes 2=No 3=Other Column #: 10 a 11 Question_ti__§ YRSATORG In___, how many years had you worked for ? 1-99 Column I: 12 Question_£1__z ENPTYPE Would you describe this employer as a federal agency, a regional agency, a state agency, a local agency, a private organization, or other? 1=Federal 2=Regional 3=State 4=Loca1 5=Private 6=Other Column I: 13 This is a blank column indicating a page turn on the survey Column #: 14 a 15 Question_ti__§ YRSATDEP More specifically, at the time of the workshop, how many years had you worked in the same department? 1-99 96 Column I: 16 W SUPRVISR At the time of the workshop, were you a supervisor? 1=Yes 2=No 3=Other 601m 3: 17 W PURPOSE Did you primarily attend the workshop to learn the environmental content, the training method, or were both equally important to you? 1=Content 2=Nethod 3=Both equally 4=Other cola-n I: 18 911mm SHARE To assist other people in you department, did you ever share the information or materials presented in the workshop? 1=Yes 2=NO 3=Other column #: 19 MW OTHRUSE To your knowledge did they ever use the information or materials you shared with them for decision-making unrelated to training other people? 1=Yes 2=NO 3=Don't know column h 20 WM OTHRTRN To your knowledge did they ever use the information or materials you shared with them to train other people? 1=Yes 2=NO 3=Don't know Cblumn i: 21 This is a blank column indication a page turn on the survey 97 Column #: 22,23,824 Questign_£1__1$ OTRAIN How many people, in your best estimate, did they train? 1-999 column I: 25 Question_£1_1§ YOUUSE Have you ever used the workshop information or materials in you job, for decision-making unrelated to training other people? 1=Yes 2=No 3=Other Column i: 26 Question_£1__1§ YOUVID To what extent have you used the workshop video in your own job, for decision-making unrelated to training others? 1=No use to 7=Extensive use column #: 27 Questign_£1_1z YOUNAN To what extent have you used the workshop manual in your own job, for decision-making unrelated to training others? 1=No use to 7=Extensive use Column #: 28 Questign_£1__18 YOUSUPL To what extent did you feel it was necessary to use supplementary materials, such as other reference guides, along with the workshop materials, for decision—making unrelated to training other people? 1=No use to 7=Extensive use Column #: 29 This is a blank column indicating a page turn on the survey 98 Colum I: 30 W TRAINER On returning back to your organization form the workshop, were you responsible for training others in your organization? 1=Yes 2=No 3=Other Column #: 31 MW YRTRAIN For how many years were you responsible for training others? 1-5 Column i: 32,33,834 gnaw YTRAIN Approximately how many people did you train using these materials? 1-999 column I: 35 W TRAINMAN To what extent did you use the manual which you received in the workshop to train others? 1=No use to 7=Extensive use column I: 36 WE TRAINVID To what extent did you use the video which you received in the workshop to train others? 1=No use to 7=Extensive use column #: 37 911W TRAINFOR To what extent did you make use of a workshop format to train others? 1=No use to 7=Extensive use Column #: 38 This is a blank column indicating a page turn on the survey 99 Column I: 39 W TRAINSUP To what extent did you feel that it was necessary to use supplementary materials, such as state level materials, for training purposes? 1=No use to 7=Extensive use Column I: 40 A I of positive workshop experiences 1-9 Column I: 41 B I of negative workshop experiences 1-9 Column I: 42 C Iof positive trainee characteristics 1-9 Column I: 43 D I of negative trainee characteristics 1-9 Colunn I: 44 E I of positive human organizational factors 1-9 Column I: 45 F I of negative human organizational factors 1-9 Colu-n I: 46 G I of positive 1-9 Column I: 47 H I of negative 1-9 Column I: 48 I I of positive 1-9 column I: 49 J I of negative 1-9 Column I: 50 K I of positive 1-9 Column I: 51 L I of negative 1-9 Column I: 52 M I of positive 1-9 Column I: 53 N I of negative 1-9 100 non-human organizational factors non-human organizational factors practicality of program format practicality of program format relevance of content relevance of content other factors other factors REFERENCES REFERENCES Anderson, J.G., & Wexley, K.N. (1983). Applications-Based Management Development: A Method to Promote Practical Application of Managerial and Supervisory Training. Personnel_anministratgr. November. 39-43- Baldwin, T.B., & Ford, J.K. (1988). Transfer of Training: A Review and Directions for Future Research. Egzsgnng1_ Psycholeax. 41, 63-105. Baumgartel, H., & Jeanpierre, F. (1972). Applying New Knowledge in the Back-Home Setting: A Study of Indian ManagerS’ Adoptive Efforts. The_lgurnal_9f_Annlied_ Behaxigral_SCienCe. 8(6). 674-694. Bell, P.E., Prewitt-Diaz, J., Molina, J., & Nendez, F.A.G. (1985). The Relative Effectiveness of Videotaped Inservice on Bilingual Education Teachers' Knowledge of Culture. ICnrnal_9f_Instructional_zs¥ChClCC1. 12(3). 140-143. Berke, E. I. (1984). Keeping Newly Trained Supervisors From Going Back to Their Old Ways. flanaggngn;_negiew, February, 14-16. 101 102 Bradlyn, A.S., Christoff, K., Sikora, T., O’Dell, S.L., & Harris, C.V. (1986). The Effects of a Videotape Preparation Package in Reducing Children’s Arousal and Increasing Cooperation During Cardiac Catherization. BfinnYiQI_B§§§§I§h_Ihfizfln¥p 24(4): 453-459- Broad, M.L., & Newstrom, J.W. (1992). Transf§z_gf_Tzaining. Reading, Massachusetts: Addison-Wesley Publishing Company, Inc. Broad, M.L. (1982). Management Actions to Support Transfer 0f Training- Trainina_and_nexelnnmenf_aonrnal. HEY: 124- 130. Byham, W.C., Adams, D., & Kiggins, A. (1976). Transfer of MCdeling Training to the Job. £ersgnngl_£§ygng1ggy, 29, 345-349. Cassileth, B.R., Heiberger, R.M., March, V., & Sutton-Smith, K. (1982). Effect of Audiovisual Cancer Programs on Patients and Families. Jguzna1_gf_flgdigal_finnga§ign, 57, 54-59, Clark, R.C. (1986). Nine ways to Make Training Pay Off. Training, November, 83-87. Cureton, J., & Cureton, B. (1983). Is Training an Ivory Tower? Trainina_and_nexelennsnt_19nrnal. February. 88- 89. Eddy, W.B., Glad, D.D., & Wilkins, 0.0. (1967). Organizational Effects on Training. T:a1n1ng_and_ Dexelgnment_lgnrnal. 22(2). 36-43- 103 Eden, 0., 8 Ravid, G. (1982). Pygmalion Verses Self- Expectancy: Effects of Instructor and Self-Expectancy on Trainee Performance. Qzganizatignal_Behayigr_and_flnman_ Performance, 30, 351-364. Eden, D., 8 Shani, A.B. (1982). Pygmalion Goes to Boot Camp: Expectancy Leadership and Trainee Performance. anznal_ 9f_Anplied_£sxshglng. 67. 194-199. Ehrenberg, L.M. (1983). How to Ensure Better Transfer of Learning- TraininC_and_negelopment_19urnal. February, 81-83. Feldman, M. (1981). Successful Post-Training Skill Application. Training_and_nexelgpmenf_ICnrnal. September, 72-75. Flanagan, S., Adams, M.E., 8 Forehand, R. (1979). A Comparison of Four Instructional Techniques for Teaching Parents to Use Time-Out. Behagigz_Tngrapy, 10, 94-102. Gagliano, M.E. (1988). A Literature Review on the Efficacy of Video in Patient Education. anrna1_gf_fledigal_ Bangation, 63, 735-792. Georgenson, D.L. (1982). The Problem of Transfer Calls for Partnership. Irain1nC_and_nexelnpnenf_aourna1. October. 75-78. Georges, J.C. (1988). Why Soft Skills Training Doesn’t Take. Training. April. 42-47- 104 Gist, M.E., Bavetta, A.G., 8 Stevens, C.K. (1990). Transfer Training Method: Its Influence on Skill Generalization, Skill Repetition, and Performance Level. Eersgnne1_ Psychology, 43, 501-523. Harmon, S.J. (1974). Management Training and Development: An Interim Project Approach: Optimizing the Transfer of Training Back to the Work Situation. Training_and_ W, June, 16-18. Huczynski, A.A., 8 Lewis, J.W. (1980). An Empirical Study Into the Learning Transfer Process in Management Training. Wales. Nay. 227- 240. Kaufman, D.M., 8 Kaufman, R.G. (1983). Usefulness of Videotape Instruction in an Academic Department of Neurology. Jgnrna1_gf_Mediga1_Edngatign, 58, 474-478. Kelley, A.I., Orgel, R.F., 8 Baer, D.M. (1985). Seven Strategies That Guarantee Training Transfer. Training_ and_Dexelonment_ICnrnal. November. 78-82- Kelly, 8.8. (1982). A Primer on Transfer of Training. Trainina_and_De!elopment_JCurnal. November. 102-106. Kotter (1988) in Broad, M.L., 8 Newstrom, J.w. (1992). Tzansfer_g£_mzaining. Reading, Massachusetts: Addison- Wesley Publishing Company, Inc. Kruger, M.J. 8 May. (1985). Two techniques to Ensure That Training Programs Remain Effective. Personn§1_ggnzng1, October, 70-75. 105 Lefton, E. 8 Buzzotta, V.R. (1980). Trainers, Learners, and Training Results. Iraining_and_nexelonnent_lournal. November, 12-18. Leifer, M.S., 8 Newstrom, J.W. (1980). T;aining_gnd_ Dexelopnsnt_19urnai, August, 42-46. Maguire, G.P., Goldberg, D.P., Hobson, R.F., Margison, F., Moss, 8., 8 O'Dowd, T. (1984). Evaluating the Teaching of a Method of Psychotherapy. Briti§n_19nznai_gf_ W. 144, 575-580. Marx, R.D. (1986). Self-Managed Skill Retention. T:nining_ W1. January. 54-57- Marx, R.D. (1982). Relapse Prevention for Managerial Training: A Model for Maintenance of Behavior Change. ACadCnY_of_Nanagement_Bexien. July, 433'441- Michalak, D.F. (1981). The Neglected Half of Training. W, 35(5), 22-23. Mironoff, A. (1988). Teaching Johnny to Manage. Training, March, 49-53. Moldofsky, H., Broder, 1., Davies, G., 8 Leznoff, A. (1979). Videotape Educational Program for People With Asthma. w, 120, 669-672. Nadler, L. (1971). Support Systems for Training. Training_ W. October. 2-7 . Napier, N., 8 Deller, J. (1985). Train Right or Don't Train at All. WWI. February. 90- 93. 106 Newstrom (1984 8 1986) in Broad, M. L., 8 Newstrom, J.W. (1992). Transfer of Training. Reading, Massachusetts: Addison-Wesley Publishing Company, Inc. Nielsen, E., 8 Sheppard, M.A. (1988). Television as a Patient Education Tool: A Review of its Effectiveness. 2atient_Education_and_connseling. 11. 3-16- Noe, R. (1986). Trainees Attributes and Attitudes: Neglected Influences on Training Effectiveness. Agadany of_nanagement_8exiew, 11, 736-49. Noe, R., 8 Schmitt, N. (1986). The Influence of Trainees Attitudes on Training Effectiveness: Test of a Model. Personnsl_Bs¥Chology, 39, 497-523. O'Dell, S.L., Mahony, N.D., Horton, W.G., 8 Turner, P.E. (1979). Media-Assisted Parent Training: Alternative Models. Behayior_TherapY. 10. 103-110- Parry, S. (1990) in Petrini, C.S. (1990). Bringing it Back to Work. Training_and_Deyelonment_19urnal. December. 15- 21. Petrini, C.S. (1990). Bringing it Back to Work. Tzaining_ and_Dexs19nment_£onrnal. December. 15-21. Pratt, C.C., Schmall, V.L., Wilson, W., 8 Benthin, A. (1991). A Model Community Education Program on Depression and Suicide in Later Life. Tha_§a:gn§aiggia§, 31(5), 692- 695. Raphel, M.A., 8 Wagner, E.E. (1974). Training Via Text, Audio Tape or TV Makes A Difference: Or Does It? TraininC_and_nexelopment_19urnal. December. 3-5. 107 Robinson, 0.6., 8 Robinson, J.C. (1985). Breaking Barriers to Skill Transfer. Training_and_nexelopnent_lgnrnal. January, 82-83. Sedlick, J.M., Magnus, A.K., 8 Rakow, E. (1980). Key Elements to an Effective Training System. Training_and_ Dexelonment_Journal. JU1Y: 10-12- Shaddock, A.J. (1989). Issues in the Production of Staff Training Media. Ihe_British_19nrna1_gf_uenta1_ thngznaiity, 35(69), 101-107. Soukup, K.M., 8 Albrecht, K.R. (1983). Using Multimedia to Replace Apprenticeship Training for Professionals. Training_and_nexelonment_lournal. December. 37-39. Sox, R.C., Marton, K.I., Higgins, M.C., 8 Hickam, D.H. (1984). Tutored Videotape-Instruction in Clinical Decision-Making. Janrnai_af_fladiga1_£dn§a;ian, 59, 188- 195. Spice, M.B., 8 Kopperl, S. (1984). Are Your Trainees Willing? Iraining_and_ne!elonment_lournal. May. 30-32- Tietge, N.S., Bender, S.J., 8 Scutchfield, F.D. (1987). Influence of Teaching Techniques on Infant Car Seat Use. Eatient_Education_and_gounseling. 9. 157-175- Trost, A. (1985). They May Love It But Will They Use It? Trainin9_and_De!elopment_Iournal. January. 78-81- Tziner, A., Haccoun, R.R., 8 Kadish, A. (1991). Personal and Situational Characteristics Influencing the Effectiveness of Transfer of Training Improvement Strategies. Jgnznai of_Qccnnational_zsxchologx. 64, 167-177. 108 Uzark, K., Rosenthall, A., Behrendt, D., 8 Becket, M. (1985). Use of Videotape to Promote Parenting of Infants with Serious Congenital Heart Defects. Parianr_Edngarign_and Counseling, 7, 111-119. Watson, C.E. (1974). Getting Management Training to Pay Off. Businaaa_ngrizana, February, 51-57. Wexley, K.N., 8 Baldwin, T.T. (1986). Post-Training Strategies for Facilitating Positive Transfer: An Empirical Exploration. Agadany_af_flanagananr_1gnrnai, September, 503-520. Wexley, K.N., 8 Latham, G.P. (1981). nayaigning_and_Training Hnnan_Raagnraas_in_grganizarigna. Glenview, IL: Scott, Foresman. Zemke, R., 8 Gunkler, J. (1985). 28 Techniques for Transforming Training Into Performance. Training, April, 48-63. "I7111111113111!“11111111111S