. “mm .‘V “ "fit‘r‘... .V . 3.53:)» 3? l. '7‘“... u. - ~ } ‘ :77?"va— . 5‘ . If s . - .I?\ v c. vuV . I! ‘ “x.“ L: W"! - £51133. .., Q ‘ ., QM»): .tv 3’ . .‘ . ‘ . . ‘ . . ntplno. (pl. . . tn 5 . ‘ . J..- V‘ :41: ,WM.K» , . : 1 L. . .. ‘ . «'51. m‘ .. 5v ntV :- 7 fr) »t .......t« . 1 I. Viki-p. I! (I. 11 . . 215.... . .3 H .u? sdltl. i. .r s: (CL. .e K. v w H A . VI .0 5 . . , . ‘ .91}. :ciw ”1mm? fl. ‘ .. s: ,v.,)~.. ....a..r.1 . L (0». yr: .(.l..s 1‘5! . \I‘I'rli’. 7“ {gm ‘ {53] Li)! Ly. . .‘Il. . n->.I...LI. . :lw...» . .. vv. -‘ Y ‘ . v l‘ '0 .0. ‘ . . .. .f if“. nu... . Lunar . . r t . .- L ,o . , V. ,I unfit... . ...Lva.kn..u..-:u....q . .(blL 1:. I v . 'Id.‘?dl.ld nor) . vfimo.h,o.$‘ru:rin;?v r1 2). . . : a». D. E; in p . . ...y§$é,. uni. (s n. u....\ .acwfi 1 xx. . L I. \ rt. \vul‘» synaflnfl. "‘2 L . , guy/9. a»: I! ‘5»- all? I; .‘ - .. ‘11.; . .V . [.565 7.3.1.3.”... x era-t1; {5.74 A III? IIIG . ‘ If} 3% ; . n 3. Ft} .2 A I. l 1:!) D t t irks c:. )1? Ito. r}.- fiuf: ; as .L... hut? 2 tr... ... .i .MPi-rnu Zuzmoc . f”. ‘ élr’. .‘ v. 7' calf... 4.2!.» c . Aw»... . 2...! nLBfliinihll. . .LS! cgedaé.«r.u?3afi. I ("nu-5c f BK”: lulu”. , it . 1 . '9': :97 Och-(Orv, l- 1“ 'I ‘5 v in flirt 2“ If» at?" (If I .Ii. 2:); o'.<.r ‘ fl)!“ Infiltan El 135;}, 3.. Ft.‘.?llvlol'(ll!l "" H t . llHlHHlIalHllllHil | UK lHlH Ill l J l 3 6 31203 00620 7264 LIERARY Michigan State University This is to certify that the dissertation entitled BEHAVIORAL CUES THAT INDICATE TEES” ‘ PRESENCE OF DEBILITATING ISSUES WHICH AFFECT TRAINING PROGRAMS, As PERCEIVED BY TRAINING PROFESSIONALS presented by Michelle M. King-Kramer has been accepted towards fulfillment of the requirements for Ph. D. . Adult & Continuing degree in Education Mayor professor Date 5/43/70 MS U is an Affirmative Action/Eqiml Opportunity Institution 0- 12771 PLACE IN RETURN BOX to remove this chockout from your record. TO AVOID FINE§ return on or baton date a». U5 DATE DUE DATE DUE =fifil MSU Is An Affirmative ActlorVEqual Opportunity Institution BEHAVIORAL CUES THAT INDICATE THE PRESENCE OF DEBILITATING ISSUES WHICH AFFECT TRAINING PROGRAMS, AS PERCEIVED BY TRAINING PROFESSIONALS BY Michelle M. King-Kramer A DISSERTATION Submitted to Michigan State University in partial fulfillment of the requirements for the degree of DOCTOR OF PHILOSOPHY Department of Educational Administration College of Education 1990 ABSTRACT BEHAVIORAL CUES THAT INDICATE THE PRESENCE OF DEBILITATING ISSUES WHICH AFFECT TRAINING PROGRAMS, AS PERCEIVED BY TRAINING PROFESSIONALS By Michelle M. King-Kramer The purpose of this research was threefold: (l) to identify behavioral cues within the employee training environment that commonly indicate the presence of debilitating latent issues, (i.e., perceptions of threat), (2) to rank order those cues according to their validity and their degree of urgency, and (3) to trace the cues to latent issues with which they may be associated. Specifically, the investigator focused on the emotional reactions of upper level management, supervisors of training participants, and training participants themselves. Thirty-seven internal training professionals from twenty states and a variety of work settings participated in the study. The Delphi technique of data collection was used, which involved three rounds of questionnaires. Final analysis of the data produced seventeen specific behaviors that training professionals often perceive to be cues to underlying issues. Further, each of those cues were linked to one or two likely latent issues. The investigator identified several notable patterns in the concerns of upper and middle managers and training participants. Employee apathy toward and ignorance of the training process also arose as common debilitating influences. This work is dedicated to my parents, Robert and Doris King. Without their inexhaustible love, generosity, and faith in my abilities, I never could have begun, let alone finish, this degree. iv ACKNOWLEDGEMENTS With heartfelt appreciation, I gladly take this opportunity to thank my friend and advisor, Dr. Cas Heilman, for his support throughout this project. Insightful, intelligent, and caring, he challenged me to ask difficult questions, then gave me the confidence to feel comfortable with answers that are, for now anyway, incomplete. I also wish to thank Dr. Cass Gentry for his diligent reading of the many proposal drafts, and his always insightful comments that boosted the academic quality of my work immensely. Dr. Richard Gardner has been a consistently supportive and enjoyable influence throughout my graduate experience. He has a knack for helping me apply academics to "real life" - such lessons are invaluable. Mary Servais, my friend and fellow student, also deserves formal appreciation. As she says, we "kicked one another down the road" toward the completion of these degrees. I must also recognize Rachel "Squeeze" Kramer, whose gestation, birth, and first year of life prolonged and complicated this project - for all the right reasons . Finally, I thank my buddy, husband, and fellow doctoral student Geoff Kramer. One day, perhaps we'll look back on our glassy-eyed, fog-headed graduate years together and remember them fondly. Then again, maybe not. vi TABLE OF CONTENTS LIST OF FIGURES AND TABLES . . . CHAPTER I. INTRODUCTION . . . . Statement of the Problem . . Definition of Terms . . . . . Relationship of Latent Issues Research Questions . . . . . Significance of this Research Assumptions of this Research Limitations of this Research Overview of the Remainder of the Dissertation CHAPTER II. Introduction . . . . . . . . An Organization Development Approach to Training Behavioral and Environmental Cues Latent Issues A Definition . . . . . . . Training Difficulties . . REVIEW OF THE LITERATURE Discounting of Latent Issues in the Lit. Empirical Support for the Concept Current Response to Latent Issues The Delphi Research Method . sumary O O O O O O O O O O 0 CHAPTER III. RESEARCH METHODOLOGY Introduction . . . . . . . . Population of the Study . . . Criteria for Selection of the Sample Description of the Sample . . Development of the Delphi Questionnaires Data Collection Schedule . . Summary . . . . . . . . . . . CHAPTER IV. FINDINGS . . . . . Introduction . . . . . . . Quantity and Quality of Participant Response Results of Round One . . . . Results of Round Two . . . . vii ix Results of Round Three . . . . . . . . . . . . 85 summary 0 C O O O O O O O O O O 0 O O O O O O O 9 2 CHAPTER V. CONCLUSIONS, OBSERVATIONS, AND RECOMMENDATIONS 94 IntrOduction O O O O O O O O O O O O O O O O 9 4 A Response to the Research Questions: QUESTION #1 . . . . . . . . . . . . . . . 95 QUESTION #2 . . . . . . . . . . . . . . . . 96 QUESTION #3 . . . . . . . . . . . . . . . . 96 QUESTION #4 . . . . . . . . . . . . . . . . 97 QUESTION #5 and #6 . . . . . . . . . . . . 97 Observations . . . . . . . . . . . . . . . . . 98 Implications for Training Professionals . . . . 112 Recommendations for Further Research . . . . . 113 APPENDICES O O O O O O O O O O O O O O O O O O 0 O 11 5 A. Telephone Script . . . . . . . . . . . . . . 115 B. Round One Questionnaire . . . . . . . . . . 118 C. Feedback Sheet Provided for Reviewers . . . 125 D. Round Two Questionnaire . . . . . . . . . . 126 E. Round Three Questionnaire . . . . . . . . . 132 F. Reminder Cards . . . . . . . . . . . . . . . 139 BIBLIOGRAPHY . . . . . . . . . . . . . . . . . . . 140 viii LIST OF FIGURES AND TABLES FIGURE 1. Flowchart: Relationship of Latent Issues to Cues TABLE 1. Response Rate of Mailed Questionnaires . . . 2. Cues Identified in Round One and Retained for Round Two . . . . . . . . . 3. Verification of Cues . . . . . . . . . . . . 4. Summary of Cues Retained for Each Round . . 5. Suggestions of Latent Issues for Each Cue . 6. Likelihood and Urgency Index . . . . . . . . 7. Relationship Between Cues and Latent Issues ix PAGE . 10 61 65 74 77 79 87 100 CHAPTER I INTRODUCTION Statement of the Problem Several factors can hinder the success of an employee training program. Many of these factors are readily apparent: the budget won't support the basic costs of program implementation, or critical union groups openly refuse to participate in the training effort because of a current labor dispute. In these cases, training professionals know from the start that the chances of training success are slim. Most dysfunctional influences on training, however, stem from more subtle factors that are difficult to detect. For instance, political tensions in the upper echelon of management may leave a line supervisor fearing for his own job security; consequently, the supervisor may discreetly discourage his subordinates from using newly-acquired skills that would reduce their need for supervision. Similarly, the informal communication patterns within the organization may by- pass a critical player who later refuses to support a new training program because she was not properly consulted. Unfortunately, many training professionals 2 fail to recognize these emotion-laden issues until long after the damage is done. (Allen & Kraft, 1982; Schein, 1985). "Over and over again improvement efforts are thwarted by invisible, unacknowledged forces. It is as if each organization were in reality two organizations - one visible, articulated, expressed in stated goals, policy statements, and procedural manuals; the other invisible, lying quietly under the surface, but actually determining what will happen in the long run." (Allen & Kraft, 1982) For the purpose of this study, these "unacknowledged forces" which are below the threshold of superficial observation will be referred to as "latent issues". Because latent issues are not tangible, training professionals must look for cues in the work environment that may suggest their presence. For instance, if a shift supervisor refuses to allocate time at the weekly staff meeting for information on an upcoming training program, this may suggest the presence of a latent issue (possibly a lack of managerial support, due to a perceived threat to his authority). If training specialists are aware that certain behavioral or environmental cues tend to be associated with latent issues, then they can look for those cues in the work environment, determine whether or not harmful issues exist, then take steps, where necessary, to adapt to and/or remedy the problem. 3 The purpose of this study is threefold: (l) to identify cues within the work environment that commonly indicate the presence of debilitating latent issues, (2) to rank order those cues according to their validity and their degree of urgency, and (3) to trace the cues to latent issues with which they may be associated. Definition of Terms Frequently used terminology and concepts used in this study that may be ambiguous will be defined in this section. The theoretical and research foundations of these concepts will be presented in the literature review section. Training professionals, (coordinators, developers, presenters), are the individuals who are ultimately responsible for the quality of the training program, and so are the peOple who must recognize cues and address the underlying latent issues. For the purpose of this study, only internal (or "in-house") training professionals will be considered. Training participants are the students or trainees in the training program. Supervisors of participants are the immediate superiors of the individuals presently acting as training participants. These supervisors are not directly involved in the actual training program, but 4 they have regular contact with the participants before and after training, and so can have significant impact on the success of the program. Upper level managers include all managers and administrators from the highest ranking authority within the organization down to the people who oversee the supervisors of training participants. Latent issues will be defined as hindrances to training effectiveness that are commonly overlooked or underestimated by training professionals. These issues are ”latent", meaning they are not stated and not readily observable; a training professional will not be aware of the latent issues unless some cue is perceived and accurately interpreted. (A latent issue may be considered the cause of a training problem, while a cue is an effect or result.) Once the issue is recognized and understood by the training professional, it becomes "manifest." For the purpose of this study, then, an issue is "latent" or "manifest" depending on the perception of the training professional. Latent issues are generally associated with any perceived threat to one's emotional, social, occupational, or financial well-being. Perceived threats that may evolve into latent issues are included in the following statements: 1. "My authority is threatened." 5 2. "My job security is threatened." 3. "My reputation is threatened." 4. "My opportunity for job advancement or a raise is threatened." 5. “My financial well-being is threatened." 6. "The loyalty of my co-workers is threatened." 7. "My right to be recognized as an individual is threatened." 8. "My right to voice my opinion is threatened." 9. "My comfortable, familiar work environment is threatened." 10. "My self-esteem is threatened." 11. "My personal autonomy is threatened." 12. "My physical well—being is threatened." 13. "Some aspect of my personal life is threatened." It is also possible for a latent issue to exist that is not related to the training program. Perhaps a manager assigned to the training planning committee is forced to miss meetings because of other, more pressing, work responsibilities. (Note the word "forced", meaning the person is not dodging or procrastinating, but truly has no choice in the matter.) Another example of a non- training related latent issue occurs when a training participant is burdened with personal or family concerns that distract her from the learning task. Whether the issue is associated with the training program or not, 6 the result is the same: training effectiveness is impaired. Cues are audible or visible indicators of underlying latent issues. (For the purpose of this study, it is assumed that all latent issues produce cues, but not all cues are recognized by training professionals.) A cue may come in many forms. It may be behavioral, meaning an observable, measurable activity of an individual or group, or it may be environmental, meaning a tangible entity that exists as the result of past behaviors (e.g., a documented organizational policy, the physical layout of a classroom, or a schedule of events). A cue may be a straightforward, unmistakable "red flag", or it may be very difficult to recognize. The following are examples of possible cues: 1. Management is slow to offer input into the goals of the training program. 2. Line supervisors who review the training curriculum set down by upper management report that one portion of the training will probably not be well received by training participants. 3. According to a long-standing company policy, all participants must attend training on their own time without remuneration. 4. A small group of training participants consistently disturbs the class by talking loudly among themselves. 5. Almost 85% of the participants flunk the final training exam. 7 6. When training is finished, participants are not allowed time to practice their new skills back on the job. An obvious and important cue in one training situation may prove meaningless in another training situation. Training professionals hold the difficult responsibility of sorting out cues from non-cues, tracing cues to their underlying latent issues, and deciding how to react to cues and issues once they are recognized. Cue validation. A cue is considered "valid" or "verified" if it regularly indicates the presence of one or more latent issues. The more consistent a cue is, the more training professionals can rely on it as a signal of underlying training complications. Cue validity indicates pervasiveness, not power. The research subjects verified cues based upon their own observations as training professionals. Intervention refers to the action a training professional takes to alleviate the problems associated with a latent issue. An example: a training participant does not concentrate on a learning task because he resents having been forced to attend the program. The order to attend training spawns a latent issue: a perceived threat to the participant's personal autonomy. In this case, the trainer could intervene by openly acknowledging the participant's frustration and 8 encouraging a brief discussion to explore and vent the feelings of resentment. There are many latent issues that may be impermeable to any intervention by training professionals. For instance, an upper level manager who inherently craves power over all others may view a particular training program as a threat to that power. In such cases, there is no acceptable intervention for a training professional; the manager's expectations are unreasonable, and so efforts to appease the threatened feelings would only compromise the training effort. Relationship of Latent Issues to Cues The best possible circumstances for any training professional would be to work in a situation where there are no latent issues, and therefore no troublesome cues. The next best scenario would be to work in a situation where all latent issues quickly become manifest, as the owners of the issues freely discuss their threatened feelings, or otherwise offer easy-to-recognize cues. Unfortunately, these fairy-tale scenarios rarely happen in real life. Most training professionals would probably agree that the latent issues associated with training programs are many and the recognizable cues too few. 9 A training professional will first be alerted to the presence of a latent issue by a cue. It is certainly possible that a latent issue could exist without producing such a cue, but in that case the training professional, unless he/she is clairvoyant, will have no means of recognizing it. This study will address only those latent issues that can be associated with observable evidence; if no cue is available, then trainers cannot be expected to detect the problem. The flow chart presented in Figure 1 was created by the investigator after reviewing the related literature and analyzing the complexities of the latent issues construct. As indicated, there are generally three groups of people, other than the training professionals themselves, who are associated with a training program: upper level managers, supervisors of training participants, and the training participants (Cells A - C). Each individual involved in the training effort perceives the impact of the program as it relates to him/her (Cell D). Based on that perception, the person may react in one of two ways Cell E). If he/she feels comfortable with the purpose and process of the training program, he/she will probably not experience any latent issues (Cell F). If an individual feels that some aspect of the program, (perhaps its goals, or the way it is planned or implemented), threatens his/her emotional, 10 he: lleve InleaeLnt C Training Participant: C. l > C131. of ) Participants D Perceived boast of Training on the Individual lisurel no Relationship of Latent Issues to (bee. Io Issue Exists c. Latent Issue Exists and Cue is knitted Does Trainer Recol- nise Oue'.’ no Issue leasins latent no rately Lin! Cue to Issue? l. Issue leooaes Initest 1 Possible Contribution to Training Failure Possible Contribution to Training Success 11 social, occupational, or financial status, then one or more latent issue may take shape (Cell G). Other latent issues that are not associated with the training program, but nonetheless pose a threat to training effectiveness are also included in Cell G. The threatened individual then reacts to the latent issue by consciously or unconsciously displaying an outward cue (Cell G). The cue may be very subtle, or it may serve as an unmistakable red flag to even the most casual observer. This is the stage that is critical for training professionals. When a cue is available, it is up to the training professional to recognize it (Cell H). When the cue is recognized, the training professional has the opportunity to link it to the underlying latent issue (Cell I). If the cue is not recognized, however, then the underlying issue will almost certainly go unnoticed (Cell J). Once the issue is properly interpreted, it then becomes manifest, (Cell K) and some intervention is possible (Cell L). In many instances, the training professional can make appropriate accommodations or negotiations (Cell L), which will encourage the individual owner of the issue to go on to contribute to the success of the training program (Cell M). 12 There are a number of circumstances that can lead to possible training failure. If no cue is available to the training professional and the issue remains latent (Cell J), or if the training professional recognizes the issue but either fails to react appropriately or is unable to influence the situation, (Cell L), then the owner of the issue will likely go on to contribute to problems in developing and implementing the training program (Cell N). The major difficulty in this process is that the training professional must not only recognize available cues, but also trace those cues to the correct latent issues and then, where possible, intervene appropriately. Research Questions The following research questions will be examined in this study: 1. What behavioral and environmental cues are perceived by training professionals as indicators of debilitating latent issues? 2. Which cues are perceived by training professionals as the most valid indicators of debilitating issues? 3. Which cues are perceived by training professionals to be associated with which latent issues? 4. Which cues can be traced by training professionals to the most urgent threats to training effectiveness? 13 5. Which cues are associated with which stage of training program implementation? (Considering the Pre—Training, Training, and Post-Training stages.) 6. Which cues tend to originate from each of the following three groups of individuals: upper level managers, supervisors of participants, or training participants? Significance of This Research This study will offer a significant contribution to the training and development field by attempting to operationalize the concept of cues as it relates to training success. The vast majority of the literature that addresses the concept of debilitating latent issues describes only generalized conditions. In contrast, this study will identify specific, readily observable gggg in the work environment that can be traced to latent issues. It is assumed that a training professional can more easily identify and address a specific cue than a generalized condition. For instance, it is commonly recognized that lack of supervisor support is an important barrier to training effectiveness. (Baumgartel & Jeanpierre, 1972; Baumgartel, Reynolds & Pathan, 1984; Laird, 1985) "Lack of supervisor support" is a generalized condition; just because you know it's important does not make it easy to spot. On the other hand, if training coordinators are aware of specific cues that commonly indicate lack of 14 supervisor support, (such as a supervisor's failure to allow training participants to practice their new skills) then they can be on the lookout for such cues. Related to the first, the second major contribution of this study will be its role as a building block for future empirical research. At present, many training professionals find latent issues difficult to define and hazardous to confront. Nonetheless, when latent issues exist but are not acknowledged, they pose an undeniable threat to the training effort. (Allen & Kraft, 1982; Argyris, 1985; Baum, 1987; Presthus, 1962) "Too often, developers tend to focus on the more tangible, concrete aspects of pre-training analysis and ignore the more nebulous factors." (Witkin, 1977) As a research topic, latent issues is certainly "nebulous", but the literature clearly affirms the need for further exploration. Thirdly, it is anticipated that this research will help to ease the organizational change process involved in training efforts. To implement a training program is to bring change to an organization, and, as Edgar Schein affirms, "One of the most salient characteristics of organizations that plagues all of us as members, consultants, managers, and citizens is their incredible resistance to change." (Schein, 1980) Organization development and organizational psychology literature has 15 established that change comes easier to organizations when participants' concerns and beliefs, (some of which may constitute latent issues) are recognized as important. (Allen & Kraft, 1982; Argyris, 1984; Baum, 1987; Beckhard, 1969; Moore, 1984) It is anticipated that the results of this study will facilitate the organizational change process by alerting training professionals to the personal beliefs of participants and other affected employees as they relate to training. Finally, a conscientious search of over 150 related dissertations and more than 200 related journal articles and books did not reveal any instances of this particular area of research being addressed in the training and development literature. While there is a plethora of prescriptive, field-oriented journal articles and books that offer vague advice on "listening to your client's needs", structured, research-oriented material on this topic is lacking. The deficiency of empirical research on the observable indicators of latent issues can partly be explained by the complexity of organizations as much as the ambiguity of the research variables. "Organizations are enormously complicated entities, interdependent with their environments and inextricably connected with the human beings making them function. Any theory of planned change must consider innumerable factors and 16 relations, and, to date, little agreement exists about the important variables to study or the key relationships to test." (Porras & Hoffer, 1986) Assumptions of This Research The fundamental assumption of this research is that influences beyond the immediate control of the training professional can significantly impact the success of a training program. Even when a training presentation is superbly designed and delivered, countless factors can interfere with the ultimate transfer of learned skills to the work setting. A second assumption of this study is that, given a quality training program and attentive, capable participants, the fewer the latent issues, the better prepared training participants will be to learn the material and subsequently transfer that learning to the work setting. In making this assumption, it is recognized that the complexities of human behavior and organizations make it virtually impossible to rule out all latent issues or to have all training participants optimally ready to learn. The assumption of these absolute conditions is inherent in any out—of- laboratory research, and must be recognized when the final results are analyzed. 17 A final assumption held in this study is that internal training professionals are better equipped to recognize and interpret cues and latent issues than external training consultants. Internal training professionals have extensive contact with the organization before, during, and after training (as opposed to external training consultants). Internal trainers, then, usually have more information on which to base judgments concerning abstruse training problems. Nonetheless, the results of this study have implications for external trainers, as well. Limitations of This Research This study was limited by the following factors: 1) Sample size. Input was solicited from forty randomly selected training professionals. While this is a sizeable number of subjects for the chosen method of data collection, it is of course a fraction of the training and development population. 2) Subjects were selected based on their willingness to participate in the study. 3) This study approached the assessment of cues and latent issues from the training professional's vantage point. Admittedly, any single observation of individual or group behavior is limited and sometimes biased. This limitation is typical of the day-to-day 18 constraints under which most training professionals must labor. Overview of the Remainder of the Dissertation A selected review of the related literature is reported in Chapter II. The major sections are: (1) an organization development approach to training, (2) behavioral and environmental cues, (3) a definition of latent issues, and (4) the Delphi research method. The design and methodology of the study are presented in Chapter III. This chapter includes a description of the population, the criteria for selection of the sample, the development of the Delphi questionnaires, and the data collection schedule. The data collected in Rounds One, Two, and Three of the study are reported in Chapter IV. In Chapter V, the investigator addresses each of the research questions, presents additional observations, and proposes the implications of this research for training professionals. Recommendations for further research are also presented in Chapter V. CHAPTER II REVIEW OF THE LITERATURE Introduction This chapter contains a review of the relevant literature related to the present study of behavioral and environmental cues that may indicate the presence of debilitating latent issues. The review addresses four major areas of scholarly writing: (1) an organization development approach to training, (2) behavioral and environmental cues, (3) latent issues, and (4) the Delphi research method. Since there is limited information on the identification of behavioral and environmental cues, and since such cues can only be fully appreciated with an understanding of their cause, special attention is given to the concept of latent issues. An Organization Development Approach to Training When a training professional initially begins preparing for a training project, the first order of business is usually to conduct a "needs assessment." A traditional needs assessment focuses on the instructional needs of the organization: exactly what 19 20 knowledge, skills, and attitudes do we want participants to learn from this training? (Goldstein, 1986) A conscientious, thorough needs assessment will ensure that the training program addresses all the critical content areas. However, many human resource development experts believe that the training professional who stops asking questions when the needs assessment is complete will almost certainly lack crucial information needed for training success. (Brunsson, 1985; Camp, Blanchard, Huszczo, 1986; Katz, Kahn, 1966; Knowles, 1984; Rogers, Roethlisberger, 1952) Just because we have established what needs to be taught, we have little indication whether or not the individuals within the organization are ready to lgggg and change. This, it is argued, is the primary reason why training professionals could benefit from adapting an organization development approach to training. To conduct a successful training program requires more than simply teaching skills to individuals. Kurt Lewin's work in social psychology clearly established that individual behavior in a group context is significantly shaped by social norms. Individual training often requires individual deviance from accepted norms. Since conformity to social "regulators" is a powerful determinant of behavior, training 21 participants are more likely to conform to patterns of expected behavior than to violate such norms by applying skills they learned in a training program. (Lewin, 1958) Daniel Katz and Robert Kahn would agree that trainers are mistaken if they believe training participants will use their new skills back on the job just as they did in the classroom: The major error in dealing with problems of organizational change, both at the practical and theoretical level, is to disregard the systemic properties of the organization and to confuse individual change with modifications in organizational variables. It is common practice to pull foremen or officials out of their organizational roles and give them training in human relations. Then they return to their customary positions with the same role expectations from their subordinates, the same pressures from their superiors, and the same functions to perform as before their special training. Even if the training program has begun to produce a different orientation toward other people on the part of the trainees, they are likely to find little opportunity to express their new orientation in the ongoing structured situation to which they return. (Katz & Kahn, 1966) One of the earliest definitions of "organization development" (OD) was offered by Bennis (1969): "Organization development is a response to change, a complex educational strategy intended to change the beliefs, attitudes, values and structure of organizations so that they can better adapt to new technologies, markets, and challenges, and the dizzying 22 rate of change itself." While several definitions of organization development have since appeared in the literature, three characteristics of OD are consistently mentioned in those definitions: an open system perspective, long-term planning, and a process rather than program orientation. Each of these three traits, it may be argued, could be included in the definition of effective training program planning strategies. In the following three paragraphs, each characteristic will be examined in detail. OD interventions that have lasting impact on an organization tend to employ an open system perspective. (Burke, 1986; Camp, Blanchard, Huszczo, 1986; Katz and Kahn, 1966) An "open systems perspective" may be described as a system that includes a series of interrelated parts and interacts freely with its environment. (Camp, Blanchard, Huszczo, 1986) In short, this means that effective OD efforts must include multiple opportunities for two-way communication and feedback between individuals and groups throughout the organization, including the communication of new ideas, feelings, beliefs, needs, goals, and environmental constraints. Camp, Blanchard, Huszczo (1986), Goldstein (1980), and Wexley and Latham (1981) agree that an open system perspective is just as beneficial for training efforts as for OD interventions. 23 An effective OD intervention is also characterized as a long-term planned change process. (Beckhard, 1969; Bennis, 1986; Camp, Blanchard, Huszczo, 1986; French, Bell, 1984) Any organizational change that demands an accompanying adaptation in human behavior takes time. The individuals involved in the change need to be mentally and emotionally prepared before the change takes place, then carefully trained, then repeatedly reminded of the benefits of the new system, and finally followed-up to ensure a workable understanding of the new skills involved. This level of intensity can rarely be accomplished through a short-term, one shot training program. In fact, some training professionals believe that the true impact of training can only be measured over the long run. (Amaya, 1980) The third characteristic of effective organization develOpment interventions is the perspective of on as a process rather than a program. Many authors stress that OD practitioners should help the organization to develop a self-generated problem-solving process, rather than swoop in and "save the organization" by telling it what to do (see, for example, Schein, 1987). OD is a process of change that never ends. Camp, Blanchard, Huszczo (1986), Katz and Kahn (1966), and Wexley and Latham (1981) believe that training should also be viewed as an on-going process rather than discrete programs that have 24 beginnings and endings. These organizational change specialists believe that a process approach to training would help trainees to help themselves with needs assessment and program planning skills. It has been established that such an approach to organizational change will reduce resistance to training efforts since the trainee is more involved in the process, rather than simply an audience to a product. (Mann, 1957) The three common characteristics of organization development discussed here, (an open system perspective, long—term planning, and a process rather than a program orientation), are also commonly believed to be characteristics of effective training programs. Whether or not change agents in the field actually use these techniques remains to be seen; no documented research addressing their prevalence was found. Nonetheless, if certain OD strategies appear to enhance training effectiveness, if these strategies help to identify and ameliorate debilitating latent issues, then it may behoove training professionals to adopt them. The ability to recognize common behavioral and environmental cues, and to link those cues to underlying latent issues, would enable training and development professionals to approach training projects with an awareness of organization-wide influences. 25 Behavioral and Environmental Cues The concept of problem recognition via behavioral cues is grounded in traditional psychological assessment. Goldfried and Kent (1972) described traditional psychology's endorsement of the "sign approach” to the interpretation of behavior as an "indirect manifestation of some underlying cause or characteristic." This is in contrast to behavioral psychology's interpretation of behavior as a "sample of the actual behaviors of interest." (Goldfried & Kent, 1972) In other words, the concept of behavioral cues, as applied in this study, is in keeping with the traditionalist's view of behavior as an effect, rather than that of behavioralists, who regard a behavior as a symptom or simply an object of investigation. Environmental cues are important indicators of latent issues because they provide tangible evidence of past behavior. For example, a company policy is an example of an environmental cue; the policy obviously exists as a result of some individual's or group's behavior, (i.e., decision) and that behavior may have been linked to one or more latent issues. Katz and Kahn (1966) believe that there are basically two types of intraorganizational cues that provide feedback for internal functioning: technical and human. The technical type of feedback includes 26 accounting for production work in the organization, such as the number of pieces produced, the use of raw materials, and the number of hours each employee spent on a job. The human type of feedback (which is the focus of this study), is much less tangible and measurable; employee morale, feelings, attitudes, and beliefs cannot be precisely recorded in a ledger at the end of the day. According to Katz and Kahn, the human type of feedback typically reaches the upper echelons only when some problem has become acute. "If there were full and accurate communication up the line then (cues) would not be needed, but the barriers to such upward communication are too numerous and too strong to ignore." (Katz and Kahn, 1966) A basic premise of the present research is that training professionals cannot afford to wait until a problem becomes acute and the "human type of feedback" meanders along its usual, indirect path to management awareness. Equipped with the ability to recognize such cues, the alert training specialist can search out behavioral and environmental indicators of latent issues early enough in the training process to effectively intervene and/or adapt to the problem. A systematic method of labeling and recognizing behavioral and environmental cues is needed. (Blake, Mouton, 1971; Lawler, Nadler, Cammann, 1980) This 27 review of the literature revealed no research-oriented writings that attempted to identify observable indicators of underlying latent issues. Practitioner literature vaguely refers to "...talking with people...observing carefully...keeping your antennae waving...joining small groups for lunch...asking lots of questions..." when attempting to gather information about the organization. It appears that "the seasoned professionals" tend to just mysteriously snoop around, following hunches and ambiguous leads until they feel confident that they understand the people and processes involved in the training effort. (Brunsson, 1985; Tichy, 1973) Organizations, with their idiosyncratic cultures, histories, and individual members, are tremendously complex. It is doubtful that many training professionals exist who can objectively, accurately, and efficiently assess the emotional climate surrounding an organizational training effort. A systematic approach to situational analysis can provide a mental map or blueprint for the training coordinator that provides structure and saves time. (Lawler, Nadler, Cammann, 1980) Edgar Schein is a strong proponent of a deliberate, systematic approach to the assessment of the invisible yet powerful forces at work within an organization. 28 While Schein refers primarily to the assessment of organizational culture, his basic concepts apply just as well to that of latent issues. Schein stresses the importance of scrutinizing such phenomenon carefully, because initial impressions and assumptions are often inaccurate. (Schein, 1985) Also, he believes that the change agent who engages in systematic observation can verify that initially "surprising" events are indeed repeatable experiences and thus an accurate reflection of the organizational climate and not merely random or idiosyncratic events. (Schein, 1985) Latent Issues A Definition: The term latent issue is defined in this study as hindrances to training effectiveness that are commonly overlooked or underestimated by training professionals and are generally associated with a perceived threat to emotional, social, occupational, or financial well-being. This definition is based upon Robert K. Merton's definition of ”latent function", meaning, "unintended or unrecognized consequences for a specified unit (person, subgroup, social, or cultural system) which contribute to its adjustment and adaptation.” (Merton, 1968) Gerald Moore (1984) later talks about "latent functions" as meaning "functions 29 that are not publicly or officially recognized by the participants as related to the actions in question." The fact that latent issues are associated with a perceived threat to one's emotional, social, occupational, or financial well-being is consistent with the beliefs of personality theorists Maslow (1954), Allport (1937) and Rogers (1951). Abraham Maslow's hierarchy of needs, which includes physiological, safety, love and belongingness, esteem, and self- actualization, closely parallels the concept of personal well-being that may be threatened when latent issues arise. Specific perceived threats that are targeted in this study include the following: 1. ”My authority is threatened." 2. "My job security is threatened." 3. "My reputation is threatened." 4. "My opportunity for job advancement or a raise is threatened." 5. "My financial well-being is threatened." 6. "The loyalty of my co-workers is threatened." 7. "My right to be recognized as an individual is threatened." 8. ”My right to voice my opinion is threatened." 9. "My comfortable, familiar work environment is threatened." 10. "My personal autonomy is threatened." 11. "My self-esteem is threatened." 12. "My physical well-being is threatened." 3O 13. "Some aspect of my personal life is threatened." Training Difficulties Associated with Latent Issues: Latent issues surrounding a training effort often interact with the stated training problem to create unanticipated (and undesirable) consequences. Gerald Moore (1984) discusses the common dilemma with which outside consultants must cope: the problems presented by the client for remedy rarely reflect latent issues that have direct baring on the success of the intervention. "The presenting problem may be due to actual, objective, manifest problems of the organization, or to underlying, latent problems that may not be apparent to those who define the presenting problem." (Moore, 1984) For example, the training specialist may be called in and asked to conduct a management training program because the client feels this will reduce turnover among supervisors. The training professional may recognize (through a combination of cues) that the actual problem is that the company is not competitive in its salary scales. The decision must be made, then, to conduct the training as requested, attempt to enlighten the client to the real (now latent) issue, or turn down the contract. 31 A second type of problem that may arise when latent issues are present is suggested by Ronald G. Havelock (1973) ”All too often, we accept (the training problem as stated) without much further thought because we are eager to move on the 'solutions.‘ If you do not take the time to study and understand the current state of the system, your change efforts are likely to be misdirected - and disappointing in the long run." (Havelock, 1973) A third consequence of unexplored latent issues may be immediate results that are desirable but short-lived: "When a program is imposed upon an organization without dealing with its 'unconscious' (i.e. latent issues") at first the change may appear to succeed, but gradually unseen forces take over until finally the change is no longer visible. The organization, like a giant, soft, resilient pillow, accepts the changes laid upon it, then gradually puffs out again as if nothing had happened. Results are only temporary - they cannot be permanent until the organizational (latent issues) are dealt with effectively and the 'soft pillow' of the organization is remolded into a shape that will support the desired innovations." (Allen & Kraft, 1982) Lastly, an ethical dilemma, articulated by change agent Roger Harrison in a 1973 interview with Noel Tichy, commonly poses a hardship for conscientious training professionals. When latent issues are not recognized and explored, efforts to bring about change may serve the needs of the organization to the detriment 32 of the individual employees. Harrison explains, "I frequently find myself in conflict between an organization's need for more effective and efficient use of its resources and the higher order needs of people in the system...." (Tichy, 1973) At this point the training professional must decide whether to deliver the service as requested, or attempt to work with the organizational decision-makers to better meet the needs of both the organization and the employees. "The costs of neglecting the psychological domain of organizations are unacceptable." (Baum, 1987) There is strong evidence in the literature to suggest that training professionals would do well to be aware of the possible presence of latent psychological issues that may surround any organizational change effort. Discounting of Latent Issues in the Literature: When present, latent issues can complicate and confound even the simplest of change efforts. No doubt there are many managers, organization development consultants, and training professionals who would prefer to work in an environment that is void of human emotions and biases. Max Weber believed that the ideal bureaucracy was a system which "eliminates from official business love, hatred, and all purely personal, irrational, and emotional elements which escape calculation." Weber not 33 only wished to ignore psychological influences on organizational behavior, but regarded "latent goals" as "errant deviations that necessarily subvert the organization's rational means and ends." (Weber, 1946) In reference to Abraham Maslow's theory of hierarchical needs (1954), Fremont Kast (1961) discusses the responsibility of the employer to meet those needs. The United States economic system, he says, has been very successful, by and large, in meeting basic physiological and safety needs, but not higher needs. Kast believes that higher order needs have assumed a much more important position in our motivation structure than they would in a society of scarcity. (Kast, 1961) Overcoming scarcity is an insufficient incentive for workers since, in most communities in the United States, scarcity no longer exists. Thus needs are pushed to higher levels. Kast believes that management should not direct its efforts toward satisfying these higher-order needs, (that is, should not strive to cope with latent issues), because, "The assumption that the business organization should be able to satisfy all of the needs of its participants and still accomplish its goals is in direct conflict with the requirements of organized behavior.... Fulfillment of these (higher-order needs) cannot be provided directly by management, but ...by the informal work group, the employee's outside professional 34 relationships, home life, social activities - in fact, his entire sociocultural environment." (Kast, 1961) Max Weber and Fremont Kast worked diligently to free the organization from "responsibility" for the personal concerns of employees. Their theories have been discarded, however, since it became apparent that, like it or not, organizational productivity is directly affected by employees' personal concerns. We now recognize that it's not a matter of social responsibility so much as desire for increased efficiency that motivates change agents to understand latent issues. Empirical Support for the Concept: The concept of latent issues within organizations, and the impact of latent issues on the effectiveness of change efforts, is discussed at length in the organization development literature. Latent issues are addressed in a more piecemeal fashion in the training and development literature, and usually with a more practical, less theoretical approach. For the purpose of research, there is much that training and development professionals can learn from the organization development field. Because latent issues are not by themselves observable, OD researchers tend to focus on positive 35 aspects of the observable work environment which may indicate the absence of latent issues. Where this study searches out cues that indicate underlying problems, these researchers documented indicators of a healthier change environment. For instance, Baumgartel and his associates (Baumgartel & Jeanpierre, 1972; Baumgartel, Reynolds, and Pathan, 1984) have conducted a line of research indicating that managers in favorable organizational climates (with freedom to set goals and a supportive environment) are more likely to apply new knowledge to work settings. Hand, Richards, and Slocum (1973) concluded that positive changes in human-relations skills eighteen months after training were due to organizational changes such as salary and promotions that reinforced the attitudes learned in training. Huczynski and Lewis (1980) found that a pre-course discussion with one's boss and subsequent boss sponsorship contributed most to the transfer of learned skills. Results from a study by Hicks and Klimoski (1987) show that a trainee's perception that he/she had a choice to attend (or not attend) a management development training program influenced motivation to learn and subsequent learning in the program. All of these studies resulted in evidence of a positive 36 relationship between the absence of latent issues and training effectiveness. Current Response to Latent Issues: How much effort do training professionals invest in seeking out, understanding, and coping with latent issues? Keichel (1982) refers to "the flight of the sea gull" as a metaphor for the typical "quick fix" approach to organizational change consulting. "It was, according to the industry joke, the sea gull model of consulting. You flew out from Boston, made a couple of circles around the client's head, dropped a strategy on him, and flew back." (Keichel, 1982) At present, no empirical studies exist that answer the question, "To what extent do change agents try‘to seek out, understand, and cope with latent issues?" However, there are many noted authorities in the organization development and training and development literature who believe that many, (if not most) professionals unwittingly advocate Keichel's "sea gull model" of consulting. In general, these authors agree that training professionals too often tend to focus on the more tangible, concrete aspects of pre-training analysis and ignore the more nebulous factors. (Baum, 1987; Brunsson, 1985; Havelock, 1973; Pilon & Bergquist, 1979; Steel, 1975; Witkin, 1977) As Howell Baum 37 described it, "To my surprise, I found that many planners do not regard their work as having a social or organizational environment. Rather, they concentrate on analyzing data in a world of pure information.” (Baum, 1987) The Delphi Research Method The Delphi method of data collection was originally developed and popularized by the RAND Corporation in the mid-1960's. (The method is entitled "Delphi” to honor the great oracle of Apollo.) (Orlich, 1978) Basically, the method involves the participation of any number of content experts in a written decision-making process. The respondents participate in three or more rounds of surveying, where they receive their own data and the data for the entire group prior to each round. The respondents are encouraged to re-think their previous responses and respond to new questions in light of the latest group consensus. By using multiple submissions of the same set of data each respondent reaffirms original opinions, modifies some, or adds additional items to the list. The technique aids in the forming of a clearly defined convergence pattern of major points, plus a well-outlined minority opinion. (Orlich, 1978) As a data collection tool, the Delphi technique has a number of advantages over other descriptive research 38 methods. First, the Delphi allows a researcher to obtain the opinions of a selected group of experts - individuals who are rarely available for group consultation. Secondly, even though the contributors to the Delphi exercise work from a distance, repeated polling and feedback from other contributors allows for a sense of ownership and involvement. From one round of questions to the next, a respondent can see the impact of his/her contribution, and see how it interacts with the contributions of others. (Orlich, 1978; Weaver, 1988) Next, this methodology allows for group interaction, while virtually eliminating the interference of personality conflicts and interpersonal politics. High status individuals or a vocal minority cannot force group discussion in the direction which they deem desirable. (Linstone, Turoff, 1975; Weaver, 1988) In effect, the Delphi technique minimizes the opportunity for debilitating latent issues to germinate within the group of respondents. Because the contributors are asked to carefully reexamine their position after viewing the responses of other members, the Delphi method will typically gather more thorough and accurate information than a one-shot 39 survey, and will likely build a degree of consensus among the contributors. (Linstone, Turoff, 1975) The Delphi technique allows the researcher to organize and prioritize the collective judgments of the polled group. According to Donald Orlich, the Delphi " ... is an excellent way to seek inputs on what may be causes or effects in problem solution." (Orlich, 1978) This characteristic makes the technique especially suited for this study of latent issues (causes) and cues (effects). As with all research methodologies, there are a few disadvantages of the Delphi technique. First of all, the data collected from respondents can be easily swayed by the wording used in the questions. It is extremely crucial that the researcher's directions and statements he understandable and value-free. (Orlich, 1978) Experts in the Delphi method seem to agree that the initial survey round should consist of open ended questions. (Orlich, 1978; Linstone, Turoff, 1975; Borg & Gall, 1979) Responses would then be edited for compilation. If the response lists are very lengthy, then additional editing, classifying, and grouping must take place so that a meaningful response pattern can be generated when the questionnaire is redistributed to the respondents. Subjectivity may enter into the editing of initial or added statements. (Orlich, 1978) 40 A final disadvantage of the Delphi technique concerns the method chosen for ranking responses when group consensus is unclear. When responses are more scattered than convergent, the method used in making the rankings might shift the priorities. Several techniques may be used to determine arithmetical scales, such as (1) total weighted average, (2) median ranking, (3) frequency rankings, (4) unweighted frequency, or (5) rank orders. "In determining the weights of a Delphi survey, Dr. Toshio Akamine, a colleague of the author, noted that items would shift upward or downward in priority, depending on the ranking method used." (Orlich, 1978) Akamine's observation illustrates a subjective characteristic of the Delphi technique as one attempts to quantify responses. "It is not to be construed that the Delphi technique is not a powerful tool. This analysis points out that there is a problem in ranking the items (when there is a lack of consensus)" (Orlich, 1978.) Ms: Training professionals can benefit from an organization development approach to training. Two-way communications, and a long-term planned change process are all common characteristics of organization development that may affect training programs. 41 The concepts of cues and latent issues are grounded in traditional psychological literature. Because these manifestations of human emotion are so complex, a systematic method of recognizing and labeling is needed. Finally, the Delphi research method was discussed at length in this review of the literature. Used appropriately, the Delphi is a powerful tool in exploratory research. CHAPTER III RESEARCH METHODOLOGY Introduction The primary goal of the investigator was to identify behavioral and environmental cues, as perceived by training professionals, that indicate the presence of debilitating latent issues. Specifically, the researcher strove to answer the following questions: 1. What behavioral and environmental cues are perceived by training professionals as indicators of debilitating latent issues? Which cues are perceived by training professionals as the most valid indicators of debilitating issues? Which cues are perceived by training professionals to be associated with which latent issues? Which cues can be traced by training professionals to the most urgent threats to training effectiveness? Which cues are associated with which stage of training program implementation? (Considering Pre-Training, Training, and Post-Training Stages.) Which cues tend to originate from each of the following three groups of individuals: upper level managers, supervisors of participants, or training participants? To that end, such cues were solicited, (via a three-round Delphi technique,) from seasoned training 42 43 professionals, then rank-ordered according to the professionals' perception of their validity and their degree of urgency. Finally, the investigator solicited suggestions from the contributing training professionals to link the cues with their most likely corresponding latent issues. This chapter includes a description of the research population and sample, the criteria for selection of the research sample, a discussion of the development of the Delphi questionnaires, and an account of the data collection and analysis procedures. Population of the Study The population for this study included the training coordinators, developers, and presenters who were permanently employed by either a private or public organization. Only "internal" training professionals were included because it was assumed that they would have extensive contact with the organization before, during, and after training (as opposed to "external" training consultants,) and so would be best equipped to recognize and understand cues and latent issues. Criteria for Selection of the Sampla A research sample of forty subjects was selected from the 1988 membership lists of the National Society 44 for Performance and Instruction (NSPI) and the American Society for Training and Development (ASTD). A sample size of forty was large enough to sufficiently minimize sampling error and small enough to make the synthesis of the respondents' ideas manageable. The sample subjects were selected from NSPI and ASTD membership lists according to the following criteria: a) b) At the time of the study, subjects worked as internal training professionals. ("Training professionals" was defined earlier as individuals who work as coordinators, developers, and/or presenters and who are ultimately responsible for the quality of training programs.) The subjects each possessed a minimum of five years experience in the field of training and development. This amount of experience ensured that the participants had adequate exposure to cues and latent issues, and so were equipped to understand the concepts and offer examples. Also, a training professional with five years of experience would most likely possess an adequate stockpile of cues and latent issues from which to draw. 45 c) The subjects worked in either private or public organizations, without restriction to industry type. Because only internal training coordinators, developers, and presenters were needed for the study, certain limitations were placed on the final selection process. For example, the coded membership information offered by the NSPI directory was used to screen out all but those individuals who coordinate, develop, or deliver training programs. An individual who listed him/herself as a "consultant" or a "student" was not likely to meet the selection criteria, and so was not contacted. During the introductory conversation, each candidate was asked how long he/she had worked in the field of training and development. Those who stated that they possessed less than five years of experience, (this included just two individuals,) were not included in the study. To maintain participant confidentiality, care was taken to never contact more than one person from any particular organization. To expedite the data collection process, and to limit cultural influence, only those individuals who worked within the continental United States were approached. To secure the forty 46 subjects needed for this study, candidates were selected using the following procedure: Beginning with the first individual listed in the 1987-88 NSPI Directory of Membership, every twelfth person on the first page was selected. Five pages later, every twelfth person was again selected. (The numbers "twelve" and "five" were arrived at randomly with a roll of dice.) If the twelfth person did not meet the selection criteria, or no longer worked or resided at the address listed, then the thirteenth name was considered, and so on, until a candidate was contacted. This procedure was repeated until the NSPI membership list was exhausted, then the selection process continued in the same fashion with the 1988 ASTD membership list. Once an NSPI or ASTD member was recognized as a possible candidate, he/she was contacted by telephone and the research project was explained briefly. (See Appendix A for the telephone script.) If the individual offered a verbal commitment to participate in the study, then he/she became a subject. Descripaion of the Sample The subject pool consisted of twenty-three men and seventeen women, representing twenty different states. Twenty-nine of the subjects were members of NSPI, eleven belonged to ASTD. Together, the subjects possessed over 47 348 years of experience in the field of training and development, with an average of 8.5 years, range of 5 - 28, and median of 11.5. The study participants worked within organizations that employed anywhere from 100 to 36,000 individuals. Thirty-one of the participants worked in the private sector, nine in the public sector, and the following occupational areas were represented: (These categories were adapted from the National Society for Performance and Instruction occupational coding system.) - Manufacturing/Industrial - Financial Services/Insurance - Utility/Communications - Service Industry Retail Sales - Health Services — Education - Military - Law Enforcement HI—‘l—‘NNQO‘OU‘I I ah 0| Total Number of Participants Nine of the subjects described themselves as training program coordinators, six work primarily as training program presenters, and twenty-five individuals work alternately as coordinators, developers, and presenters. Development of the Delphi Questionnaires The Delphi method of data collection involves the participation of any number of content experts in a 48 written decision-making process. The technique aids in the forming of a clearly defined convergence pattern of major points, plus a well-outlined minority opinion. (Orlich, 1978) Because the participants are asked to carefully reexamine their positions after viewing the responses of other contributors, the Delphi method typically gathers thorough and accurate information as it helps to build a degree of group consensus. The data collection effort involved three separate questionnaires which comprised the Delphi procedure. In the first questionnaire, henceforth referred to as "Round One", the investigator outlined three common stages of training and asked participants to report several behaviors, situations, or objects that they believed were cues. In "Round Two", the thirty-five most-often suggested cues from Round One were validated, and suggestions of possible latent issues behind each cue were solicited. In "Round Three", the investigator asked participants to estimate the degree of urgency for each of the seventeen most-often validated cues in Round Two, and sought validation on the suggested latent issues. Thus, each of the research questions were systematically approached and then validated throughout the three Delphi rounds. What follows is a detailed account of the developmental process for each questionnaire. 49 Round One Questionnaire (Appendix B): The first questionnaire was an exploratory inquiry. It included a three-stage organization of training program implementation, (”Pre-Training" preparation, "Training", and ”Post-Training" follow-up), which was designed to solicit the participants' suggestions of cues. The Pre-Training stage was described as typically including the following ten tasks: Determining Training Content Allocating Budget Selecting Instructors Program Scheduling Creating Lesson Plans Selecting Participants Preparing Supervisors of Participants Preparing Upper Level Managers Preparing Class Materials Preparing Training Site The Training stage was described as typically including the following six tasks: Participant Orientation Presentation of Content Skill Practice Participant Evaluation Program Evaluation Classroom Management The Post-Training stage was described as typically including the following four tasks: Skill Transfer Supervisor Support Upper Level Management Support Training Follow-up By guiding the Delphi respondents through these three training implementation stages, a degree of 50 response uniformity was maintained. These three stages were arrived at after consideration of the training procedures recommended by Laird (1985), Wexley and Latham (1981), Goldstein (1986), and Zemke and Kramlinger (1982). The components of each stage appear to the investigator to be standard, and it was thought that a "before, during, and after" presentation of these components would effectively guide participants through the training process with little confusion. In addition to guiding the participants through the three stages of training, the questionnaire provided a simple means for the respondents to signify from whom each cue originated. Each time a participant wrote down a cue, he/she also checked whether the cue originated from an upper level manager, a supervisor of training participants, or a training participant. This information would later help to more clearly distinguish one cue from another. Review of the "Round One” Cover Letter and Questionnaire: Before sending the first round of Delphi questions to the sample subjects, the Round One procedure and written materials were "pilot tested" by a group of six local training professionals. The individuals in the pilot group held occupational roles similar to those in the research sample. The reviewers 51 received a brief explanation of the research goals during an introductory telephone conversation (Appendix A), and were then sent the Round One cover letter, instruction sheet, and questionnaire, along with a feedback sheet (Appendix C). The purpose of the review was to solicit the following feedback: a) Does the preliminary telephone conversation adequately prepare subjects for participation in the study? b) Is the purpose of the research easy to understand? c) Is the cover letter sufficiently informative? Does it help to put participants "at ease" with the research process? d) Are the written instructions easy to follow? e) Does the layout of the questionnaire seem logical? f) How much time was needed to complete the questionnaire? (This information helped the researcher to estimate the amount of time needed from each sample subject, so that subjects may be forewarned.) Results of the Review: All six of the reviewers responded positively to the Round One questionnaire and the request for feedback. Comments were favorable; all 52 reviewers considered the introductory telephone conversation, the cover letter, the instruction sheet, and the questionnaire to be readily understandable. Also, all reviewers stated that they thought the study would be interesting and helpful. A few specific suggestions were offered: a) b) C) Lengthen the estimated completion time for Round One from "ten-to-twenty" minutes to "twenty-to- thirty" minutes. All six of the reviewers agreed that this was a more realistic completion time. (This suggestion was incorporated into the final draft of the cover letter.) Add a brief definition of "cue" at the top of each questionnaire page. Two of the reviewers believed that such a reminder would help the participants to keep their responses relevant. (This suggestion was incorporated into the final draft of the questionnaire.) One reviewer suggested that a forth round be added to the Delphi process, allowing participants to explain how they go about investigating latent issues. (This suggestion was not incorporated into the study. While the additional information is indeed relevant and important, the investigator believes it to be 53 out of the realm of the present research effort.) In addition to these specific suggestions, the reviewers also provided many positive comments concerning the organization and content of the introductory telephone conversation and Round One materials. As a result, the Round One methods and materials were finalized with few editions, (Appendix B). The pilot test also provided a small sample of cues, which reflected the clarity of the instrument instructions. Round Two Questionnaire (Appendix D): One of the unique benefits of the Delphi technique is that each round of questions is guided by the participants' ideas generated in the previous round(s). For this reason, the content of the Round Two questionnaire was finalized only after the participants had responded to Round One. As the participants suggested cues to underlying latent issues, the investigator constructed a composite list. Duplicate cues were combined and the number of suggestions tallied so that a final list of one hundred eight cues could be rank-ordered from most-often suggested cues to least-often suggested cues. The complete list of cues suggested by the study 54 participants is reported in Chapter Four. Because one hundred eight cues presents an unwieldy number for concentrated response by most study participants, a process of systematic elimination was implemented. Through this weeding-out process, the number of cues was reduced to thirty-five. Cues were eliminated according to the following criteria: a) The cues that were suggested by just one participant were eliminated. b) The cues that resulted from a training professional's error were eliminated. If the training department doesn't do its job, or if training quality is poor, then the resulting problem/complaint is due to incompetence, not a latent issue. c) The cues that resulted from a misunderstanding of the training process were eliminated. If a manager has unrealistic expectations of the training function, or makes a ridiculous training request, then the resulting problem/complaint is due to management ignorance of the training process, not a latent issue. The resulting Round Two questionnaire (Appendix D) included thirty-five of the cues suggested by the study 55 participants in Round One. The cues were separated into three groups: those that originated from upper level managers, supervisors of training participants, and the training participants themselves. The research subjects were asked to consider this composite list of suggested cues, then indicate whether or not they agreed that the suggestions were indeed cues. For each cue, the participants were asked the question, "Is this a cue?", and were restricted to a simple "yes" or "no" response. This affirmative/negative response format forced the respondents to make a definitive decision on an admittedly complex question. By not allowing respondents to say "Well ... it depends...." the investigator sought to illuminate any slight definition that may exist: aside from the confounding variables, which behaviors tend to be the most consistent indicators of underlying issues? Also in Round Two, the participants were asked to speculate the "cause" (latent issue) behind each cue by considering the possible threatened feelings experienced by each actor. Rather than use the term "latent issue", which could be confusing to the individuals whose only contact with this topic was an occasional questionnaire, the concept was described as one that results from threatened feelings. 56 The information collected from Round Two allowed the list of thirty-five cues to be even further qualified, and provided a "starter list" of latent issues which the study participants could validate in Round Three. Round Three Questionnaire (Appendix E): The purpose of Round Three was to validate the previously collected suggestions of the latent issues that cause cues, and to solicit opinion on the degree of urgency for each cue. To do this, the field of thirty-five cues from Round Two was further narrowed to seventeen. Again, this narrowing-down process served to make the list of cues more manageable for the respondents. The number "seventeen" was a logical cutting-off point, as a notable drop in cue validation percentages (from 52% to 38%) occurs between the seventeenth and eighteenth cues. Also, each of the seventeen cues selected for Round Three were validated by at least half of the participants. In such a homogeneous subject pool, only a modest percentage of participants need affirm a cue to consider it valid. (Borg and Gall, 1983) The seventeen cues were presented in Round Three along with all of the possible latent issues that were suggested by study participants in Round Two. Even if a 57 particular latent issue was suggested by just one person in Round Two, it was included in Round Three. The participants were asked to place a "1" next to the most likely threat (latent issue) for each cue, and a "2" next to the second most likely threat. Participants also had the option of writing in a different threat that didn't appear on the list. In addition, each study participant was asked to determine the degree of urgency for each cue in light of its most likely underlying issue, (i.e., the threat that he/she ranked "1"). This was done by rating the degree of urgency on a four—point Likert scale. The four ratings included: "4 = A Nearly Unsurmountable Obstacle, 3 = A Very Serious Obstacle, 2 = A Difficult Obstacle, l = A Minor Obstacle." Data Collection Schedule Because the Delphi technique involved several steps taken over a period of four months, the process of this data collection effort is outlined below. DATE ACTIVITY 5/17/89 to Initial telephone contact was made with 5/24/89 each of the forty study participants. Round One survey materials were mailed out during the same period. 6/08/89 A telephone call was made too those participants who had not yet responded to Round One. (Many of the participants who 6/13/89 7/03/89 7/17/89 8/04/89 8/16/89 12/06/89 Summagy 58 had responded could be identified by the postmark on the return envelope. If there was any doubt as to whether or not an individual had responded, then he/she was offered a reminder with a telephone call.) For those individuals who had responded to the first questionnaire, a card was sent (Appendix F) to thank them for their prompt response and to ask for their patience while waiting for the Round Two materials. Round Two survey materials were sent out. A reminder card was sent to those participants who had not yet responded to Round Two (Appendix F). Round Three survey materials were sent out. A reminder card was sent to those participants who had not yet responded to Round Three (Appendix F). A summary of the results of the study was sent to all participants. In Chapter III, the population of the study was defined, and the quantity and quality of participant response was addressed. The sample subjects were drawn from the 1988 membership lists of the NSPI and the ASTD; 40 individuals who met the criteria for participation agreed to contribute. Together, the subjects possessed over 348 years of experience in the field of training and development, with an average of 8.5 years, (range 5 - 28, median 11.5). 59 The Delphi technique of data collection was employed in this study; the development and method of analysis for each of the three questionnaires was described. A review, or "pilot test", of the Round One materials was conducted with six local training professionals. The results of that review, which were largely positive and confirming of the materials, was provided. Finally, a schedule detailing the data collection procedure was provided in the chapter. Chapter IV contains the results of the data collection conducted for this investigation. CHAPTER IV FINDINGS Introduction The purpose of this chapter is to present in descriptive form response data from the three Delphi questionnaires. The quantity and quality of the participant responses is addressed, and a complete report of the information gathered from each round of data collection is provided. This chapter contains only the reported findings of the study; a discussion of these findings appears in Chapter Five. Quantity and Quality of Participant Response Response Rate For the Three Questionnaires: The data collection effort began with forty participants. Thirty-seven of those participants responded to one or more of the questionnaires. Twenty-eight participants contributed to all three Delphi rounds. The response rates are shown in Table 1. 60 61 Table 1. Response Rate of Mailed Questionnaires. (N = 37) Mailing Number of Returned Percent Questionnaires Returned Round One 33 89 Round Two 29 78 Round Three 30 81 Available literature on the Delphi technique did not stipulate the minimum number of subjects required for quality data analysis. (As with most data collection techniques, the size of the subject pool is dependent upon the purpose of the research.) Likewise, the designation of a critical attrition rate is also left to the discretion of the investigator. Regardless of who responded to each round, the subsequent questionnaires were sent to all participants. For example, even if an individual did not respond to Round One, he or she was included in Rounds Two and Three. Participation in the latter rounds was not dependent upon participation in the earlier rounds, and this procedure increased the likelihood of adequate response to each questionnaire. Quality of Participant Responses: Judging from the introductory telephone conversations, the volume of 62 response data, and several unsolicited comments, it was apparent that the participants of this study were enthusiastic and generous in the sharing of their experiences and ideas. Participants seemed to have little difficulty understanding the instructions on the questionnaires. Only two individual contributions, both in Round Two, were discarded; those participants either did not read or failed to understand the instructions. It was apparent from the volume and variety of information offered by the participants that most spent a substantial amount of time reflecting on the topic. The ideas and suggestions shared by these training professionals quite obviously resulted from a substantial amount of experience and conscientious observation. The participants seemed eager to discuss their day-to-day frustrations associated with cues and latent issues. On several occasions, individual participants telephoned the investigator to discuss their own contributions in greater detail. Unfortunately, because of the large volume of ideas offered by participants, the investigator was forced to "screen out" many interesting contributions in order to make the subsequent questionnaires manageable. (The original data, however, was preserved and is reported below in its entirety.) 63 Results of Round One The purpose of the first questionnaire was to respond to the following research questions: 1. What behavioral and environmental cues are perceived by training professionals as indicators of debilitating latent issues? 2. Which cues are associated with which stage of training program implementation? (Considering Pre-Training, Training, and Post-Training Stages.) 3. Which cues tend to originate from each of the following three groups of individuals: upper level managers, supervisors of participants, or training participants? The Round One questionnaire (Appendix B) provided the study participants with an opportunity to describe some of the cues that they have witnessed as training professionals. A total of two hundred and seventy-three cues were suggested by the study participants. The investigator grouped all of the identical suggestions, so that one hundred eight different cues were ferreted out of the raw data. (Table 4, found later in this chapter, summarizes the distribution of the suggested cues according to training stage and actor.) A final list of thirty-five cues was compiled for the Round Two questionnaire; cues were systematically eliminated from consideration according to the following criteria: a) The cues that were suggested by just one participant were eliminated. 64 b) The cues that resulted from a training professional's error were eliminated. If the training department doesn't do its job, or if training quality is poor, then the resulting problem/complaint is due to incompetence or error, not a latent issue. c) The cues that resulted from a misunderstanding of the training process were eliminated. NOTE: The purpose of these criteria was to systematize the cue elimination process. The concepts of "training professional's error" and "misunderstanding of the training process" were interpreted and applied as consistently as possible. Still, some degree of investigator subjectivity probably influenced the screening process. Many of those cues not retained in Round Two reflect a great amount of insight on the part of the study participants, and they may provide the groundwork for later study. Therefore, the complete list of suggested cues from Round One is reported in Table 2. The cues are listed in the left hand column according to actor and training stage. The middle column shows the number of participants who suggested each cue, and in the right hand column those cues that were retained for Round Two are designated by asterisks. 65 Table 2. Guns Identified in Round One and Retained for Round Tao. liner Retaimd of for lines Round mggested cue: Suggested loo (t) 0125 FR!!! MR LEVEL MRS (hiss lbserved hung the Pre-lraining Stage: 1. hanageaent sakes a training deciSion that runs contrary to the 7 1 results of a needs analySis. 2. Hanageaent allocates aoney for a training prograa, but the aaount 6 t is grossly insufficient. 3. hamgeeent does not allocate aoney for a training program. 5 3 4. Training staff is not alloued sufficient preparation tile for an 5 * upcoaing training prograa. 5. hanageeent expects training staff to cover an inordinate anount 5 ’ eaterial in a snail aaount of training tine. 6. Hanageaent does not include training personnel in acetings for noon 4 1 products or procedures when the training of staff 1111 be necessary. 7. hanageaent does not set training goals according to long-range 3 ' organizational goals. 8. l'anageaent requests a particular type of training, then does not take 3 3 part in the subsequent planning of that prograa. 9. fianagers do not contribute to pro-training planning sessions. 3 f 10. llamgers sake last ainute changs in the training site or 3 t instructional eminent. ll. A IIMQ!’ refuses to oomroaise nth other unagers on training goals. 3 t 12. mgment insists upon imieaenting training for a non-training problee. 3 (Eliainated: imoranoe of the training process.) 13. hanageeent uants to include a topic in a training prograa simly because 2 it is a 'hot“ issue in aanageaent training today. (Eliainated: ignorance of the training process.) 66 Table 2 (cont’d.). Tamer Retained of for lines Round 91995106 910: ingested Tao l4. llanagaent hands that sort force subordinates be traimd before their 2 supervisors. (Eliainated: ignorance of the training process.) 15. Lack of fallen-w support free Training Unit. 1 (Eliainated: suggested by Just one subject.) 16. Course developaent is started vithout concurrence of supervisors and line 1 aanagers concerning content, depth, etc.. 17. A high priority training proiect is side-tracked for a perceived need in a l non-training area. 18. An insufficient meter of training staff sewers is asSigned to a training 1 project. 19. Training-related suggestions are solicited froa subordinates, then 1 categorically rejected by upper level aanageaent. 20. On-thrjob experts are penalized for volunteering as training instructors, 1 giving up their favored poSition upon return. 21. Negative co-ents in general touard training are offered during budget and l staffing aeetings. 22. hanageeent inSists upon placing a poor presenter on the instructor staff 1 as a developaental aove. 23. Job moments are not perforaing a skill properly. Rather than confront 1 that problea, aanageaent uants nee trainees to infiltrate the old ranks, iolaaentmg the skill properly. 24. lbnageeent refuses to alloe current soloyees to act as training instructors. l 25. Thug-em refuses to allou coouter based training. 1 26. A restriction is placed on the length of the participant aamal. 1 27. Training is categorized as 'non-prodictive' tiae, and so is a lou priority 1 in Willis. (Vacations, row and aaintenance scfediles take precedence.) 28. A aanager uses "scheming problees‘ as an excuse not to do training, when 1 others aanage aith little difficulty. 67 Table 2 (cont’d.). *— Mr Retained of for Tiees Round 91mm me: Suggested Tao 29. "amount sill only aooot non-referenced testing as a validation of learning. 1 30. flamgeaent sill only accept a change in soloyee turnover as an indicator 1 of training suwess. 31. Rampant hands that promotivity aeasuras of training effectiveness 1 concentrate on the process of training, (e.g., as the instructor sell Prepared, were the (HI aaterials clear, etc.) and not on the bottoa line of igact of training. Cues Observed During the Trainim Stage: 32. Hanagers do not 'sit-in' on portions of the training, as planned. 3 T 33. Hanagers coaait to training in private, but fail to deaonstrate that 2 f eta-itaent publicly to subordinates. 3‘- Instructor lacks skill or Rnoeledge to train participants or ansuer 2 fill-aestions. (Eliainated: trainer error.) 35- Nor. enough variety in presentation aethods. 1 (E 1 aainated: suggested by Just one subject.) 36- Lesson plans do not contain strategies and techniques to get the students 1 1'“!on in the learning process. 37‘ ”Be—tings or activities are schooled to conflict vith already scheduled l “lining prograas. 38' p‘f‘ticipants are forced to attend training, lith little or no preparatory l Nlanation. 39’ “ Immger agrees to act as a west apeaher for a training prograa, then 1 ‘hoaes is) for the presentation sithout having properly proared. ‘0‘ One. a training progm is unar say, the job tasks or cork enviroment are l Shiv-19d, rendering portions of the neatly-learned skills obsolete. 41. “inageaent is content to rely upon subiective course evaluation aeasures, 1 rather than invest in the developaent of objective aeasures. 68 Table 2 (cont’d.). farmer Retained of for Tiaes Round 319mm (hue: Suggested Tao y mas (beerved Thiripg the Post-Training Stag: 42. Participants learn a non skill on a nee piece of egiipaent in training, but 1 that eminent is not available uhen they return to tfeir regular 1er setting. 43. Hanageaent wands proof of return on investeent. l OTIS FROST SUPERVISORS If PARTICIPANTS Cues lbserved Durm the Pre-Training Stagg: 44. A supervisor of training participants is 51011 and haphazard in his/her 8 I. Efforts to register subordinates for training. 45. A superVisor of training participants does not provide necessary 8 3 1111‘ oraation for the needs analyses. 46. A supervisor of training participants does not attend pre-training planning 6 3 Sessaons. 0- A SUperVisor of training partiCipants does not adamately prepare subordinates 5 T '0' a training prograa, so participants cone to training not knouing chat to expect. ‘8‘ “ st-Itaervisor of training participants does not distribute training program 4 * ‘N'Oainceaents. ‘9’ “ summsor of training participants refuses to cooroaise vith other 3 ' rvisors on training goals. so 9 stastiervisor of training participants authorizes training for subordinates, 2 2 ”“t not for his/herself. 51 A WM of mm; participants does not giarantee that subordinates 2 f .111 attend training. 52 “ Slammer of training participants only allous subordinates to attend l tr‘aal'iing if it is schedaled on corsecutive days. (El a-inated: suggested by Just one subject.) 69 Table 2 (cont’d.). Timer Retained of for lines Round ingested cu: Suggested Tao 53. A supervisor of training participants calls the training coordinator eith 1 lots of giestions tout training content, coverage, participation, etc.. 54. hos or verbal announceaents of training are not supportive. l 55. Supervisors don’t know Uthh of their subordinates have been trained. 1 (Poor co-unication.) 56. Supe rvisors refuse to allou content experts to asSist training developers. 1 57- 'I can’t get his to do it right. flaybe you can get through to hia.‘ l (Indicating supervisor’s lack of understanding of chat training can do and Uhat the supervisor is responSible for.) mié mserved During the Trainm Stage: 58- 8 supervisor of training participants repeatedly pulls partiCipants out of 6 T the training program. 59- A sopervisor of training participants publicly critiCizes a training prograe. 3 3 - A SuperVisor of training participants expects subordinates to attend or 1 Prefiare for training in addition to working a full shift. (E1 immated: suggested by Just one subJect. ) 61. A supervisor she agreed to help teach a course shoes-up urprepared, saying, 1 can aing it.'. 62' 9 supervisor does not remire subordinates to attend training. 1 “w airmen-Imam 63' “ ‘Ipemsor of training participants does not reinforce tfe use of 17 t '1‘“ skills nun participants return froa training. 6" 9 Mrvisor of training participants refuses to alter the aork schedne 6 f 3° that subordinates can mic-ant nuly'trained stills. 65. A supervisor of training participants does not participate in training 4 1 m1 low-up activities. 70 Table 2 (cont’d.). mud Cue: y 66. A superVisor of training participants uho is not faailiar eith the content of the training progr- discourages coordinates froa using their hotel y-learned skills. (Eliainated: eiggested by Just one subject.) 67. R supervisor coolains that training folloe-up is a training departaent TBSp-onsibility, not a aanageaent issue. 68. A supervisor does not return the training feeaiack reelests that were 00” leted by subordinates. 69- A supervisor does not allow subordinates to attend follou-up training sesSions. 70- A superVisor does not conduct training follow-up according to a pre-arranged Structure, preferring to evaluate subordinate perforaance very casually. 71. A superVisor takes no interest in training, (”N01, ahich prograa as that?) 72- ‘1 supervisor’s only coaeent on a follow-up evaluation: 'They liked the training; they thought it as fun“. 73. A supervisor reports that he didn’t hear anything good about the training fro. subordinates. CUES FROH TRAINING PARTICIPANTS c“&mserved During the Tire-Training Stage: 7* A training participant is umiiiing to attend ore-training planning sesswnS- 75- The sarong target audience 519115 up for training. 76' ‘r‘l hing participants are unable/umilling to settle on a date or tine for ‘lning. “It: Deemed Duri the 1mm S : T’- 9 teaming participant verbalizes resenteent at having to attend training. Ittend training. 73‘ “ tJoining participant shoes diSinterest wring training by reading, not king notes, initiating side conversations, etc. lower of lines 9499:5135 14 Retained for Round Tao 71 Table 2 (cont’d.). Racer Retained of for lines Round 911mm cue: Suggested Tco 79. 4 trJining participant does not gt actively involved in the training, 6 * choosing instead to roam on the periphery of class activities. so. A training participant does not shoc-w for a long-schooled, such- 6 3 advertised training prograa. 81. A training participant freqiently leaves class for short periods of ma. 5 T 82. A training participant does not coaplete hoaecork assign-ents deeeed 4 t necessary for training success. 83. A training participant repeatedly fails to return froa class breaks on tine. 3 f 84. A training partiCipant 'chite cashes' the course evaluation, choosmg to 1 Ollt. critical co-ents. (Eliainated: suggested by just one subject.) 85. A training participant blaces ainor, inconsequential events for his inability l to learn the eaterial. 36- 8 training partiCipant expresses skepticisa about the quality of the training, 1 because it cas created by internal trainers. 87. A training participant doubts the quality of the prograa, 'I hear this class 1 'Oves sloc.' 88' 9 training participant fails to perfore cell on the final exaaination. 1 89° “ ti"aining participant reads slocly. l 90- A training participant has a qiizzical, skepticai look on her face. 1 91- 5- trgymng participant explains chy an idea or skill could not cork in l 13! her operation. 92‘ A tf‘aining participant is preoccupied cith back-hoe problees. 1 ‘3- 8* tor hours activities result in tired. “mos-over“ participants. 1 ‘14 ' cmrse evaluation fores are seldoa cooletely filled out, as trainees "3 anxious to leave. 95- “lining participants ask mestions/talk about probleas unrelated to the current topic of presentation. 72 Table 2 (cont’d.). tuner Retained of for lines Round ingested Cue: Suggested Tco 96. Training participants give poor evaluation of the training. 1 97. During course evaluation, a training participant focuses on irrelevant, l unioortant topics, (i.e., lunch aenu). Q. Not enough variety in presentation eaterials. 1 99. Instructor is inflexible to needs of the class. 1 100. A training participant belabors a saall pomt. l 101. A training participant pays excessive attention to detail. 1 102. Students share feelings that chat is taught in class in not the cay 1 things are really done back on the Job. (This say or say not be true, but it is chat the student perceives to be true that interferes cith training.) Cues lbserved During the Post-Training Stage: 103. A training participant does not apply nerdy-trained skills chen back 12 3 on the Job. 104. A training participant does not respond to training folloc-up actiVities. 6 f 105. Lack of folloc-ip support froa training unit that delivered the prograa. 1 (Eliainated: suggested by JUST one subJect. ) 106. A training participant reqiests the sue training on a folloc-up needs 1 assess-ant. 107. Literal skills transfer, but participant can’t use skill in varying contexts. 1 11!. No clear relationship betceen prograa evaluation and trend productivity. 1 73 Results of Round Two The purpose of the second questionnaire (Appendix D) was to respond to the following research questions: 1. Which cues are perceived by training professionals as the most valid indicators of debilitating issues? 2. Which cues are perceived by training professionals to be associated with which latent issues? To accomplish this, the study participants were asked to (1) determine the validity of the thirty-five most-often suggested cues from Round One, and (2) hypothesize the latent issues that are associated with those cues. Verification of Cue Consistency: A cue is considered "valid" or "verified" if the study subjects believe it regularly indicates the presence of one or more latent issues. The more valid a particular cue is, the more training professionals can rely on it as a signal of underlying training problems. For each of the thirty-five cues gleaned from Round One, the participants were asked the question, "Is this a cue?", and were restricted to a simple "yes" or "no" response. This exercise produced a verification of the cues: those behaviors or situations that were most often recognized by training professionals to be indicators of underlying training problems. Those cues in Table 3 that are labeled with an asterisk were included in Round Three of the study, (Na29). 71: Table 3. Verification of Cues. (The asaher in parenthesis folloving each cue is the original nusher assigned in Table 2.) Is This A Cue? Tes Io Retained for Round Three (1) 10. 11. h supervisor of training participants authorises training for subordinates. but not for hisself/herself. (50) A training participant shove disinterest during training by reading, not tahing notes, initiating side conversations. et cetera. (78) An upper level aanager coasits to training in private, but does not desonstrate that cossitsent publicly to peers and subordinates. (33) h supervisor of training participants publicly criticizes the training prograa. (59) A training participant verbalitee resentaent at having to attend training. (77) A training participant does not apply nevly-acquired stills chen bach on the job. (103) An upper level aanager ashes a training decision that runs contrary to the results of a needs analysis. (1) An upper level aanager does not include training staff in aeetings for nev products or procedures chen training of personnel 1111 be necessary. (6) A supervisor of training participants does not reinforce the use of sec chills chen participants return froa training. (53) 4 training participant does not shoc-up for a long-scheduled, such-advertised training prograa. (00) An upper level aanager allocates aoney for a training prograa. but the asount is grossly insufficient. (z) 22 22 21 20 20 20 19 19 19 18 18 10 10 11 11 Th 16 72 69 68 59 65 56 62 52 Table 3 (cont'd.). 755 Is This A Cue? Tes Io Retained Tor Round Three (1) 12. 13. 11. 15. 15. 17. 18. 19. 20. 21. ZZe 23. 24. A supervisor of training participants does not provide necessary inforsation for the needs analysis. (43) A supervisor of training participants refuses to cosprosise vith other supervisors on training goals. (19) A supervisor of training participants does not attend pre-training planning sessions. (46) A supervisor of training participants refuses to alter the vorh schedule so that subordinates can isplesent nevly-trained chills. (61) A training participant does not coaplete hoaevorh assignaents deeeed necessary for training success. A training participant repeatedly fails to return froa class breaks on tiae. (83) A supervisor of training participants does not distribute training prograa announceeents. (48) A training participant does not get actively involved in the training, choosing instead to reaain on the periphery of class activities. (79) An upper level aanager does not allocate soney for a training prograa. (3) An upper level aanager does not set training goals according to long-range organisational goals. (7) An upper level aanager refuses to cosprosise cith other aanagers on training goals. (11) An upper level aanager does not 'ait-in' on portions of the training. as planned. (32) A supervisor of training participants repeatedly pulls participants out of the training prograa for short periods of tile. (58) 15 13 16 13 ll 18 11 18 ID 19 55 55 55 52 an t: 52 38 38 38 38 38 34 3A Table 3 (cont'd.). Is This A Cue? Tea Io Retained For Round Three (3) 25. 25. 27. 28. 29. 30. 31. 32. 33. 34. 35. A supervisor of training participants does not guarantee that subordinates vill attend training. (51) A supervisor of training participants does not participate in training follow-up activities. (65) A supervisor of training participants is slow and haphazard in his/her efforts to register subordinates for training. (44) A training participant frequently leaves class for short periods of tise. (81) A training participant does not respond to training follov-up activities. (104) An upper level aanager expects training staff to cover an inordinate asount of saterial in a saall asount of training tise. (5) An upper level aanager does not contribute to pre-training planning sessions. (9) An upper level aanager does not allov training staff sufficient preparation tise for an upcosing prograa. (4) An upper level sanager requests a particular type of training, then does not tahe part in the subsequent planning of that prograa. (8) An upper level aanager sahes last ainute changes in the training site or instructional equipsent. (10) A supervisor of training participants does not adequately prepare subordinates for a training prograa, so participants cone to training not hnosing vhat to expect. (47) 10 19 34 34 31 31 31 31 31 24 Z4 Z4 24 77 Through the first and second Delphi rounds, the number of cues was narrowed from one hundred and eight to thirty-five, and finally to seventeen. Table 4 summarizes the cues retained for each round, according to actor and stage of training. Table 4. Russary of Cues Retained for Rach Round. Nusber of Nusher of Nusher of Cues Pros Cues Retained Cues Retained Category Round 1 Por Round 2 Por Round 3 UPPRR LRVRL NANACRNRNT Pre-Training 31 ll 3 Training 10 2 l Post-Training 2 0 0 SUPRRVISORS OP TRAINING PARTICIPANTS Pre-Training 14 h 4 Training 5 2 l Post-Training ll 3 2 TRAINING PARTICIPANTS PreoTrsining 3 0 0 Training 26 7 3 Post-Training 5 2 l TOTALS 108 35 17 78 Suggestions of Latent Issues: This portion of the Round Two questionnaire responded to the following research question: 1. Which cues are perceived by training professionals to be associated with which latent issues? Each time a participant verified a cue, he/she was asked to suggest which latent issue was associated with that particular cue. Rather than use the term "latent issue", which would require a lengthy explanation, the participants were simply asked to speculate the "cause" behind each one by completing this sentence: "The person may sense a threat to his/her . All of these suggestions of latent issues, as reported below, were presented for verification in Round Three of the study. Very little editing was needed at this stage; only when clearly synonymous terms were used did the investigator combine two or more suggestions into one. For example, all suggestions that referred to "power" or "influence" were grouped into one category called "authority, control". On a few occasions, the investigator telephoned study participants to clarify terms. The "Frequency" column in Table 5 shows the number of times each latent issue was suggested. 759 Table 5. Suggestions of Latent Issues Por Rach Cue. (The nuaber in parenthesis folloving each cue is the original nuaber assigned in Table 2.) The person say sense a Cue threat to his/her ... Frequency 1. A supervisor of training a. authority, control. ............................................ 4 participants authorises b. shill level. (Lachs confidence in his ability to learn.) ...... 8 training for subordinates, c. fasiliar routine. (Not villing to attend training.) .... ....... 2 but not for his/herself. d. fasiliar routine. (Use of nev shills requires change .......... 5 (50) of on-the-iob routines.) e. reputation as a aartyr. (Nilling to 'give' training to ........ 3 others. but not to self.) 2. A training participant a. intelligence. (Relieves he does not need training) ............ 5 shovs disinterest during b. pride. (Pears training will reveal veahnesses.) . .............. 8 training by reading, c. other vorh responsibilities. ..... . ............................. l initiating side conver- d. personal autonosy. ............................................. 5 sations, not tahing notes. e. coafortable vorh routine. ...................................... 2 et cetera. (78) f. status. (Believes adults don’t have to be as attentive ........ 1 as school children.) 3. An upper level aanager a. credibility. (Fears being asscCiated with training if ......... 5 cossits to training in probleas arise.) private, but does not b. popularity. .................................................... 9 desonstrate that c. isage as a 'hardline' aanager. ................................. 2 cosaitsent to peers and d. Job security. (Reluctant to voice an opinion.) ................ 5 subordinates. (33) 4. A supervisor of training a. pride. (Relieves old vorh practices are better than nev.) ..... 7 participants publicly b. job security. (Believes subordinates' nev shills aahe ......... 5 criticises a training her dispensable.) prograa. (59) c. superiority over training staff. or training function. ......... 7 d0 popul.rit’0 00.0.0...000....000.000.000.00...OIOOIOIOOOIIICOOOIO 1 5. A training participant a. other vorh priorities. ......................................... z verbalises resentsent at b. coafortable vorh routine. ....................... ....... ........ 5 having to attend training. c. pride. (Relieves he does not need training.) .................. 3 (17) c. pride. (Pears training sill reveal veahnesses.) ............... 5 e. pride. (Relieves her daily vorh is too isportant to aiss.) .... 2 f. personal autonosy. (Resents sandatory attendance.) .... ........ 3 Table 5 (cont'd.). 8() The person say sense a Cue threat to his/her ... Prequency 5. A training participant a. coafortable vorh routine. ...................................... 11 does not apply nevly- b. credibility. (Pears aahing sistahes.) ................. ..... ... trained shills shen hach c. approval of supervisor. (Supervisor discourages use of ......... 2 on the job. (103) c. the nev shills.) d. approval of peers. (Others discourage the use of the nev ...... 4 shills.) T. An upper level aanager a. authority, control. ....... . ................... . ...... . ......... 7 sahes a training decision b. own interests, priorities. ..................................... 6 that runs contrary to the c. pride. (Agreesent vitb needs analysis seans public ..... 2 results of a needs recognition of probleas.) analysis. (I) d. coafortable vorh routine. ...... . ............................... 2 e. reputation as a aanager vbo succeeds on instinct. not ......... 1 systesatic analysis. f. popularity. ................................. .... ............... 1 8. An upper level aanager a. authority, control. . .......................... . ................ 11 does not include training b. Job security. (Attespt to aahe self indispensable.) ........... 2 staff in aeetings for c. elitisa. (Relieves top~levei decisions are too ............... 3 nos products or procedures sophisticated for training staff.) when training of d. training expertise. (Nan: to aahe decisions without ........... 3 personnel will be consulting training staff.) necessary. (5) 9. A supervisor of training a. credibility. (Relieves the 'nev‘ shills aahe her ’old ........ 6 participants does not shills obsolete.) reinforce the use of new b. authority. (Peels subordinates’ new shills give thes .......... 5 shills vhen participants pover over his.) return froa training. (83) c. coafortable routine. (To incorporate nev shills requires ...... 4 change.) d. job security. (Relieves subordinates’ shills say .............. A aahe her dispensable.) 10. A training participant does a. other vorh priorities. ...... ..... ................ ..... . ....... . A not shov-up for a long- b. coafortable vorh routine. ................................ .. . 4 scheduled, such-advertised c. pride. (Relieves she does not need training.) .... ...... ...... 5 training prograa. (80) d. pride. (Pears training will reveal seahnesses.) ......... ...... 3 e. personal autonosy. (Resents sandatory attendance.) ...... ...... 2 11. An upper level aanager a. financial accountability. ...................................... 5 allocates soney for a b. other priority projects. ................... .............. ...... A training prograa. but the c. reputation as a tough, efficient aanager. ...................... 3 asount is grossly d. belief that training ought to be inexpensive. .................. 1 insufficient. (2) e. authority, control. ....... ..... ..... ........................... 5 81. Table 5 (cont'd.). The person any sense a Cue threat to his/her ... Prequency 12. A supervisor of training a. credibility. (Pears her ovn last of shills vill be revealed.) . 5 participants does not b. pover. (Relieves that inforsation shared is power lost.) ...... 3 provide necessary infor- c. coafortable vorh routine. (Relieves training vill bring ......... A nation for a needs unwanted change.) analysis. (45) d. status. (Relieves such sorh should be left to underlings.) .... 4 13. A supervisor of training a. authority. control. ........... ..... ..................... ....... 8 participants refuses to b. credibility. (Disguises lach of sanagenent shill .............. 2 cosprosise on training with belligerence.) goals. (49) c. ovn interests or priorities. ................................... 2 14. A supervisor of training a. shill level. (Lachs confidence in ability to contribute to ... 7 participants does not the neeting.) attend pre-training b. other vorh priorities. ... ..................... . ................ 5 planning sessions. (46) c. status. (Relieves such vorh should be left to underlings.) .... 1 d. pover. (Relieves that inforsation shared is power lost.) ...... 2 15. A supervisor of training a. credibility. (Relieves 'nev' shills aahe his 'old‘ ..... . ...... 1 participants refuses to shills obsolete.) alter the vorh schedule b. authority. (Peels subordinates’ new shills give then .......... 3 so that subordinates can pover over her.) isplenent newly-trained c. coafortable vorh routine. ...................................... 7 shills. (64) d. job security. (Relieves subordinates’ shills aahe ........ 8 his dispensable.) 16. A training participant a. personal autonosy. (Resents that honevorh is considered ....... 4 does not coaplete hosevorh sandatory.) assignnents deened b. pride. (Pears honevorh vill reveal veahnesses.) ............... A necessary for c. other vorh priorities. .... .................. . .................. 3 training success. (82) d. ability to conceal reading or vriting deficiencies. .... ....... . 1 e. coafortable vorh routines. ..................................... l 1?. A training participant a. personal autonosy. (Resents that training is sandatory.) ...... 6 repeatedly fails to b. other job priorities. .......................................... 1 return froa class breahs c. coafortable vorh routine. (Accustoned to leisurely breahs.) .... l on tise. (83) d. pride. (Pears training vill reveal veahnesses.) ......... ..... . 4 Table 5 (cont'd.). E32 Cue The person any sense a threat to his/her ... A notable drop in cue validation percentages occurs between the seventeenth and eighteenth cues. that the following situations should be considered as cues. Also, fever than half of the study participants agreed Por this reason, itens 18-35 were pg; included in Round Three. 18. A supervisor of training 20 21. 22 participants does not distribute training prograa announcenents (48) . A training participant does not get actively involved in the training, choosing instead to rennin on the periphery of class activities. (73) An upper level aanager does not allocate soney for a training prograa. (3) An upper level aanager does not set training goals according to long-range organizational goals. (7) An upper level aanager refuses to cosprosise with other aanagers on training goals. (11) (.300. . job security. ........... ..... . control over inforsation offered to eaployees. ................. . power to decide who should attend. ..... . ....................... . production and staffing desands. ............................... . reputation. .................................................... . superiority over training. ..................................... . fasiliar worh routine. ......................................... . authority, power. .... . job hnowledge and experience. .................... personal autonosy. ............................................. . fasiliar vorh routine. ......................................... . security as one face in a crowd. ...................... . ........ ability to stay within budget. ................................. perfornance appraisal. ... ...................................... job security. .................................................. other worh priorities. ............. . ........................... professional reputation. ....................................... decision sahing ability. ..... ........ ....... ...... . ............ right to voice an opinion. ........ ............. ..... ....... .... .....OIOOOOOIO long range accountability. ..... ..... ....... ....... ........ ..... short ters goals. .................. OOOOOOOICOIOOOOOO00.0.0.0... 0.ut'°rit’, ,O'EI' i'fluence‘ 00000000000OOIOOIOOOOOOIOOOOOOOODOO coafortable vorh routine. .... Prequency Table 5 (cont'd.). 823 The person any sense a Cue threat to his/her ... Prequency 23. An upper level aanager does a. status, dignity. ............................................... 5 not 'sit-in' on portions of h. other vorh priorities. ......................................... 3 the training, as planned. c. reputation as an expert. fearing that the instructor or ..... ... 2 (32) trainees nay ash a question that he/she can't answer. 24. A supervisor of training a. authority, power. .............................................. 7 participants repeatedly b. other vorh priorities. . ..... ................................... 1 pulls participants out o. ability to function as a worher. rather than a supervisor. .... 1 of the training prograa d. ego. ........................... ........ ........ ...... .......... 1 for short periods of tine. (58) 25. A supervisor of training a. power, authority, control. ............... .. .................... 5 participants does not b. other worh priorities. ......................... ...... .......... 2 guarantee that subordinates c. personal autonosy. ............................................. 2 will attend training. (51) 26. A supervisor of training a. Job shills, follow-up say highlight shortcosings. .............. 2 participants does not b. authority. control. ............................................ 5 participate in training c. other worh priorities. ......................................... 1 follow-up activities. d. coafortable routine. ........................................... 1 (651 27. A supervisor of training a. job security. .................................................. 2 participants is slow and b. other worh priorities. ... ................................. ..... 1 haphazard in his/her c. loyalty of co-worhers. ... ...... .. .............................. 1 efforts to register d. superiority over training. ..................................... 1 subordinates for e. control over subordinates’ tise. ............................... 2 training. (44) f. sanagenent style. ........ . ..................................... 1 28. A training participant a. confort level. .......................... ..... ... ...... ......... 2 frequently leaves class for 5. self esteen, fearing that training will reveal weahnesses. ..... 1 short periods of tise. c. other worh priorities. ............ ...... ....................... 3 (81) d. control. status. ............................................... 1 23. A training participant does a. control. personal autonosy. ....................... ............ 2 not respond to training 8. other vorh priorities. ..................... .... ........ . .. .. l follow-up activities. (104) c. reputation. fearing that follow-up say reveal his/her .......... 3 failure to learn the new shills. Table 5 (cont'd.). 8&1 The person any sense a Cue threat to hislher Prequency 30. An upper level aanager a. authority, control. ............................................ l expects training staff to b. other worh priorities. . ...... .................................. 4 cover an inordinate asount c. ability to solve aajor probleas with little investnent. ........ 1 or saterisl in a snail d. reputation; did not agree with training need in ............... 1 asount of training tise. the first place. (51 e. expertise; lack of tise to ash questions avoids criticisn ...... l of trainer. f. toughness; ability to ride eaployees roughshod through ......... 1 their tasks. 31. An upper level aanager does a. credibility, hnowledge of training content. .................... 3 not contribute to 8. other vorh priorities. ......................................... 1 pre-training planning c. right to voice an opinion. ..................................... 2 sessions. (9) d. accountability for possible training failure. .................. 1 e. status as a 'thinher'. not a 'doer'. ........................... 1 32. An upper level aanager does a. ability to produce quich results. .............................. 2 not allow training staff b. other worh priorities. ......................................... 2 sufficient preparation tise c. isage as a ‘tough‘ aanager. .................................... 1 for an upcosing prograa. (4) 33. An upper level aanager a. other worh priorities. ......................................... 1 requests a particular type b. power. authority. .............................................. 2 of training, then does c. status as 'thinher‘. not a 'doer'. ............................. 1 not take part in the d. belief that such worh is for subordinates. . ................... 1 subsequent planning of e. job shill, fearing failure. .................................... 2 that prograa (8) 34. An upper level aanager sahes a. authority. ..................................................... 5 last ainute changes in the b. level of service. .............................................. 1 training site or instructional equipaent. (101 35. A supervisor of training a. job security, continued need for his/her supervisor role. ...... 3 participants does not b. inch of confidence in own abilities. ............... ............ 2 ' adequately prepare c. control. .............. ......... . .............. . ................ 1 subordinates for a training prograa. so participants cone to training not hnowing what to expect. (47) 85 Results of Round Three Round Three of this Delphi study (Appendix E) served two purposes: (1) the final verification of the latent issues that were suggested for each cue in Round Two, and (2) an assessment of the degree of urgency that each cue presented, when associated with a particular latent issue. This effort was in response to the following research questions: 1. Which cues are perceived by training professionals to be associated with which latent issues? 2. Which cues can be traced by training professionals to the most urgent threats to training program effectiveness? To reach these goals, the third Delphi questionnaire included the final seventeen cues and the participants' suggestions of latent issues, both of which were gathered in Round Two. Participants were asked to place a number "1" next to the threat that was most likely associated with each cue. The sum of those votes became the "raw score" for each latent issue, which allows the reader to compare the likelihood of one latent issue to the other suggested issues. These raw scores are recorded in Table 6, Column A. The participants responded to a Likert-type scale to assess the degree of urgency for each cue, when 86 associated with each latent issue. The scale included the following range: 4 - A Nearly Unsurmountable, 3 = Very Serious, 2 a Difficult, l - A Minor Obstacle. The mean, (Column B), and standard deviation, (Column C), for each latent issue that resulted from this exercise are also recorded in Table 6. Finally, a "Likelihood and Urgency Index" was determined by multiplying the raw score with the mean score. This index, reported in Column D of Table 6, shows which of the latent issues suggested by the study subjects may warrant special attention by training professionals. [Note that the latent issues that carried the highest raw scores (Column A) also produced the highest Likelihood and Urgency Index (Column D). This is probably due to the tendency of the Urgency mean scores (Column 8) to group around 2.5, the mid-point on the 1-4 point scale offered to the study subjects. In most cases, the results of the Urgency determination exercise were indecisive.] 87 Table 6. Likelihood and Urgency Index. Likelihood Urgency of Lihelihood of Latent (hie and Urgency Issues (Scale of 1-4) Cosbined Raw Score 1 30 Index the and Possible latent Issues A R C D l. A supervisor of training participants authorises training for subordinates, but not for his/herself. (50) a. authority, control. ......................................... 7 1.57 .49 10.99 b. shill level. (Lachs confidence in his ability to learn.) 10 2.7 .90 27.0 c. fasiliar routine. (Not willing to attend training.) ........ 4 2.0 .71 8.0 d. fasiliar routine. (Use of new shills requires change ....... 2 1.5 .50 3.0 of on-the-job routines.) e. reputation as a aartyr. (Uilling to ‘give‘ training ........ l 2.0 0 2.0 to others, but not to self.) 2. A training participant shows disinterest during training by reading, initiating side conversations, not tahing notes, et cetera. (78) a. intelligence. (Relieves he does not need training.) ........ 21 2.57 .66 54.0 b. pride. (Pears training will reveal weahnesses.) ............ 4 3.25 .43 13.0 c. other worh responsibilities. ................................ 0 0 0 0 d. personal autonosy. ...... . ...... . ............................ l 2.0 0 2.0 e. coafortable worh routine. . ................................ 0 0 0 0 f . status. (Relieves adults don’ t have to be as attentive ..... 2 2.5 .50 8.0 as school children.) 3. An upper level manger units to training in private, but does not desonstrate that on-itsent publicly to peers and subordinates. (33) a. credibility. (Pears being associated with training ......... 16 2.5 .73 40.0 if probleas arise.) b! pwluit’. OOIOOOQOOOOIOIOOOOOOOOOIOIIOOOQOOOOOOOOOOO 000000 O z 2.0 o ‘00 cs meu‘.Mlin mere ssssassssssssssssssssassssassss o o o D d. job secirity. (Reluctant to voice an opinion.) . ..... 10 2.9 .70 29.0 823 Table 6 (cont’d.). Lihelihood Urgency of Lihelihood of Latent Cue and Urgency Issues (Scale of 1-4) Coabined Raw Score 2 80 Index Cue and Possible Latent Issues A R C 0 4. A supervisor of training participants publicly criticises a training prograa. (59) a. pride. (Relieves old vorh practices are better than new.) .. 6 2.8 1.07 16.8 6. job security. (Relieves subordinates’ new shills aahe ...... 4 2.5 .50 10.0 her dispensable.) c. superiority over training staff, or training function. ...... 18 2.7 .80 48.6 d. popularity. ................................................. 0 0 0 0 5. A training participant verbalises resentsent at having to attend training. (77) a. other work priorities. ...................................... 2 2.0 0 4.0 b. coafortable worh routine. ................................... 2 2.5 .50 5.0 c. pride. (Relieves he does not need training ) ............... 13 2.15 .86 28.0 d. pride. (Pears training will reveal weahnesses.) ............ 4 3.0 0 12.0 e. pride. (Relieves her daily vorh is too isportant to siss.) . 5 2.2 .40 11.0 f. personal autonosy. (Resents sandatory attendance.) ........ 4 1.75 .83 7.0 sandatory attendance.) 6. A training participant does not apply newly-trained shills when back on the job. (103) a. coafortable worh routine. .............. . .................... 10 2.3 .70 23.0 b. credibility. (Pears sahing nistahes.) .............. ........ 3 2.7 .47 8.1 c. approval of supervisor. (Supervisor discourages use of ...... 15 3.4 .61 51.0 the new shills.) d. approval of peers. (Others discourage the use of the ....... z 2.5 .50 5.0 new shills.) 539 Table 6 (cont'd.). Lihelihood Urgency of Lihelihood of Latent Cue and Urgency Issues (Scale of 1-4) Conbined Raw Score 3 SD Index Cue and Possible Latent Issues R C 0 7. An upper level aanager ashes a training decision that runs contrary to the results of a needs analysis. (I) a. authority, control. ......................................... 2 3.5 .50 7.0 b. own interests, priorities. .. ................................ 18 2.9 .31 52.2 c. pride. (Agreesent with needs analysis neans public ......... 5 3.8 .40 13.0 recognition of probleas.) d. coafortable worh routine. ................................... 0 0 0 0 e. reputation as a aanager who succeeds on instinct, not ....... 5 2.6 .43 13.0 systesatic analysis. f. popularity. ................................................. 0 0 0 0 8. An upper level aanager does not include training staff in aeetings for new products or procedures when training of personnel will be necessary. (6) a. authority, control. ......................................... 3 2.66 .47 8.0 b. job security. (Attenpt to aahe self indispensable.) ........ 0 0 0 0 c. elitisa. (Top-level decisions are too sophisticated for .... 18 2.3 .47 41.4 training staff.) d. training expertise. (Nants to aahe decisions without ........ 8 3.1 .60 24.8 consulting training staff.) 9. A supervisor of training participants does not reinforce the use of new shills when participants return froa training. (63) a. credibility. (Relieves the 'new' shills aahe her 'old ..... 7 3.0 .58 21.0 shills obsolete.) b. authority. (Peels subordinates’ new shills give then ....... 6 2.3 .64 17.4 power over his.) c. coafortable routine. (To incorporate new shills requires ... 14 3.3 .88 46.2 change.) d. job security. (Relieves subordinates' shills say aahe her .. 0 0 0 0 dispensable.) 9() Table 6 (cont’d.). Lihelihood Urgency of Lihelihood of Latent Cue and Urgency Issues (Scale of 1-4) Conbined Raw Score I SD Index Cue and Possible Latent Issues A R C D 10. A training participant does not show-up for a long-scheduled, such-advertised training prograa. (80) a. other worh priorities. ...... ........ ... .................... . 17 2.6 .77 44.2 b. coafortable worh routine. . ...... . ........................... 0 0 0 0 c. pride. (Relieves she does not need training.) . ............ . 8 2.1 .33 16.8 d. pride. (Pears training will reveal weahnesses.) ............ 2 2.5 .50 5.0 e. personal autonosy. (Resents sandatory attendance.) ......... 2 2 0 1.00 4.0 11. An upper level aanager allocates soney for a training prograa, but the asount is grossly insufficient. (2) a. financial accountability. ................................... 4 2.8 .43 11.2 b. other priority projects. .... ................................ 20 3.0 .74 60.0 c. reputation as a tough, efficient aanager. ................... 4 3.25 .43 13.0 d. belief that training ought to be inexpensive. ............... 2 2.5 .50 5.0 e. authority, control. ...... ..... .............................. 0 0 0 0 12. A supervisor of training participants does not provide necessary inforsation for the needs analysis. (45) a. credibility. (Pears her own inch of shills will be ........ 13 3.2 .89 41.6 revealed.) 6. power. (Relieves that inforsation shared is power lost.) ... 3 c. coafortable worh routine. (Relieves training will bring ... 9 unwanted change.) d. status. (Relieves such vorh should be left to underlings.) . 0 0 0 0 Nu one: a I. C: 13. A supervisor of training participants refuses to cosprosise with other supervisors on training goals. (49) a. authority, control. ............................... ....... ... 7 b. credibility. (Disguises lack of sanagenent shills .......... 3 belligerence.) c. own interests or priorities. ............ ...... . ....... ...... 20 2.15 .73 43.0 N“ as... (:3 9]. Table 6. Lihelihood and Urgency Index. Lihelihood Urgency of Lihelihood of Latent Cue and Urgency Issues (Scale of 1-4) Conbined Raw Score 2 80 Index Cue and Possible Latent Issues A R C 0 14. A supervisor of training participants does not attend pre-training planning sessions. (46) a. skill level. (Lachs confidence in ability to contribute .... 4 2.5 .50 10.0 to the neeting.) b. other work priorities. ...................................... 13 2.1 .64 33.3 c. status. (Relieves such worh should be left to underlings.) . 6 2.6 .75 15.6 d. power. (Relieves that inforsation shared in power lost.) ... 0 0 0 15. A supervisor of training participants refuses to alter the work schedule so that subordinates can isplenent newly-trained shills. (64) a. credibility. (Relieves 'new‘ shills aahe his 'old“ ........ 1 3.0 0 3.0 shills obsolete.) b. authority. (Peels subordinates' new shills give then ...... 11 3.3 .62 36.3 power over her.) c. coafortable worh routine. . .................................. 15 3.2 .31 48.3 d. job security. (Relieves subordinates’ stills any aahe ...... 0 ' his dispensable.) 16. A training participant does not coaplete hoseworh assignsents deened necessary for training success. (82) a. personal autonosy. (Resents that hoseworh is considered .... 15 1.3 .68 28.5 sandatory.) b. pride. (Pears hoseworh will reveal weahnesses.) .... ........ 4 1.8 .43 7.2 c. ot.er 'or' prioritie's OI. 000000000 COO. OOOOOOO 000.000.000.000 3 203 0‘? 6.9 d. ability to conceal reading or writing deficiencies. ......... 6 2.5 1.12 15.0 e. coafortable worh routines. .... ...... ................ ........ 2 2.5 .50 5.0 17. A training participant repeatedly fails to return froa class breahs on tine. (83) a. personal autonosy. (Resents that training is sandatory.) ... 8 2.13 .60 17.0 no other 10b prioritie.’ .0000OIOOOOOOOOIOOOIOOOOOOIOOI00000000.12 2017 037 ZEAC c. coafortable worh routine. (Accustoned to leisurely breahs.) . 4 1.25 .43 5.0 d. pride. (Pears training will reveal weahnesses.) ............ 1 3.0 0 3.0 92 §EEEE£Y A Delphi technique of data collection was employed in this study as an effective vehicle for brainstorming and group deliberation. The first round resulted in one hundred and eight different suggestions of cues, seventeen of which were ultimately designated by the study participants as the most valid indicators of latent issues. In Round Two, the research subjects were introduced to the concept of latent issues, and were encouraged to brainstorm possible issues for each cue. Round Three served to finalize the link between cues and issues, and provided the study participants with an opportunity to comment on the degree of urgency for each cue. Careful scrutiny of the data displayed in Tables 1- 6 reveals that the training professionals in this study observed most cues from those individuals with formal organizational power earlier in the training process. As the training effort moves from the planning to the implementation stages, perhaps the subjects tended to turn their focus to individuals with less political influence. In most cases, an impressively large percentage of the study participants agreed on a link between each cue and a particular latent issue. The effort to determine a degree of urgency for each one was less successful; 93 mean scores tended to hover near the median point, with proportionately large standard deviations, considering the four-point scale. Consequently, the latent issue designated as "most likely" by these training professionals was, without exception, also rated highest on the "Likelihood and Urgency Index". CHAPTER V CONCLUSIONS, OBSERVATIONS, AND RECOMMENDATIONS Introduction The purpose of this study was twofold: l) to identify behaviors and events, (i.e., cues) that commonly result when people feel threatened by a training program, and 2) to link those behaviors and events to the most likely perceived threats, (i.e., "latent issues"). For example, when a supervisor authorizes training for his subordinates but not for himself, this may be a cue that the supervisor feels threatened by the training program. Specifically, the supervisor may quietly fear that others will view him as less powerful or authoritative if they see that he "needs" training. When this fear is not clearly expressed by the supervisor to the training professional, it is considered a latent issue. In this chapter, each of the research questions are addressed, and then further observations and recommendations are offered, based upon the investigator's integrated review of the data. 94 95 Question #1: What behavioral and environmental cues are perceived by training professionals as indicators of debilitating latent issues? Seventeen cues were identified by the study participants as probable indicators of latent issues, (Table 6, Chapter 4). All of these cues are behaviors, rather than situations or objects found in the work environment. Such uniformity probably resulted because training professionals are most likely to focus on other people when looking for issues that could interfere with training. People, after all, are the source of latent issues. While environmental cues are surely available, (interoffice memos, company policies, training facilities and supplies), trainers probably are conditioned to look for behavioral indicators, which are no doubt more abundant and easier to trace to the owner of the issue. The seventeen cues identified in this study are also very specific behaviors. This is because the subjects were instructed to give examples of cues actually observed in their work settings. Without much difficulty, the list of probable cues can be expanded by generalizing to similar behaviors in similar situations. 96 Question #2: Which cues are perceived by training professionals as the most valid indicators of debilitating latent issues? The results of a cue validation exercise completed by the study participants is reported in Table 3 of Chapter 4. Generally, the percentage of affirmative responses for each one descends gradually from the most to the least valid. This suggests that the exact hierarchical order of cue validity is probably arbitrary. For the purpose of this study, only the top seventeen cues were carried into Round Three. [The number "seventeen" was a logical cutting-off point, as a significant drop in one validation percentages (from 52% to 38%) occurs between the seventeenth and eighteenth cues.] This screening-out process reduced the list of cues to a manageable number; it does not suggest that the latter eighteen cues were deemed "non-valid". Question #3: Which cues are perceived by training professionals to be associated with which latent issues? The association of cues to latent issues is numerically expressed in Table 6 of Chapter 4. The data from this study, (along with common sense), tells us that a perfectly reliable association of a particular latent issue to a particular one is unrealistic. Still, 97 in most cases, one or two latent issues stand out as a more likely explanation than the other suggested issues. Question #4: Which cues can be traced by training professionals to the most urgent threats to training effectiveness? It appears that the degree of urgency for a particular cue is dependent upon the factors surrounding the event, (Table 6, Chapter 4). The results of this study suggest that cue urgency cannot be reliably pre- determined. Question #5 and Question #6: Which ones are associated with which stage of training program implementation? Which cues tend to originate from each of the following three groups of individuals: upper level managers, supervisors of training participants, or training participants? Table 4 of Chapter 4 summarizes the number of suggested cues that originated from each training stage and employee group. It is significant that, throughout all three rounds of the study, the largest percentage of cues originated from managers and supervisors before training begins, and from training participants after training is under way. This probably indicates that training professionals tend to focus their attention on peOple who possess formal authority early in program 98 planning, and then turn their attention to the less influential employees once training begins. If so, such a habit of trainer "tunnel vision" could ultimately jeopardize the success of the training program. This observation is discussed in more detail later in this chapter. Observation 1: Further research must be conducted before the reliability of these cues can be firmly established. Consider this scenario: a training coordinator schedules a meeting for all line supervisors to discuss the content of an upcoming training program for new hires. The meeting is scheduled for a very convenient time, and it is well advertised. Still, only a handful of supervisors show-up, and most of them are strangely quiet throughout the meeting. Is this an unmistakable cue to underlying issues? Are the supervisors threatened by some aspect of the training program? There are a number of possible explanations for the supervisors' behavior, ranging from the very simple, (perhaps there is a bad case of flu going around), to the very complex, (perhaps the supervisors believe the new training program will make some of their skills obsolete, which could threaten their authority or job security.) The results of this 99 study suggest that it would be foolish to believe that such behavior from supervisors always stems from a single debilitating latent issue. Still, it appears that some behaviors and events are more likely than others to signal serious roadblocks for training professionals. The cues listed in Table 7, (page 100), were verified by the participants of this study. (This means that there was a notable drop in cue validation percentages between those cues retained and those rejected. Also, at least half of the participants believe these behaviors and events are related to latent issues.) These seventeen very specific examples of cues can be generalized, with care, to other similar situations. Additional cues may also be discovered by noting the patterns in these original seventeen cues, and then generalizing to new situations. Note the following common threads: a. In many of the cues listed in Table 1, the actor fails to invest the time necessary for effective training participation, or invests the time, but resents doing so. (See cues 2,5,10,13,14,15,16.) b. Often, the cues involve an individual's action, or lack of action, which handicaps the training program logistically. (See cues 6,7,8,9,11,12,13,14,15,17.) c. Many of the cues involve an outward sign to an individual's co-workers that he/she does not value the training program. (See cues 1,3,4,8,9.) Table 7. Relationship Retween Cues and latent Issues. 100 use The person my sense a threat to his/her l. A supervisor of training participants authorises training for subordinates, lwit not for his/herself. 2. A training participant shows disinterest during training by reading, initiating side conversations, not tahing notes, et cetera. 3. An upper level mnager comits to training in private, but does not desonstrate that oomitsent to peers and subordinates. 4. A supervisor of training participants publicly criticises a training prograa. 5. A training participant verbalises resentsent at having to attend training. 6. A training participant does not apply newly-acquired shills men bach on the job. 7. An upper level mnsger mhes a training decision that runs contrary to the results of a needs analysis. 6. An mes level manger does not include training staff in aeetings for new products or procedures (den training of personnel will be necessary. shill level. (lacks confidence in his ability to learn.) authority, control. intelligence. (Relieves he does not need training.) pride. (Pears training will reveal weahnesses.) credibility. (Pears being associated with training if probleas arise.) job security. (Reluctant to voice an opinion.) superiority over training staff or training function. pride. (Relieves old vorh practices are better than new.) pride. (Relieves he does not need training.) . pride. (Relieves her daily work is too isportant to sisa.) . approval of supervisor. (Supervisor discourages use of the new shills.) . coafortable vorh routine. . owl interests, priorities. elitisa. (Top-level decisions are too sophisticated for training staff.) training expertise. (Pants to mhe decisions without consulting the training staff.) Table 7 (cont'd.). 101 Thepersonmysensea 0|! threat to his/her 9. A supervisor of training participants a. coafortable routine. (To incorporate new shills does not reinforce the use of new requires change.) mills men mrticipants return froa training. 6. credibility. (Relieves the 'new' shills mhe her 'old' shills obsolete.) 10. A training participant does not show-up a. other worh priorities. for a long-scheduled, such-advertised training prograa. b. pride. (Relieves she does not need training.) 11. An upper level mnager allocates soney a. other priority projects. for a training prograa, but the asount is grossly insufficient. 12. A supervisor of training participants a. credibility. (Pears her own inch of shills will he does not provide necessary revealed.) informtion for a needs analysis. b. coafortable wort routine. (Relieves training will bring unwanted change.) 13. A supervisor of training participants a. on interests or priorities. refuses to cosprosise on training goals. b. authority. control. 14. A supervisor of training participants does a. other worh priorities. not attend pre-training planning sessions. b. status. (Relieves such wort should be left to underlings.) 15. A supervisor of training prticipants a. coafortable worh routine. refuses to alter the vorh schedule so that subordinates ma isplenent newly- b. authority. (Relieves subordinates' new shills acquired mills. give thm power over her.) 16. A training prticipmt does not coaplete a. personal autonosy. (Resents that heart is hoseworh assigments domed necessary considered mndatory.) for training access. b. ability to conceal reading or writing deficiencies. 17. A training participant repeatedly fails to a. other job priorities. return froa class breahs on tine. b. personal autonosy. (Resents that training is mndatory. ) 102 Not surprisingly, most of the cues identified here could be construed as passive-aggressive behavior. When employees feel (conscientiously or unconscientiously) threatened by a company-endorsed training program, they will likely choose a politically ”safe" method of resistance. Over time, such passive-aggressive resistance can "duck-bite" a training effort to death. Again, the list of cues in Table 7 is not meant to be all-inclusive. Nor are these seventeen cues guaranteed to always result from debilitating issues. The results of this study do suggest that training professionals would do well to heed the probable warnings inherent in these and other closely-related behaviors. Observation 2: Further research must be conducted before a particular one can be designated as a predictor of a particular latent issue(s). Latent issues result from an upper level manager's, supervisor's, or training participant's idiosyncratic perception of a situation. That perception is colored by countless factors, including the perceiver's personality and biases, and the organization's current political climate and culture. All of these confounding variables make it difficult to flawlessly link each cue to its underlying cause. Two factors handicapped this 103 effort to reliably correlate cues to latent issues. First, the research subjects varied greatly in their degree of interpersonal sophistication. Some were clearly more perceptive and empathetic than others. Also, communication among study participants throughout the three Delphi rounds was hampered by the relative ambiguity of terms. For instance, one subject's definition of "pride" sometimes differed from another's. (The investigator made several follow-up telephone calls to clarify definitions.) Still, the research participants succeeded in identifying one or two probable latent issues for each cue, (see Table 7). Final analysis of the latent issues reveals common themes, (identified by the investigator), in the concerns of upper level managers, supervisors, and participants: a. When upper level managers perceive a training program as threatening, it is often a concern for loss of power or status. b. Similarly, supervisors sometimes perceive the training of subordinates as a threat to their own authority. They also show concern for the loss of a familiar, comfortable routine. 0. When training participants see a training program as threatening, it is often a challenge to their personal autonomy or pride. It is not surprising that upper level managers are concerned about a possible loss of power or status; they clearly have the most to lose in that arena. Even 104 though top administrators are rarely involved directly with training implementation, they must make very visible financial and political commitments before a new training program is ventured. Consequently, a training success is often viewed as a top manager's success and a verification of power. Alternately, a training failure can potentially detract from that same manager's future influence within the organization. Supervisors of training participants, like their superiors, may feel their claim to authority is vulnerable. While upper level managers are probably most threatened by the impact of a training program on their political reputation, supervisors seem more concerned that their authority will diminish proportionately to their subordinates' increased skill level. Supervisors tend to feel a more immediate day- to-day threat from training than do upper level managers; they usually have less real power over the purpose and process of the training. That may explain why so many of the cues offered by supervisors are outward acts of resistance or rebellion, whereas cues from upper managers tend to be more subtle; supervisors may feel that they - and their reputation as authority figures - have been openly challenged. Most latent issues harbored by training participants relate to a perceived threat to personal 105 autonomy and pride. This may suggest that the participants sometimes feel coerced into attending a program that, at least in their eyes, is unnecessary, unfair, and/or unflattering. Importantly, it is apparent that most of the latent issues identified in this study result from the actor's lack of self confidence, or a perceived challenge to his/her professional competence. As frequently happens in many social interactions, behavior that appears to be motivated by resentment or apathy is in reality fueled by the actor's self doubts. If such personal latent issues occasionally influence the quality of employee training programs, should training professionals become involved in the resolution of those concerns? At what point does a professional educator's sincere attempts to pave the way for successful training cross over into unwarranted psychological meddling? Above all else, the results of this research strongly suggest to the investigator that training professionals need to be aware of other's uncertainties, fears, and needs. Trainers are often in a surprisingly powerful position in that they bring new ideas and routines into the organization. Care must be taken to properly prepare others emotionally for such changes. Lasting change will come about most efficiently only 106 when all involved can approach the task as active participants in the effort, rather than reactive defendants of their own best interests. Observation 3: Employee apathy may threaten training effectiveness. The participants of this study conveyed a disheartening, but unmistakable message as they attempted to identify the latent issues behind each cue. It seems that many behaviors that first appear as cues to latent issues may instead indicate that the actor simply does not value the training effort. For instance, if a supervisor of training participants does not provide the necessary information for a training needs analysis, his failure to contribute could be due to a latent issue: a fear that the written analysis will reveal his lack of technical skill. Another possible explanation, however, may be that the supervisor does not care enough about the training program to make the effort. General apathy, it seems, can be as big a threat to training effectiveness as some long-cultivated latent issues. (A NOTE: If a person simply does not believe that a training program is worthwhile, then that, by definition, is not a latent issue. It is, however, a serious problem for the training professional.) 107 In many organizations with which the investigator is familiar, training is historically regarded as a necessary evil; an expensive, time-consuming investment that may or may not pay-off in the long run. While this reputation as a cost center of questionable worth is quickly fading, many managers, supervisors, and trainees still view any training program with a wary eye. Trainers might successfully ward off problems associated with employee apathy by actively informing others of the goals and benefits of training. Exciting trainees about an upcoming training program, and convincing managers of a respectable return-on- investment could be especially difficult if the organization has suffered a history of mediocre training attempts. If that is the case, others in the organization may have good reason to feel apathetic, and the trainers face a formidable task. Observation 4: Employee ignorance of the training process may threaten training effectiveness. A second observation made by the investigator was gleaned from the study participants' indirect written remarks and unsolicited telephone conversations. Apparently the study subjects believe that managers, supervisors, and training participants often do not understand the purpose behind common training practices. 108 Not surprisingly, this chronic misunderstanding may be misconstrued by training professionals as resistance resulting from a latent issue. For example, if a supervisor fails to attend a pre- training planning session, the reason may have nothing to do with a latent issue. Instead, the supervisor simply may not understand the purpose of the meeting - may not understand that her input is very important. Note that the first observation, stated above, may be related to the second. If an individual does not understand the purpose behind a particular task involved in training program implementation, then he/she is not likely to value that task. The obvious remedy for employee ignorance is to freely offer explanations of the purpose behind each program goal and procedure. One of the rudimentary principles of education, after all, is to offer motivation to learners by explaining the purpose and benefits of each exercise. (A similar concept concerning employee "empowerment" has been addressed recently in the management literature. See Argyris, Lewin, or Harbison & Myers.) Observation 5: Training professionals' efforts to recognize cues may be hampered by "tunnel vision." 109 The results of this study led the investigator to believe that training professionals tend to focus their attention exclusively on upper level managers and supervisors during the pre-training stage, and then on participants during the training and post-training stage. Most likely, this pattern indicates a tendency to look for and recognize cues from only those individuals whose support is of immediate importance. The obvious drawback of focusing exclusively on certain groups during different stages of training is that support, (i.e., a working relationship that is free of latent issues,) from all groups is needed at all stages of training: a. Training professionals need active support from upper level managers before, during, and after training. This way, others in the organization will see that training is valued by top administrators, and will in turn work to make the training program successful. b. Supervisor support is critical during pre- training preparations, but it is just as important during the training stage, when participants need uninterrupted time to concentrate on learning. Also, supervisors must be actively involved during the post- training stage, monitoring their subordinates' 110 new skills and providing constructive feedback. c. Training participant involvement and support is also critical during all three stages of training. If representatives of the trainees help to establish goals for training and participate in decisions concerning training content and methodology, they are more likely to feel ownership of the training program. During training, participants will obviously learn best if they support the training goals and believe the program is worthwhile. Finally, training participant support is equally critical after the formal training program has ended; the effective transfer of skills back to the work setting ultimately depends on the efforts of the trainees themselves. The results of this study suggest that training professionals need to work closely with all groups at all stages of training implementation. With ongoing interaction, the trainers are more likely to recognize cues, work to understand the underlying issues, and make adaptations where possible to alleviate problems. When representatives of each group are involved throughout 111 all three stages, employees are less likely to harbor debilitating latent issues in the first place. Observation 6: Training professionals sometimes misinterpret particular cues for the cause of a problem, rather than a symptom. During Round Two of the study, a few of the research participants commented that they believe a "cue” is not just an indicator of an underlying problem, it is the problem. Such comments from even a small percentage of the sample are viewed by the investigator as significant, because they suggest that many training professionals are tempted to treat symptoms, rather than delve into the causes (latent issues) behind the problems. If a supervisor chastises a subordinate for trying to use a newly-acquired skill, then there is a reason for that behavior. To scold the supervisor's actions will not bring about real change, just momentary compliance, if anything. Cues, as defined in this study, are merely the symptoms of actual training obstacles. Training professionals must look further than cues to recognize, understand, and if possible, resolve the underlying problem. 112 Implications for Training Professionals The results of this study suggest a few very specific measures training professionals can take to increase their likelihood of recognizing cues and latent issues. We can involve representatives from all employee groups at each stage of training implementation. We can actively search out cues. We can educate others in the organization about what training can and cannot do, and when a particular training program is developed, we can freely inform others of its intended goals and benefits. To effectively recognize latent issues harbored by managers, supervisors, and training participants also requires that trainers possess certain interpersonal qualities. We must be skilled listeners, able to understand others' perceptions and frustrations. Edgar Schein writes, "If we want to understand what a person is doing in a given situation, and why, we must understand the person's definition of the situation." (Schein, 1980) This may require knowledge of the organization's cultural, financial, and political history, and a day-to-day awareness of the collective mood of any one group or subgroup. Such empathy requires time and energy, (commodities many trainers must ration cautiously), and it is impossible to be aware of every individual's 113 occupational and personal concerns. Still, most training professionals do have control over how a new training program is initially presented to employees, and who is involved in the early planning stages. In our daily interactions, we can ward off others' apprehensions by consistently presenting the benefits of training from the listener's point of view. Just as important for developing others' confidence in the training function, we can also openly acknowledge when someone's best interests are not served by a particular training program. When an employee views an investment in training as a personal or professional risk, program effectiveness suffers. (This remains true whether or not that view is accurate.) Ultimately, it is the training professional's responsibility, and is in the training professional's own best interest, to recognize these concerns and to work toward a reconciliation. Recommendations for Further Research The primary motive behind selecting this research topic was to begin to construct a diagnostic framework for a notoriously slippery aspect of employee training and development. Because cues and latent issues are by nature tremendously complex, this investigation has only begun to survey the territory. 114 Further research can work toward a more concrete understanding of cues, as well as employees' tendency to view the demands of training as a personal or professional threat: a. b. How do training professionals go about investigating the issues behind the cues? Once latent issues are identified, what should training professionals do about them? Are there some behaviors that will always indicate the presence of underlying issues, and can they be used as predictors? Is there a relationship between management style and the prevalence of certain cues or issues? When a training program has high involvement (or low involvement) of all employee groups, (upper level managers, supervisors of training participants, and training participants), over all three training stages, are certain cues more common than others? Perhaps extensive studies of similar organizations would help to isolate factors (latent issues) which may be present in specific configurations. Since one one can probably be associated with more than one latent issue, perhaps future studies could help to associate particular cues and issues when observed in certain organizational climates. APPENDICES APPENDIX A TELEPHONE SCRIPT 115 APPENDIX A TELEPHONE SCRIPT The following outline includes the basic information presented to and solicited from all of the study participants during an introductory telephone conversation. The purpose of this telephone contact was: 1) to inform prospective subjects of the study's purpose and method, 2) to ensure the person's eligibility for participation, and 3) to secure a verbal commitment to participation. 1. The Purpose of the Telephone Call a. Your name was drawn at random from the ASTD/NSPI membership directory. b. I am currently working on a doctoral dissertation. c. The study concerns improving the ability of training professionals to recognize obstacles to training success. 2. Determininggthe Individual's Eligibility a. Are you currently working in a training capacity within an organization? (Rather than as an external consultant?) b. Approximately how long have you worked in the training field? 3. Purpose of the Research a. As a trainer/training coordinator/training developer, you've probably discovered a number of roadblocks to training effectiveness that you don't have immediate control over. b. For instance: When a program is still in the planning stage, you may pick-up signals from your supervisor that tell you he/she is not very 116 supportive of some aspects of the training. You perceive that some problem exists, based on something your supervisor said or did. You picked-up a 922. (Additional examples are posed, as necessary, which refer to the individual's current work responsibilities.) These are problems that come up because someone else (other than you) has an idea or a concern that runs counter to the purpose of the training. Usually, the trick is to recognize these problems. Often, other people don't come right out and tell you, "I don't feel comfortable with this...." or "I don't like this...." You, the trainer, have to pick up some cues from those people, then try to interpret those cues. My question for you is, what cues have you seen that warn you of an underlying problem? Focus on: Upper level managers Training participants Supervisors of participants The written survey will guide you through three basic stages of a training project: "pre- training", "training", and "post-training". At each stage, you will be asked to write down cues that you have seen that helped you to recognize a possible training problem. The Research Methodology a. b. Forty training and development experts will participate, including yourself. We will use a Delphi study, which includes three rounds of questionnaires. After each round, I will consolidate the responses, then send them back for your review and additional comments. Benefits of the study for participants: You will be able to see how other training 117 professionals responded to the questionnaire, and in turn, you will have the opportunity to shape the final group consensus. Basically, this will be a dialogue between yourself and your peers that is carried out on paper. Also, you will receive a written summary of the results of the study. d. I will send the first questionnaire out immediately. Each questionnaire will come with complete instructions, and will take about twenty to thirty minutes to complete. f. Please complete each questionnaire and return it to me as soon as possible. Turn around within a week would be best. 9. Questions? Check participant's mailing address. APPENDIX B ROUND ONE QUESTIONNAIRE 118 APPENDIX B Date Michelle King Address Participant Name Address Dear xxxxx, Several days ago, we spoke on the telephone about a research effort concerned with improving the ability of training professionals to recognize obstacles to program success. The enclosed questionnaire is the first in a series of three inquiries that are designed to compile the opinions of forty employee training experts on this topic. As you know, a "Delphi" questioning technique will be used in this project to gather and synthesize opinions. This means that your ideas will be systematically combined and contrasted with those of 39 other professionals, and you will be able to follow the progression of ideas throughout the Delphi process. To ensure your confidentiality, I will not disclose your name, the name of your organization, or your location. Please understand that your participation is strictly voluntary; if at any point you feel uncomfortable with the process, you are free to discontinue your role in the study. This first questionnaire will take 20 - 30 minutes to complete. It will probably be the most time-consuming of the three rounds of questioning. Thank you again, xxxxx, for your interest in this project and for the effort you are about to put forth. Rest assured that you will receive a full report on the final results of this study. Sincerely, Michelle King Phone # 119 INSTRUCTIONS Several factors can hinder the success of an employee training program. Often, trainers and training coordinators will detect these obstacles only with alert, conscientious observation. For instance, if training participants, their on-the-job supervisors, or upper level managers in the organization feel in any way threatened by a training program, those feelings can interfere with the effectiveness of the training. Because feelings are not tangible, the training professional may only become aware of these problems if he/she recognizes some "cue". What Is A "Cue"? A cue is any indicator of an underlying, training- related problem. Cues may come in many forms; they may appear in the words and actions of training participants or their supervisors, or they may show themselves in a written, training-related policy handed down from top management. Below are examples of cues that indicate possible obstacles to training: a. A representative from top management promises to open the training program with a welcoming speech to participants, then forgets to show-up. b. A group of three training participants insists upon sitting in the back of the classroom, where they read newspapers and talk amongst themselves. c. The training coordinator invites all of the immediate supervisors of future training participants to contribute to the planning of the program. Less than half of the supervisors show up at the well- advertised planning meeting. d. A long-standing policy in the company stipulates that all training take place outside of regular work hours. NOte: an obvious and important one in one training situation may prove meaningless in another training situation. Use your judgement to select cues that, in your opinion, usually indicate a serious underlying problem. 120 What YOu Will Be Asked To Do: The following pages will guide you through three typical ”stages” of training. Please read the description of each stage, reflect on your experience with the many tasks involved in training, and then write down some cues that you have witnessed that indicated the presence of an underlying obstacle to training success. You will also be asked to place a check next to the description of the individual(s) who provided the cue. 121 IDENTIFYING CUIS TO TRAINING PIOBLIMS - A QUESTIONNAIRE To help trigger your memory. the training process has been divided into three stages: Pre-Training. Training. and Post-Training. Bach stage is described by a list of tests it usually includes. : Stage II: Pro—Trainin;—1 a_. The Pre-Training stage typically includes the following ten tasks: Determining Training Content Selecting Participants Allocating Budget Preparing Supervisors of Participants Selecting Instructors Preparing Upper Level Managers Program Scheduling Preparing Class Materials Creating Lesson Plans Preparing Training Site Please reflect on your experience with the Pre-Training stage. than list as many cues as you can. Use additional pages as necessary. A “CUR“: SOMETHING YOU 83!. NEAR. 08 READ UNION MAY INDI— CATE AN UNDERLYING OBSTACLB TO TRAINING INFECTIVRNBSS. CUR ONE: ORIGINATBD FROM: Training Participant Supervisor of Participants Upper Level Manager Illa CUR TNO: ORIGINATRD FROM: Training Participant Supervisor of Participants Upper Level Manager (us CUB THREE: ORIGINATBD PROM: Training Participant Supervisor of Participants Upper Level Manager (“3 Training Participant Supervisor of Participants CUB POUR: CUB OBIGINATID PROM: ___ Upper Level Manager 122 Stage s2: Training } The Training stage typically includes the following six tashs: Participant Orientation Participant Evaluation Presentation of Content Program Evaluation Skill Practice Classroom Management Please reflect on your experience with the Training stage. than list as many cues as you can. Use additional pages as necessary. A 'CUR': SOMETHING YOU SEE. REAR. OR READ NEICN MAY TNDT- GATE AN UNDERLYING OBSTACLE TO TRAINING EFFECTIVENESS. CUE ONE: ORIGINATED PROM: Training Participant Supervisor of Participants . Upper Level Manager ORIGINATED FROM: . Training Participant Supervisor of Participants Upper Level Manager CUE THREE: ORIGINATED FROM: Training Participant Supervisor of Participants Upper Level Manager ((1% ORIGINATED FROM: Training Participant Supervisor of Participants Upper Level Manager 123 , Stage an: Post-Training: k The Post-Training stage typically includes the following four tasks: Skill Transfer Upper Management Support Supervisor Support Training Follow-up Please reflect on your experience with the Post-Training stage, than list as many cues as you can. Use additional pages as necessary. A 'CUE': SONITEING YOU SEE, IEAR, OR READ NEICE NAY INDI- CATE AN UNDERLYING OISTACLE TO TRAINING EFFECTIVENESS. a.-. a..- . CUE ONE: CUE ORIGINATED PROM: Training Participant Supervisor of Participants Upper Level Manager CUE THO: CUE ORIGINATED FROM: Training Participant Supervusor of Participants Upper Level Manager CUE THREE: CUE ORIGINATED FROM: Training Participant Supervisor of Participants Upper Level Manager ORIGINATED FROM: Training Participant Supervisor of Participants Upper Level Manager 124 Pleasadaremyadiitianlmnmtsywnmyhmemingamsmuuerlymg tradnirqprdnlavs. 1 i Hiatisfiemtalrulberofmrployeesinyuir organization who are within your jurisdictim of training? hbwwmldywbestdescribethepn-imarypurpose harmfacturing/Iraiustrial of your organization? (Please died: me.) __ Mal/Immune __ Utility/Monti“ Service _ mtail Sales l-baltii Services Educatim _ Oder: Again,fiiisisfl'ef_i§tinasenesofdneeqmstiarmiresthatywwill aslcdmcmplem. Ithasbaanti'emsttine-cnmirgportimofyour participation. mwmmmmmmm. APPENDIX C FEEDBACK SHEET PROVIDED FOR REVIEWERS 125 APPENDIX C Date Michelle King Address Reviewer Name Address Dear XXXX, Enclosed is the cover letter, instruction sheet, and questionnaire that I asked you to review for my dissertation. After our telephone conversation, I believe that you have enough information about the project to approach this questionnaire as a typical research participant. Any feedback you can offer will be helpfulll (This is no time to be nice.) Please consider the following questions as you approach this task: 1. Did I (Michelle) say anything in our initial telephone conversation that was confusing or misleading? 2. Are the cover letter and instruction sheet understandable? Well written? 3. Is the survey instrument itself "approachable"? (Do you feel comfortable with the layout? Does it seem intimidating or confusing at all?) 4. Do the three "stages" of training help you to organize your thoughts? 5. Does this study seem interesting to you? Would you look forward to seeing the results? Thank you for your help! I'll be sure to send a copy of my results. Sincerely, Michelle King Phone # APPENDIX D ROUND TWO QUESTIONNAIRE 126 APPENDIX D Date Michelle King Address Participant Name Address Dear XXXXX, This is the second in a series of three questionnaires concerning cues that commonly indicate underlying training problems. I thank you for your response to the first questionnaire. As you will see, I have combined your ideas with those of your colleagues, and the resulting list of cues suggests a great deal of insight and a variety of interesting viewpoints. A word about yourself and your fellow research participants: There are forty training professionals from twenty different states participating in this study. Thirty-one of you work in the private sector, nine in the public sector. You have invested an average of 8.7 years in the training field, and you work in the following areas: 1 - Manufacturing/Industrial - Financial Services/Insurance - Utility/Communications - Service Industry Retail Sales - Health Services - Education - Military - Law Enforcement HHI-‘NNCDO‘RDUI l 40 Total Participants This portion of the study will require about 20 - 30 minutes of your time. With your prompt response, I will assimilate your ideas with those of the other participants and send the final questionnaire to you as quickly as possible. Thank you again for your input into this project. Sincerely, Michelle King 127 INSTRUCTIONS On the enclosed questionnaire, please do the following: I. Consider the Validity of Each Suggested Cue. As you may remember, a cue is something you see, hear, or read which indicates an underlying obstacle to training effectiveness. Cues some— times surface when an individual feels threatened by some aspect of a training program. The cues on the following list include only those suggested by more than one participant. Read the suggested cues carefully. and then indicate whether or not you think they are in fact cues by simply marking “yes" or “no' in the middle column. (Note that you are not offered an option to say "maybe". This is a deliberate attempt to distingunsh black from white an an admittedly gray area.) An obvaous and important cue an one training situation may prove meaningless in another training situation. By reflecting on your personal experience, select cues that PROBABLY indicate an under- lying problem that resulted in someone’s feelings of being threatened. Speculate the Cause Behind Each Cue. In many ways, a training program can be perceived as a threat to a variety of people in your organization: * An upper level manager may feel has/her AUTHORITY, PROFESSIONAL REPUTATION, or RIGHT TO VOICE AN OPINION IS somehow threatened. * A supervnsor of training part1c1pants may believe a training program threatens his/her JOB SECURITY, OPPORTUNITY FOR PROMOTION, or the LOYALTY 0F CO-WORKERS. * Finally, a training particnpant may feel his/her PERSONAL AUTONOHY, SELF ESTEEH, or COMFORTABLE WORK ENVIRONMENT is threatened. These examples of threatened feelings are not all-inclusive; there are many. many ways a training program can professionally or personally threaten people within the organization. Unfortunately, most trainers and training coordinators do not become aware of these feelings until a CUE is detected and accurately interpreted. For each of the items below that you determine to be a cue, please speculate the underlying threat(s) that the actor may be experienCing. obstacle to training effectiveness. individual feels threatened by some aspect of a training program. 128 VALIDATION AND EXPLANATION OF CUES Cues A “cue“: something you see, hear, or road which indicates an underlying sometimes surface when an Suggested Cue: Is this cue? If ”Yes". then: The person may sense a threat to his/her _____. 1) EXAMPLE: AN UPPER LEVEL MANAGER.... ... assi ns too few training staff ( members 0 handle an upcoming training program. Yes NO M Ma( 67L, «Ax—eras? , 1) 2) 3) 4) 5) 6) 7) 3) 9) AN UPPER LEVEL HANAGER.... ... does not allocate money for a training program. ... allocates money for a training program, but the amount IS grossly insufficient. ... does not allow training staff sufficnent preparation time for an upcoming program. ... expects training staff to cover an inordinate amount of maternal in a small amount of training time. ... does not set training goals according to long-range organaza ional goals. ... makes a training decisnon that runs contrary to the results of a needs analysis. ... refuses to compromise with other managers on training goals. ... requests a particular type of training, than does not take part in the subsequent planning of that program. ...does not contribute to pro-training planning sessions. Yes Yes Yes Yes Yes Yes Yes Yes YQS NO NO NO NO NO No NO No NO _ 4 1k. 415...... STEP-5'“? 129 Suggested Cue: Is this I DUO? If “Yes", then: The person may sense a threat to his/her 10) 11) 12) 13) 1) 2) 3) 4) 5) 6) 7) 8) AN UPPER LEVEL HANAGER.... ... commits to training in private. but does not demonstrate that commitment publicly to peers and subordinates. ... does not include training staff in meeting: for new products or procedures w n training of personnel will be necessary. ... makes last minute changes in the training site or instructional equipment. ... does not “sit-in“ on portions of the training, as planned. Yes Yes Yes Yes No NO NO A SUPERVISOR OF TRAINING PARTICIPANTS.... ... does not attend pre-training planning sessions. ... refuses to compromise with other superVisors on training goals. ... COOS "0t prOVIdO necessary information for the “EGGS lhfllYSIS. ... is slow and haphazard in his/ her efforts to re ister subordinates for raining. ... authorizes training for subordinates, but not or himself/ herself. ... does not adequately prepare subordinates for a training program, so participants come to training not knowing what to expect. ... does not distribute training program announcements. ... repeatedly pulls participants out of the training program for short periods of time. YES Yes YBS Yes Yes YOS Yes Yes NO No No No NO No No 130 Is If "Yes", then: this a The person may sense a Suggested Cue: cue? threat to his/her 9) 10) 11) 12) 13) 1) 2) 3) 4) 5) 6) 7) 3) 9) A SUPERVISOR OF TRAINING PARTICIPANTS.... ... publicly criticizes the training program. ... does not guarantee that subordinates will attend training. ... does not reinforce the use of new skills when participants return from training. ... refuses to alter the work schedule so that subordinates can implement newly-trained skills. ... does not participate in training follow-up activities. A TRAINING PARTICIPANT.... ... does not show-up for a long- scheduled, much-advertised training program. ... verbalizes resentment at haying to attend training. ... shows diSinterest during training by reading, not taking notes, initiating Slde conversations, e c. ... repeatedly fails to return from class breaks on time. ... does not complete homework assignments deemed necessary for training success. ... does not get actively involved in the trainin , chooSing instead to remain of t e periphery of class actiVites. ... frequently leaves class for short periods of time. ... does not apply newly-trained skills when back on the Job. ... does not respond to training follow-up activities. YES YES Yes YES YES Yes Yes Yes YES YES Yes Yes Yes Yes NO No No No NO NO NO NO NO No NO NO NO NO 131 WHAT HAPPENS NEXT: Ihen you have finished coepletinf this questionnaire, please return it in the enclosed envelo . I wil combine your ideas with those of the other study Eerticipan a. then return a revised list of cues back to you for furt er consideration. This is the second in a series of three guestionnaires that you will be asked tgbfomplete. The final questionns re will be sent to you as soon as poss e. THANK YOU FOR YOUR CONTRIBUTIONS! APPENDIX E ROUND THREE QUESTIONNAIRE 132 APPENDIX E Date Michelle King Address Participant Name Address Dear xxxxx, This is the final questionnaire involved in the three- tiered research project concerning cues to underlying training problems. Participant response to the earlier questionnaires was overwhelmingly copious and insightful. Interestingly, unmistakable patterns are beginning to emerge from what I sometimes feared would be a hopelessly complex dilemma for training professionals. In this questionnaire, the seventeen most commonly- validated cues are presented, along with the possible "causes" that were suggested by you and your colleagues. Your professional opinion, which I solicit one last time, will help to move us one step closer to group consensus on the identification of and cause behind the common "red flags" of our trade. This fall, I will send a final report on the results of this research effort. At that point, I hope that you will see your efforts Justly rewarded by a practically- useful analysis of the ideas collected from yourself and the other contributors. Thank you, once again, for your valued input. Sincerely, Michelle King Phone # w“ 133 INSTRUCTIONS On the enclosed questionnaire. you will find three columns: 8 In the left-hand column. I have listed the seventeen cues that you and the other study participants determined to be the most likely indicators of underlying training problems. These cues are not listed in any particular order. 8 In the center column. I have listed the perceived threats that study participants suggested may be behind each cue. 8 In the right-hand column. the numbers '1 - 1' provide a means for you to rate the degree of urgency for each cue. PLEASE DO THE FOLLOWING: A. Read each cue. then review the list of threats (in the center column) that were suggested by the study participants. 3. Place a ”I” next to the most likely threat for each cue. and a “2” next to the second most likely threat. Further ranking is not necessary. If cu believe that a probable threat is not listed. write it in mes to “other:” and include it in your ranking. C. Note the right-hand column. Considering the t of threat that you thought most likely caused each cue, (i.e. he threat that you ranked ' “), rate the degree of urgency. The rating system is described as follows: 4 = A Nearly Unsurmountable Obstacle. When I see this cue. I stog everything. confront the underlying p;gbl€m. and then decide w ether to continue with the training e or . 3 = A Ver Serious Obstacle. When see this cue. I continue lannin Iconductin _the training program, but consider t e reso ution to t is problem a top priority. 2 = A Difficult Obstacle. I consider this cue an indicator of a difficult. but routine, training problem. I = A Hinor Obstacle. The underlying problem indicated by this cue is considered a low prior ty problem. FINAL DETERNINATION - UNDERLYING 134 THREAT AND DEGREE OP URGENCY ) , do t~q° v °b I) b s E l I“: 8b .9 * s? t 0 l \ b. .p Q ? ‘ 0 ~ “ a A I). I, est ‘~ IA: Q, l. ay’r' 0 Rs fibfflf fig_a§. ”ha‘EQ «qf0~ ’7q6 a: e ': <§¢ ‘b ) 1"}: C," A. b t . O0 56,. is: .0 . 'bt IIANPLB: 1) An upper level mana er 2. authority 4 3 1 assi as too few tra sing 1 personal work load. staf members to handle 0 Kortunity for recognition an upcoming program. other priority programs. ___ 0 er: I) An upper level manager allocates money for a traini program, but the amount a grossly insufficient. 2) An upper level manager makes a training decision that runs contrary to the results of a needs analysis. 3) level manager 3"..." 8:” srivate, emonstrste that commitment publicly to training in but does not to peers and subordinates. pride. ( financial accountability. other priority projects. re uta ion as a tough, ef icient manager. belief that training ought to be inexpensive. authority. control. other: authority, control. own interests. priorities. reement with needs ans ysis means public recognition of roblems.) comfortable wor routine. reputation as a manager who succeeds on instinct. not systematic analysis. ularity. Egger: credibility. (Pears being associated with training if problems arise.) pularity. Igage as a 'hardline” manager. {ob security. (Reluctant 0 voice an opinion.) other: 135 ’6. f ' O 056’ .3 - °° r‘. 'P ‘§ 45 J a éfo °Z 3%" I"a “0 é . p. O ‘0 a? I»; *9 “a ’08.” 0% '0 % 1 e. . 0‘ . J _ h o 0 1+ .g C c l‘ "e r ’1' 5 . 60 C6,. Is: Q’s: °¢ 4) An upper level manager. authority, control. 4 3 2 1 does not include training Job security. (Attem t to staff in meetings for new make self indispensa le.) products or procedures elitism. (Top-level . when training of personnel decisions too sophisticated will be necessary. for training staff.) training expertise._(wants to make decisions without ‘ consulting training staff.) other: 5) A superVisor of training skill level. (Lacks con- 4 3 2 1 participants does not fidence in ability to attend pre-training contribute to mac ing.) planning sessions. other work priorities. status. (Believes such ' work should be left to underlin 5. power. Be ieves that information shared is power lost.) other: 6) A supervisor of training authority, control. 4 3 2 1 participants refuses to credibility. (Disguises compromise with other lack of management skill supervisors on training with belligerence.)r goals. own interests/priorities. other: 7) A supervisor of training credibilit . (Fears her 4 3 2 1 participants does not provide necessary infor- mation for the needs analysis. own lack 0 skills will be revealed. power. ( elieves that information shared is power lost.) _ comfortable work routine. (Believes training will bring unwanted change.) status. (Believes such work should be left to underlings.) other: 136 ) f q’cp EOQ.A) G? ‘ 05 E Q \ J q. r- a . l C 3O J 4' 9 Th, J 4’ C ‘0 Q. “ °¢ '1: P a“ - ‘b‘ J Jr 45 .r 63*Q, 9: t 9’ “b “3'0 " *0 f J: ~ A 0’ To A O “U ‘5 P Cg.fio 96 .0 J; '. E at): Gt 4% *t e 1, 6. Q 6’ 4s: “e, q‘t B) A supervisor of training authority. control. 4 3 2 1 articipants authorizes skill level. (Lacks . raining for subordinates, confidence in his ability but not for him/herself. to learn.) familiar routine. (Not_ willing to attend training) familiar routine. (Use c new skills requires change of on-the-Job routines.) reputation as a mgrtyr. (willing to ‘ ive training to others, bu not self.) other: 9) A supervisor of training pride. (Believes old work 4 3 2 1 partiCipants publicly practices are better than criticizes a training new.) program. Job security. (Believes subordinates’ new skills make her dispensable.) superiority over training staff, training function. popularity. other: 10) A superVisor of training credibility. (Believes 4 3 2 1 participants does not the “new skills make her reinforce the use of new old skills obsolete.) skills when participants authority. (Feels subor- return from training. dinates’ new skills ive them power over him. comfortable routine. (To incorporate new skills reqUires change.& Job security; ( elieves subordinates skill may make her dispensable.) other: 137 f f (66. 6. O 0%“, g g. l ‘ t ‘bo 31g 1 ‘P.C‘q° {b.g J§.* ‘3 ‘h ’c “Q J /~*. I, ’p @. (I E 0 q k ”3. J J h ”a 0 e t), ‘- I ”a ‘t J R (‘6 .0 a, See J J, o o f {b ta. ’1. ‘6’. t %... 0.: is, ll) A supervisor of training credibilit . (Believes 4 3 2 l partici ants refuses to " ”new“ skil s make his alter t work schedule old skills obsolete.) so that subordinates can authority. (Feels subor- implement newly-trained ' dinates’ new skills ive skills. them power over her.2 comfortable work rou ine. Job security; (Believes subordinates skills may make him dispensable.) other: 12) A training participant other work priorities. 4 3 2 i does not show-up for a comfortable work routine. long-scheduled, much- "' pride. (Believes she does advertised training not need training.) program. pride. (Fears training will reveal weaknesses.) personal autonomy. ""' (Resents mandatory attendance.) other: 13) A training partiCipant other work priorities. 4 3 2 l verbalizes resentment at comfortable work routine. haying to attend training. "" pride. (Believes he does not need training.) pride. (Fears training will reveal weaknesses.) pride. (Believes her daily work is too important to miss.) personal autonomy. (Resents mandatory attendance.) other: 14) A training participant intelligence. (Believes 4 3 2 1 shows disinterest during he doesn’t need training.) training by reading, pride. (Fears training initiatin side conversa- will reveal weaknesses.) tions, no taking notes, other work responsiblities. etc.. personal autonomy. . comfortable work routine. "" status. (Believes adults '”" don’t have to be as _ attentive as school kids.) other: 138 ’6. A o v f. s‘ l ..t .0 J \‘ b ‘5 0% s t o O .5 I Q <“ 0 q. ~‘ 4 b A: ‘c . “.2? I"? s ”‘6?” P389 0“ 1 fi-qb J 4s ‘0 ”q :4. ’5» ”a, ’c 0% .00 ab-1 '. S o *e .e t .e 5 .O J ”r ‘51 I I Q. 0 ° *.J t t/k T b C ’h 0‘, by». ’3‘ e. : . If: 15) A training participant personal autonomy. 4 3 2 1 repeatedly ails to (Resents that training was return from class breaks mandatory.) _ on time. other ob priorities. comfor able work routine. "' (Accustoned to leasurely breaks.) _ pride. (Fears training will reveal weaknesses.) other: 16) A training participant personal autonomy. 4 3 2 1 does not complete home- (Resents that homework is work assignments deemed considered mandatory.) necessary for training pride. (Fears homework success. " will reveal weaknesses.) other work priorities. ___ ability to conceal reading or writing deficiencies. comfortab e work routine. other: 17) A training participant comfortable work routine. 4 3 2 1 does not ap ly newly- trained ski ls when back on-the-Job. credibility. (Fears making mistakes.) agproval of superVisor. ( uperVisor discourages use of the new skills.) agproval of peers. ( thers discourage of the new skills.) other: thE USE w .. -91‘ .1 APPENDIX F REMINDER CARDS 139 APPENDIX F Thank you for your prompt reply to the research questionnaire concerning cues o underlying training problems. The second questionnaire will be sent to you as soon as a sufficient number of participants have responded to the first. I expect this to happen within the next two weeks. THANKS AGAIN FOR YOUR VALUED TIHE AND PATIENCE! Michelle King 517-393-6276 A FRIENDLY REMINDER If you haven’t done so already. please complete and return the second researc questionnaire concerning cues to underlying training problems. Your contribution is important! THANKS AGAIN. hichelle King 517-393-6276 SEND IT IN 9! If you haven’t done so already. please complete and return the third and fins questionnaire of the ”cues to training problems“ study. A summary of the research results will be sent to you in the Fall. I SINCERELY APPRECIATE YOUR HELP! Hichelle King 517-393-6276 Sent on 6-13-89 to partigépsnts whct)1 respo e pro y to the first mp questionnaire. Sent on 7-17-89 to participants who were slow to respond to the second questionnaire. Sent on B-16-B9 to participants who were slow to respond to the third questionnaire. BIBLIOGRAPHY BIBLIOGRAPHY Allen RF, Kraft C. (1982). The Organizational Unconscious. Englewood Cliffs, NJ: Prentice-Hall. Allport GW. (1937). Personality: A Psychological Interpretation. New York: Holt. Amaya T. (1980). Organization development in Japan. Journal of Enterprise Management, 2, 221-236. Argyris C. (1984). Intervention Theory and Method: A Behavioral Science View. Reading, MA: Addison- Wesley. Argyris C. (1985). Strategy, Change and Defensive Routines. Marshfield, MA: Pitman Publishing. Baum H. (1987). The Invisible Bureaucracy - The Unconscious in Organizational Problem Solving. New York: Oxford University Press. Baumgartel H, Reynolds M, Pathan R. (1984). How personality and organizational-climate variables moderate the effectiveness of management development programmes: A review and some recent research findings. Management and Labour Studies, 9, 1-16. Baumgartel H, Jeanpierre F. (1972). Applying new knowledge in the back—home setting: A study of Indian managers' adoptive efforts. Journal of Applied Behavioral Science, §(6), 674-694. Beckhard R. (1969). Organizational Development: Strategies and Models. Reading, MA: Addison-Wesley. Bennis W. (1969). Organization Development: Its Native Origins, and Prospects. Reading, MA: Addison- Wesley. Bennis W. (1986). Changing Organizations: Essays on the Development and Evolution of Human Organizations. New York: McGraw-Hill. 140 141 Blake RR, Mouton JS. (1971). In Burke WW, Hornstein HA (Eds), The Social Technology of Organization Development. Washington D.C.: NTL Learning Research Corporation. Borg WR, Gall MD. (1983). Educational Research. Fourth Edition. White Plains, NY: Longman, Inc. Brunsson N. (1985). The Irrational Organization: Irrationality as a Basis for Organizational Action and Change. New York: John Wiley & Sons. Burke WW. (1986). Organizational Development: Principles and Practices. Boston: Little, Brown. Camp RR, Blanchard PN, Huszczo GE. (1986). Toward A More Qgganizationally Effective Training Strategy and Practice. Englewood Cliffs, NJ: A Reston Book, Prentice-Hall, Simon & Schuster, Inc. French WL, Bell CH. (1984). ggganization Development: Behavioral Science Interventions for Organization Improvement. Third Edition. Englewood Cliffs, NJ: Prentice-Hall. Goldfried MR, Kent RN. (1972). Traditional versus behavioral personality assessment: A comparison of methodological and theoretical assumptions. Psychological Bulletin, 22, 409-420; Goldstein IL. (1980). Training in Work Organizations. Annual Review of Psychology, 31, 229-272. Goldstein IL. (1986). Training in Organizations: Needs Assessment, Development, and Evaluation. Second Edition. Monterey, CA: Brooks/Cole. Hand HH, Richards, MD, Slocum JW. (1973). Organizational climate and the effectiveness of a human relations training program. Academy of Management Journal, 1§(2), 185-195. Havelock RG. (1973). The Change Agent's Guide to Innovation in Education. Englewood Cliffs, NJ: Educational Technology Publications. Hicks WD, Klimoski RJ. (1987). Entry into training programs and its effects on training outcomes: A field experiment. Academy of Management Journal, 39(3), 542-552. 142 Huczynski AA, Lewis JW. (1980). An empirical study into the learning transfer process in management training. Journal of Manggement Studies, 12(2), 227—240. Kast FE. (1961). Motivating the organization man. Business Horizons. Spring, 58. Katz D, Kahn RL. (1966). The Social Psychology of Organizations. New York: Wiley. Kiechel W III. (1982, December 27). Corporate strategies. Fortune. 34-39. Knowles M. (1984). The Adult Learner: A Neglected Species. Second Edition. Houston, TX: Gulf Publishing Co. Laird D. (1985). Approaches to Training and Development. Second Edition. Reading, MA: Addison-Wesley. Lawler EE III, Nadler DA, Cammann C. (1980). Organizational Assessment: Perspectives on the Measurement of Organizational Behavior and the Quality of Worklife. New York: John Wiley & Sons. Lewin K. (1958). Group decision and social change. In Maccoby EE, Newcomb TM, Hartley EL (Eds.), Readings in Social Psychology. New York: Holt. Linstone HA, Turoff M. (1975). The Delphi Method, Techniqyes and Applications. Reading, MA: Addison- Wesley Publishing Company. Mann FC. (1957). Studying and creating change: A means to understanding social organization. Research in Industrial Human Relations. Industrial Relations Research Association, (17), 146-167. Maslow, A. (1954). Motivation and Personality. New York: Harper. Merton RK. (1968). Social Theory and Social Structure. New York: The Free Press. Moore GL. (1984). The Politics of Management Consulting. New York: Praeger Publishers. Orlich DC. (1978). Designing Sensible Surveys. Pleasantville NY: Redgrave Publishing Company. 143 Pilon DH, Bergquist WH. (1979). Consultation in Higher Education. Washington, DC: CASC: The Council For The Advancement of Small Colleges. Porras JI, Hoffer SJ. (1986). Common behavior changes in successful organization development efforts. The Journal of Applied Behavioral Science, 22(4), 477-494. Presthus R. (1962). The Organizational Society. New York: Vintage Books, Random House. Rogers CR. (1951). Client-Centered Therapy. Boston, MA: Houghton-Mifflin Publishers. Rogers CR, Roethlisberger FJ. (1952). Barriers and gateways to communication. Harvard Business Review, xxx,(4). Schein EH. (1980). Organizational Psychology. Third Edition. Englewood Cliffs, NJ: Prentice-Hall. Schein EH. (1985). Organizational Culture and Leadership. New York: Jossey-Bass. Schien EH. (1987). Process Consultation, Vol. II: Lessons for Managers and Consultants. Reading, MA: Addison-Wesley. Steele F. (1975). Consultipg For Organizational Change. Amherst, MA: University of Massachusetts Press. Tichy N. (1973). An interview with Roger Harrison. The Journal of Applied Behavioral Science, 2(6), 701- 723 e Weber M. (1946). Bureaucracy. New York: Oxford University Press. Wexley KN, Latham GP. (1981). Developing and Training Human Resources in Ogganizations. Glenview, IL: Scott, Foresman and CO. Witkin BR. (1977). Needs assessment kits, models and tools. Educational Technolggy, XVII(11), 5-17. Zemke R, Kramlinger T. (1982). Figurinnghings Out. A Trainer's Guide to Needs and Task Analysis. Reading, MA: Addison-Wesley Publishing Company, Inc. 11111111111111