n— . E ,,,,,, LIBRARY Michigan Sm - ' University ' Ill]!\llllllllllflllllllllllllllllIlllllgllll This is to certify that the thesis entitled EXAMINATION OF AN IN-SERVICE TRAINING PROGRAM FOR THE STAFF OF A JUVENILE CORRECTIONAL FACILITY USING AN EDUCATIONAL EVALUATION MODEL presented by E r i c Howa rd has been accepted towards fulfillment of the requirements for __Ph_.£.__._degree in C un el In , Personnel Services and Educational Psychology aid 55% Major professorw Fred C. Tinning, Ph.D. Date II-8-78 0-7 639 Copyright by ERIC HOWARD 1978 EXAMINATION OF AN IN-SERVICE TRAINING PROGRAM FOR THE STAFF OF A JUVENILE CORRECTIONAL FACILITY USING AN EDUCATIONAL EVALUATION MODEL By Eric Howard A DISSERTATION Submitted to Michigan State University in partial fulfillment of the requirements for the degree of DOCTOR OF PHILOSOPHY Department of Counseling, Personnel Services and Educational Psychology 1978 6‘/086'c7 L ABSTRACT EXAMINATION OF AN IN-SERVICE TRAINING PROGRAM FOR THE STAFF OF A JUVENILE CORRECTIONAL FACILITY USING AN EDUCATIONAL EVALUATION MODEL BY Eric Howard The purpose of this study was to evaluate the merits and processes of an in—service training program for the staff of a residential correctional facility for juvenile delinquents, using an educational evaluation model. It has been widely accepted that improvements in the quality of in-service training for correctional staffs are needed in order to improve the quality of rehabilita- tion services provided by such facilities. Ewing and Gregg (1978) indicate that the use of systematic evalu— ation procedures is crucial to improving the quality of in-service training. The in—service training program evaluated in this study was a two-week, on—site, multi-unit training pro— gram funded by the Law Enforcement Assistance Adminis— tration. The primary focus of the training program was On treatment techniques to be used in the facility. The . . . training program was intended to increase the part1c1pants a Eric Howard knowledge of the techniques and their skill in implement- ing them. The setting for the study was a southeastern Michigan youth facility. The facility was opened for use in October of 1977 during the course of the training pro— gram. The facility provides both detention and residential treatment services. The subjects of this study were the 27 employees of the facility who were involved in routine, direct contact with residents. The plan to evaluate the training program was based on a model provided by Robert Stake, an educational evaluator. The training program was organized into ante- cedents, transactions, and outcomes. Observations were made of these antecedents, transactions, and outcomes in order to be able to examine the relationships (contin— gencies) among them. In addition, the congruence of the program (the degree to which the events and conditions 0f the program have occurred as they were planned) was eXamined. A variety of observational methods were devised or adaPted for this evaluation. A priori standards were set for judging whether or not the antecedents, transactions, and outcomes had occurred as intended. Cognitive abili— ties, attitudes, and skill levels of the participants Were measured and compared to a group not enrolled in training who work at another facility. Multivariate and univariate analyses of covariance were used to examine several outcomes. For others, E86 SIE Eric Howard measures of central tendency were compared to a priori standards. The results indicate that a significant amount - Ci knowledge was gained by participants. Only partial support was provided by the evidence to indicate that the participants improved in their work abilities, used the information after training, and felt positively about the training experience. The evidence did not indicate that the training program resulted in improved attitudes toward the use of the techniques or that the participants shifted in their preference toward types of delinquents. These results are examined in terms of the contingencies and congruences of training. Several implications for in—service training are advanced. In-service training programs should be lengthened and should emphasize technique rather than theory. Use of proficient training presenters is essen- tial, as are follow-up mechanisms to training. Implications for future evaluations of in—service training are also advanced, including a discussion of the use of Stake's model. Observational instrumentation is lacking in this area and represents perhaps the greatest need in the field of in—service training evaluation. This dissertation is dedicated to those four individuals who have had the greatest influence on me: to the memory of my father, Ardath Howard; to my mother, Ruth Howard; to my son, Keith; and to my loving wife, Marcia. ii ACKNOWLEDGMENT S While many friends and acquaintances have helped make this endeavor possible, I would like to publicly acknowledge the efforts of several of them. Dr. Fred C. Tinning, as the director of my dissertation, is to be commended for the thoroughness and dedication to the task he displayed as well as his willingness to undertake responsibility during the project beyond that usually expected of a dissertation director. Dr. James Engelkes, as the chairman of my guidance committee, is also to be thanked for his willingness to assume additional responsi- lfllity mid-way through the project. Dr. Herbert Burks is thanked for his valuable contributions to the dissertation, particularly of an editorial nature. Dr. Don Melcer‘s good humor and contributions to the theoretical aspects of the project are much appreciated, as is Dr. Tom Gunnings’ vdllingness to serve on my committee. Several individuals provided significant guidance to me during this project. Drs. Judith Taylor and William Frey are sincerely thanked both for their extensive help with statistical and design considerations and for their iii personal friendship and support throughout. Jerry Juhr is thanked for providing a great deal of his time and effort, particularly in the development of instrumentation. Appreciation is expressed to Dr. E. L. V. Shelley fin:providing the opportunity to become involved in the training program and for being such a warm-hearted boss and mentor these several years. The administration and staff of the Monroe County Youth Center are to be commended both for their willingness to be a part of a study and their dedication to serving the needs of delinquent youth. Particular thanks go to Doug Redding, M.S.W. at the facility, for significant and invaluable help throughout the project, and to bus. Marilyn Malters for assisting in the time-consuming audio-taping procedures. Mr. Joe Shaver at the Ingham County Shelter Home freely gave of his time to arrange for participation in the study of the staff there, and his efforts are greatly appreciated. Those individuals at the Shelter Home who volunteered to engage in testing without receiving train— ing are certainly appreciated, as well. Dr. Ray Husband and Mr. Keith Ostien gallantly served in dual capacities as both raters and friends dur— ing this project and are sincerely thanked for their activities in both areas. iv Appreciation is extended to the International Rehabilitation and Special Education Network at Michigan Skate University for partial sponsorship of this project mnito the many individuals there who aided in this under- taking. Finally, I am pleased to acknowledge the fact that without the typing and editorial efforts, calmness, dedi- cation, personal love and support of my wife Marcia, this project would never have been completed. LIS Cha TABLE OF CONTENTS LIST OF TABLES . . . . . . . LIST OF FIGURES. . . . . Chapter I. INTERPRETING THE TASK Introduction to the Problem. . . . The Problem . . . . . . . . . . Purpose . Research Hypotheses Related to the Evaluation Issues . . . . . . . Intended Outcome Hypotheses . Additional Unintended Outcome Hypothesis. Overview . . . . . . . . . . II. REVIEW OF THE LITERATURE . . . . Introduction to the Review Delinquency . . . . . . . Description of Delinquency . . . . Prevention of Delinquency. . . . . Treatment of Delinquency . . . . . . Treatment of delinquency at the Monroe County Youth Center . . . The Importance of Staff Training for Effective Treatment . . . . Evaluation of Staff Training. Stake's Evaluation Model . . . . . Introduction to the Model . . . . Evaluation Model . . . . . . - Antecedent conditions. Transactions. . vi Page xiii 12 l3 l4 l4 l6 l6 l7 19 24 Chapter page Outcomes . . . . 54 Organization and processing of data . . 55 Descriptive data . . . . . . . . 56 Judgment data . . . . . . . . . 60 Relationship of the Model to the Present Study . . . 64 Summary of the Review of the Literature . . 65 III. DESIGN OF THE STUDY . . . . . . . . . 68 Introduction . . . . . . . . . . . 68 Description of the Participants. . . . . 69 Description of the Facility . . . . . . 74 Detention Resources . . . . . . . . 77 Treatment Resources . . . . . . . . 78 Description of the Training Program . . . 78 Intended Antecedents to Training. . . . 81 Intended Transactions during Training . . 82 Intended Outcomes of Training. . . . . 84 Methods of Observation. . . . . . . . 85 Anecdotal Record Form . . . . . 88 Anticipation of Training Survey . . . 88 Attitudes toward Using Concepts Survey. . 89 Cognitive Areas Objective Test . . . . 90 Correctional Preference Survey . . . 92 Critical Incidents in Child- -Care Work Objective Test . . . . . 97 Satisfaction with Training Survey . . . 98 Staff Observation Rating Scale . . . . 100 Follow— —up Questionnaire. . . . . . 106 Evaluation Procedure . . . . . . . . 107 Procedure for Data Collection. . . . . 108 Instrument administration . 108 Comparison group—-Ingham County Shelter Home employees . - - - 109 Use of the Anecdotal Record Form . . . 112 Audiotaping of trainee- delinquent 113 interactions . . Administration of Follow— up Question— 113 naire . . . . . » - - Char Chapter IV. V. Method of Data Analysis . . . Conceptual Basis. . . . . . . . Practical Considerations . . . . . . Statistical Analysis . . . . . . Null Hypotheses Related to Intended Out— comes . . . . . . . Null Hypothesis Related to Additional Unintended Outcome . . . . Summary . . . . . . . . . . ANALYSIS OF THE DATA. . . . Introduction. . . . . . . . . Intended Antecedents . . . . . . . Intended Transactions. . . Intended Outcomes Summary . . . CONCLUSIONS CONCERNING THE EFFECTS OF THE TRAINING PROCESS, EXAMINATION OF THE PROCESS OF TRAINING, IMPLICATIONS FOR FUTURE TRAINING AND TRAINING EVALU- ATION, AND SUMMARY. . . . . . Introduction and Overview . . . Report on the Effects of the Training Program . . . . Research Hypotheses. Summary of Effects of the Training Program . . . . . . . . . Relationship of Antecedents and Trans- actions to Outcomes. Implications for Future Training Programs Length of Training . . Type of Material Covered . . Style and Format of Training. . . . Expectations of Training Programs Implications for Future Evaluations of Training Programs . . . . . Stake's Model. . . . . . . . . viii Page 113 113 114 116 117 125 127 129 129 130 143 148 187 194 194 196 197 201 213 213 216 217 219 220 221 Cha AP} Chapter Means of Observation in Evaluation Design Considerations in Evaluation. Summary . . . . . . GLOSSARY . . . . . . . . . . . . APPENDICES A. DESCRIPTION OF TRAINING TRANSACTIONS. B. COGNITIVE AREAS OBJECTIVE PRETEST——POST- TEST. . . . . . . . . . . . C. ATTITUDES TOWARD USING CONCEPTS SURVEY PRE— TRAINING-—POST—TRAINING . . . . . . D. CRITICAL INCIDENTS PRETEST-—POSTTEST E. FOLLOW-UP QUESTIONNAIRE . . . . . . . F. STAFF OBSERVATION RATING SCALE INSTRUCTION MANUAL . . . . . . . . . . . . G. STAFF OBSERVATION RATING SHEET. H. SATISFACTION WITH TRAINING SURVEY. . I. CORRECTIONAL PREFERENCE SURVEY PRE-TRAINING—— POST-TRAINING. . . . . . . . . . J. INDIVIDUALS PRESENT AT THE TWO MEETINGS DURING WHICH CONTENT AREAS WERE DECIDED. . . . . . . . . K. EXPERTS IN THE FIELD CONSULTED TO FURTHER REFINE AND DEVELOP THE IN-SERVICE TRAIN— ING PROGRAM CONTENT. . . . . . . . L. ANECDOTAL RECORD FORM. . . . . . M. ANTICIPATION OF TRAINING SURVEY . . N. EXPERTS IN THE FIELD AIDING IN THE CREATION OF THE COGNITIVE AREAS OBJECTIVE TEST. ix Page 222 225 229 235 242 249 262 264 268 270 275 276 282 287 288 289 290 295 Page APPENDICES O. EXPERTS IN THE FIELD AIDING IN THE CREATION OF THE CRITICAL INCIDENTS TEST . . . . . . 296 P. INDIVIDUALS RATING STAFF MEMBERS' WORK SAMPLES ON THE STAFF OBSERVATION RATING SCALE . . . . . . . . . . . . . 297 IJST OF REFERENCES . . . . . . . . . . . . 299 .4, IIEIE I I Table 10. LIST OF TABLES Page Participants in the Training Program (by Job Title) . . . . . . . . . . . . 71 Training Session Attendance Figures — Intended Antecedent #5 . . . . . . . . . . 135 Group Means and Standard Deviations on Survey Questions Relating to Perceived Usefulness of the Training Information — Intended Antecedent #7 . . . . . . . . . . . 138 Group Means and Standard Deviations on Responses to Satisfaction with Training Survey Items Relating to Perceived Pro— ficiency of the Training Presenters - Intended Antecedent #9 . . . . . . . . 141 Stepwise Regression F Statistics for the Seven Possible Covariates on the Set of Four Post—Training Subscales of the Cognitive Areas Objective Test - Intended Outcome #1 . . . . . . . . . 150 Pre—Training Group Means on the Two Covariates Used in the MANCOVA for Intended Outcome #1 . 152 Univariate F Statistics for the Four Dependent Variables - Intended Outcome #1 . . . . . 154 Stepwise Regression F Statistics for the Seven Possible Covariates on the Set of Four Post— Training Subscales of the Attitudes Toward Using Concepts Survey — Intended Outcome #2 . 157 Pre—Training Group Means on the Two Covariates Used in the MANCOVA — Intended Outcome #2. . 159 Stepwise Regression F Statistics for the Four Possible Covariates on the Post-Tra1n1ng Critical Incidents Test Scores — Intended Outcome #3 . . . . . . . . . . 163 xi Table 11. 12. 13. 14. 15. 16. Page Group Means and Standard Deviations on Follow— up Questionnaire Responses Relating to Per- ceived Helpfulness of the Training Infor- mation During the Six Months after Training — Intended Outcome #3 . . . . . 166 Means and Standard Deviations for Ratings on the Staff Observation Rating Scale - Intended Outcome #4. . . . . . . . . 170 Group Means and Standard Deviations on Follow- up Questionnaire Responses Relating to Per— ceived Application of Information Presented in Training During the Six Months after Training - Intended Outcome #4 . . . . . 172 Group Means and Standard Deviations on Satis- faction with Training Survey Responses Relating to Attitudes about the Training Experience Immediately after Training — Intended Outcome #5. . . . . . . . . 175 Group Means and Standard Deviations on Follow— up Questionnaire Responses Relating to Attitudes about the Training Experience Six Months after Training - Intended Outcome #5. . . . . . . . . . . . 179 Stepwise Regression F Statistics for the Seven Covariates on the Set of Four Post-Training Behavior Category Sub— scales of the Correctional Preference Survey - Additional Outcome #1 . . . . . 185 xii 'M‘ ,_._ ._._. Fig LIST OF FIGURES Figure Page 1. A layout of statements and data to be col— lected by the evaluator of an educational program (Stake, 1972, p. 38) . . . . . . 57 2. A representation of the processing of descrip- tive data (Stake, 1972, p. 43) . . . . . 59 3. A representation of the process of judging the merit of an educational program (Stake, 1972, p. 47) . . . . . . . . . . . 63 4. Monroe County Youth Center employee organi— zational chart. . . . . . . . . . . 70 5. Monroe County Youth Center youth flow chart. . 76 6. Summary of intended antecedents, transactions, and outcomes of training with corresponding methods of observation . . . . . . . . 86 7. Administration of the instrument for evalu— ation purposes . . . . . . . . . . 110 8. Variable matrix for group membership vs. measure relating to Hozl (Intended Out- come #1--Increased Knowledge). . . . . . 118 9. Variable matrix for group membership vs. measure relating to Ho:2 (Intended Out- come #2--Improved Attitudes toward Use of Concepts) . . . . . . . 120 10. Variable matrix for group membership vs. measure relating to Ho:3a (Intended 122 Outcome #3-—Improvement in Ability). xiii Fig Figure 11. 12. 13. Variable matrix for group membership vs. measure relating to additional Ho:1 (Unintended Outcome #1 shifts in atti— tudes toward types of delinquents) . . . Summary of intended antecedents, transactions, and outcomes of training with corresponding methods of observation . . . . . . . Summary of results organized according to Stake's model . . . . . . . . . xiv Page 126 188 189 CHAPTER I INTERPRETING THE TASK Introduction to the Problem Review of the pertinent literature indicates that rehabilitation services in juvenile correctional insti— tutions in America have been largely unable to demonstrate positive results (Jesness, 1978). One factor that is believed to be contributing to this lack of demonstrated success is the poor quality of in—service training for the staff members working in these institutions (Coffey, 1975). Improvement in staff training has been listed as a major priority recommendation by Blair G. Ewing, Acting Director of the National Institute of Law Enforcement and Criminal Justice, and James M. H. Gregg, Acting Adminis- trator of the Law Enforcement Assistance Administration (LEAA) in their Executive Summary to the National Manpower Survey of the Criminal Justice System (Ewing & Gregg, 1978). Systematic evaluation of training effectiveness is specified by Ewing and Gregg (1978) as one of the means of accomplishing this much-needed improvement in training. The lack of available evaluation data in the current literature on the comparative merits of training pmograms and the process of in-service training itself has made improvement of the quality of training difficult, if not impossible up to this point. There has been no known reported attempt to eval- uate an in—service training program for juvenile correction staff using a systematically complete educational evalu- ation approach. It is a contention of this study that application of an educational evaluation model proposed by Robert Stake (1972, 1975, 1976, 1978) makes thorough evaluations in the field of in-service training for cor— rectional staff both possible and worthwhile. The specific purpose of this study is to implement Stake's model of evaluation fully in order to investigate the effects of processes of a multi-unit training program for correctional staff. The particular training program evaluated is a two-week in-service designed to increase the staff members' knowledge and skill levels concerning reality therapy, behavior modification, the use of effec- tive communication techniques, and the use of volunteers in corrections. The staff members enrolled in the training program were working in a newly opened county treatment and detention facility for juvenile delinquents designed to use a combination of reality therapy and behavior mod— ification as its treatment modality. The Problem Criminal activity by youth in contemporary America is regarded as a social problem of immense pro- pmrtions (Lundman & Scarpitti, 1978; Walker, 1978). Imndman and Scarpitti (1978) indicate that the number of arrests of juveniles has doubled in the last 10 years, compared to much smaller increases for those over the age of 18, and that nearly half of the arrests for serious crimes in recent years have been of juveniles. Public frustration over the mounting problem of juvenile delin— quency has been noted by social scientists and, in fact, it has been credited with providing the impetus for the initiation of over 6,500 different programs to deal with the problem since 1965 (Dixon, cited in Lundman & Scar- pitti, 1978). A census conducted in June of 1975 indicated that approximately 47,000 juveniles were being detained in public institutions and 27,000 were being held in pri— vate establishments (U.S. National Criminal Justice Information and Statistics Service, 1977). These facili- ties employed approximately 80,000 individuals on a full— Or part-time basis and expended a total of $867.9 million during that year (U.S.N.C.J.I.S.S., 1978). The return for this staggering investment of human and financial resources in terms of rehabilitation Of delinquent youth has been disappointing, judging by the rel the results of public opinion surveys and studies of rehabilitation outcomes. In 1967, the President's Com- ndssion on Law Enforcement and Administration of Justice issued its report, The Challenge of Crime in a Free Society. In the decade since the 1967 President's report, the single most important change in public attitude toward correctional activity has been the increasing disillusionment with the notion of correctional rehabilitation (Walker, 1978). The pessimism concerning the capability of these institutions to treat delinquent youth successfully would appear to be justified. In a review of 12 studies on the effects of correctional institutional treatment of over 3,000 juvenile offenders, Romig (1978) found almost exclusively negative results. In the search for the cause of the reported failures to rehabilitate juvenile delinquents success- fully in the nation's institutions, at least one social scientist has indicated that a major problem may be the inadequate training provided to the staff members who work with the institutions' residents (Coffey, 1975). The U.S. Civil Service Commission Bureau of Training (1973) reports a variety of problems in correctional training procedures that may be maintaining the quality of staff training at a low level. The report further indicates that correctional administrators are not fully aware of the benefits or importance of training. .1 “mun-A Administrative staff are unable to evaluate the effective- rmms of managers and supervisors who often carry out training and consequently are unable to reward them for quality training programs. If and when it is provided, the training is rarely systematically planned, which results in fragmented presentations most often provided to meet short-term objectives. A major study of the training and educational needs in criminal justice and law enforcement occupations has recently been completed as a part of an extensive federal survey of personnel needs in the criminal justice system (Ewing & Gregg, 1978). This study incorporates data from questionnaire surveys of 8,000 executives of state and local agencies, field visits to over 250 agencies, training or educational facilities, and results of the 1975 Census survey of criminal justice employees among other sources. The study reveals that formal training is not provided at all in a large proportion of juvenile correction agencies and facilities. Even more startling is their finding that training is often not even provided to staff members on an entry—level basis. Qualitative inadequacies noted in existing train— ing programs included too much time in training being spent on procedures and not enough time spent on "human relations" aspects of correctional work, such as how to handle crisis situations. It is a widely held assumption that highly skilled, well-trained staff are essential to the effective treat- ment of delinquents in institutions (Coffey, 1975; Jesness, 1975, 1978; Mukherjee, 1974; Romig, 1978; Trojanowicz, 1973; Wittman, 1965). Jesness (1975) indicates that ratings of perceived staff competency by residents were more predictive of residents' success upon release from the institution than the particular treatment modality provided. Romig (1978) indicates that treatment modali— ties used in residential treatment of delinquency have a better chance of success if they are defined and applied in a very specific manner by a staff well versed in the specific techniques of the modalities. It is also widely assumed that in—service train- ing should be the primary means of developing treat— ment skills (Coffey, 1975; Sandhu, 1977; Mukherjee, 1974; Trojanowicz, 1973; Alston, 1971; Katrin, 1972). In fact, Ewing and Gregg (1978) have recommended that in-service training should be made mandatory for individuals working in correctional facilities. It could be argued that formal education situ- ations should provide this type of training rather than on-site in-services conducted in the institutions. The law Enforcement Assistance Administration (LEAA) has fostered the study of criminal justice, which has resulted in an increase in the number of baccalaureate degree rmograms from 39 in 1967 to 376 in 1977 (Walker,-1978). In fact, however, individuals who work in correctional institutions tend to come from a wide variety of edu- cational backgrounds such as psychology, sociology, and education, among others, rather than exclusively from cmiminal justice programs (Sandhu, 1977). Additionally, Coffey (1975) points out that there are distinctions between training as provided by in-service experiences and education as provided by degree programs. He defines education as being broad in scope and including theoretical principles. Training, on the other hand, is more appropriately restricted to strate— gies and techniques. Coffey further contends that both training and education are necessary and that education should certainly not attempt to fulfill the role of training. The LEAA recognizes this as indicated by their support of training programs and educational curricula (Sandhu, 1977). The National Manpower Survey of the Criminal Justice System (Ewing & Gregg, 1978) has recom— mended that the distinctions between education and train— ing be more clearly emphasized by LEAA in the future. Educational assistance priorities should be less con— cerned with the education of all correctional staff and should instead concentrate on efforts to improve the skills of:managerial personnel as well as upgrading the professional staff's abilities to engage in evaluation, {VT—, __ . . I95 ass (01 tr. tr research, education, and program planning. Training assistance priorities, on the other hand, should be more concerned with the remediation of major deficiencies in training for line correctional staff. This should include not only standard training but enrichment of training as well for line personnel. Specifically, assistance should be provided to "emphasize qualitative improvement in training methods, in training faculty and in scope of training, including improved coverage of 'human relations' subjects" in both entry-level and in— service programs (Ewing & Gregg, 1978, p. 19). The professional community acknowledges the importance of improved on—site in-service training being provided to correctional staff in order for institutional progress to be made in yOuth care and treatment (Coffey, 1975; Sandhu, 1977; et a1.). Given this acknowledgment, the lack of demonstrated effectiveness of in—service training is somewhat surprising. In general, it is not known if in-service training is at all effective in improving the treatment skills of correctional staff, only assumed (Coffey, 1975). Since a lack of evaluation of training programs exists, it is not surprising that the problems of in-service training noted by Coffey (1975), the U.S. Civil Service Commission Bureau of Training (1973), and The National Manpower Survey of the Criminal Justice System (Ewing & Gregg, 1978) have not been COI Sh: th ef llIll IL corrected. A major priority recommendation of the Man- pewer Survey (Ewing & Gregg, 1978) specifies that support should be provided by LEAA for improving training methods through the use of "systematic evaluation of training effectiveness“ (Ewing & Gregg, 1978, p. 19). Evaluation of training programs could serve to: (a) demonstrate which programs are the most effective and (b) aid in their improvement (Anderson et a1., 1975). In this country's youth correctional facilities, rehabilitation professionals have no evidence that the results of in-service training are positive, or even if any results occur at all. The specific aspects of train— ing that may actually be causing positive results remain undetected as do the parameters that may be detracting from the training process (Stake, 1972). Both planned and unplanned effects remain uninvestigated. For example, a.training program presenter may intend that a certain set of concepts be learned by the staff. However, with— out implementing some type of evaluation, the presenter will not know if the concepts have, in fact, been learned or just assumed to have been learned. Information, if available, on the benefits of quality in-service training programs could be used in making decisions intended to improve rehabilitation staff skills. Improvements could be brought about both during planned sessions and during of 10 nonformal training situations (Anderson et a1., 1975; Stake, 1972). The results could help improve youth care and treatment. Purpose The present study is designed to assist in the effort to improve staff skills by contributing to the body of knowledge regarding in—service training for cor— rectional staff. Through development and implementation of an evaluation of an in—service training program for correctional staff, a determination of effectiveness can be made. The particular program chosen for evaluation was a multi-unit training program, funded by LEAA, for staff members of a newly opened residential detention and treat- ment facility for delinquents. The facility is located in southeastern Michigan and provides detention and treatment services for the predominantly rural county it serves. It is licensed to hold 18 individuals in two detention units and 27 indi- viduals in three treatment units. The facility was opened in October of 1977. The primary focus of the training program (which is more fully described in Appendix A and in Chapter III) was on treatment techniques to be used in the facility. The treatment modalities previously chosen by the admin- istrators of the county juvenile court and facility were 11 those of reality therapy and behavior modification. The training program centered on the techniques of those modalities. It also provided information on the use of volunteers in corrections and techniques for the effec- tive use of communication skills. The training program was designed to take place in the course of two weeks. The first week (pre-service) was intended to occur prior to the opening of the facility, and the second week (in- service) was to take place while the facility was in Operation. The plan to evaluate the training program was based on a model provided by Robert Stake (1972, 1975, 1976, 1978). Stake's model for evaluation, which is reviewed in Chapter II, provides for the collection of data both on the products and processes of a training program. Such data hopefully can be used to make improve— ments in future training programs. Several specific issues regarding the outcomes of the training program were investigated by the evalu- ation plan. Knowledge gained by the participants was measured. The participants' attitudes toward using the concepts learned in training were surveyed. A determi- nation was made regarding whether the participants later made use of the training information. Whether the train— ing information improved the participants' abilities was 12 investigated. Additionally, the participants were sur- veyed regarding their attitudes toward the training experience itself. A quasi-experimental design was used to investi- gate the previously mentioned evaluation issues. A com- parison group of individuals not receiving training was obtained to control to a limited extent for the effects of history, maturation, and testing (Campbell & Stanley, 1967). Research hypotheses related to the evaluation issues were established. Instrumentation was designed to gather data reflecting on the research hypotheses. Research Hypotheses Related to the Evaluation Issues The general research hypothesis of this study is that participants in the in-service training program, after their training, will demonstrate improved per- formance on a measure of cognitive knowledge relating to major principles of child management and treatment techniques presented in training; improved attitudes on a measure of attitudes toward use of the child management and treatment techniques prescribed; improved performance on a measure of ability to work with delinquent youth; appropriate application of the child management and treatment techniques via a rating of samples of their work with delinquent youths; and positive attitudes on a survey of attitudes toward the training program itself. F71, Intended Outcome Hypotheses Five directional hypotheses related to the intended outcomes of training are investigated. Hazl The in—service training participant group will demonstrate increased knowledge of the major principles presented in the in—service training program by receiving higher post—training scores on the Cognitive Areas Objective Test (see Appen— dix B) than a comparison group that has not par— ! ticipated in the in—service training. Ha:2 The in—service training participant group will demonstrate improved attitudes toward using the child management and treatment techniques presented in training by receiving higher post-training scores on the Attitudes Toward Use Survey (see Appendix C) than a comparison group that has not participated in the in-service training. Haz3 The in—service training participant group will demonstrate improved abilities to work with delin- quent youth by receiving higher post—training scores on the Critical Incidents Test (see Appendix D) than a comparison group that has not received training and by indicating self-reported improvement in abil— ities in their responses to the Follow—up Question— naire (see Appendix E). Ha:4 The in-service training participant group will demonstrate appropriate application of the child management and treatment techniques presented in training by receiving mean ratings in the apprOpriate range on the Staff Observation Rating Scale (see Appendices F and G) of samples of their work after training and by indicating via self-report on the Follow-up Questionnaire that they believe they have appropriately applied the techniques in their work following training. I la. 14 Haz5 The in—service training participant group will demonstrate positive attitudes about the training experience itself by their responses to the Satis- faction with Training Survey (see Appendix H) immediately after training and their responses to the Follow—up Questionnaire six months after training. Additional Unintended Outcome Hypothesis In addition to the previous five directional hypotheses which are directly related to the intended outcomes of training, an hypothesis relating to an addi— tional unintended outcome was established in order to investigate the effects of training on staff attitudes toward types of delinquents. Additional Hazl The in—service training participant group will demonstrate shifts in attitudinal preference regarding the type of delinquent youth with which they enjoy working, compared to a comparison group that has not participated in the in-service training, as indicated by post—training scores on the Correctional Preference Survey (see Appendix I). Overview In Chapter II, a comprehensive review of the literature relating to the field of evaluation of cor— rectional staff training is presented. In Chapter III, the evaluation methodology used in this study is described, including description of the participants, description of the setting for training, description of the eva the (he da‘ 15 the training program itself, methods of observation, evaluation procedure for data collection, design of the evaluation, and statistical methods of data analysis. Chapter IV contains the results of the analysis of the data. Chapter V includes conclusions based on the results of the analysis, implications of the results for the field of in-service training for correctional staff, and a summary of the study. CHAPTER II REVIEW OF THE LITERATURE Introduction to the Review Review of the literature relating to this study necessarily includes several topics covered in separate sections. The first section in the review of the litera— ture covers the tOpic of delinquency. Within that topic are several sub-topics. Included are the description of delinquency, its prevention, and its treatment. The section on delinquency concludes with a description of the delinquency treatment modality used by the facility which is the site of the current study. The second section reviews literature on the importance of staff training for effective delinquency treatment programs. The third section reviews literature on evaluation in juvenile corrections. Included are evaluation trends in the area of delinquency in general and in the specific area of staff training evaluation, in particular. The fourth section is a review of the literature on the evaluation model chosen for the current study. 16 Inc Rol CI 17 Included are descriptions of the model by its developer, Robert Stake, and others. Delinquency Delinquency can be defined in legal or social terms (Sandhu, 1977; Trojanowicz, 1973). Any act com— mitted by a person who is under the age of majority (and, therefore, a minor) that is against a statute is considered to be a delinquent act in the eyes of the appropriate legal system (Sandhu, 1977). Legal systems in America exist within a variety of geographical juris— dictions. According to Sandhu (1977) each of these jurisdictions, whether federal, state, or municipal, has its own definition of delinquency via its statutes. Many differences exist among these jurisdictions in how they deal with youths who violate the law and even in what behavior is to be considered delinquent. Social definitions of delinquency encompass an even broader range of behavior. Youths who have never come into contact with the police courts or social agencies may be labeled delinquent. These labels come from members Of the community who base their judgment on socio— economic status, appearance, or merely being "different" (Trojanowicz, 1973)- The difficulty is in operationalizing a defi- nition on who is labeled delinquent. Given this d1ff1— . . ' the CUlty, it is not too surprising that research into can is he .r___________________________________L 18 causation, prevention, and remediation of delinquency is exceptionally problematic. Some of the frustrations faced by researchers in delinquency are indicated by Wheeler et a1. (1967): Do we know enough about delinquency to specify the ways in which even a moderate reduction could be brought about? In terms of verified knowledge, the answer must be an unqualified no. There is, of course, a vast body of literature reporting numerous research findings, and suggesting a variety of plausible theories of delinquency causation and control. But when experiments have been conducted, the results have not been encour— aging. Indeed, as of now, there are no demon— strable and proven methods for reducing the inci— dence of serious delinquent acts through preventive or rehabilitative procedures. Either the descrip— tive knowledge has not been translated into feasible action programs, or the programs have not been successfully implemented; or if implemented, they have lacked evaluation; or if evaluated, the results have usually been negative; and in the few cases of reported positive results, replications have been lacking. (p. 409) Research in the field of delinquency has been con— centrated in several topical areas. Those areas are the description of delinquency including its causation and prediction, delinquency prevention, and the treatment of delinquency. The major findings in these areas are reviewed below in order to provide the reader with a geheral background in delinquency research. This back— ground will assist the reader in identifying the context surrounding the present study. D_e_l Description of Delinquency Several personality variables have been related to delinquency. Delinquents have been found to be more external in their locus of control than nondelinquent youth (Stauber, 1973; Howard, 1973). The individual's locus of control relates to whether reinforcement is Viewed as occurring as a result of one's actions (internal locus of control) or as a result of fate, luck, or other uncontrollable circumstances (external locus of control) (Rotter, 1954). They are also more likely to have a lower self—concept (Scarpitti et a1., 1962). A number of researchers have created personality typologies to describe delinquent individuals (Coffey, 1975). These typologies have been used in a variety of treatment situations and, according to Coffey (1975), considerable time and effort have been devoted to them. Among the typology systems is one created by Quay and Peterson (1964). This system is of particular interest to the present study because of its application in match- ing types of delinquents with types of staff members in residential treatment. Using a 55—item Behavior Problem Checklist, a 36—item Checklist for Analysis of Life History Data, and a 100—item Personal Opinion Study, they describe four behavior category types of delinquents: BC-l (Inadequate—Immature), BC—2 (Neurotic~Disturbed), BC-3 (Unsocialized-Psychopathic), and BC—4 20 (Socialized—Subcultural). Attempts to validate this typology both by using delinquent and nondelinquent comparison groups and by using concurrent measures within delinquent groups have met with moderate success (Ingram, 1970; Petersen & Thomas, 1975). A variety of causative factors have been proposed by researchers and theorists concerning the etiology of delinquency. Perhaps the most widely held belief regard— ing delinquency is that socio—economic factors are at its roots (Sandhu, 1977). This is the notion that "poverty breeds delinquency," whether by causing a conflict between lower and middle class values or by creating a situation where youth learn crime from older criminals in their environment (Trojanowicz, 1973). That socio— economic status is a primary causative factor is disputed by Trojanowicz (1973). He indicates that while lower class delinquents are quite likely to be dealt with by the courts, many delinquents come from middle and upper class backgrounds. The latter are often dealt with by private therapists and boarding schools, thereby escaping public notice. Economic considerations are seen by Tro- janowicz as only one of many contributing and descriptive factors of delinquency. Lundman and Scarpitti (1978) lend support to Trojanowicz' de-emphasis of socio- economic variables as causes of delinquency. They indicate that demographic variables relating to 21 delinquency have expanded to include a large number of middle class youth in recent years. Another possible correlate of delinquency was proposed by Glueck and Glueck (1968). They found that slightly over 60% of delinquents had a muscular body build (mesomorph) compared to about 30% for nondelin— quents. Somewhat surprisingly, this finding was provided with additional support as late as six years ago when Cortes and Gatti (1972) found three times the number of mesomorphs in a group of delinquents compared to a non- delinquent group. While these researchers hesitate to draw any causal connection between body build and delin— quency, they believe that an interaction of the two, through the environment, certainly may exist (Glueck & ‘ Glueck, 1968; Cortes & Gatti, 1972). Psychic trauma in childhood has been proposed as a causative factor by neo—Freudians Redl and Wineman (1951). Their study of institutionalized inner—city delinquents revealed that these individuals had exper- ienced traumatic, loveless early childhoods. The two authors placed these facts into a psychodynamic framework indicating the delinquents had developed deficient egos. Unfortunately, this theoretical notion has been tested by little sound research (Sandhu, 1977). Considerable investigation of family interaction styles as possible causes of delinquency has been undert that c where more ‘ that has rese 197! hue tha his PT th rr—fé undertaken. As early as 1963, Barker and Adams indicated that often delinquents come from multi—problem situations where there is much family disorganization. The parents of delinquents are likely to use physical punishment more than verbal discussion. Further it is reported that mothers are likely to be more permissive than the fathers, and in fact both parents are less consistent in their discipline measures in general than are parents g of nondelinquents. Parents of delinquents are also less likely to demonstrate warmth and affection in their homes (Glueck & Glueck, 1968). The impact of the "broken home" on delinquency has received considerable attention by delinquency researchers (McCord & McCord, 1959; Cavan & Ferdinand, 1975; et al.). A number of studies indicate that delin- quent children are more likely to grow up in broken homes than are nondelinquent children (the President's Com— mission on Law Enforcement and Administration of Justice, 1967). Recent studies, however, have indicated that the impact of divorce, separation, desertion, or death of the parents depends, to a large extent, on the attitude of the adults concerned (Cavan & Ferdinand, 1975). Another aspect of descriptive research on delin— quency has been the subject of prediction. Supporters Of prediction research indicate that if methods can be found to determine in advance which youth in society wil vi: 23 will commit delinquent acts, then treatment can be pro- vided them at an early age (California Youth Authority, 1978). One example of a means of prediction has been provided by the work of Reckless and Scarpitti (Reckless, 1967; Scarpitti et a1., 1962). They administered a number of self-concept measures to lZ—year—old boys. Later they were able to determine that the boys with negative self— concepts had gotten involved in delinquency at a much higher rate than the boys with positive self—images. The self—concept measures could then be said to contribute to a prediction of delinquency. Most written tests, how— ever, have proven to be unreliable for prediction. The judgment of teachers, upon observing their pupils' behavior, has proven to be more reliable (California Youth Authority, 1978). Prevention of Delinquency Research on the prediction of delinquency has been applied to delinquency prevention. Attempts are made to help identified potential delinquents by finding them jobs, providing recreational activities, camping programs, reading tutoring programs, group therapy, and many other programs. There have been, in fact, 6,500 attempts at delinquent prevention programs since 1965 according to Lundman and Scarpitti (1978). Unfortu— nately, Lundman and Scarpitti (1978) also indicate in a review of nearly 1,000 citations in the literature on Te——————————————————————————————*””* T“'-Il 24 delinquency prevention that they were able to find few, if any, efforts that demonstrated success in preventing delinquency. The difficulty of the prevention programs in demonstrating any success is shared by programs of delinquency treatment (Wright & Dixon, 1977). They examined over 6,000 abstracts of both prevention and treatment programs, not including institutional programs, and found that only three of them employed an adequate research design and reported positive results. Treatment of Delinquency Correctional treatment has been defined as "any activity other than investigation, surveillance, or supervision“ in which the correctional agent engages with the client "for the sole purpose of rehabilitation“ (Coffey, 1975, p. 191). These activities have taken a wide variety of forms in the field of corrections (Romig, 1978; Trojanowicz, 1973; Coffey, 1975). Cor— rectional treatment of delinquent youth can generally be divided into two major categories: community—based and institutional. Strong controversy exists in the field of cor— rections today over the relative merits of community— based versus institutional treatment (Jesness, 1978; Newcomb, 1978; Erickson, 1978; Romig, 1978; et a1.). Institutions have been called "schools for crime," ind Del di: F—T—T——h'——T 25 indicating that they do more harm than good as a result of negative peer influence (Newcomb, 1978; Sandhu, 1977). Defenders of institutional treatment for delinquents disagree, indicating that the institutions should be improved, not abolished (Erickson, 1978), and that addi- tional research efforts are needed toward those ends (Jesness, 1978). Several states in the United States have attempted to "de-institutionalize" their delinquent populations (Romig, 1978), particularly those delinquents who have been arrested only for "juvenile" crimes such as runaway and truancy (Little, 1978). The results have not been particularly successful, at least in the experience of Massachusetts (Romig, 1978). Increases in the runaway and death rates of delinquent youths occurred as did increased rates of recidivism (repeat of offenses). The controversy continues, however, as both community-based and institutional programs attempt to provide indications of successful treatment of delinquency (Jesness, 1978). The comparisons are as follows: Community-based treatment.-—The vast majority of delinquents, several hundred thousand new juveniles a year, involved in community—based programs receive juvenile probation services (Sandhu, 1977). Probation provides casework services for youth on a trial basis while they remain in the community, living at home, or 26 in a foster home, group home, etc. (Trojanowicz, 1973; Sandhu, 1977). Romig (1978) has reviewed studies empha- sizing casework aspects of probation and has concluded that it is often unsuccessful. Two major programs of delinquency treatment that have developed as off—shoots of the probation concept are divergence programs and intensive probation treatment programs. Divergence programs are designed to "divert" youths arrested for the first time, away from continued contact with the court system (Bullington et a1., 1978; Sandhu, 1977; Bohnstedt, 1978; Romig, 1978). Often volunteers or student interns are recruited to work with the youths (Schwartz et a1., 1978). The main purpose of diversion is to find the most humane treatment of some juveniles who would not have been helped, and possibly even harmed, by the court system (Bullington et a1., 1978). Youth service bureaus are seen by some as a major source of diversion of youth from the court system to social service agencies (Sandhu, 1977). Diversion programs have recently come under considerable criticism (Bohnstedt, 1978; Bullington et a1., 1978; Romig, 1978). Diversion programs have been called ineffective at reduc— ing recidivism (Romig, 1978; Bohnstedt, 1978), inefficient in terms of money saved the court system (Bohnstedt, 1978), and in fact often diverting primarily youths who would not have been processed further in the court system anyl des pro nev III to CE CE r———___ 27 anyway (Bohnstedt, 1978). The programs have even been described as being in conflict with the ideals of due process since programs are set up for youths who have never been adjudicated (legally declared to be) delin- quent (Bullington et a1., 1978). In spite of the cri- ticisms leveled against them, diversion programs continue to be pOpular alternatives to the standard court pro- cedures (Sandhu, 1977). Intensive probation treatment programs, also called probation subsidy programs, permit selected case- workers to have small limited-sized caseloads, and con- sequently have more time available for each delinquent (Romig, 1978). In some intensive probation treatment programs, such as the Institutional Alternatives Program in Ingham County, Michigan, only delinquents who might otherwise have been committed to the care of institutions are placed into the programs (Shelley, 1977). Other programs have not required such restrictions (Romig, 1978). Romig (1978) indicates a lack of demonstrable success in two major studies of the intensive probation system in California. A number of other approaches to the remediation of delinquent behavior have been attempted, according to Romig (1978). Studies of vocational programs designed to assist youth in finding jobs, gaining vocational training, or engaging in work programs have generally not and to l (to the but 28 not shown reductions in delinquent behavior as results and neither have therapeutic camping programs designed to enhance the self-images of delinquents (Romig, 1978). Group, family, and individual counseling and psycho- therapy have been attempted with juveniles with mixed but generally negative results (Coffey, 1975; Romig, 1978; Sandhu, 1977). A brief description of community residential treatment programs provides a transition into the dis- cussion of institutional treatment and serves to help distinguish between the two. Community residential pro- grams use facilities located within neighborhoods and attempt to make use of community resources, such as public schools (Commission on Accreditation for Cor- rections, 1978). Typical community residential programs are group homes, halfway houses, and foster homes (Romig, 1978). These programs, according to Romig (1978), have evidenced negative results when studied. Institutional-based treatment.—-Institutional treatment facilities require removal of the youth from the community rather than the integration of residents into community activities of community residential facili— ties. The following figures obtained in a census of juvenile detention and correctional facilities provide an idea of the relative use of such facilities (U.S.N.C.J.I.S.S., 1978). In 1975, 47,000 juveniles her thi hoe suc of di 29 were being held in public juvenile institutions. Of this number, 57% were in training schools and 24% were housed in detention centers. Both of these are considered institutional settings. Community residential settings, such as group homes, accounted for only approximately 6% of the total pOpulation. Ranches and facilities such as diagnostic centers make up the remainder of the total. Obviously, institutional treatment plays an exceptionally important role in the correctional treatment of delinquent youth in America. A variety of treatment modes have been used by staff in institutional situations throughout the years. One such mode is called "character training." The British Borstal training system uses this approach, which allows youth to choose "right over wrong" and then be rewarded for making the correct choices (Grant, 1965). Use of the dynamics of interpersonal relationships as a means of bringing about behavior change has been attempted by the Highfields Project, which originally utilized the former home of Charles and Anne Lindbergh. Instead of emphasizing the reward of good behavior, as in the Borstal system, the bad behavior was allowed to be expressed and then be "worked through" (Weeks, 1963). Considerable decision—making power was given to the inmate members of the group in the Highfields Project, as well as in the similar Provo EXperiment in Utah (Ra re] ev< th he CI 30 (Rabow, 1961). As a result of this emphasis on peer relationships, a "guided group interaction" concept has evolved. This approach makes use of the strong influence that adolescents exert on one another (Phelps, 1976). According to Phelps, the guided group interaction approach utilizes principles of William Glasser's reality therapy, Eric Berne's transactional analysis, and behavior modifi- cation. It has been further delineated by Trieschman et a1. (1969) and entitled Positive Peer Culture (PPC). The peer group is given even more responsibility in defining and attempting to alleviate the problems of its members. This approach is gaining increasing acceptance and is currently being used in the state of Michigan at institutions such as Starr Commonwealth for Boys. Behavior modification programs have been imple- mented in many treatment facilities in recent years, with a great deal of positive expectation (Ross & McKay, 1976). More than likely, this is partially a result of the enthusiasm with which many have embraced the work of learning theorists, such as B. F. Skinner (Bandura, 1969; Krumboltz & Krumboltz, 1972; Morris; 1976; et a1.). There are several basic assumptions concerning behavior modification as a treatment, according to Morris (1976). First, it is thought by behavior modifiers that all behavior problems have been learned. Second, it is assumed that each behavior problem can be considered sepa1 ment that the ment ment mod: Four and pra tre shc pa: all? US r7— separately from other problems for the purpose of treat— ment. Third, advocates of behavior modification believe that behavior problems can be modified and remedied by the prOper use and application of differential reinforce— ment. In other words, the appropriate types of reinforce— ment presented at the right time and place can serve to modify the behavior of the delinquent youth for the better. Fourth, the goals of behavior modification are specific and centered in the present. Behavior modification practices dictate that problems and procedures for their treatment be relatively narrow and well defined. They should also concentrate on the here and now, rather than past events or difficulties. The behavior modification ; approach also has been adopted by modeling groups which use videotaped feedback in treatment of delinquents (Sarason & Ganzer, 1971). According to Ross and McKay (1976), institutional behavior modification programs have little demonstrated evidence of success. Romig (1978) supports this con- tention in a review of 14 studies involving behavior modification programs with some 2,000 delinquents. Romig (1978) indicates that behavior modification tends to work to change only very specific behaviors, such as school attendance, but does not affect delinquency recidivism rates. The conclusion is that behavior modification should not be used as the solitary treatment modality in sin ste In th- be F_f———————fi I‘- 32 in a situation, but only for correction of specific and simple behaviors. Phelps (1976), on the other hand, states that one of the positive aspects of behavior modification programs has been that they have helped the staff members of institutions recognize positive behavior rather than focusing exclusively on infractions. Glasser's reality therapy techniques (1964; Glasser & Zunin, 1973) developed in the Ventura School for Girls, a correctional facility, have been gaining increased acceptance in the field of corrections (Sandhu, 1977). Glasser's techniques are regarded by many indi- viduals involved in corrections as being a logical common sense approach quite useful with delinquents (Trojanowicz, 1973). The reality therapy approach is described by means of eight principles from Glasser and Zunin (1973): Principle I: personal. The staff members should communicate that they care about the youths in their facility. Warmth, understanding, and concern are neces- sary for effective treatment. Principle II: focus on behavior rather than feelings. While not denying feelings or indicating that they are unimportant, staff members concentrate on the "doing" aspects of the youths' lives. Glasser postulates that peOple have more control over what they do than over how they feel. He also believes that constructive "doing" more positively affects feeling than vice versa. 33 Principle III: focus on the_present. Whenever the past is discussed, the past events should be related to current behavior and constructive alternatives that might have been taken at the time should be explored. A positive approach can be taken by staff members in that the focus should be on why the youths have not gotten into more trouble! Principle IV: value judgment. The staff members must help youths clarify their own behavior and then make value judgments as to whether that behavior is responsible. Responsible behavior is that which helps the person meet his own needs without hurting others. Principle V: planning. The youths are aided by the staff members in formulating realistic plans to change failure behavior into behavior that will help them succeed. The staff must be expert in recognizing degrees of moti- vation and ability in order to help youths set goals that they can be reasonably expected to attain. Principle VI: commitment. After youths have made a value judgment about their behavior and created plans to change a portion of it, the next step for the staff person is to assist the youths in making commit— ments to carry out those plans. In the early stages of treatment, youths' agreements to follow the plans only to please the staff persons are honored. why re t0 34 Principle VII: no excuses. Rather than go into why youths fail to follow a commitment, the staff person focuses on helping them create new plans, or modify or recommit to the old ones. To belabor why youths failed to keep commitments is to reinforce failure identities, according to Glasser. Principle VIII: eliminate punishment. While agreed-upon contingencies are honored, punishment of youths by staff members for failure to succeed in plans is ineffectual and detrimental to youths with already- formed failure identities. Treatment of delinquency at the Monroe County Youth Center. The administrators of the Monroe County Youth Center facility, the site of the current study, have chosen to use a combination of two treatment modali~ ties. The overall framework of the program is provided by implementation of Glasser's eight principles of reality therapy (Glasser & Zunin, 1973). These principles are integrated into a behavior modification token-economy system. Both reality therapy and behavior modification techniques emphasize the present and rely on identifying and remediating problematic behavior, rather than con- centrating on traumatic past events or underlying emotional disturbances. Each youth committed to treatment by a judge must earn his way through three separate living units (A I hoe ent eit nu as 35 (A, B, and C) to that of graduate or release status. Movement through those units is accomplished both by engaging successfully in activities described by Glasser's eight principles and by successfully earning a sufficient number of behavior modification token-economy points assigned by staff to each youth for acceptable behavior. Resident youths must determine, with the help of staff members, which of their present behaviors is causing them to not be able to meet their needs successfully without harming others. They must make value judgments regarding the irresponsibility of that behavior and commit themselves to specific plans of changing their behavior appropriately. Each of these activities is periodically viewed by the Unit Change Committee consist— ing of two staff persons and judged as to its degree of accomplishment. The development of the above procedure was based on Glasser's eight principles as interpreted by the Center's administrators. Points are assigned each day to the accounts of resident youths on the basis of their performance of acceptable behavior. The accumulated points serve several functions. They operate as reinforcers in the facility's token economy system, being used as units of exchange in the facility's "store." They also serve to provide a source of feedback several times each day as to the degree of acceptability of the youth's behavior. "Fit tut una cor The at be 36 "Fines" involving the subtracting of points are insti- tuted as a result of the occurrence of particularly unacceptable behavior. Behavior point totals are also considered whenever unit changes are being contemplated. The design of the token-economy system is based on the administrators' interpretation of the principles of behavior modification. An integral aspect of the Monroe County Youth Center's treatment approach is the use of three separate living units (A, B, and C). The purpose of having three units is to provide differential living situations for youths, depending upon their progress in the program. These differential situations are designed to provide additional feedback to residents regarding behavior and to provide built-in reinforcement for acceptable behavior. They are also intended to form groups of individuals who are roughly equivalent in ability to accept responsi— bility, level of maturity, etc. Unit A provides the greatest degree of structure and fewest privileges. Unit C provides the least structure and greatest number of privileges. Unit C youths, for example, are permitted to obtain part—time jobs outside the facility. Each unit is intended to serve as a reminder to the youth of his level of progress in the program. The staff members of the Monroe County Youth Center are expected to be aware of and appropriately use mod Gla tot use cee ot' of ce te 'I'T—__——7 is use the techniques of the institution's chosen treatment modalities. In addition to correct implementation of Glasser's eight principles and the proper use of the token-economy system, members are expected to properly use positive and negative reinforcement, extinction pro- cedures, and cognitive dissonance techniques, among others. Given the lack of demonstrated effectiveness of in-service training in corrections, however, it is certainly open to question whether this type of expec— tation is realistic. The Importance of Staff Training for Effective Treatment In order for staff members of a correctional facility to implement the agreed—upon treatment modali— ties effectively and help bring about improvement in their residents, they must be well versed and sufficiently skilled in the appropriate techniques (Romig, 1978; Jesness, 1978; Coffey, 1975; Petersen & Thomas, 1975; et a1.). Jesness (1975, 1978) found staff competency ratings to be better predictors of resident delinquents' success after institutionalization than were the types of treatment modalities used. Romig (1978) has pointed out that the greater the specificity of the treatment approach, the greater will be the chances of successful treatment of delinquents. Both increased competency and increased specificity of the use of techniques are assumed results of quality staff training (Coffey, 1975). ‘ _.i 38 That worthwhile staff training is of vital importance in successful residential treatment is widely accepted and, in fact, has been for quite some time. In Institutional Rehabilitation of Delinquent Youth-—Manual for Training School Personnel, the National Conference of Superintendents of Training Schools and Reformatories (1962) quite forcefully made the point that no institution can do a professional job if staff training is undertaken on a hit—or—miss basis. In fact, in—service training is considered too great a job for the superintendent of an institution or any one other person to direct alone. The conferees advised the appointment of a committee on in-service training in each institution, with repre— sentatives of all aspects of residential treatment being present. The purpose of these committees would be to maintain the necessary momentum in carrying out the time- consuming task of bringing together the appropriate resources needed to provide the staff with worthwhile training. In Readings on the Administration_pf Institutions for Delinquent Youth, Wittman (1965) provided an outline chapter which speaks to the issue of training and is quite appropriately entitled "Training: Key to Insti- tutional Improvement." Wittman emphasized that a staff of an institution must be provided with good training in order to grow as a treatment team and to avoid working at cross—purposes. (Wht son oft in spe 39 The National Council of Crime and Delinquency (Wheeler et a1., 1967) suggested that training of per— sonnel is such a complex issue that local communities often are unable to deal with it effectively. Its importance is underscored by the Council's urgent and specific recommendation that federal support be provided for the development of pre— and post—employment training for state and local correctional staffs. Recognition of the importance of training to residential treatment is by no means peculiar to Western culture. Mukherjee (1974), in a comparative study of the administration of juvenile correctional institutions in Delhi and Maharashtra, India, points out six reasons why effective in-service training is necessary: 1. It promotes the interest of employees in their jobs, 2. provides them with better methods of per— forming their duties, 3. prepares them to meet difficult situations, 4. standardizes procedures, 5. helps develop a band of professional per— sonnel, and 6. helps offer the best practices in adminis— tration, personnel programs, and job satisfaction. Recent support for the importance of quality staff training of correctional staff in America has been voiced fir as well (Sandhu, 1977; Petersen & Thomas, 1975; Coffey, 1975) and is described here. Petersen and Thomas (1975) indicate that when much of the staff is new and the treatment program is complex as is the case in many cor- rectional institutions, in—service training is particu- larly vital to the implementation of successful treatment techniques. Sandhu (1977) points out that one of the key aspects of the 1974 Juvenile Justice and Delinquency Prevention Act was its provision for increased funding for the training of both public and private correctional employees. The National Manpower Survey of the Criminal Justice System (Ewing & Gregg, 1978) forcefully points out the importance of quality training of correctional staff by recommending that entry—level training be made mandatory for correctional staff members and that sig— nificant support should be provided by LEAA for the improvement of both entry-level and in-service training programs. Coffey (1975) indicates that the most significant training responsibility of institutional administrators is in the area of treatment techniques. The simple, compelling reason for this is that proper implementation ! of treatment techniques will determine, in large measure, whether or not successful results are attained by the institution. Coffey believes that the quality of the ser’ tra det st SE 41 training the staff receives is the single most important determining factor in the quality of treatment provided. Evaluation of Staff Training In spite of the agreement that the quality of staff training in residential treatment situations is of such importance, surprisingly little evaluation work has been done with regard to it. The importance of evaluation to the improvement of staff training is underscored by Ewing and Gregg (1978) in their Executive Summary to the National Manpower Survey of the Criminal Justice System. Systematic evaluation of in—service training is listed as one of the conditions necessary to the upgrading of staff training. Instead of concentrating on aspects of staff training, early evaluation endeavors tended to be centered on global investigation of the effectiveness of single residential treatment programs (Weeks, 1963; Rabow, 1961; Grant, 1965). In 1965, Grant indicated that a commendable trend had begun to emerge for institutional treatment agencies to avoid promoting Epg treatment for delinquency and instead to encourage innovation and evaluation of their own programs. Prior to that time, despite various claims of success, no one approach had proven itself consistently to be more effective than the others. Conse littl work inno‘ rece is s eftc ment C0 tr da de h 42 Consequently, the evaluations that were done provided little assistance for other institutions attempting to work with delinquent youth. Despite what appeared to be a trend in 1965 toward innovation and institutional self—evaluation, Jesness recently (1978) points out that most evaluation research is still global in nature. In addition, these evaluation efforts have done little to advance correctional treat- ment. He concludes, It is currently not in the zeitgeist to believe that rehabilitation in correctional programs (especially institutions) can be beneficial. The reality is that most programs probably have no effect on some clients, beneficial effects on a few, and positively harmful effects on others. The fact that we cannot clearly identify which programs work with which youths speaks to the primitive state of our knowledge of prevention, deterrence, and correction. (p. 29) Coffey (1975) indicates that while he believes quality training to be of exceptional importance, he has no data upon which to base a recommendation of how to devise training programs. A few other dimensions of residential treatment have been studied. Jesness (1975) investigated the effect of varying the number of delinquents per unit ‘ in residential treatment. He found that delinquents in moderately sized units seemed to improve faster than those in large—sized units. Other investigators have looked into the effects of variation in treatment approach within and between institutions (Ross & McKay, 1976; Davidse (i974) succes staff valuai compai and e train and t trair quent 0the posi ing subj meet the out 6V6 1mg fi—_ 43 Davidson & Seidman, 1974; Slaiken, 1973; et a1). Glaser (1974) described studies whose results indicated that successful outcomes related positively to low rates of staff turnover and longer periods of treatment. While valuable, these evaluations are still relatively global compared to defining individual staff competency needs and evaluating training attempts to meet those needs. The accomplishment of evaluations of in—service training has lagged behind the assumption that specific and thorough evaluation is needed not only in staff training for work in residential treatment of delin- quents, but in other treatment—oriented fields as well. Other fields have found themselves in the difficult position of needing to know more about in-service train- ing but having little evaluation data available on the subject. For example, George (1974) has pointed out the need for evaluations of in—service training efforts in the field of counseling both juveniles and adults on an out—patient basis. George believes that the effects of in—service training are generally positive but believes evaluations of it could help to aid in training program improvement. Danish and his associates at Pennsylvania State University (Danish et a1., 1978) have developed a train— ing model for paraprofessionals in the field of mental health services consisting of 25 hours of interviewing and In gs} th at pi 'FT—————— 44 and counseling skills training over a period of 10 weeks. In a recent presentation in the journal, Professional Psychology, that was based on a symposium presented at the annual meeting of the American Psychological Associ— ation in 1975, Danish (Danish et a1., 1978) describes the state of evaluations in the area of training for paraprofessionals in the field of mental health services: Although the need for, and the value of para— professionals is well established, the relevance of training for this personnel has not yet been determined. The first question in the determi- nation of the relevance of training is whether it is needed. Members of this symposium have con— cluded elsewhere that training is necessary. However, a more basic question remains: What should the nature of the training be and how should the training be delivered? (p. 16) In order to begin answering these important questions, Danish et al. have examined the paraprofes— sionals' verbal behavior during and after their own training program (which is a college—level course) and have concluded that the paraprofessionals in training do learn the skills (Danish & D'Augelli, 1976; Danish, D'Augelli, & Brock, 1976) and have not lost them entirely after a seven-month period (McCarthy, Danish, & D'Augelli, 1977). While these results may seem modest in scope, they represent a significant contribution to the field, given the lack of other available evaluation data. Gordon (1976) provides several models for train- ing paraprofessionals in federal probation work but is unable to point to any completed evaluation of in-service or 0V1 bl tr VE 45 or pre-service training. In the field of social work, over a decade ago, the specific recommendation was made by the Arden House Conference concerning manpower and training in corrections that "research and evaluation of various approaches to field work training" should be undertaken (Meeker, 1966, p. 140). Perhaps the dilemma of not having sufficient evaluations of in-service training available is best summed up by Jorgensen and Scheier (1973). Regarding the effects of training on court volunteers, they indi- cate: The foundation assumption of this book has been that the trained court volunteer is far more effective than the untrained one. Yet, it must be admitted we have no convincing proof of this as yet in research which contrasts the actual performance of the trained versus the untrained volunteer and beyond that, compares the relative effectiveness of different types of training. (p. 333) Concerning the evaluation of in—service training for rehabilitation personnel in residential facilities, only one topical area has received significant attention to date. The effects of facilitation training on cor- rectional staff have been explored by several experi- menters. Alston (1971) found that: 1. An in—service facilitation program improved both professional and subprofessional personnel's ability to communicate facilitative conditions. l.— otl 46 2. The perceptions of the subjects of each other significantly improved during training. 3. The attitudes of the represented vocational groups toward each other did not change. 4. The greatest improvement of ability to com- municate facilitative conditions occurred during the first half of the training. 5. More defensive individuals exhibited lower levels of ability to communicate after training. Katrin (1972) studied the effects of a 40—hour training program designed to increase 14 female cor— rectional officers' ability to be supportive and helpful with inmates. Training of just one—third of the staff resulted in resident responses indicating some increased confusion and disorderliness in the institution. In Katrin's opinion, this can be expected in the short run from any program change until it has had a chance to "settle in." Quay (1977) is presently in the process of dev— eloping one of only two other attempts, known to this writer, to evaluate other types of training for cor— rectional staff. Quay provides correctional staff with two—week seminars funded by LEAA in behavior modification, transactional analysis, and facilitation training. He is attempting to determine the effects of the training by means of pre— and posttesting of a cognitive nature. By exam hporta modalit have i] being this, of the intent heasu (1975 uatic edge delh ihdi is k drel jec res rec f0] 19 47 By examining scores on tests that include items on the important concepts and techniques of the treatment modalities, Quay hopes to demonstrate that his seminars have improved levels of staff knowledge. Plans are also being discussed for examining "program integrity." By this, Quay means the degree to which the actual operation of the correctional institution later matches its stated intent to use certain treatment modalities. This is a measure of transfer of training which Anderson et a1. (1975) indicate is an important aspect of training eval- uation. In other words, do the trainees take the knowl— edge learned in training and apply it to their work with delinquents? The other known attempt to evaluate training for individuals working with youths in residential settings is being undertaken by the Michigan Association of Chil- dren's Agencies' In-Service Training Project. This pro- ject provides training for workers in private and public residential facilities, a minority of whom work in cor- rections, during a two-year course. The course lasts for four semesters of 2% hours a week for a total of 70 hours (Michigan Association of Children's Agencies, 1977). The project, which is partially funded by LEAA, emphasizes developmental psychology. Treatment modalities, such as transactional analysis, are discussed primarily in relation to how they relate to developmental psychology. in prog Directo trainee have be assessr items' are be inclue rated have Train Progr gaine of t tune pert COhe hoe Pl; ad ti 48 Evaluation of the training program is presently in progress, according to Mr. Rich Liberatore, ACSW, Director (Liberatore, 1978a). Supervisor ratings of trainee performance in work settings after training have been attempted. However, the project's current assessment of the observation instrument is that some items' definitions are overlapping and the categories are behaviorally unclear. Future refinements of the instrument will likely include more complete definitions of the categories to be rated. Surveys of trainees by means of questionnaires have provided a pool of data presently being analyzed. Trainees have been surveyed at the end of the training program as to their perception of their own knowledge gained during training, their perception of the relevance of training, whether the training has affected their functioning in their work, their perception of instructor performance, and their perception of the degree of group cohesion during training. Stake's Evaluation Model Ipproduction to the Model Robert Stake's (1972, 1975, 1976, 1978) evaluation model is used as a conceptual guide for the evaluation plan of this study for several reasons. It is flexible, adaptable to unique learning situations, and is rela— tively easily understood by readers of the evaluation report cation nether ceptut ing, a log evalr Tran exam and on t ind the rel the 49 report (Taylor, 1976). The model assists in the identifi— cation of relevant training components by providing a method of classifying them. It also requires a con- ceptual sequencing of conditions existing prior to train— ing, training events, and training outcomes. This lends a logical order to the evaluation process. Stake's model has been called both "transactional evaluation" and "responsive evaluation" (Taylor, 1976). Transactional evaluation approaches tend to focus on examination of the processes and interactions of a program and the uniqueness of the educational setting rather than on the extensive use of analytic techniques. As Taylor indicates, the approach is particularly responsive to the needs of programs in that its flexibility allows for relatively complete evaluations of complex programs in their developmental stages. The lack of available evaluation data on the process and effects of in—service training for correc- tional staff members, in effect, places each such program into the position of being considered a new beginning. Consequently, a flexible, adaptable approach, such as that provided by Stake, is well suited to initiate the evaluation of a program that takes place in a unique educational setting. In addition, the conceptual sequencing of conditions existing prior to training, teaching events, and training outcomes, as proposed by Sta eva par bee Ew 50 Stake, lends order and organization to the program it is evaluating, as well as the evaluation itself. This is particularly important in an educational field that has been regarded generally as being haphazard (Coffey, 1975; Ewing & Gregg, 1978). Stake's evaluation method has been used to examine the effects of other training programs including one designed for special education majors at Michigan State University studying vocational education for the handicapped (Kozlowski, 1976). It has also been used to evaluate a training program that aids the professional and paraprofessional staff in institutions in Michigan for the severely mentally impaired (Frey, 1976). Stake has applied his model of evaluation to several seemingly widely divergent areas, which testifies to the flexi- bility of his approach. He has evaluated science edu— cation curriculums (Stake & Hall, 1975; Stake & Easley, 1978) as well as fine arts and movement dance programs (Hoke & Stake, 1976). Evaluation Model The following is a description and explanation of the model of evaluation proposed by Stake. The sources of information for this description are his writings on the subject from 1972 through 1976 and a personal com— munication in 1978. atio: eval intu eva] are vis con the 7p!E:——————————————______'_______T______T__—T 51 Stake (1972, 1975, 1976) indicated that evalu- ations have both informal and formal aspects. Informal evaluations rely on casual observation, implicit goals, intuitive norms, and subjective judgment. Informal evaluations are seldom questioned because the results are characteristic of familiar perspectives but are of a wide range of quality. Formal evaluations use checklists, structured visitations by peers, standardized testing, and controlled comparisons. Stake suggested that most individuals in the position of evaluating a program tend to ignore these formal procedures in favor of informal ones such as ask- ing the opinion of the presenters or considering the logic of the program. Admittedly, there have been many problems in the use of formal procedures. For example, the vast majority of psychometric testing is designed to differentiate among individuals at a certain point in time rather than to assess the effects of training on a group. Consequently, it becomes difficult to demonstrate training effects that do exist. Stake has recently indicated an increased will— ingness to rely on informal evaluation procedures (1975, 1978). He believes the loss of exact measurement is compensated for by the increased responsiveness of the evaluation procedures in determining the evaluation needs Of the training program. Stake indicated, however, that evalt they diss corr acae a p: pro dat mat of as pr he fi_ + -1 52 evaluations should speak to the audiences for whom they are intended (1972). This study, as a doctoral dissertation evaluation project, must speak to both correctional training presenters and the professional academic community. The need for "hard data" for such a project is clear. As a result, formal evaluation procedures which tend to provide statistically analyzable data are used whenever possible in this study. Fortu- nately, Stake (1972, 1975, 1976) provided a great deal of assistance in organizing formal evaluation procedures as well as informal ones. Stake (1972) indicated that one of the main problems of past formal evaluation work is that there has been too little care taken in defining the conditions that exist prior to training and training activities themselves. Further, there is not enough effort made to follow training activities through to their logical outcomes. When educational or training programs would fail to produce the desired outcomes, and the conditions considered necessary for learning had not been defined prior to and observed during training, the evaluators would be at a loss to explain the deficiencies in the Program. The deficiencies could be described but not explained. Ausubel (1960), as well as Stake (1972), [ advised that evaluators would be wise to spend time Prior to the training program trying to determine the condit: of the gran i organi indice ing tl ing p: In ev atten trair paid whet desc Stat are con des and Ste th: ac ma h fi— .fi 53 conditions necessary for learning. Previous experience of the evaluators with the content of the training pro- gram is almost a necessity for this kind of advanced organization to take place successfully. Stake (1972) indicated that not enough work has been done in measur— ing the difference (or lack of it) between what a train- ing presenter intends to do and what one actually does. In evaluating training or educational curricula, more attention should be paid to the contingencies among training conditions, activities, and outcomes and less paid to individual differences among participants. While some controversy has existed concerning whether evaluations should concentrate on complete description of programs or on judgment of their merits, Stake (1972) affirms that pppp description and judgment are essential and, in fact, basic to evaluation. A complete evaluation of a training program should fully describe it and fully judge it as to its relative merits and deficiencies. In order to accomplish these ends, according to Stake (1972, 1975), an evaluator should gather data from three different sources: antecedent conditions, trans— actions, and outcomes. Organizing the sources of infor— mation in this manner aids both in describing and judging. Antecedent conditions. An antecedent condition is "any condition existing prior to teaching and learning whi ff which may relate to outcomes" (Stake, 1972, p. 36). The experience of staff members in combination with their aptitudes and interests constitutes a complex antecedent. The aspects that a training program suggests must occur in order for intended transactions to produce desired outcomes are also to be considered antecedent conditions (Frey, 1976). For example, that a training session is scheduled each working day for two weeks, and that facili— ties be made available for all training sessions, are antecedent conditions. Transactions. Transactions are "the countless encounters of students with teacher, student with student, author with reader, parent with counselor--The succession of engagements which comprise the process of education" (Stake, 1972, p. 37). Examples of transactions are presentations of films or lectures, group discussions, and the administration of tests, as well. Outcomes. Outcomes are "the consequences of education--immediate and long-range, cognitive and cona— tive, personal and community-wide" (Stake, 1972, p. 37). They are the results of the antecedent conditions together with the program transactions. In other words, training activities interact with training conditions in order to produce training results (outcomes). Changes in skill levels of the sidere i975) trans colle clas: anon even duri its: lea the ske tre as in 55 levels or attitudes, as well as any modification of behavior of the participants in training or education, are con— sidered outcomes. Organization and processing of data. Stake (1972, 1975) has provided the data categories of antecedents, transactions, and outcomes in order to stimulate data collection rather than to simply further sub-divide and classify it. He, in fact, cautioned that the boundaries among these three classifications of conditions and events are not even always distinct. For instance, during a training transaction, an outcome may present itself that can function as an antecedent for subsequent learning. A hypothetical example of this would be if the participants in a training program who had been skeptical of the usefulness of reality therapy as a treatment modality then became more positive toward it as a result of a particular training presentation. The improved attitude on the part of the participants can be considered an outcome of training, but it also functions as an antecedent to learning in that it would motivate increased attention to training transactions relating to reality therapy during the remainder of training. Stake's organization of evaluation data and scheme for its processing takes into account the fact that data occasionally fit into more than one of the three categories of antecedents, transactions, and outc dose are by ces 56 outcomes. He advised that the data be organized into a description matrix and a judgment matrix, which he states are not very different from data categories proposed by Stufflebeam (1971) and Provus (1971). The pro- cessing of data within the descriptive matrix is accom- plished by means of examining contingencies and con— gruences. The processing of data within the judgment matrix is accomplished by means of comparison and judg— ment. These terms and processes are more fully delineated in the following sections. Descriptive data. Descriptive data can be recorded as either intents or observations. These cate- gories represent what training presenters intend and what observers of the training perceive. The organization of descriptive data is represented by Stake in Figure l. The organization of judgment data which is discussed later in this chapter is also included in the figure. The Description Matrix represented in Figure 1 includes an intents cell for each of the three sources of data: antecedents, transactions, and outcomes. Stake (1972) indicated that "intents," as used in this matrix, include the terms "goals," "objectives," and "intended student outcomes." Intents are made up of planned-for environmental conditions, planned—for lectures, and planned-for activities, as well as planned—for student behavior. Listed by the evaluator in this .Amm .d .mbma .oxcumv Ecumomd HMQOAumospo no mo soumoao>o may we pmuooaaoo on 0D pump can mucoEwummm mo usomoa m .H oMDmHm xenon: pcofimpsh xflmumz coflumflmomoo moEOOpso 57 mcoHuOMmcmmfi mangoflucm mucopooopcm mucofimpsb mppppgppm mcoflpc>momno mDCoDCH three if pt pate Inte Stal sta' an an dal do he th CE 58 three-cell column are events that may happen including, if possible, the goals and plans of the participants. Included as intents are those effects antici- pated, hoped for, and even those that are feared. Intents can be global or tightly behaviorally defined. Stake (1972, 1975) indicated that any conglomeration of statements of goals can be used acceptably as a part of an evaluation. The Description Matrix in Figure 1 also includes an observations cell for each of the three sources of data. Stake (1972) indicated that the evaluator must decide which characteristics to observe and by what means he will observe them in order to collect data for this three-cell column. The evaluator will have to choose certain characteristics because he would be unable to observe all aspects of the training process. While giving special attention to participant outcomes, he should not neglect incidental and even undesirable out— comes, antecedent events, or instruction transactions. Observations are sometimes made in a direct personal way and sometimes instruments are used. Pos- sible instruments that may be used are biographical data sheets, inventory schedules, structured interviews, questionnaires, and many types of psychometric tests. Descriptive data can be processed by two principal means, according to Stake (1972). Figure 2 represents the processes of examining contingencies and congruence. 59 Descriptive data Intended Observed Antecedents s CONGRUENCE E Antecedents LOGICAL EMPIRICAL CONTINGENCY CONTINGENCY Intended Observed Transactions CONGRUENCE Transactions LOGICAL EMPIRICAL CONTINGENCY CONTINGENCY Intended Observed Outcomes CONGRUENCE 5 Outcomes Figure 2. A representation of the processing of descriptive data (Stake, 1972, p. 43). table and c that actit uate thre Date evai vat if pr: tr ra 6O Contingencies are the relationships among var- iables and are examined among antecedents, transactions, and outcomes. The evaluator needs to identify outcomes that are contingent upon certain instructional trans- actions and antecedent conditions. As intents are eval— uated, logical connections should exist between the three cells in the Intents section of the Descriptive Data Matrix. For the Observation cells, however, the evaluator needs to collect empirical data. Congruence is examined between intents and obser- vations. The data for the training program are congruent if that which was intended happened. In order for a program to be fully congruent, the intended antecedents, transactions, and outcomes would all have to occur. This rarely happens and, as Stake (1972) indicated, often should not happen. The evaluator is able to compare the cells across the rows to note any discrepancies and to determine the degree of congruence for that row. While congruence does not indicate any reliability or validity of outcomes, it does note whether that which was intended did, in fact, occur. Judgment data. Stake (1972) provided a judgment matrix for the organization of judgment data (refer back to Figure 1). Judgment data can be recorded as either standards or judgments. Standards refer to the presenters' own standards of acceptable quality, or standards estat dents stane shou Judg thos for ties abo nor the the ant 61 established by others, and can be recorded for antece— dents, transactions, and outcomes. As Stake indicated, standards are “statements by certain experts as to what should happen in a situation like ours“ (1972, p. 45). Judgments occur after it has been determined whether those standards have been met and can also be recorded for antecedents, transactions, and outcomes. Stake describes judgments as being "data on how people feel about aspects of our situation" (1972, p. 46). In other words, standards provide data on what should happen in the training program, and judgments provide data on whether those events and conditions actually did occur and whether those occurrences were sufficient for a judgment of acceptability on the part of the evaluators or others. If the goal of a training program is to achieve excellence, measurement of that excellence requires explicit standards. However, most training or education programs are not reviewed by "standard—oriented" evalu- ations, according to Stake (1972). Well-documented standards, in general, are not commonly available because information on similar programs often has not been sys- tematically recorded. Consequently, evaluation is complicated by a multiplicity of standards which vary from individual to individual and from reference group to reference group. This f It is accor are t atior info clos to c incr eva jué cor us ju tr 62 This is to be expected, of course, in a free society. It is the responsibility of the evaluator, however, according to Stake (1972) to make clear what standards are being used. Stake (1972, 1975) indicated that formal evalu— ations tend to specify criteria more carefully than do informal evaluations. Unfortunately, evaluators who closely define the criteria tend to pay less concern to outside standards of acceptability. Stake proposed increased awareness of relative comparisons in formal evaluations, whenever possible. Stake (1972) provided two means of processing judgment data. One involves the use of two types of comparisons: absolute and relative. The other is by use of a process of judgment. These comparisons and judgments are represented in Figure 3. The matrix in the upper left corner is the Descriptive Matrix of Figure 1. The matrix at the upper right represents standards in the absolute sense, with multiple standards in evidence. Absolute comparisons stem from the use of the opinions of reference groups as to what constitutes an acceptable level for antecedents, transactions, and outcomes. The matrix at the lower left represents descrip- tions of certain characteristics of similar programs to be used as standards. Direct comparisons can be made 63 T I I ABSOLUTE JWII DESCRIPTIVE DATA COMPARISON STANDARDS ------- from—---—-- e— > -----of—---— ONE PROGRAM EXCELLENCE ‘ e: > i I x r- -------- l— -------- w. I r- --- ’ .e I \ I I F RELATIVE —L COMPARISON I I DESCRIPTIVE DATA JUDGMENTS k ------ from ------- ANOTHER PROGRAM Figure 3. A representation of the process Of judging the merit of an educational program (Stake, 1972, p. 47). usin tern dec gro eva has mm 64 using these descriptions. This type of comparison was termed "relative comparison" by Stake (1972). The process of judgment occurs as the evaluator decides which of the available standards of reference groups and/or characteristics of other programs to use in making judgments. Prior to making a judgment, the evaluator must determine whether or not each standard has been met with unavailable standard data being esti- mated. Stake (1972) indicated that the act of making a judgment then rests with deciding the relative impor— tance of each standard. In other words, the evaluator must decide how much attention to pay to the standards of each reference group's point of view. Using a combi— nation of relative and absolute comparisons, represented by the arrows pointing to the lower right of Figure 3, the evaluator can obtain a composite rating of merit, with possible qualifying statements, which completes the judgment process. Relationship of the Model to the Present Study As Stake (1972) indicated, there are two main uses of the model for evaluation methodology. One use involves assessing the effects of a program on its par— ticipants in order to assist in making a judgment as to its overall efficacy. This type of evaluation is called "summative evaluation." The other use involves assess modifj tatior is ge: evalu of ev cater uati< the that of f pari tiv. fut gra has be in ii L 65 assessing specific Objectives and activities in order to modify and revise the training program during its presen- tation or during later presentations. This second type is generally referred to as "formative evaluation." There has been disagreement in the field of evaluation methodology as to which of these two types of evaluation should be encouraged. Stake (1972) indi— cated that Scriven, for example, prOposed summative eval- uations that use a relative comparison approach to answer the question,-"Which program is better?" He indicates that Cronbach, on the other hand, encouraged the use Of formative evaluations using absolute standard com— parisons to answer the question, "How can we do it better?“ \\This study is undertaken essentially in a summa- tive evaluation framework. However, it is expected that future implementations of this and other training pro— grams for correctional staffs will be revised on the basis of this evaluation. AS such, this evaluation can be considered also to have a formative dimension. Summary of the Review of the Literature The review of the literature in this chapter included several topics. As a background to the problem, literature regarding delinquency was reviewed including its description, prevention, and treatment. A pronounced lack of demonstrated success of institutional treatment of de went there the s of q with the nif: tut por eva of 19' hi tr ft r—f *i 66 of delinquency was noted in the literature. The treat- ment modalities of behavior modification and reality therapy, which are the chosen methods of treatment of the site of the current study, were described. Literature relating to the recognized importance of quality staff in-service training was then reviewed, with at least one author (Coffey, 1975) contending that ”” the poor quality of in-service training of staff has sig— nificantly contributed to the lack of success of insti— tutional treatment of delinquency. The review then sup— ported a contention of the current study that systematic evaluations are needed in order to improve the quality *9” of in-service training in corrections (Ewing & Gregg, 1978). h Only facilitation training has received any sig- nificant evaluational attention in the area of in-service training for correctional staff (Alston, 1971; Katrin, 1972). Evaluations of other types of in—service training for correctional staff, using more complete evaluation techniques, are presently in progress in two more locations (Quay, 1977; Liberatore, 1978b). No known evaluation of in-service training for correctional staff has been completed using systematic educational evaluation technology. Stake's evaluation model (1972, 1975, 1976, 1978), which was chosen for application in the current study, was rev model t present is din the de fi 67 was reviewed and explained. The relationship of the model to the current study was also described. With the review of the pertinent literature presented in this chapter as a background, the reader is directed to Chapter III, which describes and discusses the design considerations relevant to this study. reat rel tra are CHAPTER III DESIGN OF THE STUDY Introduction The purpose of this chapter is to provide the reader with a description of the study. Each of the relevant components of the design is delineated. The training program participants, subjects of this study, are described with respect to a variety of demographic characteristics. The particular setting for the study, the Monroe County Youth Center, is described with regard to its purpose, functions, and unique features. The plan for evaluating the training program is explained. Robert Stake's model of evaluation was chosen as a guide for this evaluation of the training program. His conceptual framework for evaluation was explained in Chapter II. The description of the treatment intervention for the study, the training program itself, is presented in this chapter according to that framework. Stake encourages the collection of data used to evaluate programs from a variety of sources (1972, 1975, 1976, 1978). The methods for the observation and collection of data for this evaluation are described as to their purpose, source, 68 and I cedm and i The chap test to] 69 and reliability measures. The section on evaluation pro- cedures and implementation in this chapter describes how and when the methods of data collection were applied. The analysis of the collected data is described in the chapter in terms of the specific null hypotheses to be tested and the specific statistical analysis techniques to be used. Description of the Participants Twenty-seven employees of the Monroe County Youth Center routinely are involved in direct contact with juve— niles housed in the facility (see Figure 4). They are responsible for the carrying out of treatment and/or detention plans for residents. Seven job titles are included in this group (see Table l). The largest sub- group within this group is that of detention shift super- visor, totaling ll individuals. There are nine group leaders who fulfill a similar function in the treatment units of the facility. The remaining seven individuals are three teachers, the treatment coordinator, the detention coordinator, the special service coordinator, and a unit therapist. These 27 employees were the primary subjects for the plan to evaluate the effectiveness of the training program. The mean length of employment experience with children for that group was 51.3 months, with a range from zero months to 178 months. The mean education . a .uumco Hmcoflummficmmuo owmoHQEo umucoo £u30> >u2500 woucoz v ousb.m m2~b_ Pm mmao -mmaam pwufimmEH >Hamcofluosm QHWMW Emamm man how muouosuuwcH u mmoeoamemzH .H.m umxuo3 wumu Udanu u 300 .Illltllllll mmmo doomo (madam msz Emumm mmmxaoz mach mmdu OAHIU BmHmdmsz O39 BHZD H mmOmH> OZHZmmm SUU\MUH>mmm mOBmmmm m>HE mfiszDBm UZHQAHDm EZMQZmBZHmmmDm _ mmo~>mmm B1300 LC MCFUEmHC mmUQDh m90>LDW OCACaMLE MO EOflUQLaUHUC< -p U ‘cqwtt‘nflll WUCUUUUQUC< WUCUUCH TTWD UOLQTZ AQCOdJfl>hWWQQ nfibmvm—J CTF—zv 86 .cofiuo>uwmno mo mpOzqu asap «Ucflcqmuu OutoEoousc Hmcodufipp< m oucmuowoam Hmconuumuuo tumom smumv >m>u3 unoccoHummSO QDIBOAAOL an Aocacflmuu nouum mcucoE xflmv o commouuoo zufl3 mamasmuu a AUCHCHMMUIamomV >m>uam acacflmue LuHB ceauomumfiumm Ac .m Aocacamuuauouum mnucos xfim. muqmccOAumono man3oH~om an Achcflmuu noumm mcucoa xam .ocwcfimuunuwom. maccm mashed ceaun>uwuno wumum Am .v Amcficflmpu umuum mcucoe xnmv munmccoflumwsa Q5|30HH0L Ab Amcacwmuueumom tong. nmme mucmpmocH Hmowuwuu Am .m flocflcfimuueumom noumv >w>usm wumwocou mean: pumzou moCSDHuu< .m Aquacqmuueumoa pcm Imudv amok m>fiuooflno nomad m>fiuficoou .a moEoouso O moEOUuJO one chcdmnu umuuo mfiucOE xwm An .mcofluommcmuu .mucopoooucm coccmucfi uo >umEEsm .c ousoba pouuwuwud uccsrcdfior Cc_c«mub numoq >~wumMUwEEw Ac Ucflchmuu usonm QCSDAuum w>AmeOQ muchumCOEwp H~w3 cofizfimo Hmcowuod Ab po>uwmno Aw :ofiumEMOwcfl om: Hafiz xuo3 :« >ua~flnm po>OuQEH mosqficcowu mo ow: cum3ou mmcsuwuum pw>0umEH mocha AmoaQOu mo oopwa3ocx pommmuocw mo cm>u :fl mocczoeeceoouso accodb¢p€< mmEoouao Amzwcflmuu OCAMDUV mommu0wps< .Ehom puoomm Amquuocd mcofluommcmne < xhocoamd .>H uwuamcu wo uxou com macauommcmub chmcwcuulumoav 3 cowuomwmflumm mo cowuumm >p0umfim qcficflmub >o>usm qcficawue sud chflcamuulumoav >m>u=m qcficqmuk Luw3 cowuummmflumm Amcficflmuu gawpspv ELOL chooom Hmu0pomc< Aocflcqcuu umuwm wcucoe xflmv oufimccowummsa 23:3OHHcm An chACAMHqumomv >o>usm OCHCHMuE £ua3 conuomwmfiucm Am Awacflmuu Ocfiuscv Ehom buoomm HmuOUuocd chficfimuu manuspv Enos puooom amuocuoc< Amcficwmuu UCMMSCV Euom buooom Amquoocd Amcflcqmuu manuspv Enos buouom kuOOUoc< chflcqmuu madman. ELom chooom amquooc< cofluocw mpsufiuud Hmoflumuomze An cofiuumm umwumucH Hmcowuoa Am chMCHmuulwuav >o>u5m acacampb uo :oflummflofiuc< .OH .0 .H wucopwuouc< xmm3 Ucoumm ocfiusv mucwvfimmu :uMB uowucoo m>mz ou cOmuwm macaw comm DCQAUAWOHQ on HAM: uumum madcflmue >Hwnflucm pmucommum mono amofimOu comm mummucoo ccm mosgflczumu Howmm: can Hmofiaomud mpH>0ud Hafl3 mcoflmmow poaqcuco mm cmzu oboe uo: .muwnEoE uwmuw >Hco :ofimmmm comm um mocmccouum EJEACME ucoouwm w>ums>uzaflm voumsuumucfics :flmEmu HHH3 wcoflmmwm mcoflmmsomflc macho Hanan vac ovumd no“ mannafim>m wwquflfidowm copfl>0um m>mp Hmcofiuosuum:fl awe conufluum Hmowumuoonu an umwuwucw HMCOmumm Am Oshawwuu ppm30u occupuum manmuo>Mw ammuflcw .H mucopouwucd Anon: cwfizv pom: pozuw: Hmcofium>uwmno mucoucH data, but i greater ext of data coI developmen' when appro On reliabilit separate H were used. across ite related. consistent across ea( tency imp construct Consisten Of an ins resPonses With chi] iStered i Subllfilcts cedure i, stUdY. a of less - CO(Effici 87 data, but formal procedures were relied upon to a much greater extent. The following are the primary methods of data collection used in this evaluation. Their development is delineated and reliability measures, when appropriate, are provided. On those instruments that the calculation of reliability measures was considered appropriate, separate Hoyt's coefficient alpha statistical procedures were used. They determined the degree of homogeneity across items for each group of items believed to be related. This procedure provides an estimate of how consistently the subjects' patterns of responses hold across each group of items. A high degree of consis- tency implies the existence of a unitary underlying construct within those items. This measure of internal consistency constitutes an estimate of the reliability of an instrument. The calculations were run on the responses of the 27 subjects who had direct contact with children in Monroe County and who also were admin— istered the pre? and post-instruments, along with 10 subjects from a comparison group (see Evaluation Pro— cedure in this chapter). For the purposes of this study, a Hoyt's coefficient alpha reliability estimate Of less than zero was considered indeterminate. A coefficient of zero to .30 was considered very weak; .30-.50 to good; nated or the degi consider upon re did not awe-"ev— scale f Anecdot data re all of simple occurr Specia attend when t avail, Attit and a 88 .30—.50 was considered poor; .50—.7O was considered fair to good; and .70—l.OO was considered good to excellent. Item analyses were run and some items were elimi- nated on the basis of two criteria. If an item suppressed the degree of reliability obtained by its scale it was :onsidered a candidate for removal from that scale. If ipon re-examination of the content of the item it also iid not appear to be related to the other items in the scale for one reason or another, it was then removed. anecdotal Record Form The Anecdotal Record Form was designed to record data relevant to a number of intended antecedents and all of the intended transactions (see Appendix L). This simple form allowed for a written record of what actually >ccurred during training sessions on an hourly basis. Special note is taken on the form of interruptions and attendance. Relevant data were also recorded regarding when the sessions were held and what facilities were available, as well as how often staff members initiated :ransactions by means of discussion. anticipation of Training Survey The Anticipation of Training Survey (entitled attitudes toward Training) was made up of 17 items and was created in order to gather data on the experiences and attitudes of staff members regarding in-service trainine (see Ap agree his/her the tra prior 1 includ and th experi explor are me train: four much" m m devei attil area refl COnc Stat the app: 89 training prior to participation in the training program (see Appendix M). Seven of the items are essentially demographic in nature. The respondent is asked to rate his/her own familiarity with the five main subjects of the training program and to indicate the sources of prior knowledge. Four items deal with the degree of interest in participating in the training program, including which areas are of most interest to the person and the degree to which he/she believes the training experience to be necessary. The remaining six items explore attitudes toward training in general. Statements are made regarding the importance (or lack of it) of training and the respondent is asked to pick one of four choices from "agree very much" to "disagree very much" for each. Attitudes toward Using Concepts Survey The Attitudes toward Using Concepts Survey was developed to gather data regarding the staff members' attitudes toward some of the key issues of the topical areas (see Appendix C). These responses were assumed to reflect on the respondent's attitudes about use of the concepts. The survey is made up of 16 items which are statements about the overall effectiveness of each of the various approaches or about specific aspects of each approach. Five items are concerned with tenets of new: ~. le'f—r reality cation, five ar rection dropped further other : item a As wit respon "agree these for U istra Parti A Hoy the t obtaj There Conn and L092 th area 9O eality therapy, three items deal with behavior modifi- ation, three items relate to communication skills, and ive are concerned with the use of volunteers in cor- ections. Upon item analysis, three items had been rOpped because they suppressed reliability and, upon urther examination, appeared to be unrelated to the ther items in the subscale. One behavior modification 'tem and two communication skills items had been dropped. 8 with the last six items of the previous survey, the respondent is asked to pick one of four choices from "agree very much" to "disagree very much" on each of these 16 items. Hoyt's reliability coefficients were established for the total survey and its subscales,.based on admin- istration of the pre-training survey to the 40 training participants in attendance at the first training session. A Hoyt's reliability coefficient of .68 was obtained for the total survey. The subscale reliability coefficients obtained were the following: .71 for Use of Reality Therapy Concepts, .57 for Use of Behavior Modification Concepts, .68 for Use of Communication Skills Concepts, nd .80 for Use of Volunteers in Corrections Concepts. ognitive Areas Objective Test The Cognitive Areas Objective Test was created 0 assess specific knowledge in each of the topical reas (see Appendix B). The test contains 40 multipl with a areas a therapp for cor volunt items bility unrela one be items been of th knowl trair areas this know assu the bas€ 40 1 tra Tes 91 multiple—choice items which were created in consultation with a variety of experts (see Appendix N). The topical areas are covered as follows: 14 items for reality therapy, 10 items for behavior modification, 15 items for communications skills, and six items for use of volunteers in corrections. Upon item analysis, seven items had been dropped because they suppressed relia- bility and, upon further examination, appeared to be unrelated to the other items. Two reality therapy items, one behavior modification item, two communication skills items, and two use of volunteers in corrections items had been dropped. The basic assumption behind the construction of the Cognitive Areas Objective Test is that increased knowledge in these topical areas as a result of the training should be reflected in items relating to those areas. In addition, while it is extremely difficult in this type of situation to demonstrate that increased knowledge directly improves work performance, it is assumed that there is a positive relationship between the two factors. Hoyt's reliability coefficients were established based on administration of the pre-training test to the 40 training participants in attendance at the first training session. A Hoyt's reliability coefficient of .60 was obtained for the total Cognitive Areas Objective Test. The sub—tests within the total test were found ?. 773375.??‘1153 '3 to have Reality Communi Correct Correct proced any of outcom Survey sever part inten of ti 0f t1 ment inst data cons mate abm del Cat fun of 92 to have the following reliability coefficients: .35 for Reality Therapy, .54 for Behavior Modification, .31 for Communication Skills, and .57 for Use of Volunteers in Corrections. Correctional Preference Survey An instrument was included in the observational procedures that collected data not directly relating to any of the established antecedents, transactions, or outcomes. The instrument was the Correctional Preference Survey (Quay, 1968) (see Appendix I). This was done for several reasons. While a change in attitudes on the part of staff members toward delinquent youth was not an intended outcome of training, it was felt investigation of that area might shed some light on unintended outcomes of training. In addition, data collected by the instru- ment in this setting would contribute to the field of instrumentation in corrections by supplying normative data to add to that already collected (Ingram, 1970). The Correctional Preference Survey was originally constructed by Dr. Herbert C. Quay (1968) as a means of matching correctional workers on the basis of opinions about youth and correctional practices with types of delinquents in a correctional setting. As briefly indi— cated in Chapter I, delinquents can be classified into four behavioral categories by means of a combination of three scales (Quay & Peterson, 1964). The Behavior Problem behavie studie: delinq quent is the factor case 1 scale Opini naire was 6 beha from Cate atte Dist of. The cru He Be. The 93 Problem Checklist consists of 55 items describing deviant behavior. It was developed out of factor analytic studies comparing the behavior of delinquent and non— delinquent children. An observer who knows the delin- quent well responds to the checklist. Another scale is the 36-item History form also developed by means of factor analytic methods. Anyone with access to complete case history material can fill out the form. The third scale developed by factor analysis is the Personal Opinion Study, which is a 100-item true~false question- naire filled out by the delinquent. Each of these scales was develOped by Quay and Peterson (1964). The adolescent is then classified into one of four behavioral categories on the basis of T—scores resulting from a combination of these three scales. The Behavior Category 1 (BC-1) (Inadequate-Immature) youth is described as easily flustered and confused, lazy, having a short attention span, and often truant. The BC-2 (Neurotic- Disturbed) youth is characterized by anxiety, feelings of inferiority, social isolation, guilt, and depression. The BC-3 (Unsocialized—Psychopathic) youth is often cruel, defiant, and shows little concern for others. He tends to be manipulative and untrustworthy. The BC-4 (Socialized-Subcultural) youth is a gang delinquent who shows intense loyalty for his delinquent peer group. He deme looks some 5 at the Federz years ment categ types one) Thei pati duri Yout intc con fee you wit wit yor be 94 He demonstrates no serious personality difficulties but looks to his peers for most of his reinforcement. This classification scheme has been used with some success in placing delinquents into treatment units at the Kennedy Youth Center, which is operated by the Federal Bureau of Prisons for youthful offenders 12—20 years of age (Ingram, 1970). Each cottage has a treat— ment program designed to meet the needs of the behavior category group housed there. The Correctional Preference Survey was developed as a result of the theoretical position that certain types of adults would be more successful working with one behavioral category of youths than with the others. The BC—l treatment agent should be an exceptionally patient person who is tolerant and calm, especially during the eXpected emotional outbursts of the BC-l youth. The BC-2 treatment agent should be able to enter into personal relationships with disturbed youth and be able to deal verbally with long—standing emotional conflicts. He/she would be sensitive to the intense feelings of adolescents. The treatment agent for BC—3 youths needs to be tough-minded, direct, and candid without being willing to engage in long discussions with manipulative youths. Adults working with BC-4 youths must have strong beliefs and determination and be perceptive enough to recognize group attempts at manipul sonally elicit youths items r The it the re four-p the nu relati direct the s‘ but 13 the p inste chap] anal} PEro to t succ Plac resp of t thar 95 manipulation. The person must be exceptionally per- sonally secure in these situations. The survey consists of 60 items designed to elicit the opinions of the individual toward types of (ouths and styles of correctional practice. Fifteen items were written to represent each behavior category. The items are written in the form of statements that :he respondent is asked to agree or disagree with on a four-point continuum. Classification is determined by :he number of items answered in the keyed direction :elative to the average number of responses in that lirection by a normative sample of correctional workers. The Ingram study (1970) involved administering :he survey to 50 volunteers at the Kennedy Youth Center, zut n9: using the results to place the volunteers with :he previously categorized groups of youths. Placement, -nstead, was made on the basis of impressions of the :haplains who ran the volunteer program. Upon subsequent analysis, it was demonstrated that a significantly higher >ercentage of "correctly placed" individuals (according :o the survey) proved to be rated by the chaplains as successful in their volunteer work than those who were >laced with the wrong type of group. In addition, those respondents who scored below average on all four scales >f the survey proved to be less successful as a group :han those who scored above average on at least one of the SC 14 vol and we retest survey tical of th for B It is nifie proce betwt the ing BC-2 coei pro} reL cor and cos and reI Ste 96 the scales. It is interesting to note that all of the 14 volunteers who both responded strongly on the survey and were placed correctly were classified as successful. Ingram reports internal consistency and test- retest reliabilities based on administration of the survey to 62 volunteers. He used Spearman-Brown statis- tical procedures to estimate the internal consistency of the four BC scales, obtaining coefficients of .62 for BC-l, .33 for BC—2, .48 for BC—3, and .69 for BC—4. It is indicated that each of these coefficients is sig- nificant at the .01 level of probability. A test-retest procedure was undertaken with a gap of seven months between the two administrations of the survey. While the method of calculation was not indicated, the follow- ing coefficients were obtained: .39 for BC-l, .36 for BC—2, .50 for BC-3, and .43 for BC-4. Each of these coefficients was also significant at the .01 level of probability. Ingram also calculated product moment cor— relations between the BC scales. The only significant correlations were found to occur between BC-l and BC-2 and between BC-3 and BC—4. The Hoyt internal consistency coefficients on this population proved to be uniformly and considerably lower than the Spearman-Brown estimates reported by Ingram. The coefficients obtained in this study were the following: .42 for BC—l, .24 for BC-2, .01 for BC-3, and .53 for BC—4. Critic Care W tive T situa‘ Appen four apprc be it ther sup exa re] 97 Critical Incidents in Child- Care Work Objective Test The Critical Incidents in Child-Care Work Objec— tive Test was develOped in order to present typical work situations to the staff members in written form (see Appendix D). Participants were asked to select among four choices for each of 10 items the choice that best approximates what they themselves would do. It could then be investigated whether the training had any effect on their simulated performance from pretest to posttest. A modified critical incidents technique was used to develop the test based on that reported by Jaques (1959). The critical incidents technique is a method of collecting and organizing observations of especially effective and ineffective job performance behaviors. Usually a large mailing of questionnaires is made to supervisors and on-the-job staff asking them to list examples of particularly effective and ineffective job- related behaviors they have themselves experienced or have observed. The responses are analyzed, organized, and interpreted by the investigator. While it was not within the sc0pe of this study to undertake a complete critical incidents project for the field of child-care work, certain aspects of the technique were incorporated into the development of this test. A variety of individuals who are or have been engaged in residential treatment work were consulted informs or inei they he additi cult a are li work 5 situat Cdet ation evalu was C Satis \ Surve \ e10pt fact twee into Sati The ment of: Its Pro, 98 informally and asked to provide examples of effective or ineffective child management and treatment practices they have experienced or observed (see Appendix 0). In addition, they were asked to provide examples of "diffi- cult and typical" situations that child-care workers are likely to experience in their work with adolescents. These responses were used as a pOpulation of work situations from which 10 were selected. Each situation calls for a response on the part of the child- care worker. Four choices were created for each situ- ation with only one being keyed as appropriate by the evaluator. The Hoyt internal consistency coefficient was calculated to be .39. Satisfaction with Training Survey The Satisfaction with Training Survey was dev- eloped in order to gather data on the degree of satis— faction with the training of the participants after the two-week program (see Appendix H). The survey is divided into three sections entitled Attitudes Toward Training, Satisfaction with Training, and History of Training. The first section includes 11 items consisting of state- ments to which the respondent is asked to choose one of strongly agree, agree, disagree, or strongly disagree. Items are concerned with the length of the training program and each of the major topical areas, the usefui topic durin the i ham suffi trair tion they forc Spac any item ext hav top cat the rec to to] th 99 usefulness of the information presented on each of the topics, whether questions were adequately answered during training, and whether the respondent will use the information he/she has learned. Also included are items requiring responses concerning whether there were sufficient Opportunities for discussion, how expert the training presenters were, and to what extent the par— ticipants believe they are prepared to implement what they have learned. The first section concludes with a forced rank ordering of the estimated importance of each of the major topical areas in that respondent's work. Space is provided for respondent comments concerning any of the above-mentioned issues. The second section is made up of an eight-part item which requests the respondent to indicate the extent to which he/she considers each of the areas to have been presented during the training sessions. Eight topics are listed. Reality therapy, behavior modifi— cation, and communication skills are each divided into the two areas of theory and techniques. The principles of adolescent psychology and use of volunteers in cor— rections are also included. Each respondent is asked to select one of four choices for each of the eight topics to indicate the extent to which he/she considers that topic to have been covered: completely presented; thoroughly, but not completely presented; minimally preser a reqi progrz it in conta wheth durir Staf: rese actu trai vidr fiel He] deg wor tre IEC was th me 100 presented; not presented. The section concludes with a request for comments on any aspect of the training program that might help the training presenters improve it in the future. The third section consists of questions regarding contact with children during the training period and whether or not the participants received any training during that time in addition to the training program. Staff Observation Rating Scale One of the primary difficulties in evaluation research is how to determine whether the participants actually use the information they learn in in—service training sessions. Quay (1977) is one of the few indi— viduals attempting to investigate this problem in the field of in—service training for correctional staff. He has labeled this difficulty that of discovering the degree of "program integrity" of a facility. In other words, does the staff of a facility actually use the treatment modalities it claims to be using after having treceived the appropriate training? A method of investigating the above question was developed for this evaluation (see Appendices F and G). It involves recording observations and later rating the interactions according to a set procedure. An agree— ment had been entered into with the Monroe County Youth Center and the Monroe County Intermediate School District to Vidt facili anothe of a d being vatior vatio nonve have The N Depar catie asse: haVe Prof aide Tree duc tap gro Sid giv 101 to videotape trainee—delinquent interactions in the facility. This agreement was terminated by them when another facility and school district were sued by parents of a delinquent as a result of videotaped observations being shown in a college classroom. Thus the obser- vation plans had to be changed. The decision was made to use audiotaped obser— vations. While audiotapes do not record important nonverbal interactions that videotapes include, precedents have been set in using them to rate therapeutic behavior. The Michigan State University College of Education, Department of Counseling, Personnel Services, and Edu- cational Psychology, for example, uses audiotapes in assessing the level of competence of master's candidates in counseling practicum settings. Several researchers have made use of audiotapes to assess the skills of para— professionals in training to be mental health service aides (Danish et a1., 1976; McCarthy et a1., 1977). An arrangement was made with the Coordinator of ’Treatment of the Youth Center for audiotaping to be con— ‘ducted within the facility. It was agreed that audio- 1taping carried out by the coordinator and a selected group leader would be less obtrusive than having out— siders come in to do on-site ratings. The instructions given the taping agents were to: —. . . .. ._,._..1 wymmm were I tacit betwe as op quenc pose the prog whe bet Pre det in 102 1. Obtain samples of ongoing activity that were representative and typical of daily life in the facility. 2. Choose times to tape when verbal interaction between staff and residents would normally be occurring, as opposed to times such as late at night when the fre- quency of interaction would be very low. 3. Conduct the taping as unobtrusively as possible. 4. Remind interested staff and residents that the taping was being conducted to evaluate the training program rather than individuals in the facility. 5. Include as many staff members as possible in taping sessions. 6. Record completed interactions and situations whenever possible, attempting to keep each session to ‘between five and 20 minutes. 7. Record the time, situation, and staff members present at each session. t The purpose of making these observations was to jdetermine if the staff members were using the training ‘information. Therefore, a rating system, the Staff Observation Rating Scale, was devised including important aspects of the two treatment modalities taught in the training sessions: behavior modification and reality therapy. Aspects of communication skill techniques were integi dices scori the t force Bers tion rein pre sct 103 integrated into the rating system as well (see Appen- dices F and G for the rating instructions and sample scoring sheet). A previously developed instrument was used as the basis for the rating system. The Positive Rein— forcement Observation Schedule (PROS) was developed by Bersoff and Moyer (1976). The scale consists of descrip- tions of 10 categories of behaviors that are positively reinforcing in nature. Mediators of reinforcement, such as behaVior modifiers, teachers, and therapists, are assumed to emit such behaviors in various frequencies during their work with children. It is a basic tenet of behavior modification that the frequency and type of reinforcing behavior emitted have varying effects on the children involved. The PROS has been used as a reinforcement preference schedule, but its use as an observation schedule is of greater importance to this study. In three studies reported by Bersoff and Moyer (1976), raters observed interactions in classroom settings. Frequency rates of teacher behavior described by the 10 categories were recorded by various combinations of seven raters. High inter-rater reliabilities resulted with correlations between pairs of raters usually exceeding .90 and per— centage agreement in a time sampling format repeatedly exceeding 80%. instr of pc an ex of t] for thei audi Moye obs acc 104 While the PROS has proven to be an effective instrument for the measurement of type and frequency of positively reinforcing behavior, it does not include an evaluative component. An estimate of appropriateness of the behavior emitted is not included. The PROS instrument was modified in several ways for use in this evaluation. Two of the 10 categories of positively reinforcing behaviors were removed because of their reliance on nonverbal information unavailable on audiotapes. Two neutral categories used by Bersoff and Moyer (1976) in other experimentation were included for observation. An evaluative component was added to accompany the frequencies of occurrence of categories of behavior. The rater is asked to rate each behavior on a scale from one to seven regarding its appropriate- ness in the given situation. A major addition to the PROS was brought about by the inclusion of staff behaviors relating to the tenets of reality therapy. Reality therapy concerns itself with the modification of problematic behavior. A category describing staff reaction when confronted with such behavior on the part of residents was created. The appropriateness of response is rated on several dimensions, such as emphasis on behavior rather than underlying motivations, and encouragement by the staff person of appropriate commitment and planning on the part of the resident. instr feedt degre were Acce rati Dime beha stai bac} prd res app and cat th] 105 Communication skills are also included in the instrument. The frequency of active listening and feedback responses are recorded and rated as to their degree of appropriate usage. Active listening responses were already included as two categories of the PROS: Accepts Feelings (AF) and Accepts Ideas (AI). Dimensions of effective use of feedback are incorporated into the ratings of the response to problematic behavior category. Dimensions included were specificity of problematic behavior described in feedback, the degree to which the staff person's intentions were clear while giving feed- back, the degree to which the consequences of the problematic behavior were pointed out, and the degree to which alternative behaviors were offered to the resident by the staff person. Two raters were used to rate the frequency and appropriateness of staff person behaviors in the 15 audiotaped situations (see Appendix P for the qualifi- cations of the raters). Three training sessions of three hours each were provided to the raters. Inter- rater reliabilities were established for the two raters on four training audiotaped situations obtained at the Ingham County Shelter Home, prior to their rating of the 15 situations. Spearman rho correlation coefficients were calculated for the two raters on their ratings of appropriateness for each of the categories (positive reinf respe thom meas acro the four mate Spa ro_i 106 reinforcement responses, listening reSponses, neutral responses, and responses to problematic behavior) across those four situations. These correlations provide a measure of monotonicity or agreement between raters across several situations. These Spearman rhos for the rank order of the ratings of apprOpriateness on the four categories were averaged to yield an overall esti- mate of inter-rater reliability of rho = .87, with no Spearman rho for a category being less than .80. Follow-up Questionnaire A questionnaire was developed for the purpose of gathering follow-up data six months after the two-week training program (see Appendix E). These additional data reflect on the lasting effects of the program. The ques- tionnaire was kept brief because of the large amount of instrumentation already administered to the Monroe County staff. The questionnaire responds to four basic issues. Data are collected on the extent to which the training information has been applied by the respondents. Data are also collected on the extent to which they feel the information has been helpful to them. The respondents are asked to indicate their overall attitude toward the training and finally to list its strengths and weaknesses. Demographic data are requested including name, job title, date of administration of the questionnaire, and which training sessions the respondent attended. The respone has appliet areas of t The extent to each in scale. Tt cated by 1 responses worthwhil the secon rating of the sessi up to thr training this eVa 0f imple describe Procedui Vided. With a ; investi 107 he respondent is asked to rate the extent to which he/she as applied information learned in each of the topical reas of training by selecting one of four choices. he extent to which this information has been helpful 3 each individual is also indicated on a four-point sale. The overall attitude toward training is indi— ated by responses to two items. In the first, the esponses include a four-point scale rating of how orthwhile the in—service has been for the staff. In he second, the responses include a four-point scale ating of how much each respondent personally enjoyed he sessions. Lastly, the respondent is asked to list 9 to three specific strengths and weaknesses for the raining program. Evaluation Procedure This section describes the procedures by which ais evaluation was undertaken. The chronology and means E implementation of the methods of data collection ascribed in the previous section are indicated. The rocedure for statistical analysis of the data is pro- ided. The section, and this chapter, are concluded ith a list of the specific evaluation outcomes to be ivestigated. Procedure pg the advent first pres istered a istration T data. Th date of a of time 5 children separate test inc (entitle toward r tive Tes Criticaj adminis- ing. 1 Gathere length ing wit responc' SESsioI 108 >rocedure for Data Collection Instrument administration. On the same day of :he advent of the training program, but prior to its First presentation, 40 training participants were admin— .stered a 28-page "paper and pencil" instrument. Admin— .stration of this instrument took approximately one hour. The instrument included collection of demographic ,ata. The respondents were asked to provide their name, late of administration of the test, job title, length .f time at that job, length of experience working with ehildren, education, and college major, if applicable. The instrument actually was a combination of reparately conceptualized surveys and tests. This pre— ,est included the Anticipation of Training Survey entitled Attitudes Toward Training), the Attitudes ;oward Using Concepts Survey, the Cognitive Areas Objec— .ive Test, the Correctional Preference Survey, and the tritical Incidents in Child-Care Work Objective Test. A 30—page posttest form of the instrument was ,dministered on the last day of the second week of train- ,ng. In addition to the demographic information also 'athered in the pretest of name, date, job title, .ength of time at that job, length of experience work— .ng with children, education, and college major, the "espondents were asked to indicate which training iessions they had attended. They also were asked to indicate i during tra they might the postte Training t Survey, t7 rectional in Child- C employees instrume1 who work Figure 7 occurred trations the Shel No two ( no two t are cer Certain County experie adminis for the (Campbe form 0. 109 'ndicate if they had had contact with resident children uring training and to specify any other type of training they might have received during this time. Included in the posttest instrument were the Satisfaction with Training Survey, the Attitudes toward Using Concepts Survey, the Cognitive Areas Objective Test, the Cor— rectional Preference Survey, and the Critical Incidents in Child—Care Work Objective Test. Comparison group—-Ingham County Shelter Home employees. Modified forms of the pre— and posttest instruments were administered to a group of 10 individuals who worked at the Ingham County Shelter Home (see Figure 7). The administration of these instruments occurred within a week of the corresponding adminis— trations in Monroe County. The purpose was not to use the Shelter Home group as a true control group. 0 two correctional settings are exactly alike, as are 0 two correctional staffs. While the situations re certainly not identical, they are comparable in ertain respects. Both staffs work with children in a ounty correctional facility and are roughly similar in xperience and education. The purpose then of this dministration was to control, to a limited extent, or the effects of history, maturation, and testing (Campbell & Stanley, 1967). The modifications in the orm of the instrument administered to the Shelter Home 110 :11! .11! . .womomusm coaumsam>w How #Gwfisupmcw one mo COHpmuuchflfipm .h wusmwm #QmeSHU.mGH wcoz pcofisuumcH oawc momhoamfim “voodoo woouflo wcoz mcflzflmme amapnmm T odoasuomcH MHHQ womh0HmEm uOMpsoo poohflpooz Any ucofiduumcH viii mcflcflmnuapmom acflcflmne Haze pcofisuomcH FNHC wmo%onEm uUM#coo “ocean Amy vcmfipmwne maficflmupnoom mmsouw oEom uwuaonm hucsoo EmnmcH Housoo nubow wucsoo ooucoz group were the pretes the postte was provid Ti 10 indivie instrumen‘ those pre pated in time cont Of these (Comparat the Monrc 10b tith descript SPECiali of emplo 37.7 mon The mean Complete to 13 Y6 SouthCeI deilit males a County 111 roup were that the Anticipation of Training section in he pretest and Satisfaction with Training section in he posttest were removed, since no training program as provided for them. The Shelter Home comparison group is comprised of 0 individuals who volunteered to take the pre— and post— nstruments. It should be noted, however, that all of hose present at a staff meeting volunteered and partici— lated in the study. Each of these subjects was in rou— ine contact with the residents of the facility. Nine f these staff members were classified as supervisors comparable to shift supervisors or group leaders in he Monroe County Facility). ‘The other staff member's ob title was that of Shelter Home caseworker. His job escription and functions are similar to the treatment pecialist position in Monroe County. The mean length f employment experience with children for the group was 7.7 months with a range from 12 months to 60 months. e mean education level of the group was 14.70 years mpleted, with a range within the group from 10 years 18 years. The Ingham County Shelter Home is located in uthcentral Michigan in the city of Lansing. The cility is used as a temporary holding placement for les and females under the jurisdiction of the Ingham unty Probate Court. It is licensed to house 21 individual stays are W) treatment introduce County Pr the time the Shelt geted be} the Shel procedur formulat official instead, Placemer Cont inge bEhaVio] PIOVidet nature . SChools after t during gator ‘ form w. 112 individuals for not more than 30 days, although longer stays are not unusual. While being classified as a holding facility, treatment aspects of a behavioral nature have been introduced into the Shelter Home programming (Ingham Sounty Probate Court Shelter Home Committee, 1977). At the time of admission, the child's caseworker meets with he Shelter Home caseworker to discuss tentative tar- geted behaviors for the child during his/her stay at the Shelter Home. Within 10 days a more formal staffing procedure is scheduled during which a treatment plan is formulated. Release from the Shelter Home does not pfficially depend upon performance in the program but, instead, on the caseworker's finding suitable subsequent placement. Privileges of various types, however, are fontingent upon the child's maintaining acceptable ehavior. Both recreational and academic activities are rovided within the facility but are of a supplementary ature. Children in residence generally attend public chools nearby. Use of the Anecdotal Record Form. Immediately fter the administration of the pre—training instrument, uring the first day of training entries by this investi— ator were begun in the Anecdotal Record Forms. One orm was used for each day of training. In addition to recording . dance and A number 0 the course 55 Within twe ing progre the facil later. R three wee completed E Approxime two-week was admi a regula the ques “1&3 bases f, (1972). from de DireCt seCOnd' 113 recording a description of training transactions, atten— dance and number of interruptions were kept on the forms. A number of incidental occurrences were noted during the course of the training as well. Audiotaping of trainee-delinquent interactions. Within two weeks of the conclusion of the two-week train- ing program, the audiotaping procedure was completed in the facility. This procedure was repeated six months later. Ratings of the tapes took place approximately three weeks after the second set of tapings had been completed. Administration of Follow—unguestionnaire. Approximately six months after the conclusion of the two—week training program, the Follow-up Questionnaire was administered to the individuals in attendance at a regularly scheduled staff meeting. Administration of the questionnaire took approximately one—half hour. Method of Data Analysis Conceptual Basis Stake‘s evaluation model provides two potential bases for making judgments about a training program (1972). The first, relative standards, are obtained from descriptions characterizing similar programs. Direct comparisons can be made using these descriptions. Second, absolute standards stem from opinions of reference level for back to F unavailab T character reference determin: exceeded final ju a compos Descript the fine m absolut dEpendi t0 Stak Program COmparj termed Potentj PI‘OgraI Partic 114 eference groups as to what constitutes an acceptable evel for antecedents, transactions, and outcomes (refer ack to Figure 3). If reference group standards are navailable, they must be estimated. The evaluator must decide which of the available haracteristics of other programs and/or standards of eference groups to use in making judgments. After determining whether standards have been reached or exceeded by the observed data, the evaluator makes a final judgment of the program. This is done by creating a composite rating of merit using the selected standards. Descriptive and qualifying statements often accompany the final recommendations regarding the program. Practical Considerations The decision to emphasize either relative or absolute standards is generally made by the evaluator depending upon the purpose of the evaluation, according o Stake (1972). If the purpose is to determine which rogram does the best job, relative standards (direct omparisons) will be used. This approach is generally ermed "summative evaluation." The evaluator informs otential consumers as to the comparative merits of the rogram. Evaluators wanting to know how to improve a articular program often rely on absolute standards. hey are concerned more with the process of education and with t among ante cases (rei "formative be concent gram for I Tl cerning c. at this t on other it more d PIOgrams also make cult sinc trarily 5 Similar t evaluatic Presence ConSequei aSPeCts, ing prOg; tunity t- it‘serVi logiCal 115 and with the manipulation of logical contingencies among anteCedents, transactions, and outcomes in such cases (refer back to Figure 2). This type of evaluation, "formative evaluation,’ requires evaluative efforts to be concentrated toward improving and preparing the pro- gram for later use. The need for summative evaluation research con— cerning correctional in—service training is most urgent at this time. The lack of reported evaluation material on other correctional in-service training programs makes it more difficult for facilities to choose or develop programs for themselves. The lack of relative standards also makes initial summative evaluation attempts diffi- cult since many of the standards must be somewhat arbi- trarily set rather than obtained from evaluations of similar training programs. The need for summative evaluations becomes even more critical since their resence will begin setting standards in the field. onsequently, this study emphasizes its summative spects, reporting the merits and effects of the train— 'ng program despite the limitations provided by the lack f comparison standards. At the same time, this study provides an oppor- unity to begin looking at the process of correctional n—service training. Once outcome measures are obtained, ogical contingencies can be examined with an eye toward future impr of the cont unsuccessfu acteristics The ing the dat most heavil whether th standards the null h comes. Th aspect of rence of t to the sue failure t a more fo Chapter v E transactj vidES the evalufitie intended program Variety 116 uture improvement of training. The initial examination if the contingencies that have caused successful and insuccessful outcomes constitutes the formative char— rcteristics of this study. The above—mentioned dictate the format for examin- -ng the data obtained by the investigator. The first and most heavily emphasized step is that of determining whether the intended outcomes did occur, based on the standards set for each outcome. This is done by testing :he null hypotheses related to each of the intended out— :omes. This step primarily relates to the summative aspect of the study. Then the occurrences or nonoccur— rence of the antecedents and transactions can be related to the successful production of each outcome or to its failure to be produced. This latter step constitutes a more formative aspect of the study and is included in Zhapter V. Statistical Analysis Statistical examination of the antecedents, :ransactions, and outcomes of the training program pro— rides the substance for the summative aspects of this evaluation. The congruence between that which was intended and that which occurred concerning the training >rogram (refer back to Figure 2) is indicated by a rariety of statistical procedures. Data were examine (means and : compared wi (refer back Dat this evalua sophisticat occurrence section of be tested. Hozl K There train: Test] group T Pants in knowledge tOpical a Areas Oh: this out. indiCate iStEred training 117 Data regarding the antecedents and transactions ere examined primarily via measures of central tendency means and standard deviations). These measures are ompared with the standards previously set for them refer back to Figure 1). Data concerning outcomes, the primary focus of .his evaluation, were assessed by considerably more ;ophisticated statistical methods. Investigation of the >ccurrence of these outcomes is stated in the following section of this chapter in terms of null hypotheses to >e tested. Null Hypotheses Related to Intended Outcomes m There will be no significant difference in post— training scores on the Cognitive Areas Objective Test between the in-service training participant group and the nontraining comparison group. The first intended outcome was that the partici— >ants in the training program will possess increased :nowledge of the major principles presented in each :opical area as a result of training. The Cognitive ireas Objective Test was used to collect data concerning :his outcome. Changes in this measure were inferred to .ndicate changes in this variable. The test was admin— .stered to the training participants both pre- and post— .raining. It was simultaneously administered to a comparison . one-way mul mithe post this outcom within the Figure 8). measures r; posttests : point out scores in assumption the preteg be the ca: (pre— and than the : ately. T Chaos to Monroe COunty Shelter HOme Figure 8. 118 comparison group that did not receive the training. A one—way multivariate analysis of covariance (MANCOVA) on the posttest scores was used to determine whether this outcome had occurred. within the test were used as dependent variables Figure 8). The four subscale scores (see Posttest scores were used as the dependent measures rather than difference scores between pre— and posttests for several reasons. Anderson et al. (1975)‘ point out several disadvantages of using difference scores in evaluation research. First, there is an assumption underlying the use of difference scores that the pretests of groups are equal. be the case. (pre— and post-) Second, This may or may not the reliability of the two tests used in combination is invariably lower than the reliability of either one of them used separ- ately. Third, regression effects tend to cause more chaos to difference scores than to posttest scores. Monroe County Shelter Home Figure 8. Cognitive Areas Objective Test (Post-Training) Reality Therapy Behavior Modification Communication Skills Use of Volunteers N=27 N=lO Variable matrix for group membership vs. measure relating to Hozl (Intended Outcome #l——Increased Knowledge). Pri occurrencel explore the covariates variables present jo dren were tigated by in the ste T1 at .05. \ analyses ' variables analyses .05 alpha Ho:2 \ There trai] betW1 and Outcome niques “ to that Using Cc occurrex Was inf. 119 Prior to the statistical test for the outcome ccurrence, a stepwise regression analysis was done to xplore the possible inclusion of several factors as ovariates in the MANCOVA. The independent demographic ariables of years of education, length of time at resent job, and length of experience working with chil- ren were considered for use as covariates. Also inves— igated by the regression analysis for use as covariates n the statistic test were the subscale pretest scores. The a priori alpha level for the MANCOVA was set t .05. When the MANCOVA was significant, univariate nalyses were used to explore each of the four dependent ariables. The alpha level for each of the univariate nalyses was set at .0125, which evenly distributes the 05 alpha level to each of the analyses. Ho:2 There will be no significant difference in post— training scores on the Attitudes toward Use Survey between the in—service training participant group and the nontraining comparison group. The hypothesis relating to the second intended utcome of improved attitudes toward using the tech— iques was tested for occurrence by procedures similar 0 that for Intended Outcome #1. The Attitudes toward sing Concepts Survey was used to collect data on the ccurrence of the outcome. Change on this instrument as inferred to indicate change in its respective variable. and compari after the t performed c scores. T1 were used . Figure 9). Monroe County Shelter Home Figure 9. 2 come #1, set at . analyses Variable analYSEE '05 alp} 120 variable. The survey was administered to the training and comparison groups at the same time, prior to and after the training period. Regression analyses were performed on the demographic variables and the pretest scores. The four respective subscales within the survey were used as dependent variables in the MANCOVA (see Figure 9). Attitudes toward Use of Concepts Survey (Post—Training) Reality Behavior Communication Use of Therapy Modification Skills Volunteers Monroe =27 County N Shelter N=lO Home Figure 9. Variable matrix for group membership vs. measure relating to Ho:2 (Intended Outcome #2——Improved Attitudes toward Use of Concepts). As with the hypothesis relating to Intended Out- come #1, the a priori alpha level for the MANCOVA was set at .05. When the MANCOVA was significant, univariate analyses were used to explore each of the four dependent variables. The alpha level for each of the univariate analyses was set at .0125, which evenly distributes the .05 alpha level to each of the analyses. Ho:3a There w trainin between and the Ho:3b The ine in the: that t] help t< after ' Th members wi quent yout on this or was admini Participa] same time retically Were used Sis of CC rElating 0f the Ag thOSe dox to deter] alpha 1e. third in QueStiOn 121 Ho:3a There will be no significant difference in post- training scores on the Critical Incidents Test between the in-service training participant group and the non—training comparison group. Ho:3b The in-service training participants will indicate in their responses to the Follow-up Questionnaire that the in—service training was of little or no help to them in their work during the six months after training. The third intended outcome was that the staff members will have an improved ability to work with delin- quent youth. Two methods of collecting data reflecting on this outcome were used. The Critical Incidents Test was administered pre- and post—training to the training participant group and to the comparison group at the same times (see Figure 10). Scores on this test theo- retically infer ability levels. The post-training scores were used as a dependent variable in a univariate analy- sis of covariance (ANCOVA) to test the null hypothesis relating to this outcome. Prior to the implementation of the ANCOVA, stepwise regression analyses similar to those done on the previous two outcomes were performed to determine covariates for the ANCOVA. An a priori alpha level of .05 was set for the ANCOVA. An additional source of data reflecting on the third intended outcome was obtained from the Follow-up Questionnaire. One question within the questionnaire surveyed pa gained in t Responses t perceived c youth. A I lated. A a four—poi the group‘ to indicat their work Monroe County Shelter Home Figure 1C Li The : will of t samp 122 surveyed participant Opinions regarding whether knowledge gained in the training had helped them in their work. Responses to this question are inferred to reflect self— perceived change in ability to work with delinquent youth. A mean of responses to the question was calcu— lated. A 3.0 (equivalent to "agree" on the question) on a four-point scale was set as a standard mean score for the group's responses to the question. This was inferred to indicate that they felt the training had improved their work abilities. Critical Incidents Test (Post—Training) Monroe _ County N—27 Shelter _ Home N-lO Figure 10. Variable matrix for group membership vs. measure relating to Ho:3a (Intended Outcome #3—-Improvement in Ability). Ho:4a The ratings of the Staff Observation Rating Scale will indicate neutral or inappropriate application of the techniques presented in training in work samples obtained after training. Ho:4b The in—service training participants will indicate in their responses to the Follow-up Questionnaire that they have seldom or never applied in their work during the six months after training information pre- sented in the in-service training sessions. The participant training p1 relating tc the Staff < Questionna Th raters tra by the two on a sever tained wit as a minir were, in ; learned 1; Obtained POSitiVe nentral r Mean rati aSSumed t modifica1 response: members' the reSp asSumed therapy. fourth i 123 The fourth intended outcome was that the training participants will use the information learned in the training program. Data reflecting on the hypotheses relating to this outcome were obtained from two sources: the Staff Observation Rating Scale and the Follow-up Questionnaire. The audiotaped observations were rated by two raters trained in the rating procedure. A mean rating by the two raters of 5.0 or above (4.0 equals neutral) on a seven-point rating scale of appropriateness con— tained within the Staff Observation Rating Scale was set as a minimum standard to indicate that the staff members were, in fact, appropriately using the information learned in the training program. Mean ratings were obtained for each of the four staff behavior categories: positive reinforcement responses, listening responses, neutral responses, and responses to problematic behavior. Mean ratings on the positive reinforcement category were assumed to reflect on the staff members' use of behavior modification techniques. Mean ratings on the listening responses category were assumed to reflect on the staff members' use of communication skills. Mean ratings on the responses to problematic behavior category were assumed to reflect on the staff members' use of reality therapy. A further source of data reflecting on the fourth intended outcome was provided by the Follow—up Questionna surveyed p had applie during the this quest not the pa A mean of 3.0 (equi‘ set as a : responses indicate training H0:52 In-se neut: EXpe: thei Surv Ho:5 In-s neut six Fol] Pants w; trainin. relatin the Sat Questio 124 Questionnaire. One question within the questionnaire surveyed participant opinions regarding whether they had applied knowledge gained in training in their work during the six months after training. Responses to this question are inferred to reflect on whether or not the participants had been using training information. A mean of responses to the question was calculated. A 3.0 (equivalent to "occasional use" on the question) was set as a minimum standard mean score for the group's .responses to the question. This would be inferred to indicate that they felt that they had been using the training information. Hoz5a In-service training participants will demonstrate neutral or negative attitudes about the training experience itself immediately after training in their responses to the Satisfaction with Training Survey. Hoz5b In—service training participants will demonstrate neutral or negative attitudes about the training, six months later, in their responses to the Follow—up Questionnaire. The fifth intended outcome was that the partici- pants will demonstrate positive attitudes about the training program. Data reflecting on the hypotheses relating to this outcome were obtained from two sources: the Satisfaction with Training Survey and the Follow-up Questionnaire. Te in the Sat to attitué after trai with a vaJ could reS] the secti: attitudes responses of 3.0 on response) attitudes training. I in the F< t0 attit A mean 0 A standa the most indicate Six mont m The: tra, SUr' gro 125 Ten questions containing a total of 34 sections in the Satisfaction with Training Survey were related to attitudes about the training program immediately after training. While the sections concerned themselves with a variety of aspects of training that participants could respond to differentially, a composite of all the sections theoretically reflected on the participants' attitudes toward the training as a whole. A mean of the responses to the questions was calculated. A standard of 3.0 on a four-point scale (4.0 is the most positive response) was set as a standard, inferring positive attitudes on the part of participants immediately after training. Four questions containing a total of 10 sections in the Follow—up Questionnaire were believed to be related to attitudes about the training program six months later. A mean of the responses to the questions was calculated. A standard of 3.0 on a four—point scale (4.0 again being the most positive response) was set as a standard to indicate positive attitudes on the part of participants six months after training. Null Hypothesis Related to Additional Unintended Outcome Additional Hozl There will be no significant difference in post- training scores on the Correctional Preference Survey between the in—service training participant group and the nontraining comparison group. In are direct] Hus additi in order t< attitudes ' reflecting Preference training t comparison four Behav erences of staff mem} scores we] a“Priori . Monroe County Ingham County Figure 1: for the 126 In addition to the previous null hypotheses which are directly related to the intended outcomes of training, this additional null hypothesis was established and tested in order to investigate the effects of training on staff attitudes toward types of delinquents. The source of data reflecting on this unintended outcome was the Correctional Preference Survey which was administered pre- and post— training to the training group participants and to the comparison group at the same times (see Figure 11). The four Behavior Category subscales on this test infer pref- erences of types of delinquents with which correctional staff members enjoy working. The post—training subscale scores were used as dependent variables in a MANCOVA. An a-priori alpha level of .05 was set for the MANCOVA. Correctional Preference Survey (Post—Training) BC—l BC—2 BC—3 BC—4 Monroe _ County N_27 Ingham _ County N"lO Figure 11. Variable matrix for group membership vs. measure relating to additional Hozl (Unin— tended Outcome #1 shifts in attitudes toward types of delinquents). Prior to the implementation of the MANCOVA, step- wise regression analyses similar to those done on previous outcomes were performed in order to determine covariates for the MANCOVA. In the event the MANCOVA was significant, 1mivariate of each of alpha level set at .012 to each of The reader witf relevant c discussed. Monroe Cor ing Progre 0f demogra for the 5+ described unique fe treatment this Chap Stake's u in terms COmes. 5 ehcourag. SOUrces. 127 univariate analyses would be used to explore the influence of each of the four dependent variables. The a priori alpha level for each of the univariate analyses would be set at .0125, which evenly distributes the .05 alpha level to each of the four analyses. Summary The purpose of this chapter was to provide the reader with a description of the study. Each of the relevant components of the design was described and discussed. The subjects of the study, employees of the Monroe County Youth Center who participated in the train- ing program, were described with respect to a variety of demographic characteristics. The particular setting for the study, the Monroe County Youth Center, was described with regard to its purpose, functions, and unique features. The training program, which is the treatment intervention for the study, was presented in this chapter according to the framework provided by Stake's model of evaluation. The program was described in terms of intended antecedents, transactions, and out- comes. Stake's model, which was described in Chapter II, encourages the collection of data from a variety of sources. The methods of observation that were designed or obtained for this study were described in this chapter, with respec measures, v Th< mentation . methods of and after and perfor prior to t training ; were also viduals w] Observatiq dence wer ing prOgr attitudin Participe 1 0f Obserx to the f: 0f the t' Plished 0f centr reSUIts 128 with respect to their source, purpose, and reliability measures, when appropriate. The section on evaluation procedures and imple- mentation in this chapter described how and when the methods of observation were applied, prior to, during, and after the training program. Cognitive, attitudinal, and performance measures were administered to participants prior to the training program and then again after the training program had been completed. These measures were also administered to a comparison group of indi— viduals who did not participate in the training program. Observations of participants' work with youths in resi— dence were made during the several weeks after the train— ing program and then again six months later. A follow-up attitudinal measure was also administered to the training participants six months after training. The data collected by the above—mentioned methods of observation were used to test six hypotheses relating to the five intended outcomes and one unintended outcome of the training program. The data analysis was accom- plished by means of MANCOVA’s, an ANCOVA, and measures of central tendency compared to a priori standards. The results of the study are reported in Chapter IV. An ses that r training p However, t cedents a1 Prior to i Stake's e‘ 0f intend lection 0 evaluatio OCCurrenc exClusior Or treat: Presenti] ChrOnOlo. emphasis avoided . Cified i CHAPTER IV ANALYSIS OF THE DATA Introduction Analysis of the data evaluating the null hypothe— ses that relate to the five intended outcomes of the training program is of primary concern to this chapter. However, the analysis of data collected on intended ante— cedents and transactions is presented in this chapter prior to the analyses to test the null hypotheses. Stake's evaluation model encourages the specification of intended antecedents and transactions, and the col— lection of data reflecting on them, for a purpose. Often, evaluation and research studies tend to emphasize the occurrences of training or treatment effects to the exclusion of close consideration of whether the training or treatment had been conducted exactly as planned. By presenting the results of the analyses in Stake's logical chronological order, it is believed that an under— emphasis of the intended antecedents and transactions is avoided. Each intended antecedent and transaction is spe— cified in this chapter. The data for each antecedent 129 and transac dency or de measures 0: intended a] to the a p Th analyzed h ing on Int of multiva occurrenca a univari Outcomes if obtain Previousl Changes 1‘ Participa MANCOVA. null hyp 0f testi and Circ in Order the 10 5 gram an( OCcurre 130 and transaction are reported by measures of central ten- dency or description, whichever is appropriate. These measures of central tendency or description for each intended antecedent and transaction are then compared to the a priori standards set. The data reflecting on the intended outcomes are analyzed by several statistical procedures. Data reflect- ing on Intended Outcomes l and 2 are analyzed by means of multivariate analysis of covariance (MANCOVA). The occurrence of Intended Outcome 3 is tested by using a univariate analysis of covariance (ANCOVA). Intended Outcomes 4 and 5 are tested for occurrence by determining if obtained data means compare favorably with standards previously set. An additional unintended outcome of changes in preference of type of delinquent with which participants enjoy working is tested by means of a MANCOVA. For each intended outcome, the corresponding null hypothesis is stated prior to reporting the results of testing. Intended Antecedents Antecedent events are those conditions, situations, and circumstances whose presence is considered necessary in order for learning to occur. The following are the 10 specified antecedents for this training pro— gram and the analyses of the data reflecting on their occurrence . l. The gran towa Data obtained by istered as pretest ins istered to results are both weeks of six sect the partic; sessions t< (12 througj Th interest i A "three" interest 5 interest j interest ; at least : positive : attitude dard devi pants! re 131 1. The staff members enrolled in the training pro— gram will have an initial favorable attitude toward receiving training. Data reflecting on this antecedent condition were obtained by the Anticipation of Training Survey (admin— istered as the Attitudes Toward Training section of the pretest instrument) in two parts. The survey was admin— istered to all 40 of the participants in training, but results are reported here only for the 27 who received both weeks of training. Three items containing a total of six sections (8a, b, d, e; 10; and 11) reflected on the participants' personal interest in the training sessions to follow. Six items, of one section each (12 through 17), reflected on the participants' theoreti— cal attitudes toward training in general. The items reflecting on the participants' personal interest in training were keyed on a three—point scale. A "three" response for the items represented the greatest interest in training. A "two" response indicated moderate interest in training. A "one response indicated no interest in training. A standard of the group scoring at least 2.0 (equal to or above that of a moderately positive response) had been set to signify a positive attitude toward training. A mean of 2.77, with a stan- dard deviation of .210, was obtained for the 27 partici- pants' responses to all of the six sections which indicate th Therefore, The retical att on a four—} represente( A "three" attitude t a moderate “one“ resp toward tre score equ. a moderat attitude with a st 27 Partic theoreti< fore! th. Pants' p attitude of the P each se< fact, he the tra trainin 132 indicate the degree of personal interest in training. Therefore, this standard was met and exceeded. The items reflecting on the participants' theo- retical attitudes toward training in general were keyed on a four-point scale. A "four" response on the items represented the most positive attitude toward training. A "three" response represented a moderately positive attitude toward training. A "two" response represented a moderately negative attitude toward training and a "one" response represented a more negative attitude toward training. A standard had been set that a group score equal to or above 3.0 (equal to or above that of a moderately positive response) would indicate a positive attitude toward training, in general. A mean of 3.37, with a standard deviation of .377, was obtained for all 27 participants' responses to all six items that indicate theoretical attitude toward training in general. There— fore, this standard was met and exceeded. In summary, data were collected on the partici- pants' personal interest in the impending training and attitudes toward training in general by the two sections of the Anticipation of Training Survey. The data from each section indicated that the participants did, in fact, have positive attitudes about being involved in the training experience themselves and about in-service training in general. This intended antecedent is, therefore, start of t] 2. Te st De indicate 1 for the 51 than the intended would occ ducted on day the r weeks la1 cate the cussion: Small g small 9 each of 133 therefore, indicated to have been in existence at the start of the training sessions. 2. Ten instructional days will be provided for the staff during the course of the training. Data recorded on the Anecdotal Record Forms indicate that only seven days of training were provided for the staff members during the training program, rather than the intended 10. In addition, an assumed, unrecorded intended antecedent was that the two weeks of training would occur together. Instead, the training was con— ducted on three consecutive days the first week, one day the next week, and on three consecutive days seven weeks later. 3. Facilities will be available during the training sessions for both large and small group dis— cussions. Data recorded on the Anecdotal Record Form indi— cate that facilities were available for large group dis— cussions. Separate rooms were not readily available for small group discussions, but the large room accommodated small group discussions adequately in the opinion of each of the training presenters. 4. Instructional training sessions will remain uninterrupted. De indicate 1 20 hours I minor suc' room. On facility impromptu considere Insurance day's se: caused a session. first f< were en: ing. T days of each se dUring 22.95 i AneCdoi 134 Data recorded on the Anecdotal Record Form indicate that a total of 11 interruptions occurred during 20 hours of training. Nine were considered extremely minor such as individuals coming into or leaving the room. One interruption was considered moderate. The facility administrator brought in soft drinks and an impromptu break was called. The final interruption was considered to be a major one by the training presenters. Insurance agents speaking to the group caused the final day's session to be started one hour late, which in turn caused a drastic shortening of the scheduled summary session. 5. At least 85% of the staff persons will be in attendance at each session. A total of 40 participants were enrolled for the first four days of training. A total of 27 participants were enrolled for the second set of three days of train— ing. The standard for attendance during the first four days of training is that 34 participants would attend each session (85% of 40). The standard for attendance during the second set of three days of training is that 22.95 participants would attend each session (85% of 27). The results of attendance data recorded on the Anecdotal Record Form are reported in Table 2. Day A .bL’JNH T the first sessions. Standard a number Even if was not Plied a1 WEek of for the iUdivid 135 Table 2 Training Session Attendance Figures Intended Antecedent #5 % of Standard Day Attendance Standard Total Attained l 40 34 100 yes 2 40 34 100 yes 3 35 34 87.5 yes 4 31 34 77.5 no 5 20 22.95 74.1 no 6 16 22.95 59.3 no 7 20 22.95 74.1 no The standard for attendance was attained during the first three sessions and was not for the last four sessions. 6. Only staff members of the facility will be in attendance with not more than 25 participants enrolled. The Anecdotal Record Form data indicate that the standard for this intended antecedent was set at too low a number since there are 32 staff members of the facility. Even if the standard were set at 32, however, the standard was not met since the probation staff of the county sup- plied an additional eight participants during the first week of training, bringing the total enrollment to 40 for the first week. During the second week only those individuals who were staff members of the facility and had direct fixing the 7. Th us all D; dition we the train tered the within tr eight see the degre material administ ticipant had left Sections t informat userI ; 0f the With Tr A "four POSitiV 136 had direct, routine contact with residents were enrolled, fixing the enrollment at 27. 7. The training sessions will provide practical and useful child management and treatment techniques and concepts for the staff members. Data reflecting on this intended antecedent con- dition were obtained from two sources. At the end of the training sessions the 27 participants were adminis- tered the Satisfaction with Training Survey. Included within that survey were two items containing a total of eight sections (2a, b, c, d; 4a, b, c, d) reflecting on the degree of perceived usefulness of the training material. In addition, the Follow—up Questionnaire administered six months after training to the 20 par— ticipants remaining in employment at the facility (seven had left) included two items containing a total of eight sections (3a, b, c, d; 4a, b, c, d) asking whether the information had been applied and whether it had been useful in their work. The responses reflecting on perceived usefulness of the training material included in the Satisfaction with Training Survey were keyed on a four—point scale. A "four" response on the items represented the most positive perception of usefulness of the training material. Two questions were posed to the participants. ‘2' ",7 kg First, par they felt of the for asked to 5 in each 0: T? keyed on two items the train whether t during t1 asked whe work dur Satisfac Question ing infc equal t, to the e of trai ceived trainin this st snestic and tw, 137 First, participants were asked to indicate how useful they felt the information presented in training in each of the four main topical areas was. Second, they were asked to indicate whether they will use the information in each of the four main topical areas in their work. The Follow-up Questionnaire items were also keyed on a four—point scale. A "four" response on these two items represented the most positive experience with the training information. The first question asked whether they had applied the information in their work during the six months after training. The second question asked whether the information had been helpful in their work during the six months after training. The results of the responses to the items in the Satisfaction with Training Survey and the Follow-up Questionnaire reflecting on the usefulness of the train— ing information are reported in Table 3. An a priori minimum standard of mean responses equal to or greater than 3.0 of the participant group to the questions reflecting on this intended antecedent of training was set to indicate at least moderate per— ceived usefulness of the training information by the training participants. It can be seen in Table 3 that this standard was met for all four questions (two questions in the Satisfaction with Training Survey and two questions in the Follow—up Questionnaire) for Group Me Topicai Area Reality Therapy Behavior Modificat Communice Skills Use of Voluntee; \ 138 Table 3 Group Means and Standard Deviations on Survey Questions Relating to Perceived Usefulness of the Training Information Intended Antecedent #7 Satisfaction with Follow-up Topical Training Survey Questionnaire Area Will Have Has Been userI Use Applied Useful Reality mean 3.05* 3.65* 3.40* 3.05* Therapy S.D. .945 .487 .754 .780 Behavior mean 3.43* 3.58* 3.20* 3.10* Modification S.D. .676 .504 .894 .641 Communication mean 3.o4* 3.65* 3.17* 3.00* Skills S.D. .878 .573 .786 .795 Use of mean 3.05* 2.77 1.70 1.75 Volunteers S.D. .848 .869 .801 .851 * Met or exceeded 3.0 standard. three of t1 Reality The Skills wer informatio was seen a not meet t (immediate later), 0: (six mont I compared intended therapy municati Voluntee includir areas we ing SUI 139 three of the four tOpical areas. The tOpical areas of Reality Therapy, Behavior Modification, and Communication Skills were all perceived as having provided useful information. The topical area of the Use of Volunteers was seen as "useful" immediately after training but did not meet the standards set to indicate it would be used (immediately after training), had been used (six months later), or had been helpful in the participants' work (six months later). 8. Each tOpical area will be presented in its entirety. Data collected on the Anecdotal Record Form, when compared to the intended transactions, reflect on this intended antecedent. Behavior modification and reality therapy were covered as intended. One session on com- munication skills was drOpped, however. The use of volunteers was not covered as intended. A session including a summary presentation of all of the tOpical areas was drastically shortened. 9. The training staff will be proficient in pre— senting the topical information. Data reflecting on this antecedent condition were obtained from four items in the Satisfaction with Train— ing Survey containing a total of 10 sections (3; 4b, to t) ing seve Sur\ to \ and dis tai 140 c, d, e; 7b, 0, d, e; 8). This survey was administered to the 27 training participants at the end Of the train- ing sessions. The items asked the participants to respond to several issues related to training staff proficiency. Survey items 3 and 8 asked the participants to indicate to what degree their questions were adequately answered and to what extent there was sufficient Opportunity for discussion during training. Items 7 and 10 each con— tained four sections. They asked to what degree the training presenters were sufficiently expert in each Of the major tOpical areas and how well the training pre— senters had held the participants' interest in each Of the areas during training. Each of the 10 sections was keyed on a four- pOint scale. A "four" response on each represented the most positive perception by the participant. Mean responses Of 3.0 or more (equivalent to or greater than a moderately positive individual response) were set as standards for data reflecting on this intended antecedent. The results Of responses to these sections of the survey are summarized in Table 4. The standards were met and exceeded for the responses indicating that questions were adequately answered and that sufficient Opportunity for discussion was provided. The standards were met and exceeded for responses indicating that the 141 Table 4 Group Means and Standard Deviations on Responses to Satisfaction with Training Survey Items Relating to Perceived Proficiency Of the Training Presenters Intended Antecedent #9 Survey Question Mean Segiiiign Questions Adequately Answered 3.36* .581 Sufficient Opportunity * for Discussion 3.41 .666 Trainers Sufficiently Expert in: * Reality Therapy 3.32* .568 Behavior Modification 3.55* .510 Communication Skills 3.14 .640 Use Of Volunteers 2.76 .995 Trainers Held Interest in: * Reality Therapy 3.14* .640 Behavior Modification 3.41 .503 Communication Skills 2.91 .668 Use Of Volunteers 2.77 .869 Total (Each section from above * considered equally) 3.25 .367 * Met or exceeded 3.0 standard. 142 training presenters were perceived as being sufficiently expert in the topical areas of Reality Therapy, Behavior Modification, and Communication Skills; but the standard was not met for the topical area of the Use Of Volunteers in Corrections. The standards were met for responses indi- cating that the trainers held the participants' interest in the topical areas of Reality Therapy and Behavior Modification but not in the topical areas of Communication Skills and the Use Of Volunteers in Corrections. An estimation Of the participants' perception of the overall proficiency of the training staff is provided by the total mean Of all 10 sections. The standard of 3.0 was met and exceeded by this mean. In summary, data reflecting on the perceived pro- ficiency of the training staff were obtained by items in the Satisfaction with Training Survey. The data col— lected and analyzed indicated that, in general, the par— ticipants perceived the training staff as having been proficient in presenting the topical information. Several specific areas within that overall positive perception were regarded less positively by the training participants, however. While the trainers were regarded as being expert in the other topical areas as indicated by mean responses exceeding 3.0, this was not demon— strated regarding their perceived expertise on the Use of Volunteers in Corrections. While the survey responses 143 indicated that the trainers held the participants' interest in the other two areas, this was not demon- strated for the topical areas of Communication Skills and the Use Of Volunteers in Corrections. 10. Each staff person will have at least some con— tact with children in residence during the second week of the training program. Data reflecting on this antecedent condition were Obtained during the administration of the post—instrument at the end of the training sessions. A section entitled Training History was included in the Satisfaction with Training Survey. Each participant was specifically asked among other questions if he or she had had direct contact with youths in residence during the second week of train— ing. Twenty of the participants responded that they had engaged in such contact and seven indicated that they had not. The standard had been set that all of these individuals should have at least some contact with resi— dent youths. Therefore, the standard for the demonstra— tion Of the occurrence of this intended antecedent was not met. Intended Transactions Transactions are those activities and inter~ actions during training that are intended to bring about 144 the desired training outcomes. The following are the specified transactions of this training program and the description of the data reflecting on their occurrence. 1. Training will take place during one three—hour session each working day for two weeks. Data collected on the Anecdotal Record Form indicate that the training took place during three three— hour sessions the first week, during one three—hour session the next week, and during two three-hour sessions and a two-hour session seven weeks later. The intended transaction took place in a greatly modified, and some— what reduced, form. 2. During the first week of training (pre-service): (a) The first day will include one—half hour of introduction by Dr. Ernest Shelley, admin- istration of the pretest instrument for one hour, and one hour of didactic presen— tation on the subjects of the use of volun- teers and the psychological aspects of adolescence. Data recorded on the Anecdotal Record Form indi- cate that the transactions for the first day occurred as intended except that the subject of the use of volun— teers was not covered. 145 (b) Reality Therapy will be the subject of the second day. There will be 1% hours of didactic presentation, one hour of small group discussion, and one—half hour of large group discussion. Data collected on the Anecdotal Record Form during the second day of training indicate that reality therapy was the main subject of the second day. Trans- actions, however, consisted of three hours of didactic presentation rather than including one hour of small group discussion and one-half hour of large group dis- cussion. The topic of the use of volunteers in cor— rections was also integrated into this didactic presen— tation for approximately one—quarter of an hour. (c) Behavior Modification will be covered the third day. Transactions will include one hour of lecture, one hour of demonstration, and one hour of discussion. Data collected on the Anecdotal Record Form during the third day of training indicate that behavior modifi— cation was the subject of the third day. Transactions, however, consisted of two hours of didactic presentation, one-half hour of demonstration, and one—half hour of discussion. 146 (d) The topical area of communication skills will be introduced on the fourth day. Transactions will include one—half hour of introduction and didactic presentation, one and one—half hours of small group exercises and one—half hour of large group discussion. The last half hour will be used for obtaining feedback from the participants concerning what they feel would be the most beneficial emphasis of training during the following week. Data collected on the Anecdotal Record Form indi- cate that the topical area of communication skills was the subject of the fourth day of training. The trans— actions occurred as intended. (e) Transactions involving the topic of communi— cation skills will be available for the fifth day of training but abandoned if the staff members, on the fourth day, express a strong preference for other activities. Data collected on the Anecdotal Record Form indi- cate that during the last half hour of the fourth day participants expressed an interest in more training on reality therapy for the fifth day. Consequently, trans— actions on the topic of communication skills were 147 abandoned and transactions on the topic of reality therapy were presented instead. Transactions included one-half hour of didactic presentation, 1% hours of viewing and discussing videotapes on reality therapy, and one hour of combined presentation and discussion on reality therapy. 3. During the second week of training (in—service): A minimum of transactions will be planned prior to consulting with the staff during the first week of training (pre—service). Data collected on the Anecdotal Record Form indi— cate that the "second week of training" actually con— sisted of the fifth, sixth, and seventh days of training. The transactions covered during the fifth day of training are described in the previous section (Intended Trans— actions 2e). On the sixth day, transactions relating to behavior modification were provided. Three hours of com- bined didactic presentation and discussion occurred. On the seventh day of training, one-half hour of summary regarding the training topics was presented in a didactic fashion, a group summary discussion was undertaken for one—quarter of an hour, and the posttest was administered for one hour. 148 Intended Outcomes Intended outcomes are the anticipated results or outputs of a training program. The following are the five specified intended outcomes of this training program. These are accompanied by the corresponding null hypothe— ses. The analysis of the data reflecting on each null hypothesis then follows. Intended Outcome #1 The participating staff members will possess increased knowledge of the principles presented in the major topical areas of the training program. m There will be no significant difference in post— training scores on the Cognitive Areas Objective Test between the in-service training participant group and the nontraining comparison group. The null hypothesis Hozl is rejected based on the results of the MANCOVA statistical procedure performed (E = 4.34, p i .008). The only univariate analysis proving to be significant, however, was for the topical area of behavior modification (E = 16.60, p i .000). Data reflecting on this null hypothesis were obtained through administration of the Cognitive Areas Objective Test. This instrument was administered pre— and post—training to the 27 Monroe County Youth Center participants who have routine contact with youths in 149 residence. It was also administered at approximately the same times to a comparison group of 10 individuals employed at the Ingham County Shelter Home. This was done to control, to a limited extent, for the effects of history, maturation, and testing (Campbell & Stanley, 1967). The Cognitive Areas Objective Test contains 40 multiple—choice items. These items are divided into four topical area subscales as follows: 12 items for reality therapy, nine items for behavior modification, 13 items for communication skills, and six items for use of volun- teers in corrections. A one—way MANCOVA statistical procedure was used to test for the rejection of the null hypothesis. The dependent variables used in the analysis were the four subscale scores obtained on the post—training adminis— tration of the Cognitive Areas Objective Test. Prior to examination of the MANCOVA, the relationship was explored between the set of four dependent variables (post—training scores on the four topical area subscales) and a set of seven independent variables including three demographic variables (education, experience, and time at present job) and the four pre-training subscale scores. A regression analysis was performed to determine which of the demographic variables or pre-training scores should be used as covariates in the MANCOVA. 150 The overall multivariate test of the hypothesis of no association between the four dependent variables (post-training subscale scores) and the seven independent variables (three demographic and four pre-training sub— scale scores) indicated that a significant relationship did exist (E = 2.46, p i .001). A stepwise regression analysis was then used to determine the contribution of each of these independent variables to that relationship. The results of the stepwise regression analysis are pre— sented in Table 5. Table 5 Stepwise Regression E Statistics for the Seven Possible Covariates on the Set of Four Post-Training Subscales of the Cognitive Areas Objective Test Intended Outcome #1 Covariate E E< Pre-training Cognitive Areas Test Subscales Reality Therapy 4.56 .006 Behavior Modification 4.82 .004 Communication Skills 2.13 .108 Use of Volunteers 1.41 .259 Time on Job 0.61 .661 Experience with Children 0.81 .154 Education 0.89 .485 As a result of the stepwise regression analysis, five of the independent variables were eliminated for possible use as covariates in the MANCOVA. Eliminated 151 were the three demographic variables and the pre—training subscales for communication skills and the use of volun— teers in corrections. Left for possible use as covariates were the pre—training subscales for reality therapy and behavior modification. In order to explore whether the pre-training behavior modification subscale should be eliminated for consideration of use as a covariate, the order between it and the pre—training reality therapy subscale was reversed and another multivariate analysis was undertaken. This had to be done because reality therapy could not be examined for use as a covariate in the previous stepwise regression analysis since it followed behavior modifi— cation which was significant. As a result, the test for significance of the behavior modification pre-training subscale was confounded by the significance of the reality therapy pre—training subscale since these two 3 tests are not independent. This multivariate analysis tested the relationship between the four dependent variables and these two remaining independent variables. The results of this analysis indicated that a significant relation— ship exists (E = 4.61, p i .0003). The reversed order of the two variables was then used in a stepwise regression analysis to examine the contribution of each independent variable to this significant relationship. The results of this analysis indicate that the pre-training subscale 152 score on behavior modification should also be included as a covariate in the multivariate analysis of covariance (g = 4.91, p i .004). The group means on the two pre—training subscale scores used as covariates are presented in Table 6. Table 6 Pre—Training Group Means on the Two Covariates Used in the MANCOVA for Intended Outcome #1 Covariate Monroe County Ingham County Youth Center Shelter Home Pre—Training Cognitive Areas Test Subscales Reality Therapy 5.88 5.90 Behavior Modification 5.38 4.20 In spite of the lack of apparent difference between the groups on reality therapy pre—training sub— scale means, this subscale was included, along with the behavior modification pre—training subscale, for use as a covariate in the MANCOVA as a result of the stepwise regression analysis. These two variables were retained as covariates in the analysis because they might have given a false impression of significance if they were not controlled for in some manner. Their retention as Covariates adjusted the post-training scores for influence by the pre—training scores. 153 The regression analysis procedure indi- cated that use of two covariates in the MANCOVA was necessary. The two covariates retained were the pre- training subscale scores for reality therapy and behavior modification. The regression analysis also indicated that five variables could be justifiably dropped from use as covariates in the MANCOVA. The variables not included as covariates were the demographic variables of education, experience, and length of time at present job, and the pre—training subscale scores for communication skills and use of volunteers in corrections. A one—way MANCOVA was used to test for the rejec— tion of the null hypothesis. The alpha level for sig— nificance was set at .05. The four post—training sub- scale scores within the Cognitive Areas Objective Test were used as dependent variables in the analysis. The training and nontraining settings were used as independent variables. In the event that the MANCOVA was significant, univariate analyses would be used to explore the influence of each of the four dependent variables. The alpha level for each of the univariate analyses would be set at .0125 which evenly distributes the .05 alpha level to each of the four analyses. The overall MANCOVA test of association between the settings and the set of four dependent variables indicated a significant difference exists between the 154 two groups (3 = 4.34, p i .008). This significant dif- ference occurred even after pre-existing differences were controlled for by use of two covariates in the analysis. In order to investigate the effects of each of the subscales on the overall MANCOVA, four univariate analyses were done. The results of these analyses are presented in Table 7. Table 7 Univariate 3 Statistics for the Four Dependent Variables Intended Outcome #1 Dependent Mean Univariate df < Variable Square E —— B — Reality Therapy 15.0756 6.80 1,30 .014 Behavior Modification 25.8970 16.60 1,30 .000* Communication Skills .7453 .18 1,30 .671 Use of Volunteers .0028 .00 1,30 .965 *p < .0125 The univariate analyses indicate that the behavior modification subscale post-training variable is the major contributor to the difference between the groups indi— cated by the overall MANCOVA. A least square estimate, adjusted for covariates, indicates that the adjusted estimate of difference between the two groups on the behavior modification post-training subscale was 2.01 in favor of the Monroe County group. 155 In summary, the null hypothesis relating to the intended outcome of the participating staff members pos- sessing increased knowledge of the major principles pre- sented in each topical area is rejected. The significant overall MANCOVA indicated that learning had occurred to a statistically significant degree. However, upon uni— variate analysis, only the behavior modification subscale proved to be a statistically significant contributor to the overall significance obtained. Intended Outcome #2 The staff members will have improved attitudes toward using: (a) reality therapy techniques (b) behavior modification techniques (0) communication techniques (d) volunteers in corrections There will be no significant difference in post- training scores on the Attitudes toward Using Con- cepts Survey between the in-service training par- ticipant group and the nontraining comparison group. The null hypothesis Ho:2 is not rejected based on the results of the MANCOVA statistical procedure per- formed (3 = .88, p i .488). Data reflecting on this null hypothesis were obtained through administration of the Attitudes toward Using Concepts Survey. This 156 instrument was administered pre- and post-training to the 27 Monroe County Youth Center participants who have routine contact with youths in residence. It was also administered at approximately the same times to a com— parison group of 10 individuals employed at the Ingham County Shelter Home. As with the Cognitive Areas Objective Test, this was done in order to control, to a limited extent, for the effects of history, maturation, and testing (Campbell & Stanley, 1967). The Attitudes toward Using Concepts Survey con- tains 16 statements to which the respondent is asked to choose one of four responses from "strongly agree" to "strongly disagree." Five items are concerned with use of reality therapy, three items are concerned with behavior modification, three with the use of communi- cation skills, and five items with the use of volunteers in corrections. The statistical procedure used to test for the rejection of this null hypothesis was similar to that used to test for Ho:l. A one-way MANCOVA was used with the dependent variables being the four post—training subscale scores on the Attitudes toward Using Concepts Survey for each of the two groups. Prior to the exami- nation of the MANCOVA, a regression analysis was done to determine which of the independent variables should be included as covariates. Considered for inclusion 157 were the demographic variables of education, experience, and time at present job, as well as the four pre—training subscale scores. The overall multivariate test of the hypothesis of no association between the four dependent variables and the seven independent variables indicated that a significant relationship did exist (E = 2.46, p i .001). A stepwise regression analysis was then used to determine the contribution of each of these independent variables to that relationship. The results of the stepwise regression analysis are presented in Table 8. Table 8 Stepwise Regression E Statistics for the Seven Possible Covariates on the Set of Four Post—Training Subscales of the Attitudes toward Using Concepts Survey Intended Outcome #2 Covariate In In IA Pre—Training Attitudes toward Using Concepts Survey Subscales Reality Therapy 6.98 .001 Behavior Modification 1.47 .241 Communication Skills 4.30 .008 Use of Volunteers 1.72 .178 Time on Job 2.15 .106 Experience with Children .87 .495 Education .82 .525 As a result of the stepwise regression analysis, five of the independent variables were eliminated for possible use as covariates in the MANCOVA. Eliminated 158 were the three demographic variables and the pre-training subscales for behavior modification and use of volunteers in corrections. Left for possible use as covariates in the analysis were the pre-training subscales for reality therapy and communication skills. The reality therapy subscale qualifies for use as a covariate, but the pre- training subscale for communication skills remained to be explored because of its order in the stepwise regression analysis. In order to investigate whether the pre—training communication skills subscale should be eliminated for consideration of use as a covariate, the order between it and the pre-training reality therapy subscale was reversed and another multivariate analysis was performed. This analysis tested the relationship between the four dependent variables and these remaining two independent variables. The results of this analysis indicated that a significant relationship exists (E = 5.48, p i .0001). The reversed order of the two variables was then used in a stepwise regression analysis to determine if the pre-training communication skills subscale should be considered for use as a covariate in the MANCOVA in addition to that for reality therapy. The results of this analysis indicate that it also should be included as a covariate (E = 6.42, p i .001). 159 The group means on the pre—training subscale scores for these two covariates are presented in Table 9. Table 9 Pre—Training Group Means on the Two Covariates Used in the MANCOVA Intended Outcome #2 Covariate Monroe County Ingham County Youth Center Shelter Home Pre-Training Attitudes toward Using Concepts Survey Subscale Reality Therapy 3.29 2.84 Communication Skills 3.51 3.30 These two variables were retained as covariates in the analysis because they might give a false impression of significance if they were not controlled for in some manner. Their retention as covariates adjusted the post- training scores for influence by these pre-training scores. In summary, a regression analysis procedure indi- cated that use of two covariates in the MANCOVA was appropriate. The two covariates retained were the pre- training Attitudes toward Using Concepts Survey subscales for reality therapy and communication skills. The regression analysis also indicated that five independent variables could be justifiably dropped from consideration as covariates in the MANCOVA. The variables not included as covariates were the demographic variables of education, 160 experience, and length of time at present job, and the pre-training subscale scores for behavior modification and use of volunteers in corrections. A one-way MANCOVA was used to test for the rejection of this null hypothesis. The alpha level for significance was set at .05 and the four post—training subscale scores of the Attitudes toward Using Concepts Survey were used as dependent variables in the analysis. The two settings (training and nontraining) were used as independent variables. In the event that the MANCOVA was significant, univariate analyses were used, as with the first outcome, with alpha levels of .0125. The overall MANCOVA test of association between the settings and the set of four dependent variables indicated that the set of four dependent variables did 22E demonstrate a significant difference between the two groups (E = .88, p i .488). Univariate analyses were not run as a result. In summary, the null hypothesis Ho:2, relating to the intended outcome of the participating staff mem- bers possessing improved attitudes toward using the tech- niques presented in training at the end of training, is not rejected. Intended Outcome #3 The staff members will have an improved ability to work with delinquent youth. 161 Ho:3a There will be no significant difference in post- training scores on the Critical Incidents Test between the in-service training participant group and the nontraining comparison group. The null hypothesis Ho:3a is 29: rejected based on the results of the ANCOVA statistical procedure per- formed (E = .50, p i .484). Data reflecting on this null hypothesis were obtained from the Critical Incidents Test. The Critical Incidents Test was administered pre— and post-training to the 27 Monroe County Youth Center participants who have routine contact with youths in residence. It was also administered at approximately the same times to the comparison group of 10 individuals employed at the Ingham County Shelter Home. As with the previous null hypotheses, this was done to control to a limited extent for the effects of history, maturation, and testing (Campbell & Stanley, 1967). The Critical Incidents Test consists of 10 described hypothetical situations in a residential treatment facility which call for a response on the part of an adult working there. The respondent is asked to choose, among four, the choice of responses most closely describing what he or she would do in that situation. Only one answer is keyed as being correct. A total number correct score for each respon- dent is obtained. Significantly improved total scores after training are inferred to indicate improved ability to work with delinquent youth. 162 A one-way ANCOVA was performed on the Critical Incidents Posttest scores of the two groups in order to determine if the training participants had improved significantly in their scores. Prior to this, however, the relationship was explored between the dependent variable (post-training scores) and a set of four inde- pendent variables including three demographic variables (education, experience, and time at present job) and the pre-training Critical Incidents Test scores. A regres- sion analysis was performed to determine whether any of the demographic variables or the pre-training scores should be used as covariates in the ANCOVA. The overall multivariate test of the hypothesis of no association between the dependent variable and the four independent variables indicated that a significant relationship did exist (E = 6.77, p i .0007). A step- wise regression analysis was then used to determine the contribution of each of these independent variables to that relationship. The results of the stepwise regression analysis are presented in Table 10. As a result of the stepwise regression analysis, three of the independent variables were eliminated for use as covariates in the analysis of covariance. Elimi- nated were the three demographic variables. Qualifying for use as a covariate in the analysis was the set of pre-training Critical Incidents Test scores. The group 163 means on this variable for the two groups were 4.83 for the Monroe County training group and 4.30 for the Ingham County nontraining group. This variable is retained as a covariate in the analysis because it might give a false impression of significance if it is not controlled for in some manner. Its retention as a covariate adjusts the post—training scores for influence by the pre—training scores. In summary, a regression analysis procedure indi— cated that the use of a single covariate in the ANCOVA was necessary. The covariate retained was that of pre— training scores on the Critical Incidents Test. The regression analysis also indicated that three demographic variables could be justifiably dropped from consideration for use as covariates in the ANCOVA. Table 10 Stepwise Regression E Statistics for the Four Possible Covariates on the Post—Training Critical Incidents Test Scores Intended Outcome #3 Covariate E p : Pre-Training Critical Incidents Test Scores 18.17 .0002 Time on Job 2.28 .142 Experience with Children 2.55 .121 Education 1.68 .206 164 A one—way ANCOVA, with the pre—training Critical Incidents Test score as the only covariate, was used to test for the rejection of this null hypothesis. The alpha level for significance was set at .05. The post— training Critical Incidents Test was used as the depen— dent variable in the analysis. The training and non- training settings were used as independent variables. The ANCOVA test of association between the set- tings and the dependent variable indicated that a sig— nificant difference was 293 demonstrated to have existed between the two groups (E = .50, p i .484). Consequently, administration of the Critical Incidents Test both pre- and post-training to the two groups and analysis of their responses did not provide evidence for the rejection of this null hypothesis. Ho:3b The in—service training participants will indicate in their responses to the Follow-up Questionnaire that the in—service training was of little or no help to them in their work during the six months after training. The null hypothesis Ho:3b is rejected for the topical areas of reality therapy: behavior modification, and communication skills but not for the use of volunteers in corrections, based on comparisons between a priori standards set for mean response scores and those actually obtained. The Follow-up Questionnaire was the €’ 165 source of data reflecting on Ho:3b. The questionnaire was administered to the 20 routine-contact participants remaining in the employ of the facility (seven had left) approximately six months after training. One question specifically asked if the respondents had found the information learned in training helpful in their work during the six—month period after training. A "four" response represented the most positive perception of the helpfulness of the training information. A group mean of at least 3.0 (equivalent to a response repre- senting a moderately positive perception of the helpful- ness of the training information) was set as a minimum a priori standard for the rejection of Ho:3b. Table 11 demonstrates that the standard of 3.0 was met for three of the four topical areas. The topical areas of reality therapy, behavior modification, and com— munication skills were perceived by the participant group as being at least moderately helpful to them in their work during the six months after training. The topical area of the use of volunteers in corrections was 292 per- ceived by the participant group as being at least mod- erately helpful. Consequently, only partial evidence is provided for the rejection of this null hypothesis. In summary, two sources of data were used to test the null hypotheses related to Intended Outcome #3, that the staff members will have improved abilities as a 166 result of training. Analysis of the data from the first source, the Critical Incidents Test, provided insufficient evidence for the rejection of the null hypothesis. Analy- sis of the data from the second source, the Follow—up Questionnaire, provided only partial evidence for the rejection of the null hypothesis. Table 11 Group Means and Standard Deviations on Follow-up Questionnaire Responses Relating to Perceived Helpfulness of the Training Information During the Six Months after Training Intended Outcome #3 Topical Area Mean S.D. Reality Therapy 3.05* .780 Behavior Modification 3.10* .641 Communication Skills 3.00* .795 Use of Volunteers 1.75 .851 * . Met or exceeded 3.0 standard. The training participant group immediately after training did BEE perform significantly better than the comparison group on the Critical Incidents Test, which is assumed to reflect on the ability levels of correc— tional staff. The participant group, however, did per— ceive that the reality therapy, behavior modification, and communication skills information learned in training had been helpful (improved their abilities) during the 167 six months after training, as reflected in their responses to the Follow—up Questionnaire. These results are interpreted to imply only partial, and relatively weak, support for the rejection of the null hypothesis relating to the occurrence of Intended Outcome #3. Intended Outcome #4 The staff members will implement what they learned during the training program. Ho:4a The ratings of the Staff Observation Rating Scale will indicate neutral or inappropriate application of the techniques learned in training in work samples of in—service training participants obtained after training. The null hypothesis Ho:4a is rejected for the topical areas of behavior modification and communication skills but not for reality therapy, based on comparisons between a priori standards set for mean ratings and those actually obtained. Data reflecting on this null hypothesis were obtained from the Staff Observation Rating Scale. Fif— teen audiotaped work samples of trainee—delinquent interactions were made in the several weeks after train— ing and then again six months after training. These tapes were then rated by two raters trained in the Staff Observation Rating Scale procedures. The tapes 168 were rated with regard to the appropriateness with which the major principles presented in training were being applied by the training participants in their work. These ratings were assumed to reflect on Ho:4a. The raters were asked to tabulate all of the responses of staff persons on the audiotape into the behavior categories on the Staff Observation Rating Sheet. A rating of the appropriateness was then recorded for each response (see Appendices F and G). Included in the rating system were six behavioral categories under the heading of Positive Reinforcement Responses. These categories, and their respective ratings, were designed to reflect on the participants' use of behavior modification techniques. Two categories were included under the heading of Listening Responses. These categories, and their respective ratings, were designed to reflect on the participants' use of communication skills techniques. Under the heading of Responses to Problematic Behavior, one category is rated on six dimensions. These dimensions relate to the appropriate application of reality therapy techniques. No categories were included for the use of volunteers since no observ— able techniques were involved in that topical area. A heading with two categories was included, however, for neutral responses unrelated to treatment techniques. 169 Each category was rated on the basis of a seven— point scale of appropriateness. A rating of "one" indi- cated the lowest degree of appropriateness. A rating of "seven" indicated the highest degree of appropriate- ness. A rating of "four" indicated a neutral response on the part of the staff member, which was neither par- ticularly productive nor particularly destructive in the judgment of the raters. An a priori minimum standard of 5.0 for the mean ratings of all of the responses within each heading was set to indicate appropriate application of the techniques by the training program participants. Table 12 lists the results of the ratings of the par- ticipants' responses. It presents the means and standard deviations of the ratings by both raters of all of the responses within each heading for the work samples imme— diately after training, six months later, and the com- bined total of both times. The results of the mean ratings of the Staff Observation Rating Scale shown in Table 12 indicate that the a priori standards were met for responses under the heading reflecting on the application of behavior modifi- cation techniques (Positive Reinforcement Responses) and for responses under the heading reflecting on the appli- cation of communication skills (Listening Responses). The standard was not met for responses under the heading reflecting on the application of reality therapy tech- niques (Responses to Problematic Behavior). 170 Table 12 ‘ Means and Standard Deviations for Ratings on the Staff Observation Rating Scale — Intended Outcome #4 Immediately Six Months Total T After After (Combined) raining Training Positive * * * Reinforcement Mean 5.36 5.75 5.56 Responses S.D. 1.148 1.043 1.108 Listening Mean 5.52* 5.32* 5.42* Responses S.D. .877 1.061 .977 Neutral Mean 5.12* 5.21* 5.17* Responses S.D. 1.135 1.071 1.106 Responses to Problematic Mean 3.27 2.54 3.10 Behavior S.D. 2.074 1.967 2.067 * Met or exceeded 5.0 standard. It should be noted, however, that the raters also had been instructed to rate the appropriateness of neutral, nontechnique-related responses. These responses were judged to have exceeded the 5.0 standard, as well. This would indicate the possibility of the existence of a "response set" on the part of raters which led them to rate most responses as being slightly above neutral, lessening the strength of the evidence indicating that the behavior modification and communication skills tech- niques had indeed been applied appropriately. It would, however, strengthen the evidence that the reality therapy responses were not applied appropriately. 171 In summary, analysis of the data collected by the Staff Observation Rating Scale dictates that the null hypothesis (Ho:4a) relating to the apprOpriate appli- cation of the techniques learned in training be rejected for behavior modification and communication skills tech— niques but not for reality therapy techniques. Ho:4b The in—service training participants will indicate in their responses to the Follow—up Questionnaire that they have seldom or never applied in their work during the six months after training, infor- mation presented in the in-service training sessions. The null hypothesis Ho:4b is rejected for the topical areas of reality therapy, behavior modification, and communication skills but not for the use of volunteers in corrections, based on comparisons between a—priori standards set for mean responses and those actually obtained. Data reflecting on Ho:4b were obtained from one item in the Follow—up Questionnaire. This questionnaire was administered to the 20 routine-contact in—service training participants who remained employed at the facility six months after training (seven had left). One four-part item in the Follow—up Questionnaire (3a, b, c, d) specifically asked the participants to indicate the extent to which they had applied information pre— sented in training sessions in their work during the six 172 months since training. A "four" response represented the greatest subsequent extent of application of the information presented in training sessions. A group mean of at least 3.0 (equivalent to a response indicat— ing moderate application of the information) was set as a minimum a priori standard for the rejection of Ho:4b. The results of the analysis of the responses to this four—part item of the Follow—up Questionnaire are presented in Table 13. Table 13 Group Means and Standard Deviations on Follow-up Questionnaire Responses Relating to Perceived Application of Information Presented in Training during the Six Months after Training Intended Outcome #4 Topical Area Mean S.D. Reality Therapy 3.40* .754 Behavior Modification 3.20* .894 Communication Skills 3.17* .786 Use of Volunteers 1.70 .801 * Met or exceeded 3.0 standard. It can be seen that for three of the four tOpical areas the 3.0 standard was met. Information presented in training on the areas of reality therapy, behavior modification, and communication skills was perceived by the training participant group as having been applied 173 in their work during the six months after training to at least a moderate degree. Information presented in the topical area of the use of volunteers in corrections was EEE perceived by the training participant group as having been applied in their work to at least a moderate degree. In summary, analysis of the data collected by the Follow-up Questionnaire dictated that the null hypothesis (Ho:4b) relating to the self—perceived degree of appro- priate application of the techniques learned in training be rejected for the topical areas of reality therapy, behavior modification, and communication skills but not for the topical area of the use of volunteers in correc— tions. Evidence regarding the two null hypotheses related to Intended Outcome #4, that staff members will implement what they learned during the training program, is somewhat contradictory. Ratings of audiotaped staff observations indicate that the participants, on the average, applied behavior modification and communication skills techniques at least at a minimal level of appro— priateness but did not demonstrate appropriate appli- cation of reality therapy techniques. On the other hand, the evidence indicates that participants felt that they had applied the techniques of each of these three topical areas (including reality therapy) in their work. It 174 would seem at least possible that the participants did not fully understand what was involved in the proper implementation of reality therapy techniques. Intended Outcome #5 The staff members will have positive attitudes about the training experience. Ho:5a In-service training participants will demonstrate neutral or negative attitudes about the training experience itself, immediately after training, in their responses to the Satisfaction with Training Survey. The null hypothesis Ho:5a is rejected based on comparisons between a priori standards set for mean responses and those actually obtained. Data reflecting on this null hypothesis were obtained from the Satis- faction with Training Survey. It was administered to the 27 routine-contact training participants immediately after training. Data were obtained from 16 items containing a total of 34 sections. Results of the analysis of the responses to these items are summarized in Table 14. Please note that several sections of data reported in this table have previously been reported in Tables 5 and 6. In Table 14 the overall estimate of attitudes toward the training experience provided by a grand mean 175 Table 14 Group Means and Standard Deviations on Satisfaction with Training Survey Responses Relating to Attitudes about the Training Experience Immediately after Training Intended Outcome #5 Item Mean S.D Length of time for overall training * was adequate 3.00 .756 Length of time for each topic was adequate: Reality Therapy 2.61* .941 Behavior Modification 3.00 .798 Communication Skills 2.57 .992 Use of Volunteers 2.32 .945 Extent to which the following were covered: Reality Therapy Theory 3.57: .662 Reality Therapy Techniques 3.09* .921 Behavior Modification Theory 3.30* .703 Behavior Modification Techniques 3.36* .790 Communication Skills Theory 3.13* .869 Communication Skills Techniques 3.00 .853 Use of Volunteers in Corrections 2.60 .995 Find the information useful on: Reality Therapy 3.05: .945 Behavior Modification 3.43* .676 Communication Skills 3.04* .878 Use of Volunteers 3.05 .848 Will use the information on: Reality Therapy 3.65: .487 Behavior Modification 3.58* .504 Communication Skills 3.65 .573 Use of Volunteers 2.77 .869 Questions were adequately answered 3.36* .581 Sufficient opportunity for discussion 3.41* .666 176 Table 14 (continued) Item Mean S.D. Trainers sufficiently expert in: Reality Therapy 3.32* .568 Behavior Modification 3.55* .510 Communication Skills 3.14* .640 Use of Volunteers 2.76 .995 Trainers held interest in: Reality Therapy 3.14* .640 Behavior Modification 3.41 .503 Communication Skills 2.91 .668 Use of Volunteers 2.77 .869 Will need more help on: Reality Therapy 2.52 .680 Behavior Modification 2.63 .647 Communication Skills 2.57 .662 Use of Volunteers 2.17 .717 Total (Grand Mean) 3.14* .218 *Met or exceeded 3.0 standard. 177 of responses to the Satisfaction with Training Survey indicates that the participants were positive in their attitudes toward training, immediately after the training experience (mean = 3.14 which exceeds the 3.0 a priori standard). Further investigation of the results of the responses to this questionnaire indicate, however, that training participants perceived several aspects of the training program to be superior to others. While the length of time spent on the overall training program was seen as sufficient, only the indi— vidual topic of behavior modification was seen as having been given an adequate length of presentation. On the other hand, both the theory and techniques of the topical areas were seen as having been covered adequately, except for the use of volunteers in corrections. While the participants Viewed the information presented in each of the topical areas as being useful, they did not indicate that they themselves would use the information presented on the use of volunteers in cor— rections. Concerning the style of the training presenters, they were seen as adequately answering questions and providing sufficient opportunity for discussion. They were regarded as being sufficiently expert in all topical areas except for the use of volunteers and as holding the participants' interest in reality therapy and 178 behavior modification but not communication skills or the use of volunteers in corrections. Somewhat surprisingly, the participants‘ responses indicated a desire for "more help" regarding each of the topical areas in spite of responses to other items which indicated they felt the topics had been covered ade— quately. The item read, "I feel I will need more help before I will be able to understand and use in my work: ." (each of the topical areas). The participants were to respond from strongly agree to strongly disagree. They may have become confused regarding the direction of the item, which might account for this puzzling and apparently contradictory result. Ho:5b In—service training participants will demonstrate neutral or negative attitudes about the training experience in their responses to the Follow—up Questionnaire six months later. The null hypothesis Ho:5b is not rejected based on comparisons between a priori standards used for mean responses and those actually obtained. Data reflecting on Ho:5b were obtained from four items containing a total of eight sections in the Follow—up Questionnaire. It was administered to 20 routine—contact participants (seven had left the employ of the facility) six months after training. Results of the analysis of the responses to these items are reported in Table 15. 179 Table 15 Group Means and Standard Deviations on Follow—up Questionnaire Responses Relating to Attitudes About the Training Experience Six Months after Training Intended Outcome #5 Item Mean S.D. Have applied in work: Reality Therapy 3.40: .754 Behavior Modification 3.20* .894 Communication Skills 3.17 .786 Use of Volunteers 1.70 .801 Has been useful in work: Reality Therapy 3.05: .780 Behavior Modification 3.10* .641 Communication Skills 3.00 .795 Use of Volunteers 1.75 .851 How worthwhile was training 2.50 .761 Personal enjoyment of training 2.60 .681 Total (Grand Mean) 2.72 .968 * Met or exceeded 3.0 standard. 180 The results summarized in Table 15 demonstrate that while three of the tOpical areas (reality therapy, behavior modification, and communication skills) were seen by the training participant group as having been .applied and useful in their work, information presented in the topical area of the use of volunteers in cor— rections was seen as neither having been applied nor useful. The group means for the items reflecting on how worthwhile the training had been and how much they had personally enjoyed the training also failed to meet the 3.0 a priori standard. It is interesting to note, how— ever, that while the 2.60 mean of the responses to the item reflecting on personal enjoyment of the training sessions did not meet the 3.0 standard, 70% of the par- ticipants responded that they did, in fact, enjoy the sessions most of the time. The highly negative responses of the remaining 30% of the participants caused the standard to not be met. The grand mean, which is an average of responses to all of the items on the Follow—up Questionnaire, of 2.72 also failed to meet the 3.0 standard. When the responses relating to the topical area of the use of volunteers in corrections are removed, and the grand mean re~ca1cu1ated, it becomes 2.98. Clearly, the topical area of the use of volunteers in corrections is the greatest detractor in an estimate of participant attitudes toward training six months after training. 181 In summary, the results of the analysis of data obtained by the Satisfaction with Training Survey support the rejection of the null hypothesis Ho:5a that the in- service training participants will demonstrate neutral or negative attitudes about the training experience itself, immediately after training, in their responses to the Satisfaction with Training Survey. An overall positive attitude is indicated by the grand mean of responses to the survey, which exceeded the 3.0 a priori standard. Especially positive attitudes were expressed toward most aspects of the presentations on reality therapy and behavior modification. Less positive atti— tudes were expressed about aspects of the presentations on communication skills and the use of volunteers in corrections and whether the participants would need more help before applying what they had learned in training to their work with delinquents. The results of the analysis of the data obtained by the Follow-up Questionnaire do not support the rejec- tion of the null hypothesis Ho:5b that the in-service training participants will demonstrate neutral or nega— tive attitudes about the training experience, in their responses to the Follow-up Questionnaire, six months later. While important aspects were rated positively (the degree of application and helpfulness of the train— ing information on reality therapy: behavior modification, 182 and communication skills during the six months after training), the grand mean of responses to the Follow-up Questionnaire items did not meet or exceed the 3.0 a priori standard. The participants' responses to items relating to the use of volunteers in corrections were particularly low, perhaps affecting the participant rating of overall worthwhileness of training and personal enjoyment of training, which also failed to meet the 3.0 standard. The rejection of Ho:5a and lack of rejection of Ho:5b is interpreted as providing only partial support for the occurrence of Intended Outcome #5. This appar- ently contradictory result is further discussed in Chapter V. Additional (Unintended) Outcome Hozl There will be no significant difference in post— training scores on the Correctional Preference Survey between the in-service training partici- pant group and the nontraining comparison group. The null hypothesis related to this additional unintended outcome is not rejected based on the results of the ANCOVA statistical procedure performed (E = 1.56, p i .217). The Correctional Preference Survey (Quay, 1968) had been administered pre— and post—training to both the training and nontraining groups in order to determine if any change had taken place, consequent to the training, in the participants' preference of types of delinquent with which they enjoy working. As 183 indicated in Chapter II, changes in attitudes toward types of delinquents on the part of participants were not an intended outcome of the training presenters. The Correctional Preference Survey was administered in order to gather data on possible incidental changes occurring as a result of training and to further the field of instrumentation in corrections by supplying normative data adding to that already collected on this instrument (Ingram, 1970). The Correctional Preference Survey was developed as a result of a theoretical position that delinquents can be classified into four behavioral categories and that certain types of adults would be more successful working with one category of delinquent than they would others. The Correctional Preference Survey purports to determine the type of delinquent with which an indi— vidual would be best suited to work. This is done by presenting the individual with 60 statements relating to style of correctional practice and types of delinquent behavior. The respondent is asked to agree or disagree with these statements on a four—point continuum. Clas— sification of the individual is determined by the number of items answered in the keyed direction relative to the average number of responses in that direction by a normative sample of correctional workers. 184 In order to determine if the training partici— pants had responded significantly differently to the post-training administration of the Correctional Preference Survey, a MANCOVA procedure was undertaken. The alpha level for significance was set at .05. The four post- training Behavior Category subscales of the Correctional Preference Survey were used as dependent variables in the analysis. The training and nontraining settings were used as independent variables. In the event that the MANCOVA was significant, univariate analyses would be used to explore the influence of each of the four dependent variables. The alpha level for each of the univariate analyses would be set at .0125 which evenly distributes the .05 alpha level to each of the four analyses. Prior to examination of the MANCOVA, the relation- ship was explored between the set of four dependent vari- ables and a set of seven possible covariates in the analy— sis. Considered for use as covariates in the analysis were three demographic variables (education, experience, and time at present job) and the four pre—training Behavior Category subscales of the Correctional Preference Survey. A regression analysis procedure was performed to determine which of the above-mentioned variables should be used as covariates in the MANCOVA. The overall multivariate test of the hypothesis of no association between the four dependent variables and 185 the seven independent variables indicated that a signifi- cant relationship did exist (E = 2.23, p i .003). A stepwise regression analysis was then used to determine the contribution of each of these independent variables to that relationship. The results of the stepwise regression analysis are presented in Table 16. Table 16 Stepwise Regression E Statistics for the Seven Covariates on the Set of Four Post—Training Behavior Category Subscales of the Correctional Preference Survey Additional Outcome #1 Covariate E E : Behavior Category 4 5.96 .001 Behavior Category 3 3.17 .030 Behavior Category 2 2.89 .042 Behavior Category 1 2.04 .120 Time on Job .39 .816 Experience with Children 1.58 .213 Education .61 .658 As a result of the stepwise regression analysis, the three demographic variables were eliminated for possible use as covariates in the MANCOVA. While the pre—training Behavior Category 1 subscale of the Cor- rectional Preference Survey could also have been elimi— nated as well, a decision was made to include it as a covariate, along with the other three pre—training sub— scales. Various reorderings of the pre—training sub- scales in other stepwise analyses confirmed that BC—2, 186 BC-3, and BC-4 would qualify for use as covariates while BC—l would not have to be included as a covariate. In spite of the fact that inclusion of BC-l as a covariate might slightly reduce the possibility of detecting dif- ferences that do exist in the MANCOVA, it was included with the other pre-training subscales for several reasons. First, the primary intention of the MANCOVA was to examine potential differences in the groups on the entire survey, rather than on any individual sub- scale. Second, unlike the Cognitive Areas Objective Test and Attitudes toward Training Survey whose sub- scales were intended to be independent of each other, the scoring of each subscale of the Correctional Preference Survey is relative to and dependent upon the other subscales. For these reasons, it was decided that the pre-training subscales should be treated as a group for the purpose of consideration as covariates. Since three of the subscales qualified for use as covariates, all four were included. In summary, a regression analysis procedure indi— cated that the three demographic variables could be eliminated for use as covariates. The pre-training BC-l subscale could also have been eliminated but was not on the basis of logic. Consequently, all four pre-training subscales were used as covariates in the MANCOVA. 187 The overall MANCOVA test of association between the settings and the set of four dependent variables, using the set of four pre—training subscales as co— variates, indicated that a significant difference was ESE demonstrated to have existed (E = 1.56, p i .217). As a result, univariate analyses were not run. In summary, the null hypothesis relating to the additional outcome of the participating staff members significantly changing preferences of types of delin- quents with which they enjoy working was not rejected. Summary The purpose of this chapter was to report the results of the statistical analyses of the observational data related to this study. The reader is directed to Figure 13 for a summary of these results, organized according to Stake's model of evaluation. For the con- venience of the reader, Figure 6, which summarizes the intended antecedents, transactions, and outcomes of training and the corresponding methods of observation, is repeated here as Figure 12. As indicated in Figure 13, some of the standards .set for the antecedents were met and some were not. The participants did demonstrate an initial favorable attitude toward training in their responses to the Anticipation of Training Survey (Intended Antecedent #1). The intended 10 instructional days of training did not occur 188 .coaum>uwmno mo mtozums @CWUCOQmmuuoU :uaB Agencamuu named nmumv >w>hsm wocwpomwua HmcofluoouuouuleOUuso HocoHuapp< Amcficfimpu woumm mcucoe xemv wsfloccoflummoo QSI3OHHOL An Ranacflmuunumomv >m>usm quchub LDAB coauoowmarmm Am .n Amcacamuu uwuuo mtucoe xamv ouamcccflbmwaa isoBOHHon «b Aocachuu umuwm mnucoe xam .macflmuulumOQV mHmom chrmm coflum>uwmno uumum Am .v “acachuu uwuum wLuCOE mev mnemccofiumwso dsn3oHHom An Amnecwmuuuumom Iona. uwme mucmanCH amofiuauu Am .m Amadcamuulumom nomad >o>usm mummocoo moan: pum30u mopsuduu< .N AD:a:amuunuwom pom loumv uwme w>auumflno mmmp< m>wuacmou .~ mGEOUU—JO ocazamuu mo meouuso pco .mcofluummcmxu .mucopououcm popcoucfl no >umEESm .NH wusosu chcwmub pouuouou; ucm:?:wfi mobum m:o:CE xflm AD H. to was» ca motoroz: c @EOUuSO HchwufiUC< c:s:wctb lumca >chmLUcEEfl Am UCHCMQMu unons opsuauuo o>auflm0d wumuumcosop Hafiz :oMCado Hchwuod An pw>ummno Am codumEuomcq mm: HA4: xuo3 CH >uaaa£m pw>0uaEH .m mmnwficnomu mo mm: pumBOu mOUSuHuum Uo>oumEH moose accedOD mo woun~30cx pwmmwuocu 17"” onOUuso A@:flCAMuu mcflunpv momou0ap5< .EhOm puoowm Howevumn< mcofluomwcmue IIIlllllllillllllllllllllll < xflpcoaad .>H uwummzu mo uxwu wow tlilllllltlll mcofluomwcmue Aged:flmuuuumomv >w>usm guacamue :ufl3 :ofluomwmflumm no cofluomw >u0umw: madcampe .oH Amca:amuunum0dv >o>usm mcflcamue Lufl3 ceauowwmaumm .m AOCACAMMD qcfluzpv Ehom whooom HGDOpomcm .m AOCACAMMD uwumm mnucoe xflwy oufimccoflummso asu3oHHOa An AmcficwmuUIuwomv >w>usm madcfimue nufi3 comuumwmflumm Am . nocwcfimuu mcfiuspv Euom puOoom Hoquowcd . n o Amcficamuu ocauspv Euom phenom Hmuocomc< .m AmcficfiMuu mcfluscv Ehom puouwm amquumc< .v AmCHCAMuu mewuspv Euom puOuwm Hmuocuoc< .m Aocacemuu ocfluspv shod puoowm HmDOpowc¢ .m :ofluoow mpsbfluu< Honeymuooce Ab conuoom umwuoucH Hmcomuwa Am “weacflmuuloudv >o>xzm Ucflcflmue mo coaummflonucc .H wUCOUQU®UC< xo¢3 tcoowm qcfiusp mucwpflwmu Lufi3 uomucoo o>n£ ob :omumm mmmum 50mm uroaUaoone on Haas Lemon acacaone .o >Hmuauco pwucmwonm mmum Hmofimou comm . m mumwocoo pcm mosqflccowu Hommm: can Hwofluumud wpa>oud Haw: wcofiwmww .h pmHHOucw mm can» macs uoc .muwnEmE ummum >Hco .o coawmmm zoom on wUCMpcouum ESEHCME unmouwm o>wwu>unvflm .m pmumsuumucwcs season HAM: mconmwm .v mcoflmmsomflp moOum HHMEm paw wanna now mannaflo>m mmfluaaflumm .m popfl>oud m>mp amcofluuouumCA :09 .N opsufiuuo kufiumuowsu An ummuwucfl HmCOmumm Am mcflcqmuu pum30u mpsuwuum wabmuo>mu Hwfluflcu .H mucwpwowuc< Atom: cogzv pom: pocuoz Hmcoflum>uwmno mucmuCH .Houoe w.wxmam Ou wratuooum comacmuuo muazmmn no >umEE=m .nu wusmwu yes he: Ttap:MumxlmEOUu:C Hoseaufivpt qu uo: uuotcmum AL qu Uhmvcmuw Aw .m muwuucaHo> mo mm: 0:» new um: uo: asp mHHfixm :ofiumuflcsesou can .cOwuuUMwflnoe M0w>mrmn .>amnw£u >uflamwu new awe oumucmuw An >mnuw£u >uaa~wu no“ qu uo: usn mauaxm :OMumoMcsfikoU vac nodumofluHEOE uoH>mnwfl how uwfi phancmuw Am .v mhwquZHO> no on: ecu new uUE uoc uzn maawxm :oqumoflczfiEou ccm .coHumoauwuos ~0w>mzwn .>mmuwzu >uwamwu new qu pumczmum An uUE no: Unmvcmum Hm .m qu uo: phmucmum .N 502 numeroum .H mosouuzo 189 <>oozos no.1:0E00uzo Hoseauath o>onm no o.m we cams an w>on~ no o.m mo :me Am >e nmoamro do axon mom w:0auumm:mub qu uo: tnmpcnum .OH umE th¢t¢Mom .m qu ucc tnmvcauw .m mummucnfio> mo mm: new be: usn .mHmem coaumUflcsezou .COHUMUAMMGOE xofl>msmn .>Qmumnu >uaflmwu LCM nos Unavcmum An mmwum use» flaw new owe puntcmcm at .w boE uo: pthcMuw .0 >Hco m>mp cough umu_u umE transmum .m 50E uo: that:Mum .v soE charcmaw M m ~05 to: thatctbm one mtnmt:q.u :50m # Wucopmmwwflfl mucmEbtzfi .m wwwcomwwu co v>onw no o.m mo :maE AA mmwcwumMumoumnm mo mmcauau co o.m wo cums Am .v o.m m0 cme an <>ooz< :0 outmoauarmam to ~o>ua mo. in .m <>ooz<2 :0 outmoaoatman mo Hwaoa mo. .m unsuccesm Huumumquwum <>ouzwa mo. .H moE00uso >H umudmao do axon wow MCOHHUNWENHF uuoucou w>mz cu good .OH 0>onn no o.m mo coo: .m muoucmmwhm osmcdmuu mo zodcamo Accomuom .m w>onm Mo o.m no :moE an 0>onm no o.m mo chE am .> mm cmzu once 0: .muwnEmE uwnum >Hco .o x9e: vacuum EdEmcqs mc.NN .xwm3 umumu EzEwcflE «m .m usuuo Cu wucm0muflcmwm mo macauasunwuca 02 .v muwucowwud O:L:_M»u mo :owcwto _m:0muea .m pwuw>0um wMOE no m>mn :08 .N m>0£m no o.m to :meE An o>onm to o.N no :mo& Am .H mucovoowuca mvumocaum 190 in full, according to the Anecdotal Record Form (Intended Antecedent #2). Facilities were available for large and small group discussions in the Opinion of the training presenters, according to the Anecdotal Record Form (Intended Antecedent #3). The sessions did not remain entirely uninterrupted, as indicated by the Anecdotal Record Form (Intended Antecedent #4). The attendance standards were met on only the first three days of train- ing, according to the Anecdotal Record Form (Intended Antecedent #5). The Anecdotal Record Form also indicates that nonstaff members were in attendance at training sessions (Intended Antecedent #6). The responses to the Satisfaction with Training Survey suggest that practical and useful techniques and concepts were presented in training, in the opinion of the training participants immediately after training (Intended Antecedent #7a). The responses to the Follow-up Questionnaire indicate that the training participants still held the same Opinions about the usefulness of the topical areas six months later except for the topical area of the use of volunteers in corrections (Intended Antecedent #7b). In the opinion of the training presenters, as indicated by the Anecdotal Record Form, not all of the topical areas were presented entirely as planned (Intended Ante- cedent #8). Responses of participants to the Satisfaction with Training Survey suggest that the training presenters 191 were regarded as being proficient in presenting the topical information (Intended Antecedent #9). The Training History section of the Satisfaction with Train- ing Survey indicates that not all of the training par- ticipants did have contact with residents during the second week of training (Intended Antecedent #10). The transactions of training were observed by means of notes taken by this evaluator on the Anecdotal Record Form and by means of audiotaping of training sessions. The reader is referred to the text of this chapter for a summary of the training transactions that occurred, compared to that which was intended to occur in training. Some of the standards set for the intended out- comes of training were met and some were not. Post- training responses of the participants to the Cognitive Areas Objective Test provided evidence to indicate that they possessed increased knowledge of principles pre— sented in the major topical areas of the training pro— gram (Intended Outcome #l). Post-training responses of the participants to the Attitudes Toward Using Concepts Survey provided insufficient evidence to indicate that they possessed improved attitudes toward using the techniques taught in training (Intended Outcome #2). Post-training responses of the participants to the Critical Incidents Test provided insufficient evidence 192 to indicate that they demonstrated improved ability to work with delinquent youth (Intended Outcome #3a). Responses of the participants to the Follow-up Question— naire six months after training provided evidence reflect— ing on the same intended outcome to indicate that the participants felt the information provided in three of the four topical areas had improved their abilities, however. Somewhat contradictory evidence was also found for Intended Outcome #4 that the information will be used after training. Results of the Staff Observation Rating Scale procedure provided evidence that behavior modification techniques and communication skills were being applied at at least a minimal level of appropri— ateness but that reality therapy techniques were not. Since the topical area of the use of volunteers has no readily observable techniques, no attempt was made to observe them. Responses of participants to the Follow— up Questionnaire six months after training provided evi— dence to indicate that they perceived themselves to have been appropriately applying the techniques of behavior modification, communication skills, and reality therapy but not the use of volunteers in corrections. Post— training responses of participants to the Satisfaction with Training Survey provided evidence that they demon- strated positive attitudes about the training program immediately after training (Intended Outcome #Sa). Six 193 months later, however, in their responses to the Follow- up Questionnaire, insufficient evidence was obtained to indicate that they still maintained positive attitudes about the training (Intended Outcome #5b). Post-training responses of participants to the Correctional Preference Survey provided insufficient evidence to indicate that they had significantly shifted in their preferences of types of delinquents with which they enjoy working (Unintended Outcome #1). The next chapter is the conclusion to this study. The reader is referred to Chapter V for discussion of the results and their implications. CHAPTER V CONCLUSIONS CONCERNING THE EFFECTS OF THE TRAINING PROCESS, EXAMINATION OF THE PROCESS OF TRAINING, IMPLICATIONS FOR FUTURE TRAINING AND TRAINING EVALUATION, AND SUMMARY Introduction and Overview The specific purpose of the study was to investi— gate the effects and processes of a two-week in-service training program for the staff of a juvenile correctional facility. The study encompasses a broader purpose as well, however, of applying educational evaluation tech- nology to the field of in—service training in corrections. The need for such evaluation is gaining wide acceptance. It is seen by some as the key step toward improving in— service training in corrections (Ewing & Gregg, 1978). Yet the literature does not help to specify what type of evaluation approach or whose evaluation model would best lend itself to this evaluation endeavor. This is probably due to the embryonic stage of the field of evaluation of in-service training for correctional staff and the con— sequent 1ack of reported results. 194 195 Stake's model (1972, 1975, 1976, 1978) has been used in this study as the guide for the evaluation of the particular training program in question. In addition to determining the specific effects and processes of the training program, an estimation of the value of Stake's model for use in this field is of interest as well. The model has been applied in the study thus far 1. Define the training program in terms of intended antecedents to training, intended transactions during training, and outcomes intended to be consequent to training (see Chapter III). 2. Define and develop methods of observing these events and conditions (see Chapter III). 3. Determine a priori standards by which the observation of the events and conditions are judged (see Chapter III). 4. Make judgments based on the standards applied to the observations (see Chapter IV). As a continuation and completion of the evaluation process, Stake's model has been applied in this concluding chapter to: 1. Create a composite evaluation report on the effects of the training program. 2. Examine the processes of the training program by examining the relationships of the antecedents and 196 transactions of training to the outcomes of training. This is done by noting what effect the occurrence or lack of occurrence of the antecedents and transactions (which Stake terms "congruence") had on the occurrence or lack of occurrence of the intended outcomes and also by examining whether the assumed logical ties between the antecedents, transactions, and outcomes (which Stake terms "contingencies") were, in fact, valid. The information from the above-mentioned two steps in the evaluation process was then used to assist in the reporting in this chapter of implications for future in—service correctional training programs. Suggestions regarding future evaluations of in— service correctional training programs are made, based upon the present evaluator's experiences with this study. The chapter concludes with a summary of the study. Report on the Effects of the Training Program The effects of the training program are first stated in this section in terms of the directional research hypotheses which were supported or not supported in varying degrees. These hypotheses are related to the intended outcomes of training. This section concludes with a summary of the effects of the training program on its participants. 197 Research Hypotheses Hazl The in—service training participant group will demonstrate increased knowledge of the major principles presented in the in—service training program by receiving higher post-training scores on the Cognitive Areas Objective Test than a com— parison group that has not participated in the in—service training. This hypothesis was supported. The participant group did demonstrate a significantly greater amount of post—training knowledge on the Cognitive Areas Objective Test than the comparison group, even when pre-training scores were covaried out. UpOn univariate analysis, the only topical area that was shown to contribute signifi- cantly to the overall significance was behavior modifi— cation. The topical area of reality therapy approached significance as well, however. Ha:2 The in—service training participant group will demonstrate improved attitudes toward uSlng the child management and treatment techniques pre— sented in training by receiving higher post- training scores on the Attitudes toward USing Concepts Survey than a comparison group that has not participated in the in-service training. This hypothesis was not supported. The partici— pant group demonstrated no significant post-training improvement in attitudes on the Attitudes toward Using Concepts Survey when compared to a group that had not participated in the training. 198 Ha:3 The in-service training participant group will demonstrate improved abilities to work with delinquent youth by receiving higher post—training scores on the Critical Incidents Test than a com— parison group that has not received training and by indicating self-reported improvement in abili— ties in their responses to the Follow—up Question— naire. This hypothesis was only partially supported. The participant group demonstrated no significant improvement in abilities as demonstrated by their post—training scores on the Critical Incidents Test when compared to a comparison group that had not received training. The participant group did report in the Follow-up Question— naire that the topical areas presented in training of reality therapy, behavior modification, and communication skills did help them in their work during the six months after training. Ha:4 The in—service training participant group will demonstrate appropriate application of the child management and treatment techniques presented in training by receiving mean ratings in the appro- priate range on the Staff Observation Rating Scale of samples of their work after training and by indicating via self—report on the Follow—up Ques— tionnaire that they believe they have appropriately applied the techniques in their work following training. This hypothesis was only partially and relatively weakly supported. The participant group received mean ratings in the appropriate range on the Staff Observation 199 Rating Scale of samples of their work after training on the dimensions concerning behavior modification and com- munication skills but not on several dimensions related to reality therapy. On the other hand, the participant group reported on the Follow—up Questionnaire that they felt they had been correctly applying behavior modifi— cation, communication skills, and reality therapy tech- niques during the six months after training. Ha:5 The in-service training participant group will demonstrate positive attitudes about the training experience itself in their responses to the Satis— faction with Training Survey (immediately after training) and their responses to the Follow-up Questionnaire (six months after training). This hypothesis was only partially supported. The training participant group reported positive attitudes immediately after training as indicated by a grand mean of their responses to the Satisfaction with Training Survey. Six months later, in the Follow—up Questionnaire, they did report that they had been helped in their work by training in the areas of behavior modification, com- munication skills, and reality therapy. However, the grand mean of their responses on the Follow-up Question— naire did not meet or exceed the a priori standard set to indicate positive attitudes about the training exper- ience. 200 Additional (Unintended) Outcome Hozl The in-service training participant group will demonstrate shifts in attitudinal preference regarding the type of delinquent youth with which they enjoy working compared to a compar- ison group that has not received training as indicated by postetraining scores on the Cor— rectional Preference Survey. This hypothesis was not supported. The post— training scores of the participant group on the Cor- rectional Preference Survey demonstrated no significant shifts from the pre—training scores when compared to a group not receiving training. Summary of Effects of the Training Program Evaluation data that were obtained and analyzed concerning the training program, as it was presented, provided evidence to support that a statistically sig- nificant difference in amount of knowledge was acquired by participants as a result of training when compared to a group not receiving training. The topical area primarily contributing to this significance was behavior modification. The area of reality therapy also approached providing a significant contribution. The evidence provided only partial support to indicate that the participants significantly improved work abilities as a result of training, only partial support to indicate that the training information was used after training, and only partial support to indicate 201 that the participants felt positively about the training experience itself. The evidence provided no support whatsoever to indicate that the training program resulted in improved attitudes toward the use of the topical area techniques or that the participants shifted in their attitudinal preference toward types of delinquent youth with which they enjoy working. Relationship of Antecedents and Transactions to Outcomes The previous two sections of this chapter have provided indications of whether or not the intended out- comes of training did, in fact, occur. These indications have been based on evidence gathered and analyzed to test the research hypotheses that have been set up in relation to the intended outcomes. The intention of this section is to provide the reader with additional information about the processes of in-service training for correc- tional staff. Each intended outcome of training that was specified prior to training is listed. For each intended outcome, the antecedents to training and trans— actions during training that appear to have had an effect on the occurrence or nonoccurrence of that outcome are identified and discussed. 202 Intended Outcome #1 The participating staff members will possess increased knowledge of the principles presented in the major topical areas of the training program. This intended outcome was demonstrated to have occurred. Several factors may have contributed to this successful occurrence. The participants evidenced an initial positive attitude toward receiving training (Intended Antecedent #1) which indicated motivation on their part to benefit from training. The participants also regarded the training presenter for behavior modifi- cation to be particularly proficient (Intended Antecedent #8). That presenter's proficiency may well have sig— nificantly contributed to the overall significance achieved on the Cognitive Areas Objective Test, since only the univariate test for behavior modification proved to be significant in the MANCOVA. The presenters for reality therapy and communi— cation skills were seen by the participants as being pro- ficient but less so than for behavior modification. The presenter for the use of volunteers in corrections was not seen as being proficient. Reality therapy approached significance in its univariate test on the Cognitive Areas Objective Test MANCOVA, but communication skills and the use of volunteers in corrections did not. It should be noted that the presenter for the use of 203 volunteers in corrections is a nationally recognized expert in that field. He did not cover the area as he had intended in training because he became involved in discussing other matters relating to the therapeutic treatment of delinquents. That shift, however, was reflected in the results of the Cognitive Areas Objec- tive Test and in the Satisfaction with Training Survey, which indicated no positive results concerning the topical area of the use of volunteers in corrections. Several factors that might have been expected to prevent the intended outcome from occurring should be mentioned. As indicated in Chapter IV, under Transactions and Intended Antecedent #8, the training was somewhat shortened. While this did not prevent the overall occurrence of the outcome, it may have reduced the pos- sibility of achieving significance for each of the topical areas in their univariate tests. The same can be said about attendance (Intended Antecedent #5), interruptions (Intended Antecedent #4), and contact with resident delinquent youths (Intended Antecedent #10), all of which were specified intended antecedents of training that failed to meet a priori standards set for them. r.~—- . 204 Intended Outcome #2 The staff members will have improved attitudes toward using: (a) reality therapy techniques (b) behavior modification techniques (c) communication techniques (d) volunteers in corrections This intended outcome was not demonstrated to have occurred. The most likely explanation for this actually has little to do with intended antecedents and transactions. The participants evidenced quite positive attitudes about using the concepts prior to training. It is, of course, more difficult to demonstrate improve- ment of positive attitudes than negative ones because of a ceiling effect in a case like this. Actually, regression effects could have been expected to cause a drop in attitudes after training. This did not occur, as the attitudes did not drop, and, in fact, rose nonsignifi- cantly. Two antecedents that may also have influenced the lack of demonstrated occurrence of this outcome are the failure of the attendance standards (Intended Ante- cedent #5) and the shortening of the training itself (Intended Antecedent #8). It may well be that a short-term training program, such as this one, is more likely to influence knowledge gained by participants 205 than attitudes which are probably more deeply ingrained. This would be true especially if a number of individuals have not attended all of the training sessions and con- sequently had not given the training program a full chance to influence their attitudes. Intended Outcome #3 The staff members will have an improved ability to work with delinquent youth. Only partial, and relatively weak, support was provided by the evidence to indicate that the partici- pants improved their abilities as a result of training. While the standard was met to indicate that the sessions will provide practical and useful techniques and con- cepts (Intended Antecedent #7), and that the training presenters were proficient in their presentation of reality therapy, behavior modification, and communication skills techniques (Intended Antecedent #9), still only self-perceived improvement of abilities on the part of participants could be demonstrated. There are several possible explanations for the weakness of the evidence supporting the occurrence of this outcome. Not all of the participants had contact with resident youths during the second week of training (Intended Antecedent #10). It may be that considerable practice on the job may be necessary for improvement 206 in ability to take place in this type of situation. A number of participants in training did not have contact with resident youths during the training period because the facility was not yet functioning at capacity, and there were not enough residents to go around! Another possible explanation for the weakness of the evidence is that of the failure to reach the atten- dance standard during the second week (Intended Antece- dent #5) and the shortening of the training program (Intended Antecedent #8). These may have affected per- formance on the measure of ability more than that on the measure of cognitive knowledge. While individuals might be relatively easily influenced to modify how they would answer a theoretical question relating to reality therapy, etc., they may be more resistant to changing how they would hypothetically handle a situation involving delin- quent youth. Consequently, as might have been the case with Intended Outcome #2, less than maximum attendance and any reduction in training at all reduces the chance of changing this type of more deeply ingrained pattern of behavior. It must be noted, however, that the Critical Incidents Test has a very short "track record" at this point and may not have detected changes in the partici— pants' abilities that did exist. It is likely that actual changes in the abilities of the participants 207 after a relatively brief training period were slight. Consequently, instrumentation more sophisticated than the Critical Incidents Test would be needed to detect them. Continued refinements of the Critical Incidents technique would likely be of great benefit in similar evaluation endeavors in the future. Intended Outcome #4 The staff members will implement what they learned during the training program. Only partial, and relatively weak, support was provided by the evidence to indicate the occurrence of this intended outcome. The participants indicated that they had appropriately applied the training information on reality therapy, behavior modification, and communi- cation skills relatively often during the six months after training. However, two trained individuals, rating audiotaped samples of the participants' work both imme— diately after and six months after training, indicated that the participants were applying positive reinforce- ment and communication skills techniques only at a minimal level of appropriateness and as a group were not appropriately applying key aspects of reality therapy. Favorable pre—training and post—training attitudes toward using the techniques on the part of the participant, as indicated by the Attitudes toward Using the Concepts 208 Survey, would indicate that they would desire to apply the techniques appropriately in their work. The con- fidence they had in their own abilities to do so after training may have been limited, however, as indicated by their responses in the Satisfaction with Training Survey indicating they “need more help." There are several possible explanations for the apparent weakness with which the participants applied the training information. While the participants indi— cated that the training had provided practical and useful information (Intended Antecedent #7), the information may have been more theory—oriented and less technique— oriented than the presenters intended. This would likely result in increased chances of significance on the Cognitive Areas Objective Test and reduced chances of appropriate ratings on the Staff Observation Rating Scale. Another possible explanation for this weakness is indicated by the failure to achieve the a priori standard set for each person having contact with resi— dents during the second week of training (Intended Ante- cedent #10). It may be that participants needed sig— nificantly more practice implementing the techniques than they received during the training period. Since many of the training participants had no contact with residents during that time, they had less chance to practice implementing the techniques while training was occurring. It is clear that there was no sufficient 209 mechanism built into the training program or the facility itself to insure appropriate application of the training information after training by the participants. Intended Outcome #5 The staff members will have positive attitudes about the training experience. Only partial support was provided by the evidence to indicate that this intended outcome did occur. In one of the most interesting sets of results, it would appear that the participants' positive attitude about the training experience immediately after training may have eroded somewhat during the six months following training. Of course, it must be immediately noted that the Satisfaction with Training Survey and the Follow-up Questionnaire were never intended to be compared with each other, only for each to provide its own set of data. Nevertheless, the participants were positive in their attitudes toward nearly all aspects of training imme- diately after training. Later they were only positive relative to specific techniques of reality therapy, behavior modification, and communication skills learned in training having been applied and helpful to their work during the six months after training. They were less positive regarding their enjoyment of the overall training program itself. Fltt)[‘ “‘l.‘ 210 Several explanations for this situation can be advanced. While the participants seemed pleased with the training program immediately afterwards, they indi- cated a lack of confidence in their own abilities to apply the information in their work as indicated by responses to the Satisfaction with Training Survey. This lack of confidence would seem to be justified, given the weakness of evidence to indicate that they had apprOpriately applied the training information in their work (see Intended Outcome #4). As they exper- ienced six months of dealing with resident delinquents in their work after training, the participants may well have become even more firm in their opinion that the training sessions had not adequately prepared them to face such a task and consequently caused a reduction in the positive attitude with which they had regarded their training experience. Other possible explanations include the training being too short and having insufficient follow-up built into the training process. As the participants were administered the post-training packet of paper-and-pencil evaluation instrumentation, they may well have sensed that they were performing significantly better on many of the items in the Cognitive Areas Objective Test. This may have caused them to feel quite positive about the training experience since they had obviously learned 211 something during it. This positive feeling may have then been reflected in the Satisfaction with Training Survey later in the packet. The truth of the matter may have been that the training program was just too brief to adequately prepare them to face the tasks of applying the techniques to their work. Insufficient follow—up by the training presenters or facility administrators during the next six months after training may then have had a disheartening effect on the participants. Of course, it would have been interesting to see how the participants would have responded if they had been offered more and repeated in—service training sessions. It is also possible that dissatisfaction with the facility's program itself on the part of participants "spilled over" onto their attitudes about training. The Follow-up Questionnaire was administered for convenience in a packet with a questionnaire to be used to evaluate the performance of the facility itself (not related to this study). A variety of attitudes and "gripes" were aired by participants in the self-evaluation questionnaire for the facility. It is possible that some of the frus— trations of the line staff were then carried over to the Follow-up Questionnaire section of the packet. 212 Additional (Unintended) Outcome #1 The staff members will demonstrate significant shifts in attitudinal preference regarding the type of delinquent youth with which they enjoy working. The evidence did not indicate that this outcome occurred. Actually, it was not intended to occur as a result of training. Instead, the outcome and the Cor- rectional Preference Survey were included as a matter of curiosity to see if the training program would affect the preference of the participants for working with certain types of delinquents. The specified antecedents or transactions had not been logically linked to this particular outcome and, as a result, it is difficult to examine them in light of its nonoccurrence. Nevertheless, it would appear that one of the possible explanations for the nonoccurrence of Intended Outcome #2 (improved atti— tudes toward use of concepts) could well apply to the nonoccurrence of this outcome. It is unlikely that such a brief training program can significantly affect atti- tudes as easily as cognitive knowledge. This would seem to be true for attitudes of various sorts. The ceiling effect explanation advanced for the nonoccurrence of Intended Outcome #2 would not seem to be indicated for this outcome. Shifts in preference for types are dif- ferent from a continuation from positive to negative as found in Intended Outcome #2, which can easily "ceiling out." Li 213 Implications for Future Training Programs Several implications can be drawn for future staff training efforts in the field of corrections from the evaluation results of this study. These implications are organized into four main areas in this section. The first area concerns the length of training. The second area relates to the type of material covered. The third area concerns the style and format of presentation. The fourth, and last, area concerns the expectations of training designers and those who fund training efforts. Length of Training The preponderance of evidence indicates that a two-week training program does not allow sufficient time to produce demonstrable desired outcomes. Despite the participants' stated satisfaction with the overall length of training and their feeling that the topics were covered thoroughly (except for the use of volunteers in corrections), evidence indicates that several problems may have been related to the length of training. The participants indicated a lack of confidence in their ability to apply the material without further help at the end of training, which may have been a reflection of the length of training or a reflection of their own lack of confidence as they began in a new work situation. They expressed dissatisfaction with the individual 214 lengths of time spent on reality therapy, communication skills, and the use of volunteers in corrections. The length of training may then have contributed to the minimum significance on the MANCOVA test of the post—training Cognitive Areas Objective Test (only one univariate, behavior modification, being significant with only reality therapy also approaching significance). It may have also contributed to the minimum ability of the participants to demonstrate that they were applying the information in their work later as well as the minimum amount of evidence to show that their abilities to work with delinquents were improved as a result of training. The implication of the evidence is that in—service training programs for correctional staff should be designed to be longer than the two-week, 24-hour training program evaluated in this study. Before the reader thinks that this implication should be obvious, several facts should be noted. The two training designers (Dr. Ernest Shelley and this investigator) together have over 40 years exper— ience in corrections. These two never came across any information in the field, or in the literature, to indi— cate that a training program of this length would not be likely to accomplish all of the goals! In addition, the facility and court administrators requested, and desired, to obtain funds for this particular length of training. 215 It is the impression of those two training designers that such requests, and subsequent training programs, are the norm in the field of corrections, not the exception, and, as such, the administrators were perfectly justified in their request. The evidence obtained by this evaluation, however, suggests that the confidence displayed in a two- week training program is not entirely justified. It should be noted that one way to "lengthen" training is to reduce the number of topics and cover the remaining ones more thoroughly. An alternative to be considered is having training programs be repeated on a scheduled basis, rather than lengthened. The recommendation to lengthen future training programs is echoed by Ewing and Gregg (1978) in their Executive Summary to the National Manpower Survey of the Criminal Justice System. While citing no evidence to support this recommendation, they indicate that training programs should be a minimum of 40 hours. However, some researchers believe that longer tra- ditional "knowledge—oriented" training programs may not necessarily result in the desired changes of behavior in the participants (Watzlawick & Weakland, 1977). They suggest instead implementing short-term highly behavior- oriented training programs. Certainly, modifying the type of material covered and the style and format of training must be considered as seriously as lengthening standard training. 216 Type of Material Covered The training designers and presenters attempted to emphasize technique and de-emphasize theory whenever possible. The results of the study and the review of the literature indicate that this is exactly what should be done. The study indicated that a significant amount of information was learned but that much less evidence was obtained to indicate that the information improved the participants' abilities or that they could even success- fully apply the information. This implies that tech- niques and their practice should be emphasized whenever possible in in-service training programs in corrections. Ewing and Gregg (1978), once again without citing evidence to support this recommendation, have recently advised the same thing. The training designers and presenters also attempted to emphasize human relations aspects and the humane treatment of residents, rather than more "guard- oriented" management of individuals. While the present investigation was not designed to test the efficacy of such an orientation and, consequently, presents no evi- dence to support it, Ewing and Gregg (1978) have indi- cated that this approach should be emphasized in cor- rectional training throughout the United States. ‘0;*—. -—1‘_ . 217 Style and Format of Training The training program evaluated in this study used a very traditional approach to training. Didactic presen- tations provided the core of the training and were alter- nated with discussions and videotapes. This format met with the general approval of the participants, at least immediately after training. Still, as a result of the lack of unequivocal support of the participants toward training six months later and because of the weakness of the evidence supporting the occurrence of several of the important intended outcomes, this format must be ques— tioned. Several conclusions regarding style and format are clear as a result of the study. Well—prepared, dynamic speakers are, of course, well received and highly successful in conveying the information to the partici- pants. The presenter for the topical area of behavior modification was rated the highest in proficiency and ability to hold interest in the topic. It is not sur— prising then that the topical area of behavior modifi— cation also was the only significant contributor to the MANCOVA on the post-training Cognitive Areas Objective Test scores, according to univariate analyses. Another conclusion is that flexibility in format is only to be desired to a certain point. The presenter of the information on the use of volunteers in correc- tions allowed himself to neglect that area in favor of 218 conveying other important treatment information to the participants. However, he never returned to that topic and, as a result, other potentially helpful information was never provided. In addition, when the participants indicated that they would prefer an additional session on reality therapy to another one on communication skills, this was done. Each of these changes, however, then seemed to be reflected in the weakening of the evidence to indicate successful outcomes of training concerning the topical areas of the use of volunteers in corrections and the successful application of communication skills. While interruptions were kept out of the training program for the most part, one significant interruption did occur at the end of training. Inflexibility, with regard to insurance salesmen who wish "a few minutes" of the train— ing time, would be well advised. An additional conclusion regarding the style and format of training is that a follow—up mechanism should be an integral part of any training program for correctional staff. Given the difficulty the participants had in appropriately applying the techniques taught in training, it is clear that additional follow—up was needed beyond that provided. Follow—up training sessions with partici— pants are only seen as a minimum measure. Supervisory staff should be coached in how to assist line staff in the application of training techniques. Periodic on— site observations should also be made after training by 219 training presenters to make immediate determination of problem areas in participants‘ application of the tech- niques. The evaluator would like to see other styles and formats of in-service training for correctional staff developed and evaluated. At this point, so little is known about what works in the field of in—service train- ing for correctional staff that additional information would be invaluable. Simulation training, for example, as used by Tinning (1973) in the field of medical edu— cation, has been used in a variety of fields. Appli- cation of this technique to the field of corrections would certainly be difficult but may well serve to sys— tematize training efforts and provide a new worthwhile approach to in—service training in corrections. Expectations of Training Programs This last area concerning implications for future in—service training efforts relates to the expectations that training designers and those who fund training efforts can realistically have of in—service training. This study has demonstrated that even relatively brief (but probably of average length) in-service training pro— grams can be expected to improve the participants' stores of knowledge on treatment modalities if the information is conveyed by proficient presenters to motivated par- ticipants in a setting conducive to learning. Brief 220 training programs are unlikely to result in easily demonstrable changes in attitudes or abilities, however. Training designers, presenters, and evaluators need to work closely with the administrators of cor- rectional facilities to better define what their expec— tations of training programs actually are. Such defi— nition will help serve to further direct the efforts of training designers and presenters. For example, if the training program funders desire demonstrated changes in abilities, training programs will have to be lengthened and their orientations modified. Such changes will, of course, likely be expensive. Clearly, the federal govern- ment in the form of the Law Enforcement Assistance Admin- istration is willing to assist by providing additional funding but will rightly insist that an evaluation com- ponent be built into the training efforts (Ewing & Gregg, 1978). Implications for Future Evaluations of Training Programs That additional evaluation efforts will be made in the field of in—service training for correctional staff is a foregone conclusion. The need for such efforts is obvious and is practically demanded by Ewing and Gregg's report (1978). Additional monetary support for such efforts by LEAA for doctoral dissertation research was even recently announced in a national journal ("Of Counseling Interest," 1978). Evaluation efforts in this 221 field must seemingly start with the basics. As Jesness (1978) indicates, in the field of corrections even the obvious must be demonstrated. Several implications for future evaluations have become apparent to this investigator through the course of this study. They are organized into three main areas in this section. The first area deals with the use of Robert Stake's model for the evaluation of in-service training programs in corrections. The second area dis- cusses implications regarding methods of observation to be used in this field. Thethird area describes impli- cations regarding design considerations of evaluations. Stake's Model Stake's model was selected for use in this study by the present evaluator primarily for two reasons. It is a flexible model that can be adapted to a "different" type of educational situation. It also was recommended by several experienced evaluators to an individual who was more familiar with in—service training in corrections than the means of evaluating educational programs. It was found to offer several advantages for use in evaluating an in-service training program for cor— rectional staff. It was indeed flexible, relatively easy to understand, and could be applied by a relative novice in evaluation technology (with considerable effort, how— ever). Stake's system of organizing the training program 222 into intended antecedents, transactions, and outcomes is also seen as being quite helpful in that this serves to focus both the evaluation and training efforts. As Ausubel (1960) points out, advance organizers of any sort are quite worthwhile. Stake's system proves to generate a great deal of data but also provides a sufficient system for organizing it. It also assures that attention is paid to aspects of the training program other than just outcomes of training. The model has several disadvantages for use in this field, as well. It does not include a cost-effec- tiveness component which would have provided additional worthwhile data. It also does not assist the evaluator a great deal in setting a priori standards and making consequent judgments. Stake (1975) even indicated that he, himself, knows relatively little about making judg— ments regarding programs. Stake's primary focus is on the generation and organization of evaluation data. This focus has the advantages of encouraging the evaluator to use a variety of observational measures and helping him to organize the resulting data but tends to neglect inter— preting that data. Means of Observation in Evaluation It would seem that deciding what to observe in the field of in—service training in corrections is less of a problem than finding a means to observe it. Every 223 method of observation used in this study, except the Correctional Preference Survey, had to be created by this evaluator. This would indicate a dearth of instru- mentation in this area. Even the Cognitive Areas Objective Test had to be develOped for this study. There is a clear need for further refinement of instrumentation to measure cognitive gain in training. A problem that will have to be overcome in this area is the lack of a reliability model that genuinely fits this type of evaluation measure. Test— retest reliability does not fit very well since after training the participant group is supposed to exhibit little variance in responses as a group and to change in their responses from those on the pre-training test. Measures of internal consistency, such as used in this study, have an inherent weakness. If a topical area is made up of several different key concepts rather than just one, the internal consistency will be reduced. The evaluator must try to find a balance between adding in homogeneous items to increase the estimate of reliability and keeping the instrumentation broad enough to allow for a variety of aspects to be investigated. Attitude measures relating to evaluations of in— service training programs for correctional staff must become quite sophisticated if they are to detect dif— ferences that do exist as a result of training. As indicated previously, any attitude changes as a result 224 of relatively brief training proqrams are likely to be subtle. Ceiling effects in pre-testing of attitudes must be carefully avoided. Of course, attitude change may well not be one of the key issues to be investigated in such evaluations. Certainly more critical measurement issues are how to measure changes in ability levels and how to determine whether or not the training participants are using the training information. The Critical Incidents approach, used in other fields and pioneered for the field of in—service training in corrections in this study, would seem to exhibit promise both as a training and assessment tool. The Staff Observation Rating Scale approach is time-consuming and expensive but provides an extremely close look at what the training participants are doing in their work after training. An alternative to be considered in future evaluation efforts would be to obtain recognized experts in the various treatment modality fields who did not participate in presenting the training and have these individuals observe the actual functioning of the training participants in their work. These individuals could then make judgments about whether the techniques were being applied appropriately after training. A similar approach is now being considered by Quay (1977). More effective means must be found to assess entry level sophistication of training participants. 225 By the time this evaluator had seen the computer printout of the statistical results of the pre—training instrumen- tation, the training program was over and it was too late for any adjustments in the program itself. Design Considerations in Evaluation Hemphill (cited in Payne, 1974) indicated that there are many differences between evaluation and research from problem selection procedures to hypothesis testing to the means of collecting data. He stated that evaluation efforts generally must be less precise and controlled than experimental research because of the pressures imposed on the evaluator by the real world situations and the evalu— ator's client. The evaluator of this study would question whether these distinctions are totally necessary. In fact, several of the suggestions made in this section encourage making evaluation efforts more research-oriented. Certainly situational pressures do affect evalu- ation efforts. For example, since the facility which was the site of the present study only opened during the training period, no opportunity was available to engage in pre-training observations of participant-resident interactions. Nevertheless, whenever possible, pre—post experi- mental designs should be used for evaluation efforts_of this type. In addition, quasi-experimental designs can 226 be used in evaluations, such as repeated time sampling to determine the immediate and long-range effects of training. If a means of obtaining random samples of participants in training and nonparticipants can be found, this of course would be ideal. A variety of dimensions of in-service training in corrections need to be explored in order to insure that the quality of in—service training will improve. The above—mentioned designs could be applied in investi— gating these dimensions. The following list of potential evaluation activities is provided in the hope that it will stimulate further examination of the processes and merits of various in—service training programs in cor— rections. 1. Determine the length of training that is maximally cost—effective. 2. Investigate the differential effects of concentrated versus more spread-out training proqrams. 3. Compare the effects of theory-oriented versus technique—oriented training. 4. Determine if different types of material should be presented to groups of employees with dif- ferent job classifications (teachers, child—care workers, etc.). 5. Determine which, if any, topical areas should be included in training for all job classifications. 227 6. Compare didactic versus participant involve- ment approaches to training. 7. Investigate innovative approaches to staff training, such as simulation training (Tinning, 1973). 8. Investigate effects of participant input into training processes. 9. Determine the optimally efficient number of topical areas to cover in training. 10. Determine the relationship of staff recep- tivity toward training to the outcomes of training. 11. Determine the relationship of staff satis- faction with training to performance after training. 12. Investigate staff personality, motivational, educational, and experiential variables likely to affect performance in training programs. 13. Determine the relationship of facility admin- istrators' commitment to training to the outcomes of training. 14. Determine the optimally effective number of staff participants to be enrolled in training at one time. 15. Investigate the relationship of staff per— formance on cognitive measures to performance in work situations. 16. Compare the effectiveness and ease of appli— cation of various evaluation models to the field of in- service training in corrections. 228 l7. Determine the optimally effective means of observing staff performance in work situations. 18. Determine the optimally effective means of detecting changes in knowledge, attitudes, and behavior that have resulted from in—service training programs. 19. Determine the most appropriate models of reliability and validity to apply to training evaluation observation. 20. Determine the most efficient means of assess- ing entry-level sophistication of training staff partici- pants. 21. Determine the most efficient means of detect- ing areas of misunderstanding of training material on the part of training participants during training. 22. Determine the most effective mechanism for follow-up after training prOgrams to insure the proper application of training material. 23. Investigate the relationship of the quality of in—service training programs to the effectiveness of the facility staff's work with residents. 24. Attempt to overlay the decision—making compo- nents of another evaluation model onto Stake's model, in evaluations of in-service training programs. 25. Determine which individuals have changed their behavior as a result of an in—service training program and attempt to determine why they did. 229 Summary The purpose of this study was to evaluate the merits and processes of an in—servicetraining program for the staff of a residential correctional facility for juvenile delinquents using an educational evaluation model. It has been widely accepted that improvements in the quality of in-service training for correctional staffs are needed in order for improvements in the quality of rehabilitation services provided by such facilities to take place. Ewing and Gregg (1978) indicate that the use of systematic evaluation procedures is crucial to improv- ing the quality of in-service training. The in-service training program evaluated in this study was a multi-unit training program funded by the Law Enforcement Assistance Administration for staff mem- bers of a newly opened residential detention and treat— ment facility for juvenile delinquents. The training program was designed to take place in the course of two weeks at the site of the facility. The primary focus of the training program was on treatment techniques to be used in the facility. The administrators of the county juvenile court and facility had chosen reality therapy and behavior modification to be the primary treatment modalities. The training program provided learning experiences related to those modalities and also regarding the effective use of volunteers in corrections 230 and techniques for the effective use of communication skills. The training program was intended to increase the staff participants' knowledge of the above-mentioned topi- cal areas and also their skill in implementing the related techniques in their work with delinquents in residence at the facility. The subjects in this study were the 27 employees of the facility who were involved in routine direct con- tact with the residents. Seven job titles were included in this group, with the largest two sub-groups being detention shift supervisors with 11 individuals and treatment group leaders with nine individuals. The remaining seven individuals were three teachers, the treatment coordinator, the detention coordinator, the special service coordinator, and a unit therapist. The setting for the study was the Monroe County Youth Center which is located in southeastern Michigan in a predominantly rural area. Construction of the facility was completed and it was opened for use in October of 1977 during the course of the training pro- gram. The facility serves dual functions in that it provides both detention (temporary holding placement) and residential treatment services for the Monroe County Juvenile Court. The detention side of the facility is divided into two units containing a licensed maximum of nine residents each. The treatment side of the facility is divided into three units of nine residents 231 each. Both the detention and treatment units use a combination of reality therapy and behavior modification as their treatment modalities. The plan to evaluate the training program was based on a model provided by Robert Stake, an education evaluator. Stake's model for evaluation provides for the collection and examination of data both on the products and processes of a program. In accordance with this model, the training program was organized in terms of intended antecedents to training, intended transactions during training, and intended outcomes of training. Ante- cedents to training are any conditions that exist prior to training that may relate to the outcomes, such as attitudes toward training on the part of the partici- pants. Transactions are any engagements between training presenters and training participants, such as lectures and discussions. Outcomes are the results of antecedent conditions in interaction with proqram transactions. Included as outcomes are changes in skill levels, knowl— edge, attitudes, and behavior on the part of participants. Stake's model encourages making observations of each of these antecedents, transactions, and outcomes in order to be able to examine the relationships (contin- gencies) among them. In addition, the congruence of the program (the degree to which the events and con— ditions of the program have occurred as they were planned) can be determined. A variety of observation methods 232 were devised or adapted for this evaluation. This involved developing cognitive tests, attitude surveys, an anecdotal record form, a critical incidents test, and a staff observations rating system. When appropriate, reliability measures were obtained. A priori standards were set to be used for judging whether or not the ante- cedents, transactions, and outcomes had occurred as intended. To measure whether the participants had gained in knowledge as a result of training, a pre- and post— training cognitive test was developed and administered to the participants and a comparison group not enrolled in training who work at another facility. To measure whether the participants became more positive in their attitudes toward using the techniques, a pre- and post— training attitude survey was developed and administered to the same groups. To measure whether the participants improved in their work abilities, a critical incidents test was developed and administered to the same groups. To measure whether the participants later used the infor- mation learned in training, a post-training staff obser- vation rating procedure was developed and implemented. In addition, a post-training follow-up questionnaire was developed and administered to provide indications as to whether the participants felt they had been using the information. To measure whether the participants had positive attitudes about the training experience after 233 training, a post-training attitude survey and a follow-up questionnaire were developed and administered. To measure whether the participants changed in their attitudinal preference toward types of delinquents with which they enjoy working, the Correctional Preference Survey (Quay, 1968) was administered pre- and post-training to the participants and the comparison group. Multivariate and univariate analyses of covariance were used to examine several of the intended outcomes of training. For others, measures of central tendency were compared to a priori standards. These tests were used to examine evidence relating to the six directional hypotheses. The results of this examination indicate that a significant amount of knowledge was gained by participants as a result of training. Only partial support was provided by the evidence to indicate that the participants improved in their work abilities, used the information after training, and felt positively about the training experience itself. No support was provided by the evidence to indicate that the training program resulted in improved attitudes toward the use of the techniques or that the participants shifted in their attitudinal preference toward types of delinquents with which they enjoy working. These mixed results are examined in terms of contingencies among antecedents, 234 transactions, and outcomes and congruences between that which was intended in training and that which actually occurred. Several implications for in-service training are advanced as a result of this study. Future in-service training programs should be lengthened. Short-term training programs should only be expected to have an impact on knowledge, rather than attitudes or even behavior. Techniques should be emphasized rather than theory whenever possible. Flexibility in approach is helpful only to a point. Use of proficient, well- prepared training presenters is essential. Follow-up mechanisms to training are advisable. Several implications for future evaluations of in—service training are also advanced. Several advantages and several disadvantages concerning the use of Stake's model in evaluation efforts in this area are discussed. The use of other models should be explored as well. Observational instrumentation is lacking in this area and represents perhaps the greatest need in the field of in-service training evaluation. GLOSSARY Antecedent Condition A term used by evaluator Robert E. Stake (1972) to denote any condition that exists prior to teaching that may relate to the outcomes of training or teaching such as attitudes toward training on the part of participants. Congruence A term used by evaluator Robert E. Stake (1972) to denote an estimation of whether or not the events and conditions of a training program have occurred as they were intended. Contingency A term used by evaluator Robert E. Stake (1972) to denote a relationship of the variables in a training program. Contingencies are the logical and empirical relationships among antecedents, transactiOns, and outcomes of training. Corrections For the purposes of this study, governmental or private efforts to remediate delinquent behavior by incarceration and/or treatment of adjudicated delinquents. 235 236 Delinquent (adjudicated) Any youth (under 17 years of age in the state of Michigan) who has been declared by a court to have violated a law, including all those in residence at the Monroe County Youth Center. Descriptive Data A term used by evaluator Robert E. Stake (1972) to denote descriptions of what training presenters intend and what observers of training perceive. Descrip- tive data are organized by Stake into a descriptive matrix including intents and observations of antecedents, transactions, and outcomes. Didactic Presentation A training transaction during which the training presenter primarily attempts to impart knowledge to the participants via lecture or nonpartici- pant demonstration. Evaluation Model For the purposes of this study, the type of evaluation procedure advanced by Robert E. Stake (1972). This model encourages the organization of the training program into antecedents, transactions, and out- comes. The data are organized into intents, observations, standards, and judgments. The evaluator, using the data that are organized in this fashion, then examines the congruences and contingencies of training. By comparing the results of training to that which was intended, or to other training programs (if they are described in the 237 literature), the evaluator creates a composite estimate of the merits and processes of the training program. Eypothesis Two types of hypotheses are necessary to this study. Research hypotheses are statements of pro— posed significant differences, such as between training— participant and nonparticipant groups on various measures. Null hypotheses are the corresponding statements of pro— posed nonsignificance that must be rejected based on the evidence in order to accept the alternative hypotheses. In-Service Training Program For this study, a short-term (two-week) training program for the staff of the Monroe County Youth Center, which is a detention and treatment facility for delinquent youth. The training program, which was implemented in the fall of 1977, was designed to cover the following topical areas: Reality Therapy As used in other residential treatment settings such as the Ventura School for Girls and as defined by Glasser and Zunin (1973). Behavior Modification As used in other resi- dential treatment settings, such as the Maple Village Campus of Starr Commonwealth for Boys, and as defined by Bandura (1969) and others. 238 Communication Skills For the purpose of this training program, synonymous with empathy, or facilitation training as defined by Alston (1971). Use of Volunteers in Corrections As conceptualized by Jorgensen and Scheier (1973) and Shelley (1971) regarding unpaid individuals assisting in the oper- ation of a juvenile correctional facility. Intent A term used by evaluator Robert E. Stake (1972) to denote that which is intended in relation to training prior to its implementation. Stake advises the advance defining of intended antecedents to training, transactions intended to occur during training, and intended outcomes of training. Intents include planned—for environmental conditions, planned-for lectures and activities, and all forms of goals and objectives. Judgment A term used by evaluator Robert E. Stake (1972) to denote a decision made by an evaluator after it has been determined whether or not a standard has been met for an antecedent, transaction, and outcome. Judgments provide data on whether the occurrence of the antece— dents, transactions, and outcomes are sufficient for a judgment of acceptability on the part of evaluators or others. Judgment Data A term used by evaluator Robert E. Stake (1972) to denote descriptions of standards and judgments. 239 Judgment data are organized by Stake into a judgment matrix including standards and judgments related to antecedents, transactions, and outcomes. Observation For the purposes of this study, any of the variety of methods of gathering data on the intended antecedents, transactions, and outcomes. Included as methods of observation in this study are objective multiple—choice tests, questionnaires, attitude surveys, anecdotal record forms, audiotaped observations, and personal observation. Outcome A term used by evaluator Robert E. Stake (1972) to denote a consequence of a training program. Outcomes are the results of the antecedent conditions in inter— action with the program transactions. Included as out- comes are changes in skill levels, attitudes, and any modification of behavior on the part of training program participants. Residential Treatment Facility Specifically for this study, the Monroe County Youth Center. In general, a setting for youths that requires an individual to live within the grounds of the facility and whose stated intention includes the behavioral and/or emotional treatment of that youth. 240 Residential Treatment Facility Staff For this study, all individuals who have routine contact with youths in the Monroe County Youth Center and are responsible for carry— ing out treatment and/or detention plans in that facility. Standard A term used by evaluator Robert E. Stake (1972) to denote a statement indicating what should happen in a given situation. Such statements can be made by the evaluators or others and in this study are often commonly accepted a priori statements of acceptable levels for statistical tests of significance. Stake's Model of Evaluation (See Evaluation Model) Statistical Procedures Analysis of Covariance (ANCOVA) A statistical procedure used in this study to test for effects, whereby covariates are inserted into a design in order to remove extraneous variation from the dependent variable. Regression procedures are used to remove the variation in the dependent variable due to one or more covariates. A con- ventional analysis of variance statistical pro- cedure (ANOVA) is then performed on the corrected scores. An anova is a statistical procedure used to test the simultaneous effects of several factors. 241 Multivariate Analysis of Covariance (MANCOVA) A statistical procedure based on the ANCOVA model which has been extended to test a set of dependent variables simultaneously. Stepwise Regression Analysis In this study, a statistical procedure used to determine which of the available independent variables should be included as covariates in an ANCOVA or MANCOVA procedure. It is a statistical technique by which the relationship between possible covariates and dependent variables is examined to determine which of the possible covariates might give a false impression of significance if not controlled for in some manner. Transaction A term used by evaluator Robert E. Stake (1972) to denote an engagement between a training pre- senter and training participants. Examples of trans- actions are lectures, filmed presentations, group dis- cussions, and the administration of tests. Transactions work within the framework of antecedents to training in order to produce outcomes of training. APPENDICES APPENDIX A DESCRIPTION OF TRAINING TRANSACTIONS FIRST WEEK APPENDIX A DESCRIPTION OF TRAINING TRANSACTIONS Training Program Schedule I. October 5, 1977, Wednesday A. Hour.1 1. Introduction by Jim Standifer, Administrator of Court Services 2. Dr. Ernest Shelley's introductory remarks a. Why interested in Monroe County Youth Center-- uniqueness of detention/treatment program b. Why pretest (to evaluate training) c. Why everyone involved in training 3. Jim Standifer--attendance mandatory a. First week—~all staff b. Second week--Monroe County Youth Center staff who have routine contact with residents 4. Administration of the pretest Hour 2—-Administration of the pretest Hour 3 l. ,Didactic presentation--Dr. Shelley--general principles of delinquency treatment a. b. No expendable people—-need even "hard to work with" ones (political implications) c. No one is hopeless d. Nothing works all the time with everyone--learn from our mistakes e. Treatment is everyone's job 1) Sociopath cured by night supervisor (personal anecdote) '2) Alcoholics Anonymous--people helping each other 2. Dr. Shelley—-princip1es of adolescent psychology worth noting a. 11 to l7--in-between years (kids know they're kids, adults know they're adults) b. Transition consists of jumping ahead and falling back Individual differences 242 -.~.—’pn-“- II. 243 c. Adolescent often feels intensely about rights of adulthood, revulsion about its responsibilities] d. In process of discovering identity 1) Sexual identity-—overemphasized by many pro- fessionals 2) Personal identity--underemphasized by many pro- fessionals e. 10% severely disturbed of delinquent population 10—15% not severely disturbed but need some pro- fessional help 75% only psychologically different in that concept of adult male role is distorted, immature, etc. so male models are important. What is the role of women in this field? 1) Women--persons first 2) Women--run counseling groups a) Helped inmates understand women more than bodies b) Mother figures f. Poem (written by inmate, challenging professionals to understand, help) g. Closing--plea for staff to try to remain enthused because they are affecting others' lives h. Kahlil Gabran verse dealing with children October 6, 1977, Thursday Hour l--Didactic presentation—-Dr. Shelley on reality therapy A. 1. 2. 3. Show of hands-—who has heard of reality therapy? Faddishness of psychology as new science (why not try it?) Rigidness of Freudian psychology resisted these new approaches Challenge of orthodoxy—-any system work if believe in it enough Glasser (Ventura School for Girls)--used ideas that were not novel, just neglected because of Freud Abraham Lowe--Chicago Recovery, Inc.--follow—up groups of patients for each other demonstrating effectiveness of group situations Glasser--understanding past doesn't necessarily affect present feelings, behavior Problems usually result from needs not being met (love, be loved, feel worthwhile) Hour 2--Didactic presentation continues 1. Fulfilling these needs responsibly without interfering with someone else is the key to reality therapy I like you, want to be your friend--is only a start to professional relationship Emotionally disturbed behavior is really irresponsible behavior "Mental illness"-—not included in the vocabulary of reality therapy _ Check physical, diet, etc. first (in response to question from participant) Break taken 244 Hour 3--Didactic presentation continues 1. 2. Present and future are emphasized—-not past Commitment to change in behavior on the part of adolescents a. Act leads to feeling b. Society concerned with act, not feeling c. Acts easier to control than feelings Importance of volunteers--post institution (briefly dis— cussed) How to use reality therapy a. Set limits for behavior b. Constructive arguments to change behavior c. Pin the kid down regarding changes d. Verbal shock if necessary to convince adolescent to make changes III. October 7, 1977, Friday Hour l--Didactic presentation-—Dr. Vincent Scalese--behavior A. modification 1. Generated topics to talk about. Participants indicated interest in: Contingency contracting Fitting reinforcement to behavior Negative reinforcement Importance of consistency Emotional change and value change . Limitations and problems of behavior modification Reinforcements for work (to personally understand behavior modification) a. Will change values based on reinforcement b. Increased values of other behaviors determine actions c. Manipulate environment Premack principle-—associate low frequency with high frequency to change behaviors Problems—-developing reinforcement schedule, maximizing reinforcement How to reinforce appropriate behavior Questions from audience HICDQIQU'Q) Hour 2--Didactic presentation continues 1. UlsbeN 0\ Cognitive dissonance-—convincing self you're doing right thing (why work when could steal) Fading (reducing institutional reinforcement) Change in self-image Modeling Behaviors are learned--interventions should, therefore, include teaching Questions from audience Break taken 245 Hour 3--Didactic presentation continues 1. 2. Life space interviewing—-Rad1 Antecedents, behavior, consequence--what do differently? Problems of behavior modification a. Targeting behavior--being creative b. Not just correcting old behavior but leading to new life styles c. Treatment tasks 1) What intend to accomplish? 2) What problem to be corrected? 3) What intervention are you going to employ? 4) Evaluation--is it really working? Demonstration a. "Bells" 1) Participants chosen to be the "white rat" with others to serve as the "reinforcers" by ringing bells 2) More complex task-~shaping behavior 3) Two opposing reinforcements (importance of con- sistency) Relabeling as a reinforcement tool IV. October 11, 1977, Tuesday Hour l--Communication Skills--Mr. Gerald Juhr A. 1. 2. 4. 5. Generates situations where communication skills are necessary Theoretical considerations a. Reality therapy--understanding as a need b. Behavior modification—-communication as a reinforce— ment c. Rogerian therapy-~communication as a therapy Three skills a. Active listening b. Positive Constructive Feedback 0. Negative Constructive Feedback Affective/cognitive messages-‘how to recognize each Demonstration-~interview of presenter, role player Hour 2-—Communication skills continue 1. 2. Participation exercise presented Demonstrations--triads a. Practice active listening b. One speaker, one interviewer, one observer Discussion of active listening Hour 3--Communication skills continue 1. Participant demonstration——I see (behavioral), I infer (subjective) Lecture on feedback Role playing in groups of six--active listening and negative feedback Asked about emphasis for second week of training (reality therapy expressed as interest) SECOND WEEK V. November 28, 1977, Monday A. Hour 1 1. Reality therapy videotape—-Glasser as parole officer with role player (parolee) a. Glasser asks, "What's your plan?" b. Lets parolee work if this causes him no trouble 0. Insists he go to continuation school until job is obtained d. Warns parolee about fights 2. Stopped tape-~discussion B. Hour 2--Tape continues 1 Discuss McDonald's job, what likely to be paid? 2 What to do so people will not take advantage of you 3. Apartment? Talk about it later 4. What do you want to take in school? 5. Discussion led by Dr. Shelley C. Hour 3-—Tape continues 1. What interested in in school? 2. Points out how much better be to go to school 3. Why not want to go to school-~teachers hassling him (reality of teachers not thinking he's changed) 4. Parolee volunteers that liked state school so he will "not go for" the officer's threats 5. Glasser puts it back in his court-—doesn't talk about those feelings 6. Discussion--several topics raised a. Natural consequences of behavior. Consequences in discussion of plan b. Kid is in control? Too loose? c. Reality therapy reinforces community standards and underachieving? d. Glasser weak on "What I might be" compared to "What I am?" e. Role of professional (according to Dr. Shelley)-- to push a little beyond community standards f. Check options in each situation and also review decisions afterwards VI. November 29, 1977, Tuesday A. Hour l--Group and Individual Treatment Planning (Behavior Modification) Dr. Vincent Scalese 1. Detention personnel as well as treatment personnel need to be aware of how to set up treatment plans. This will set positive tone for groups even during short stays in detention VII. VIII. 2. Generated topics a. Kid input into treatment planning--how to get it? b. Escaping from detention image (babysitter) c. Individual contracts with kids d. Fading process (reducing institutional reinforcement) 3. Outline of treatment planning Problem assessment Tasks for reinforcement Dissonance creation Evaluation of previous steps 0400‘!!! Hour 2--Treatment Planning Topic continues 1. Problem assessment a. Focus of goals, label problems in format 1) Behavior observation and interviews 2) Point system can be helpful here 3) Look at needs trying to fulfill b. Can identify pattern of behavior--compare notes with other staff members 2. Tasks for reinforcement--identify what you want to increase 3. Stages a. Defining the problem b. Determining the objective c. Choosing a method d. Evaluating the success of the plan Hour 3--Treatment Planning Topic continues 1. Creation of dissonance—-putting pressure on (sometimes not appropriate for some kids) Evaluation--important for staff and child Escape as topic (how to reinforce nonescape) Discussion about treatment ig_detention Fading (has to go to self-reinforcement) macaw November 30, 1977, Wednesday A. Hour 1 1. Insurance salesman—-45 minutes (unplanned interruption) 2. Fifteen to 20-minute review of behavior modification, communication skills, and reality therapy Hour 2 l. Posttest administered 2. Announced next session (follow—up) Dec. 14 or Dec. 15, observation of staff interaction with residents taking place until then December 14, 1977 Follow-up Session with Group Leaders on Group Management Issues A. Hour 1 l. Introduction--why this type of training and follow—up 2. Discussion--questions 248 a. What do if kid argues points (talk, don't argue) b. What do for negative kid, "losing" kid (have to repeat, bring back to subject, relabel) Hour 2--Continuation of discussion 1. Opposite--intelligent kid who argues for its own sake 2. How much creativity do you sacrifice with system? 3. Reality therapy presentation--eight aspects Hour 3--Mr. James Longhurst 1. Group management techniques from experience 2. Positive Peer Culture techniques 3. Group meetings run in Positive Peer Culture format J APPENDIX B COGNITIVE AREAS OBJECTIVE PRETEST--POSTTEST fiz- t rim_‘ _.'_"_—_.L‘ TH ' ' — 4‘ APPENDIX B COGNITIVE AREAS OBJECTIVE PRETEST--POSTTEST Reality Therapy 1. PrOponents of reality therapy treat delinquency as being a problem involving: a) emotional disturbance x b) irresponsibility versus responsibility c) inadequate home life ____d) society 2. Reality therapy deals primarily with: a) the past b) the present c) the past and present x d) the present and future 3. The following treatment technique(s) deal(s) primarily with the present and future: a) behavior modification b) reality therapy x c) both d) neither 249 250 Reality therapists believe that child-care workers can be effective with children as a result of: a) b) X C) d) the authority they exercise the similarity of the child-care worker - child relationship to that of a parent and child modeling and direct interaction providing timely reinforcement According to reality therapy, which is more important? X a) behavior motivations emotions morality Reality therapy indicates that: d) the child should discover right and wrong for himself the child-care worker should point out instances of right and wrong "right and wrong" have no place in child— care work "right and wrong" deal with administrative policy Reality therapy indicates that right and wrong should be defined a) x b) __c> d) In reality a) X b) by: an individual society families a committee therapy, "commitment" means: a promise on the part of an adult to "stand by" a child a decision to improve behavior 251 c) spelling out rewards and punishment d) dedication on the part of workers 9. According to reality therapy, which is more important? a) why we act the way we do _§__b) what we do c) how we feel d) religious beliefs 10. To a reality therapist, whether or not a child views himself as a failure is: a) of little consequence _f__b) a key issue c) of moderate importance d) only important concerning commitments 11. A child—care worker familiar with the concepts of reality therapy avoids the use of punishment because: X a) it doesn't work for delinquents who View themselves as failures b) he would rather build on the child's strengths c) it tends to cause disruptions in a group setting d) one person should not impose his/her values on another 12. According to reality therapists, commitments to improved behavior made by delinquents: X a) are best made in a group setting b) are best made only with adults present c) should not be made at all d) should only be made between the delinquent and his peers 252 13. Reality therapists believe that if a plan of improvement by a delinquent fails, it is best to: a) x b) C) 61) find out why it failed modify and redevelop the plan help the delinquent understand why explain the meaning of a promise to the delinquent 14. Reality therapists believe that delinquents' "excuses" for not succeeding: x a) should not be allowed are necessary in order for them to preserve their dignity in a group situation should be closely examined by the child-care worker and delinquent together really are valuable clues regarding the delinquents' defense mechanisms 'FSGKWJ‘v—‘a, _—. .--.--.e- ~. _ _-_~- __-~ 253 Cognitive Areas Use of Volunteers 1. An effective volunteer program is only as good as its: a) selection procedures b) training c) supervision x d) a, b, and c 2. Actually, volunteers should only be used for: a) relationship building with children b) reducing work loads for correctional workers c) clerical duties X d) a variety of duties 3. Volunteerism in corrections is: x a) growing b) staying approximately the same size c) just past its peak in terms of number of programs d) decreasing 4. Surveys have shown that: a) nearly all volunteer programs reduce recidivism X b) good volunteers are as dependable and dedi- cated as paid staff c) a and b d) neither a nor b 5. Volunteers function best when: X a) they are given specific tasks to accomplish b) they are given little guidance and a free rein to be creative 254 ____c) both a and b ____d) neither a nor b 6. Volunteerism generally: _§__a) saves money for courts b) costs courts money c) has no monetary effect on courts d) none of the above 7. One of the benefits of using volunteers is: a) to provide a variety of interpersonal con- tacts for children b) to save money for the court c) to educate the public about the needs of corrections X d) all of the above 8. Volunteerism is most consistent with the tenets of: a) reality therapy X b) democracy c) behavior modification d) socialism 255 Cognitive Areas Behavior Modification 1. The rewarding of appropriate behavior is defined by behaVior modification as providing: a) conditioning X b) positive reinforcement c) contingency contracting d) a and b 2. In general, reinforcement is most effective: a) after the youth has had time to think the situation over x b) immediately after the behavior has occurred c) simultaneously with the behavior of the youth —— d) when the youth requests it 3. Negative reinforcement is defined as: _§__a) the removal of an aversive stimulus ____b) a punishing stimulus c) a reward or punishment that is ineffective d) a harmful stimulus 4. The following is an example of negative reinforcement: X a) a parent yells at a child and when the child runs away, he finds that he no longer hears yelling b) a child tries to sneak an extra cookie and gets a slap on the hand from his mother c) a child-care worker finds that punishing a particular child seems to do no good d) a child-care worker loses his temper and slaps a child . 256 5. When a child imitates the behavior of a child-care worker, it is usually the result of: a) positive transference x b) modeling c) affective conditioning —— d) positive reinforcement 6. To be most consistent with the principles of behavior modification, when a child is misbehaving you should indicate to him that he is: a) losing privileges .53-£12"..— .. x b) not earning privileges c) earning punishments d) letting you down 7. In purely behavior modification treatment, who or what is primarily responsible for change: x a) the reinforcement b) the child c) the child-care worker ____d) the relationships 8. A contingency is: a) the natural result of a behavior b) the setting in which a behavior occurs x c) the agreed-upon consequence of behavior d) an agreement to work together on a problem 9. A boy is able to relate better with his peers as a result of learning how to relate more appropriately with a child—care worker. This is an excellent example of: x a) generalization b) positive transference 10. 257 c) modeling d) positive reinforcement A girl's behavior deteriorates rapidly upon release from a residential setting to her parents' home. A behavior modification expert would be most likely to say that this is a result of: a) her parents' lack of skill in setting up reinforcement b) lack of generalization of behavior x c) both a and b d) neither a nor b .4 V 258 Cognitive Areas Communication Skills 1. Active listening is best defined as: a) paying extra close attention to the other person x b) indicating a tentative understanding of what a person has intended to communicate c) encouraging a person to talk about his problems d) all of the above 2. Active listening is an important skill because: a) it is helpful to tell people what you think about them b) it is difficult to listen to people unless you are really interested in what they say X c) people often need help in accurately com- municating their feelings d) delinquents speak a "different language" and you have to be sharp to understand them 3. Active listening includes: X a) paraphrasing, observing, and questioning b) observing, questioning, and monitoring c) questioning, monitoring, and clarifying d) monitoring, clarifying, and paraphrasing 4. Paraphrasing is most effective when done: X a) tentatively b) decisively c) with deep feeling d) with as little feeling as possible ————-— V 259 Ehen a child tells you about a feeling, it is helpful 0: a) ask a direct question in order to get a direct response b) tell the child you understand how he feels x c) share your perception of his feeling tentatively d) all of the above A good "active listening" response: a) switches the focus of the conversation to the listener x b) indicates your perception of what the person said c) indicates how you think the person's problem can be solved d) does not necessarily have to demonstrate that you are interested in what was said A "cued question" is one that: a) helps a person formulate his thoughts and continue talking b) lets a person know that you understand X c) imposes direction and value on the con— versation d) gives a person clues as to what has been said The purpose(s) of active listening is(are): Check one or more X a) to facilitate conversation X b) to help understand another person X c) to help another person understand himself d) to help lead the other person into seeing your point of View 9. 10. 11. 12. 13. 260 It is possible to "actively listen" to: a) verbal messages b) nonverbal messages c) a tone of voice X d) a, b, and c The best definition of positive feedback is: a) a compliment concerning behavior b) a statement that is intended to be helpful x c) an indication of support for behavior d) letting a person know that you like him/her The best definition of negative feedback is: a) an honest statement of negative emotion X b) a statement disapproving of behavior c) a statement that is not helpful to the person d) a statement expressing disapproval of a —— person To be effective, feedback to delinquents should be: X a) specific b) positive in nature c) done only in one-to-one situations d) all of the above Good feedback to delinquents about behavior indicates: a) that the behavior can be changed —— b) the intent of the person in giving feedback —— c) the consequences of the behavior X d) all of the above 'l 14. 15. 261 Feedback should emphasize: a) b) X C) d) emotions motivations behavior intentions A delinquent will be more likely to "hear" feedback if: X a) b) C) d) the statements are tentative the statements are delivered forcefully the statements refer to unusual behaviors you demand the delinquent's attention first APPENDIX C ATTITUDES TOWARD USING CONCEPTS SURVEY PRE-TRAINING--POST—TRAINING Check one (v’) 1. APPENDIX C ATTITUDES TOWARD USING CONCEPTS SURVEY A a d D PRE-TRAINING--POST-TRAINING A = Agree very much a = Agree somewhat d = Disagree somewhat D = Disagree very much for each: Behavior modification is an effective treat- ment approach with delinquents. If a delinquent changes his behavior in order to obtain rewards, it is doubtful that his attitude has really changed. If a child is consistently rewarded for his good behavior, he is unlikely to accept real responsibility for his actions. It is important to punish bad behavior prior to rewarding good behavior. Good communication skills on the part of child-care workers are important factors in the treatment of delinquents. Encouraging an unhappy child to talk about his feelings generally only makes him, and those around him, feel worse. A child is generally better off when listen— ing to good advice than when he is talking about his own problems. 262 10. 11. 12. 13. 14. 15. 16. 17. 18. 19. 263 Discussions about feelings with children should be handled by treatment specialists rather than child-care workers. Criticism usually has a negative effect on children. Reality therapy is an effective treatment approach with delinquents. Adolescents generally end up in institutions because they have not accepted responsibility for their own behavior. You can help a child to develop a sense of responsibility. A child—care worker should encourage a child to focus on his behavior rather than his emotions. A child—care worker should encourage a child to focus on the present and future rather than his past life. Using volunteers to work with delinquents in a treatment facility is generally worthwhile. Time spent training volunteers would be more effectively spent working directly with children. Most volunteers lack the skills necessary to really help delinquents. Children act out and "test" volunteers, which destroys gains made in treatment. Even though they lack specific skills in work- ing with children, volunteers benefit delin— quents. APPENDIX D CRITICAL INCIDENTS PRETEST--POSTTEST APPENDIX D CRITICAL INCIDENTS PRETEST--POSTTEST Try to imagine yourself as a staff person who is faced with the following situations. Please choose the answer that most closely approximates what you would do, even if it differs from what you think you "should" do. 1. A boy comes up to you and asks for a privilege not extended to the others. He has been in the institution for three weeks and has been extremely quiet. This is his first unsolicited interaction with an adult. You: a) give him the privilege in order to positively reinforce his approaching you b) make it clear to him in a nice way that privileges must first be earned x c) indicate that he can't have that privilege but encourage him to talk about other things he likes to do d) explain to him how unfair that would be to the others A boy becomes angry after losing at ping pong and throws his paddle down. He swears at you when you suggest that he calm down, saying, "It's none of your damned business, you bastard." You: x a) point out to him that while most people feel bad when they lose, it doesn't have to lead to more trouble and worse feelings b) tell him that if he continues to swear, he will have to be restricted from playing in the game room for a short period of time c) ask him to leave the game room immediately and return when he has calmed down d) point out his childish behavior to the other boys and allow them to put social pressure on him to behave more appropriately 264 111,1 1 265 7 Critical Incidents 3. A boy says to you, "I had a great home visit this weekend! I stayed out all night! It pissed off my mom, but it was worth it." You: a) give him a mild punishment or an extra duty in order to make sure he is not rewarded for his misbehavior b) ignore his statement so he won't gain attention for his negative behavior x c) ask him to talk about why he was willing to behave in a way that hurts his relationship with his mother d) take away his home-visiting privileges so the trouble won't continue A girl says to you, "I really feel lonely. Nobody cares about me or Shows any interest in me. I could disappear and nobody would even notice." You: a) convince her that some of the other children do like her b) suggest some ways that she could spend more time around you so she would not be so lonely c) try to help her by pointing out a number of the things she does to alienate other people and give her clues about how to be more popular x d) try to get her to talk about the part she plays in this problem In,a setting where a behavior modification system is being used, the boys are looking forward to being able to attend an upcoming basketball game. You see a new boy misbehaving. You know that if he continues, he will not be able to attend. You: a) quietly let him know that he is losing his privilege by misbehaving X b) tell him he is not earning his privilege when he misbehaves c) enter the situation and distract the child from his mis- behavior so that he doesn't feel defeated before he really gets started in the program d) say nothing in order to let him find out more quickly for himself how the system works 266 Critical Incidents 6. A girl who has been in your group for quite a while says to you, "I know I promised you I'd work extra hard to pass math this time, but I changed my mind. It's just too much work." You: a) let her know that she will be letting you down if she gives up b) remind her what a promise means in terms of responsibility and urge her to make a real effort x c) ask her which parts of math are giving her trouble d) help her to see that breaking promises can become a habit and that it would be better for her not to give up A boy announces loudly to you the first thing in the morning, "I'm going to mess up today and really be bad!" You: a) restrict him as a punishment for being negative b) warn him that you will be watching him closely that day X c) ask him why he thinks so d) ignore him so as not to reinforce his negative behavior A large boy in the unit teases others mercilessly and picks on smaller boys often. You come into a room and find two smaller boys he had previously tormented fist—fighting with him, and in fact, beginning to best him. You: a) let them fight until you think the bigger boy has learned a lesson about picking on other boys b) have the three boys put on boxing gloves and go at it again in a more controlled setting c) break up the fight, lecture the smaller boys about fighting fairly, and the larger boy about teasing X d) break up the fight and encourage the boys to talk about their relationship problems A boy whose behavior has improved greatly during his two months in residence mainly due to your influence spends most of his spare time with you and is obviously attached to you. You: X a) encourage him to spend more time with other staff and peers b) encourage the development of your relationship since it is helping him 267 c) neither encourage nor discourage any relationships. Let him decide for himself d) discourage his spending so much time around you 10. A delinquent in your unit has complained about the rules so much that you lose your temper and really yell at him, even swearing mildly. You: a) place the boy in isolation until he can verbalize to you what he has done wrong b) explain to him why complaining makes the situation worse c) ask him to tell you why he complains so much x d) apologize and explain why you got angry at him APPENDIX E FOLLOW-UP QUESTIONNAIRE __ Ill.fr.l##x[> APPENDIX E FOLLOW-UP QUESTIONNAIRE Name Job Title Date 1. l Did you attend any of the in-service training sessions presented by Dr. Shelley and Rick Howard last October and November? Yes If yes, which training sessions did First week Wednesday, October 5 Thursday, October 6 Friday, October 7 Tuesday, October 11 Second week Monday, November 28 Tuesday, November 29 Wednesday, November 30 Rate the extent to which you have applied in (D U) 0 III NO you attend? your work, (Check one) information learned in each of the following areas presented in the in—service sessions. (Check one for each area.) Behavior Modification Reality Therapy Communication Skills Use of Volunteers Daily Occasionally Seldom Never 268 Did Not Attend 269 Rate the extent to which in-service training in the following areas has helped you in your work. (Check one for each area.) Very Somewhat Of Little Not Did Not Helpful Helpful Help Helpful Attend Behavior Modification Reality Therapy Communication Skills Use of Volunteers _—_—_——_—— How worthwhile do you feel the in-service training sessions have been for the Monroe County Youth Center staff? (Check one.) Very valuable Somewhat valuable Of some value Of little value How much did you personally enjoy the in-service training session? I enjoyed them very much. I enjoyed them most of the time. I rarely enjoyed them. I did not like them. I did not attend them. Please indicate the major weakness(es) of the training program (if you attended). 1. 3. Please indicate the major strength(s) of the training program (if you attended). 1. 2. APPENDIX F STAFF OBSERVATION RATING SCALE INSTRUCTION MANUAL APPENDIX F STAFF OBSERVATION RATING SCALE INSTRUCTION MANUAL Introduction The purpose of this rating procedure is to obtain an estimate of how often and how appropriately staff persons in a correctional insti- , tution for delinquents exhibit certain behaviors. Audiotaped obser— ' vations have been made in the institution and are presented to you in separate situations ranging from five to 20 minutes in length. One Staff Observation Rating Sheet will be used for each situation. Rating of the staff behavior requires two steps. First, a fre- quency count of behaviors is recorded. Next, a rating of the appro- priateness of the respective behaviors is undertaken. It is suggested that you listen to each situation twice; the first time record fre— quencies of behavior, the second time rate the appropriateness of the behavior. How to Record Frequencies of Behavior Categories of staff behavior have been listed and defined (see Definitions and Symbols). Each staff behavior should have a category in which it can be tabulated. In order to be able to properly cate— gorize the behaviors, you will have to become quite familiar with the definitions and symbols provided. It is, in fact, expected that several training sessions will be necessary in order for the distinctions between them to be adequately learned. As a first step, of course, you should memorize the category definitions and their corresponding symbols. For each complete staff behavior, record a mark in the Frequency column in the appropriate category on the Staff Observation Rating Sheet. For example, if the following took place on the tape: "Where is the mop?" (Pause) "Does anyone know where the mop is?," you would record ggg_mark (not two) for the AQ category under the Neutral Responses heading in the Frequency column. How to Rate Appropriateness of Behavior For each behavior category, one rating scale is provided, with the exception of RPM--Response to Problematic Behavior which is rated on six 27C) 271 dimensions. The rating scales range from one (highly inappropriate response) to seven (highly appropriate response). After listening to the tape for a second time, on the Staff Obser— vation Rating Sheet rate the appropriateness of the behaviors for which you have recorded marks. Make a mark under the number which best indi- cates the appropriateness of that staff behavior, from one to seven in the area on the right side of the Staff Observation Rating Sheet (Rating of Appropriateness of Behavior). In rating the appropriateness of behaviors you should consider two factors. First, was that type of response indicated, given the situation? Second, how skillfully did the staff person accomplish the response? It is understood that it is quite difficult to judge the appropriateness of someone's actions even when you know all of the parties involved and the situation very well, much less when you don't. However, attempt to use all available clues including the circumstances, tones of voice, and even the results of staff behavior, in rating behavioral appropriateness of these situations. High ratings of appropriateness should be given for those behaviors on the part of staff members that promote healthy rapport between resi- dents and staff and are clearly helpful in the remediation of maladaptive behavior on the part of residents. Neutral ratings of appropriateness should be given for those behaviors that, while not being particularly helpful in the promotion of rapport or particularly helpful in the pro- motion of adaptive behavior, are not at all destructive to either of these. Low ratings of appropriateness should be given for any behavior on the part of staff members that interferes with healthy relationship building or interferes with the treatment of delinquent behavior (includ- ing missing the opportunity to promote the above). The rating of appropriateness for the RPB--Response to Problematic Behavior category bears special mention. Note that six dimensions of appropriateness are provided. High degrees of appropriateness on some dimensions will not necessarily mean equally high ratings on others. The first dimension is that of intentions. If the intentions of the staff person are clearly to help the child, rate the response as being highly appropriate. If the intentions are unclear or even destructive, rate the response lower. The second dimension considers how clearly the staff person described the specific problematic behavior with which he was concerned. High appropriateness ratings should be given for responses in which the staff person emphasized and described to the youth in clear under- standable terms the problematic behavior he saw. The third dimension relates to whether alternative behaviors to the problematic behavior were pointed out to the youth by the staff person. Did the staff person just say, "Stop bugging him or I'll send you to your room" or did he offer other alternatives to the poor behavior? High ratings are given for the latter. 272 The fourth dimension relates to whether the staff person empha- sizes present behavior or past behavior. High ratings should be given to emphasis of the present or future. Low ratings should be given for over-emphasis of past events in the youth's life. The fifth dimension is concerned with whether or not the conse- quences of the problematic behavior were pointed out to the youth. High appropriateness ratings should be given for responses that help youths see how problematic behavior hurts them. The sixth dimension considers whether the staff person encouraged the youth to commit himself to improving his behavior, make specific plans to do so, or make new plans if old ones have failed. High ratings should be given for attempts to do this with youths. Moderate ratings should be given for ignoring this aspect and low ratings for discouraging this activity. Thank you for your time and effort in this endeavor. It is hoped that you will have helped to contribute to a better understanding of the correctional treatment of delinquency and improvement of correctional staff training as a result. 273 Definitions and Symbols The following are the symbols and definitions for the categories of the staff behavior you will be observing. Note that there are four main headings for these behaviors: Positive Reinforcement Responses, Listening Responses, Neutral Responses, and Responses to Problematic Behavior. Please locate these headings on the Staff Observation Rating Sheet before you go on. Under each of these headings are several categories of behavior, except for the Responses to Problematic Behavior heading, which is rated for appropriateness on several dimensions, but consists of only one category of behavior. The symbols and definitions of each of these categories is provided below. Please become familiar with these defi- nitions. Positive Reinforcement Responses Categories under this heading refer to staff behavior on the audiotapes that seems intended to, or does, elicit increased rates of certain behavior on the part of residents. CRD Administration of Concrete Rewards (Direct): Giving of direct concrete rewards such as candy, money, or free time. This cate— gory also consists of those instances when the staff member gives concrete but symbolic rewards (such as giving flashcards to a youth contingent upon correct answer to that card) which have no backup or other value. CRT Administration of Concrete Rewards (Token): Giving of symbolic rewards which will be redeemed for direct concrete rewards at some future time. Common examples are poker chips, tallies, colored sticks, stars, stickers, etc. AAB Affirmation of Appropriate Behavior: Verbal contact indicating approval, commendation to a youth that his responses are correct or acceptable, or that his behavior is appropriate. Verbal Affirmation may either be loud or soft, and consists of such examples as "That's good," "Fine," "You‘re studying well." RP Rapport-Praise: Evaluative reactions which go beyond the staff member's level of simple affirmation of positive feedback by ver— bally complimenting the youth. RP communicates a positive evalu- ation or a warm personal reaction to the youth and not merely an impersonal communication. Staff person responses are considered RP if the content (Yes, Umhumm, Fine, Good) is accompanied by com— munication of warmth, joy, or excitement. AM Adjuvant Mastery: Urging, prompting, fostering, promoting con— fidence and success, providing encouragement for response production. When the youth refuses to participate, the staff person may suggest guessing, give encouragement ("You just got the last one"), or systematically employ a graded series of suggestions. Includes encouragement to finish a job or to do it better, if done so as to promote confidence and success. 274 k AE Aiding by Example: Demonstration of appropriate behavior by staff person when the youth is either nonresponsive or incorrect in exhibiting expected response. Listening Responses Categories of behavior under this heading refer to the staff behavior on the audiotapes that is intended to, or does, elicit increased open communication of residents. .As such, this behavior can also be considered to be positively reinforcing. The distinction between these behaviors and those under the previous heading is that these behaviors specifically reinforce communication behavior on the part of residents. AF Accepts Feelings: Staff member accepts and clarifies the feeling tone of the youth in a nonthreatening manner. Feelings or youth's emotions may be positive or negative. Predicting or recalling feel- ings is included. The staff person accepts feelings when he says he understands how the youth feels, that he has a right to those feelings, and that he will not punish the youth for his feelings. AI Accepts Ideas: Clarifying, building, or developing ideas sug- gested by the youth. Staff person may paraphrase the statement, restate the idea more simply, or summarize what the youth has said. The key staff person behaviors are clarifying and developing ideas. Simple restatement without building such as when staff person verbalizes youth's answer during recording on chalkboard or test booklet is not scored. Neutral Responses Categories of behavior under this heading refer to the staff behavior on the audiotapes that is never intended to be reinforc— ing or critical of behavior on the part of residents. AQ Asks Questions Asking questions by staff person. In this category neither positive nor negative evaluation of the youth is present in the question. NEVR Nonevaluative Verbal Response: Other responses by staff person which are neither criticism nor affirmation of behavior initiated by the youth. Responses to Problematic Behavior This heading contains only one category--responses by a staff person to inappropriate or problematic behavior by a youth. Includes staff behavior ranging from admonish— ments, such as, "Don't do that!" to extended discussions of past problematic behavior. Criticisms, verbalized disapproval of behavior, suggestions to youth regarding advised changes in behavior, etc., fall in this category. APPENDIX G STAFF OBSERVATION RATING SHEET APPENDIX G STAFF OBSERVATION RATING SHEET sit 3 UR?) '1 Raina ', " Appropriateness of Pehavior Highly Highly inappropriate Neutral Appropri 1'0 es Behavior ponse ResPonse RESPOHS' Category FI‘ guency Total 1 2 3 4 5 7 Positive " ‘ * Resronses -— CRD CPT AAB RP All _ ______ .1_ AF 1’ —— Listening :sponses AF AI Neutral Refl‘onses 8') HGV‘P Region , z; Problematii RATING OF ASPECTS OF __ RESPONSES TO PROBLEMATIC BEHAVIOR Rrv . ‘ Intentions Clear — —_ I _ _. — PM): mMed and Specific —‘ Alternative Behaviors Offered —_ ' “~— Present Situation Emphasized _— ______,_....___ CMences—‘of—Behavior Made Clear —_ —— ~--—- ----- 5mm mane—fixing oTzzEp—lanmng 275 APPENDIX H SATISFACTION WITH TRAINING SURVEY APPENDIX H SATISFACTION WITH TRAINING SURVEY Name Date Job Title Length of time at that job Length of experience working with children Education (last year completed) If college, what major INSTRUCTIONS The following pages contain some questions that deal with opinion or attitude and some that are factual in nature. Please answer all questions. When you do not fully agree with any of the choices provided, please choose the answer that most closely approximates your thoughts. Which training sessions did you attend? First week Yes No Wednesday, October 5 _____ Thursday, October 6 _____ _____ Friday, October 7 ______ ______ Tuesday, October 11 276 Second week Monday, November 28 Tuesday, November 29 Wednesday, November 30 ATTITUDES TOWARD TRAINING We need information concerning the training sessions that you attended. This information is for the improvement of our program. Please circle the letter which best reflects your feelings about each statement. Please read each question carefully and after you have completed the test, check to make sure that you have answered every question. A = Strongly agree a = Agree d = Disagree D = Strongly disagree A a d D l. The length of the in-service training (two weeks) was adequate for me to understand the concepts. Comments: 2. I have some reservations about the usefulness of some or all of the information presented on: A a d D a) Principles of Adolescent Psychology A a d D b) Use of Volunteers in Corrections A a d D c) Reality Therapy A a d D d) Behavior Modification A a d D e) Communication Skills Comments: A a d D 3. My questions were adequately answered by the training presenters. Comments: D? P W W W W V W P W QJQQJQJQ- QQJQJQQ- QIQIQ-QIQJ 278 4. In my work, I will use the information on: a) Principles of Adolescent Psychology b) Use of Volunteers in Corrections c) Reality Therapy d) Behavior Modification e) Communication Skills Comments: 5. The length of time spent on each of the topics was adequate: a) Principles of Adolescent Psychology b) Use of Volunteers in Corrections c) Reality Therapy d) Behavior Modification e) Communication Skills Comments: 6. I feel I will need more help before I will be able to understand and use in my work: a) Principles of Adolescent Psychology b) Use of Volunteers in Corrections c) Reality Therapy d) Behavior Modification e) Communication Skills Comments: >u>u>n>u> W W W a F OJQJQIQJQJ 0.: 0404040.: DID-040.10.: 279 7. I believe the training presenters were suf— fiCiently expert in the areas of: a) b) C) d) e) Principles of Adolescent Psychology Use of Volunteers in Corrections Reality Therapy Behavior Modification Communication Skills Comments: 8. There was sufficient opportunity for discussion during the training sessions. Comments: 9. I am satisfied with how much I have learned about: a) b) C) d) e) Principles of Adolescent Psychology Use of Volunteers in Corrections Reality Therapy Behavior Modification Communication Skills Comments: 10. The presenters held my interest concerning: a V b) V c d) e) Principles of Adolescent Psychology Use of Volunteers in Corrections Reality Therapy Behavior Modification Communication Skills Comments: 280 11. I would like to learn more about: A a d D a) Principles of Adolescent Psychology A a d D b) Use of Volunteers in Corrections A a d D c) Reality Therapy A a d D d) Behavior Modification A a d D e) Communication Skills Comments: 12. Please rank order in the order of importance to you in your work the following aspects of the training (1 indicates that aspect most helpful to you, down to 5 which indicates that aspect least helpful to you). Please provide a ranking number for every aspect. Principles of Adolescent Psychology Use of Volunteers in Corrections Reality Therapy Behavior Modification Communication Skills SATISFACTION WITH TRAINING SURVEY Please indicate (by circling) the extent to which you feel the following areas were presented during the training sessions. 2380 = Completely presented Thoroughly, but not completely presented Minimally presented = Not presented C T M N Principles of Adolescent Psychology C T M N Use of Volunteers in Corrections C T M N Reality Therapy — Theory C T M N Reality Therapy — Techniques 281 C T M N Behavior Modification - Theory C T M N Behavior Modification — Techniques C T M N Communication Skills - Theory C T M N Communication Skills - Techniques Please provide any comments regarding any aspect of the training that might help us improve in-service training seSSions in the future: TRAINING HISTORY Did you have contact with resident children during the training sessions? Yes No If yes, was it during the first week of training? Yes No in between the two weeks of training? Yes No during the second week of training? Yes No If no, for what reason? Did you receive any other additional training or information regarding the following since the beginning of our in-serVice training sessions? a) Principles of Adolescent Psychology b) Use of Volunteers in Corrections c) Reality Therapy d) Behavior Modification e) Communication Skills lllll If yes, please describe: APPENDIX I CORRECTIONAL PREFERENCE SURVEY PRE-TRAINING--POST-TRAINING APPENDIX I CORRECTIONAL PREFERENCE SURVEY PRE-TRAINING--POST-TRAINING On the following pages are items designed to determine the preferences of correctional personnel for working with certain kinds of delinquent youths based on their opinion about correctional practices, the characteristics of youths, and other factors. It is to be expected that there will be considerable differences among youth workers on most of the items. Please do not omit any of the items even though you may feel some doubt about your responses. It is best not to take too long to respond; your first choice is likely closer to your real feelings. To indicate your response simply circle the position of the Scale (A, a, d, D) which reflects your feeling or opinion about the statement. 282 283 Correctional Preference Survey A = Agree very much a = Agree somewhat d = Disagree somewhat D = Disagree very much 10. 11. 12. 13. d D d D d D d D d D d D d D d D d D I think I have more patience than the average correctional worker with delinquents who are impulsive and have short attention spans. I think a staff person should have very, very firm ideas about what is right and what is wrong. I like to challenge a delinquent to make a real commit— ment to a nondelinquent way of life. A child's worst enemy in a correctional school is boredom. I do not like a living unit where the adolescents are not good in competitive activities. The kinds of kids who get my goat are those who will always lie to gain their own ends. Delinquent behavior is generally a sign that a child is emotionally disturbed. I think most delinquents have to learn not to follow the crowd. Most delinquent kids would be o.k. if they had the work skills to compete in the outside world. I don't mind admitting to an adolescent that I can make mistakes. You can learn a great deal about a child simply by noting with whom he/she associates. I feel more comfortable working with delinquents who are not easily bothered by things than with those who are more easily upset and tend to show their feelings. The trouble with most delinquents is just that they have used the wrong means to get the right ends. 14. 15. 16. 17. 18. 19. 20. 21. 23. 24. 25. 26. 27. 28. 29. 30. A a d D A a d D A a d D A a d D A a d D A a d D A a d D A a d D A a d D A a d D A a d D A a d D A a d D A a d D A a d D A a d D A a d D 284 My style of working with delinquents is casual rather than strict. Disciplinary actions for rule violations are best handled without a lot of bother about what may have led up to the situation. I prefer to work with adolescents who have a lot of self— confidence rather than ones who are weak and need pro— tection from themselves and others. With many boys and girls, you have to move the whole group to move them. If kids can be kept busy with something they like, they will cause much less trouble. I think that talking to most delinquent kids just gives them an opportunity to "con" you. When a child makes a decision, I am willing to take a chance and go along with him. The idea that adolescents are delinquent because of forces beyond their control is generally nonsense. Delinquents don't have much of a chance of solving their problems unless they will talk about them. I prefer to work with delinquents who are group-oriented rather than individualistic. I find working with tough kids a real challenge. I'd very much rather work with adolescents who are quick to understand than with those who may need repeated directions. If a delinquent flares up for no apparent reason he should be protected from the consequences of his behavior. Some kids have to be "led by the hand" into becoming more able to look out for themselves. I enjoy trying to reach the shy, socially isolated individual. I find it is very difficult to work with the kind of child who is always demanding attention. I don't mind working with kids who go along with the pro— gram but don't really change their feelings about things. 31. 32. 34. 36. 37. 38. 40. ‘41. 42. 43. 44. 45. 46. 47. A a d D A a d D A a d D A a d D A a d D A a d D A a d D A a d D A a d D A a d D A a d D A a d D A a d D A a d D A a d D A a d D A a d D 285 Working with boys and girls who like me one day and are mad at me the next doesn't bother me at all. I would rather work in a living unit where the structure is clear and the limits are very tight. It is important to keep some delinquents out of situations which are too difficult where they might fail and feel unhappy about it. I don't mind working with kids who are always trying to put one over on you. I do my best work with kids who are not really delinquent in their attitudes but tend to be clumsy, helpless and easily led. Putting your arm around a child in a parental manner is often very important. I prefer working with adolescents who keep their personal problems to themselves. I find it difficult to work with boys and girls who admit they're wrong in doing something but do it anyway. Having a kid think I am paying a lot of attention to him doesn't bother me at all. I think it is better to discipline boys and girls who need it without a lot of talk about what may have led up to the particular problem. I would rather work in a living unit where boys and girls are given responsibility to make decisions about the rules within the unit. I don't mind working with scared and anxious delinquents. Scared and unhappy children need a chance to express themselves to an adult willing to listen. Most delinquents are not much different from other kids. I think it is almost always wrong to use ridicule with delinquents. If you are open and honest with delinquents, they will be honest with you. When I was a child, I had quite a few delinquent kids as acquaintances. 48. 49. 50. 51. 52. 53. 54. 55. 56. 57. 58. 59. 60. A a d D A a d D A a d D A a d D A a d D A a d D A a d D A a d D A a d D A a d D A a d D A a d D A a d D 286 I think I can work well with boys and girls who get con— fused easily and need lots of attention. Delinquents who try to "con" me and others are hard for me to like and work with. It bothers me to have to work with boys and girls who act much younger than their age. I find it more satisfying to work with the kind of delin- quent who seems to have an understanding of his problems. I prefer to work with "average“ kids rather than those who are more difficult or mixed up. I think a close relationship with a counselor is the best thing that happens to a child in a correctional school. Officers or counselors should rarely let the children know that they (the staff) are wrong or have made a mistake. It is particularly difficult for me to work with the kind of boy or girl who is always testing institutional procedures. When most delinquent kids learn right from wrong they will be o.k. I have the patience to go over the same thing time and again without expressing anger. The most important thing a staff person can do is to set a good example. I would like to lead group counseling sessions where delinquents discuss intimate personal problems. I prefer to work in a cottage where something new is always going on. APPENDIX J INDIVIDUALS PRESENT AT THE TWO MEETINGS DURING WHICH CONTENT AREAS WERE DECIDED APPENDIX J INDIVIDUALS PRESENT AT THE TWO MEETINGS DURING WHICH CONTENT AREAS WERE DECIDED Mr. Earl Dunlap Director, Monroe County Youth Center Mr. Perry Freelon 1( Coordinator of Detention, Monroe County Youth Center Mr. Eric Howard Psychologist, Ingham County Probate Court Training Designer, Presenter, and Evaluator Mr. Doug Redding, M.S.W. Coordinator of Treatment, Monroe County Youth Center Dr. E. L. V. Shelley Chief Psychologist, Ingham County Probate Court Training Designer and Presenter Judge Harry Sietz Presiding Judge, Monroe County Probate Court Judge McCauley Sietz Judge, Monroe County Probate Court Mr. James Standifer Director of Children Services, Monroe County Probate Court 287 APPENDIX K EXPERTS IN THE FIELD CONSULTED TO FURTHER REFINE AND DEVELOP THE IN-SERVICE TRAINING PROGRAM CONTENT APPENDIX K EXPERTS IN THE FIELD CONSULTED TO FURTHER REFINE AND DEVELOP THE IN-SERVICE TRAINING PROGRAM CONTENT Mr. Gerald Juhr Graduate Assistant, IRSEN, Michigan State University Mr. James Longhurst Counselor, Starr Commonwealth for Boys Dr. Herbert C. Quay Director, Program in Applied Social Sciences, University of Miami Dr. Vincent Scalese Assistant Director, Maple Village Campus, Starr Commonwealth for Boys 288 APPENDIX L ANECDOTAL RECORD FORM APPENDIX L ANECDOTAL RECORD FORM Date Hour 1 Activities End of Hour Attendance Number of Interruptions Hour 2 Activities End of Hour Attendance Number of Interruptions Hour 3 Activities End of Hour Attendance Number of Interruptions APPENDIX M ANTICIPATION OF TRAINING SURVEY APPENDIX M ANTICIPATION OF TRAINING SURVEY Name Date Job Title Length of time at that job Length of experience working with children Education (last year completed) If college, what major INSTRUCTIONS The following pages contain some questions that deal with opinion or attitude and some that are factual in nature. Please answer all questions. When you do not fully agree with any of the choices provided, please choose the answer that most closely approximates your thoughts. 291 ATTITUDES TOWARD TRAINING Check One (V) 1. How familiar are you with the principles of: very somewhat not familiar familiar familiar a. reality therapy b. behavior modification c. adolescent psychology d. use of volunteers in corrections e. effective communication 2. Your exposure to the principles of reality therapy has primarily been through: (Check only the most relevant one) academic class in-service training personal reading professional workshops___ no exposure 3. Your exposure to the principles of behavior modifica— tion has primarily been through: (Check only the most relevant gag) academic class in—service training personal reading 4. 5. 6. professional workshops___ no exposure Your exposure to the principles of adolescent psy— chology has primarily been through: (Check only the most relevant gag) academic class in—service training personal reading professional workshops___ # no exposure Your exposure to the principles of the use of volun— teers in corrections has primarily been through: (Check only the most relevant 92g) academic class in—service training personal reading professional workshops___ no exposure Your exposure to the principles of effective communi— cation has primarily been through: (Check only the most relevant ggg) academic class in-service training personal reading professional workshops no exposure 293 In your work, how much have you applied the principles of the following? extensively occasionally never a. reality therapy b. behavior modification c. adolescent psychology d. use of volunteers in corrections e. effective communication Please rate your interest in learning about the following: very somewhat not interested interested interested a. reality therapy b. behavior modification C. adolescent psychology d. use of volunteers in corrections e. effective communication Which of the following would you most like to learn about? Please rank 1 to 5—-do not omit any or double number. a. reality therapy b. behavior modification d. e. adolescent psychology 294 use of volunteers in corrections ————__ effective communication 10. Do you feel this in-service training is necessary for you personally? yes no unsure 11. Your previous experience with in-service training has been: very helpful somewhat helpful of little help no previous in-service training 12.—l7. Check One (V) 12. A a d 13. A a d 14. A a d 15. A a d 16. A a d 17. A a d A = Agree very much a = Agree somewhat d = Disagree somewhat D = Disagree very much Even if you learn worthwhile theory in training, you can't really improve your actual skills away from the job. Once you have experience working with children, training is of little benefit. Training is much more important for those who have little experience working with children than for those who have a great deal. The ability to listen to people in an effective manner is something that cannot be taught. The ability to criticize children effectively can be improved through training. Most people who have an interest in children have no trouble communicating with them. APPENDIX N EXPERTS IN THE FIELD AIDING IN THE CREATION OF THE COGNITIVE AREAS OBJECTIVE TEST APPENDIX N EXPERTS IN THE FIELD AIDING IN THE CREATION OF THE COGNITIVE AREAS OBJECTIVE TEST Mr. David Brown Staff Psychologist, Ingham County Probate Court Dr. William Frey Coordinator of Research, IRSEN, Michigan State University Mr. Gerald Juhr Graduate Assistant, IRSEN, Michigan State University Dr. Vincent Scalese Assistant Director, Maple Village Campus, Starr Commonwealth for Boys Dr. E. L. V. Shelley Chief Psychologist, Ingham County Probate Court Dr. Judith Taylor Program Evaluator, Tri—County Community Mental Health Board 295 APPENDIX 0 EXPERTS IN THE FIELD AIDING IN THE CREATION OF THE CRITICAL INCIDENTS TEST APPENDIX O EXPERTS IN THE FIELD AIDING IN THE CREATION OF THE CRITICAL INCIDENTS TEST Mr. David Brown Staff Psychologist, Ingham County Probate Court Mr. Gerald Juhr . . Graduate Assistant, IRSEN, Michigan State UniverSity Dr. Vincent Scalese Assistant Director, Maple Village Campus, Starr Common— wealth for Boys Dr. E. L. V. Shelley Chief Psychologist, Ingham County Probate Court Dr. Judith Taylor Program Evaluator, Tri-County Community Mental Health Board 296 APPENDIX P INDIVIDUALS RATING STAFF MEMBERS' WORK SAMPLES ON THE STAFF OBSERVATION RATING SCALE APPENDIX P INDIVIDUALS RATING STAFF MEMBERS' WORK SAMPLES ON THE STAFF OBSERVATION RATING SCALE Mr. Keith Ostien B.A. Psychology, Houghton College, New York, 1969 M.A. Rehabilitation Counseling, Michigan State University, 1970 Expected completion Ph.D., June 1979, Rehabilitation Counseling, Michigan State University Experience: 1970—1972 United States Army—-Worked in drug abuse treatment and research 1972—1973 Outpatient clinician at Flint Community Mental Health 1973-1974 Clinical supervisor of inpatient unit of Flint Community Mental Health 1974—present Full—time position as certified psychological examiner at the Psychological Evaluation and Treatment Center 1975-present Staff Psychologist, Ingham County Probate Court 297 298 Dr. Ray Husband B.A. Mathematics, Central Michigan University, 1968 M.A. Guidance and Counseling, University of Michigan, 1973 Ph.D. Counselor Education, Michigan State University, 1978 Experience: 1968-1970 1970—1972 1972-1974 1975-present Counseling Field Experience, Youth for Christ International Mathematics teacher, Clawson High School, Clawson, Michigan Vocational and Financial Aids Counselor, Spring Arbor College, Spring Arbor, Michigan Residential Administration and Residential Counseling, Michigan State University LIST OF REFERENCES LIST OF REFERENCES Alston, P. P. The effects of facilitation training pro— vided a combined group of professional and sub- professional personnel in a juvenile correctional facility (Doctoral dissertation, University of Georgia, 1970). Dissertation Abstracts Inter- national, 1971, __, 3863—A. (University Micro- films No. 71-3710, 152) Anderson, S. B., Ball, 8., Murphy, R. T., and Associates. Encyclopedia of educational evaluation. San Francisco: Jossey-Bass, 1975. Ausubel, D. The use of advance organizers in the learning and retention of meaningful verbal materials. Journal of Educational Psychology, 1960, 51, 267—272. “— Bandura, A. Principles of behavior modification. New York: Holt, Rinehart & Winston, 1969. Barker, G. H., & Adams, W. T. Glue sniffers. Sociology and Social Research, 1963, 47(3), 298—310. Bersoff, D. N., & Moyer, D. Positive reinforcement obser— vation schedule (PROS): Development and use. In E. J. Mash & L. G. Terdal (Eds.), Behavior—therapy assessment diagnosis, design, and evaluation. New York: Springer, 1976. Bohnstedt, M. Answers to three questions about juvenile diversion. Journal of Research in Crime and Delinquency, 1978, lE(l), 109-123. Bullington, B., Sprowls, J., Katrin, D., & Phillips, M. A critique of diversionary juvenile justice. Crime & Delinquency, 1978, 33(1), 59—71. California Youth Authority. A review of the literature on the early identification of delinquent—prone chil- dren. Criminal Justice Abstracts, 1978, ;g(2), 173. 299 300 Campbell, D. T., & Stanley, J. C. Experimental and quasi- experimental designs for research. Chicago: Rand McNally, 1967. Cavan, R. S., & Ferdinand, T. N. Juvenile delinquency ‘ (3rd ed.). Philadelphia: Lippincott, 1975. j Coffey, A. R. Correctional administration: The manage— ment of institutions, probation andlparole. Englewood Cliffs, N.J.: Prentice-Hall, 1975. Commission on Accreditation for Corrections. Manual of standards for adult community residential services (Rockville, Md., 1977). Criminal Justice Abstracts, 1978, lg(l), 80-81. Cortes, J. B., & Gatti, F. M. Delinquency and crime: A biosocial apppoach. New York: Seminar Press, 1972. Danish, S. J., & D'Augelli, A. R. A training program for para—professionals: Rationale and implementation. Professional Psychology, 1976, 1, 38-46. Danish, S. J., D'Augelli, A. R., & Brock, G. W. An evalu- ation of helping skills training: Effects on helpers' verbal responses. Journal of Counseling Psychology, 1976, g;, 259—266. Danish, S. J., D'Augelli, A. R., Brock, G. W., Conter, K. R., & Meyer, R. J. A symposium on skill dis— semination for para-professionals: models of training, supervision, and utilization. Profes- sional Psychology, 1978, 2(1), 16—37. Davidson, W. S., & Seidman, E. Studies in behavior mod— ification and juvenile delinquency: A review, methodological critique and social perspective. Psychological Bulletin, 1974, El, 998-1011. Erickson, T. M. Schools for crime? Journal of Research in Crime and Delinquency, 1978, lE(l), 31—34. Ewing, B. G., & Gregg, J. M. H. The national manpower survey of the criminal justice system: Executive summary. Washington, D.C.: National Institute of Law Enforcement and Criminal Justice, Law Enforce- ment Assistance Administration, U.S. Department of Justice, 1978. Frey, W. D. Michigan State University in—service training project evaluation for the 1975-76 project year. Unpublished paper, 1976. 301 George, R. L. In-service training for counselors: Teach- ing old dogs new tricks. Counselor Education and Supervision, 1974, ll, 314-31 . Glaser, D. Remedies for the key deficiency in criminal justice evaluation research. Journal of Research in Crime and Delinquency, 1974, ll, 144-154. Glasser, W. Reality therapy. Crime and Delinquency, 1964, lg, 135-144. Glasser, W., & Zunin, L. M. Reality therapy. In R. Corsini (Ed.), Current psychotherapies. Itasca, Ill.: F. E. Peacock, 1973. Delinqpents and nondelinquents Glueck, S., & Glueck, E. in perspective. Cambridge: Harvard University Press, 1968. Involving paraprofessionals in the helping The case of federal probation. Cambridge: Gordon, M. T. process: Ballinger, 1976. Grant, J. D. Delinquency treatment in an institutional setting. In H. C. Quay (Ed.), Juvenile delin- quency--Research and theory. Princeton: D. Van— Nostrand, 1965. . & Stake, R. E. Evaluating movement/dance in The National Elementapy Hoke, G. A., a downstate district. Principal, 1976, §§(3), 52—59. Howard, E. Internal-external locus of control: parison of delinquent and non-delinquent male Unpublished masters thesis, Ball adolescents. State University, A com- 1973. The Ingham County Probate Court Shelter Home Committee. ___ final report and recommendations of the shelter home committee. Unpublished report, February 25, 1977. (Available from Ingham County Probate Court, 300 West Kalamazoo Street, Lansing, Michigan, 48933.) Ingram, G. L. Matching volunteers and juvenile offenders. In P. F. Zelhart & J. M. Plummer (Eds.), Institute on research with volunteers in juvenile delin- Fayetteville, Arkansas: University of uenc . 1970. Arkansas, 302 Jaques, M. E. Critical counseling behavior in rehabili— State University of tation settings. Iowa City: Iowa and Office of Vocational Rehabilitation, U.S. Department of Health, Education, and Welfare, 1959. Jesness, C. F. Comparative effectiveness of behavior mod— ification and T.A. programs for delinquents. Journal of Consulting and Clinical Psychology, 1975, 33, 758—779. Jesness, C. F. Small may be beautiful but is it cost effective? Journal of Research in Crime and Delinquency, 1978, lE(l), 25—30. & Scheier, I. H. Volunteer training for Metuchen, N.J.: The Jorgensen, J. D., courts and corrections. Scarecrow Press, 1973. Katrin, S. The effects on women inmates of facilitation training provided correctional officers. Unpub- lished doctoral dissertation, University of Georgia, 1972. Kozlowski, V. The adequacy of a course in vocational edu- cation for handicapped students for special edu— cation undergraduate majors. Unpublished doctoral dissertation, Michigan State University, 1976. J. D., & Krumboltz, H. B. Changing children's Prentice—Hall, Krumboltz, Englewood Cliffs, N.J.: behavior. 1972. Liberatore, R. Personal communication, July 13, 1978. (a) Liberatore, R. Personal communication, July 17, 1978. (b) Little, A. D. Cost and service impacts of deinstitution- alization of status offenders in ten states: Responses to angry youth. Criminal Justice Abstracts, 1978, lg(2), 217. Lundman, R. J., & Scarpitti, F. R. Delinquency preven- tion: Recommendations for future projects. Crime and Delinquency, 1978, 33(2), 207-220. McCarthy, P. M., Danish, S. J., & D'Augelli, A. R. A follow-up evaluation of helping skills training. Counselor Education and Supervision, 1977, ll, 29-35. 303 McCord, W., & McCord, J. Two approaches to the cure of delinquents. In S. Glueck (Ed.), The problem of delinquency. Boston: Houghton Mifflin, 1959. Meeker, B. Expansion of field work and internship facili- ties for training of correctional personnel. In C. S. Prigmore (Ed.), Manpower and training for corrections—-Proceedings of an Arden House Con— ference June 24-26, 1964. New York: Council on Social Work Education, 1966. Michigan Association of Children's Agencies. In-service training progpgm project summary. Unpublished pamphlet, 1977. (Available from M.A.C.A. In- Service Training Project, P.O. Box 6189, Grand Rapids, Michigan, 49506.) Michigan Judicial Institute. Handbook of Legal Terms. Unpublished pamphlet, 1977. (Available from Michigan Judicial Institute, P.O. Box 30048, Lansing, Michigan, 48909.) Morris, R. J. Behavior modification with child: A sys- tematic guide. Cambridge: Winthrop Publishers, 1976. Mukherjee, S. K. Administration of juvenile correctional institutions: A comparative study in Delhi and Maharashtra. New Delhi: Sterling Publishers Private Limited, 1974. National Conference of Superintendents of Training Schools and Reformatories. Institutional rehabilitation of delinquent_youth: A manual for training school personnel. Albany: Delmar Publishers, 1962. Newcomb, T. M. Characteristics of youths in a sample of correctional programs: Differences associated with programs and time spent in them. Journal of Research in Crime and Delinquengy, 1978, l§(l), 3—24. Of counseling interest. Guidepost, July 13, 1978, p. 8. Payne, D. A. (Ed.). Curriculum evaluation: Commentaries on purpose, process, product. Lexington, Mass.: D. C. Heath and Co., 1974. Petersen, D. M., & Thomas, C. W. Corrections: Problems and prospects. Englewood Cliffs, N.J.: Prentice- Hall, 1975. 304 Phelps, T. R. Juvenile delinquency: A contemporary view. Pacific Palisades, Calif.: Goodyear, 1976. President's Commission on Law Enforcement and Administra— tion of Justice. The challenga of crime in a free society. Washington, D.C.: Government Print— ing Office, 1967. Provus, M. Discrepancy evaluation. Berkeley, Calif.: McCutchan, 1971. Quay, H. C. Correctional preference survey. Unpublished survey, 1968. (Available from Dr. Herbert G. Quay, University of Miami, P.O. Box 248074, Coral Gables, Florida, 33124.) Quay, H. C. Personal communication, September 27, 1977. Quay, H. C., & Peterson, D. R. The questionnaire for measurement of personality dimensions associated with juvenile delinquency. Unpublished paper, 1964. Rabow, J. The Provo experiment in delinquency rehabili- tation. American Sociological Review, 1961, EE, 679-696. Reckless, W. C. The crime pgoblem (4th ed.). New York: Appleton—Century—Crofts, 1967. Redl, F., & Wineman, D. Children who hate-—The disorgani- zation and breakdown of behavior controls. Glencoe, Ill.: The Free Press, 1951. Romig, D. A. Justice for our children. Lexington, Mass.: Lexington Books, 1978. Ross, R. R., & McKay, H. B. A study of institutional treatment programs. International Journal of Offender Therapy and Comparative Criminolggy, 1976, 39(2)! 165-173. Rotter, J. B. Social learning and clinical psychology. Englewood Cliffs, N.J.: Prentice-Hall, 1954. Sandhu, H. S. Juvenile delinquenqy: Causes, control and prevention. New York: McGraw-Hill, 1977. Sarason, I. G., & Ganzer, V. J. Modeling: An approach to the rehabilitation of juvenile offenders. Wash- ington, D.C.: Department of Health, Education, and Welfare, 1971. 305 Scarpitti, F. R., Murray, E., Dinitz, S., & Reckless, W. The good boy in a high delinquency area. In M. E. Wolfgang, L. Sowitz, & N. Johnston (Eds.), The sociology of crime and delinquency. New York: John Wiley and Sons, 1962. Schwartz, I. M., Jensen, D. R., & Mahoney, M. J. Volun- teers in juvenile justice. Criminal Justice Abstracts, 1978, lu(2), 235. Shelley, E. L. V. (Ed.). Volunteers in the correctional spectrum: An overview of evaluation, research, and surveys. Boulder, Colorado: National Information Center on Volunteers in Courts, 1971. Shelley, E. L. V. Personal communication, June 15, 1977. Slaiken, K. A. Evaluation studies on group treatment of juvenile and adult offenders in correctional institutions: A review of the literature. Journal of Research in Crime and Delinquency, 1973, lg, 87—100. Stake, R. E. The countenance of educational evaluation. In C. H. Weiss (Ed.), Evaluating action programs: Readings in social action and education. Boston: Allyn & Bacon, 1972. Stake, R. E. Program evaluation: Particularly respon— sive evaluation. Paper #5 in Occasional Paper Series. Kalamazoo: Evaluation Center, College of Education, Western Michigan University, 1975. Stake, R. E. Making school evaluations relevant. The North Central Association Quarterly, 1976, 29(4)! 347-352. Stake, R. E. Personal communication, June, 1978. Stake, R. E., & Hall, M. E. Evaluation report 1975 of prgject city science. Urbana: Center for Instructional Research and Curriculum Evaluation, University of Illinois, 1975. Stake, R. E., Easley, J. Case studies in science education: Final report to the National Science Foundation. Urbana: Center for Instructional Research and Curriculum Evaluation, University of Illinois, 1978. 306 Stauber, D. H. Internal-external locus of control: A comparison of delinquent and non-delinquent female adolescents. Unpublished masters thesis, Ball State University, 1973. Stufflebeam, D. L., Foley, W., Gephart, W., Guba, B., Hammond, R., Merriman, H., & Provus, M. Egu— cational evaluation and decision making. Itasca, Ill.: Peacock, 1971. Taylor, D. B. Eeny, meeny, miney, meaux: Alternative evaluation models. The North Central Association Quarterly, 1976, 29(4)! 353—358. Tinning, F. C. An experimental study investigating the effects of real and simulated clinical training on psychomotor, affective and cognitive variables during real clinical performance of first year osteopathic medical students. Unpublished doc- toral dissertation, Michigan State University, 1973. Trieschman, A. B., Whittaker, J. K., & Brendtro, L. K. The other 23 hours: Child care work with emo— tionally disturbed children in a therapeutic milieu. Chicago: Aldine, 1969. Trojanowicz, R. C. Juvenile delinquency——Concepts and control. Englewood Cliffs, N.J.: Prentice-Hall, 1973. U.S. Civil Service Commission Bureau of Training. Study shows obstacles to effective training. Federal Trainer, No. 16, June 1973. U.S. National Criminal Justice Information and Statistics Service. Children in custody: Advance report on the juvenile detention and correctional facility census of 1975. Criminal Justice Abstracts, 1978, lu(2), 215—216. Walker, S. The challenge of crime in a free society after ten years. Crime and Delinquency, 1978, l£(1), 1—12. Watzlawick, P., & Weakland, J. (Eds.). The interactional view. New York: W. W. Norton, 1977. Weeks, H. A. Youthful offenders at Highfields——An evalu— ation of the effects of the short-term treatment of delinquent boys. Ann Arbor: University of Michigan Press, 1963. 307 Wheeler, S., Cottrell, L. S., & Romasco, A. Juvenile delinquency-—Its prevention and control. Presi— dent's Crime Commission's Task Force Report on Juvenile Delinquency and Youth Crime, 1967, 409- 428. Wittman, G. P. Training: Key to institutional improve- ment. In W. E. Amos & R. L. Manella (Eds.), Readings in the administration of institutions for delinquent youth. Springfield, Ill.: Charles C. Thomas, 1965. Wright, W. E., & Dixon, M. C. Community prevention and treatment of juvenile delinquency: A review of evaluation studies. Journal of Research in Crime and Delinquency, 1977, ll, 35—67.