EVALUATION OF AN INSTRUCTIONAL MODEL FOR TEACHING COUNSELOR TRAINEES HOW TO ESTABLISH BEHAVIORAL OBIEOTIVES IN COUNSELING Dissertation for the Degree of Ph. D. MICHIGAN STATE UNIVERSITY MARK ALBERT HECTOR 1973 LIBRAR! Michigan St - 3 University This i_s tmoertgfyfihatefae thesis entitled _ . EVALUATION OF AN INSTRUCTIONAL MODEL , FOR TEACHING COUNSELOR TRAINEES _ HOW TO ESTABLISH BEHAVIORAL OBJECTIVES IN COUNSELING i. I presengiy t 4 Mark AficiEIf has been accepted towar§gfulfillment of the requiremefi'tg for Ph.D . degree in Education Major professor ; I ABSTRACT EVALUATION OF AN INSTRUCTIONAL MODEL FOR TEACHING COUNSELOR TRAINEES HOW TO ESTABLISH BEHAVIORAL OBJECTIVES IN COUNSELING By Mark Albert Hector The study was designed to evaluate a training model for teaching counselor trainees how to establish behavioral objectives in the counseling process. The emphasis of the model was to encourage the trainees to make references to the three components of a behavioral objective--terminal performance, conditions, and criteria. An advance organizer in the form of a scale to record the references of a counselor to the components of a behavioral objective was the unique feature of the model. The main treatment group received training with the scale and two control groups did not. When the main treatment group engaged in a role-play counseling interview that was intended to establish a coun- seling objective, however, they did not make significantly more . references to the components of a behavioral objective than did two other control groups. Reasons for the lack of significant results and other unexpected findings are discussed. EVALUATION OF AN INSTRUCTIONAL MODEL FOR TEACHING COUNSELOR TRAINEES HOW TO ESTABLISH BEHAVIORAL OBJECTIVES IN COUNSELING By Mark Albert Hector A DISSERTATION Submitted to Michigan State University in partial fulfillment of the requirements for the degree of DOCTOR OF PHILOSOPHY Department of Counseling, Personnel Services, and Educational Psychology 1973 . 4r 0 1",} '2‘ 53- .93 I ACKNOWLEDGEMENTS ‘p -‘. 0. :23 My committee of Bob B. Winborn, Herbert M. Burks, Jr., Richard G. Johnson, and Dozier W. Thornton, deserve special thanks for their help. Judy, my wife, also supplied much support. 11 TABLE OF CONTENTS LIST OF TABLES . LIST OF FIGURES CHAPTER I. PROBLEM, RATIONALE, AND RELATED RESEARCH Need Purpose . Theory Review of the Literature Hypotheses . . . . . . . . . . . . . . . . Overview II. EXPERIMENTAL DESIGN AND METHODOLOGY . Subjects Treatments Procedures Instruments . Design and Statistical Analysis . III. ANALYSIS OF RESULTS . Recorder Correlations and Dependent Variables . Results of Major Hypotheses . Results of Secondary Hypotheses . iii ii viii PAGE l3 19 20 22 22 23 33 40 42 45 45 SO 53 CHAPTER IV. SUMMARY AND DISCUSSION Summary . Discussion Conclusion APPENDICES . A. Comments Made to Subjects by Main Experimenter One Week Before the Experiment Student Consent Form . Directions Given to Each Subject Before the Role-Playing Session Descriptions of the Two Roles That Were Presented to the Subjects . Behavioral Objectives for Counseling . Counseling Objectives Two-Part Paper on "Identify Concern" and "Select Concern for Counseling" Counseling Behavioral Objective Process Scale and Directions for Its Use . CBOP Scale Form Used by Two Trained Recorders Tape Evaluation Session Directions for Subjects for Tuesday (No-Delay Group) Tape Evaluation Session Directions for Subjects for Tuesday (Delay Group) iv PAGE 76 76 79 93 98 98 100 101 103 105 108 109 112 114 115 116 CHAPTER PAGE L. Tape Evaluation Session Directions for Subjects for Thursday (No-Delay Group) . . . . . . . . . . . . . . . . . I 117 M. Tape Evaluation Session Directions for Subjects for Thursday (Delay Group) . . . . . . . . . . . . . . . . . . . 118 N. General Guidelines for Role-Players . . . . . . . . . . . . 119 0. CBOP Scale Recording Form Used by Group 1 During the Training Phase of the Experiment . . . . . . . . . . . . . . 120 LIST OF REFERENCES . . . . . . . . . . . . . . . . . . . . . . . . 121 TABLE LIST OF TABLES Product-Moment Coefficients of Correlation Between the Two Trained Recorders for the Role-Play Evaluation Summary of Observed Cell Means of Subjects' References Corrected for'Length'of RoleéPlay Interview Univariate Test of Hypotheses I, II, and III: Number of Total References Subjects Made to the Components of a Behavioral Objective Multivariate Test of Hypotheses I, II, and III: Terminal Performance, Conditions, and Criteria Scores . . . F Values for Group Variance Comparisons: Total Scores F Values for Group Variance Comparisons: Terminal Performance (TP), Conditions (CO), and Criteria (CR) Scores Variances for the Three Main Treatment Groups: Terminal Performance (TP), Conditions (CO), Criteria (CR), and Total (TOT) Scores Univariate Test for Problem Presented by Role—Players and Treatment Groups: Number of References to the Components of a Behavioral Objective . vi PAGE 47 49 52 54 57 58 6O 63 TABLE PAGE 3.9.. Multivariate Test for Problem Presented by Role-Players and Treatment Groups: Terminal Performance, Conditions, and Criteria Scores . . . . . . . . . . . . . . . . . . . 64 3.10. Univariate Test for Problem Presented by Role-Players: Terminal Performance, Conditions, and Criteria Scores . . . 65 3.11. Means and Standard Deviations of Conditions and Criteria Scores: Problem Presented by Role-Players . . . . . . . 66 3.12. Univariate Test for Treatment Group and Feedback: Total Number of Correct Selections During Role-Play Evaluation . . . . . . . . . . . . . . . . . . . . . . . 69 3.13. Multivariate Test for Treatment Groups and Feedback: Number of Correct Selections During Role-Play Evaluation . . . . . . . . . . . . . . . . . . . . . . . 70 3.14. Univariate and Multivariate Tests for Treatment Group and Feedback (Segment A): Total Scores and Terminal Performance, Conditions, and Criteria Scores . . . . . . 73, 3.15. Univariate and Multivariate Tests for Treatment Groups and Feedback (Segment B): Total Scores and Terminal Performance, Conditions, and Criteria Scores . . . . . . 74 3.16. Means of Total Scores on Segment B Comparisons . . . . . . 75 vii LIST OF FIGURES FIGURE PAGE 2.1. Outline of the Experiment . . . . . . . . . . . . . . . . . 34 3.1. Mean Number of References to Components of a Behavioral Objective for Each Group . . . . . . . . . . . . . . . . 48 3.2. Corrected Total Scores of Subjects by Groups . . . . . . . 55 viii CHAPTER I PROBLEM, RATIONALE, AND RELATED RESEARCH Need Counselor educators are increasingly concerned about finding the most efficient ways of teaching behavioral counseling skills (Hendricks, Ferguson, & Thoresen, 1973; Horan, 1972; Jakubowski-Spector, Dustin, & George, 1971; Miller, 1972; Stewart & Hinds, 1970; Thoresen, 1972; Winborn, Hinds, & Stewart, 1971). As behavioral counseling skills come to be taught in more counselor education programs, there is a need to determine the best way to teach these skills. In the past, counselors have typically been taught to establish relationships, gain rapport, be warm and accepting, and so on. Teaching models were established to teach these skills. With the recent emphasis on teaching counselors such skills as establishing behavioral objectives, 'identifying contingencies of reinforcement, and determining behavioral (change strategies, counselor educators must develop new teaching models. The more specific nature of counseling skills to be taught requires new specific teaching methods. I One of the more important skills to be learned by counselors who are interested in behavioral counseling is the formulation and use of_ behavioral objectives. There is extensive literature regarding the use 1 of behavioral objectives in instruction (Duchastel & Merrill, 1973). However, little has been written about the use of behavioral objectives in counseling. There are no studies which investigate the best way to teach counselor trainees how to employ behavioral objectives. Purpose The primary purpose of the study was to evaluate an instructional model for teaching counselor trainees how to establish behavioral objectives when counseling with clients. 'Advance organizers were a basic feature of the model. The instructional model consisted of four basic components which the trainees experienced in succession. First, the trainees read a general introduction which specified that the learning experience would deal with setting objegtives for counseling. Second, the trainees read specially prepared materials on how counselors establish behavioral objectives for counseling with their clients. These materials were taken in part from the materials used in the Michigan State University M.A. program in counseling (Winborn, Hinds, & Stewart, 1971). Third, the trainees read a unit on how to identify and tally the parts of a behavioral objective as they listened to a counseling interview. This third section was essentially aimed at teaching the trainees how to use a scale (advance organizer) for recording the number of times a counselor mentioned the components of behavioral objectives in an interview. Fourth and finally, the trainees heard a series of audiotape interview segments in which they used the scale to tally the number of times they heard the counselor mention any of the components of a behavioral objective. As stated before, the purpose of this study was to evaluate which aspects of the model are most important in terms of teaching counselor trainees how to establish behavioral objectives in the counseling pro- CESS. Theory This experiment was directly concerned with five theoretical areas or issues. 1. The content of behavioral objectives. 2. The controversy over the use of behavioral objectives. 3. The use of behavioral objectives in counselor education programs. 4. The use of behavioral objectives in the counseling process. 5. The use of advance organizers. Each issue is discussed in this section on theory. It will be noted that, with the exception of the third and fourth areas, most of the theory reviewed here does not deal specifically with counseling or counselor education. This situation arises because only recently has much been written about the uses of behavioral objectives in counselor education and in the counseling process. The theory that is reviewed here deals mostly with the learning process. Theoreticians, however, usually see their work as having application to a wide range of learning situations. The Content ongehggioral Objectives Two areas of instruction have stressed the importance of objectives (Gagné: 1964). The first is technical training in the military services. The close relationship between the training programs and the skills that leaders wanted to develop in trainees forced the program planners to look very closely at the outcomes of their training programs. With outcomes being of prime importance, the planners were led to state specifically the objectives for the training program. More recently the extensive work on programmed instruction has given an impetus to the development of behavioral objectives. It is generally agreed that clearly stating the instructional objective is the first requirement when preparing programmed instruction materials. Extensive work has been done regarding the theoretical foundations of behavioral objectives. Bloom (1956) has developed a taxonomy of educational objectives for the cognitive domain. This taxonomy has six categories which together form a hierarchy with the higher classes of objectives built upon the lower classes of objectives. Krathwohl, Bloom, and Masia (1964) have carried out similar work for educational objectives in the affective domain, and presently a taxonomy for educa- tional objectives in the psycho-motor domain is being developed (Gronlund, 1970). The work of Bloom, Krathwohl, and Masia has been on a fairly abstract level. It has, therefore, not been used extensively in practical situations. Mager's book (1962) entitled Preparingglnstructional Objectives, however, has been a widely used tool for assisting practioners in developing objectives. This book is very practically written and is directed mainly at the classroom situation. Others have outlined specific requirements and instructions for writing behavioral objectives (Banathy, 1968; DeCecco, 1968; Gagné: 1965), but Mager's work seems to be the most popular and widely read. According to Mager, an instructional objective contains three basic parts. First, it is a statement of the terminalgperformance. The terminal performance statement describes the desired observable behavior of the student after instruction. Second, it is a statement of the conditions under which the terminal performance is to occur. The con— dition statement describes the environment in which the student is expected to perform. Third, it is a statement of the criteria for evalu- ating the terminal performance. The criteria statement describes the level of acceptable performance on the part of the student. Taken to- gether, the three parts--terminal performance, conditions, and criteria-- form a precise instructional objective which can be objectively measured. Much of the material in Preparing Instructional Objectives is devoted to the specifics of writing instructional objectives. Gronlund (1970) has also specified several rules to follow when writing instruc- tional objectives. In both cases, the main stress is on dealing with and describing observable behaviors. ln summary, the stress upon the use of behavioral objectives in instruction has occurred in the military and in programmed instruction. Theoretical taxonomies of educational objectives have been written. More practically, specific instructions on how to write behavioral objectives have been given. The Controversy Over Use of Behavioral Objectives Not everyone, however, supports the use of behavioral objectives. There is presently a controversy over their use and application. Some have questioned the value of behavioral objectives (Atkin, 1969; Ebel, 1970). Eisner (1967) has stated that the issue of whether or not to use behavioral objectives in curriculum construction and teaching is basic- ally an empirical question depending on the situation. Ebel (1963) has pointed out that the use of behavioral objectives in the instructional process is a limiting factor that hinders creativity. When objectives are set, there is usually a strong tendency to instruct only to those topics. Ebel fells that if all topics to be taught in school were specified in behavioral objectives, the objectives would become too complex and unworkable. The majority of writing relative to behavioral objectives supports their use. Flanagan (1967) states that a lack of well-defined instruc— tional objectives has prevented the use of modern decision-making pro- cedures. Mager (1962), Gronlund (1970), and Gagné’(1964) have advocated the use of instructional objectives to make the teaching process more systematic and effective. Bandura (1969) has encouraged the setting of behavioral objectives in any behavior change program that is syste- matically implemented. Almost all types of professional counselors seem to be concerned with the goals or objectives of counseling (Lorr, 1965; Michaux & Lorr, 1961). This concern transcends the theoretical framework of the counselor. Practitioners of many different theories have dealt with the subject (Rogers, 1951; Sullivan, 1954; Williamson, 1950; Wolberg, 1954). In some cases, the theorists have dealt mainly with the objectives for the counselor, and in othenm they have stressed the objectives for the client. Overall, however, most treatment approaches spend very little time with the problem of selecting objectives (Bandura, 1969). Most of the effort has been given to describing the process of counseling. Thus the controversy involves those who question and those who support behavioral objectives. The questioning has to do with the value of objectives, when they should be used, whether they limit creativity, whether they narrow the scope of instruction, and whether it is workable to draw up behavioral objectives for a total school curriculum. The Use of Behavioral Objectives in Counselor Education Programs With the application of learning theory principles to the counsel- ing process, a greater interest in the objectives of counseling has been encouraged. In the area of counselor training, Thoresen (1969) proposed a systematic model of training that incorporates the use of performance objectives. “The performance objectives are written by the counselor educators, and they follow the guidelines set down by Mager (1962). The whole training program as proposed by Thoresen is based on performance objectives. In order to graduate from the program, counselor ~ trainees must demonstrate the required skills of good counseling as stated in the performance objectives. The counselor education program at Michigan State University which uses behavioral objectives as a guide for trainee performance was described by Winborn, Hinds and Stewart (1971). They cite examples of what typical behavioral objectives might be for counselor trainees. One objective would be, "After five weeks of training, the student and a client will implement at least one counseling strategy that uses the principle of positive reinforcement." Hendricks, Ferguson, and Thoresen (1973) have described the counselor education program at Stanford University. This program incor- porates behavioral objectives from a competency-based approach. The students in this program must show that they have attained specific performance levels in skills required by the program. Horan (1972) beiefly reviewed the Michigan State University and Stanford programs and discussed how they both make extensive use of behavioral objectives as a guide for their students. He also describes how to construct an instructional goal for counselor education. His com- ments are based on the format for writing instructional objectives as specified by Mager (1972). Thus, an examination of the use of behavioral objectives in counselor education programs reveals a proposed model which includes objectives for trainees. Additionally, in the literature, two counselor education programs now in operation which incorporate stated objectives for trainees are described. The Use of Behavioral Objectives in the Counseling Process The first clear statement encouraging the use of behavioral objectives in the counseling process was made by Krumboltz (1966a). In another article the same year, he advocated that " . . . the goals of 9 counseling must be stated in terms of specific behavior changes defined by each individual client and agreed to by his counselor" (Krumboltz, 1966b, p. 10). This statement implies that the goals of counseling should be stated differentially for each individual client. To arrive at differentially stated goals for each client, the counselor and the client will spend time negotiating and specifying a goal statement in terms of the client's behavior. Bandura (1969) has dealt with the use of behavioral objectives in counseling. He points out that social change and psychological change projects have traditionally never articulated their goals very clearly. This failure has been one of the main reasons for their limited successes in the past. Bandura (1969) also discusses the factors that hinder the setting of behavioral objectives in counseling. One factor he points out is that counselors have been taught and encouraged to use social reinforce- ment on a noncontingent basis. For example, a counselor who attempts to use unconditional positive regard with his client is using noncon- tingent social reinforcement. Bandura encourages the use of selective reinforcement which is designed to help the client reach some agreed- upon objective. He thinks that a systematic plan of reinforcement is more efficient in changing behavior than a random procedure. Hinds (1970) has described why behavioral objectives should be implemented in the counseling process. In describing the reasons, he covers three main benefits that can be obtained through the use of behavioral objectives in counseling. These benefits are also discussed by Krumboltz (1966a) and Bandura (1969). There seems to be much 10 agreement among different writers on these benefits. The first benefit comes from merely stating the counseling goal in a clear and objective manner. When the counselor and client perform this task, the whole counseling process is forced to be more specific. There is increased communication when both the client and the counselor know what the client is expected to be able to do at the end of counseling. The second benefit of stating behavioral objectives for counseling is the guidance they provide in the selection of change strategies. Bandura (1969) notes that if the objectives are poorly defined, the counselor and the client have no rational basis on which to select appropriate treatment procedures. A behavioral objective that incor- porates a description of the terminal performance, the conditions under which the terminal performance will take place, and the criteria by which it will be evaluated typically gives a strong indication as to what strategy might be employed to meet the objective. Related to the issue of strategy selection, Krumboltz (1966a) states that a consequence of using behavioral statements for counseling goals is that the counselor, clients, and interested citizens are able more clearly to anticipate what counseling can accomplish and what it can not. Specific strategies can be used to reach specified objectives. Amorphous and more wonderful sounding goals are not involved. The third benefit is related to evaluation. If the behavioral objective is adequately stated (Mager, 1962), then there is no question as to whether it is obtained by the client. Bandura (1969) states how the evaluative function is related to strategy selection. ll Behaviorally defined objectives not only provide guidance in selecting appropriate procedures, but they serve an important evaluative function as well. When desired outcomes are desig- nated in observable and measurable terms, it becomes readily apparent when the methods have succeeded, when they have failed, and when they need further development to increase their potency. This self-corrective feature is a safeguard against perpetuation of ineffective approaches, which are difficult to retire if the changes they are supposed to produce remain ambiguous (p. 74). In summary, behavioral objectives in counseling are seen as individual for the client and to be reached with selectiVe reinforcement, not with noncontingent reinforcement. The benefits of behavioral objectives in counseling are increased client-counselor communication, guidance in change-strategy selection, a clearer anticipation of the results of counseling, and greater ease in evaluating the outcomes of counseling. The Use of Advance Organizers Ausubel (1963) formulated the concept of the advance organizer and carried out a series of research studies to test the concept. Some of this research is reviewed in the next section. The theoretical concept of advance organizers is covered in this section. Ausubel (1963) proposed the existence of a cognitive structure that is "an individual's organization, stability, and clarity of knowledge in a particular subject-matter field" (p. 26). It is this cognitive structure which plays a major role in the organization and retention of new material. The cognitive structure or advance organizer provides a frame of reference for new experiences. With this frame of reference, the individual is rapidly able to organize mentally what otherwise would be a confusing mass of material. 12 Ausubel suggests that a general organizational approach to the presentation of any new material can be progressive differentiation. Goldstein, Heller, and Sechrest (1966) describe progressive different- iation and advance organizers as follows: One of the ways of providing for progressive differentiation is by the use of advance organizers. An advance organizer provides a cognitive framework or structure into which new material can be interwoven. The nature and inclusiveness of the advance organizer will depend on the subject's prior level of organization of concepts in the field in which he is learning. . . . The organizer is introduced before the learning material and is at a higher level of abstraction and inclusiveness than the material to be learned (p. 243). An example of the advance organizer concept might be found in the case where the author of a textbook has made extensive use of chapter and section headings. By looking through chapter and section headings, a student can gain an overall understanding of the organization of the book. When actually reading the book, the student is able to fit the new material into an overall outline which would be called an advance organizer. A concluding comment should be made in order to clarify the two concepts of behavioral objectives and advance organizers and their use in this experiment. Duchastel and Merrill (1973) have pointed out that in a sense behavioral objectives are advance organizers. When objectives are given to a student before a learning experience, it is logical to assume that they help to organize the experience of the learner. Summary There have been several attempts to determine what should go into a behavioral objective. Ideas relative to the content of behavioral 13 objectives are still changing, just as the debate over their usefulness continues. Increasingly, however, behavioral objectives are being used in counselor education programs as guides for trainees. In addition, trainees are learning how to use them in the counseling process. Regard- less of the manner in which objectives are used, they can be thought of as advance organizers, a concept formulated by David Ausubel. Review of the Literature The research reviewed here deals with the use of behavioral objectives in the learning process and with advance organizers. There is no research at present on the evaluation of different counselor educa- tion models which are aimed at teaching behavioral counseling skills. The literature is extensive and rapidly expanding regarding the use of objectives in education generally, and counseling more specifically. Duchastel and Merrill (1973) have done a comprehensive review of the empirical research relative to behavioral objectives and learning. In their review, they noted that behavioral objectives seem to form three main instructional functions. They can be interpreted as providing direction for teaching and curriculum development, or providing guidance in evaluation, or providing facilitation of learning. Duchastel and Merrill focused their review on the third function which has to do with the use of behavioral objectives as facilitators of learning. Within this third area, they categorized the research into four groups dealing with (a) general research involving behavioral objectives as a facilitator of learning, (b) the use of behavioral objectives with different types of learning, (c) the use of behavioral objectives with different learner l4 characteristics, and (d) the use of behavioral objectives relative to the time it takes to learn to a given criterion. The first area of research investigated "the hypothesis that students provided with behavioral objectives would achieve more than students not provided with objectives." Duchastel and Merrill (1973) summarize this research as follows: (This) group of studies is difficult to summarize because of the lack of consistent results across investigations. On immediate retention, measured by a posttest, five studies reported a significant effect due to the availability of behavioral objectives, while five studies reported no such effect. On measures of delayed retention, two investigations found objectives to enhance performance and one did not find this facilitative effect. ln summary, the availability of objectives was found to facilitate learning in certain instances, although the generalizability of these instances is not easily determined (p. 57). The second area of studies sought to investigate the possibilities of interactions between the type of learning and the availability of behavioral objectives. Five out of the seven studies reviewed in this area revealed no significant interactions between the use of objectives and the type of learning. One of the two studies that did not support the major conclusion was carried out by Yelon and Schmidt (1971). They found that objectives had either a neutral or interfering effect relative to a problem-solving task. The third area of studies investigated interactions between the use of behavioral objectives and learner characteristics. Interactions were found regarding each of the learner characteristics of reasoning ability and certain personality characteristics. No interactions were reported regarding each standardized test of ability results and the state of learner anxiety over a short period of time. With respect to 15 aptitude, conflicting results have been reported. The final area of research reviewed by Duchastel and Merrill (1973) deals with the variable of the time needed to learn to a specific criterion, and the use of behavioral objectives. The overall findings here seem to indicate that the use of behavioral objectives does not decrease the amount of time needed, and in one case, the use of objectives increased the amount of time needed. Ausubel (1960) tested the hypothesis that the learning of unfamiliar but meaningful verbal material can be facilitated by the introduction of advance organizers. He tested two groups of 40 subjects each on their ability to retain some material they had read three days earlier. The material they read was a 2,500-word passage on the metallurgical properties of plain carbon steel which was an unfamiliar topic to all the subjects. The two groups were equated on the basis of sex, field of specialization, and ability to learn unfamiliar scientific material. The experimental treatment of interest had to do with the type of material that the subjects studied before they read the main 2,500-word passage. The experimental group read on two different occasions "a SOO-word introductory passage containing substantive back- ground material of a conceptual nature presented at a much higher level of generality, abstraction, and inclusiveness than the steel material itself" (p. 271). Ausubel was careful to note that the advance organizer passage contained no specific material that would help the experimental group with the multiple-choice test that all the subjects took three days after reading the main passage. The control group read a historical introduction to the process of steel making. The experimental group 16 performed significantly better on a 36-item multiple-choice test cover- ing the material in the main passage. The better results of the experimental group which used the advanced organizer was credited to two factors: (a) the selective mobilization of the most relevant existing concepts in the learner's cognitive structure for integrative use as part of the subsuming focus for the new learning task, thereby increasing the task's familiarity and meaningfulness; and (b) the provision of optimal anchorage for the learning material in the form of relevant and appropriate subsuming concepts at a proximate level of inclusiveness (Ausubel, 1960, p. 271). Investigating the concept of advance organizers further, Ausubel and Fitzgerald (1961) attempted to find out whether they "could facilitate the learning and retention of unfamiliar meaningful material by increas- ing the discriminability between the new material and related concepts already established in cognitive structure" (p. 268). The new material had to do with Buddhism and related concepts of Christianity which were considered to be already established in the cognitive structures of the subjects. In this study, 155 subjects were divided into three treatment groups which each studied a different type of advance organizer. The first group studied a comparative organizer which pointed out the similarities and differences between Buddhism and Christianity. The second group studied an expository organizer which outlined the principles of Buddhism at a high level of abstraction and made no reference to Christianity. The third group studied a historical introduction which merely gave human interest material about the life of Buddha and Buddhism. The third group was intended as a control group. 17 After the three groups had studied the organizers or introduction, all the subjects read a "2,500-word passage dealing with Buddhist concepts of God, immortality, soul, faith, salvation, morality, and responsibility. These concepts were elaborated in considerable detail" (Ausubel & Fitzgerald, 1961, p. 267). In order to test the learning and retention of the passage, two 45-item multiple—choice tests covering the material were given three and ten days later. After three days, the group that had the comparative organizer did significantly better on the multiple-choice test than either the exposi- tory organizer or historical introduction groups. After ten days, both the comparative and expository groups did better than the historical group. These results seem to support the use of advance organizers, especially those that compare material soon to be learned with already existing knowledge. Clearly, a major factor here is the state of the already existing knowledge. Ausubel pre-tested all the subjects on their knowledge of Christianity and found that those who were more knowledge- able about Christianity did significantly better on the Buddhism test regardless of what group they were in. Thus in the learning and retention of unfamiliar ideational material that is relatable to established concepts in the learner's cognitive structure, both comparative and expository organizers appear to be effective only in those instances where existing discriminability between the two sets of ideas is inadequate as a consequence of the instability or ambiguity of established concepts (Ausubel & Fitzgerald, 1961, p. 247). Ausubel and Fitzgerald (1962) investigated the effects of an advance organizer, antecedent learning, and general background knowledge on the learning and retention of two unfamiliar passages on the subject 18 of endocrinology. The model they used for the experiment was similar to that of earlier studies (Ausubel, 1960; Ausubel & Fitzgerald, 1961). Their major finding of relevance to this experiment was that general background knowledge facilitated the effect of the advance organizer. This result was suggested to be due to a positive interaction between the treatment groups (advance organizer and control) and the level of previous general knowledge about endocrinology. It is important to show that the use of advance organizers is not bound by the content of the subject matter being taught. Ausubel has concentrated on verbal learning of unfamiliar but meaningful materials having to do with iron ore production, religion, and endocrinology (Ausubel, 1960; Ausubel & Fitzgerald, 1961, 1962). Scandura and Wells (1967) have used essentially the same experimental model as used earlier by Ausubel and have applied the advance organizer concept to the teaching of the mathematical concepts of groups and topology. They found, as did Ausubel, that advance organizers facilitate the learning of new material. Summary The research reviewed on behavioral objectives in the learning process lacks'consistent results, and often significant differences are not obtained with respect to the facilitation of learning. It can be said that the use of behavioral objectives either helps the learning process or does not affect it. With the exception of one study (Yelon & Schmidt, 1971), they have been consistently shown not to hinder the process. 19 The research on advance organizers shows that the learning of unfamiliar meaningful verbal material and mathematical concepts is facilitated by advance organizers. Similarities between advance organizers and behavioral objectives have also been pointed out. Hypotheses The three main hypotheses that were tested in this experiment make reference to materials which are described later in this thesis. The Counseling Behavioral Objective Process Scale is described in Appendix H. The three basic components of a good objective are described by Mager (1962) as terminal performance, conditions, and criteria. The written description of how counselors can use behavioral objectives (Stewart, Burks, Engelkes, Johnson, & Winborn, 1972) is in Appendix E. In the following hypotheses, reference is made to counselor trainees. These trainees came from the Michigan State University M.A. counseling program. A later section in this dissertation entitled "Subjects" describes the population and the sample for this study. There are three main treatment groups in this experiment, and they com-. prise the major independent variable. The treatments on the three groups are essentially three different ways of teaching prospective counselors how to establish behavioral objectives with their clients. The tallies relative to the Counseling Behavioral Objective Process Scale are the major dependent variable. -flypothesis l: Counselor trainees who have had training with the Counseling Behavioral Objective Process Scale (advance organizer) will 20 make significantly more responses regarding the three components of a good behavioral objective with a role-playing client as measured by the scale than counselor trainees who have not had training with the scale. Hypothesis ll: Counselor trainees who have studied a written description of good behavioral objectives for counseling and have had training in recognizing good counseling performances relative to the written description will make significantly more responses regarding the three components of a good behavioral objective with a role-playing client as measured by the scale than counselor trainees who have not studied the description. Hypothesis lII: Counselor trainees who have received feedback during training exercises to recognize good counseling performances relative to setting up behavioral objectives for counseling will make significantly more responses regarding the three components of a good behavioral objective with a role-playing client as measured by the scale than counselor trainees who have not received feedback during training exercises. Overview The remaining three chapters of this study will be organized in the following way. In Chapter II, the experimental subjects, treatments, procedures, and instruments will be described. The testable hypotheses and the design are also stated. Finally, the type of statistical 21 analysis is described. Chapter 111 is devoted to an analysis of the results. Chapter IV concludes the study and includes a discussion of the results. CHAPTER II EXPERIMENTAL DESIGN AND METHODOLOGY Subjects The subjects for this study came from two sections of an M.A. degree-level course in theory and application relative to the counseling process offered at Michigan State University during the winter term of 1973. The main experimenter was team teaching these sections with another instructor. All of the students in these two sections began their counseling training in the fall term of 1972. Participation in the study was voluntary on the part of the students, and those who agreed to participate signed a consent form (Appendix B). It was explained to the students both verbally and on the consent forms that participation or non-participation in the study in no way affected their grade for the course. It was also explained that everyone who did not participate in the study would be exposed to the major ideas which were presented in the study. Of the 61 students in the two sections, 60 agreed to participate. One student rersed to take part because of personal reasons. Some demographic information collected on the subjects for this study gives a more detailed picture of the sample. The average age was 27.65 years, with the youngest being 22 and the oldest being 43. 22 23 Twenty-three of the subjects obtained their Bachelor's degree from Michigan State University, with 18 obtaining their degrees elsewhere in Michigan and the remaining 19 obtaining them outside of the state. Half of the 60 subjects had undergraduate majors in psychology or sociology, with the other half majoring in a variety of subjects. Everyone had either a Bachelor of Science degree--24—-or a Bachelor of Arts degree—-36. Twenty-six had teaching certificates and 34 did not. Finally, the distribution of major fields within the counseling program was heavily biased in favor of rehabilitation counseling. Thirty-seven subjects were in rehabilitation counseling, 13 were in community college counsel— ing, 9 were in Secondary school counseling, and l was in elementary school counseling. Treatments Three main treatment groups were employed. All three groups experienced conditions that were similar in many ways. As a means of describing the treatments, four sections will follow. The first section describes the conditions that were common to all three groups. The second section describes the conditions that were unigue to Group 1, which experienced the instructional model of main interest in this experiment. The third and fourth sections describe the conditions that were unique to Group 2 and Group 3, respectively. Group 2 and Group 3 were control groups for the instructional model. Following these descriptions, there is a composite diagram showing the various relation— ships among the three main treatment groups. This.diagram is displayed on page 34. 24 Eyperiences Common to All Three Treatment Groups (1) One week before the experiment, the experimenter met with all subjects and informed them of the evaluation, described it briefly, and requested everyone who would participate to fill out a Student Consent Form. They were also told that participation was voluntary and that their course grade would in no way be affected by whether or not they agreed to participate. The list of specific points covered by the experimenter in this pre-experimental session and a copy of the Student Consent Form are in Appendix A and Appendix B, respectively. (2) When the subjects arrived at the room where they were to go through the first part of the evaluation, they were placed around the room as far from one another as possible. The room was a regular classroom, and the subjects came in a series of groups which averaged eight members. (The schedule describing how the 60 subjects were assigned to come in small groups is described in the next section on procedures.) The object of spreading the subjects around the room was to decrease the possibility of a given subject's looking at or being disturbed by the work of another subject. (3) All the subjects were provided with programmed booklets that were especially assigned to each person. Differences in the programmed booklets provided the only treatment differences in this experiment. (4) After the subjects were seated and given a programmed booklet, an assistant to the main experimenter started a specially prepared audiotape recording. This instructional tape was prepared so as to tell the subjects how and when to move through the booklets. The only verbal 25 instructions the subjects received during this phase of the experiment was from the instructional tape. (5) When the subjects were instructed to start through the programmed booklet by the instructional tape, all of them first encount- ered a short one-page written description of the general focus of the experiment entitled "Counseling Objectives" (Appendix F). (6) After reading some other materials that concerned the unique aspects of each treatment group, all subjects went through a discrimina- tion training phase. (These materials are described later in this section.) This training had to do with learning to identify counselor statements, questions, and summaries that refer to the three basic com— ponents of a behavioral objective: terminal performance, conditions, and criteria (Mager, 1962). All subjects heard a series of counseling interview segments lasting approximately five minutes each. In these segments, the counselor (the main experimenter) was attempting to establish objectives for counseling with different clients. In some cases, the counselor made several references to the components of behavioral objectives, and in others he made none at all or only a few references. These short interview segments were specially prepared for the instructional tape. After each pair was played, all subjects were asked to pick which was the best example of counseling. The programmed booklets provided a place for them to write their answers. There were four pairs or eight interview segments presented on the instructional tape. 26 (7) After the discrimination training, all the subjects participated in one lS-minute counseling session with an individual who presented a role-played problem. The subjects were requested to take the role of a counselor. Before the role-playing session, the subjects were reminded by a written message given to them by the assistant experi- menter that the general focus of the evaluation was the establishing of objectives for counseling (Appendix C). (8) The role-players were, for the most part, doctoral students in college counseling or counselor education programs at Michigan State University. There was a total of 18 role-players, 8 females and 10 males. Ideally, the role-players should have been randomly assigned to each particular role-playing session. The personal schedules of the role-players prevented the use of this procedure. As a result, the role- players came according to a schedule that was made up relative to when each was free to participate in the study. (9) Several days before the experiment, the main experimenter met with each role-player and gave him or her a short description of the role to be played. There were two roles. One concerned a teacher who was unhappy with teaching and was trying to decide whether to leave the profession. The other was anooffice employee with a heavy drinking problem that was beginning to affect his work. The experiment was conducted on a Tuesday and Thursday of the same week. The teacher problem was presented on Tuesday, and the drinking problem was presented on Thursday. (See Appendix D for the descriptions that were given to each role- player.) Different roles were performed on different days so as to 27 lessen the possibility that the subjects would be able to help each other by talking about their experiences. In addition to each role-play description, the role-players were given a more general description of how to behave in the interview. Again, the purpose here was to standardize the role—players as much as possible. These instructions also included comments about how to tape record the interview and how to make sure the subjects were in the right place at the right time. (See Appendix N for a copy of the role—play instructions.) (10) After the short lO— to lS-minute role-played counseling session, all subjects were provided with a short paper entitled "Behavioral Objectives for Counseling" and were asked to read it. They were also provided with the Counseling Behavioral Objective Process Scale and a description of how to use this instrument. These two documents will be described later in this section. (11) All subjects then played two counseling interviews each of approximately seven minutes length on audiotape and were asked to rate the counselor's performance using the Counseling Behavioral Objective Process Scale. The counselor in these two interview segments was the main experimenter. The 11 items listed above describe the experiences which were common to all subjects in the experiment. The three main treatment groups also had unique experiences. These unique experiences are described next in the following order: Group 1, Group 2, and Group 3. For each group the unique experiences will be numbered in the order that they occurred. 28 It is intended that the numbers will correspond across the three groups. In other words, when Group 1 is engaged in the event described in (5), for example, Group 2 and Group 3 will also be engaged in the event described in (5). During the same event, different groups may be doing the same thing. For example, if Group 2 is doing the same thing as Group 1 during event (5), then the description under Group 2 for event (5) will be "The subjects had the same eXperience as-Group 1."; Other- wise, the unique activities of each event will be specified. Group 1: (l) A week before the actual experiment, the subjects were informed in their regular classes about the experiment and its conditions. Subject consent forms were filled out at this time. The subjects also were asked not to interact with any of the other subjects during the actual experiment or after it. (2) When the subjects arrived for the experiment, they received a programmed booklet and pencil and were asked to take a seat. (3) All instructions regarding how and when the subjects were to work through the booklets were on a specially prepared instructional tape which was started by the main experimenter's assistant when the subjects were ready. The subjects moved through the booklets on commands made on the instructional tape. (4) Initially in the programmed booklets, the subjects read a general introduction to the experiment. This paper was entitled, "Counseling Objectives" and can be found in Appendix F. 29 (5) The subjects read a paper on how to establish behavioral objectives for counseling. This paper is entitled "Behavioral Objectives for Counseling" and can be found in Appendix E. (6) The subjects read directions for a scale designed to measure the frequency of counselor efforts to establish counseling objectives. They also were presented with a copy of the scale. The scale is called the Counseling Behavioral Objective Process Scale (CBOP Scale), and can be found along with directions for its use in Appendix H. (7) The subjects then heard five pairs of counseling interview segments. Directions in the programmed booklet asked them to make tallies for each counseling interview using the CBOP Scale. After they had made tallies for each pair, they were asked to pick the better counseling performance relative to setting up behavioral objectives based on which of the two segments received the highest tally. Places were provided in the programmed booklets for the subjects to record their tallies and pair selections. At this point in the process, Group 1 was divided into two equal groups. Throughout the period when the subjects were recording tallies and selecting interview segments, half of Group 1 received feedback on which of the two setments in each pair actually contained a higher tally. The other half of Group 1 was asked to think back over the pairs and to make sure they had made the correct choice. They received no feed- back. Again, it should be emphasized that the difference in treatment experiences was provided by the materials in the programmed booklets. All the subjects listened to the same instructional tape. 30 (8) After the training session, the evaluation of the subjects' performances was started. Immediately after the last pair of interview segments was presented and feedback or no feedback was given, the sub- jects were asked to counsel a live role-playing client for approximately 10 to 15 minutes. The subjects were told two things before the live evaluation. First, they were reminded that the focus of the project was on establishing objectives for counseling. Second, they were told to assume that they had already talked with the client for a period of time and that they had agreed on what the basic problem for counseling was to be. This session lasting approximately 10 to 15 minutes was audiotape recorded for later evaluation. (9) After the live role-playing session, each subject received the "Behavioral Objectives for Counseling" paper and the "Counseling Behavioral Objective Process Scale" and was asked to read them again. The last thing the subjects were asked to do was to make tallies on two approximately seven-minute counseling interview segments using the CBOP Scale. Group 2: (l) The subjects had the same experience as Group 1. (2) The subjects had the same experience as Group 1. (3) The subjects had the same experience as Group 1. (4) The subjects had the same experience as Group 1. (5) The subjects had the same experience as Group 1. (6) At this point, Group 2 had a different experience from Group 1. The subjects in Group 2 were asked to reread the paper entitled 31 "Behavioral Objectives for Counseling." They had already read it in (5). This rereading experience was intended to be a control experience for the presentation of the CBOP Scale to Group 1. (7) During this phase of the experiment, the four pairs of counseling interview segments were played. The subjects were asked to listen carefully to each pair and then to select the best counseling performance. Group 2 subjects were not told to use any specific criteria in their selection process. Just as in Group 1, at this point in the process Group 2 was divided into two equal groups. Half of Group 2 received feedback on whether their selections were right or wrong, and the other hald received no feedback. Of course, the subjects who received feedback were unaware of the means by which one interview segment was determined to be better than another. (8) The subjects had the same experience as Group 1. (9) The subjects had the same experience as Group 1. 22.9.22; = (l) The subjects had the same experience as Group 1 and Group 2. (2) The subjects had the same experience as Group 1 and Group 2. (3) The subjects had the same experience as Group 1 and Group 2. (4) The subjects had the same experience as Group 1 and Group 2. (5) At this point, Group 3 read different materials from Group 1 and Group 2. Group 3 subjects read a two-part paper dealing with identifying concerns for counseling and selecting concerns for counseling. This material comes from the Michigan State University counselor training 32 materials (Stewart, Burks, Engelkes, Johnson, & Winborn, 1972). These materials had been presented to the subjects earlier as a part of their regular class work. It was presumed that this material would not be new to the subjects. A copy of this two-part paper can be found in Appendix G. The experience of reading this paper was intended to be a control experience for the presentation of the paper entitled "Behavioral Objectives for Counseling" which was presented to Group 1 and Group 2. (6) All subjects in Group 3 were then asked to reread the two- part paper they had just read. This rereading experience was intended to be a control experience for the presentation of the CBOP Scale to Group 1. (7) The subjects had the same experience as Group 1 and Group 2. (8) The subjects had the same experience as Group 1 and Group 2. (9) The subjects had the same experience as Group 1 and Group 2. By way of conclusion regarding the treatment descriptions, it would be useful to comment on the rationale behind the composition of the three main groups. Group 1 was the main group of interest in this experiment. This group represented the model to be evaluated which is, described in Chapter 1 in the section on purpose. Group 2 and Group 3 were intended mainly to control for the variables present in Group 1. In the strictest sense, then, this experiment is pp; a comparison study between Group 1 and the other two groups. It may be noted possibly that Group 2 and definitely Group 3 represent procedures for counselor training that would seldom be used. Therefore, if it is shown that Group 1 is taught by the better method, 33 nothing is really proved. This criticism would be significant if the experiment were a comparison between these methods. The point of this experiment was to determine what variables in Group 1 are important to the model, or in other words, what procedures do, in fact, produce change in the behavior of the subjects. An outline of the experiment for Groups 1, 2, and 3 is shown in Figure 2.1. Procedures The subjects came from two classes of Master's degree students in the Michigan State University counseling program. All of the subjects were in their second term of classes in the program. There was a total of 61 students in both classes, and all of them were willing to partici- pate except 1. This 1 student did not participate for personal reasons. With the exclusion of 1 student, the total sample size was 60 subjects. This number was appropriate from the standpoint of allowing for equal cell sizes in the three main treatment groups. Subjects were assigned to the different treatment groups using a random procedure. The principle of random assignment to treatment groups was relied upon to equate experimental and control groups with respect to relative learning ability and background knowledge. First, the subjects were numbered one through sixty. Second, a random ordering of the numbers one through sixty was obtained from the Michigan State University Control Data 3600 computer. Third, the six main treatment categories were specified in the following order: 31; Group 1 Group 2 Group 3 Verbal intr oduction to Same as Group 1 Same as Group 1 experiment in regular class session Feedback Subjects make tallies on interview with CBOP Scale while listening to interview segment Subjects pick better of each pair based on highest tally Subjects receive feedback on which is the better selection possess-s eeeee Subjects read "Counseling Objectives" Subjects read "Behavioral Objectives for Counseling" Subjects read about the CBOP Scale No Same as Group 1- feedback Same as Group 1- feedback Subjects receive no feedback oooooooooooo Eesdbeek... PROGRAMMED BOOKLET Same as Group 1 Same as Group 1 Subjects reread "Behavioral Objectives for Counseling" Training Session (The Following sequence was repeated five times:) Same as Group 1 Subjects read two-part paper on identifying concerns for counseling Subjects reread the two- part paper on identifying concerns for counseling Feedback No Feedback No .............. .Eeedbeek...................I..Eeedbeek. Subjects Same as Subjects Same as listen to Group 2- listen to Group 3- interview feedback interview feedback segment segments Subjects Same as Subjects Same as pick better Group 2- pick better Group 3— of each feedback of each feedback pair pair Subjects Subjects Subjects Subjects receive receive receive receive feedback no feedback no feedback feedback follows: Scale. interview. the client. TAPE EVALUATION During the evaluation which took place after the subjects went through the training session, all groups had the same experience. A description of the experience This experience ended the evaluation. 1. The subjects participated as counselors in a 10- to lS-minute role-play They were asked to establish an objective for counseling with 2. After the interview, the subjects read the paper entitled "Behavioral Objectives for Counseling." 3. The subjects next read the "Counseling Behavioral Objective Process Scale" and directions for the scale's use. 4. The subjects than listened to two, 5- to lO—minute counseling interview segments on audiotape and made tallies for each interview using the CBOP FIGURE 2.1. Outline of the Experiment. 35 1. Group l--feedback 2. Group l--no feedback 3. Group 2--feedback 4. Group 2--no feedback 5. Group 3--feedback 6. Group 3-—no feedback Fourth, the student assigned to the first number of the randomly ordered numbers was placed in Group l--feedback. The student assigned to the second number of the randomly ordered numbers was placed in Group 1--no feedback. This procedure was continued, recycling through the six treatment groups until all 60 subjects were assigned to a group. As stated in the section describing the treatments, the treatment differences were created solely by the type of booklet each subject was assigned during the actual experiment. All subjects listened to the same instructional tape. With these conditions, it was not a concern that each treatment group participate in the experiment as an intact experi- mental section. The subjects, therefore, came in experimental sections whose average size was between seven and eight. The smallest experi— mental section consisted of four subjects, and the largest had ten subjects. The experiment was run on Tuesday, February 20, 1973, and Thursday, February 22, 1973. Two experimental sections were processed in the morning of each day, and similarly, two experimental sections in the afternoon. Individual subjects were assigned to come at times which were most convenient for them. No attempt was made to randomize the 36 times at which subjects participated because of difficulties of coordinat- ing the schedules of so many people. As described in the treatments section, after the training phase of the experiment was completed, all subjects went through an evaluation phase. The evaluation phase consisted of a 10» to 15-minute counseling interview with a role-playing client. Before participating in the evaluation session, each subject was given a written statement asking him to go to a specific room, and either to meet a role-player who was already there or to wait for one who would come (Appendices J, K, L, and M). These directions also included a brief statement regarding the nature of the problem that the role-player would present and a reminder that the focus of the experiment was on setting objectives for counsel- ing. Note that "behavioral objectives" was not mentioned here. Group 3, to this point in the experiment, had not had any exposure to "behavioral objectives." It was mentioned earlier that after the training sessions the subject would either meet with the role-player immediately or have an approximately 10- to 15-minute delay. Half of the subjects were in the delay setting, and the other half experienced no delay. After the subject was assigned to a given training session, each experimental section was randomly divided in half and assigned to either the delay or no-delay group. The reason for employing a delay or no—delay factor was a lack of role-players. Using the procedure described here, a given role-player would present a role to one subject, and then would immed- iately go to another room where another subject would be waiting. The 37 role-player would then present the same role again. The actual time schedule for Tuesday's sessions was as follows: Training Session One (six subjects) 8:05-9:10 AM Taped Evaluation Session One--no-delay (three subjects) 9:10—9:25 AM Taped Evaluation Session One-—delay (three subjects) 9:25-9:35 AM Training Session Two (eight subjects) 9:15-10:20 AM Taped Evaluation Session Two-—no-de1ay (four subjects) 10:20-10:35 AM Taped Evaluation Session Two--delay (four subjects) 10:35-10:50 AM Training Session Three (ten subjects) 4:05-5:10 PM Taped Evaluation Session Three-~no-delay (five subjects) 5:10—5:25 PM Taped Evaluation Session Three-delay (five subjects) 5:25—5:40 PM Training Session Four (ten subjects) 5:15—6:20 PM Taped Evaluation Session Four—~no—delay (five subjects) 6:20-6:35 PM Taped Evaluation Session Four--delay (five subjects) 6:35-6:50 PM Similar sequences and times were followed on Thursday. The only dif- ference between the training structure from Tuesday to Thursday was 38 section size. The experimental section sizes for Thursday were as follows: Training Session One (eight subjects) Training Session Two (four subjects) Training Session Three (four subjects) Training Session Four (ten subjects) A total of 18 different people participated as role-players in this experiment. Fifteen of the 18 role-players were doctoral students in the counseling program at Michigan State University. They varied from first year students to third year students. Ten were males and 5 were females. The other 3 role-players making up the total of 18 were females, and each had reached the Master's degree level of education. When assigning role—players to taped evaluation sessions, no regard was taken for the sex or degree of training of the role-players. Ideally these factors should have been included in the design, but the problems of dealing with large numbers of people who had very busy schedules precluded a systematic assignment of role-players to taped evaluation sessions. Because of the possible unsystematic variability of role-player behavior, the main experimenter made an attempt to standardize the roles that were presented as much as possible. Whiteley and Jakubowski (1969) point out several difficulties of counselor trainee evaluation if the behavior of coached clients is not consistent across trainees. The main experimenter used the following procedure in an attempt to standardize role—player behavior. 39 (1) The main experimenter met with all role-players for 30 to 45 minutes to explain their part in the experiment. These meetings were usually held individually, with the main experimenter going to the homes of the role—players. (2) At these meetings, the role-players read a description of the role or roles they were to play (Appendix D). The idea was expressed to each role—player that the main function of the role descriptions was to provide him with information he should use in responding to the subjects who were pretending to be counselors. (3) The main experimenter also gave each role-player an overall description of how the role should be carried out (Appendix N). The following points were emphasized with each role—player regarding how he was to play his role: (a) Be an agreeable client. (b) Do not lead the counselor or present difficult problems. (c) If the counselor asks questions, freely respond using the information given in the role description or anything you can add to it. (d) Do not dominate the session. Make your own answers or com- ments fairly short and to the point. (e) Try not to behave in such a way so as to increase the subject's anxiety. (f) Do not feel that you have to use all the information in the role-play description. Let the counselor draw the informa- tion out. 40 After the points regarding the actual role—play were covered, the role-players were specifically shown how to operate the tape recorder so that a recording could be made of the role-played evaluation. It would be useful at this point to explain that the main experimenter went to a great deal of effort to insure that high quality tape recordings would be made during the role—play evaluation. The cassette tapes that were used were all new and were a recommended brand name. All cassette tape recorders that were used were checked for their recording capability. In the rooms where the role-played evaluations took place, the main experimenter arranged the chairs, table, recorder, and microphone so as to insure optimal recording conditions. The microphone itself was taped to the table so it would not be moved by the participants. The role—players were told about the taping procedures and were asked not to move the chairs and tables. They were also asked to give a booklet to the subject and to show him or her how to operate the tape recorder after the role-play evaluation. The booklet and tape regarding the tape evaluation were placed in the room before the role-play evaluation. After giving the subject the booklet and illustrating how to work the machine, the role—players were to leave the room. lnstrumepps The main instrument used in this experiment is called the Counseling Behavioral Objective Process Scale (CBOP Scale). (See Appendix H.) This scale was devised by the main experiménter as a device 41 for recording the number of statements a counselor makes regarding the three parts of a behavioral objective as specified by Mager (1962). lt is assumed that anyone who uses this scale has had an introduction to the concept of a behavioral objective and its three parts: (a) terminal performance, (b) conditions, and (c) criteria. Specific directions for use of the CBOP Scale can be found in Appendix H. The CBOP Scale was used in a variety of ways throughout the experiment. The subjects in Group 1 used it during the training phase. As they listened to the four pairs of counseling interview segments, they tallied the number of statements made by the counselor to the parts of a behavioral objective as a means of selecting the better segment of each pair. The tally form they were provided to record counselor statements for a given interview segment can be found in Appendix 0. The CBOP Scale was also used by two trained recorders who listened to the tapes made by the subjects and the role-players during the role-play evaluation phase of the experiment. The recorders' tallies provided the measurements of main interest in the experiment. The form the trained recorders were provided to tally the number of the subject's statements for the role-played interview segment can be found in Appendix 1. In order to insure objectivity on the part of the two trained recorders, they participated in approximately two hours of training with the main experimenter. This training consisted of making tallies while listening to the interview segments on the instructional tape. The training ended when the two recorders obtained approximately a .80 correlation in their inter-rater reliability. 42 The CBOP Scale was also used by all subjects at the end of the experiment. After the role-play session, all subjects were given a paper describing the concept of behavioral objectives (Appendix E), the scale, and directions for its use. After reading these materials, the subjects played two short counseling interview segments and used the scale to tally counselor statements relative to behavioral objectives. The form they used to make their tallies was the same as used by Group 1 in the training phase of the experiment (Appendix 0). A comment may be useful in understanding the use of the CBOP Scale. It should be stressed that the scale is basically a means of keep- ing an accurate tally. The judgemental aspects of using the scale come in two categories. First of all, the one making the tallies must decide when the counselor has made a statement. The rules for this decision are spelled out in the directions for the scalels use. Second, the user must decide if a given statement refers to terminal performance, conditions, criteria, any combination of the three, or to none of them at all. This decision is based on previous reading relative to the characteristics of these three components. Designjand Statisticgl Analysis The major independent variable of interest in this experiment was the group treatment. There were three main group treatments. The second independent variable of interest was the feedback or no feedback component in the training phase. The group treatment variable was crossed with the feedback—no feedback variable, producing a basic three 43 by two fixed effects model. There was a total of 60 subjects in the experiment, and they were randomly placed under the six treatment conditions. The major dependent variables of interest were the number of references made to the components of a behavioral objective by each of the subjects during the role-played session. These references were tallied by trained recorders who listened to audiotape recordings of the role-played interviews. The trained recorders used the CBOP Scale procedures just as the subjects in Group 1 did during the training phase. For each subject, the recorders reported the number of references made by the subjects to terminal performance, conditions, and criteria. They also recorded a total score for each subject,. which consisted of the sum total of references to terminal performance, conditions, and criteria. Several other variables were utilized in order to make the analysis as complete as possible. These variables are listed as follows: (1) the length in minutes of each role—played interview, (2) the day the subject participated in the experiment (Tuesday or Thursday), (3) the time of day (AM or PM) the subject went through the experiment, (4) whether the subject participated in the role—play session immediately after the training session or whether the role- play session came approximately 10 to 15 minutes after the training session (delay--no delay), (5) (6) (7) (8) 44 the total number of correct answers for each subject during the training phase of the experiment, the number behavioral evaluation the number behavioral four pairs of tallies relative to the components of a objective made by the subjects on the tape at the end of the experiment, of tallies relative to the components of a objective made by the subjects in Group 1 on the of counseling interviews they heard during the training phase of the experiment, and the sex of the subjects. CHAPTER III ANALYSIS OF RESULTS The analysis of the data obtained in this experiment was performed on the Control Data Corporation 3600 and 6500 computer systems in the Computer Center at Michigan State University. The level of significance for all tests was .05. Recorder Correlations and Dependent Varipples The dependent variables of major interest were tallies of the references made by subjects to the components of a behavioral objective during a role-play evaluation. These tallies were made by the trained recorders as described earlier. In order to insure a certain degree of consistency between the Uwo recorders, the main experimenter arbitrarily determined that when the two recorders disagreed by eight or more on any given subject's total score tally, the recorders would evaluate that subject again. After the initial evaluation of the 60 subjects, it was found that the two recorders disagreed on five subjects as determined by the arbitrary rule. The recorders were asked to evaluate the five subjects again. After the second evaluation, the two recorders fell within the criteria on all subjects. In order to check the consistency between the trained recorders, product-moment coefficients of correlation were calculated using the 45 46 traditional raw score formula. The correlations are listed in Table 3.1. From Table 3.1, it can be seen that the two recorders obtained correlations over .80 on two of the three components and the total score. Clearly, the recorders had a more difficult time in agreeing on when the subjects made references to the conditions of behavioral objectives. On the conditions component, they had a correlation of only .66. After making the tallies for all 60 subjects, the recorders also reported that conditions was the most difficult of the three variables to identify. The observed cell means of the subjects' references as tallied by the recorders are shown in Table 3.2. A graphical representation of the observed cell means of the subjects' references corrected for time and ignoring the feedback--no-feedback component can be found in Figure 3.1. The subjects' references were calculated in the following manner: A + B T where A = Recorder A's tally, B Recorder B's tally, and T 2 Length of evaluation tape in minutes. As there was a high correlation between the two recorders, it was decided to total both tallies for each subject. The sum of the tallies was divided by the length of the role-play evalua- tion. This procedure was followed because there was a wide range in the length of the role-play evaluations. The range of interview length was from four minutes to nineteen minutes with a mean length of twelve minutes and a standard deviation of 3.5 minutes. In order to simplify terminology for the rest of this chapter, all references to terminal performance, 47 TABLE 3.1. Product-Moment Coefficients of Correlation Between the Two Trained Recorders for the Role-Play Evaluation Behavioral Objective Component Recorder Correlations Terminal Performance (TP) . . . . . . . . . . . +.82 Conditions (CO) . . . . . . . . . . . . . . . . +.66 Criteria (CR) . . . . . . . . . . . . . . . . . +.8l Total Score (TP + C0 + CR) . . . . . . . . . . . +.87 1 30 pGroup 1 (Advance Organizer): X X Group 2 (Advance Organizer Control): H Group 3 (Advance Organizer Control): H H 1.20 I- 0 L14 8 u 1.10 '- o (I) H Li 8 1.00 " (.0 .LJ 5 I- ; .90 o o. B 8 i: u .80 '- a.) on :> I: "-4 Q 8 u—I Q) 3 .70 '— 41, (I) .0 ~I-I O P I a; ‘. .m U .60 i.- "\ \ {/O H C \ u’ o H \ ‘( "-4 R 5‘ r 3 F4 030 '- 0 cu m I «n 0 r4 I.) O I __ a: .~..O ’0 “I .: {é ..:'U u: I" o M 1,0 -i- - ..I . - I I I, ...-.._.. '<. (4': :If,‘ \_. "" ' 1’91 10'... Induce Components of a Behavioral Objective ,I’II_.I_II{E 3.1. Mean '.\II_I:‘I:.'_";.: 0f Relaxantes 1‘0 C’J‘IIIP’."EIGF‘.L‘3 of J‘- B‘C’I’Iavioz'a? I"; ("It 1' are it: Each Group. 49 TABLE 3.2. Summary of Observed Cell Means of Subjects' References Corrected for Length’of Role-Play Interview Terminal Performance Conditions Criteria Total Group 1 - Feedback 1.00 .32 .49 1.80 Group 1 - No Feedback 1.07 .47 .63 2.17 Group 2 - Feedback 1.07 .44 .70 2.21 Group 2 - No Feedback 1.17 .32 .50 1.99 Group 3 - Feedback .79 .23 .44 1.47 Group 3 - No Feedback .84 .24 .35 1.44 50 conditions, criteria, and total score are considered as transformed scores which have been corrected for the length of the interview. In all cases, the transformation is made by dividing the score of interest by the length of the role-play interview in minutes. It may be helpful to note here that more "meaningful" scores could be obtained by using the formula A + B 2T S: instead of the one indicated above. With A + B in the numerator, there is, in essence, a double score, and this situation might well be repre- sented in the denominator with 2T instead of T. The T value was used, however, for reasons of simplicity and because the statistical computa- tion is basically no different in either case. Results of Major Hypotheses The three major hypotheses of this experiment were tested using the univariate analysis of variance procedure. The following are restatements of the major hypotheses relative to the results of the analysis. Hypothesis 1 There was no difference between the number of responses regarding the components of a behavioral objective made by the subjects who had training with the Counseling Behavioral Process Scale and those subjects who did not have training with the scale. 51 Hypothesis 11 There was no difference between the number of responses regarding the components of a behavioral objective made by the subjects who studied a written description of good behavioral objectives for counseling and that of those subjects who did not study the description. Hypothesis III There was no difference between the number of responses regarding the components of a behavioral objective made by subjects who received feedback during training exercises to recognize good counseling per- formances relative to setting behavioral objectives and that of those subjects who received no feedback. The remainder of this chapter describes the data which support the findings reported above. The univariate analysis of variance relative to Hypothesis 1, Hypothesis II, and Hypothesis III, are shown in Table 3.3. The dependent variable used here was the total number of references to the components of a behavioral objective disregarding whether they were terminal performance, conditions, or criteria references. The three main hypotheses are concerned only with the total number of references. As indicated in Table 3.3., there was no significant inter- action between the independent variables of group and feedback. The results regarding the main effects are interpreted as providing no evidence that differences existed after the treatment among any of the three treatment groups as measured by the total score for each subject. In Table 3.3, it should be noted that the tests of Hypothesis I app Hypothesis 11 are reported in the first row which is designated "Group 52 TABLE 3.3. Univariate Test of Hypotheses I, II, and 111: Number of Total References Subjects Made to the Components of a Behavioral Objective Source df 88 MS F p less than Group (G) 2 4.79 2.39 1.28 .29 Feedback (F) l .02 .02 .Ol .91 G X F 2 .88 .44 .24 .79 Error 54 100.83 1.87 NOTE: Description of p value: If the null hypothesis is true, the p value is the probability of obtaining an F ratio as large or larger than the ratio calculated for this sample. If the p value is less than the chosen c<3 the null hypothesis is rejected. 53 (6)." In addition, no significant differences were found between the feedback and no-feedback groups. It is also useful to consider the individual scores the subjects obtained relative to the three components of a behavioral objective. Even though differences were not found when analyzing only the total scores, it is conceivable that there might be differences relative to the component scores. These scores can be analyzed together using a multivariate analysis of variance test. Results of the multivariate analyses of Hypothesis 1, Hypothesis II, and Hypothesis III are shown in Table 3.4. As with the univariate analysis of variance of the total scores, there was no significant interaction between the independent variables of group and feedback indicated in Table 3.4. The results regarding the main effects are interpreted as providing no evidence that differences existed after the treatment among the three groups on any of the dimensions. In addition, no significant differences were found between the feedback--no-feedback groups. Results of;$econdary Hypotheses As previously noted, no significant differences were obtained among the three main treatment groups relative to the total scores. Upon examination of the frequency distribution of the total scores by groups, however, it appeared that Group 1 had a greater variance than the other two groups. (See Figure 3.2.) Initially, there was no reason to suspect that one particular group would have a larger variance than another, so a two-tailed rejection region was selected, and the null and alternative 54 TABLE 3.4. Multivariate Test of Hypotheses I, II, and III: Terminal Performance, Conditions, and Criteria Scores Source df Multivariate F p less than Group (G) 6, 104 .53 .78 Feedback (F) 3, 52 .42 .74 G X F 6, 104 .88 .52 55 .maoOHO we muommnjm wo mmpOUm HmDOH umuomupoo .w.m mmaon newcmq 3mH>umucH mmamlmaom pow nmuuouuou mmuoom HMDOH 000000000000000000000000000000000000 086420864208642086420864208642086420 7666665 555544.444.33313222221LLLL00.0.0.0. X X X X XXXXXX X X X XX X X X X X XX XXXXX XX XX XX XX X X X X X X X X XXX X XXXXX X XX X X X X X muommnsm Agonucoo nonficwwuo mocm>ocU¢v H moose mo nonssz 56 hypotheses for the populations were determined as follows: Hays (1963, pp. 351—352) has suggested the following test for the comparison of two sample variances; ~1 and N —1 In this case, F should approximate an F distribution with N O 1 degrees of freedom. The F values which compare the total score variances are shown in Table 3.5. The sample variances used in this analysis were obtained by pooling the variances of the feedback and no feedback groups for each of the three main treatment groups. This procedure required the loss of two degrees of freedom for each group. The critical values for a two-tailed test using 18,18 degrees of freedom are 2.604 for the upper region and .384 for the lower region. As indicated by Table 3.5, it can be seen that the variance for Group 1 is significantly different' from the variance for Group 3. The other group comparisons did not reach significance. Corresponding calculations were made for the terminal performance, conditions, and criteria scores. The F values for these comparisons are shown in Table 3.6. It can be seen from Table 3.6 that two of the three comparisons for the condition scores were significant. The condition score relative to the Group 1/Group 2 comparison almost reached signim ficance. Of the other comparisons, only the comparison between Group 1 57 TABLE 3.5. F Values for Group Variance Comparisons: Total Scores Group l/Group 2 2.32 Group 1/Group 3 2.63 Group 2/Group 3 1.13 NOTE: The critical values for a two-tailed test at the .05 level with 18, 18 degrees of freedom are .384 and 2.604. 58 TABLE 3.6. F Values for Group Variance Comparisons: Terminal Performance (TP), Conditions (C0), and Criteria (CR) Scores Group 1 / Group 2 Group 1 / Group 3 Group 2 / Group 3 TP 1.90 3.01* 1.58 CO 2.59 17.54* 6.77* CR 1.76 1.09 .62 *The critical values for a two-tailed test at the .05 level with 18, 18 degrees of freedom are .384 and 2.604. 59 and Group 3 for the terminal performance scores reached significance. The actual variances for the three main treatment groups relative to the terminal performance, conditions, criteria, and total scores are shown in Table 3.7. I It can be seen from Table 3.7 that in every case except one, Group 1 had a larger variance than Group 2, and that Group 2 had a larger variance than Group 3. The exception is where Group 3 had a larger variance than Group 2 on the criteria score. The above finding regarding the significantly larger variance on total scores in Group 1 as compared to Group 3 suggests that the instructional model of main interest has a tendency to produce extreme total score values. It would be of interest to look closely at those subjects who fell below the mean and those who fell above it to see if there are any characteristics which distinguish them. It should be kept in mind, however, that this analysis was not originally proposed at the onset of the experiment. Therefore, only very limited conclusions can be reached. Several demographic variables were collected on the subjects. They were (a) sex, (b) age, (c) location in which Bachelor's degree was earned, (d) type of Bachelor's degree--B.A. or 8.8., (e) counseling program major, and (f) possession of teaching certificate. It seemed possible that if a subject had a teaching certificate, he might have been exposed to the behavioral objective concept, and possibly might have been helped in the learning process. This explanation does not account for why those who did not possess the certificate would be hindered. 60 TABLE 3.7. Variances for the Three Main Treatment Groups: Terminal Performance (TP), Conditions (00), Criteria (CR), and Total (TOT) Scores Group 1 Group 2 Group 3 TP .59 .31 .19 C0 .26 .10 .02 CR .34 .19 .31 TOT 3.09 1.33 1.18 61 A Chi-squared test known as Fisher's Exact Test (Conover, 1971, pp. 154—166) was performed, testing for no relationship between indi- vidual total scores above or below the mean of Group 1 total scores and the possession of a teaching certificate. Fisher's Exact Test was used mainly because it is the exact Chi-squared test, and the number of cells was small. Using a two—tailed test with .05 as the target level of significance, no relationship was found between the two variables. There is little reason to suspect that any of the other variables listed above would be related to high or low total scores. As previously stated, the experiment was run on Tuesday and Thursday of one week. On each day, all role-players presented the same problem. On Tuesday, the role had to do with a teacher who was con- templating the idea of changing his career from teaching to real estate. On Thursday, the role dealt with a person who was drinking to excess. It is of interest in this analysis to see whether the presentation of different roles had an effect on the number of references the subjects made to the components of a behavioral objective. It is also of interest to see whether the presentation of different problems had an interaction effect with the treatment procedures. If there was a sig- nificant interaction, the implication is that certain treatments are more effective with certain client problems. The following analysis was carried out in an attempt to test for these effects. A univariate analysis of variance was conducted with the type of problem presented and the group treatment as independent variables and with the total number of references made to the components of a behavioral 62 objective as the dependent variable. The results of this analysis are contained in Table 3.8. As indicated by Table 3.8, there was no signi- ficant interaction effect or group effect. There was a significant effect relative to the problem presented at the .0226 level. With a significant effect relative to the problem presented, it becomes necessary to look at the multivariate analysis of variance of problems and groups with the terminal performance, conditions, and criteria scores as dependent variables. The results of this analysis are contained in Table 3.9. A significant effect was obtained as a result of the problem presented when the three basic scores were considered together. As a post-hoe procedure, the univariate analysis of variance tests for each of the three scores relative to the problem presented were considered. The results of this analysis are shown in Table 3.10. It can be seen from Table 3.10 that the conditions and criteria scores contributed the most to the significant differences obtained relative to the different problems presented. The means of the conditions and criteria scores relative to the problems presented are shown in Table 3.11. Surprisingly, there is an interaction effect in this result. The problem presented on Tuesday was associated with more criteria references and fewer conditions references. The reverse is true for the role presented on Thursday. During the role-play evaluation, the subjects participated in the evaluation immediately after the training phase or they had a 10- to 15— minute delay. It is of interest to see if this delay had an effect on 63 TABLE 3.8. Univariate Test for Problem Presented by Role-Players and Treatment Groups: Number of References to the Components of a Behavioral Objective Source df 85 MS F p less than Problem (P) 1 9.09 9.09 5.52 .02 Group (G) 2 3.17 1.59 .96 .39 P X G 2 5.29 2.64 1.60 .21 Error 54 88.97 1.65 64 TABLE 3.9. Multivariate Test for Problem Presented by Role-Players and Treatment Groups: Terminal Performance, Conditions, and Criteria Scores Source df Multivariate F p less than Problem (P) 3, 52 4.32 .0086 Group (G) 6, 104 .70 .65 P x G 3, 104 1.04 .40 65 TABLE 3.10. Univariate Test for Problem Presented by Role-Players: Terminal Performance, Conditions, and Criteria Scores Source MS Between MS Within Univariate F p less than Terminal Performance .52 .34 1.54 .22 Conditions .62 .13 4.96 .03 Criteria 2.26 .24 9.33 .0035 NOTE: Degrees of freedom for hypothesis = 1; Degrees of freedom for error = 54. 66 TABLE 3.11. Means and Standard Deviations of Conditions and Criteria Scores: Problem Presented by Role—Players Conditions Criteria Mean Standard Mean Standard Deviation Deviation Problem (Tuesday) .20 .24 .43 .34 Problem (Thursday) .30 .35 .27 .32 67 the performance of the subjects in the role-play evaluation. A possible hypothesis might be that the delay gave the subjects more time to think and to organize the training experience. This additional time could be a key factor relative to the performance level in the role-play evaluation. A univariate analysis of variance test was carried out to compare the total scores made by the delay and by the no—delay groups. No significant difference was found (p <:.8730). This finding indicates there is no evidence that the delay--no-delay factor had an effect on the number of references to the components of behavioral objectives made by the subjects. During the training phase of the experiment, all subjects were asked to select from four pairs of taped counseling interview segments the best counseling performance relative to establishing objectives. One of the counseling performances in each pair was designated as better than the other if the counselor in it made more references to the components of a behavioral objective. The number of references to the components of a behavioral objective in each interview segment was tallied by the same two trained recorders who were used to record the subjects' per- formances in the role-play evaluation. The number of correct selections made by each subject (the best possible score was four correct responses) was totaled and used as a dependent variable. It is of interest to know if Group 1 made more correct selections than the other two groups. It is also of.interest to know if the subjects who received feedback did better than those who received no feedback. 68 Two types of analyses, with groups and feedback as the dependent variables, were performed t4 answer the above questions. First, a univariate analysis of variance test with the total scores was performed. The results of this test can be seen in Table 3.12. No significant differences were obtained relative to either independent variable or interaction. It should be noted that the .00 value for the sum of squares resulted from the fact that the average of the three cell means for the feedback groups equaled the average of the three cell means for the no feedback groups. Second, a multivariate analysis of variance test with the number of correct selections for each of the four pairs was performed. This test was performed to see if there was any relationship between groups and the four pair selections. The test also analyzed the possible relationships between the feedback component and the pair selections. The results of this test can be seen in Table 3.13. Similar to the univariate test, no significant differences were found. It is worthwhile noting, however, that the effect due to feedback seems to be approaching the .05 level with p less than .0839. After subjects participated in the role-play evaluation, they were given the materials on behavioral objectives in counseling-—used by Group 1 and Group 2 in the treatment--and the Counseling Behavioral Objective Process Scale (CBOP Scale)--used by Group 1 in the treatment-- to read. When they had read these materials, they played two audiotaped counseling interview segments on the tape recorder already in the room. These two segments are called tape evaluation segments A and B. As the subjects listened to the tape evaluation segments, they were asked to 69 TABLE 3.12, Univariate Test for Treatment Group and Feedback: Total r Number of Correct Selections During Role-Play Evaluation Source df SS MS F p less than Group (G) 2 27.13 1.07 2.10 .13 Feedback (F) 1 .00 .00 .00 1.00 G X F 2 .40 .20 .39 .68 Error 54 27.40 .51 70 TABLE 3.13. Multivariate Test for Treatment Groups and Feedback: Number of Correct Selections During Role-Play Evaluation Source df Multivariate F p less than Group (G) 8, 102 .84 .57 Feedback (F) 4, 51 2.18 .08 c x F s, 102 .69 .70 71 use the CBOP Scale to record the number of references made by the counselor to the components of a behavioral objective. They used the same format for recording their tallies as was used by Group 1 during the training phase. The main purpose for having all the subjects participate in this tape evaluation at the end of the experiment was to see whether practice with the CBOP Scale helped an individual become a more accurate recorder. If practice with the scale helped, then the subjects in Group 1 should have recorded more accurately than the subjects in Group 2 or Group 3. In order to measure the degree of accuracy of each subject, a reference set of tallies was required. The trained recorders listened to the tape evaluation segments A and B and recorded the number of counselor references to the components of behavioral objectives. The averages of their tallies provided the reference set of tallies to which each sub- ject's tallies were compared. The actual means of comparison consisted of taking each of the subject's four tallies--(a) terminal performance, (b) conditions, (c) - criteria, and (d) total score--and subtracting the corresponding trained recorders' average tallies. This set of difference scores consisting of positive and negative values was then converted to absolute values. (Subject tallies) - (Trained recorder tallies) = X These values (denoted by X in the preceding equation) were, in effect, the measure of how close to the trained recorders each subject was able to record using the CBOP Scale. As stated before, the hypothesis under consideration here would state that Group 1 tallies should be more 72 consistent with the tallies of the trained recorders than the tallies of Group 2 or Group 3. The analyses on tape evaluation segments A and B were carried out separately. For each analysis, a univariate analysis of variance test was performed on the total scores, and a multivariate analysis of variance test was performed on the three scores which comprised the total score. In both cases, the independent variables of interest were group treatment and feedback--no feedback. The analysis for segment A is presented first. The univariate test on the total scores and also the multivariate test on the three individual scores are shown in Table 3.14. As indicated by Table 3.14, there were no significant differences between the treatment groups or the feedback-—no feedback groups on any of the scores., The analysis for segment B is presented similarly to the analysis for segment A. Information that is parallel to the information presented in Table 3.14 is contained in Table 3.15. From Table 3.15, it can be seen that there were no significant differences on the multivariate test for the three individual scores, but there was a significant difference on the univariate test. For the total scores, a significant difference was found between the treatment groups at the .0241 level. It is useful to look at the total score group means after having found this differ- ence. These means are presented in Table 3.16. Unexpectedly, Group 1 had the highest mean of the three groups. As stated before, the scores being considered here are absolute value difference scores, and a low score indicates high agreement with the trained recorders. Group 2 had the lowest score. 73 TABLE 3.14. Univariate and Multivariate Tests for Treatment Group and Feedback (Segment A): Total Scores and Terminal Performance, Conditions, and Criteria Scores 1. Univariate Test on Total Scores Source df SS MS F p less than Group (G) 2 13.23 6.62 1.65 .20 Feedback (F) l 3.27 3.27 .82 .37 G X F 2 5.83 2.92 .73 .49 Error 54 216.00 4.00 2. Multivariate Test on Terminal Performance, Conditions, and Criteria Scores Sources df Multivariate F p less than Group (G) 6, 104 .68 .68 Feedback (F) 3, 52 1.66 .19 c x F 6, 104 .85 .53 74 TABLE 3.15. Univariate and Multivariate Tests for Treatment Groups and Feedback (Segment B): Total Scores and Terminal Performance, Conditions, and Criteria Scores 1. Univariate Test on Total Scores Source df SS MS F p less than Group (G) 2 37.20 18.60 4.10 .02 ‘Feedback (F) 1 .60 .60 .13 .72 G X F 2 14.80 7.40 1.59 .21 Error 54 251.40 4.66 2. Multivariate Test on Terminal Performance, Conditions, and Criteria Scores Sources df Multivariate F p less than Group (G) 6, 104 1.34 .25 Feedback (F) 3, 52 .64 .59 c x r 6, 104 .99 .43 TABLE 3.16. 75 Means of Total Scores on Segment B Comparisons Group 1 5.1 Group 2 3.3 Group 3 3.6 CHAPTER IV SUMMARY AND DISCUSSION Summary The purpose of this study was to evaluate an instructional model for teaching counselor trainees how to establish behavioral objectives when counseling with clients. Many counselor educators are advocating that counselor trainees be taught behavioral counseling skills. Little is being done, however, to evaluate instructional procedures to teach these new skills. This study proposed a new teaching model and evaluated its components through the use of two control groups. The subjects in this study were 60 M.A. degree candidates at Michigan State University. These subjects were randomly divided into three major treatment groups. The first group experienced the instruc- tional model which was of main interest. They read a general paper on objectives in counseling, a paper on behavioral objectives in counseling, and a paper describing a scale for recording counselor references to the three main components of a behavioral objective--terminal performance, conditions, and criteria. The second group read the general paper on objectives in counseling and also the paper on behavioral objectives in counseling, but instead of reading about the scale they reread the behavioral objectives in counseling paper as a control experience. The 76 77 third group read the general paper on objectives in counseling, but instead of reading about behavioral objectives in counseling, they read a paper on how to identify client concerns. While the first group was reading about the scale, the third group reread the paper about client concerns. The design described here enables an evaluation for each of the experiences encountered by the first group. After all three groups had read the papers allocated to them, they went through a training experience. Everyone heard four pairs of counseling interview segments on audiotape in which the counselor was attempting to establish behavioral objectives with the clients. In some cases, the counselor made several references to the components of a behavioral objective, and in others, he made only a few references. All subjects were asked to pick the "better" counseling performance in each pair. Group 1 was asked to use the scale which they had previously studied and to pick the interview that had the higher tally while Groups 2 and 3 were given no criteria on which to make a selection. Throughout the training phase of the experiment, each of the three groups was randomly divided into a feedback--no feedback mode. Half of the groups received feedback as to which was the better segment of each pair of counseling interview segments, and the other half received no feedback. The better segment in each pair was determined by which segment contained more counselor references to the components of a behavioral objective. Immediately after the training phase of the experiment, all subjects participated in an evaluation phase. In the evaluation, they 78 played the part of a counselor with a role-playing client. As the counselor, each subject was asked to establish an objective for counsel- ing. These role-played interviews were taped, and trained recorders listened to them and recorded the frequency of counselor references to the components of behavioral objectives. These frequencies were the major dependent variable in the study. It was hypothesized that the group which had exposure to and training with the scale would make more references to the components of a behavioral objective than the other two groups. It was also hypothesized that a group that had exposure to a paper describing the components of a behavioral objective would make more references to the components than a group that did not have the exposure. Finally, it was hypothesized that groups which received feedback during training as to which was the better counselor performance in establishing behavioral objectives would make more references to the components of a behavioral objective than those groups that received no feedback. None of the three main hypotheses was supported. There is no evidence, therefore, to indicate that differences existed among the groups after the treatment procedures were implemented. An unexpected finding indicated that the subjects, regardless of what treatment group they were in, made a different number of responses relative to two com- ponents of a behavioral objective depending on the type of problem that was presented to them. It was also found, but not predicted, that the group that experienced the instructional mode of main interest had a significantly greater variance than one of the control groups. 79 Discussion Although no significant differences were found relative to the three main hypotheses, it is useful to investigate some of the known aspects of the experiment in retrospect. It is possible that some of these aspects can clarify the results of this study and can suggest lines for future investigation. The following topics are discussed: theory, sample, treatments, instruments, evaluation, design, and statisti- cal treatmmu; The chapter ends with a conclusion and some recommendations for future studies. Theory It is possible that the outcome of this study was affected by an error in, or misinterpretation of, the literature. Determining whether there has been an error or misinterpretation is difficult, however, because there are few studies which are directly related. As stated previously, the main intent of this study was to evaluate an instructional model for teaching counselor trainees how to establish behavioral objectives in the counseling process. The literature contains several descriptions of different behavioral counseling programs, and there is no reason to believe that the model proposed in this study incorporates substantially different procedures from those that have been proposed and described. The present study made use of Mager's format for writing behavioral objectives. The Counseling Behavioral Objective Process (CBOP) Scale which was used in the treatment program and provided the basic dependent 80 variable for evaluation was built directly on the three parts of a behavioral objective as specified by Mager (1962). Advance organizers were a major feature of the instructional model. The concept of advance organizers was formulated and researched by David Ausubel, and he has consistently found that they facilitate learning (Ausubel, 1960; Ausubel & Fitzgerald, 1961, 1962). There is a difference between the type of advance organizer that Ausubel used, how- ever, and the type that was used in this study. Ausubel stressed that his advance organizers contained no information that was to be presented later in the main body of information to be learned. The CBOP Scale used in this experiment was directly related to the training phase and the later evaluation. Stating some contrasts between the type of research Ausubel has done and this experiment would be helpful. Ausubel has been interested primarily in the learning of written material through the mode of reading that can be evaluated with multiple-choice tests. The present experiment dealt with learning through the modes of reading and listening which were evaluated through the verbal analysis of a role—played situation. The subjects were required to demonstrate behaviorally what they had learned. The modes of learning and evaluation are not equivalent, and, therefore, the application of Ausubel's research to the present experiment can be questioned. Performance on multiple-choice tests and behavioral per- formance may not be significantly correlated. This point may explain some of the reason why Ausubel has consistently obtained significant differences, and none were obtained in this study. 81 In summary, it appears that this experiment was testing an instructional model which is closely related to models described in the literature. Additionally, the use of the behavioral objective concept seems to be based directly on what has been proposed in the literature. The type of advance organizer and the way it was used, however, may be a major source of error in the results. Easels Uncontrolled characteristics of the sample which was used in this experiment may have had an effect on the outcome. There are two different aspects to this consideration. First, there is the issue of how repre- sentative the sample was of the larger population to which it was intended that the results of this study could be generalized. Second, there is the issue of subject characteristics. Some subject characteristics may have had an effect powerful enough to overcome weaker treatment effects. It was the intention of the main experimenter that the results of this study could be generalized to the population of counselors-in- training at the M.A. degree level (Cornfield & Tukey, 1956). This population includes counselors who are training for work in elementary schools, secondary schools, community colleges, and rehabilitation agencies. As noted in Chapter II, 37 of the 60 subjects in the present experiment were training to be rehabilitation counselors. This fact is clearly a limitation on the generalizations that can be made regarding the sample. It can be stated in favor of the sample, however, that they all came from the population of interest and that at the time the 82 subjects were admitted to the Michigan State University program, there were no special selection criteria in favor of rehabilitation counselors in effect. It seems apparent when the factors stated here are considered along with the subject description presented in Chapter II, that the over-representation of rehabilitation counselors-in-training is the main difficulty regarding the representativeness of the sample. Otherwise, there seems to be no evidence which would indicate that the sample used in this study was a unique group. What special characteristics of the sample, both observed and hypothesized, might have led to the non-significant findings? Twenty-six out of the 60 subjects had earned teaching certificates, and of those that had no teaching certificate, 15 had majored in psychology. It is probably that a majority of the teachers and psychology majors were exposed to the concept of behavioral objectives. As stated previously, there is presently a great deal of interest in the use of behavioral objectives in the learning process (Duchastel & Merrill, 1973). If several of the subjects in the sample were already familiar with the idea of behavioral objectives, then the treatment that Group 1 was exposed to did not incorporate ideas that were new to the subjects. In effect, Group 2 and Group 3 had already been exposed to the major concept in the instructional model of interest, and this fact would decrease the probability of finding significant differences. In a future experiment dealing with the same issues, it would be advisable to control for the knowledge the subjects have regarding behavioral objectives. This plan 83 could be carried out by giving the subjects a test on behavioral objectives before the experiment. Another characteristic of the sampe which may have increased the probability of nonmsignificant results was the fact that all the subjects were in the Michigan State University Counseling Block Program. This program is a unified sequence of courses in which extensive use is made of specifically stated objectives for all the students (Winborn, Hinds, & Stewart, 1971). When a student enters a counseling course, he is typically told on the first day what is required in order to obtain a particular grade. Even though the concept of behavioral objectives had not been taught to the subjects before this experiment, it is possible that some may have learned it by induction. In summary, there are problems with the representativeness and unique characteristics of the sample. It is a possibility that these factors contributed to the size of the error variance. baggage When treatment procedures are poorly planned and carelessly administered, they introduce unexplainable variance into the analysis. This variance, of course, makes it more difficult to obtain significant differences. In this experiment, the treatments were highly standardized in terms of their presentation to the subjects. As described in Chapter 11, all subjects heard the same instructional tape. The various differences among the treatment groups were created solely by the composition of the programmed booklets that subjects read while listening to the instructional 84 tape. The subjects went through the treatments in experimental sections of about six subjects each. All experimental sections were administered by an assistant to the main experimenter. Other than asking the subjects to spread themselves out in the room and giving them specially desig- nated treatment booklets, she gave them no directions. All directions the subjects needed for the experiment were on the instructional tape. It is evident that the treatment sessions were highly standardized and consistent across the 60 subjects. It is useful, however, to con- sider factors within the treatments that might have contributed to the error variance. In any experiment of this type that involves reading and concen- tration on the part of the subjects, there is the question of how actively they participated in the procedures. It is possible that some subjects did not read the material carefully or misunderstood what was being asked. Three different cases in Group 1 were identified in which a subject did not seem to follow the specified procedures. In one case, it did not appear that the subject used the CBOP Scale during the training phase. The subjects were asked to make tallies on a specially prepared form any time they perceived that the counselor made a reference to one of the components of a behavioral objective. This subject had no tallies for any of the eight interview segments that were played. A second subject seemed to use the CBOP Scale properly during the training phase, but when it came to the role-play evaluation she did not make any verbal references to the components of a behavioral objective. Both of the trained recorders gave this subject zero tallies for all categories. It 85 seems probable that she misunderstood the purpose of the role—play evaluation. A third subject fell asleep during the first two taped segments in the training phase. The assistant experimenter awoke this subject and reminded her that participation in the experiment was volun- tary. After this reminder, she participated in the rest of the experi- ment. The above three examples are the only instances that came to the attention of the main experimenter in which subjects were probably not following the experimental procedures. The case of the subject falling asleep during the training phase of the experiment suggests another possible difficulty with the treat- ment procedures. The instructional tape was approximately 45 minutes in length, and much of it consisted of counseling interview segments. It has been the main experimenter's experience that counselor trainees who are not specially motivated soon become bored when asked to listen to audiotaped counseling interviews for an extended period of time. This problem would be especially evident in the case of Group 2 and Group 3 when they were asked to reread certain materials. In addition, Group 2 and Group 3 did not have any specific task to perform while listening to the counseling interview segments. In a few instances, after the experiment, some subjects did comment to the main experimenter about being bored while listening to the instructional tape. Motivation on the part of the subjects could also be questioned. In the student consent form that all the subjects signed before the experiment (Appendix B), it was clearly stated that participation or lack of it in the experiment in no way affected the grade for the course. 86 Stress was laid on the benefits to be gained by participating, but in the final analysis, participation was strictly voluntary with no consequences attached to careless or to unmotivated participation. Another factor relative to the treatment procedures that might have contributed to the unpredicted results has to do with the concept of modeling. It is possible that the key experience in the treatment program was the mere listening to the eight counseling interview seg— ments. For many subjects, it might have been obvious as to which counseling performances were superior. The interview segments were similar in format except for some in which the counselor made several references to the components of behavioral objectives, and in others very few references were made. It may have been possible to distinguish between these two types of interviews just by listening to them without any specified criteria. Counselors who made several references to ther components of behavioral objectives as compared to counselors who made only a few may have sounded to counselor trainees as though they were doing a superior job. Thus the subjects may have been presented with a model of superior counseling, through the four pairs of counseling inter— view segments without the additional learning aids of advance organizer, paper on behavioral objectives in counseling, or feedback within training. There are some issues with the Group 1 treatment specifically that should be considered. Did the subjects in Group I apply what they learned in the training phase to the role-play evaluation? It is possible that a given subject understood the use of the CBOP Scale and was 87 motivated during the training phase, but when he or she came to the evaluation, no transference was made. As described above, this case seems to have occurred with one of the subjects in Group 1 who made no behavioral objective references in the role-play evaluation. In order to promote this transference as much as possible, it was stressed to the subjects that the training procedures were aimed at setting objectives for counseling. In addition, just before the subjects went into the role-play evaluation, they were again reminded that the focus was on setting objectives. The first hypothesis of this study had to do with looking for significant differences between Group 1 and the other two control groups. No significant differences were found among the three groups, but unexpectedly it was determined that Group 1 had a significantly larger variance than Group 3 relative to the total scores. These findings are described in Chapter 111. It is of interest to look closely at the anslysis results dealing with the first hypothesis in graphical form (see Figure 3.1, page 48). Keeping in mind that significant differences cannot be implied, the graph gives a possible indication of trends. Note how close the Group I and Group 2 scores are to each other. These are the two groups that had access to the paper on behavioral objectives in counseling. All three scores on Group 3 are below those of the first two groups. Does this graph represent some significant treatment effect which at present is being covered by unexplained variance? Possibly future studies of this nature can answer the question. 88 Instruments The reliability of the instruments used in any experimental study has impact depending upon the precision with which they can be inter- preted. The instrument of main interest in this experiment--The Counselw ing Behavioral Objective Process (CBOP) Scale--was developed by the main experimenter. Its major function is as a recording device to tally the number of references a counselor makes to the components of a behavioral objective. These references can be counselor statements, questions, or summaries. The three main components are terminal performance, condi— tions, and criteria. (For further details regarding the CBOP Scale, see the section on instrumentation in Chapter 11.) It would seem that questions regarding the reliability of the scale would have to do with the reliability coefficients of the tallies made by individuals who are presumably expert in identifying counselor references to the components of behavioral objectives. The CBOP Scale was developed especially for this experiment so there are no data regarding its reliability except that of the two trained recorders. Their overall correlation of inter- rater reliability relative to identifying any behavioral objective components was over .85, and their weakest correlation on any one component was .66. This evidence seems to indicate that a fairly high degree of reliability between recorders can be obtained with the instrument. The possible exception is in regard to the conditions component where the recorders obtained the .66 correlation. Part of the conclusion section of this chapter deals with the problems encountered with the conditions component. 89 Probably the single factor which most prevents very high relia- bility is not determing what type of statement the counselor makes, but agreeing among recorders as to what constitutes a counselor statement. (See the directions for the use of the CBOP Scale in Appendix H.) The main experimenter arbitrarily determined that one counselor statement consisted of anything a counselor said between any two client statements. Even with this definition, it was relatively difficult to tally the number of counselor statements in a counseling interview. The content validity of the scale may be examined by reading the paper on the use of behavioral objectives in counseling (Appendix E). It can be seen that the CBOP Scale is an application of the generally straight forward ideas in this paper having to do with terminal per— formance, conditions, and criteria. The CBOP Scale was used essentially in two different ways in the experiment. First, it was used as an advance organizer for Group 1 in the training phase of the experiment. It was the most basic component of the training model being evaluated. If the scale was too difficult to understand in the time the subjects were given to study it, then the hypothesized advantage that Group 1 had in using it would have been minimized. This possibility is another explanation for the non- significant findings. The second manner in which the scale was used in the experiment was in regard to evaluating the subject's performance in the role-play evaluation. To the degree that the recorders could net achieve perfect reliability in their tallies, an error variance was introduced into the analysis. 90 Evaluation There were two main evaluation phases in the experiment. The role-play evaluation took place immediately after the treatment phase when the subjects played the part of a counselor talking with a roleu playing client. The tape evaluation took place after the role—play evaluation. During this time period, the subjects listened to two taped counseling interview segments. The subjects evaluated the counselor's performance using the CBOP Scale. It is possible that situations or events in these evaluations might have contributed to the lack of treatment differences. Some of the unexplained variance that occurred in the role-play evaluation might have been due to the unique characteristics of the 18 role-players. Role—player characteristics such as sex, age, and physical appearance were not taken into consideration in the design of the experi- ment. In addition, some role-players were used more than once. The role-players were assigned to the role—play evaluation sessions relative to their availability. It should be noted, however, that the main experimenter made a serious attempt to standardize the role—play presentations as much as possible in spite of the previously discussed problems (Appendix N). There is clear evidence in the experiment that the controlled aspects of client behavior had an effect on the behavior of the subjects. There were unexpected significant differences in the number of references the subjects made to the components of a behavioral objective depending on whether the client was talking about leaving the teaching profession 91 or decreasing the frequency of his drinking. The actual experimental results in this area are confusing. On the teaching problem, the subjects made more references to the criteria component as opposed to the condi- tion component. The reverse was the case regarding the drinking problem. Any speculation as to the reason for this result would necessarily be very tentative. After the role-play evaluation, the subjects went through a tape evaluation where they listened to two counseling interview segments called segments A and B. The intention of employing this procedure was to com- pare the CBOP Scale recordings of the subjects with trained recorders. On segment A, no significant comparisons were found, or in other words, none of the three main treatment groups compared more favorably with the trained recorders than any of the other groups. In contrast on segment B, results were found which seem to indicate that Group 2 recorded more consistently with the trained recorders than did Group 1. Again, these results were unexpected. Group 1 had previous training with the scale during the training phase of the experiment, and it was expected that . their recordings would have the highest agreement with the trained recorders. It can be stated, however, that Group 2 was exposed to the materials dealing with behavioral objectives in counseling before they listened to the tape evaluation segments. It is worth noting here that in both the role-play evaluation and tape evaluation the content of the problems presented by clients seemed to have an effect on the behavior of the counselor subjects. In the first case, depending on which client problem was presented, certain 92 subjects made more references to some of the components of behavioral objectives. In the second case, depending on which client problem was presented, certain subjects were able to obtain a higher degree of agreement with trained recorders when compared with other subjects. Both of these results seem to point to the issue that Heller, Myers, and Kline (1963) have researched. They investigated the effect that different client behaviors have on counselor behaviors and found a definite relationship between the two. Design and Statistical Treatment The lack of significant differences between the main treatment groups may be due to a poor experimental design or to an imprecise analysis of the data. The design question is considered first. Overall, there are no major flaws in the design of the experiment. The basic intent of the design was to control for the various parts of the instructional model, and this task was performed by the two control groups. There were two aspects of the role-play evaluation segment which probably tended to introduce unwanted error variance. The first problem was the variability of the role-players. As stated before, with time limitations and so many people involved, it was impossible to have the role-players extensively trained. The second problem had to do with the presentation of two different roles in the role-play evaluation. Unexpected significant differences were found in the performances of the subjects depending on which role they encountered. A better procedure to follow would have been to present the same role for all subjects on both days. The two 93 different role-played problems added uncontrolled variance to the main treatment effects. The major portion of the analysis was done using the analysis of variance approach. Univariate and multivariate analysis of variance procedures were appropriate for use with the behavioral objective com— ponent total scores and individual scores, respectively. Conclusion, The purpose of this experiment was to evaluate an instructional model for teaching counselor trainees how to establish behavioral objectives with their clients. The design of the experiment was organized so that individual parts of the model could be evaluated in order to find out what parts were critical in facilitating the learning of the subjects. No significant differences were found between the performances of the subjects who experienced the instructional model of interest and the subjects who experienced control learning models. The data provide no evidence to indicate which aspects of the main instructional model are critical in the facilitation of learning of the subjects. The remaining topics to be covered in this conclusion section can be grouped under four headings: (a) significant differences between group variances, (b) problems with the criteria component, (c) suggestions for counselor training models, and (d) suggestions for future studies. Significant Differences Between Group Variances Possibly the most interesting finding of this study was the significantly larger variance on the number of references to the components 94 of a behavioral objective made by the subjects in the main instructional group as compared to the subjects in the third control group. This result indicates that some of the subjects were helped by the instructional model, and others were possibly hindered by it. Future studies would be useful in determining the specific characteristics of students that are indicators of whether a particular instructional model will help or hinder learning. Problems With the Conditions Component The conditions component of a behavioral objective unexpectedly presented difficulties in this experiment. As noted previously, the two trained recorders obtained a correlation of at least .80 on the terminal performance, criteria, and total scores. On the conditions component, their correlation was .66. The subjects also appeared to have difficulty understanding the conditions component. As indicated by Figure 3.1 and Table 3.2 (pages 48 and 49, respectively), it appears that the subjects made fewer references to the conditions component than to either of the other two components. In order to see if the subjects made significantly fewer references to the conditions component, two t tests with correlated data were performed. When the number of terminal performance references was compared with the number of conditions references, a t value of 11.92 was obtained. A t value of -4.13 was also obtained for a similar comparison between conditions references and criteria references. Using a two-tailed test at the .05 level with 59 degrees of freedom, the critical values are 2.00 and —2.00. Both of the t values reported above are clearly in the rejection regions. It is, 95 therefore, evident that the subjects made significantly fewer references to conditions than to terminal performance or to criterial No research has been done which might suggest the reason why the subjects made significantly fewer references to the conditions component. Two possible hypotheses can be suggested. First, it is possible that by the way in which the three components are defined, the conditions factor requires fewer references when it is being specified as compared to the other two. There is no evidence to support this hypothesis, however. A second alternative is more likely. It is possible that the conditions component is more difficult to understand. Perhaps the sub- jects were not able to understand fully the concept in the time allowed and, therefore, then made fewer references to conditions and referred more frequently to the components they understood. If the conditions component is more ambiguous and more difficult to learn, then future instructional models should_take this fact-into consideration. Possibly more time should be spent explaining the condi- tions component in addition to the presentation of more examples. Suggestions for Counselor Training Models If progress is to be made in the training of counselors, better instructional models will have to be developed and evaluated. In many cases, models are developed in a haphazard manner with little regard to the principles of efficient learning procedures and then are seldom carefully evaluated. There is a need for more objective evidence regard- ing the best manner in which to teach the many new counseling skills that are being proposed. 96 It is worth noting at this point an evaluation weakness of this study. In the final analysis, the most important evaluation of any instructional model must rest with how much the client was helped as a result of what the counselor learned. In terms of this study, the most important evaluation would deal with whether or not the establishing of behavioral objectives helps clients solve their problems. Unfortun- ately, the major emphasis in this study was on the simple question of whether or not the counselor attempted to establish a behavioral objective. A final point can be made regarding counselor training models and this study. A central idea of this study was to teach counselor trainees how to use the evaluation instrument with which their performance would be evaluated. This method seems especially useful in terms of helping the students to focus on what they are to learn. This study centered on the skill of establishing behavioral objectives. With the teaching of different counseling skills, other evaluation instruments such as the CBOP Scale would be developed. Suggestions for Future Studies With regard to what was learned in the present study, there are some suggestions for improving future studies. (a) The subject's knowledge of behavioral objectives should be assessed before the learning phase. This assessment could be accomplished through the use of a multiple-choice test. (b) The subjects should be motivated to learn the concepts being presented. In a typical setting, motivation is facilitated through the use of grade assignments. (c) The time it takes individual 97 subjects to learn specific concepts would be worth investigating. It is likely, as the present study has indicated, that learning about the conditions of a behavioral objective takes longer than learning about terminal performance. (d) Care should be taken to make the learning experience as interesting as possible. Subject boredom was a problem in the present study, and this factor introduced uncontrolled error into the results. (e) Finally, further analysis of the differences between discrimination and performance evaluations should be made. The results of this experiment were inconclusive regarding the comparison between the subjects' cognitive understanding of the scale and their performance with it. It is possible that a student may learn a concept well enough to discriminate it on a paper-and-pencil test, but not be able to employ the concept in his own performance. (Ausubel's experiments were based primarily on discrimination evaluations through the use of paper-and- pencil tests.) In terms of counseling skills, it is imperative that counselors be able to "perform" based on cognitive discriminations. APPEN DI CE S APPENDIX A COMMENTS MADE TO SUBJECTS BY MAIN EXPERIMENTER ONE WEEK BEFORE THE EXPERIMENT COMMENTS MADE TO SUBJECTS BY MAIN EXPERIMENTER ONE WEEK BEFORE THE EXPERIMENT All subjects in the experiment came from two classes of Master's level counseling students who were meeting regularly during Winter Term on the Michigan State University campus. The main experimenter was the instructor for one of these classes. The main experimenter went to both classes on February 13, 1973. This date was one week before the experi- ment was scheduled to take place. The following list of points was covered by the experimenter, and time was provided in which the students could ask questions: (1) An evaluation will take place during class time next Tuesday and Thursday, February 20 and 22. (2) The instructional evaluation is to be a part of the experi- menter's Doctoral dissertation. (3) The students will take part in an evaluation of different instructional methods relative to teaching counselors how to establish objectives in the counseling process. The experimenter will be trying to find out which instructional method is most effective. (4) In the evaluation, all the students will follow the same general format. They will (a) read some instructional materials, (b) listen to some counseling interview segments, 98 99 (c) counsel a role-player for a short lO- to lS-minute interview which will be audiotape recorded, and (d) evaluate a counselor‘s performance on a short audiotape interview segment. (5) During the evaluation, programmed booklets, tape recorded instructions, and other restrictive features will be used. These features will be employed in order to control the situation so the different teaching methods can be closely evaluated. (6) The evaluation experience will not be related to the student's grade for the course. (7) The students will be free to participate or not to participate. (8) After the students have been through the evaluation, they will be requested not to discuss it with others. (9) While going through the actual evaluation, the students will be requested not to talk or to behave in any way that might disturb the concentration of others. (10) Students will report for the evaluation of their regular class according to a special schedule during the experiment week. The main experimenter handed out the Student Consent Forms and read it to the students. He requested their signdtures and collected them. APPENDIX B STUDENT CONSENT FORM STUDENT CONSENT FORM Date The evaluation project regarding setting counseling objectives has been explained to me and I understand its conditions. I will participate in the evaluation project of my own free will. My grade in the course will not be affected by my participation or lack of it. I understand that I am free to discontinue my participation in the project at any time. I understand that all the data collected in this project will be treated with strict confidence and that I will have free access to the results of the project after the evaluation is completed. I understand that no beneficial effects are guaranteed for me and that I will be randomly selected and placed in one of six different learning groups. ' I understand that any aspects of the project that are not explained before it is carried out will be explained to me upon its completion. Signed 100 APPENDIX C DIRECTIONS GIVEN TO EACH SUBJECT BEFORE THE ROLE-PLAYING SESSION DIRECTIONS GIVEN TO EACH SUBJECT BEFORE THE ROLE-PLAYING SESSION Read the following material and then go to room in the 250 area. There you will meet a role-player who will talk with you about his/her unhappiness with being a teacher. The role—play session should last about ten minutes. Remember that the focus of this experience is on setting objectives for counseling. Please remember not to talk to any other members of the class on the way to your role-play room or after the experience is over. Leave this booklet in the room you are now in. As a reminder, rip off the right bottom corner of this page which has your room location on it. lOl APPENDIX D DESCRIPTIONS OF THE TWO ROLES THAT WERE PRESENTED TO THE SUBJECTS DESCRIPTIONS OF THE TWO ROLES THAT WERE PRESENTED TO THE SUBJECTS DESCRIPTION OF ROLE-PLAY The student coming into the room will have been told that they are to talk with someone who is dissatisfied with his/her school teaching job. Your role will be as follows: You have taught mathematics in a local high school for six years and you are now unhappy with the job. Preparing lesson plans and coping with discipline problems in the classroom are things you would rather forget. You're strongly considering quiting teaching and becoming a real estate salesman/saleswoman. Your most basic problem now is to make a decision whether or not to stay in teaching before school contracts come out in May. The main condition on this decision is that you must be able to earn at least $8,000.00 a year to support your family. Use this role for all role-plays on Tuesday. 103 104 DESCRIPTION OF ROLE—PLAY The student coming into the room will have been told that they are to talk with someone who has a drinking problem. Your role will be as follows: You work hard at the office all day and when you come home you always have a drink. The problem is that the first drink always leads to another and before long you are in a stupor for the rest of the evening. This is affecting your work and it looks like you may not get an upcoming promotion unless you limit the drinking. What you really want to be able to do is to have no more than two drinks on any evening after work. You need to reach this level within a month or you can consider the promotion lost. The main condition or situation that is relevant to the problem is that it takes place in your home after a day's work. Use this role for all role-plays on Thursday. APPENDIX E BEHAVIORAL OBJ ECT IVES FOR COUNSELING BEHAVIORAL OBJECTIVES FOR COUNSELING PLEASE READ THE FOLLOWING STATEMENT VERY CAREFULLY. IF YOU FINISH READING BEFORE YOU ARE INSTRUCTED TO GO ON, REREAD THE STATEMENT TO MAKE SURE YOU UNDERSTAND THE BASIC IDEAS EXPRESSED. An outcome objective represents the established goal that the client and counselor will work toward. Its focus is upon the identified concern of the client and a specification of his desired behavior as a result of counseling. Although objectives are individually established, all have commonly defined parts. The discussion and exercises that follow will prepare the learner for the task of establishing objectives with clients. In a discussion of behavioral objectives for the instructional process, Mager (1962) presented the following as characteristics of well-written objectives: 1. An identification of the terminal performance by name. 2. A description of the conditions under which the behavior is to occur. 3. A specification of the criterion of minimum acceptable performance. TERMINAL PERFORMANCE Terminal performance as used in a counseling objective, is a specific statement that describes what a client will be doing as the result of counseling. A number of terminal performances might be sought by a client who has difficulty in establishing a social relationship with girls. One client might complain of stuttering caused by extreme anxiety. Another might report no particular anxiety, but a general lack of skill in conversing and appearing attentive. A third might identify his own aggressive acts toward girls as being detrimental to his interest. In each instance a different outcome behavior could be identified. Specificity is the key factor. Statements such as "Making him aware of his behavior" and "Helping him feel comfortable" are of no assistance. Both are vague and do not specify the behavior that is to be altered. The following are some statements of outcome behavior that are specific 105 106 and indicate the exact change that the client wishes to make in his behavior: Increase the frequency of contacts with girls. Discriminate between insensitive and hostile remarks. Decrease the number of self references in conversation. Respond calmly. . Increase the length of contact with a girl. Decrease the number of times that the girl must break a silence. O‘kn-L‘bJNr—I CONDITIONS Conditions, as used in counseling objectives, take on a variety of forms. In some instances "conditions" might refer to the specifying of environmental situations. Examples: 1. When taking a test 2. Given a social situation . 3. The next time the teacher reprimands you . In other instances "conditions" might refer to a state with the person. Examples: 1. When you are faced with an important decision . 2. When someone makes you angry . . . 3. The next time that you feel blamed In either instance "conditions" refers to the state in which the client finds himself at the time that the outcome behavior is expected to occur. CRITERIA In addition to specifying what the client will do and under what condi- tions, it is necessary to state how well he is expected to do it. In counseling objectives this may take a number of forms, which are presented below with illustrations: Form Illustration Specific time period . . . for twenty minutes daily. . once daily for two weeks. 107 Form Illustration Specific degree . . . on four of five occasions. . with less than a 10 percent error rate. Task accomplishment . . . until the project is completed. . until the instructor approves. Change of personal state . . . without crying. . while remaining relaxed. The statement of counseling objectives can be simplified if the conditions (if any) are stated first, then the terminal performance, and finally the criteria. Presume that a sophomore girl reports that she has no friends in school and would like to make friends. She believes that it would be easiest to do this on the school bus which she takes twice daily. The objective, by parts, might be: Conditions When (you are) riding on the bus. Terminal behavior . . . begin and continue a conversa- tion with another student Criteria . . . so well that you can recall his name, address, classes, and two interests or hobbies. Collectively, the counseling objective would be: When riding on the bus, begin and continue a conversation with another student so well that you can recall his name, address, classes, and two interests or hobbies. Another example of an objective would be: Given that there are at least three lathe operator openings in Detroit the client will call two companies over the phone before next Saturday and find out what is required when applying for these jobs. DO NOT TURN THE PAGE WAIT FOR INSTRUCTIONS BEFORE YOU TURN THE PAGE. IF YOU FINISH BEFORE BEING INSTRUCTED TO GO ON, PLEASE REREAD THE ABOVE MATERIAL. APPENDIX F COUNSELING OBJECTIVES COUNSELING OBJECTIVES PLEASE READ THE FOLLOWING STATEMENT VERY CAREFULLY. IF YOU FINISH READING BEFORE YOU ARE INSTRUCTED TO GO ON, REREAD THE STATEMENT TO MAKE SURE YOU UNDERSTAND THE BASIC IDEAS EXPRESSED. Almost all counselors agree that objectives are an important part of counseling. Not all agree, however, on where the emphasis regarding objectives or goals should be placed. Some feel that objectives for the counselor are the most important and should be emphasized. Others feel that objectives for the client are the most important. In this learning experience we are not interested in the debate for whom objectives should be set. We have arbitrarily chosen to consider objectives or goals that are for the client and his or her life experiences. A client objective for counseling is a statement describing what the client wants things to be like when counseling is finished. The basic purpose of counseling is for the client and counselor to work together to help the client reach the goal or objective. Objectives for counseling are typically determined by the client and counselor early in the counseling process. As counseling proceeds, they help to provide direction to the process. At some point near the end' of counseling, it is natural to see whether the client has reached the predetermined objective. One means of determining if counseling has been successful is to see if the objectives set at the beginning of counsel- ing have been met at the end of counseling. The main emphasis of the learning program you are now in has to do with objectives for counseling. Please keep this fact in mind throughout the experience. D O N O T T U R N T H E P A G E WAIT FOR INSTRUCTIONS BEFORE TURNING THE PAGE. IF YOU FINISH BEFORE BEING INSTRUCTED TO GO ON, PLEASE REREAD THE PREVIOUS MATERIAL. 108 APPENDIX G TWO—PART PAPER ON "IDENTIFY CONCERN" AND "SELECT CONCERN FOR COUNSELING" TWO-PART PAPER ON "IDENTIFY CONCERN" AND "SELECT CONCERN FOR COUNSELING" PLEASE READ THE FOLLOWING STATEMENT VERY CAREFULLY. IF YOU FINISH READING BEFORE YOU ARE INSTRUCTED TO GO ON, REREAD THE STATEMENT TO MAKE SURE YOU UNDERSTAND THE BASIC IDEAS EXPRESSED. A. Identify Concerns One of the most necessary skills a counselor must possess is the ability to identify a client's important concerns. The counselor must be able to hear what a client is really saying and what actually bothers him or her in order to assist the client with the concerns. Distinguishing significant from insignificant material in a client's presentation is a subtle skill. Several clues to remember while determining important concerns are as follows: I. The reason the client gives for seeking counseling (the "present- ing problem") frequently is not the real reason or greatest concern(s) the client has. The counselor, therefore, should listen_garefully and explore numerous topics with the client before narrowing discussion to one concern. It is sometimes impossible for a counselor to determine the client's specific problem(s) immediately. The client will frequently present a large amount of material at the beginning of counseling. The counselor must constantly be sorting important from unimportant material. The counselor should not hesitate to question the client in order to determine the importance of a topic. For example, the counselor could ask, "Has your lack of friends been the reason you came to our office?" or "You want me to assist you in career planning. Is that correct?" The counselor may find it helpful to think of important careers as falling_into categories. The counselor may use any categorical system which is useful to him or her. For example, some coun- selors view concerns as either personal, educational, vocational or social, while other counselors might categorize concerns as those involving other people or those involving only the client. 109 110 5. The counselor should determine the client's important concerns by examining the preoccupations of the client and the strength of the emotional significance attached to a topic by the client. Emotional significance may be indicated by: a. the client's speech changing at the moment he discusses a particular topic--becoming higher or lower pitched, more rapid or slower in speed. b. the frequency the client talks about a topic--either an obvious avoidance of a topic or constantly returning to a particular topic. c. changes in posture, facial expression, body movements or eye contact whenever the client talks about the topic. In other words, any significant change from the usual behavior of the client is likely to indicate the importance of a topic. Select Concern for Counseling Once a concern has been identified for immediate consideration, the counselor has to direct the client's verbalizations so that they focus on the concern. The most effective use of interview time occurs when the client is assisted to talk at length about a limited number of topics. Because he forgets, wishes to avoid further dis- cussion, becomes anxious or merely wanders, however, the client does not always continue to explore the identified concern as the counselor might wish. Only through a continued attempt to focus on the concern does the counselor bring the client back to the central theme each time that he strays or rambles. Some of the following might be helpful leads for the counselor: "You were telling me about why your father doesn't understand you . . ." "Concerning your difficulty in meeting strangers, you were saying . . ." "The big campus . . . what is it that bothers you about it . . .?" "Lonely . . . you were saying that you felt very lonely.” A client's verbalization at any moment has both content and feeling. Both may prove useful in helping the client to focus on his concern. Content refers to the objects, actions, or information that the client refers to in his verbalization. His father, his car, his late registration, his refusal to participate, his comments to a friend, and his knowledge of his choices all refer to content. 111 However, they may be less important than the essential attitudes (feeling) expressed by the client concerning the content and what it means to him. Example: "I drove from Detroit last night and nothing happened." Content refers to the act of driving from a given point at a given time of day without incident. In the case of most adults, this particular comment seems rather uneventful. Why, then, did the client make it? Does it have particular psychological significance for him? Perhaps it indicates that he has just done something that he did not feel that he could do. The counselor would wish to focus upon whether the client would wish to elaborate. The counselor might respond by using one of the following responses: "That sort of surprised you . . . like, what might have happened?" ”Yeah . . . something could have, but.it didn't." "Nothing happened . . . I'm not sure I understand." As the counselor attempts to help the client focus on his concern, he may wish to pick up content or feeling. Some examples: "Perhaps we could get back to your problem with assignments." (Content) "Earlier you mentioned that you didn't know any sources." (Content) "And now it's hard to even talk about taking an exam." (Feeling) "You said you would keep the appointment, but it makes you angry." (Feeling) The presence of both feeling and content can be readily seen in the final two leads. A bit of content is presented (exam or appointment), but the underlying feeling ("hard to talk" or "makes you angry") is emphasized. Use of this type of response not only returns the client to the selected concern, but involves him more deeply by tying in his feeling or attitude about what is being said. A feeling response helps to push the client further toward self-exploration. DO NOT TURN THE PAGE WAIT FOR INSTRUCTIONS BEFORE YOU TURN THE PAGE. IF YOU FINISH BEFORE BEING INSTRUCTED TO GO ON, PLEASE REREAD THE ABOVE MATERIAL. APPENDIX H COUNSELING BEHAVIORAL OBJECTIVE PROCESS SCALE AND DIRECTIONS FOR ITS USE COUNSELING BEHAVIORAL OBJECTIVE PROCESS SCALE AND DIRECTIONS FOR ITS USE PLEASE READ THE FOLLOWING STATEMENT VERY CAREFULLY. IF YOU FINISH READING BEFORE YOU ARE INSTRUCTED TO GO ON, REREAD THE STATEMENT TO MAKE SURE YOU UNDERSTAND THE BASIC IDEAS EXPRESSED. Directions: Listen to the counselor in the audiotaped interactions that will be played. When using this scale, evaluate only the statements made by the counselor. Listen for three different categories of counselor statements. These three categories are (1) Terminal Performance (2) Conditions (3) Criteria Note how many times the counselor makes any kind of statement relative to each of the three categories. In this study, a counselor statement may consist of one or several consecutive remarks or questions. Count one statement as everything said by the counselor in between client responses. When you hear a counselor statement, determine whether there are any references in it to any of the categories. If there are such references, make tallies on the tally form which is described below. One counselor statement may produce three tallies, one in each category, if all three categories are mentioned. It might, likewise, produce two tallies in two of the three categories if only two categories are mentioned. Similarly, make one tally for one category or no tallies if no categories are referred to. Make a tally relative to one of the three categories if the counselor makes any kind of reference to the category. A reference to terminal performance, for example, might be a question that leads the client in the direction of terminal performance, a statement specifying terminal performance more clearly or a general summary statement relative to terminal performance. The following interview segment is an example of how to count counselor statements: ' 112 113 Client: I'm afraid I will not be able to find a job. STATEMENT Counselor: It's a scarey feeling. . . . You're really :3 1 wondering what to do now.- Client: I just don't know what to do. Counselor: What would it be reasonable to say is the STATEMENT most basic thing you want to get out of 2 counseling? Client: Well, getting a job. Counselor: Okay then . . . could we say that getting a STATEMENT job is our basic goal? . . . In other words, 3 that's what we would be working toward. Client: Yeah, but I have no idea of how to go about it. Counselor: Before we discuss the "how," I'd like to make sure our basrc task is set . . . the thing STATEMENT we re aiming for IS to get you a job . 4 Now . . . when do you feel you need this job by? What time limits do we have? Client: Well . . . I pretty well have to have one by the end of the month. In the above short interview segment, there were three terminal perform- ance statements (Statements 2, 3, and 4) and ppg criteria statement (Statement 4). Note that we are not interested in individual sentences, but in whatever the counselor says in between client responses. (If the counselor says anything in regard to one of the three categories, make a tally for that category.) For each taped interview segment you hear, there will be a space provided where you can keep a tally. It will look like the following diagram: Terminal Performance: 1 2 3 4 5 6 7 8 9 10 Conditions: 1 2 3 4 S 6 7 8 9 IO Criteria: 1 2 3 4 5 6 7 8 9 10 In the example given above, there were three terminal performance state- ments, one criteria statement and pg conditions statement. The tally form for this segment would look as follows: Terminal Performance: 1 2 B 4 5 6 7 8 9 10 Conditions: I 2 3 4 5 6 7 8 9 IO Criteria: ‘ l 2 3 4 5 6 7 8 9 10 Important: The total score for a given interview segment is the sum of the numbers of statements in all three categories. In the above example, the total score is four. (3+O+l=4) D O N O T T U R N T H E P A‘G E WAIT FOR INSTRUCTIONS BEFORE TURNING THE PAGE. IF YOU FINISH BEFORE BEING INSTRUCTED TO GO ON, PLEASE REREAD THE PREVIOUS MATERIAL. APPENDIX I CBOP SCALE FORM USED BY TWO TRAINED RECORDERS CBOP SCALE FORM USED BY TWO TRAINED RECORDERS Subject Number Rater Terminal Performance Conditions Criteria TP Total CO Total CR Total 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 1 2 3 4 5 6 7 8 9 IO 11 12 13 14 15 16 17 18 19 20 l 2 3 4 5 6 7 8 9 10 ll 12 13 14 15 16 17 l8 19 20 114 APPENDIX L TAPE EVALUATION SESSION DIRECTIONS FOR SUBJECTS FOR THURSDAY (NO—DELAY GROUP) TAPE EVALUATION SESSION DIRECTIONS FOR SUBJECTS FOR TUESDAY (DELAY GROUP) Read the following material and then go to room in the 250 area. Wait there for approximately 10 minutes until a role-player comes. Then the role-player will talk with you about his/her unhappiness with being a teacher. The role-play session should last about ten minutes. Remember that the focus of this experience is on setting objectives for counseling. Please remember not to talk to any other members of the class on the way to your role-play room or after the experience is over. Leave this cooklet in the room you are now in. As a reminder, rip off the right bottom corner of this page which has your room location on it. 116 APPENDIX N GENERAL GUIDELINES FOR ROLE-PLAYERS (l) (2) (3) (4) (5) (6) (7) (8) (9) (10) (ll) (12) (13) (14) GENERAL GUIDELINES FOR ROLE-PLAYERS. For this role-play, you should be in room at A student will come in shortly after the above time. The student's name should be Check to make sure you are role-playing for the right student. See attached sheet for a description of the role. Guidelines for role play. (I am interested in what the counselors say to you and not what you say.) (a) Be an agreeable client. (b) Do not lead counselor or present difficult problems. (c) If the counselor asks questions, freely respond using the information on the attached sheet or anything you can add to it. (d) Do not dominate the session-—Make your answers or comments fairly short and to the point. (e) Try not to behave in such a way as to increase the counselor's anxiety. A (f) Do not feel that you have to use all the information in the role-play description. Let the counselor draw the information out. The tape recorder and microphone should be all set up when you come into the room. When the interview is to start you push the switch on the microphone. The recorder will already be set with the tape and will be in the record position. Stop the session after approximately ten minutes by pushing_the stop button on the tape recorder. (Not the stop on the microphone.) The tape has only 15 minutes on a side. After you have stopped the role-play, take the tape out of the tape recorder and take it with you. ’ Give the student the programmed booklet and the cassette tape that will already be in the room. (Tell the student to read the booklet and then play the tape. Put the tape in the recorder and show the student which button to push to start it.) Leave the room and allow the student to listen to the tape. Now go to room where will be waiting for you. Follow the same procedure as outlined above. Before you leave, please give to me the two tapes that you have made. 119 APPENDIX 0 CBOP SCALE RECORDING FORM USED BY GROUP 1 DURING THE TRAINING PHASE OF THE EXPERIMENT CBOP SCALE RECORDING FORM USED BY GROUP 1 DURING THE TRAINING PHASE OF THE EXPERIMENT Now listen to pair number Rate each segment as you listen to it. Segment A: Terminal Performance: 1 2 3 4 S 6 7 8 9 10 Conditions: 1 2 3 4 5 6 7 8 9 10 Criteria: 1 2 3 4 S 6 7 8 9 10 Segment A Total Score (The total number of checks for the three categories) Segment B: Terminal Performance: 1 2 3 4 5 6 7 8 9 10 Conditions: 1 2 3 4 5 6 7 8 9 IO Criteria: 1 2 3 4 5 6 7 8 9 10 Segment B Total Score (The total number of checks for the three categories) Circle the segment below (A or B) you think is better based on which one you gave the higher Total Score. A B D O N O T T U R N T H E P A G E U N T I L Y O U A R E T O L D T O D O S O 120 LIST OF REFERENCES LIST OF REFERENCES Atkin, J. M. Behavioral objectives in curriculum design: A cautionary note. In R. E. Anderson, G. W. Faust, M. C. Roderick, D. J. Cunning- ham, & T. Andre (Eds.), Current research on instruction. Englewood Cliffs, N. J.: Prentice-Hall, 1969. Ausubel, D. P. The use of advanced organizers in the learning and reten- tion of meaningful verbal material. Journal of Educgtional Research, 1960, 2;, 267-272. Ausubel, D. P. The psychology of meaningful verbal learning. New York: Grune & Stratton, 1963. Ausubel, D. P., & Fitzgerald, D. The role of discriminability in mean- ingful verbal learning and retention. Journal 9; Educggiongl Research, 1961, 22, 266-274. Ausubel, D. P. & Fitzgerald, D. Organizer, general background, and antecedent learning variables in sequential verbal learning. Journal of Educational Psychology, 1962, 2;, 243-249. Banathy, B. H. Instructional systems. Palo Alto, Calif.: Fearon, 1968. Bandura, A. Principles of behavior modification. New York: Holt, Rinehart & Winston, 1969. Bloom, B. S. (Ed.). Taxonomy of educgpiongl objectives. New York: Longmans, Green, 1956. Conover, W. J. Practical nonparametric statistics. New York: John Wiley & Sons, 1971. Cornfield, J., & Tukey, J. W. Average values of mean squares in factorials. Annals of Mathematical Statistics, 1956, 21, 907-949. De Cecco, J. P. The psychology of learning and instruction: Educational psychology. Englewood Cliffs, N. J.: LPrentice-Hall, 1968. Duchastel, P. C., & Merrill, P. F. The effects of behavioral objectives on learning: A review of empirical studies. Review of Educggional 121 122 Ebel, R. L. The relation of testing programs to educational goals. In The impact and improvement of school testing_progpams, The 62nd Yearbook of National Society for the Study of Education. Chicago: University of Chicago Press, 1963. Ebel, R. L. Behavioral objectives: A close look. Phi Delta Kappan, 1970, 22, 171-173. Eisner, E. W. Educational objectives: Help or hindrance. School Flanagan, J. C. Functional education for the seventies. Phi Delta Kappan, 1967, 48, 27-32. Gagné: R. M. The implications of instructional objectives for learning. In C. M. Lindvall (Ed.), Defining educational objectives. Pittsburgh: University of Pittsburgh Press, 1964. Gagné: R. M. The conditions of learning. New York: Holt, Rinehart & Winston, 1965. Goldstein, A. P., Heller, K., & Sechrest, L. B. Psychotherapyyand the psychology of behavior changg. New York: John Wiley & Sons, 1966. Gronlund, N. E. Statipgrbehaviorgl objectives for classroom instruction. London: Macmillan, 1970. Hays, W. L. Statistics. New York: Holt, Rinehart & Winston, 1963. Heller, K., Myers, R. A., & Kline, L. V. Interview behavior as a function of standardized client roles. Journal of Consulting Psychology, 1963, 21, 117-122. Hendricks, C. G., Ferguson, J. G., & Thoresen, C. E. Toward counseling competence: The Stanford program. Personnel and Guidance Journal, 1973, Q1, 418-424. Hinds, W. C. Behavioral objectives: Implementation in the counseling process. Paper given at the annual convention of the American Personnel and Guidance Association, New Orleans, 1970. Horan, J. J. Behavioral goals in systematic counselor education. Counselor Education and Supervision, 1972, 11, 162-170. Jakubowski-Spector, P., Dustin, R., & George, R. L. Toward developing a behavioral counselor education model. Counselor Education and Supervision, 1971, 19, 242-250. Krathwohl, D. R., Bloom, B. S., & Masia, B. B. Taxonomy of educational opjectivesJ handbook II: Affective domain. New York: David McKay, 1964. 123 Krumboltz, J. D. Behavioral goals for counseling. Journal of Counseling Psychology, 1966, 13, 153—159. (a) Krumboltz, J. D. (Ed.). Revolution in counseling. Boston: Houghton—Mifflin, 1966. (b) Lorr, M. Client perceptions of therapists: A study of the therapeutic relation. Journal of ConsultingfiPsychology, 1965, 22, 146-149. Mager, R. F. Preparing instructional objectives. Palo Alto, Calif.: Fearon, 1962. Michaux, M. W., & Lorr, M. Psychotherapists' treatment goals. Journal of Counseling Psychology, 1961, 8, 250—254. Miller, T. V. Simulation and instructional objectives in counselor education. Counselor Education and Supervision, 1972, $2, 83-87. Rogers, C. R. Client-centered therapy. Boston: Houghton-Mifflin, 1951. Scandura, J. M., & Wells, J. N. Advance organizers in learning abstract mathematics. American Educatiopgl Research Journal, 1967, 4, 295-301. Stewart, N. R., Burks, H. M., Engelkes, J. R., Johnson, R. G., & Winborn, B. B. Systematic counseling materials. Unpublished instructional materials, Michigan State University, East Lansing.(Mimeo.), 1972. Stewart, N. R., & Hinds, W. C. Behavioral objectives to direct simulated experiences in counselor education. Videotape and paper presented at American Personnel and Guidance Association, New Orleans, March, 1970. Sullivan, H. S. The psychiatric interview. New York: W. W. Norton, 1954. Thoresen, C. E. The systems approach and counselor education: Basic features and implications. Counselor Education and Supervision, 1969, 2, 3-17. Thoresen, C. E. Training behavioral counselors. In F. W. Clark, D. R. Evans, & L. A. Hamerlynck (Eds.), Implementipg behavioral programs for schools and clinics. Champaign, 111.: Research Press, 1972. Whiteley, J. M., & Jakubowski, P. A. The coached client as a research and training resource in counseling. Counselor Education and Supervision, 1969, 2, 19-29. 124 Williamson, E. G. Counselingyadolescents. New York: McGraw—Hill, 1950. Winborn, B. B., Hinds, W. C., & Stewart, N. R. Instructional objectives for the professional preparation of counselors. _Counselor Education and Sppervision, 1971, 19, 133-137. Wolberg, L. R. The technique of psychotherapy. New York: Grune & Stratton, 1954. Yelon, S. L., & Schmidt, W. H. The effect of objectives and instructions on the learning of a complex cognitive task. Paper presented at the annual meeting of the American Educational Research Association, New York, 1971. M'lllllllmllllllllllll 111111 11111111111111“ 3 1293 01015 0899