AN EVALUATION OF THE OBJECTIVlTY OF AN AUDIOMSUAL COUNSELOR SGALE WHICH COMPARES THE BEHAVICIRS OF M. A. CANDIDATE COUNSELORS WITH THE BEHAWORS 0F PH. D. CANDIDATE COUNSELORS Thosis fot the Dogma of Ph. D. MlCHiGAN STATE UNWERSITY Gerald G. Griffin 196.6 LIB R A R: Michigan 5-: ;‘: i University This is to certify that the thesis entitled AN EVALUATION OF THE OBJECTIVITY OF AN AUDIO-VISUAL CWNSELOR SCALE WHICH COMPARES THE BEHAVIORS OF M.A. CANDIDATE C(IINSELORS WITH THE BEHAVIORS OF PH.D. CANIEDATE CCII NSELORS presented by Gerald. G. Griffin has been accepted towards fulfillment of the requirements for Doctoral degree in Philosophy / ’ ' I I I ' ‘ " J, . I/ , '1'" , ’ ’(‘é/{L |/ I O ('1' "’ r ‘1, r I I k P ' Majoé profeséor A33T£AC 3H SVALUATIOH OF THEL mMIfIf OF M AJDIO-VISUAL COUI'S 3140.1 Soil... 'IIICEI CI) 33A £33 F13 BEHAJlC-BS Or i‘-'..A. CANDIDATE CCU 3110 L3 UfHTRBBJMWD$3m3HLD.C&DEMT3UL'CL$B by Gerald G. Griffin This 103 esrch examined three in H10 oth eses associated with the A ‘ develop ent of an Audio—Visual Counselor Behavior Scale (A-. Sea 1:). v Or 1 C.‘ :is A-; tca‘e was developed for the purpose of making available an objective ins rur e‘.1t which would compare the response patterns of gaster's degree candidates in Counseling an ad Guidance (L.A. Candidates) with the response patterns of Doctoral de ;:ree candidates in Couzsclin3 and Guidance (Ph.3. candidates). The A-Y Scale consists of five forced choice, dichotor ous disens ions: the Affect-Cornitive, Unders tanding-I onunderstane , Specific- ..ons1ocific, Sleoratoryl onexploratory, and Effective-Noneffective ii: Lensions. The research consisted of three separate studies. The Scale Bdscrinination Study tested the hypothesis that the A-V Scale would significantly discriminate the response patterns of Ph.s. candidates from those of K.A. candidates on its five diaensions. The Oojectiw itJ StudJ ested the hypofi mi that the A-V Scale would obtain hi3h interjudge agreement when used for the ratin: of counselor behaviors on audio-video tapes. The Sensitivity Study tested the hypothesis that the A-V Scale is sensitive to chan3e in the counselor behaviors meaw mred by its d'me mic :3 a 9.) a result of supervision which focuses on behaviors within these Gerald G. Griffin dimensions. For the Scale Discrimination Study, three trained counselor judges used the A-V Scale to independently rate 20 responses from each of ten 30 minute audio-video taped counseling sessions. Five of the counselors engaged in these interviews were K.A. candidates. The other five counselors engaged in these interviews were Ph.D. candidates. Each of the ten interviews was an initial personal-social problem interview with a client. The rating scores obtained by the two classes of counselors on each of the five dimensions of the A-V Scale were statistically compared using ‘ti-tests. There were significant differences in the response scores of the two classes of counselors on all five dimensions of the A-V Scale. The responses of the Ph.D. candidates were more frequently rated affect, understanding, specific, -xploratory, and effective than were the responses of the L.A. candidates. For the Objectivity Study, three trained counselor judies used the A-V Scale to independently rate four minute seaments from each of J: audio-video counseling tapes. Forty—five of these tapes involved K.A. candidates and eight involved Ph.3. candidates. Hoyt's analysis of variance method was applied to the ating scores. Two types of interjudge reliability and internal consistency coefficients were obtained. One was an index of item objectivity and internal consistency, and the other was an index of a global rater objectivity and internal consistency. Item interjudge reliability coefficients of .89, .79, .76, .5}, _._‘ and .31, and global rater interjudge reliability coefficients Gerald G. Griffin of .95, .91, .39, .70, and .93, were computed for the Affect- Cognitive, Understanding-Honun‘crstanding, Specific-Nonspecific, Exploratory-Ionexploratory, and Effective-Noneffoctive A-V Scale dimensions respectiv 1;. For these same scale dimensions, item internal consistency coefficients of .96, .92, .91, .8l, and .93 respectively, and global rater internal consistency coefficients of .98, .979 oyb, .37, and .98 respectively were computed. For the Sensitivity Study. ten HDEA Guidance and Counseling Institute trainee volunteers were randomly assigned to two groups of five; ne group comprised a supervised counselor-interrogation group (Supervised Group), and the other group comprised a non- supervised client-interrogation group (Nonsupervised Group). Each member of both groups underwent six 30 minute video taped counseling sessions with a college or high school student. The Supervised Group focused on affective, understanding, specific, and exploratory responses during interrogaticn 1112' the Eonsupervised Group had no particular focus during interrogation. The middle twenty responses of the first (initial) and sixth (terminal) counseling sessions for each of the ten counselors conprising the two groups were rated by three trained counselor judges. These initial and terminal ratings within each of the Supervised and Nonsupervised Groups were statistically conpared using dependent *t"-tests. The results of these tests indicate that‘for the Supervised Group there was a significant increase in the rating scores, from the initial to the terminal ratings, on the affect, understanding, specific, exploratory, and effective dimension categories of the Scale's five dimensions. For the Gerald G. Griffin honsupermised Group there was no significant score increase, from the initial to the terminal ratings, on these five Scale dimension categories. A3 EUALUATIOI CF THE OBJECTIVITY or a: {DMD-VISUAL coma-max sous; macs comma Tia BSHAVIORS OF n.A. CANDIDATE COURSELORS 41TH '33 BEHAVIORS OF PH.D. CAEDIDATE counssLoas By .. .. e. Gerald G? Griffin A THESIS Submitted to Kichigan State University in partial fulfillment of the requirements for the degree of DOCTOR OF PHILOSOPHY Department of Counselins, Personnel Services L) 'and Educational Psychology 1966 ' 1’j'O "' "I .fi'1 'fia'flfi -1C;L.. 14.0 U..._J.L.l3 I .Jisa to e mress my sincere gratitude and appreciation to my ajor Professor and Program Chain'nn, Dr. ITornan Kagan, not only for providing me the opportunity to pursue this research endeavor but also for t1 e very hel>ful support and consul ation he provided towards its fulfilln nt. In addition, I an especial..y grateful to Dr. Richard Rank, Dr. Bill Kell, and Dr. Bernard Cora-.1 for the scientific refinement they added to the research through their critical evaluations and individual contributions. Lg; sincere thanks 13 also extended to those many counselors V who helped in the develogI1ent of the A-J Scale, 1-: no participated in the studies of t‘m'Ls research, and who devoted their time and skills as judies. Particularly do I wis h to acknowledge nr ti ude to Robert ..Ta.rd, Ifarbara Fuller, John Eiartsell, Joe l-Lesanno, 31.13. 130300, Patricia Parisian, Robert Campbell, and .511 Goluoel 1". I would like to 1::ake note, too, of my appreciation to L3 -111 ..enc 1301. for t11e1::a11y secretarial services she rendered in assoei . v.1. 1.. '..ith this resea1 ch. Ffinaffly, to 1:1 wife, Patricia, 11.51311 to c::' rse s :i.y gratitu-ie for her pat'.1_en e111; c139 )eration and uncierstaz'a‘infi; 1111:1511: the 11311118. F 'L‘ " 1‘ "i .11 r ‘ .‘1 "‘ N ' . " '-.A —" e. 'l‘ ‘t-' '(' J‘ ‘ , 6 f‘ 3.‘ 1.35. s roses.-c-1, 11.11.- uo extend rys . ..patu m. ». 1,0 111.1 for . 1 . -.,:,. .m 11“,. . v .1...-:_-.-,_ ,,J.. ,1 v 1 “,1 . . . 0,. A ..-,,-. .2 . . ENQUJ._.-;-Q £4110 1.3.1.1011.) {53-41.15, .11.,31513, 13.... 8119 U... tendent 3.11 ~.2.l.L_L~.«..‘.'A I 'l“ 'L" -. u ' ‘-.. \r- .1 r‘v. ,o . . J" — J' of . . -. x 4.-.»... .-l..:rc~::..1.-_on document. r‘ v . ..I,‘ V. bfififl in» I. ”or III. 1V. '1 ‘ Ty 1‘. \ ".1 ;! A—La-‘l 1 .bk" _ -‘.J4 . O O T ' " -: > '3 JOEL--5 .L‘LVLJUF. 13: 'I' . .3 JILC‘J‘JLCGL . . H . T - . 7‘ 5‘. 1 .— .fi "1 ow 'fif‘ . . V \ .‘ n¢- AA‘*‘)iN) k)“ AtbhiLlJLl.) w1'- ' rv“"-'| L uJA, V .-W-‘,‘ 1. .1) 1.:,.4;1".;".'.f COLICLULSICLIES AID IliFLICATIOITS 3 Bl BLIOILXE’I LI . . . H. 1* k'AGL‘.’ 1’)\ \11 (7‘. 7l C‘ ’3 13¢ ’- \,l / _’ LIST OF TABLLS ABLE PA" h L?) j.u. Comparison of the loan (K) Scores TGtVOOH the Ph.D. and E.A. Candidates on Each of the Five Dimensions of the A-V Scale . . . . 7? §.~. Ito; Interjudce Reliability and Internal Consistency Coefficients Computed from the Scores of the Scale Diserininetien Study . 73 Kn O \J) O Iteu Interjudge ieliability and Item Internal Consistency Coefficients Computed from the Scores of the Objectivity Study . . . . 7% r’ ' 7- »\ \ I fin'L ‘ ' '3/ ~ . ‘ . g . vv J- 3.4. gleshl “beer IAtGTJUL’Q ?elianitity and Internal A - ‘av _ -v _ .— .a. “-0. ‘. 3 .0- ‘ “ .I- V, ‘n " J_ ‘r‘ ~a-r/ .~.'.A.I(.ILII- I... :~."_'. ' . ‘.",(".:.‘J-(J Cf) ~k J .A. .. - \j’. . Due ,_ --, _ __-.-,'_2,, -¢-.'f. '.,) “f x I, c..'. but/Jr, 4”...“ ..IV.£\x/ o o o o (J j. . Conscrison of the Xeen (K) Scores Between the En.2. and 3.3. Candidates on the Cognitive, Honunderstandint, Fenspecific, and Ionexploratory C togories and Effective-Noneffective Di ension, of the A-V Scale . . . . . 7C \J'x Q\ . Leon (1) Scores of the Initial Interview Rating for both the Konsupervised and Supervised ' ‘ :vogos on the Five Dimensions of the A-V Scale 77 5.7. Coxpgiidon of the Kean Scores of the Initial and Terminal Interview Ratings for the {ensupervised (Client-Interrogation) Group gn tie Five Dimensions of the A-V Scale . . 8 5.3. Ce Sinison of Loan Scores of the Initial and Terminal Interview Ratings for the Supervised (Counselor-Interrogation) Group on the Five Dimensions of the A-V Scale . /; 7v Ll. - O ; 5%. LIST OF APPEYDICES A Conpilation of Counr elor Video Veroal Sehaviors as Observed on Audio-Video Tapes , . Categonisction of Counselor Audio-Video Zehavi01 3 According; to Behavior Categories , , Definitions and Illustrations of the A-V Scale's Dimension Categories . . , , , Q 0 Clarifi ation of Frequently Occurring Rating Problems and Questions , , . . . . . oc‘ulstical t -Iest Inalfsis Cenparing the ”-7 Scale Dimension Scores of the Scale Discrir' ztion Study’s Five Ph.D. Candidate' cith Those of the Five K.A. Candidates . , . Statistical Itex Analysis of the Scores of the Scale Discrixi.nation Study Based Upon Hoyt's Get utation of Reliability by Anal sis of Valixnice , . . . . . . . . . StatiLt ieal Ite Analysis of the Scores of the 0330( mti tjf Study 3ased Upon hoyt's Compute tion Reliability oy Anal;si.s of Variance . . . 0 “Leo ;lieal Cloaal Rater Analysis of the Scores of the Oh jectivity Study 3ased.Upon Hoyt's Ce Jutation of Reliaoility by Analysis of \VCLiO Ice . o O O O O O O 0 0 Statistical "tt-Tcst Analysis Comparing the A-T Scale Scores of the Objectivity'Study's Ifiqfit Ph.D. Candidates with Those of the Forty-Five I.A. Candidates . . . . . . sependent-“t” Statistical Analysis Computations Conparing the Konsupervised Group's Initial and Eerjinal Interview Scores . . . . . 3e enlont- t” Statistical Analysis Computatiors C; paring the Supervised Group's Initial and Terminal Interview Scores . . . . . Ill ‘4 r3 "Nf ”’K }_J (_ \ K11 or] r, L k; [\ff‘ l‘v CR; 12R THE PROELBf This research is primarily an objectivity evaluation of an Audio-Visual Counselor Behavior Scale (A-V Scale) developed to compare two classes of counselors at Hichigan State University. «ssentially, Iaster”s degree candidates in Counseling and Guidance (I.A. candidates) constitute one class of counselors while Doctoral degree candidates in Counseling and Guidance {;h.D. candidates) constitute the other class of counselors. The A-V Scale consists of five forced-choice, dichotomous dimensions which measure the extent to which each of these two classes is characterized by affective, understanding, specific, exploratory, and effective responses in initial personal-social problen interviews. In the remainder of this chapter the problem will be more explicitly defined under the headings of Specific Problems, definition of Terms, Delinitations, Basic Assumptions, vaotheses, Heed for Study, and O~sanization of the Stu.y. L.) o A“ Speci:_c Problems The specific problems of this research are: «L 1. Can the two defined classes of counselors be signi- licantly differentiated on any of the dimensions of the A—V Scale? N 0 Ion objective is the A-V Scale for measuring these th;ensions? -1- Ix) - how sensitive is the A-V Scale for indicating change in counselor behaviors on its five dimensions as a result of supemision which focuses on behaviors within these dimensions? Definition p_f_ Terms The terms validity, objectivity, and reliability are defined in the following manner: l. Validity. The extent to which the A-V Scale does the job it is employed to do. That is, the property of the scale to differentiate the video-verbal behaviors of ILA. candidates from the video-verbal behaviors of Ph.D. candidates. 2. ijectivity. Interjudge reliability based upon Hoyt's analysis of variance. This is the average reliability of ratings based upon the average of interjudge correla- tions. 3. Reliability. Reliability based upon the internal analysis of the average ratings of judges employing Hoyt's analysis of variance method. gelimitations The major dehmdtations of this study are: l. The video taped counseling; sessions employed in the individual studies halting up this research, with the exception of one objectivity study, were based upon relatively small numbers of counselors (I'J'ith diverse theoretical orientations) at I-‘Iichigan State University. -3- This was necessary because of the special problems, xpense, and time involved in video taping, and because of the limited number of counselors available for the type of video taping required. The counselor verbal behaviors studied were laboratory behaviors in which both the counselors and clients were aware that their counseling interviews were being video taped. Basic Assumptions The basic assumptions upon which this research is based are: l. Counselor responses which are affective or cognitive, understanding or nonunderstanding, specific or nonspecific, exploratory or nonexploratory, and effective or non- effective, are available on audio-video tapes for judges to observe and measure by a rating scale. Both H.A. and Ph.D. candidates in Counseling and Guidance have a pattern and manner of counseling associated with these behavior dimensions which are consistent throughout their counseling interviews when such interviews pertain to a client's personal-social concerns. Those A-V Scale dimension behaviors which are character- istic of the counseling practice of Ph.D. candidates, but are not characteristic of the counseling practice of N.A candidates, are assumed to be learned by the Ph.D candidates as a result of their training. gmmtheses The general expectations which this research examined are: First_§ypgthesis. The frequency of affective, understanding, specific, exploratory, and effective responses in initial personal-social problem counseling interviews, as measured. by the A-V Scale, will be significantly greater for Ph.D. candidates in Counseling and Guidance than for M.A. candidates beginning their practicum experiences in Counseling and Guidance. Second Hyppthesis. Based upon Hoyt's analysis of variance method, the A-V Scale will have adequate objectivity for differentiating between Ph.D. and M.A. candidate responses on each of its five dimensions. Third.Hypothesis. Counselor trainees who receive professional supervision.which focuses on.exandning affective, understanding, specific, and exploratory responses will receive higher frequency ratings for these categories on the A-V Scale than will counselor trainees who do not receive this pro- fessional supervision. Most instruments which have been developed to measure counselor performance have had poor objectivity, and therefore, poor reliability. In recent years the demands of an ever expanding, service oriented society have created a tremendous press for counselors. These demands have not only necessitated an increased need for adequate counselor education progr-ms, but they have necessitated the need for adequate counselor education programs WhiCh could accelerate counselor training. However, the educational and counseling professions have been seriously hampered in their ability to experiment with various counselor accelerated programs because they have lacked an objective counselor behavior measuring instrument . Organization 9_f_ thg M A review of relevant rating measurement methodology, related research, and common counseling elements are presented in the following chapter. The development of the A-V Scale is presented in Chapter III. The methodology for testing the three hypotheses set forth in the present chapter is presented in Chapter IV. The results of this research are presented in Chapter V. The summary, conclum‘ons, and implications are presented in Chapter VI. CHAPTER.II FflfViEu' Ol‘ LITZU1L I): The review of literature presented here is divided into four areas, each with a summary: relevant general principles in the construction and use of rating scales; methodological considerations in the development of rating scales; previous counselor behavior measurement; and common elements of counseling pra tice that are important in their evaluation. The terms *therapist”a nd couns elor are used interchange- ably. Construction and Use of Pating Scales A joneral review of the value and use of rating scales, and the important considerations associated with their develop- 5 :3 (+- \O :1: ‘e presented.in this section. ”...3 ratizg instrument generally consists of two com- ponents; a descm ption of the behaviors to be considered and a set of altCLnDtiVO res we ses fr on which the rater is to choose for each person rated‘ (Ielmstadter, l96fi). A rating scale is one o: the ;:est peHJul r techniques for describing hrs a1 characteris‘ics (Helrzsta dter, l96~z Tor3erson, 1953). A rating scale, states el‘r ‘l' 5 "2 "\“ . " t" ‘ ”It “. “" "" ’- -!" ~ '. -'-.' l\’ r us-.. 1.» v1“ .00 (‘L' (".2 . ‘0 i.'_ '.,.: for dte Cl':ilfr-‘.‘.{; the w y in which others react to an individual in question, and for this situation they are valid by definition. Helmstadtcr (l964, p. 199) remarks, ”...if sufficient care and trouble are taken, rating scales can be used to obtain highly satisfactory quantitative descriptions of many behaviors which cannot be studied by other measuring devices.” Test construction is a never ending process (Tyler, 1964). Rating scale development is a special form of this process. In the initial stages of test construction there are no specific rules or techniques rhich can tell a person exactly what to measure in a particular situation. The researcher ultimately decides what the measure will include based.upon his experience and judgmental skill (Helmstadter, 196M). However, helmstadter presents the following steps which a test constructor must follow if his instrument is to be based upon sound construction. 1. The first and most essential step is a careful delimitation and breakdown of the area(s) or trait(s) involved. 2. Once the areas of knowledge and/or particular traits which need to be included have thus been laid out, he next stei is to list the behaviors which the examinee should exhibit if he possesses the knowledge or characteristics under consideration. 3. When the test items have been written, the next major step is that of trying them out on an analysis group of s1bjeets who are representative of the population for which the total test is being prepared. .’\ 4. Kort, the items are to be organized into a resonably administered format and an appropriate system for scoring the test is to be develeped. 5. The last logical step in develOpnent is hat of naking a final eval ation of the test, as revised and re- organized, on a new analysis group. {elmstaater (1964) points out that even with the best possible psychological measuring instruments, errors will be made. He mentions that what test constructors should do is, "...to learn what sorts of error can occur; to develop procedures for estimating this error; and, insofar as possible, to utilize methods of test construction and administration which minimize these errors." When rating scales are used, errors arise from rater char- acteristics and from the behavior selected for measurement. Rater biases and constant errors of leniency (tendency to be overly generous in descriptions), halo effect (rating a given individual in the same way on all behaviors), and error of central tendency (the avoidance of using extreme positions on a rating scale) are errors which arise from rater characteristics (Helmstadter, 1964). One of the most recent innovations in the rating scale area has been the development of a forced choice technique which has been specifically designed to overcome these difficulties (Highland and Berkshire, 1951). In a forced choice rating the judge is required to consider not just one attribute, but several characteristics all at one t‘me. he general format requires that only one or a few of the behaviors listed in each item be selected as applicable. To reduce error ariSing from the behavior selected, Helmstadter ‘ 7 ting behaviors which can be dependably rated. He 23 E C) (i- (.1 L“) a O 0 highest for those characteristics which are Specific entities. Tie major logical objection to the rating scale approach is that ratings are often based on only broad impressions about the person being ‘ated. Therefore, for rating scales to be useful to researchers ana workers in applied fields, numerical ratings must in some way relate to relevant observable behaviors. Recent studies (Gobhavt a.d Sort, 1958; Guilford, 1959; and Thorton, 19b3 s of specifically defined characteristics obtained on carefully developed scales can have S validity, while judgments of broadly defined traits are often susject to irrelevant influences. Eel stadter (196%) offers four major techniques for the improvement of ratings. First, rate only relatively Specific ‘ . aly characteristics which "...are readily discernible and have a hig out meaning for a wide variety of judges.” Second, be careful in the preparation of the rating scale itself. Select behavior labels which "...are short enough to fit on the scale yet sufficiently'unanbiguous to result in general uniformity of meaning for all potential raters. Incluce a concise description and a few examples along with each behavior label.” Decide upon the best number of steps to be used. Five and eleven see; -10- ..L 1 to be the o;mt1'al nunbor of steps above and below which lb does not pay to proceed (Chazpney and 2;.rshall, 1939). After the individual behavior scales have been developed, organis- then into a complete racin in tvuzent. Third, “...use careful selection and training of the judges who will do the rating.‘ Forth, (_) ...have the judges make their ratings independent}r and then {I pool tzeir res ts.- SUJBUaIfir ratings cale consists of a description of the be to be considered and a set of alternative resporses from Uzich the rater is to choose for each person rate . It is the most direct n asur currently available for determining the way in which othe‘s react. Common errors of the rating approach, such as ater biases, errors of leniency, halo effect, nd errors of central tendency, are gelmer mil avoided by the forced choice technique. If sufficient care and trouble are tagen in the devel lopment and use of rating ins' oruxents t1 er can DO used to obtain 1 "h 3 satisfactory quantitativcc descriptions of 21an; behaviors which _L ‘0 1 ‘ cannot be squaiec oy oth r measuring devices. Ratings are improved by selection of traits for neasurewent which relate to is and relevant observable behaviors; areful preJarat- n (A cale itself, including appropriate selection and aese"i;tio; of behavior levels and proper organization of individual behavior scales into a complete rating instrument; careful selection and training of the judges who will do the rating; and the peeling of the results of indeflaen eit judge ratings. Ratin" Scales and fiethodological Considerations Ac lesion of the pertinent methodological considerations presented in this section. These 51 U) 0 {a i..J O P. L) related to rat 5.- considerations refer o TOllflJl_le, objectivity, and validity, each of Inlic:1 is assoc M1 ited with a different kin: of error, and each of which represents a separate criterion for evaluating a test ('eltsstesel, 196U). .1th aza; test, relio.bi]-ity of the techniole end vali <15.t ' of the results are two highly si'nificant rechodelogical considerations (Tyler, 196%; Helnstadtcr, 1964; Ruck, 1963; Lindquest, 19Cl). In the case of rating scales, which are us ed by a "roup of judges to rate st..crs, a reliability consideration of ingortance is that of interjudge agrcexcnt (Helxstidter, 1969; Ebel, 1951; Hoyt, 1931). The central nethodelogical question for scale, then, nay be stated: how use “11 (valid) and objective J. (degree of interjudge reliabi t7) is the scale for the surgese Following is a general overview of reliability, objectivity, and v: lidity considerations and their relation to measure: .ent Selig 1b: 411g; /n a o a End 1(190)) en has 303 that a measuring dcv1ce cannot be valid unless it is first of a1. reliable. The 1959 Glossarv of 11,0.s1rez'10nt Tens," published by the Califorrfia Test .; tree-.13., /\/’) defines reliability (in common rith others such as inch, 192) and Tyler, 19C:) as the le ree to w a an individual would obsain the sane score on test if the test were read irjstered E.) 4‘, ‘n~- "1 ‘1. .A .1 (“'11. . (n v -:‘ ., .s 4- m -* . to one sL .e incavjduaaq 1:113 “301 ssL.rj oi “easure .enL, Lerls distinj"shtm three tvpes of reliability coefficients: j ‘1’ n15: o 4‘ _fi (‘"‘ ”’1‘. H. o J; .- . - A‘,‘ r -. | -, 4‘. COL-.. ..L.--C_L .ILL .. L) .JL.-_/:1..L.l. .1" —v._~ -— W A correlation between test and retest with sene period of time intervening. The test-retest situation nay be :ith tno ferns of the sane test. 2. Coefficient of eluivalence u correlation between scores from two ferns javen ct essentially the same time. 3. Coefigicient of internal cons is er 9: A ‘easure ba.sed on internal ancljsi-sofd1ta obtained on a sfnfile trial efa . tes Tore nro: inent of these are the sWnlit- elf method and the Kuder-lichardson and ..l n hogs analjsi s o; variance netheds. fieli1yilit7, in each of these cases, is defined as “...the ratio Of true score variance to observed score variance and is an index of tle agount of 'ariable error (chance errors varyinfi from ° .1. .1. person to person and fro: one tixc to tle next) in a sesc’ There are four classical ways of estimating rclia m; tr (best- retest, parallel for , split—half, and finder-Richardson), all of which may be expressed 11 n for-: analorous to the quotient of the observed variance ninus the error variance to the o variaace. P1ch of these procedures is basically different in that '...it defines chat is meant by error in a slirhtly fferent wag. cl-ss (ter (l‘/’) points out that the proce(lure for '1 ’3 deter ining the test-retest correlation ...defines as variable error any tine to tine fluctuations;'f the procedure for deter inins parallel for; reliability ‘...defines as variable error both time to time and fern to for; fluctuations;“ the procedure for deter- Lining split-half reliabilit" "...defines variable error only as fluctuations from one half of the test to the ne::..;'2 the procedure for cote: inizg finder-Richardson reliability (Iuder and Richardson, 1937) '...defines variable error in terns of consistency of per- forance fro:.; iter. to item." Loyt (thl) has presented a procedure for estimating reliability which defines a variable error in a sonewhat different way from these other aggroaches, although the results are identical with Kuder-Richardson reliability. Hoyt's final basic way of defining what is n ant by variable error involves a slightly different breakdown of observed score variance from what is usually presented. According to Hoyt‘s formulation, ”...variation in the performance of an individual fret iter to item is not considered error. Rather, it is a real (non-error) difference, an intraindividual difference, and one which should not be involved in the estination of reliability.’ Total variation observed, then, is conceived to be made up of three coxoonents: 'true interindividual differences, intraindividual differences, and error interindividual differences. According to Hoyt a better definition of reliability is expressed by the quotient h of the true interindividual variance (error variance subtracted 1Wa~ *- the obs rved variance minus the intraindividual variance) to the aserved variance Linus the intraindivioaal variance. Using an P O 1 l 1‘ "/V. (s ' ' Q’fi . A A \ r ‘1 ~ 7 V (w 4| 1 . —. ~- anal;s1s o- «_riance procedure, then, lovt's 1e11aoil tj can be estimated by I‘__;] = 1-13 individuals - IZS residuals 175 individuals I-IS individuals = mean square of deviations from the il1rklvidual's means l-LS residuals 2‘ z-nean square of the deviations left over after individual and item variation have been removed. The reliability of rating scales is affected by the major considerations for f ornulating an observational measurement. The number of scale categories and the effect of the length of the judged lin‘telial appear to influence reliability (Helmstadter, 1961+; Bendig, 199%; Bendig and Sprague, 199+; Symonds, 1922). Reliability of rating; scales is enhanced by clear and. ummbiguous directions (Helrlstadter, 1961}; Benc‘ig, 1953), preciseness of rating criteria (Helmstadter, 1960'; Rose, 191W), uniform scoring procedures (I-Izmlin, 1951:»), use of adequate samples of observed behavior (Helmstadter, 196+; Hardin, 199+), verbal definitions of scale categories and illustrations of behaviors which fit various scale categories (Bendig, 1953; Albee and Hamlin, 1950), and the peeling of independent jucre ratings (Iielmstadter, 19’»; Gordon, 1924). The factor which influences scale reliability the most, he: ever , is interju r“e agreement (IIelmstadter, 1961+; Ruch, 1963; Benrlwi; ll55; 3011618 and Spraguea 1954)“ ‘L. .-bj] O] ajectiw lfielgastadtor (1964) defines objectivity,r as the extent to which -13.. pers0111 errors have been avoided in rating. Thus, a measure "hick is relatively free from pers sonal errors is referred to as' objective. Per:;;lied an analysed in various sec;Lal settings by neldnan, Hyman, and Hart (1951). I. The use of an interview or pretest in the formulation of a ratinr scale has preven to be especially valuable by Rose (1947). epersonal charactel of the judges appear to be significant in observational analysis. Bendig aW1d Wrague (195/!) found thct the individual rater reliability increased rith hi 3her levels of education, and the va riance depended grea tly'upon the length of the scale and the experience of the rater. In a fellewaup study'using the sane rating scale, Bendig (1955) again found that the educational level of the judm affected the reliability. Here bias measures were found in the undergraduate "lower education" judges. Helzm tr.d oer (196%) declares that the careful selection and J‘J. training 01 ulOSO jud:es the will do the rating is one of the most effective ways of insuring adequate results from the rating scale approach to measurement. He indicates that the firsat rule or who I’ll]. have the op aortunity to ma}: e the necessary observations of the subject in pertinent s.tuatiens. is important tO‘try to secure raters who are willing to maze grud3 ents free of distortion. One of the first tasks in tlm etraining of raters to use a scale apprepriately, notes Helmstadtcr, is to convince they of the value of honest and accurate ratings no It also helps to point out the types of errors which raters are li. :cly to r:.l:e azd provide some .L!‘O tiezn 3for avoiding .L‘ U; LC‘.‘ . Ya lilit “c Validity is the most important consideration in the con- struction and use of all t31)es of tests (Kelmstadter, 196%; Tyler, 1963). Rob abi ity is morelv a hes ste the er d of validity (Ruch, 1963). A measuring device can not be valid unless Lalrstadte" 105%) states that ‘...reasure:ent of bellavioral Claractew astics is indirect. Zecause of this, one cannot be con>letely co: tain that a test measures the precise characteristic for which it U’C sdesigned. Thus, an error can be made in using tests which is the sane for every individual to which the test is aoplicl and the sane every'tine it is used. Such errors are called constant errors. In test measurement, the problem of these const nt or; ors is the problem of validity.” Validity is an indication of the extent to which a test is not influenced Ly constant err Cronbach (lBQG) and others point out that validity may have a varie OtJ o; xeanings. The the basic approaches to the deter- mination of validity are logical analysisa 1nd empiric al analysis ( Zlossary oi Hcasurenent Terns”, 1959). Content validity and it01 structure are t3Q)es of va.ljdit" 1v logical analy81s. Content validity refers to "...he" '.e Ll the content of the test samples the subject matter , or situation .Dout wllic.1 conclusions are to be drawn.” This r'y be determined the opinions of a jury of experts. Content validity includes face validity, which invol a simple evaluation of co: Itent adequacy; logical validity, in which it is '1 1"\ U) Q) carefully deter.- ined I'FlIO'UlOI‘ the test behavior i . representa- tive sample of the behavior in question; and factorial validity, in Mulch the e:~;tent to which certain factors contlibute to total test scores are determined by exmfining the many relationships among; test-taking behaviors (IIelmstadter, 1961;). Item structure includes corroborative evidence from item analysis supporting the other elmracteristics of the test and item coztmposition emphasizing conceptual clarity in the expression of items. Data shoxing the relationship between test and some later (eliterion) behavior is called empirical validity. Concurrent validity and. predictive validity are types of validity determined by empirical analysis (”Glossary of Ifeasurement Terms”, 1959). Concurrent validity refers to how well test scores match measures of contemporary criterion performance. Predictive validity relates to how well predictions made from the test are confimed by the data collected at a later time. A type of validity in which both logical and empirical methods any be used in its determination is construct validity, validity in which the theory underlying a test is validated. Cronbach (1957) defines construct validity as an analysis of the meanil'z; of test scores in toms of psychological constructs. Rank (196+) points out that this definition requires that in order to establish construct validity, the test-m. {er must 1;;easuro a constluct which is tied into an appropriate theoretical framework. EIeL‘zstadter (1962:.) states that construct validity "...is evidence which seems to establish the test as a measure of so: :0 11;.rpothcsized trait which may affect performance in a variety of ways in rsny different kinds of situations." He adds that ”...construct validity involves such diverse evidences as group differences, perfornanee change as a result of experimental :-anipula tion of certain variables, axltitrait-nultinethod matrices of correlations, internal consistency in.ornation, and studies of the tes -ta:ing proces.." Keehl and TacCorquodale (55) suggest that no test has approached the ideal level of construct validity, indicating that this is a difficult type of validity to establish for a t st. lelnstadtcr (193 ) feels that it is; rather naive to ask whether a test is valid. Cronbach and heehl (1955) point out that 1 test is never rea’ly ”validated." Rather, a principle for making certain kinds of inferences about persons who obtain "ivcn test scores is verified or refuted. Information taat is gathered to determinee z'netly fnat kind of infereimce can be drawn from test scores is referred to as validity evidence. The question should not be t"Is the test valid?” but “Is the test valid for such and such?“ Ruch 1963) states that "...the validity of a measuring instrument is the extent to which it measures that Vhich it was designed to measure." Lindquest (l 961) points out that validity y for a pal ticular purpose with the essential question being how well does a tee t do the job it is employed to do. Lindquest (1961, p. 622) states: The validity of any test is its vali dity as an indicator of individual differences i n some pa articular function among the nezoel s of some specified group. Ty]. er (193:) foo ls in e:ca:ining validity that the proper question 1: is "...;just what is it that this test does measure?" rather that the old validity question of "...to what extent does this test measure what it purports to measure?” He feels that the content of the test 1;:ust be analyzed and many correlations em: ined with different criteria in various groups before the answer can be 131mm. 'l‘yler's view is that it is not necessary for a psycholoList to have an exact definition of what he hopes his test will measure at the beginning of his test developl 1ent. Tyler (1963, p. 29) cor-azents: ...i.f he has a general idea about the character- istic and its relationship to either theoretical concepts or practical situations, his precision in defining it will increase as he tries out his test in series of separate research studies. Bayroff, Haggerty, and Rundquist (195%) point out that the use of rating scales and judges for the assessment of segments of behavior carries with it certain problems and ‘ ~awbac:s. One of the most important problems seems to be the obtaining of :i1a::'1.11ruz:1 rating validity. These writers note that ratings are subject both to rater contamination (agreement of ratings due to the .LO ct that the same person makes two evaluations) and technique contamination (extent to which validity coefficients are al 'tifici all inflated because of reseltzblance of technique). They found that the most effective way of increasilg the rating validity was to: l. Avera e the ratings :tade by the raters. 2. have raters 1dcnt1f‘y their rating- s. ). .iavc the crucial. rating; completed early in the series. In 19.11 1113:; stated that the rating f hmsmn character was practicable "...if the rating is done under conditions as rigorous as “no follm-Zing:” 1. If each fiml rating given a person is the average oz- three independent ratings, each one made on a scale as objectified as the man-to-nan comparison t;po scale. 2. If the scales on which the ratings are made are c01 :parable and equivalent, having been made in conferences under the instruction of one skilled in rating scale work. 9. If the ‘hrec raters are so thorourr 11y acquainted with the person rated that they are conpetent to rate. If validation evidence identifies a number of scale items which sigfificantly distinguish between the criterion groups obtained in one sampling, the test constructor may wish to try them out on a second sample. This is known as cross-validation. If the sale results occur on the second sample, the researcher can have a fair degree of confidence that his finding will hold up in subsequent studies (Helnstadter, 1961+). Cross-validation is accomplished by trying out a previously developed and refined scale on a new group that resembles closely the one for which the scale will ultimately be used. Slur-'13 r”; Ijothodological pl‘OlDlOZ‘lS are the most prevelant questions regarding; rati: 3. Two of the most highly significant mothedolofical considerations concern the reliability of the rating technique and the validity of the rating; results. The reliability of a rating ‘techzfirplc refers to its consistency as a measuring instrument. Tirls indicates its relative freedom f ron variable error. Personal errors, witch are due to the impressions and biases of the raters, are a special class of variable error. They tel-{e on sufficient zragnitude when rating scales are used. A special index of reliability, known as objectivity, reflects the extent to which personal errors have been avoided in ratings. Lack of objectivity is one of the most common causes of unreliability. The greater the objectivity of an instrument, the greater its reliability will be. Using pretests in the forrmlation of rating scales, developing precise rating criteria, providing clear and unambiguous directions and category definitions, providing examples of scale categories, following uniform scoring procedures, providing adequate samples of behavior to be rated, and pooling of independent judge ratings are all techniques which reduce variable error and thus increase scale reliability. Personal errors are reduced, and thus objectivity. and reliability increased, when careful attention is given to the selection and training of the judges who are to do the rating. The educational level and knowledge of the judges seems to be of particular importance. ‘ The validity of test results is the most important consideration in the construction and use of a rating instrument. The problem of constant error is the problem of validity. Information that is gathered to determine exactly what kind of inferences can be made from the test scores is referred to as validity evidence. The major validity criterion of any instrument is how wen does the instrument do the job it was e:‘::ployed to do. The validity of rating results is increased, according to expert opinion supported by some experimental studies, by averaging judge ratings, having judges identify their ratings, and having the crucial ratings completed early in the series. Counselor Behavior heasurement Some of the most relevant research efforts over the past years to objectively measure various aspects of counselor behavior are reviewed.in this section. A brief historical overview of counselor measurement efforts preeeeds a discussion of the signi- ficant counselor behavior scales related to the present research. Histogz Laswell (1929) reported the need for more adequate recording of the data of the analytic hour if such interviews were to be used for research. In subsequent articles (1935, 1936) he reported the usefulness that electrically recorded verbatim.data could have for this purpose. Symonds (1939) suggested to psychologists that phonographic recordings might provide the objective data for the study of the interview process. Rogers (1942) described the ways in which such data might be used for training and research, and Covner, one of his students, wrote a series of articles (1942, 194h) spelling out the details of the rationale and technique of phonographic recording. Subsequent research under the general supervision of Carl Rogers validated the assumptions regarding the use that could be made of verbatim data. -25. It was recognized by early investigators that in order to conform to the rigors of the more objective approach characteristic of psychology, it was deemed necessary to translate the raw data of the interview into a form that could be amenable to statistical analyses. Lasswell (1938) devised a system of categorizing what the patient said during the psycho— analytic intervieW'by designing codifications for the patient's positive and negative attitudes towards himself, towards others, and towards the therapist. A system was developed by Porter (1943) for the codification of what the therapist said. Porter's system of categories was refined by Snyder (1945), who added categories regarding the statements made by the patient and, in part, studied their interaction. Other attempts to objectively describe the activity of the therapist followed the early efforts of Lasswell, Porter, and Snyder. In 1948 Keet codified the statements of the therapist to differentiate between expressive and interpretive activity. Finesinger (1948) classified therapeutic activity into three levels of activity: low, moderate, and marked, while Dittman (1952) classified therapist activity into questions reflecting feeling, clarifying feelings already expressed, clarifying feelings somewhat less surface, and depth interpretations. Similarly, Carnes and Robinson (1955) categorized therapeutic activity into clarifications, tentative analyses, interpretations, and urgings. In 1953, Collier focused on a continuum of activity level, while Harway, et. a1. (1955) and Rausch, et. a1. (1956) focused on a dimensional analysis of interpretive activity. D Signiiieant Counselor Behavior Scales The earliest validated counselor-rating scales of any Significance are Rogers' Relationship Inventory Scale (1962) and Andersons' Interview Rating Scale (1962). Rogers' scale 3 is one whi a per .zits the client to rate the counselor. It eonsi sts of three sectioxm which focus on the type of relation- Q ship provifcd by the counselor. The first section consists of 72 items thich describe possible positive and negative feelings cr‘ at'i'vles trhich the counselor may have toward the clie mt; e.g., f‘-at tines he feels contempt for re." The client is asked to rate on a four point scale, ranging from, ”I strongly feel this is true, to ”I strongly 1801 that it is not true," the extent to whiehh e feels each of these 72 items app ply'to 's can melor as he relates to the client. The second section of Rogers' scale consists of two questions asking the client to rate, along a seven point scale ranging from extremely dis- ied toe ompletelys ti. sfied, how he feels about the relationsnio wifi1the cou11selor. The third section consists of eight sentence completions concerning the counselor and his counseling; 0.3., ”the counseling seemed very .* Although the Rogers' Relationship Inventory Scale has demonstrated some utility for counselor eva aluation by clients, its major linitation is ‘nat it is tied too closely to the theoretical concepts of Rogers' wn counseling viewpoint. It focuses upon ulnatuu, Positive Regard, Congruence, and Unconditionali y. Thus, it contains no questions related to effective or ineffective use of counselor questions, confrontations, interpretations, and ”7 -.u - other counselor behaviors not emphasized as important by Rogers. Also, the scale depends entirely upon the subjective impression of the client doing the rating rather than upon objective measure- ment of counselor behaviors. Andersens' Interview Rating Scale is an attempt to provide an operational definition of ideal rapport, defined in terms of effective communication in counseling interviews. It consists of 50 items that are based upon a consensus among a select group of counselors and clients concerning the meaning of rapport. Eighteen of the items refer to client behaviors and attitudes; e.g., “the client distrusts the counselor," while the remainder refer to counselor behavior; e.g., "the counselor is very patient." The items are described in terms of specific behaviors land attitudes, each of which are rated on a five point scale ranging from ”never existing" to "always existing." The 50 items are evenly divided.between items representing high and low rapport. The Pearson correlation between the ratings of the counselors and those of the clients, based upon median scores, was 0.98. Anderson and Anderson have demonstrated that a core of agreement among clients and counselors can be reached concerning the nature of ideal rapport, using the Anderson Interview Rating Scale. The effectiveness of the scale as a measure of rapport in counseling has also been supported by other inde- pendent investigators. However, Andersons' scale limits itself to an effectiveness of communication concept and does not draw broadly upon other counselor behaviors such as aspects ~23- ased upon vie al perception and confrontation. Like Rogers' scale it relies more Upon svb' t'v 'mpr ssiens than uoon u..—, J. .u.. _... 9.3801633“.A ea.) 1 objective observations. . 0 .0 L. as part of a atienal Institute of Lental Health project, Rioeh, et. al. (l, 3), developed a scale for rating counselors in interviews which used outside profes smouals rather than clients as judges. Zioch's scale was uuch simpler than either the Rogers or .inderson scales. It consisted of tno Slall sectiozs: one for rating the counselor on his general counseling competency during the interview and one for rating the counselor on auto— criticisn. The interview section consisted of nine global rating s 0.3., “re spect for the patient, " and the section on autocriticisn.consisted of five global ratings; e.s. ”shows ° V at.areness of major neakness or weaknesses,“ both of which were marked on a five pei.t scale from poor throusl excellent. no aiocn Scale was applied to eight 40 year old mothers who had one year of training in the project's two year Hental health Pregran of individual and group procedures designed to train them to do psychotherapy'under lixited conditions. Four professional judgesr rated thes e counselors on each gloeal din ension using the five point seal». is a group, the eight counselors receiveda :mcan score above 3.0 on each dixension. In addition, the clients of these counselors were cliuiw Ml eval1ated to be fUnctioning better. These results vere considered positive. a?" _ a1" ‘1‘) '1‘ . .4 " ' ' "V I“ J. 'L . V (- J' As ‘v N r. .03 enough caese tentative results Laj suggest so 0 utilitr I 'r‘ - " a - \ 1 r‘ .'L ‘ -a .x..-' D C 1-,, u 5,. - lo: bfl‘ AHOC. seaae, lbs use seeus lihited ior caree reasons. H. .J- .‘ . '0 1 o ,'.L' 1 - ‘ '0‘ ‘H J. h ."fi sires, no rellasilicf stur: es 1:e1e reporcel for the sc.13 Second, the scale is too general and too global, and thus does not permit rating of specific counselor verbal behaviors. And third, like the other scales reviewed, it relies upon subjective impressions. During the 1963-6@ academic year the Interpersonal Process Recall (IPR) Project members develOped the Counseling Process Effectiveness Scale to measure counselor trainee effectiveness. IPR is a new technique in stimulated recall methodology designed to secure the maximum effect from the replay of video tapes in counseling through emphasis on the visual element during recall. This technique was developed during the course of investigation into the dynamics of empathy at Michigan State University's College of Education by Kagan, Krathwehl, and Miller (1963). The Counseling Process Effectiveness Scale was developed by IPR project m-nbers after an unsuccessful search for a suitable scale. It consisted of 33 behavioral and.feeling tone items, a global rating, and a question asking the judges to write adjectives or phrases which.described the trainee. The 33 behavior items, e.g., "the counselor paces the interview inappropriately," were each rated on a four point scale with one end of the four point continuum corresponding to ineffective and the other end corresponding to effective. A few of the items were taken and modified from Andersons' Interview Rating Scale. The global rating was adapted from Rioch's scale and.was scored on a representation of the normal curve with baseline divided into eight equal segments. The judges were asked to rate each counselor in comparison with counselors of comparable experience mm - WU- in order to establish an absolute rather than a relative judgment so the general quality of the counselor as a product could be evaluated. On the third part of the IPR project's scale the judges were asked to indicate the main factors, in order of , importance, vhich led to their particular rating cu‘eaeh counselor. As part of the development and progress of the IPR project, Ward (1965) used the IPR technique to compare the effects of three different methods of education on counselor trainee effectiveness as measured by the project's scale. Fifty-four hichigan State University Counseling and Guidance trainees were divided equally into three counselor education treatment groups: a video recall, an audio recall, and a control group (standard counselor training). Ezch of the groups had three thirty minute counseling sessions with high school students. The video recall group had each of its counseling sessions video taped and replayed for recall purposes with an interrogator. The audio recall group had each of its counseling sessions audio taped and replayed for recall purposes with an interrogator. The control group had its counseling sessions audio taped but received no stimulated recall sessions with an interrogator. At the end of this training each of the 54 counselors held a counseling session with the same coached client (a college student actr-ss). This session was audio-video taped and used as the criterion for evaluating treatment effects. Hard employed three professional counselor educators s judges to rate each of the 5% counselor's criterion tapes for counselor trainee effectiveness using the Counseling Process "v -/-_- Effectiveness Scale. For each criterion tape they rated two separate four minute segments. One four minute segment was rated early in the interview session (the first to the fifth minutes) anc. the other was rated later in the interview session (the ninth to the thirteenth r-damtes). Using this scale Ward found no sigfificant differences between the three groups in counseling ability as rated by ‘he three professional counselor educator judges. Ward found that the three professional judges did not rate the 51+ counselors participating in the study in the same way; their ratings often differed markedly from each other, and these differences were found to be statistically significant. Based upon a two-way analysis of variance, the range of interjudge agreement between any two judges ratings for the first four I':Iil‘fll’t0 seguents was .31 to .46 with a mean of .36. The range of interjudge agreement between any two judges ratings for the second four minute segments was .52 to .61 with a mean of .56. These low objectivity findings suggest that the Counseling Process Effectiveness Scale may have questionable utility. Kelz (1966) constructed the Counselor Effectivenes Rating Instrument, a measure for assessing counselor trainee effectiveness. This rating instrument consists of eight major categories to appraise counselor effectiveness in interview situations. The eight categories of the scale are: (1) Appearance, (2) Expression, (3) Relationship, (1+) Commnication, ( 5) Knowledge, (6) Perception, (7) Interpretation, and (S) Ten-dilation. The subdivisions of these categories represent personal (relationship) and technical (professional) old-fonts of counselor effectiveness which are I)” .‘- - (L " \ rated on a five point scale ranging from unsatisfactory to out- standing. Accor "1 g to Kelz, this scale was developed by collecting effective counselor characteristics from several references and from discussions with experts in the field of counselor education. Iielz selected those characteristics f or rating which he considered to be relatively independent of the nature of the problem and the 1’:10'U10Cl of counseling. The counselor subjects used by Kelz to test his scale were tl'firty members of a I‘Iational Defense Education Act Guidance and Counseling Institute conducted at the Pennsylvania State University d111‘ing the academic year 1960-61. Standardized interview situations were developed using student actors who, it was felt, cmld meet the standards considered necessary for the portrayal Of high school counselees. Two interview situations, each em- pha sizing problems commonly faced by secondary school counselors, V'Tere developed. One situation focused on personal factors f-‘J- cilitating counseling relationships (interview S) and the other eP-lfphasized technical proficiencies such as the integration and £3“Iblalication of data (interview P). During the study a panel of six trained professional judges from the Permsylvania State Urfiversity observed, via closed- circuit television, two 15 minute interviews (interviews S {11343 P) for each of the thirty counselor trainees. Ratings of counselor effectiveness using .Kelz's scale were made indepen- dently by the judges during and directly following their ob— SE>J:'vat:°'-.ons of the interviews. The correlations of interjudgo a‘LII‘COI‘ECITt on interview S ranged from. .23 to .77 Earth a mean of -33- .53. This conpared to a range of .21 to .65 on interview P, with a mean of .455. The average correlation between ratings given by pairs of independent raters was . 50. The average agreement of individual judges with the pooled judge ratings was .78 for interview S and .73 for interview P. The judgments of counselor effectiveness were correlated with the rankings of the counselor trainees made by counseling practicum supervisors to establish the validity for the ratings. The correlations obtained were .42 and .28 for interviews S and P respectively. Although Kelz's Counselor Effectiveness Rating Instrument ajppears to be useful for assessing the counseling proficiency of counselor trainees, it lacks favorable test validation and is characterized by low objectivity, and thus, low reliability. Stitching from the global and four to five point continuum counselor rating scales, Rank and DeRoo (1965) developed a forced Choice, check-off Counselor Response System (CBS) Scale for the IT‘Leasurement of each counselor statement along six dichotomous dLimensions as follows: (1) Affective-~Cogrdtive Content, which indicates whether or not reference to affect is present in a C=lotained a range of reliability ratings on the scale's six c3:111ez1sionsfr'..:o .75 to .99 with marginal reliability resulting Oh the Rest1ictive--E:_, ansive dimension. Thus, in irrit' a1 .JiDliCJtiOl'l to audio taoes, Rank and DeRoo have demonstrated Very f‘avorab-..e interjudge objectivity and reliability across the scale's six dimensions or the counselor rating scales developed to date, Rank and C‘I.’ 'JJ' Deloo's 013 30.10 aooears to be the nest obgective, it is “no t. ‘1 .) O , 1 E ’3' 0 nos; specific in the counselor echo -viors i one of the least tied to any one couns eling t1eoqy; and i r the '2 st cozzparative si' 1' >licity and ease of use by raters. ‘escriptive rather than an evalLative ‘-J L}. a l f. ‘ because its dimensions do not neces s |_) the counselin r negative characteristics for it pay be limited at this stage of I:>c3haviors it r .3 , for evaluatin; counselor educatieh (1 CV e101 tent ..) “rims and counselor trainee progress, and may have greater ’n research as an ’nstvunent to identify counselor 1lress A“ " _0 w Q - o ’ 0". 00-10. 1210?. (lfej) h s demonstrated the tthe co11dition 01 0.1) nulzs presentation of naterialt a rater has a s b O <+ 1a -1 ~ (“J-.1111. O - l‘4 cijJTOCt upon his rati;1gs. he presented audio, visual, ant of a nuxDer of Ward's criterion tape seguents to 'tLl iree separate groups of four judges. Each segment was rated L‘fi‘I \oeaLl" 1‘ ti 1e "udges of each group on each of fourteen counselor J ”31'12’dwie,‘ along a four 1oint coz1tinuun r-n i1” , ’KDlriablés; e.g., 'e:ri’xremel;,rT scored 1': J. T'L'ie tion 1 inpreceptible” (scored 1), to conai ‘1.“ {“Krerage inter-rater QSTOOHGHD value for thee audio-viM1al r'e inte1-rater agreenent for the visual consition ‘K? A .2. _ - . A ‘1€1s .7u, the averab ‘»‘ ‘- (n . ,3 .'-'1 -- a ’ —' ' - ~ ‘1‘ ‘ 31s .Lw, and he average value for th audio condition 1;'as ,uq, 1 p flS‘to the visual node of presentation lca he" inter—rater agreement than he audio mode because it he; ' 17"" value. 1 ~ . «m ”71‘8",“ ~' ‘3" ‘J.'. _ L.) kl... .L ,-~o~.. L) terature indicates that H. ‘ revieV'of the l ’) c -J("_ velopmental studies of the past have failed to produce an objective, and thus reliable, instmtcnt for evaluating counselor behaviors. flthougl'x a majority of these scales appear to have good face validity, they lack“ objectivity. Agreement among judges using these scales is poor. Although attempts to objectively measure counseling inter- view data date back to 1929, counselor rating scales did not become validated until the develo; ment of the Rogers' Relationship Inventory Scale and the Anderson Interview Rating Scale of 3.962. Both of these scales had some research validation, but Rogers' scale was too closely. tied to his own counseling ‘vi ewpoint, and the Anderson scale did not draw broadly upon various types of counselor behaviors, nor was it designed to Duke use of viwal cues available now through video tapes. Both S cales presented rather cumbersome rating assignments based upon f Our or five point scale continumtzs that were rated according to the subjective impressions of clients. These two scales, hits the Idoch Scale, the IPR Cmnseling Process Effectiveness scale, and Kolz's Counselor Effectiveness Rating Instnment, which f (allowed in development, are, for the most part, global type rating scales. The IELOCh scale, developed after the Rogers' and Anderson 6'2 cales, uses professional judges rather than clients as raters. This P. s also true of the Counseling Process foectiveness Scale and the Counselor Effectiveness Lia-ring; Instrmient which followed ‘1‘ ‘ -, a V o o '! me 3100.» scale in development. Follox‘nng the fonaat of tne v) . hOgors" and .mderson scales, these latter two scales present Jo‘ - ‘- - t. r! (3 “fir-A \« A ‘ ‘ 1" V ‘- .‘ '-":‘ rather cumbersoue ratin as Signnents sod u)on fear or live point scale continuuxs. In addition, they are not very objective and .13 o: h“ .1 .1 r! ‘ "“1 r 1 r v f‘." fi ‘ r1 . ‘ rellaolc scales, and they suffer frox vclf meal valilatioa Of “"7 the counselor rating sea les revie"ed £10 the L..-'- . . czounselor measureficnt literature, Rank and DeRoo's Counselor ' ' - ‘J' u‘ fl A ~ ~ “’- . ‘3‘ w ~ 4‘ ‘~ 3 J' . ‘ ‘ " files pens SJSUCJ uCulC appears so oe the hose ooJecoiJe and zausuable scale for neasur :3 coxm selor behaviors. Unlike the CJLimn'scal es ernnined, it is least biased toward any one coun— :5celin' “~001‘23 it is not ve lue laden; it deals with specific f \_) :111d obs rv: Talc oelavi rs ratl er tn; iglobala and im ”erred <3<3nstructs' and it presents a rating assign: ent ch aracterized '13:; sixplicit; and ease of ere ‘tion However, the CRS Scale fiL:3 designed to describe, rather than evaluate, counselor verbal IJehaviors. It neasures certs mi counselor behav’ ors and {Ittenpts to avoid placing a value as to their desirability in (counseling. T't‘ *3 pos‘ible e: :ce: tion of the IPR Counseling Process ’bJ-A. LEffectiveness Scale, and Kelz”s Cou.ns elor Effectianess P iting :IW1S crunent, all ‘ escales reviewed were developed primarily to I:easurc audio taped counseling behaviors and are unao‘e to talce dvrnoe e of visual ucs available on video tapes. Visual cues ppecr to enhan cc interjudge arfi elent and thus add to the C37je“tiV13y of a counselor scale. " Lselifi; Elevents Co: on to Various Counselinr Practices “his part 3’ the review of the research was done only aftel 9 video taped Verbal Couyrelfi_n: behaviors were collected and tested in an effort to deterziine the cormon counseling elements or dimensions characterizing these behaviors. This review attez-zpts to specify those elements of counseling which are common or relevant to all counseling practice, regardless of counseling orientation. suvmarizing the research data on counseling, 1 “the "Review of Fxhlcatimlal Research" (April, 1963, pp. 181L-185) concludes that the trends in the data of counseling research incticate that experienced counselors probably produce better counseling results than novice counselors and that most experienced counselors choose their 1:1ethods to fit the counseling task rather than a stereotyped counseling orientation. Fiedler's findings (1.950, 28) suggest that the better trained therapists of different Schools agree more with each other than they agree with less ‘bI‘ained therapists within their own school. Fiedler's findings (1950, 29) also suggest that expert therapists of diff eling Orientations formed similar relationships with patients. In Cliscmssing ccnnseling process Cartwright and Lerner (1963) point out that frequently studies which compare the respective Jpercentages of successfully treated cases suggest that some common elements exist among the various approaches to counseling, I‘egamiless of theoretical orientation. Truazc and Carlchuff (19610 mention that an examination of the mar; foztmlations designed to describe successful and effective counseling reveals such common elements as the .‘E‘ ellorfing: re 1. The abi 'ty 0 the counselor to accurately understand the client. "f3 _Jy- . The ability of the counselor to communicate this understanding to the client. j. The ability of the counselor to be able to accept the client while being integrated, mature and genuine within the counseling or therapeutic relationship. CPhey'note that these elements of the psychotherapcutic relation- :3Irip are aspects of the therapist’s behavior and are common lea]_enents in a wide variety of psychoanalytic, client-centered, .gzrqd.ec1ectie approaches to psychotherapy and counseling. These :z‘€>curring themes are emphasized by psychoanalytic theorists such sans; Alexander 1948), Forenczi (1930), Schafer (1959) and Halpern aizdxi Lesser (1960); Client-centered theorists such as Dymond (1949), e712n4rard (1959), Rogers (1951) and Snyder (1961); and eclectic 1:11130rists such as Rausch and.Bordin (1957), Strunk (1957), and S trupp (1960). Truax and Carkhuff (1964) indicate that another aspect of 131343 counseling process commonly pointed to by these approaches IJJCIS been the client's exploration of his feelings, his values, 11EELS relationships, his fears, and other concerns. Host of -tiflese approaches, they add, consider the client's self-exploration ‘tLCD be one of the central happenings resulting from the process c3’1."psycl-lotherapy. They conclude this self-exploration is 1Tlc>st likely to take place when the counselor communicates EL(aceptance and understanding to the client. Based upon their <3vrn research and that of others, Truax and Carkhuff 1964) ifeel that the findings of importance to a theoretical model fOr evaluatiig psychotherapy were that patients who explored t«leir feelings and concerns were and more throughout psychothera; y also demonstrated the greatest constructive personality change, while patients who underwent very little self -e:-.jfl.oration throughout the course of parchotherapy showed little constructive personality change. x‘uiother crucial variable for succesle psychotherapy implied by the previous practitioners of analytic, client- contered, and eclectic theory has been that of concreteness. They all 2:39.50 references to specific problem areas and affect and regard abstract interactions as non-therapeutic and even defensive. True}: and Carldmff (1964) note that, "...perhaps one of the most useful learnings is that the cases lfigh in understanding, high in patient intrapersonal exploration, and 17:1. gh in case outcome seen heavily loaded with specificity or concreteness of interaction." They conclude that this seems ‘to serve three important functions: 1 First, by enmring that the therapist's response does not become abstract and intellectual and, thus, more earlotionally removed from the patient' 5 feeling and experiences. Second, by forcing the therapist to be more accurate in his understanding of the client. Third, by influencing the client- to attend with Specificity to problem areas and emotional conflicts. .Smmmry A review of significant counselor behavior literature strongly Suggests that the counseling practice of experienced and trained Counselors, regardless of theoretical orientation, is characterized by corrion elements. Specifically, these coiugzon elements are that -"r’..'-- \ e:q)erienced and successful counselors (l) sensitively and accurately understand their clients and comrIunicate this understanding to then; (2) canmnicate acceptance to their clients in a genuine and integrated manner; (3) penrit and encourage self-exploration by their clients; and (1!) deal specifically with the important concerns of their clients. A“: I ~ fife ‘3 -I'P ‘J.-J..LJ.LA14l‘b .L .L 31‘1“." ‘fl (xv-«fivvfi m HIV WA.) 314 ..‘J-c—bel ..JL-‘L The Si: ingor steps followed in the dcvele'.):1ent of the 1‘1-7 Scale (Eigv‘c .., page 1‘3) are oi scu ed here. These steps we e: ‘ 1. Sb ‘erea ti on and co:".pilation of O.‘.‘.’.’::LO-VZ‘..C‘1.CO taps . co .‘.. so -.or behaviors. 2. C0. .30, OL'ILzation of 5110 observed a_1.1d_-'Lo_~.;;,100 taped CO~b‘\:1\JO--Ol‘ 1301111112103". 3. Organ." nation of categories into a scale format. I, .- " ‘ ‘5‘ If. Jimj_):Lj..0 4‘ e: t' couns elir ion of tae scale to c- -e reti1.; of am‘io-yfi .lee 1g? tapes 5. Definition of scale co.;'1e1'1sions. I1. Develo 1.:ent of a scale scoring S‘fs‘tej‘. ,v-n‘ - 1r 5 a V 1-10 --.-. Scale as developed for the purpose of co :parxu ' tge .A essence 35"”[301’113 of 11.1‘1. cand. dates t-I'Lth those of Ph.D. ca.:e1";fi.e"t . T: “ ""3 'n 4°1\ ‘: DI' fie . "\ t‘aé-H. '7 " N fiL\-‘ ‘7‘ "- J': I’ ‘3 peev-u.Lc)Lz O... L: .O (LLL-_01311CO 3.1.. -.-L‘LL."1:; 1.». I‘C.:...IO.L1. 4.0 C.-UOCQO~U-LU-; ° , .Lj.,..'- .l.‘. .. . '1 '. ,. m J.‘ .... l‘ -L‘ -- 1 . " . . ‘ -. . is Lucio the co17.-1:,.,_1.inu -.aote “1.53 0.. ..ae t. 0 Classes 01 coanselers ~'. ‘° 1"" -- ‘V- .1 ‘9 .... ..,‘ 1 V.. 1-. . ‘. ° . J- '- . J- alll Ci-.. -.-‘. _.-.1e scale (_.C‘."Jlf)73‘.;~.. 1:. axis research is an _-l -: ‘r.-»‘ J" . I“ v “\ 1 ‘.\ . J‘ . - 'c --- ~ ‘ . J- to .LCAU-.V1-—:, 3-1.. OwJGCol.’91J . oasure Specific an; relevan. . “'14.... . 'n‘l- . .3 ° f. . .. .L . . 4.1 coun:.e.-z-..g v0.3.1...oles ..aicn see. to disc-1" 1.:in..te 1e“:'.een LA-C .1 .-~,. ‘ ‘ I'\ . '.- 1‘ -I > A. r‘. N h ,‘. ‘ J. ‘ 001.12130.5.1.1.‘\J l)Cll.'.V-J-Ol‘ Oi _'...-.. .-.: 1;. l..D. C..Y‘.-.Q.Q.0.L,OS. The first step in the development of the A-V Scale 7.11:: to observe the "“L~'O-".’:-‘LC‘O ta )ed counseling be} aviors of Loth " 1... , . ..1- 1,..." - , ,. ~ - . 1-10 \lCVOlOHI-O-:b of this scale .10..» eased prinaril upon (31133.3. - 1 1 1,- U .sC.’.LU. ’b". -* 3/4,- -43. 348 moyémm mSmmzooo >4. 1... H 93mg D a... . .. n . nod mo 9“ 2H 2H d— m— in «H 6.. 4H C— a a N c m d r a a H N n q Joule w gum « 8 30mm we a u @333: «.5303: 1.3“an— .ruov§.na show: 1.53 1:02 Inga £85323 umuonwom peace—50 , III "3.5 Boo—.35.." "out; 35m 95.53 umzommnm MOH>$fin MQHMHBO >14 n: ot- cl . .. . t' I '4 "' I U .'.al I. n... Ll --.-,-- ‘ 05111; 7.0: "Les 0.1.6. “11.3. canafidates. Audio-13-1100 counseling; tapes 0 of t-1ese 17.13 classes 0'1- , counselors, ave -3- :‘1-11010 at "he 13301135111 State 1 Diversi- C0---ere 0f 311110.11; 3.:01 010 013.5crve:.1 The audio-"1130 9 001121303111; tapes 1030 of 0011-.selors 0131331113 out personal-social 11:001-310.: 1.1-0.1310 1.13-1.11 clients at IicEfi-;;an 51:0. be University. .' l f. I .L p L] w ~ 0 I‘ J. thee 0 03:10:31.1? 3-:-“01s. 1110 1:13., 17:13 11110 .01“ 0.10 1:101131710rs o- .11.: V ~ ( .0 I. 1 ‘0‘ l‘ .. C ‘_L .0 _‘ 'Q a ‘ _f‘ I‘D-.3. 00.11-11.10- and 0110 13.01, 1.218 1.0.00 101“ the 0011a.-.0 :5 0... l J ,- 1 0 $3.... cr11--.1..f_.1;cs (11301.3.-- :1, p. 101). For each list, the Toe-'10 1.13 .3;- Irere 00‘ 1313.108. under 03.1.1101: an Affective 11011151115 01' :1 '-J -.H: J.-." -r ‘ ‘.‘, .- ,. . ”J. N J. . ° . 1307,11 . 0 3.0.1..- u. ”3011-1301013 spate-e111,: 1. 1:1.01-r 010 1 ~1~ed t0 the e-gien'Us 3005:1113 or alibi-Javier: were 1' sted 11111101: the Afif‘ective L - \_) cool- --\l~. kt.-- ’V‘ A " '. ' '~ - - ‘fi .1 ~ 1 Q-s v '. q ‘r- r‘ “- " 1.0.1-131.“ CO‘U.:..'ZO--OI‘ state .10 .‘t5 .;-3-I'.c s e 0-1 s0 leer. my 13030101100 v~a’.~ ’L>~. ‘, . J‘ 1 -t ' ’*u ~v - .1 . .n - . - to or 1:000 -.=-0:'-0-1 0f tzlc 0.1.10; 10's 3001111955 .030 113.011 11.111 0 '- n ,-.v.--, , 1 -.-,, 1. . .- '1 - "J. . . .0 °..: n11... t;1e .0;,.~1.-- 0 ‘ e set-113.11). 1.0 1.1.0 1.1.0:. more p11? sed 01 01.1.1.0 1 , (10130121911170 c..'1.ss:'1-f..01‘.cie11s 0f 1’30 1011:1033 ‘1 0:51; fr0311(,‘.1‘bl" 07"“5‘Vn‘ 'L - . 'f‘ ‘w n ‘ "‘ \~-'\r~ .1 - ~ 1,0 00 031cm.1*0r;1;01'3-t10 0- 03.01. 0-11-5.5: 0- 00111.5--0111. f‘nJ‘ ..'- VA ”6"". r“ ' 2.“ ’YNr‘n'n0rf‘t’J -'I(I 10,--.-.:€.-1J-1.‘..)JA C ‘J.'.O '_ .'._).,_ .‘ z A A __A._ . ' , w 1 n .- ". '1 . . ..10..0-.3 . '30 Ts. 1e . .0u-.se_-0r 3.1.13 3.:: w1‘. ‘ m-.. - 1 . 4: , - 4.x. .. a -..e 1.11---.~.-'1.0‘ 0133033.---5201 co.._'_):1..1-e:1f0: 0110 1.1.1.1.-..- VV \“—. - u 'Ir‘~\nr‘_ J‘ 'L‘.‘ (V -\ v1‘ ‘31 A" f ‘A ’L‘. 71‘ T‘ fl‘flf—V (‘AJ‘ \f‘ C‘\ .oCIJ. t.) CO. .N/I.... «_o'-.‘. L‘O UA.O;)('I CO. 3... L‘v‘sas ‘ .L b-10 - LAQJ. CgL’L-egl.»LLL\J'/U . 3.0 .. '-.- _,-,-1.'1 - 47.1.0“ -. 1- ., 0 ‘I .. n ., ‘ ° . - m1 1101030315140 .-..-.er1:-1ce 00051. -0 013-00111, 11:01.1 “0111-55 00 1"11‘18021. 1-10 .1.-- . .0 ~ .1 . . -'~ - _ 4.1 ~» 1 .- -, - . 4-1. W905) 0~ 9'3— C‘V—L‘ 35 Closer-1133-11; 0-110 1mm. 43-11. .8 1.030 01:13:30 -.‘ . ‘S V" ‘ .‘l-t J“ (- 6' 'c‘ A‘-"‘ v ‘- ‘-. '3“ «u. <. J“. ‘\ .s ,- ...1:Lc.. .030 --ssexl .1031: free-110.101,; 101.113 0110 (309.11-01.10 seeds-11”. w- .'--'..- ,.'. J.“ .!.,_ _‘n V j - ‘ I‘. 0-“ 7 ‘ '1 ~ , ' 3 1. .1-11 CO.-.0--1.0\5, _..3 0' )1“?- OJ. 00.10.131.011) ..SCI’IL; 1" f‘ 110 911...). 001111-1100? .l.- A o ~-,_J l M9 x -.p~' "-.. .A1‘ I. J.- . —.—.. n -. 0 , . ¢ .“ . .- '1 -~—.*. . .3. . .110 00. .41.---0.-...-5~- 0. 0 10.50 03501'V.-11.1-011s Wis .11- .- 1.111 . 1 1 1;. 0 ~ V -145... \ were those which were most frequently listed under the Affective heading. It appeared that the two Classes of counselors were apparently differentiated on the basis of affective and cognitive responses. Because the Affective and Cognitive categories encompassed a broad range of counseling behaviors, additional categories were sought for finer categorization of the compiled counselor behaviors. The review of the literature (Chapter II, pp. 37-41) suggested that responses which were "understanding," " specific,"A and "exploratory" were common and important counseling behaviors characterizing the practice of many counselors of diversified orientations. These three types of behavior designations appeared useful and relevant as categories for classifying the compiled counselor'behaviors. Based primarily upon the clinical definitions of these counselor behavior designations from the literature, an effort was made to classify the compiled counselor'behaviors as to whether they were understanding, exploratory, and/or specific. Briefly, understanding is defined as the counselor's ability to communicate to the client the fact that he knows what the client is feeling or'talking about; exploratory is defined as the counselor’s assistance and skill in helping the client to explore his feelings, values, and other concerns; and specific is defined as the counselor characteristic of being concrete and getting to the core of the client's problem areas. A majority of the behaviors compiled for Ph.D. candidates seemed to appropriately fit under one or more of these categories (indendi: I, a}. 135-137). A majority of the beha.iors co piled A - "‘ 3 ' 3 r- fir ‘ '3'. I‘ ‘v‘ '-- ' "A ‘ . J‘ W- 9‘. ‘ ‘ v" " Io: t..c __,.... 1.1 J. Lat tes and .-et see. so apropr'iately 4.-.: 111-103? A $1) (i O r 3 F I b C) O E} )3 O O O I.) C a 9 0' W o (3') 0 CO . O . A '1 ‘ ' ' ’1 A. . w ~*er, bf uSinn -nonunee -ou““k‘“L, n ' fl: ' . 1 -.., . --,- . (..- A. J. 0 .. .I.‘ \ WORSUOCi_lC,' “f” me o Mlo~ato beha.j r (as gnaeio-u, ..a ‘O \ ‘ . J“ " J“ '.‘ ‘1 ‘ - 1 o - ' fl t 3 -" ‘ - ' . ‘ . I l - I N ‘ -., -.v.-x., O. '- 'J ‘ '31 ’3 CU. u 10 . o... s . I r (H b) L)’ v :- ~ ' \'- ' v. " -. ' 3: \ " "‘3 -"-.~ - 3 ‘ . >3: - - fl ‘ - \ 1 “(N ‘1’ l,/\\ l I- ‘v‘, n" - ()L'r') a. .‘. 'J ~‘.. .1.\-KI A) ~ ' 0-...” I. - _.. -.K} A .‘1- ;.fl~_3_‘.-_l_,. ~-.\L1 . .."--\.."-.:. I.) / . ~ .6 -.- \ _L _.- ., - .. U l - ’ a s. ’ 13_u:;-;wfr;l;h.".. ‘a;inee as the coan selor's failure to ‘13:, -’\"'\'. Win 011 .f“ 11:1 \fi‘“ 3 4") +1.“ 1 . Y‘t J‘lxq'i‘ \o '~fir-v"rrw‘ v7r‘.")"L t“(‘\ lx'CV -i.'.s1v «L C‘)- _.Ak‘ls -(w UL! Ll Vné'xl PhlolOoA UA‘LHU .\I :3-..) tolLLskJ ‘1-- -,-"‘ - ~ 3. , ~ -\r- e. 'L - 4' . a r . w‘. ‘ leze one in -uslmng; none;;eloraeony is defined as the counse o3n r’s :7. -.. .v. w .1 .- -4 '1 x.- .. - a... faila-a uO aeup .ie clie ant to enplore “l3 feelings and concern“, W and nonspecific is eefinee as the eouzselor characteristic of being vague ang.keeping clear of the central corcorns of the client. :13 .-. "I 0 a I'M A '0 . . v .n . r” .‘ v “ 'L f ' f‘ Lao e-. “b categories of affective, he itive, bnoerseanding, an>rn ntazy, Specific, oriiecific, explorato 'y, and effective. ra'inQJs were avora red across the three jud'res to give one rating per response for the Effective-Ebneffective diir1ension and each of the four categories mentioned. The initial and ten~1inal ratings for both the Supervised and Nonalpervised Groups were cozfzpared for siQLificant differences on all five dimensions of the A~V Scale. The dependent "t"-test was the statistic used JW‘ for: 12.1:111 3 131 1s comparison at the .05 level of confidence. Summary The no thodologies of the Scale Disczirfination Study, the Objec 1.°vit3, 3111'? , and the Sensitivity Study were discussed in this chapter. These three studies tested the three hypotheses set forth in Chapter I.- The 1:1ethodolegy of the Scale Discrimimtion S‘bidy was designed to test the hypothesis that the frequency of affective, understanding, specific, exploratory and effective responses in initial personal- social problem counseling interviews will be significantly rrreater for Ph.D. candidates than for I..A. candidates ber'inning their practicum e:q>eriences. Three trained counselor juwes used the A-V Scale to rate 20 responses from each‘of ten 30 minute audio- video 1.2 .ped co1.;1seli11g interview sessions. These ten taped counseling; sessions involved five 11.. . degree candidates and five Ph.D. candida tes 0:131.ng in personal-social probl e.1 interviews with clients The rat'ng scores of both groups were coar1pared on the five (L1? .01 ions of the A-'.' Scale by means of ”t”-tests. Heyt's analys s f variance 1. cthod '1 'as .121 0 applied. to the ratinr scores to establisn an ind-0:: of iten objectivity. The Iet 1odolor;y of the Oojectivity Stu :13] w .2. designed to test the 11;:101.th3.8 that the 311-! Scale will have adequate objectivity for Ch..1.1.0“8‘.’11.--.-t1LnU between ‘12.!1. and Ph.D. candidate responses 011 each of its five dj.1:1.s1e1 ion .3. Three trained counselor judges used. the 1‘.—‘1' Scale to rate four minute segm- nts from each of 53 audio-video couns cling tapes. Forty-five of these tapes involved 12.11. candidates and eight involved Ph.D. candidates. All 53 counselors were engaged in 21 i1r'1 1.3al intervicw with the same coached client. Hero's analysis of variance was applied to the rating; scores to establish two sets of objectivity and internal consistency coefficients. One set was an index of item objectivity and internal consistency and the other a global rater objectivity and internal consistency. The rating scores of the 21.11. candidates were co-zzpared Ui th those of the Ph.D. candidates on the five dimensions of the A-V Scale by means of "t -tests. The methodology of the Sensitivity Study was designed to test the hypothesis t‘ at counselor trainees who receive professional supervision '1I1I121- ch focuses on examining affective, understanding, specific, ande ::11'L0ratory responses wil 1 receive higher frequency ratings for these categories on the A-V Scale than Iill c0111.." lor trainees 1.7110 do not receive this professional training. Ten I1DEA Gui (121100 and Counseling: Institute trainees partic:'1.p.1.ted in the study. Five of thes e counselor trainees belonged to 2. Supervised Group which received profe ..::°101-12.l interro- ga. on .1; :.;ea;1s of 2.11dio-v1deo time re 112.3; of their counseling interviews. These interrovation sessions evaluated the effect that affective,'understanding, specific, and xfiloratory responses woulc have on the counselor trainee's clients. This group received six of those interrogation sessions. The other five counselor trainees belonged to a Eonsupervised Group who also were exoosed to the audio-video tape replay of their own counseling sessions. However, their interrogation sessions were carried out with the Client, rather thanmthenselvcs, by a colleague team nefiber. These interrogation sessions pursued no particular emphasis during the audio-video replay. The pre and post tapes of the five trainees belonging to each group were rated by three counselor judgeS'using the A-V Scale. For each group, the rating scores of the pre and post tapes were statistically conpared.using dependent ”t "t-tC Sts . ..v o‘ The neonats of the: Chapter IV are repo; 130d here three sections; '1 discus Discri":'Lnation S'bidy, a discu I " ~v-f‘ov—a‘ _ -.. 41.4-1.1.1“) sion 01 the rcsu now-‘7 mqfi~j ‘1-—..4 L4- ‘1 1r '3'“wa mr“ Luddabu ("'1 VJ. ezthod ole. ice.-- procedures described in J‘ J '11. of the Scale ssion of the romlts of the Objec- tivity Study, and a (U. sous on of the results of the Sens trusty 301103“. Results of the Scale Discrinination Studg-Jr The hypothesis tested by the Scale mscriifizmtion Study was that the frequency of affect, and effective responses than for 7.2.5.. candidates. which woul CL r01. ”lect on the so next section. will be greater understanding, speCific, exploratory for Ph.D. candidates 1lso, objec"c.ivit3r data 1' s collected eond hypothesis discussed in the Anajsis of Sea- le mSCl'lI tinat ion Data Table 5.1, page 72,511.111r1rizes the statistical "t" amiys's of Appendix: E, p. 132. The analysis presented in this appendix compares, on 00 ch 0-“ the five dimensions of the A-V Scale, the scale see; 1 s of the five Ph.D. candidates with those of th \ 11.91. cant’aeates. it can be seen from inspection of this Table -w .l. .0“. . >0 ‘ ‘3‘ ,L: ‘ I ._ that t.- 1e.‘o is a spa c.1_st:1.eall‘; (51.10 :zio:1s 1 steel one scores a J,“ ' l‘ r - '1... '1 ’.‘ I. {’1‘.‘ 01. the . ._-.. 01111.1.(L1tes. 11-0 significant difference on all five of the Ph.D. candidates and these responses of the Ph.D. candidates I“ -70- were nere frequently rated affect, understanding, specific, exploratory, and effective than were the responses of fine L.A. candidates. Table 5.1 -- Camparison of the Kean (X) Scores Between the Ph.D. and X.A. Candidates on Each of the Five Dimensions of the A-V Scale " - Y! A _ ‘ ‘Ar'\ (3 Judges, 5 Ph.D. counselors, 5 h.a. counselo-s/ Aff.- Underst.- Spec.- Exnl.- Eff.- Cogg . l'onunderst . lTonspec . Hone r’n P:1.D. J 35-2 3&2 35.4 34.11 58.8 1.0410 X 2.3—0? 23-00 20.2 20.2 37.]. t _ ratio 5.9** 13.4** 7.nx* v 5*» 5.9** necessary: t .05 = 2.31 necessary: t .91 = 3.36 **significant at .01 level Aff.-Cog. = Affect-Cognitive v Undorst.—Tonunde:st. = bnderstan‘ing-Ionunderstandin; ll 3pcc.-Tonsoec. Specific-Ionspecific 'IL... .2. 7 l . -'. $011022} 11 . = Ezploratommlionexgfloratory I - ‘ Eff.-Ioncff. = H fective-honeffective These findings support the hypothesis tested. They suggest that the dizcnsions of the A-V Scale are able to discininate between the responses of these two classes of counselors. These results, then, indicate that the scale has validity for this purpose. fauljsis 0-; lnterjgdge Reliabilit" Data noyt's analysis of v.riance method for estinating reliabi_ity was applied to the scores of the five Ph.D. candidates and the five ‘. l -.n "\'I‘ A‘L '3 a: - "1 . a '1‘ Q— m, ‘ Q r‘ - 1‘ \' 4‘. ~A 1 11.1.. 051.1... ’ .uCS l‘c. uCU. 4.1."; lat-LI) Solid-u, . .L..lO Oil-ta all t CO. L11.tC. LIV-LL _ ~. -J‘) .3, .v.1”o ‘ J. J-‘ o n . '1-- o o ‘ -- ,: 0 1113-1 ‘...~'-C.. 1,-.1-1. SUE-L113 1c.l zealdsis 157130.306 are presented in O ‘ U ..a-.1 , . 1- - . . ° .2. .' ..1 -- . '1.-.. 1.,.-.1..2. ...-- “3901-1-1.“ F, 3. 130. .-1e 1.1.111 llluOl"Jllk.‘.i_;G 1elaz1- 01.1111”. (oe IeCL. 1:31.) and. itez: 3419013211 consiste: 1c, coefficients catputed 3;; t.1-'1.s . n1,_‘_0 ‘ - _ n _‘ 1 n J_1 _po. 1. _ .11111-5111-3 arc :e1ortec‘. in Table 5.2 or eacn 01 1,110 -ILJO oi. on: 1.111s the -‘.-'1" . Scale . out r A - .1,->--c d... -- Ite;- Intern-16150 Deli-'1 oilit " nc- Interzwl eons-.stencJ Coefficients Cor.a 1 to c. fro: the Scores of the Scale lecrizixmtion Stir-Ty \.,/ k3 jmlg; cs, 10 counselors .i‘f.- 'Jnderst.- Spec.- Expl... :f.- 00;. ITenunderst. Ions sec. Honor-“pl. I; 1te‘.‘d'ud{_;e hells-Lilli); .792 .816 .817 .753 763 . ens:- stenc; . 919 . 930 .929 .903 .906 These rez‘mlts suggest that perhaps the dimensions of the 11’ 31-. Scale have suite favorable objectivity and reliability l o Q‘21‘CCUVLZE1D'P‘ . lesfl-ts f the glijectivitl at 11‘" o .1. c \ N g ‘L‘he h;‘ goth-cs l .1 tested 13;; the Objectivity Study was that the a. - ficn-ln 1" 1'.1 all "“.‘Il‘l‘ .2"- ‘9‘.‘ .1317 ‘0 " -.- g L) [‘1.-V -1-—¢.- ALHV C u“~0&;v“v~ U0 0 JJOC U1 n J J. or 7 .L ’1 1ere1t~acin~ between ZS.-‘.. and. $11.3. candidate r- sponses on each of its five dizcnsions. ..;0 ob ccti‘vit; Stud; also served as a type of — ‘ nvfi ‘ "‘-1 ,-—.~ 'r' ‘ '9" - n“ . . ~-~. ~ “‘1‘ fl cross-...1.-1..-e.1-ac.s.-1-1 r v.10 Scale Discm1.i.1s. ion .1 cues, -72- Jnal;si:‘ of Intezjudge Reliability Data Hoyt's analysis of variance method for estimating reliability was anal-iod- to the scores f or the f orty-five 17.3.. and eight Ph.D. ca Iii-dates rated in this study. Two sen. rate sets of inter judge reliability and internal consistency coefficients were conjmte-J. by this analvsis. The first set of coefficients conputed were the iten interjudge reliability and internal consist no; coefficients based upon a comparison of individual counselor resuonse ratings. “nose coefficients are reported in Table 5.3 for each of the five dimensions of the .-V Scale. The Statistical analysis upon which the computation of these coefficients is basedzis presented in A3);>endix G, page 165. .. a -- Iter- Interju dgge Reliability and Item Internal Consistency Coefficients Computed from the Scores of the Objectivity Study (3 judges, 53 counselors) Affu- Underst.- Spec.- E:Cpl.- Eff.- Cog. Lionunderst. lionspec. l-Ionexpl. lionef f . Inter j'u dfje «chant ‘12; .5390 .792 . 763 . 590 .8124. Internal Consistency .961 .919 .906 .312 .92 She seco 1d se ‘ of coefficients computed U ere the global rater interjudgc reliabiliy and internal consistency coefficients based upon a conparison of the total dimension scores for each counselor rated. Chose coefficients are reported in Table 5.@, page 75, for each of the five dimensions of the i-V Scale. The statistical -75- analysis upon."hich the COHputation of these coefficients is based is presented in Appendix H, page 206. The internal cons istenc; coefficients reported refer to the naiimum likeli- hood estimate of reliability. Table 5.M - Global Rater lnterjudge Reliability and Internal Consistency Coefficients Computed from the Seer s of the ijectit'ty Study ; Judges, 53 counselors) iff.- Underst.- Spec.- Expl.- Eff.- Cog. Ionunderst. Ionspec. YONGKDl- Ioneff. Interjuaje 10113.2 . . 353. . 3:1 .335 . 697 .934 :anC r ‘16.]. Consistenc: ,th .903 .,59 .373 .977 It can be seen unen inspection of Table 5.3 that the item interrud;;e relia oiliw .nd the maximum estimate of item reliability (internal consistency) coefficients are rather high for a counselor rati.g scale. Although loner, these coefficients compare favorably with those obtained from the ratings of the Scale Discrimination o I, Study'reported in Table 5.2. The results reported in Ta ale 5.w indicate that thee e coo:fic:m nts are lligher "hen the total or glooal scores for each dirension are compared. nese findings support the hypothesis tested. They suggest that the lizensions of the L-V Scale have favorable objectivity for the rating of Ph.D. and X.A. candidate reSponses. This hi Mr- Objecti'ic” is accompanied, as :‘ould be er)ected, by a hiSh Irujsit1c>s ui4r to o: reliability. l- \J finalisis f Jata Cogparins Ph.D. with X.A. Candidates Table 5.5 summarizes the statistical “t” analysis of 1“» ”H .,. " .4...“ P1" _ .Q -84“ . F .2. ,1 a t“ :1 c‘ .h‘.‘r _gppenci. -L, e 217. 1-10 011 ,y sis presentet 1n .1...) appenoa-.. compares the proportionate scores of the eight Ph.D. candidates :ith those of the forty-five I.A. candidates on the Cognitive, -2'onunderstann‘ing;, Nonspecific, a nd Nonexploratory categories of the 1—7 Scale. A co: :parison of the proportionate scores of the Lifective-Koneffective dimension is also made between the two classes oi counselors. -aele 5.5 -- COMPCTiSOH of the Kean (X) Scores Between the $11.3. and 11.31." at -10 1'da ates on the Cognitive, Nonunderstanding, Ionspecific, and Ionexploratorv Categories, and Effective- onoffective binension, of the A-V Scale (3 Judges, 8 Ph.D. counselors, 45 7.3. counselors) Eff.- Cog. Uonunderst. Honspec. Honexpl. Noneff. 311.0. 3; 3.8 5.3 7.0 21.1.9 ,1 -: ’3 O ‘3. / 9 l 2 1.0.0. as do.) 70/ vex) 90k 50 t - ratio 3.5** 14.l** ll.8** 8.5** h.9** necessary: t .?5 4 2.00 necessarf: t .Ol = 2.66 *asi'nificant at .01 level Inspection of Table 5.5 discloses that there is a statisticallv signifi ant difference at the .01 level between the scores of the %1 D. candioct es and the scores of the K.A. candidates on all five scale dimensions. The responses of the I.A. candidates were 3 more frequentlv'ratcu cognitive, nonuneerstanding, nonspecific, nonexploratozy and less effective than were the responses of the -77- Ph.D. candidates. These results support the finc‘inr's of the x.) Scale 7 .):_sc i‘111ation Study reported in Table 5.1, and again L 10 five «’51.: 1e11sions of the 1’1—V Scale are able to discrbimte bettreen the responses of 23A. and Ph.D. candidates. Results of the Sens'tivi v Stud; The Sensitivity Study testeo the hypothesis that counselor trainees 17;: receive 111110 ‘essional s:u)ervi sion .rhich focuses on 1 53125111115 affective, unnerstandinb, specific, and exploratory responses trill receive higher frequency ratings for these categories on the -‘-.-'."S ‘cale than will counselor trainees v.10 do not receive at. s are; es siornl training. Sable 5.6 reports the i1fitial.eans of the Lionsu >erv1 sed Group 11th t_1e initial means of the Supervised Group on the five ‘0 l '1‘ eigensions oi the A-! Scale before interrogation sessions. 1a‘olo 5.6 -- Iloan (3'?) Scores of the Initial Interview Rating for both the lfonmpervised and Supervised Groups the Five Dimensions of the A-‘J Scale (3 judges, 10 counselors) .f f . - U11d.erst.- Sp ec.- ficpl .- E’.‘ .- Cog;. lionunderst. 1 'son 1300. 101102-2131. Zioneff . -fons‘ 1 10”"! so: .2 1.2 13.2 .3 3.’% 27.? Taole 5.7,:1age 73, margarizes the statistical comparisons of‘ 4-1 -,_.;.!-- '1 ..., 1 ,. _m a 4.1..» (M. b " ‘u. 14-10 1-1.1.10.-- _-eaa scores Cl LLO IT:on :upervis ed 11. cup 01010 n n- supe‘misod client interrogation with the tenzinal mean scores of -78- 05810115.) . o D (.7 _9‘1 9 _.. 1"- 5.,“ ' _1 a ‘~f' -L' ‘_ «L . ‘ 0 this are”; 14119511“ the g1ons u)ervised clienti :1te1rogatio. N The data axe conputations upon which this analysi L1 F! n m 0‘ ,1: m (D Q. ’1) “3 CD prese nLeCZ in Anuendix J, page 228. '1‘,hle 5.7 -- ConDarison of the Kean Scores of the Initial and Terminal Interview Ratings for the ‘Jonsupervi sed (Client-Interrogation) Group on the Five Dimensions of the A-V Scale (3 judges, 5 counselors) Aff.- Underst.- Spec.- Expl.- Eff.- Cog. Ionunderst. Honspec. Ionexpl. Ioneff. Llicial '2 1.2 3.2 .3 3.4 27.2 Toninal ‘7 1."; 2.4; 1.6 2.0 25.3 t - rntio 2.5 2.2 necessary: t .05 = 2.73 The comrarison of the mean scores of the initial and terminal ratings of Table 5.7 show that there were no significant changes on any of the five dinonsions of the A-V Scale. Since the means for the Understanding-Tonunderstending, Exfflol atory; onexploratory, '1) .nd Effective-Soneffective dixensions decreased rather than increased iron the initial to the terminal ra.tings o, no t-tests were neeesSarf. It was apparent that these three dimension scores had not chanfied in the nocessarv directioz. AAJHO 5.3, page 7?, summarizes tae statistical conparison of the initial :ean scores of the Supervised Group before supervised -‘ counselor into: rogation with the terminal mean scores of this group folloz'ng the supervised counselor interrogation sessions The data and co:1nutations upon thich this analysi sis based are -79.. presented in A} m aniX II, 1X1? 0 232- Table 5.8 .- Conparison of Ilean Scores of the Initial and Ter‘timl Interview Ratings for the Supervised (Counselor-Interrogation) Group on the Five extensions of the EI-V Scale (3 judges, 5 counselors) Aff.- Underst.- Spec.- Expl.- Eff.- Cog. , Ronundel st. Nonspoc. Nonexpl. Honeff. Initial I 2.6 6.8 3.4 7.2 31.0 Tonia-1a]. Li .2 4.0 14.6 16.6 51.0 t - ratio 3.3? 3.3” 6.l** 1.5.6“ 111.21' -leCOSSC-ll‘ff: t .05 = 2.73 *dmificant at .05 level necessary: t .Ol = 1.5.60 MSiSnificant at .01 level The comparison of the mean scores of the initial and tcniinal fl r1“ 8 0:; mole .85 shot: that there were stati sti cally significant .0 changes on each O.‘. the five dimensions of the .»\-V Scale. :3. co: :;_>arison of thes fi ICLings I'ith those reuorted in Table 5.7 suggest that the supervised counselor interrogation Irith counselor trainees, Irhica focuses on o:ca::°1.:1ing affective, understanding, specific, and ez-qfloratom' counselor responses, incre-.ses sigzri- ficantly the froquency of counselor trainee responses I-lilich are rated affective, understanding, specific, and er .1loratory. These ed. (4‘ findizrs "u”:uort the 1‘1'not1.esis tes '1 “\‘1\~‘QW'P N’s.) L. .L-L‘ — rue results of the Scale Discrimination Study, the Objectivity StucL, a nd tne Sensitivity Study are reported in this chapter. The findings of the Scale Discrimination Study suggest that the five dimensions of the A-V Scale significantly discriminated between the responses of Ph.D. candidates and those of M.A. candidates. The responses of the Ph.D. candidates were more frequoztly rated affect, understanding, specific, exploratory, and effective than were the responses of the K.A. candidates. The results also suggest favorable objectivity for the A-V Scale in making these discriminations. Item interjudge reliability coefficients of .72, .82, .82, .76, and .76, and item internal consistency coefficients of .92, .93, .93, .90, and .91, were Obtained for the Affect-Cognitive, Understanding-Honunderstanding, Specific-Zionspecific, Exploratory-Nonexploratoxy, and Effective- honeffective dimensions respectively. The findings of the Objectivity Study suggest that the A-V Scale has.favordhle objectivity, accompanied by favorable 11121111111111 reliability estimates, for the rating of Ph.D. and M.A. candidate responses. Item interjudge reliability (objectivity) coefficients of .89, .79, .76, .59, and .81, and global rater interjudge reliability coefficients of .95, .91, .89, .70, and .93 were computed for the Affect-Cognitive, Understanding- 1~§onunderstanding , Sp emifi c-I’onspecifi c , {1:201 omtory-IIonexfloratory, and Effective-Ioneffective A-V Scale dimensions respectively. For these same scale dimensions, item internal consistency (maximum est'uate of reliability) coefficients of .9 , .92, .91, .81, and .93 respectively, and global rater internal consistency coefficients of .98, .97, .96, .87, and .98 respectively were computed. The findings of this study also support the findings (N '1 —c1- of the Scale Discrimination Study that the A-V Scale significantly discmtdnatcs between the responses of Ph.D. candidates and those of ILA. candidates. The findings of the Sensitivity Study suggest that supervised counselor interrogation with counselor trainees, which focuses on errandning affective, understanding, specific, and ezqnloratoxy counselor responses, increases significantly the frequency of counselor trainee responses which are rated affective, understanding, specific, and exploratory. CHAPTER VI SUI'I'XRIL, CONCLUSIONS HID II-ZPlICATIOZTS This chapter consists of three sections; a summary of this research, the conclusions of this research, and.a discussion of the implications generated by this research. Sunnarv This research examined three hypotheses associated with the development of an Audio-Visual Counselor Behavior Scale (A-V Scale). This A-V Scale was developed for the purpose of making a. vai-11ble an objective instrument which would compare the res pens patterns efI aster's degree candidates in Counseling and Guidance (K.A. candidates) with the response patterns of doctoral degree candidates in Counseling and Guidance (Ph.D. candidates). Its development was based upon the clinical observation of counseling behaviors on audio-video tapes followed by a categorization of these behaviors according to constructs clinically evaluated to be relevant and descriptive of H.A. and Ph.D. candidates. The A-V Scale consists of five forced choice, dichotomous dimensions. These dimensions are the If feet-Cognitive, Understanding-Nonunders t: nding, Speci fic- honspeeific, 111lorat01 - one: :ploratory, and Effective-Noneffective (131821"? ens . The three hypotheses examined by t}1is research are: Firs t I guxat es1s The frequency of affective, undersea 1nding, specific, exploratoflfi nd effective responses in initial personal-social problem counseling interviews L1 as measured by the A-V Scale, will be 9 C1 -Jo ) ..J H) H. 0 P3 :3 Ca IJ \" gseator for Ph.D. candilates than it will be for M.A. candiiates beginning their practicum experiences in Counseling and Guidance. Second Hypothesis The i—? Scale will have adequate objectivity for differenti- ating between Ph.D. and K.A. candidate responses on each of its five dimensions. (1" o ~1_" _L1 ‘3 ‘ 1h1rd.1n;x)miosis Counselor trainees who receive professional supervision which focuses on examining affective, understanding, specific, and exploratory counselor responses will receive higher frequency ratings for these categories on the A-V Scale than will counselor trainees who do not receive this professional training. The Scale Discrimination Study tested the first hypothesis. For this study, three trained counselor judges used the A-V Scale to independently rate 20 responses from each of ten 30 minute audio-video taped counseling sessions. Five of the counselors engaged in these interviews were M.A. candidates. The other five counselors engaged in these interviews were Ph.D. candidates. Each of the ten interviews J38 an initial personal- social problem interview with a client. The judges had not seen the tapes prior to the rating session, nor did the; know the counselors they were to rate. The rating scores obtained by the two classes of counselors on each of the five dimensions of the A-V 3cale were statisticallv eoapared using ”t -tests. There were significant differences in the respons scores of the two classes of counselors on all five dimensions of the A-V Scale. The responses of the Ph.D. cmxdidates were more frequently rated affect, understanding, specific, exploratory, and effective than were the reSpenses‘of the M.A. candidates. Hoyt's analysis of variance method was applied to the rating scores. Item interjudge reliability coefficients of .79, .82, .82, .76, and .76 were computed for the iffect-COgnitive,‘Understanding-Ionunderstanding, Specific- Nonspecific, Exploratory-Honexploratery, and Effective-Noneffective dimensions respectively. For the same dimensions, item internal consistency coefficients of .92, .93, .93, .90, and .91 respectiVely Jere obtained. The Objectivity Study tested the second hypothesis. For this study, three trained counselor judges used the A-V Scale to independently rate four minute segments from each of 53 audio-video counseling tapes. Fortybfive of these tapes involved K.A. candidates and eight involved Ph.D. candidates. All 53 counselors were engaged in an initial interview with the same coached client. The judges had not seen the tapes prior to the rating session, nor did they know the counselors they were to rate. Hoyt's arnlysis of variance method was applied to the l item objectivity and internal consistency, and the other was an index of a global rater Objectivity and internal consistency. {‘3’ —\7J )— a Interjudf;e reliaeilj.ty refers to objectivity while internal consistency refers to the maximum likelihood estimate of reliability. Item intorjudge reliability coefficients of .89, .79, .76, .59, and .Ol, and global rater interjudge reliability coefficients of .95, .9l, .39, .70, and .93, were computed for the Affect- Cognitive, Understanding-Nonunderstanding, Specific-honspecific, Tvnlor.torr-1oaexnloratorv and Bffective-Honeffeetive A-V Scale dimensions respectively. For these sane scale dimensions, item internal consistency coefficients of .96, .92, .91, .81, and .93 respectively, and global ra ater internal consistency coefficients of .98, .97, .96, .87, and .93 respectively'werec mr‘.puted. The dimension Hoe es of the fortyAfive N.A. candidates were statis- tically compared'with those of the Ph.D. candidates by means of "t -tests. There were significant differences in the response scores of the two classes of counselors on all five dimensions of the A-V Scale. These findings supported thos e obtained by the Scale Discrimination Study which tested the first hypothesis. The Sensitivity Study tested the third hypothesis. For this study, ten UDEA Guidance and Counseling Institute trainee volunteers were randomly assigned to two groups of five; one group comprised a supervised counselor-interrogation group (Supervised Group), and the other group comprised a nonsupervi sed client- interrogation group (Eonsupervised Group). Each member of both groups undeleent six 30 minute video taped counseling sessions with a college or high school student. Following each counseling session 00 ch ne:1ber of the Supervised Group received a one hour audio-video recall couns 0101-int errogation which evaluated the out," 11’- effect that affective, understanding, specific, and exploratory respons- 3 would have on the counselor tra inee's clients during the coulseling interview. Following each counselin ng session each member of the N01asupervised Group either observed through a one view: .ir ror while a colleague team member interrogated his client foro one hour, or he interrogated his colleague team member's client for one hour while his team member Observed the interro ation. There was no particular emphasis to this client The middle twenty responses of the first (initial) and sixth (terminal) counseling sessions for each of the ten counselors comprising the two groups were rated by three trained counselor judges. r1hose judges had no knowledge as to which of the two groups each of the ten counselors belonged. These initial and terminal ratings within each of the Supervised and Honsupervised Groups were statistically compared using dependent "t"-tests. The results of these tests indicate that for the Supervised Group there was a significant increase in the rating scores, from the initial to the terminal ratings, on the affect, understanding, specific, exp101~atory, and effective dimension categories of the Scale' 3 five dimensions. For'the honsupervised Group there was no significant score increase, fran'the initial'to the terminal ratilgs, on these Mi Scale di 1e sion categories. Conclusion In drawing conclusions based on the findings of this research, it is necessary to consider certain variables which ight have limited or influenced these findings. First, there were the major delimitations outlined for this research in Chapter I. This refers ‘ I .9. )rinarily to the fact that the behaviors studied in this research were laboratory behaviors and that the number of counselors Observed in two of the three studies of this research were relatively'srmdl, Second, there'was the fact that the ) counselors and judges used in this research were primarily associated with bflchgan State University}, Third, there was the possible Operation of special factors which could have influenced the outcome of tne findings. This would include the operation of such variables as rater bias, the overtraining of judges, the convenience and desirability of selected tapes, and the nonverbal conveyance of rating attitudes resulting from ju r“es rating in the 8 me room. Konetheless, within these limitations, the findings of this research appear to warrant the following concbusions about the hypotheses presented in Chapter I: l. The data of this research indicates that the frequency of affect, understanding, specific, exploratory, and effective responses in initial personal-social problem counseling interviews, as measured by the five dimensions of the A-V Scale, is significantly greater for Ph.D. candidates 1n Counseling and Guidance than it is for yr 1.A. candidates beginning their practicum experiences in Counseling and Guidance. Therefore, when used for the purpose of discriminating between the responses of 1 Their training, however, was diversified and acquired at various other universities. {I (W -'.,,S ‘- .1 those two classes of counselors, the A-V Scale appears to have adequate validity. a. The data of this research suggests that the A-V Scale has high interjudge reliability for discriminating between the responses of H.A. and Ph.D. candidates on each of its five dimensions. Therefore, the A-V Scale seems to have adequate objectivity. Compared to the low objectivity findings reported for most other counselor evaluation scales, these findings appear most favorable. The data of this research suggests that counselor \J) o trainees who receive professional supervision which focuses on examining affective, understanding, specific, and exploratory responses will receive significantly higher frequency ratings for these categories on the A—V Scale than will counselor trainees who do not receive this professionally supervised training. Therefore, the A-V Scale seems to have adequate sensitivity for measuring the subsequent change in these category behaviors which are.associated with special counselor training. inplications This section is divided into two parts; a discu351on of the implications for counselor education and the implications for future research. Implications for Counselor Education Affective, understanding, specific, and ermioratory counselor behaviors seen to be associated with Ph.D. candidates in Counseling and Guidance but do not seem to be associated with H.A. candidates beginning their practicum experiences in Counseling and Guidance. This would suggest that these counselor behaviors are associated with, and are perhaps relevant to, the counselor training process. If such an indication proves valid, then special emphasis on these counselor behaviors during training can perhaps accelerate counselor education. The A-V Scale would then possibly become a potential evaluator of this type of accelerated counselor education. The affect, understanding, specific, and exploratory dimension categories may be central to effective counseling or they may have nothing to do with effective counseling. This we do not know. However, if it can be satisfactorily demonstrated that these category behaviors are significantly related to effective counseling, then this would indicate that these types of behaviors are relevant to the counselor training process. If this be the case, the A-V Scale can serve as a cumulative progress report of a counselor trainee's change in counseling behaviors as he progresses through the various phases of his counseling develop- ment. Hhether or not these category behaviors are significantly related to effective counseling will have to be substantiated by further research. luring the course of the rating sessions which were a part of the three studies compiising this research, a new value and :w n I use of the J‘s-V Scale was suggested. IIDBA Institute and other I-LQSCOP:S levol counselors who served as judges said they understood more fully the importance of affective, understanding, specific, L and ez-zlfloratomr responses on the part of the counselor because they had been required to focus on and evaluate clearly defined aspects of the counselor's verbal behavior. This may have given them a very useful and meaningful set, one which they had not acquired before, for evaluating counselor behaviors. As a result of their rating experiences with the A-V Scale, the importance .0 of affective, understanding, specific, and exploratory reSponses for effective counseling became more apparent to them. As a result, some of these counselors reported more of an emphasis on these counseling behaviors in their own counseling practice. They also reported that through the use of the Scale in rating audio-video tapes they learned more about what is "good" counseling than they had ever learned in their Easter level practicum experiences. Could it be that the use of the A-V Scale itself for rating purposes by counselor trainees would be a valuable supplement to regular counselor trailing pro gram procedures in developing basic counseling understanding and grot-rth as well as serving as a means for providing a relevant model of counselor behaviors with which to identify? Is it possible, then, that by using the A-‘J Scale to rate experienced and inexperienced counselors, the Scale can contribute directly to the education of counselor trainees by providing exerci ses for focusing on essential counselor behavior as well as providing realistic opportunities for observing the effects affective, understanding, specific, and exploratory responses, or the lac!r of hen, has on clients in real cmu'zseling intervieX-J situations? Could it be that such experiences would accelerate counselor education and give a sound counseling perspective early in the training period? The answers to these questions can only be acquired through further research with the Scale. Iraalipajti0 is for Future Research Further development and evaluation of the A-V Scale is needed before definitive conclusions can be made as to its characteflstics and value. The implications for the types of researeh'v- which seem most promising for the A-V Scale are: l. Cross-validation of the A-V Scale on audio-video taped counseling interviews at other universities using other trained judges at each of the universities. 2. Ecmating two groups of counselors A and B. Train group A along the dimensions of the A-V Scale. Train grcup 3 along some other counseling dimensions. Rate both groups as to global effectiveness and er mine whether group A is rated higher. Establishment of more intermediate counselor levels of LI) 0 e:;)e;ience and. training and evaluating the Scale's sensitivity to differentiate these levels. 1+. Applying the Scale to the rating of counselor trainees in various counselor education programs and observing J how satisfactoxily it detects change in their counseling behaviors as they progress through the programs. Eva nation of the Scale's value for accelerating counselor kn 0 educational growth as a result of counselor trainees using the Scale themselves in counselor rating sessions. 0 ill ‘ll‘ollll‘lr'! 9. 10. 12. I) /. Bibliography Albee, G. 7. and Harlin, R. K. Judgment of adjustment f_on irawings: the applicability of rating scale ethods. J. Clinicalfifsvchol. 1950, 6: 363-365. Salerrnder, F. Fundarentals of Pch hoanalvsis. ETew Yorlm .‘-.Ol" con, 19113. American Educational Research Association. ReViGU'Of Educational Res :earcl., Guidance, Counseling, and Personnel Services, April, 1903, pp. 184-185- Anderson, R. P. and Anderson, G. U. Development of an instrument for measuring rapport. Eprsonnel and Guidance Journal. l9o2, 41: 18-24 Bayroof, A. G., Haggerty, H. 3., and Rundquist, E. A. validity of ratings as related to rating techniques and conditions. Personnel PsychologZp 1954, 7: 0a 11? J-""“./ 0 Bondig, A. I. The reliability of self-ratings as a function of the amount of verbal anchoring and of the number of categories on the scale. J. Applied Psychol. 1953, 37: 38-41 Bendig, A. 7. Reliability and the number of rating scale categories. J. Applied Psychol. 1954, 38: 38-40 Ihmxhgh A, V. Rater experience and case history judgments of adjustxmelt J. Clinical Psychol. 1955, ll: 127—132. Bendig, A. J. and Sprague, Janine. Rater experience a.nd the reliability of case history ratings of adjustment. J. Consult. Psychol. 1954, 18: 207-211. California Test Bureau. A Glossary of Measurement Terms, Konterey, California, 1959. Carnes, E. F. and Robinson, F. P. The role of the client talk in the counseling interview. Educ. and PSLChOl.LeaSXt.19M8, 0: 635-644. Cartwright, R. D. and Lerner, B. Emuathy need to change & andi Drovenent in.psychotlleraoy. J. Consxlt:_ A Pmrchol... 19L)3, 27: 133-1145. Cattcll, J. L. Cited by floodworth, Robert S. and Harold Scholsoerg. Experiuental sychology, Revised Edition. Net Yer : Henry Belt and Company, 1954. in _.Q‘)- r/ ‘9 (113411111033 73. and Earsl1z111,:;. Optinal refinement of the rating; sca ael c. J. fixnplied Psychol. 1939, 23: 323-331. Collier, R. H. A scale for rating; the responses of the therapist. J. Consult. Psychol. 1953, 17: 321- 372C. Coxr:1cr,2. J. Studies in phcncgraphic recording; of verbal tclial: I. The use of phonographic recordings in counseling practice and researcl . J. Consult. §_S*':'chol. 19132, 6: 1019-113. Cori-nor, B. J. Studies in phone rap 11:7 tie ial: 11:. The cord. pletene es counseling intervie on reports. J. "115', 30.1ul-203. 0 recording of verbal s and accuracy of Gen. P3; hol. O .a‘ 9 n so 0 1... CO‘J‘ 10;, s. u. DoucLleS'll‘l ahonor'ra 1111c reco ‘dirgs of verbal :aterial: II". Ifritten reports of interviews. J. Implied ngchol. 19m, 28: 39-9-3. 'I, f‘- u o c :3. 1.. ' ‘anach, L. J. Response sets and test valichty. Zane. and Ps:/‘(:l1ol...ea"1_t. 191-16, 6: 475.1; 34. 20. Cronbach, L. J. Essentials of Pshchclcggcal Testing; 230:: Ioriz: Ila- rpc cr 3110s., 1960. .1 I Q Q - - o o o 2*" Crone: .en, L. and Loehl, P. Construct validity in ‘jsychological tests. Psychol. Bull. 1955, 52: 35(1-302. at) "“' DeP oo, .,'. II. A study of the relationship between selected 5.4. Dittz‘an, A. T. The in a verbal behaviors of counselors and variables of the IIoltm-‘an in}: blot test and the Rolze ach dog; Katie :1 scale. Unpublished doctoral dissertation, 19C5, 9'. -ichi'an State Univcr sity. er personal proces in 1 svchotherap": 4- f‘ nU L) developnent of research neth od. J. Ab onorm. and Sec. ..n (- Psychol. 1952, 2 9" """ LlJLhL, 1‘1, . 21L. 1);. 10nd, R. .7. see 1e for the 1:1easurencntoer1pathic ability. J. Consult. Psvchol. 191.9, 3: 127-133. 535- 3001.311. Estimation of the reliability of ratings fif‘f' .s eqoletrika, 1951, 16: 407_h2. LL). Feld‘"g.ny Jo Jo, Zb’i‘fln, Hg, and £13. 11% C. .. A fiol d "131153 05 intend. ever effects on the quality of 3‘1?va (Lita. 3.101.210 Cpizfion 5118'le bellJ— L951, 55 principle of rel-.:.-t1.o and neocartharsis 23. 17:5, 001101", F. E. The concept of an 1.10 al therapeutic relationship. J. Cansult. Ps jchol. 195C, 14: .- 19—..11-5. 29. 13‘fi. cdler, I‘. 3. A ceg.11a1*i son of therapeutic relations hips in psychoanalytic, nondiroctive, and Adlerian therapy. J. Co1m11t. P sychol. 1950, 11-4: 1"’6-LH5. 30. mmsinger, J. 3. Psychiatric interviewing: I. Some 11121116111105 and procedures in insight therapy. 11102?. J, Parchiat. 19"?, 105: 137—195. 31. F3. “3111:, G. II. On the history of the objective investigation of the process of psychotherapy. J. Pslchol. 1961, (‘1 Duo 09-950 52. Fl“;“3.1’112, G. II. and Sweetland, A. A study of the process of 1 sychotherapy. J, Consult, Pachol. 1952, 26: 135—138. 33. Ge‘ohart, G. and Hoyt, D. Personality needs of under and over achievers. J. Implied Psycho; 1958, 2: 125—128. 31+. Guilford J. Personalitv. New York: I~IcGraw-Hill, 1959'. 35- Gordon, Kate. Group judgments in the field of lifted weights. J,_Eb:per. Psychol. 1924, 7: 398—1100. 36' Hal ..).)ern, II. and Lesser, L. Empathy in infants, adults, and. psychotherapists. Psychoanal. Rev. 1960, 27: 32-42. 37 ‘ Hagz‘lin, II. 2:. The clinician as judge: implications of a 50110 es of studies. J. Consult. Psychol. 1954, 18: 233-7338 38. I-Ianzay, I3. I., Dittman, A. '1‘.. Raush, H. L., Bordin, E. 8., and 75”].01", D. The neasure2:ent of depth of 1111301 are- ta 131011. J. Consult. Psychol lo 55, 19: 247-253. 39. I-Iel'astac tor, G. C. Principles of Psychological Measurement. Lew York: Appleton-Century-Crofts, 193+. 1.50. Highland, R. and Berkshire, J. A methodological study of forced-choice oerf or111ance ratings. f 3108. Bull. 51-9 IL‘EIC. San Antonio, Texas, 1951. 41. Hoyt, C. "est diability ob‘taincd 13; ans 1731:: of variance. . 12:51.. 111,191, 131-10 ’0‘ 0. a Q 1' ,_ 5- r: ZIQ. 0:71), C. Cit -. by Ebel, R. 7’ easuringj Educ Lona}- Acifievezzert. o f 315101-2001": Cliffs,Ie1.r Jers 0/: Pre1fiMcIIau,j-91>5. ‘ (In... Iii .II 1' L'53. J <9 izrofec‘u, S. I-thou relationship versus mmimlotion :1 counseling on I. ‘szcfilotherany. J. Ind. Psychol. ‘Jo I .L gnu 15; 17¢.179. ‘..//, '._J M'- - “”5 5.1:, :39 I“L"‘at11'~'ohl, D. R. , and 13.3.10: 3., 3:;i mlotcd rCCSiI ill Lhera)v u Jinn VidCO tape--a C386 Sbldj. n / I‘ A ,- J- CWUL’O—lnii P810110» 1:03, la: 37-443. '5. 110 ct, C. 3. Two verbal techniques "n a. pinioturec ounsolinf situation. P3101101. 1301106: 1*. 1M3, 6-2: 110. 2954'. ..a do. 2:03.23, J. If. The c‘i.c'v'olop:.:ez’lt and ovanotion of o. “109.571.1208 01‘ coansolor effectiveness. :E‘-'C.-'1 Journal, 1366 in ;)~'J.::.’.. y'r -,__- - . . o ’ 0 fi' f‘ ' ~.-/. :_1_. (2310:, 0., . Clotucu, 17.0.1121 ileln, .—.. 33131121."; fro-r: "1‘.-’3 reqcordcc‘. tnormy inter-vi ow: a coazlmrative study of r‘iiforont segment lengths . J. Consult. Psycho?“ L} .L" Q - " A . . LC: ' L 2'»: no.‘(, )Ir9-3/7. a ‘4, I./‘, Y’— Q n .' . . . a m -'-‘z.1-o_ :9 .:. 2.911 “a cr‘ontson, . The theory of :s5:-::o.tlo.1 o; o 9‘ _ o . ._ o 7 _ 0 'fi / ooso .n‘ello311_z..:,'. 133.1323- strike, 19?", 2: l5l-l-0. H '1 M —- 0 ' W. Q? or? 0.7.1, .1. .J. The Tl‘OQl on; of adec ate we- 3011.13.13}; records: a. torso )0351. -.‘W‘. J. 1 .lrciiot. 1:329, f‘v’ V _f :0 103’7-1OL\6. so .. .-., ‘l ’1' - ." F 2 0 .-.o.ss.-o.-l, :1. D. Jel‘bol re; erence cs and ‘ 1:, s"! ..ologic .1 changes Curing the :sychoan a1 rtic i..terv1 e1 2. '_:-:‘.1J..-.n..1ff commnicotion. Psionoonol. Rev. 1:335, 73 'P) . :1— 0-211'. tip—v u "'1 wk . lassvoll, 1:. 3. Cert/9.51.21 1.1‘032105. tic c212 n es during tzt'vol psychoanalytic) i: tervietfi‘ . P yl‘choanal. v“’\/ A ' box. 19-0. 23: 2.9-273. C'A _ .24. uélSSITOJL, II. D. 1 provisiono 11 classification of spool ' -otr' 1938, 1: 197-20%. ”f! 3 . 1 ‘3. r"" 1“ vI‘ I ~ ‘~ » .0 h _ _L ‘ 0 DJ. ILLJCIfl-uso, 11.. :. D081“1’1 and analysls of iw5el'."_.lef1oi: 31_ ~ - x .w- '*t " . . .,- . 22" :svcwolo' , 2.21-: .‘;.C:dC£T.lO.1. Boston: ZIourolmon .-j.:..‘2.n p'A Oran CO..1 93.11;" ,lDJ-J, (.>L;. 6': e ' d- r‘.‘ “ ~71 - w~1 7'. m wn‘ 1‘7. 4.3..“1 :11 Lou- ‘ J, u. 2' o 4.10“ U...C’-.;~ . .LaSU.C. -Lu 4. {$3.13. 8:1: , -o ‘ y ‘( A V _.. . 3, 1 Av/fl ”1110011151. 1: 301'f6:_‘2.1-ta1 £101-35»- "1-0 . , 2-1-... 55. 1.90.17 ’ P. 3' L LLLL CihCCorguodgle, K- "3”t1~’..;;1 voliJit)’ of'fi..-1";or"~rltii1~r; constaxcts. In: 27. ii. 123‘7. "V‘- 1 a. m‘ r $.- v' " I ‘- _.. . f, (its 0 163 3.7. wor’ . -.o.. 1.01%., -~2.cr..:.ll... . I ..an ..z / ‘fi '0 5;. Porter, _.. ..., Jr. Tho C‘Lovcloortont and evaluation of o. re of c 121:: 03221;; intervi '.-.‘ procedures: I. The devaflogztent. Educ! and Psycho}... Yeast. 19:":3, ’2; . fl 5.1 ”A --... - ->—a\l. llama all?! . Yul. P\ _f - I A “7. FOP-"1301‘, 3. H., Jr. The clevelowzzent 5.11.1.1 evaluation 0: J 5. :0:sure of counseling; and 1v1lcr'vie1jarocec‘mres. I. The 01.101.115.13bion. Educ :1an P5301101. 2705.330. 19’33 3: “15-233. 3’]. 27:13.12, 2‘1. C. The 5.53””1'110 01" com selor-treinee 110110011121 ons of fairinrviox-I protocols before and a; “ter an inten sive procticmzt 031015102100. Doctoral dis sorts cion, 12361:, Ynivcrsity of 13.112100 01351.. 59. 31* 1.;1‘2:, 31. C. :1 .d 30.05, '..'. II. Counselor response system of the ‘ Dollavim interaction description syst0 Pager read at 1:110 ALGA Convention, 1-':'1-m1051.:1olis, ,I‘V ‘3 -‘ “ vv _ 3.‘ ‘ ‘ ‘ g -, — v. .0. ‘ ‘ 1,3. -«Z‘. 612.. -;. --., 0110-1306131», 313101‘, 3. .11 1114-0-13, J., o- . . U, A ‘ ‘ “5.1.0,, .I., -fon‘i.” .3. 8., Jitt. man, A. 1‘. 51.1.0 22:.” s, ‘I. r. A w. 011010.111 3.119.111.1320 of depth f iz'x'borgrctation. J. Consult. P1591331 :11. 195-5, .730: ."'T‘ 3" "/-' .. :1 T: r. - .- -- - . .. .. -, 1-0. «‘ ~1L:,-;, --.. and zon‘in, .3. .2. ..':1.:r th in pers onelita (10110101321011: 51.:.;‘-. in lt-S;r010t11ore‘.".3’. PS.','Cl‘iE‘~._t- 1957’ An , O a .10. ’1, 291-263. ’m -- ‘J; ‘4 ‘ "' 'fi‘- q c Q’- F‘H .\ ‘ L . “—-OC-I, . o, ..b JCS, C. , Fl..‘-‘1.:l, A. 11., JSQ1~I‘J.'&J',, ..). SQ, -0..5.n, II. 3., 9.1.6. 31‘ - oer, 3. Lotions-l Institute 01‘ 2.012351 Health pilo and study in *z-oiningj- cental 110911211 059.11.:30--0rs. The 300:1. J. ’thhopzjgcriot'g. 1903, "X’I. ’ A ’flr) /" u L-J- /I o , 03 '\ r0 ' 0 r‘ '3 1 . r. ‘ w '3 -, 1‘ ' ‘v V, - flowers, 0. a. CO‘II’ISanA :1.-v10. ;s 0.05110121- 1 . Boston, «.1 22011le ”on I-‘ifflin C0: 1.1.1157, 3.;102. (a “‘- -.0 0.5, C. R. 1:3. ectr'ically recorded interwi ears in "41111011. ‘ 1;; I)S;’C.100.-Or"lf)0uulc intelwiwss. Inter. J. 3:. $05077.) 4,01" )j. LU. l;’11’o ’ 12: [4‘29 "35. p. a 1.1 rh '- a o Rogr-‘crs, C. 21. Client-centered TILCI‘QEY. Corbridge, O - ‘.O Q ‘ ' A 21'.’0:0ice Press, 193i. / f‘ f‘ Q: Q . ‘. 60 11032118 C. 3. opo-.sor: Lat-lieu, P. on - curse-11d, J. P. 311“ 1035313131 05 31‘0“? 0011113011an t0 m‘osequent ac;1 1610.210 L‘Clrf011?2--co a" £110 00110;. level. C001erativc research. 7“ b Project no. 037194.? 3fl-sconsin :syc-xiatric Institute, ‘ 3 v o - .n _ r r 70 ‘* " F (V 5 _ / bits-versit; of .4 sconsin, maison, ..2isconsin, 1202. «37.3000, ZI. Inte11w.101.:1'_ng to test for valic-ity and rolicoility. I--tern. J. of Ouizio or and .‘Lttifiude 20.130130?" 9377, 1.: 1031—101. / v. o -‘ H . 113. Mich, 17.3. I‘syc‘1010'12' and life Chicago: ocott, gorcsrzan, 5.:6. :30. ‘ iz-tth edition, 19?}. 69. 72. 75. 7' 3:11:15 3.. 0. Is the rating of human character practicable? J. 3:13.10. Psichol. 19? , 12: 1:25-1:33 and LIBS-501. s clxafcr, R. Generative empathy in the treatment situation. Psvehoanal. Quart. 1959, 23: 34252-373. Silapiro, S. and Eberhalt, J. C. Interviewer differences in an intensive interview survey. Intern. J. of Ozzizfion and Attitude Research. 1947, 1: 1-17. Srvder, 3!. U. An investigation of the nature of non- directive psychotherapy. J. Gen. chhol, 1945, 33: 193-223. SIUClOl‘, 2.7. U. The Psyghotheraoy Relationship. New York: I-La.c:j.11an, 1901. Ste 3311, Joseph. Responses to hypothetical counseling situation as a prediction of relationship orientation in school counselors. Unpublished doctoral dissertation, University of Wisconsin, 1963. "C’I'unlg 0., Jr. Expath : a review of theory and research. Psychol. Newsletter. 1957, 9: 47-57. 76. Sta—11px), H. H. Hature of psychotherapist's contribution to the treatment process. Ardh. Gen. Psychiat, 1960, 3: 219-231. 77- Szntloncls, P. II. On the loss of reliability in ratings due to coarseness of the scale. J. Laser, Psychol. 1922, 7: 1956-461. 78' Syztzonds, P. N. Notes on rating; J. Applied Psychol. 19259 9: 188-1950 79- Sjfi-Londs, P. Ii. Research on the interviewing process. J. Echlc._Esxc1mli 1939, 30: 356-353- 80. Thornton, G. The effect upon judgments of personality traits of varying a single factor in a photograph. g. Soc. Psycho; 1943, 18: 127-1118. 81. Thornton, G. The effect of wearing glasses upon judgments of personality traits of persons seen briefly. J. igflied Ps;~[_chol. 19134, 28: 203-207. C3 (‘0 o Torgerson, 2']. Theory and Methods of Scalian Ifew York: John Kiley and Sons, 1953. 83. Truax, C. B. and Carklmff, R. R. Theory and research ha in counseling and psychotherapy. 11pm: Journal. Day, 19:31:}, 880-335. and Ifiea mrez‘c nts . Engl evood Cliffs , 32:;rfler, L. 2. Test / u L a 1 [_ : ren ul 8-;13Jl, Inc . , 19e3 . “0*: Jersey 85. E'fajdcer, H. II. and Lev, J. Statistical Inference. New York: Zlolt, Rinchart and 'Jinston, 1953. ‘-.7'::2.3_3, G. 1‘.. and Johnston, J. A. Counselors look at therzsolves on video tape. J. lenselinLPsychol. 1963, 10: 232-236. CO C\ o r . 87. ZFaZ‘d, R. Educating counselors by video-tape using the interpersonal process recall technique. Unpublished electoral dissertation, 1965, Inchigan State University. 33. 312:: ’tson, P. D. and Kanter, S. 8. Some influences of an expci'uental situation on the psychotherapeutic process: a report based on 1&4 treatment interviews, of the reactions of a patient and therapist to observation, recording, art‘s. physiological measurement. ngchosoztz. lied” 1956, 18: 457—470. 89. ~ Testnan, N. C. A comparison of three modes of rating counseling interaction. Unpublished doctoral dissertation, 1965, University of L'Iisconsin. 90- lander, c., Almad, F., Bandnra, p... and Han, L. Dcwendency of patients psychotherapists' responses and aspects of psychotherapy. J, Conmlt. Psychol. 1962, 26: 129—134. 91- 2:013, R. and l-Iurray, personalities . . A. An experiment in judging Lof Psychol. 1937. 3: 345-365. C453 APPENDICES -100- APPENDI X A A WI'IPILATIOI‘J OF COUNSELOR VIDEO VERBAL BEHAVIORS AS OBSERVED ON AUDIO-VIDEO TAPES -l Ol— VIDEO VERBAL BEHAVIORS CHARACTERIZIKG MASTER'S CANDIDATE COUNSELORS Cognitive l o Sllperfi 0131 2. Clumsy structuring of interview 3. Nonverbal behavior indicates anxiety and discomfort with client 4. Prestructures client's reply 5. Distant from the client emotionally 6. Avoids client problem areas 7. Reflects or rephrases cognitive aspects 8. Expresses no feeling about client 9 0 V9. gu 9 10. Moves client away from problem area 11. Questions cognitive aspects of client's verbalization 12. Makes general comment in response to client's verbalization 13. Agrees with client's cognitive verbalization 14. Repeats client's comments verbatum 15. "Yeah," "um huh," "I see," "right,” 16. Comments on room 17. Makes judgmental comment concerning client or client's verbalization 18. Adds additional cognitive content or meaning to client's verbalization in restatement effort l9. Leads client in specific cognitive direction through suggestive comments 20. .Lectures or sermonizes to client 21. Philosophizes to client 22. Conducts monopolizing conversation 23. Advises 24. Offers cognitive evaluation of client's tuation 25. Pleads for information 26. Excessive Verbalization response to client cognitive content 27. Anxiety laughter Affective 1. Inaccurately reflects feelings 2. Inaccurately interprets feelings 3. Reflects ambiguity 4. Leads client in irrelevant affective direction through suggestive comment 5 5. Adds additional affective meaning to clients verbalization in reflection effort 6. Excessive verbalization response to client affective response content 7. "Yeah," "um huh," "I see," "right," Video Verbal Behaviors Characterizing Ph.D. Candidate Counselors Cognitive 1. Questions cognitive aspect of client's verbalization 2. ”Yeah," "um huh," "I see’n "right," 3. Reflects cognitive aspect of client's verbalization 4. Reconstructs or simplifies cognitive presentation Affective 1. Responds at the same emotional level as the client 2. Free from discomfort with client 3. Questions client's feeling 4. Appropriately reflects feelings 5. Appropriately clarifies feelings 6. Appropriately interprets feelings 7. Asks personal-intimate questions 8. Discusses client's problem areas 9. Reassures client concerning affective aspect of Verbalization 10. Offers affective evaluation of client's situation 11. Questions to clarify more specifically general feelings expressed by the client 12. Expresses to client the feeling which client's Verbalization elicits from him 13. Expresses his feeling about client 14. Suggests related affective area for discussion 15. Points out to client his reluctance to discuss an affective area 16. Points out to client his refluctance to discuss more deeply his feelings 17. cqeah," nun huh," "I see," "right," APPEE-IDIX B CATEGORY. ZATIOI‘J OF COUNSELOR AUDIO-VIBE.) BEHAVIORS ACCORIIING TO BEHAVIOR CATEGORIES B. CCU ZISESLOR VIDEO VEZBAL BEHAVIOR CATEGORIES AND IILUSTRATIOI-IS Affective This categorv includes all counselor responses which make reference to some affective or feeling aspect of a client's verbal communi cation. Illustrations 1. Responses which enter the feeling frame of reference of the client, which deals with expressed concerns, needs, and wants of the client as expressed or implied.by the client. 2. At the same emotional level as the client. Emoting with the client on his feeling level. 3. Responses which reflect, deal with, clarify, or’interpret the feeling of what the client is saying. Understanding The counselor's ability to communicate to the client the fact that he knows what the client is feeling or talking about. Counselor communicates an understanding of client's feelings to the client. Illustrations l. Counselor reflects client feelings accurately Counselor attempts to express in fresh words the essential attitudes (not so much the content) expressed explicitly by the client. The counselor attempts to mirror the client's attitudes for his better selfaunderstanding and to show the client that he is being understood by the counselor. 2. Counselor clarifies feelings in a specific, non-vague, rial-iner e The counselor clarifies what was implicit in what the client said; he reflects relationships or meanings for the client which are suggested or implied in the feelings or ideas brought about by the client's actual responses. 3. 5. -106- Counselor interprets feelings appropriately. Interpretation refers to any remark which interprets, analyzes, explains, or draws inferences. Counselor presents to the client an hypothesis about reliationships or'meanings of his attitude behaviors, which he had not considered before, for his consideration. Counselor's verbal responses and accompanying behavior demonstrate primarily an interest in and comfort with the client and thereby conveys counselor acceptance to the client. a. Counselor's attention and concentration are on the client. b. Counselor's verbal responses demonstrate that he is following what the client says (e.g. "yes," "I-Mim," ”un huh," "I see," etc.) o. Counselor gives indications of approval which seem to give reassurance and encouragement (e.g. Rgood," "I feel that's a fine idea.") These may be positive confirmations by the counselor of the adequacy of client behavior, attitude, feelings, etc., as alluded to in the verbalizations of the client (e.g. "you are a capable person;" "you can.be consisten ;" "you can be reasonable;" "you can be organized:" "you can feel better."). d. Counselor's nonverbal manner is free of discomfort and anxiety (there is an absence of continuous shifting in the chair, looking away from the client, or continuous movement of arms, hands, and legs). Counselor questions client's feelings generally or questions to have client clarify more Specifically the general.feelings which he has expressed. This is a request for general elaboration and expression of feelings. Counselor expresses his feelings about the client or the feelings which the client's verbalizations have elicited from him. This includes: a. Statements of counselor's feelings about the client as a person. b. Statement of counselor's feelings about something the client has said, done, or thought. 0. Statements in which the counselor's feelings are expressed as being his own. C. D. -197- Specific Counselor's interaction with client is characterized by concrete (specific) responses concerning the client's problem.areas and emotional conflicts. Illustrations l. 2. 3. Counselor proposes or suggests that the interview be devoted primarily to the objective of exploring the client's problem areas and emotional conflicts. Counselor's responses reflect, or are directed toward, the core of the client's remarks rather than their peripheral aspects. Counselor's responses move the interview discussion in the direction of the client's problem areas and emotional conflicts or bring the client's discussion back to these topics. Counselor's responses are relevant to the client's prdblem areas and emotional conflicts rather than being irrelevant as characterized'by abstract, intellectual, lectural, extraneous, etc., type of responses. Exploratory The counselor's assistance and skill in helping the client to explore his feelings, values, and other concerns. Illustrations l. 2. 3. Counselor suggests possible exploration of affective area referred to either*explicitly'or implicitly by Client 0 Counselor points out to the client his reluctance to explore an affective area. Counselor asks personal-intimate question of client. Counselor'is willing to explore with the client, in a nondefensive manner, the import of his (i.e., the counselor's) behavior traits which the client observes in their relationship. F. -lCIS- Cognitive This category includes all counselor responses which omit any reference to affective or feeling aspects of a client's verbal communication. illustrations 1. Response is distant from the client emotionally. 2. Superficial reSponse which passes over emotional implications of client's verbalization. 3. Not with the client emotionally. 4. A response which is not on the feeling level of the client. 5. A response which does not enter the client's emotional frame of reference-it is at a different level from the client's feelings. 6. Own feelings or apprehension preventing counselor from making emotional contact with the client. 7. Clumsy structuring of interview. 8. Anmering factual questions. 9. Repeat of client's remarks verbatum. 10. Responses devoid of any reference to client affect- feeling, emotion, mood, attitude, etc. ll. matter of fact responses, informative responses, general and superficial discussion responses. Nommderstanding Counselor does not communicate an understanding of the client's feelings to the client. illustrations 1. Counselor empres uses confusion as to cognitive or 2. affective meaning of client's verbalization. Counselor's responses show evidence that he has pre- judged the client and thereby has classified, stereotyped, or categorized the client. G. 3. 7. Counselor's responses inaccurately reflect the client's feelings. Counselor's responses of clarification of client's feelings are attempted in a vague, non-specific manner. Counselor’s responses encourage or reflect ambiguity. Counselor's responses in restatement or reflection add unwarranted additional affective or cognitive content meaning to client's verbalization. Counselor responses reflect the cognitive aspects of the client's verbalization when affective aspects are central. , 8. Counselor repeats client's comments verbatum. 9. Counselor pleads for information. 10. Counselor verbal and nonverbal responses indicate a lack of interest in, and discomfort with, the client. a. Counselor looks away from the client rather than concentrating on him. b. Counselor does not appear to be following what the client says. Nonspecific Counselor's interaction with client is characterized by nonconcrete (nonspecific) responses concerning the client's problem areas and emotional conflicts. Lllu strati ens l. 3. Counselor's reSponses reflect, or are directed toward, the peripheral aspects of the client's remarks rather than toward their core meanings. Counselor responses are related primarily to the cognitive content of the client's verbalizations and deal only superficially with the affective content. Counselor responses are irrelevant to the client's problem areas and emotional conflicts. They are characterized by abstract, intellectual, and extraneous responses. a. Counselor encourages expansion of superficial cognitive discussion. .11 C... b. Counselor intellectually debates cognitive or affective aspects with client. 0. Counselor remains silent and simply permits client to wander in his verbalizations. d. Counselor is excessive in his response to client cognitive or effective content. H. 15 onexplorat cry The counselor's failure to help or permit the client to explore his feelings, values, or concerns. Illustrations l. He—structures clients reply 4.. Counselor monopolizes conversation, philosophizes with, or lectures to client. 3. When client presents affective content, counselor response does not encourage further exploration of this content. 1+. Counselor's responses move the interview discussion in a direction away from the client's problem areas or emotional conflicts 3. Counselor makes judgmental or evaluative responses to client’s Verbalizations. 6. Counselor's responses are characterized by advice- giving and question-answeling. 7. Counselor's responses are structured to specifically lead the client in definite cognitive or affective directions rather than permitting the client to . initiate these directions. APPENDIX C DEFINITIONS AND ILLUSTRATIONS OF THE A-V SCALE'S DDIETJSION CATEGOKEES —llZ- TIT OF EVMHATIOI‘I FOR THE A-V CCDNSELOR BEI-IA‘JIOR SCALE The urit for evaluation for the A-V Counselor Behavior Scale is the counselor-client unit. A client verbalization is emitted or elicited and the counselor's response to it is then rated on all five dimensions of the scale according to the dimenfion mtegory definitions. One forced choice rating is made independently for each of the four specific counselor behavior dimensions and a fifth rating is made evaluating the effectiveness of the counselor's response. Generally, each client-counselor unit is jxdged independently of preceding units, although the congruence between the direction and level. of counseling movement by the counselor with the direction and level of counseling concerns the client is attempting to present must continually be kept in awareness when making rating judgments. The scoring of all dimension categories must be based upon concrete and observable behavior and is not to be based upon what a judge may believe could possibly be occurring within the counselor. 13112313810113 OF THE A-V (XDUITSFLOR BEHAVIOR SCALE The Cognitive-16f ective Dimension This dimension indicates whether a counselor's response is in reference to some affective or feeling aspect of a client's verbal communication (an affective response) or whether his response omits any such reference to affect or feeling in the client's verbal communication (a cognitive response). These categories are more clearly defined as f ollows: A. Affective Regponses Affect refers to emotions, feelings, fears, moods, desires, urges, impulses, fantasy, dreams and any ideas, attitudes, beliefs, convictions, etc., which are based upon such re- ferents as these. A counselor affective response is one which makes reference to any of this type of affective material on a feeling level regardless of how congruent it is with that being expressed by the client. An affective response is not determined by the extent to which the coun- selor's manner or communication is affective, but rather, it is determined solely by the fact that the counselor responds throughcomnent (reflection, clarification, interpretation, etc.) to underlying affective elements, or to what he implies are underlying affective elements, in the client's verbal and nonverbal expressions as they are revealed in his inter- actional discussion with the counselor. B. .114- Examples: 1. “You feel quite inadequate in these situations." 2. "You act that way because you are angry." 3. "You feel better when she mothers you." 4. "How do you feel when your parents argue?" 5. "Maybe that's not what you want at all." Coggiclve Responses A counselor cognitive response is one which is on a non- affective plane in that it is devoid of any basic concern, reference, or recognition of client feelings, emotions, moods, attitudes, or any other affective aspects which may be implied in the client's communication. Such a response passes over any affective implications inherent in the client's remarks. The counselor responds at a cognitive, non-feeling level or asks a question or makes a statement concerning cognitive or content material. Such cognitive responses are more formal, informative, matter of fact, of a general discussion order, and socially acceptable oriented, and are altogether on a different plane from responses which ‘mske emotional contact with the client through discussion of feelings, attitudes, etc. Cognitive responses include pnswering factual questions, "busy" talk, discussing triviali- ties, repeating client remarks verbatum, exclusive talk about third parties without affective reference back to the client, any any general educational, intellectual, vocational, con- versational etc., type discussions. II. .115. Examples: 1. "Yes, that seems to be a logical step to take." 2. "Your father was an attorney, huh. Well that seems to give you a good background." 3. "How are you feeling today?" (If intended in a general sense) 4. "I think your performance in your science course would support a decision to enter medicine." 5. "So you want to enter graduate study. I think that ' 8 nice. " The Understanding--Non-Understending Dimension This dimension relates to how well the counselor's response communicates to the client the fact that he understands, or is seeking to understand, the client's basic problem. These categories are more clearly defined as follows: A. Understanding Responses These are responses which imply that the counselor understands the basic underlying affective or cognitive concerns of what the client is talking about, or is trying to get enough information from the client (through appropri- ate reference to what the client is feeling and expressing and are appropriately timed to the significant cues given by the client. Examples : 1. "These feelings make it difficult for you to get along with others." 'I I‘ . . I s 0“ 4 . . V . ”I l '7: o. a-l J .3 . .. I . ‘3 51". r 2 \v'. As.‘ : 0' "‘ .9 _.- ' . I . ,L . .4 a"7 _‘ .4.,' ,. -. a ;|' I 4'“ B. -116- 2. "In other words, you feel inadequate and self- conscious in these situations."° 3. "You really want to be a big man." 4. “Could you tell me more specificallywhat you mean by this odd feeling about your brother?" 5. "Could you tell me more about that?" Non-Unders tandigg Resgnses These are responses which imply that the counselor lacks an understanding of the basic underlying affective or cognitive concerns of what the client is talking about, or are responses which indicate the counselor is making no attempt to obtain apprOpriate information from the client from which he may derive an understanding of the client's underlying concerns. Non-unders tending responses are those which make inappropriate reference to what the client is feeling or expressing, are inapprOpriately timed to sig- nificant cues, or are responses which are made after several significant client clues have either been ignored or entirely missed. Examples: 1. CL: "When he said that, I just turned red and clutched my fists." CO: "Some people don't say nice things." 2. CL: "When I showed my mother my grades and she said they were acceptable, but I should be 4 fl. . . . . . . a . . ..c I: II 0 : v I. l . . 1 v 0 . v . u . r v , All, i L .. x I w‘. . . . III. .117- doing better, I felt like quitting college and telling her to go to hell." C0: "What would you do if you quit?" 3. CL: "Sometimes I have queer feelings about my parents." CO: "I see." 4. CL: "When Ivan put his arm about~,me, I started crying." C0: "Ybu.mentioned that Ivan likes the classics." 5. CL: "I'm.so self-conscious in a crowd.- (CO: Silence). I just can't speak to groups because of my nose (CO: Silence). My brothers always made fun of my nose (CO: Silence). My aunt said once I looked like an eagle. Boy, I hated her guts (CO: Silence). I just don't like my family." C0: "When you go to college you can get away from your family. The Specific-~Non-Specific Dimension This dimension indicates whether the counselor is dealing directly with the client's basic problem by confining his re- sponse to a concrete (specific) one quite relevant to the pre- sented problem (a core response), or whether the counselor passes over dealing directly with the client's basic problem and gives a more general, or non-specific response not too relevant to the presented problem (a peripheral response). These categories are more clearly defined as follows: r 1 a A .A. \ .x I . .u. .\. up . I. r. ~ . . I. I 0 II. a . . u 1 . . . x. J. J » .s .4 . .. . A. i118- Specific Resppnses These are core type responses which indicate that the counselor is handling directly and honestly the basic problem presented explicitly or implicitly by the client. The counselor's remarks are keyed to the central aspects of the problem which the client is feeling and communica- ting. The counselor thus responds to the underlying con- cern, feeling, or meaning--the primary message--of the client's remarks such that his responses zero in and are congruent with the level of affect or concern being ex- ~pressed by the client. Specific responses are those which influence the client to attend with specificity to his problem areas or emotional conflicts and concerns. They are relevant to these problem areas or emotional conflicts and concerns, and move the interview discussion in this direction or bring the client's discussion back to these tepics. Specific responses are usually directed at underlying affect, but may consist of cognitive responses if awareness of affect is not essential to the working through of the counseling prOblem. Examples : 1. CL: "I've gotten all A's this year and I still feel lousy. " CO: "You're sensing that obtaining such high grades is not such a satisfactory solution to your feelings of inadequacy as you thought." a. . _ \I . .. . . . ' .. a on, hm p - .bJ ! \. . 1r. . a . \- ..|.. . . y .1 Q1... . .034 .f . . o a . J . , . h. , . . ‘e A . . ‘ ~ — c' , . . . . I I. . p1 - u. . a . r .A . .. . u a . J 4. 7‘ v » l - r x, . r .r .34 , . I V L p 5 .- . I. h a ‘l. . .- .- . s a .I. .1 VI. . . . . fl « I. .vlls I n V.. I 0 .na . . .. . o I, 1's (C .d a“ , ..). . .. a , e .. v ~ . | ..) .. If; I. . |\ 4. . o. .. A on V. B. 2. CL: 3. CL: 4. CL: CO: 5. CL: CO: ‘ .119- "It's a common thing for men to have mistresses in EurOpe. ‘Why do they have to make such a fuss about it over here?" "What's that got to do with your sexual problem?" "Hell, what do I care about being on.time, my boss never says a good word about my work and he didn't do a thing about getting me that raise." 7 "When your boss doesn't continually praise and take care of you, you become angry and take it out on him by being late." "When I try to make love to my wife anymore, I just can't perform." "Ybu still look on your'wife as your mother." "It's my twenty-second birthday tomorrow. I hate growing old." ‘ "It's more fun being a little girl, then you don't have to face these adult responsibilities." Non-Specific Responses These are peripheral type responses which indicate that the counselor is not dealing directly and honestly with the basic problem presented explicitly or implicitly by the client. Rather, the counselor responses stay away fromnthe client and his problem, or at best handles the problem at a superficial, surface level. These responses indicate that‘ the counselor has missed or ignored cues which reveal the t ‘ ' ' V: ‘ 7t ‘ '1 6 L J ~ v! .‘ )Wf , , . . ‘.,‘.,.),..,,7 ‘. \flj". ‘ . ":1: I , ' r" : w. ‘, l. .. :r a! ,- 'h‘ . 'Jf‘ ' T‘f' . Tut. (l‘ ' . r . T‘“ r .‘ - ..a" ’ ‘. A. o '. 2‘ '3 H'.’ I A o‘ a s ..l ‘.-. ‘-‘ . ~l (an a . , ,‘ , s r' ' ,‘»‘ ,, . L ‘1‘ L 4‘ ) .} ‘ ‘1.: . .2 ‘ .1 . ' a .v‘ - q 0‘», . _ . ‘ , 4 -‘ _ } .I . \ \ ~‘ " . V ‘1‘ ‘4 ’ -- fx " ,C r s! J“ A ., .. .., .. ‘ -3 .’ a. .A. ‘ . .l ‘v '. Vy‘ , . o l ' ‘ ..-. ~': . o . _ " If:f.¢( . _ ‘1 - . H , . 3" ~“ -. . ‘~ a ‘l ,l 5 O ' I . — ~ ~ . J. . , z.’ . . . _ ,- .- "A{l‘.f‘ 1. . ‘ ~I. ‘0‘ . l-D x= . J T. ' . . .. . .. . I L - a .~ .. .- f ..a - ~Av ‘I ‘3 (153: _'.‘ v -ta , 'A f. -.‘ J“\- "J , . _.j l . l' a u‘ I o 0 va- . i . ... l " ' -. . ,\ . If .‘ , ‘,‘: . . '5 « .‘.L ' s‘a , ~ n ‘\ .. L W' . _ . \f- , | . ‘ ck. .. .. J ‘ ‘, ‘- _. . ,n +3.. 3J'». .1 _....- ....-- _..--. _...-. . .. . ‘1"..‘1 5,.“ _... . . . ‘,‘ s. I s. .. I .J ‘ 7 a, '. ., r .: ' l’_ L ‘ . , < 4.- r \ , . . p‘; . . . r-.~’ . _. n .. . 3.. . . , .«zu' ~ .1 , e e I' '- F ' ~ .J. . , , ;120. client's main problem areas. In such responses the caunb selor concentrates on aspects of the client's Verbalizations that are of less importance than the main message which the client has attempted to comunicate. The counselor's re- sponses refer mostly to cognitive generalizations or super- ficial cognitive specifics rather than their underlying concerns, and thus such responses are not congruent with the level of affect being expressed by the client. Non-specific responses are those in which the counselor becomes abstract, intellectual, or extraneous rather than confining his remarks to the specifics of the client's problem areas or emotional conflicts. Such responses tend to be irrelevant to, or deal inappropriately with, these problem areas or emotional conflicts and thus move the inter- view discussion in a direction which avoids them. Such re- sponses encourage expansion of superficial cognitive content; leads to debate with the client over cognitive or affective concerns; are typified by excessive response verbalization and verbal monopolizing on the part of the counselor; and are illustrated by all sorts of lecturing, sermonizing, and philosOphizing to the client by the counselor. Examples: 1. CO: "I've gotten all A's this year and I still feel lousy." CO: "I wouldn't feel so bad if I were you. There are a lot of kids who would give their eye-teeth J. L., IA 7. 2. CL: CO: 3. CL: C0: 4. CL: CO: ‘-121- for those grades. Feel thankful and count your blessings." "It's a common thing for men to have mistresses in Europe. ‘Why do they have to make such a fuss about it over here?" "Well you know it's really not the Christian thing to do. Anyway, it's wise not to develop relationships which are frowned upon by our society. Have you considered locating in Europe?" "Hell, what do I care about being on time, my boss never says a good word about my work and he didn't do a thing about getting me that raise!" "Don't you think though, that consistently being on time will be to your advantage in the long run? It looks much better on your record and shows a sense of maturity and responsibility. Your boss is probably a very busy man. I'm sure he will make some comment on your work in the future. And, don't be so disappointed about the raise. A lot of people have to get by on a lot less than you're making." "When I try to make love to my wife anymore, I just can't perform." "I imagine that's very frustrating to your wife." v \ WING.) ‘I". 1 . .1 .3: . A ‘- 515."; . c;£ ‘r . . If. . . I ..r .‘;_l . . V’I‘", - ' I ' I . . A a . . . . S . . ‘ . l v I . I I I O l' . . U, . .W . . . \u , ‘ I ‘. ‘ . . A. a w . ,1 v ...a -122- 5. CL: "It's my twenty-second birthday tomorrow. I hate growing old." C0: "My lands, child, that's not old. You're real young yet. You have your whole life ahead of you." IV. Explorative - NondExplorative Dimension This dimension concerns the extent to which the counselor's response permits, encourages, or elicits further exploration on the part of the client of any aspect of his problem areas or affective concerns. These categories are more clearly defined as follows: A. Egploratopy Resppnses Counselor exploratory responses are responses which first, indicate that the counselor has recognized some por- tion of the client's basic problem and, second, has made some verbal reference of this back to the client in such a way that the client is free from any defined or ”limited" structure in his response, and is thus free to respond to any degree and depth to the counselor's reference that he 'may choose. In addition to the verbal response, the counselor's whole manner creates an atmosphere for further exploration. An exploratory response, then, is one which concerns some aspect of the client's basic problem and at the same time permits and encourages the client flexibility and freedom in his response. Such responses are often open ended and V. . t . I v. . .w ' it . a _. .. ._ ‘ a. .1 .[ In ..IJ .1 I. .3 ok 9...... . . ..« . . 7.. . A K: .14 A o . I. .. 4 . . \J "u. 4. u \ )u . . i. . I. .1 IA . ... . ‘ a. u”; ,. v .7} 4 A . . .. _. . . »v ’A I. ..a 0!). ,,._ A. .IJ. .. - . . r . I .‘ r .0. .. . .. v f .. H. . .,. LI; 5‘. r» J . f B. ’3 .12)- allow the client to explore his own feelings and to expand upon them. Examples: 1. "It seems that your anger is really directed toward your father." 2. "Perhaps your need for such high esteem has some- thing to do with your choice of medicine as a career?" 3. "Maybe we should discuss a little more these strong attitudes about sex." Non-Exploratogy Resflnses Counselor non-exploratory responses are responses which indicate no recognition of the client's basic problem, and thus no attempt to search them out further, or responses which structure or limit the client's response. A "yes" or "no" answer or a "pat" answer is often implied by such responses. Non-exploratory responses are those which give the client little opportunity to explore or expand, or to express himself freely. Examples: 1. "What is your average in English so far this year?" 2. "That's a nice place to study, isn't it?" 3. "What was your uncle's middle name?" a C. I J Ir ,7 'l' .- . ",1. -\ -‘ .'¢' _ .. . .‘I" ‘ r g‘ - ' v‘ ‘ ‘ , t . J 0 . , ,__ .. “_...- .--... --v- ,...- «4....-- . — . , .. 00' . r " “' L) . . _. . O .' ‘._, 64's. I o l‘ ;_L.. ‘ , ,.. . , .s. , . . ; “ r v. ’ -- . : . . "(‘7 F ‘ “.2 u. . ..1:. :3 ..~ J I. .I. . ‘ ' ' “ ' n .a .I J. Li. ..:_:. ., " - a l‘ C ' . .- , s . .~ ..l!‘ [,1 .L; ', Y.‘ —i l '. > I p -12 COUNSELOR RESPONSE EVALUATION This evaluation is made independently of the four counseling dimension ratings and is a dichotomous rating whereby the coun- selor's response is marked either effective or non-effective. This rating is not based upon an evaluation of the client's response to the counselor's remarks but, rather, is based solely upon the judge's professional impression as to how appropriately, from a counseling vantage point, the counselor's response deals with the counselor's verbalization and thus contributes to the development and maintenance of counseling progress. Counseling progress is generally defined as concentration of discussion on basic problems and feelings of immediate concern to the client such that there develops for the client an increasing undistorted awareness of their influence and consequence on his behavior and relationships with others. In this regard, the judge decides globally--taking into consideration the counselor's total manner and behavior, including nonverbal--the effect the counselor's response will have or could have, toward permitting or encouraging the client to move in a di- rection of counseling progress. If the judge evaluates the counselor's response as effective he can either check a 4 or a 3 under the heading "effective." A rating of 4 indicates that the judge's professional impression is that the response is about as "good" or the "best" possible in terms of counseling progress. A rating of 3 indicates that the judge's a a]. W . .11. I. , \‘ . , I ~ -.« v "- .. ~ ‘ ‘ _ . " ._ - l , v - , y ~ ‘ ~ r , ' ’ . ' ..I . t' . .. . . .\ .' . I V v . »' L K ' . I . .j A I . r) .| A’ p - I i ‘ L \ . . . ‘ I ‘v N u 3 ~ g (I. _' ,. ' ‘ . l, .' ‘~ ‘ I , ~ ( . ‘ ‘. ‘ . a; .J ‘ ' ‘ ' ‘ . . A . . . . ' I y I. ‘ .1 ‘ if ”4 .-. .J. .v I. s“, . l ’ ‘ - ' J, l 1 ‘l. .N'(,'. ..’ f . 4 .I ~ . ' .1..\ - ‘ "I. 0' \“’ . 'J ; “ .. - : -,-i f (\ (j-.-,',v~\ , VA >-‘- x' d- .‘ -- ‘ v .-‘ ‘J. ..a , ' ev .‘ . ‘ ' I A ‘1‘ x ' d v \v . w '9 ‘...4 . l .l, e professional impression is that the response is effective toward some counseling progress but it was not the "best," that is, a more effective response or communication could have been given. If the judge evaluates the counselor's response as non~effective he can either check a 2 or a 1 under the heading "nonveffective." A rating of 2 indicates a response which contributes in no way to counseling progress (sort of neutral) while a rating of l, in addition to this, indicates also a complete lack of understanding or concern regarding the client's problem situation, or is a response which is definitely detrimental to counseling progress. APPEIDI K D CIARIFI CATIoi-I OF FREQUENTLY ocwsrcme RATING PROBLE‘I-TS AND QUESTIONS -126. Right Side and Left Side Scoring Tendencies For a given counselor response, judges should be particularly cautioned against rating the first dimension and then marking all the remaining 'nensions to follow suit: e.g., marking the first dimension cognitive, and then the remaining dimensions nonunder- standing, nonspecific, nonexploxatory, and noneffective, or marking the first dimension affective, and then the remaining dimensions understanding, Specific, exploratory, and effective. Each dimension.is to be scored independently according to category definitions. Possible Counselor Dimension Scoring Patterns According to the dimension category definitions, the following scoring patterns are possible: 1. A response can be cognitive and still be understanding, specific, and/or exploratory. 2. A response can be affective without being understanding, . specific, and exploratory. 3. A response can be affective and understanding without being specific and exploratory. 4. A response can be understanding and specific without being affective or exploratory. 5. A response can be affective, understanding, and exploratory without being specific. A reSponse, however, can never be exploratory without being understanding. Recuggipg Judge Qgestions Concernipg Dimension Category Definitions Although it is not possible to anticipate the nature of all judge questions concerning category definitions, the important ..l' (0 recurring ones are presented here along with the answers of clarifi cation: c1) so &) L; "If the counselor demonstrates affect himself in his response, is tlfis scored affect?" "1'0, not necessarily. The main criterion is whether the counselor is referring back to some affective aspect presented by the client. The important thing is what he is referring back to, not his manner in referring." 7 fhat if the counselor ma (95 reference back to affective aspects or :u 1plica“ ions in the client's remarks which are not justified b3, the client's remarks or manner. Is this scored affect?“ "Yes. However, if you feel the affective reference is inappropriate you would also score the response nomnderstanding, nonspecific, nonexploratory, and probably noneffective." ‘Uhat if the counselor refers to or discusses the feelings and concerns of third parties which are in no way related to the concerns or involvement of the client. Is this scored ffect?" "No, such a response is scored oogrdtive. To be scored affect, (Bantam of feelings and concerns of third parties must be directly related to the client and tied in with the feelings and concerns of the client." "Are all questions by the counselor to be considered as seeking understanding?" “No. The question has to be related to clarifying for the counselor some aspect of major concern, explicitly or implicitly implied by the client. Random questions asked by the counselor from the 'top of his head' to alleviate satiety, to continue a cognitive discussion unrelated or leading; away from the client's rain concerns, or which in any way soon unrelated or inappropriate to what the client is trying to equress, are scored nonunderstanding." ”If the counselor fails to respond to several important affective client leads but finally responds to a lesser type of client lead, is this scored nonunderstanding?” I] In "3"OS . cc /-7¥ /'130.. ::If the couns selor nail nt ins a cognitive discussion with the client when the clie1 t' we remarks are implying more affective nonundcrstanding?” concerns, is this see n L». 5 ”Yes. Jhencver a counselor maintains a cognitive discussion 0" er festive concerns, his res sponses are scored nonunderstanding. "Ehen is a cognitive response on the part of the counselor scored undcrs handing?” ”Jhenever the cognitive content to which the counselor is responding seems to be the important concern of the client and is not a coveraup for underlying affective prOblems or concerns. This also applies to the specific and exploratory categorieS. Respon cs are not scored understanding if it is obvious that affective problems are of more concern to the client than c0gnitive content, but the counselor continues on a cognitive plane. In such cases the counselor's response is also nonspecific and nonexploratory." "Can a counselor's response be specific without being understanding?" "Yes. A client can present point blank to the counselor a central concern of his and the counselor may comment on it without demonstrating any understanding of its significance or may comment on it is such a‘waj that there is no further pursuit of it." "flhat is the main criterion for specific?" "When the counselor's response stays at the same emotional level as the client and.it is able to zero through the client's verbage right to his central concerns without being side-tracked by less important affective or cognitive concerns.“ "If a counselor's response is such that it is possible that the client could respond in any way that he chooses, is that response scored exploratory?“ “he. For a counselor's response to be scored exploratory it is essential that the counselor's response not only permit the client to respond to arw'length or any depth that he chooses, but it must also reflect that the counselor has some under- standing that there is some aspect of real concern to the client and his response is thus encouraging further elaboration of this concern. This concern of the client may be vague and still not specifically identified by the counselor, but he “2- w -131- understands there is some kind of concern there and is seeking further orflorntion or clarification of it. ”Iln not quire clear as to the difference between an effective response scored 4 and one scored 3," “1n effective response scored 3 is one in which the counselor is appropriately with the client and.is helping him to express more clearly his feelings. An effective response scored H-is tl's also, but in addition it is one which demonstrates rather exact appropriateness and timing such that the response vividly reflects back to the client the essence of what he is feeling.“ APPEI‘IDI X E STATISTICAL "t"-TEST ANALYSIS COMPARING THE A-V SCALE DIMENSION SCORES OF THE SCALE DISCRII-‘INATION STUDY'S FIVE PH.D. CANDIDATES WITH THOSE OF THE FIVE M.A. CANDIDATES Affect-Cognitive Dimension Ph.D. Candidate Deviation Squared Counselors Scores From Yean Deviation 29 -6 2 38.hb' no 1+.8 23 . 01:: 2. 2 4.81; 8 . 614- 8 C4 f.) U G L‘- VV »-’\/V k») S o 2.8 2. 2.81; Sun 176 74.80 3 5 .1— 35.2 variance (821): ZQ.8 = 14.6 5 K.A. candidate Deviation Squared Counselors Scores From Yean Deviation (B) 26 4.8 23.024. (E) 2 -1.2 1.1m (F) 20 —1.2 1.1% (H) 20 -1.2 1.144 (I) :20 -1.2 1.44 .1U1 106 28.80 ' 5 §% 21.2 variance ($22) = 28.8 = 5.8 5 t: 331-332 = 35.2-21.2 2 14.0 = 14.0 = 6.2 "'— 0 2 “14.6 + 5.8 (g) in]. S l + S 2 %: +-;EV> 2 b A - ‘4 0.1 degrees of freedom (df) = »4 ~ v o _ _ o .l + ;.7 - t1. "" lo - 2 "' v w 9”. Lu._/ 1 3.36 significance at .05 level significance at .01 level firm-fl Understanding-Konunderstanding Ph.D. Candidate Counselors Scores (1.) 31;, (C) no (D) 40 (G) 37 ( ) A0 Sun 191 5 ii 38.2 1:.A. Candidate Counselors Scores (B) 24 (E) 20 (F) 21 (n) 20 (I) 20 Sun 105 E 5 3% 21.0 't = 38. 2 - 21.0 = l .2 = V 5.8 + 2. (é) V1.6 (if = 8 .gLeni"ieaneo at .05 level sj,3nifieance at .01 level K») ‘~J) VON J Dimension Deviation Squared From Kean Deviation -4.2 17.64 1.8 3.24 1.8 3.2M -1,2 1.44 1.8 3.2a 28.80 9 Variance (8” ) = 28.8 — 5.8 1 r J Deviation Squared From Kean Deviation 3.:) 9.0 -1.0 1.0 0.0 0.0 -1.0 1.0 -l.0 1.0 12.0 variance (322 )= ___9_= 2.4 5 12.2 = 13.& 1.28 H O\ Specific-Ionspecific Dimension Ph.D. Candidate Deviation Squared Counselors Scores From Kean Deviation (A) 27 -0.0 70.56 (J) “v 3.6 ..a. Sun 177 105.20 v 5 L1 35.4 Variance (321) =10§. = 21.0 “ 5 I-I.A. Candi<.‘.ate Deviation Squared Counselors Scores From Kean Deviation 21 .8 60 (3) . (E) 20 -.2 .04 (F) 20 -.2 .04 (H) 20 -.2 .04 (I) 2 -02 col-L Sun 101 .80 ? 20.2 variance (322) = .8 = .2 Kn t = 35.0 - 20.2 = 12.2 = 15.2 = 7.4 2.05 21.0 + .2 2 4.2 2 '3 df = 8 significance at .05 level = 2.31 significance at .01 level = 3.36 '7 -.L A f . I 3) Q- wloraeo"‘— 'onozrp101atory Dimension Ph.D. Candidate Counselors Scores (A? 27 (C) L50 (D) 37 (C?) 38 (J) 40 Su. 182 7:1 36.4 L.A. Candidate Counselors Score (3) 21 (E) 20 (F) 20 (H) 20 (I) 20 Su. 101 5 31, 20.2 t.:364-20.2=16.2= \/ 221+ + ,2 (3) 4.7 5 CT) df significance at .05 level significance at .01 level Deviation Squared From Kean Deviations -9.4 88.36 3.6 12.96 .6 .36 1.6 2.56 3.6 12.26 117.20 variance (821) = 11 .2 — 23.4 Deviation Squared From.Mean Deviations .8 .64 -.2 .04 -.2 .04 ’02 .0“ -.2 .04 .80 Variance (S72) .8 = .2 5 Effective-Yoneffective Dimension Ph.D. Candidate Deviation Squared Counselors Scores From Kean Deviation (:1) 1158 —lO.3 116. 6L; (C) 62 3.2 10.26 (D) 58 - .3 .6@ (s) 60 1.2 1.44 W .611. 7-2 .2322. Sun 29% 180.80 1 5 ii 58.8 Variance (521) = 180.8 = 36.1 Deviation Squared X. A. Candidate Scores From Dean Deviation Counselors (3) AZ 6.9 2a.01 (‘3) 37 "' cl .0]. (F) 3]- “601 3702]- (x) 38 .9 .81 (I) 28 .9 .81 Sun 186 62.85 - 5 ‘2? 37.1 Variance (52?) = 62.85 = 12.4 ' 5 t = 58.8 q_37.1 = 21.2 = 21.2 = 6.9 3.12 _— 2 5 df = 8 significance at .05 level = 2.31 significance at .01 level = 3.36 APPENDI X F STATISTICAL ITEI AI‘EALYSIS OF THE SCORES OF THE SCALE DISCR'U-E'LNATION STUDY BASED UPON HOYT'S COIIPUTATION OF RELIABIIITY BY ANALYSIS OF VARIANCE ..'L"" 2,- ‘. 39ft's Statistical Anallgis for Determining haximum Likelihood Estimate of Reliabilig (Internal Consistenc ) and Interjudre Reliability QQ. Sum of Sguar es garlangg K ,C 6’? ? 9 Counselors d—l SSR = gig .. @_ S R = DDR /R-1 (ROMS) C RC Raters C 5 9 £5 2 1 ‘L S ‘ - H ‘- (Co—wn > C-l 58¢ _.LRiL 1.19.. S c 95c/c-1 ..\ - n V " 2 4" error (2~l)(c-l) So. = SST - as. - SSC 8 a = Saw/(a.1) ‘36 fifc Total RC-l SST = $2 X2 - 6 x 2 RC (1) Kaximuvaikelihood Estimate 2£_Reliability (Internal Consistency) The reliability of the average ratings or the total score on the test. 2 2 rtt = 00 R - 53 E 2 SS R (2) Entorjudge Reliability The average reliability of all the raters or items when between- rater variance is to be removed. This is done when decisions are based on averages of complete sets of ratings from all observers or ratings. SS E r " SS - tt R _Cd SSR + SSE (3) Interjudg e Reliability The average reliability of all the raters or items leaving in the between-rater variance. This should be done when decisions about pupils are based on a single score or rating which may be given by different raters. 830 + 331 a R .. C-l 89R + (SSC + SSE) '1 J! I". “1" U '- -4- .nitive Dimension £3 Affect-Co Sum ters Response Counselor QB/OQJQJ/O/OBQJ/O/O 531J/O 36.4.46 1121129~1122111212112 112112n41122211212122 2121122119~221_1210~212 O 1234 ..)/0700901 Add) fry/on/nonynu l 11.11 11192 1 A nu. 3h». 334/0536 3.414554 (.2333 11111221121111111111 21111122121112222111 11211122121222212111 1234567890HZAJQVJ/O7m090 1 111111112 Up K8 /0 /O /Q /0 22222 9... 2 n4 on A4 Op 0... 7w fig 2 1212.02 5 C Sum 4m- Raters (Aff.-Cog. continued) 6/0/06/O/O/O/O/O/O/D/U/O/O/O /036/O/033/D/O/0u45/O/O/D/Oh~.hw.3/O BOJQJWrBQ/aJQJQ/B 0L2222226¢20422222 21222112221222221112 1111111111 2222229~9~2222222 2122211222222227~2212 111211—1111 2220.~2229~o/~?~22222 2120~2112221122221112 1111111111 8 S n MUHCU 750 D!) O _.l n/H 3 a 1L11 \. a) nu 11n4 anw :J/o Ozno n/nu 1 9w 1* 4, (.267. 1.11 n09 11 - 700 O/ O l 9‘1 thw fJ/D n/OU 0/0 1 2 l 1112 111 3 Re Counselor (Aff.="og. continued) Raters Sum .l. 3333333333 1.11111111111111111. 1111111111111111111. 1.111111111111111111 m m 0 PH 23hr 5,0 73 90 MW 111111112 an Counselor Nu. 3333.33a330J333333333 llllllllllllllllllll 119~11111111111111111111111.111111111111 1111111111111111111111111111111111111111 l 2 3:4 5/0 700 Q/ 10 6 5.4666 3.4 5366/0/06 Oglln/undnéllZlOLA/gn/JZZ OLZZZOLZlZZlA/ezn/uA/LZ 29~19~221111n¢0~d¢29~ l 2 34 126 73 0;, O l 2 3h. 5 l l l l l l a} l -l (Aff.-Cog. continued) (1 01.173”! Jhtors Counselor 55/0 (0. 6 22222 04 0;. ad Ad 2 11717~?~ol e S n 0 .01/0 700 9 O "s1_111111o »O. . . a333330JOJ33333333a33333 111117.11111111111111 11111111111111111111 11111111111111111111 l 2 9).” rJ/o 7.3 9 O 2 3b. 56 78 90 l «lwlqlwfixulalfllulfin/w 30-41/02?)OJ333A)aJ33¢J30\/n)33nfl/ llllllllqlmlwllllllllllwl 1111111111111111111_11111111111111111111 lllllfilmlllnlullfilllllwl1uqlm1 l 2 a211,”. 5-70 700 m7 0 l 2 3.1... rJ/O 7. a H, O, “I. .71“ 1A.!“ 1 1L 1.” .1-.. 11 7. l_ O n a _1/1': AT. ri- \ (Aff.-Cog. continued) Raters l 2 _3 Sum Counsrz-CLor .{csponse J l 2 2 2 6 2 2 2 2 6 3 l 2 2 5 £1, 2 2 2 6 5 2 2 2 6 {3 2 2 2 6 7 2 2 2 6 8 2 2 2 6 9 l 2 l 4 10 2 2 l 5 ll 2 2 2 6 l2 2 2 2 6 13 2 2 2 6 14 1 2 2 5 15 1 1 2 4 16 2 l 2 5 l7 2 2 2 6 13 2 2 2 6 l9 2 2 2 6 20 2 .2. .1. .2 Sumz 281 292 276 849 Sum2=3975 (849)E=720, Sum 443 476 428 g x‘=134-7 (Sum)2 240,401 Source of Variation df Sum of Squares (SS) variance (82) Counselor 3212.- 220,801 = 123:7(ssR) 122.2 = .62(32R) ReSponsos 199 3 00 ‘ 199 Raters 2 240,401 - 720,801 = .7(ssc) ‘1__ = .35 (32C) 200 00 2 Error 398 145.7 - (123.7 + .7) = 21.3(383) 21. = ,05 39’ Total 599 1347 - 720,801 = 145.7(33 ) OO ‘ T (1) Internal Consistency = .62 - .0 = .212 ._ _..:2:F_J§ (2) Intorjgdgg Reliability (between-rater variance removed) = '62 " 005 = 07,2 .52 + TB-T) (.175) (3) Intgrfiud 2 Reliability (between-rater variance left in) = 127.7 - (~Z + 21.3 = .729 2 127.7 + (.7 + 21.3) :7 1"“.1 x) \ I. , ndingrDixension n cl ugLonunderst Understand'n~ Raters Sun OJ 02189 J11 2 024 5/0 73 90 l 2 o3 . ( ‘ 0.. R Counselor 46/0 556652706 5336.4 55.45 1202292722212721112a42212 lZZlZZlenLZleleln/NZ 20920212929~1020L211212211 .HN. .r 1:..Iulwl_l.l_ql~ 64.4 333.4 55533333)...Y 3333 2.111111111111119222111111111_2211111111 211111221111111111ull 222l11122211111113_17* 123.0(5/0780/ lO ,0 6 IO /0 2.0 2 2 0p Ad 0;. 972 2 22 d; 7” 9H 2 n4 n4 1 2 c.,.hw r) (Undersb . -Zfonundcrst . cont inu ed ) Sum 2) O S 1 .0. 3 .. ..a S I am TI. 01‘ 1:“ 0 OJ C011 n /O/O/O/O/O/O/O/O/O/O/O/O/D/O/O 22222229~2222222 n4n4222222222n4222 A.L222229~22222222 go 7.8 90 l 9 9:74 .1340 7.8 O/ O 111. lljxllaimnina /O/O/O/D/O/O 566666666656 5/0 22222222222222222222 2222n42122222222222169 222220n2922222222219~2a~ 1 Op 31.0.4. 5/0 7nu Q/ n 9 19 U OHZBbrS/nv? 1 111111 1.8 {.4 c3 3 a) .4 3, 3 3 23 a) a) lailn/“llwilll llllllllll lqilllllalflwlnav.‘ l 0‘, 3-4. rJ/U 7‘09 0/ AU 1“ Pg -12! (Underst.-Honunderst. continued) Raters 9~ 1 3333333333 11111111111111111111 1111111111 11111111111 dosgonsc ll 12 13 l4 15 m W 16 D 20 Counselor 363333333333333QJ3333 12111111111111111111 12111111111111111111 12111111111111111111 l 2 3.4 5/0 7nUQ/m 1 02 3b. ”..)/0 7nv 0/ 6/0/7046/0 rJCJ/O 55/06/020 2221220204202729~2292 222222192219227~29~ 22219~Og 0....12nalnd2922 01 0” 3nd,. .LJ 11.!“ 1.1.11” er; .- -l inuod) J. v (Undorst.-Konunderst. con- Raters 9n” 5,. A, . 1L. 66/066 22222 22222 22222 G S O Dvonfooo/no 311.117.?" p u. ‘ ounselor '1 J ( 3333a BBBBQ/BBBQJAJQJBBOJOJ 1111111111111111111111111111111111111111 1111111111111111111111111111111111111111 1111111192111111111111111111111111111111 .l. 9~ 91/4 CJ/U 7.0UO/O 23 Y5/O an“ 1 l l 333333333a233BnJGJfiJOJAJBQJ 1111111111111111111111111111111111111111 llllllllllllllllllll 111111111111111111111111111111111111111. f 3/0 7nU 0’0 123:“. 5/0 700 0/0 3rd w l 1!— 7. l .1. w; A,” 2 .|_ l -1 171,0- —— - / (Caderst.-Ionundorst. continued) Waters 1 2 _1 Sum Counselor Rosuonse J l 2 2 2 6 2 2 2 2 6 3 2 2 2 6 a 2 2 2 6 5 2 2 2 6 6 2 2 2 6 7 2 2 2 6 8 2 2 2 6 9 2 2 2 6 10 2 2 2 6 1.1 2 2 2 6 12 2 2 2 6 13 2 2 2 6 14 2 2 2 6 15 2 l 2 5 l6 2 2 2 6 l7 2 2 2 6 8 2 2 2 6 l9 2 2 2 6 2 .2. ..a. .2. .6... Sum.) 297 290 299 886 Sun12=4318 (886)2=781+, 993 Sam; 491 470 U97 25'X2=1H53 (Sum)2 261,710 Source of Variation df Sum of Sguares(SS)_ Variance ‘82) Counselor 1+ 18 - 8b 96 = 131.0 1 1.0 = .658 .25.. 2'36%" .2_._. Responses 199 199 Raters 2 231,710 - 234,996 = .3 .2__ = .15 200 00 2 39' Total 599 1458 - 284,926 = 119.7 ' 00 (l) lgyornal Consisteugg = .658 - .046 = .220 (2) Intorjuige Reliability (between-rater variance removed) = .653- .ous = .316 0 6—58 + (3‘1)(o 0746) (3) Intorjugge Reliabilitg (between-rater variance left in) = 131.0 - (.3 + 18.u) = .811 2 131.0 + (13 + 1835) Raters CifiC-llonspocific Dimension ?)0 __g r1 .3 35643663366433534434 unnflJq/333353333333330J3 6/06/06 11221221122111212212 lllllllllellllllllW+ 22222 12211221122211111111 11111111111111111111 22222 192211221122111..le111 2211.1111211111111111 20~220N e S n 0 Pl 2 3h. 56 78 90 1233. 5/0 78 MW 1.1.11.1qlalwigl MUN 12 3.456 789 1231:.er r 10 Counselor A B C (Spec.-Uonspec. continued) Raters Sum 66/06/0666/0/06/0666 20L22222227~22222 222222222222222 20422222220~9~222n4 Rosnonoe 6 7 8 9 O 2 3 '+ 5 6 7 Counselor 6566/043u/O66/D/0/D/D/Ouu56 921.9292921.121292v272929292929212129292 29.2222le222222222le 229~22112222205222110~04 l 2 3.4567890 1.72314 56 73 90 l... ljfillllez 33?;3333nfl/033 1111111111..— 1*111111111 1212121.1_1212121212 1 09 «3.4 5/0 7.00 gm «nu -152- (Spoc.—Fonspoo. continued) Raters Sum esponse I) -~ Counselor 3333333333 11111111111111111111 1111111111111.111111 1.111111111111111111 11 12 13 15 16 17 18 l’) 20 33333333q/OJ3333333333 1111111.111.11111111111111111_111.111111 llllllllllllllllllll llllllllllllllllllll 11ohqynw:JAUnroonynunu02QJthJAovlnoo/no 1 111111112 6564/Oubr 55.4 5/0 5/06 21212122091229~22 9fi9292929211117292119292929202 ZZZlZlelOQlZlOun/u l 2 3U. 5/0 7nU 0/ AU 0 l u 2 6/. 1171 hr 1 p.) l 7 L”? ””‘J/H. (Spoc.-Uonsnec. continued) Raters Counselor Response 6/06/06 22222 22222 222292 17 18 1 20 16 32’30J3333333333333333 111.1111111111111111111111111111111111w1 11111111111111111111 11111111111111111111 123+rJ/0 78 O/OJRU ll 33333333333333.333333 111111111111111111111 111111111111111117111 lllllllllllllllwiflllll 1231.». 5/0 78 0.,OJZ3L15 11 1111 l {3 17 1d 19 2 O (SDOC.-Ionspoc. continued) Raters l 2 3 Sum Counselor Response J 1 2 2 2 6 2 2 2 2 6 3 2 2 2 6 4 2 2 2 6 5 2 2 2 6 6 2 2 2 6 7 2 2 2 6 3 2 2 2 6 9 l 2 2 5 10 2 2 2 6 . ll 2 2 2 6 12 2 2 2 6 13 2 2 2 6 14 2 2 2 6 15 l l 2 4 l6 2 2 2 6 17 2 2 2 6 3 2 2 2 6 19 2 2 2 6 20 .2. .2. .2. .6. Sum. 270 280 281 839 Sum2=3897 (839)2=703,921 31m" 1+3!» 1:40 1:43 g X72131? (Sum )2 234,645 Source of Variation df Sum of Squares(SS) ‘ Variance (82) Counselor QQQZ - ZOE3221 = 125.80 12§.8 = .632 RGSponSGS 199 3 00 199 Raters 2 224,645 - 0 21 = .03 ‘;92’ = .015 200 00 2 Error 398 143.8 - (125.8 +.03)=17.97 12.92 = .04 3 0 5 Total 599 1317 - 20%;.921 = 143.80 00 = .222 (1) Internal Consistengl = .632 - .045 . 32 (2) Interjudgg Reliability (between-rater variance removed) = .6 2 - .045 = .01 .632 + (3—1)(.045) (3) Enterjudgg Reliability (between-rater variance left in) = 125.80 - (.03 + 17.97) = .812 2 125.80 + (.03 + 17.977 Sum ry Dimension A Foncxnlorato Raters 7?:933101‘atory-1 Ll U‘5ébkpUr/O/D33/0/0433.5h~..4h~.3v4 “43333335433333.430233 5/0/05/0 110227.2le22111.222le 11111111221111121111 292929.72 1.2211221122le111111 lllllllqillllllllllll 12222 02221102211nan/2117.21.11.11 227_llllllellllllW.ll 22212 , O 1:" 0.. OJU. r) 12..-.45678901234567390 1234567890uo,.34567 O 1111 1 1 1 1 1,. l «I. at” l a!“ «in OH 1 .|_ _.I 18 A R0 sponsc Counselor A B C 'J .I,u \ u _j» _l‘ ' Raters n:91 . -l Tone:-:p1 . continu ed ) .‘ LL: ( /O/D/.U/O/O/O/O/O/O/O/O/U6/D6 642222220u220~2222 222222222222222 2229.. u26¢n¢7~229~9~29~ 8 8 RI. 1 O . 0.6 780 0/0 2 oJuUr wag/O 7mg n90 MW 1711~lllfinnlql~ng .3“ lor Counsc 666664446 56666664 556 222221120222222221222 22717~0921120922222229~12 22222121212222221122 l 2 3.4 4&6 7.8 90 u 2 6 78. my . _l ther 1 111111 20 QIBBQ/BBQ/QJBB 1111111111 1.1.1.1.1.1.1.1.1.1. 1111111111 1 2 3b,. r36 7.8 9m. E -157- (Exp1.-Ionexpl. continued) Raters Sum Response Counselor 3333333333 1111111111 1111111111 1111111111 33333333333333333333 11111111111111111111 11111111111111111111 11111111111111111111 1234567890u234567890 1 1111111 2 666.46 5:4 5555/0 56/0 222122227~7~22222 222221122127~2022 202212~21112121_022 1 2 «)4 56 78 90 2 «)4 ..D 11 ..111 «11 (Exp1.-Nonexpl. continued) Raters 66/0/06 22222 22222 22222 wcsponse m N 18 D 2 q Counselor 33333333333333333333 11111111111111111111 11111111111111111111 11111111111111111111 1234567890fl23 567890 1 11 1111112 M H 33333333333333333333 11111111111111111111 11111111111111111111 11111111111111111111 11.253n4p3/0n(oon7nu 1n '7 f‘ — 5‘:- ._ / (Exp1.-fionoxplo Continued) Raters l 2 3 Sum Counselor Tesponse J 1 2 2 2 6 2 2 2 2 6 3 2 1 2 5 4 2 2 2 6 5 2 2 2 6 6 2 2 2 6 7 2 2 2 6 G 2 2 2 6 9 1 2 2 5 10 2 2 2 6 11 2 2 2 6 12 l 2 2 5 13 2 2 2 6 14. 2 2 2 6 15 1 1 2 u 16 2 2 2 6 17 2 2 2 6 3 2 2 2 6 19 2 2 2 6 .2. .2. .2. .62. Sum 276 278 291 845 Sum2=3933 (845)E=714,.25 81111120 428 434 473 g X“=l3 3 5 (Sum)“ 238,141 Source of Variation df Sum of Squares (58) ~ variance (32) Counselor 3223 - 14 02 = 121.0 121.0 = .608 ReSponses 199 3 00 199 Raters 2 238,141 - 214,025 = .70 :22, = .350 200 00 2 Error 398 145 - (121.0 - .7) = 23.3 22.? = .059 39 Total 599 l - 14 02 = 145 600"2 (1) Inter-mil Consistengy = .608 - .252 = .903 ‘ .608 (2) Interjudgg Reliability (between rater variance removed) =.608-.059 =. {'6 .603 + (341711059) (3) Intcrjudgc Reliability (between rater variance left in) = 12.1 - (.2 22.01) : 07 22 121 + (.7 + 23.37 0 -160- Effective-Koneffeetive Dimension Raters 301186 ‘2‘) . “Cb Counselor 67n0 77mO/6/099/0550/78 7/0 7. 91nd «)33hW3n/123G222 233332 3 01.227123322332113271222 91. n)3r/~ 2339;. 20/322 230.1371 2 9.. 7.1013675/07n0O/O 0* GJ 315/0 7nd 0/0 111.1. ; 1.11.13 A 8 76/0667688666/067/0555 222220921332229~23229~2 21?.727.71711Jq/91919171912191o2929.0192 a1,n1.n2 02 02 on 02 n2 51.51.02 02 02 n2 o2 o2 02 11 11 11 lOLOJhWKJ/O7nUO/0n21/ r l 1111. 17 13 19 23 11' 15 16 an 10 10 10 9 10 QJ o) 3 3 «J A) OJ 3 a.) 3 1..“ 9” OJ 71”. r) Pg —j. 6]_- .f.-Koneff. continued) -fifl 0' (1 Raters Sum 34 34 3444 34 3444 3 44 3333433333334 34 34 344 33333333 8 s m .016 7.3 90 1. 2 31.1. rJ6 78 90 MW 31711111111116“ F R Counselor 9990067799902u996889 11 ll 33OJ332 2 BBAJOJU.“ 3332 333 333.4422 2333344332323 330J3n32 32 91/333414 332 2 33 123456789 10 5666 333/0/0/0 22221111626291 22221111112202 loan/10111191912 l 2 QJU. 5/0 7nU 9m Wu (Eff.-Uoneff. continued) Raters Sum )OnSB Ros; Counselor 66666/06/0/06 9wn49~0L29~2227~ 9;.0420420422204 n4 «4 2 2 2 Ad on Ad ad 0L nwcjnwnwnwnwnwzJAchnwhw:chAuchjgucjnw O~2111112221111222211 1*2222222211122211222 111111112227~220u22221 .l 9” 0.4M- EJ/J 7nd Q/ 10 ll 1? 13 L 5 6 l7 9 O 90/0/7987000/0u00 9 OLOO 111 3330.." 33333334 330) 333332 2 332 3.4 bf 33 3332 OJQJn/w 2 3n3néhw «QM/w. a) l 2 a)” 5,0 78 9 O u 2 n.2,»- r) (Eff . -Z?onoff . continued) Sum 3 9990/9 33333 33nJOJ3 023 3.?J3 l. ”.os‘mnso 16 17 n J 19 20 Counselor 6556/0 5.451266666666665 2222211110L2222222222 2112042221222220u9~2221 22222212122220~222222 al— 2 3h”. 5/0 78 O/ 10 64433655466/0/055666/05 21211210412222227~227~2 2111122212222119~229~1 2211122122222220una2nd?" O l 2 34 5/0 78 9O 1 2 3.4 5,0 7.8 9 1.1. 1L1;11._llqi~l I ‘4 A (3:f.-uoneff. continued) Raters l 2 43 Sum Counselor Response J 1 4- 3 3 10 2 4 4 3 11 3 3 3 3 9 1.3, 1+ 3 3 10 5 14, 14, 1+ 12 6 3 3 4 10 7 3 4 4 ll 8 3 4 4 ll 9 3 3 3 9 10 3 3 3 9 11 3 4 3 10 12 3 3 3 9 l3 4 3 4 ll 14 3 3 3 . 9 l5 2 2 3 7 16 3 3 3 9 17 3 4 4 ll 8 4 4 3 ll 19 4 4 4 12 20 .3. .3. L .2. am 476 474 491 1441 Sum 2=11381 (1441 )2=20764: 1 31117122 12 54 1252 1351 i X -3857 (Sum) 692,333 Source of variation df Sum of Squares £§Sl_, Variance {82) Counselor n381 - 2 076 481 = 332.9 322.2 = 1.672 Responses 199 00 199 Raters 2 6926222 - 2 076 481:. . = .4’0 2 O 00 _‘g— ) Error 393 396.2 - (332-9+.9) = 62.4 62.4 = .157 T t 1 "99 3357 2,0E6,% 1 396. 2 39‘ o a 3 ~ < - = (1) Internal Consistency = 1.672 — .157: 906 1.672 (2) lgfierjugge Reliability (between-rater variance removed) :1 672 - 0;? = 9 6 1672 + (B-lW 1577 (3) Inter’1d39 Reliability (between-rater variance left in) = 332.9 - 1,9 + 62.4) = .260 ’) L, 333'? + (19 +“525¥7_ APP EITDI X '3 STATISTICAL ITEI AIL-XLfSIS 0‘7 TH Ls) SCORES OF THE CBJSCI‘IVITY STUDY BASED UPON HOL’T'S COIIPU'I‘ATIOI‘I OF RELIABILITY BY AI-IJ‘LLYSIS OF VARIAI‘TCE .165- / ," ~130- :ensien Affect-Cognitive 313 Raters Sum Resoonse Counselor 6 3.333a9/O/O/O/O EKG/C 302:4 BBQ/33333333366 551w. 333333/0/0/0 OJQJOJQJOJQ), 62 11 11 11 11 11 62 62 62 62 62 62 62 11 11 62 11 11 11 11 11 11 11 11 11 11 11 11 62 62 62 62 62 11 11 11 11 11 11 62 62 62 11 11 11 11 11 11 921111111111929292921192921111111111111111111111111111119291111111111111111111929292111111111111 91111112220,2912211111111111111122221111111292le1111 1.11.01 OJHW rJ/O 1.“ 2 QJIIW 5/0 71 2 314 5/0 ?8 O/l 2 «314 1.2 31H». 51 2 «)4 5/0 1 2 3'4 r311 0201/14 ,rJ/O .i‘i Ill}... -1157- ) Aff.-Coy. continued ( Raters SL1 37‘: A 1’186 0330)0J3333333333333333333333333333356533.4 3333333/C 1111111111111111111111111111111111111111111111111111111111111111119272921111921111111111111192 11111111111111111111111111111111111111111111111111111111111111111111929211111111111111111111o2 1111111111111111111111111111111111111111111111111111111111111111119292111111111111111111111192 o .91123412345,07812345/07112312341234561234112345,.D78123 S x} R( Counselor 9 10 11 12 13 14 .-Cog. continued) Raters «4.2 A 1) Denise Re Counselor 145336 BGJBQJAJ/O 30.236 33333333333336333333333014n3336331130) 7292111192111111111192111111921111111111111111111111111192111111111111111111111111111111111111 1111111192111111111192111111921111111111111111111111111192111111111111111111111111111111111111 1162111162111111111192111111921111111111111111111111111192111111111111111111111111111111111111 rind. 5/0 700 1 Ad OJrUr 51 2 3b. 51 2 31“.? 5/0 7.8 90 l 2 3h. 1 9N 3n. 1 2 3.4 5/0 78 90 l 01 1.14... _.l_ l 19 20 21 22 2 24 18 . continued) {r L) (Aff.-C01 Raters Sum A 8 S n m 1 51 S Re Counselor 33n233333333333330233333n1JOJ333333333333336/O/O/O5/0 333 11111111111111111111111111111111111111111111111111111111111111111111111111119.9292929292111111 1111111111111111111111111111111111111111111111111111111111111111111111111111929292921171111111 1111111111111111111111111111111111111111111111111111111111111111111111111111329272929291111111 1234512312345671234123.4463/0123456711234561123412345 on QJ OJ 3) 25 26 27 2:3 29 \ 30 31 ’3‘ £1 (Aff .-Cog. contimed) Resoonso Counselor Big/333333333333330J3/O53H. 3334333633n)33h~. 3,3n_JA},n)QJSn.).OJn«J.GJ llliLllLillLllllllZZlo,~1112111211111211_L.LLlLLLlL ililLllilllllllllllelllllllllelllllillllRHLLLL Llilll.LilllllLllllZ21illllilllelllllLiiLiRiiLL to 7.3 i 9., 3a. .51 2 3a.. ..)/0 73 9mm 1.2qu 2 3.4 51. 2 oJflv 5/0 1 2 374 5/0 7.3 l 2 3 ml 5 , 7. 34 3 30 37 8 9 continued) ‘1‘ (a. (Eff .-Co Raters Sum A 333n33333330J3333333333330J3/0/D333333.06OJOJDrSr01033.3/Q/O/O llllflflkLlllLVLlllllllllllW_l22111l11221.i22ZXlLLn/NZB .lllllll.iill_llllllllllilllg21111112211.12221iL222 «L.L.L.L1L.;.;1L1u1+nuI*1*1*I*1*1*1*1_1*1*1*1*1*1*obvslu1*1u1.1*1.9~9~1*1*1*1*Quasi Lwin4n¢o~ 01180 Jul 2 43):“: 5,0 o... 2 3d. 5,0 700 l 2 3E.5/O 7i 2 «3%... 5,40 l 2 3.4 51 2 3H“. CJ/O ..lm 2 ”3.4 l 0;. 3U: 5 7 Q has. Counselor ’L 42 43 44 [+5 46 WI. 1; (Aff.-Cog. continued) Raters 1’1 3330J.fu/O/Or)5./OU.105663rO/O 30J36/O/OOJ3366/0— lli.inp9‘.2n/~OLQ~9~29~22127~lll2092111222— llLlné9~20~12129~9~217~7~lllZZn/klllGun/~2— la!”I.loan/Hallnan/ulOLWLA/NZHLZn/Jlll2221.11222“ “l 2 3&1 VJ]; 9w nQU. 5/0 7.!” 2 02.4 r.) ,0 71* 2 3M. 5i 2 34 5/0 Loo)onse CouheeLor pg 50 51 52 53 12566¢1 7 2].)~ Sum2=u321 (11 367 383 1121 371 ‘ 475 523 457 9‘. '7‘. 1) 419,019 (Sum)2 (Aff.-Cogj. continued) Analysis of Variance Source of , Variation __ df Sum of figuares LBS)» Variance 158) Counselor Responses 3L2 412i - l32§oaofll = 102.0(SSR) 102.0 = '330(SZQ) 3 939 312 ‘ Raters 2 412019 .. lazgeaénl = 6.5(ssc) 9&5; = 3,4(520) 313 939 2 Error 624 146.7 - (l32+6.8) = 7.9(ssE) L2 = .013 6:34 Total 938 was - 1,256,641 = 146.7(ssT) 939 (l) Internal Consistengy =_;330 336013 = .g%% = .961 (2) Intcrjudgg Reliability (between-rater variance removed) = O I- .01 = .820 .330 + 3-1 .013 (3) Interjgdgg|ReLiabili§1 (between-rater variance loft in) = 132.0 - 56,8 + 2.22 = .820 132.0 + (as: 7.9) -17! L— Uonundcrstanding,Dimension 1' 3 L111 57- A \ Understanr Raters Sum A Response Counselor ((3 3/0 3.0.1 3/0 066506435333333333333/06 5/0434 333QJ/Ohw..b joJ/O uJO)...L- Quin/will?“222222210¢llllllllllll229~22121ul1 21.XllizzZZlZlel.lllllll.l—lllLZZIlellll 2l2i2l2222222112llllllllllllZZZlellll .i 9., c.,).HLr rJ/Q I. 2 3h”. ..)/0 7i 09 BR. .126 7,00 91 2 3h. IA 2 3a. 5.1. O,” 9J9: 5/0 .1 LP 6 a . ..L3_i9w_l‘_l?wntull .|*n/NlleO‘._ll,Ll |_n/HVu./.ill0.flm,lwom 2 3h”. 5|" 3 n) M. K7? . Ill-IIIIIIIIII |.|l|l|..1| III .erst . -ITe:1underst . continued) Raters 01130 ..)}..Uv. O-/3n)330J33u4/.33BQJOJOJOJOJ/O/O IO 3330J3333331/IQ /O 53nw/50JQJOJAJOJNW. CJ/O llll lllllllllllllllz2 zilllliilllludthLIlfi/vllwlil_2i? lllllllllllllllllll222lllllllllllflZZIlu/llllll22 l/ll Pl 9“ H) u. I‘ u.) "j “ elor S U/ lllllllllllllllZZlelllllllll2211llllllll22 L 2 OJDr ..)/D 78 l 2 3:4 5/0 7.1“?” ”2.1.. Oh 3a. 1 05.. 3a. 5/0 i 2 u) M. 1L0“ nigh”. 50 7mu All a,“ Q i3 14 i5 st . ~130nunderst . contimed) ‘5 (Uncle Raters h». 5336 333336 3336 33333333333336 333333333333333364 221121111121112111111111lllllellllllllllllllZl lllllellllellellllllllllllellllllllllllllZl l211lellIlellellllllllllllellllllllllllll22 4567001234512345123456789m1234l23.41234567009m1234 mmom Cmmdw 18 m 20 21 22 8 2# -l"/ 7- (Underst. . -l-Ionunderst . c ontimled) Raters m 4.4/0 314. 53nd. 33333334333333333333333333333366/066 nfl/BQJH/ C 2.].2112121111....11211llllllllllllllllllllzzz22111...,"i BIlllelllllllllllllllllll11111111111111222221111 A l221Zlelllllllllllllllll1.111111111111222221111 123.451231234567123.4123hw561234567123456123412017.,wa Response Couns elor 25 26 2'7 28 29 30 _um (Underst.-Honunderst. continued) Raters Sum kwmw mmnmflor 33h. 33333333333333366 34.43333366 333333533311». 353333 11211111l11111111122122111112211111121111111111 llllllllllllllllllz2111111112211111121112121111 llllllllllllllllllz2111111112211111111111121111 % N w w m 3% 35 ~179- (Underst . ~33 onundor st . continued ) Raters 333333333333333333333333666 33333366.“. 34/06/04 33/0/06 11111111111111111111111122211111122112222211222 llllllllllllllllllllllllzz211111122111222111222 llllllllllllllllllllllll22211111122211222111222 123456123456781234567123.45612345123fi561234123h~5 Respo ¢‘e 1&7 1&8 1+2 43 Counselor 1+1 wL. (Undorst.-Nonunderst. continued) Raters Sum Rosoonso (3cn1rn3eflaoz- 3.4/03666/0/0/0/0/05/0/03/0/Ohwn4hW/066U.33/06/0— ll2122222222222122222222211222— 120u122222222122120ulll222111222— :1—2345123hw5671234567l2345123456 Ly) 5C) 7 a9 @937 (1183>2=1399u» 404 1183 3111,12: 385 394 Smn é x2=1671 529 586 556 466,677 (amwz 1 (3'1 - _...-Vl- .- (Underst . -llonunderst . c ontinu ed) sis of Variance Source of Variation di‘ Sum of ngares (SS) Variance L52) Counselor Responses 312 £221 - 1,322,182 = 155.27 1 .2 = .498 3 939 Raters 2 I+66,6zz - 1,322,162 = .56 :16. =.280 313 939 2 Error 621} 180.60 - (155.27 + .56) = 24.77 24 = .040 Total 938 167.1. - 1,322,482 = 180.60 939 (l) Intemal Consisten = #8 - 040 = 1+ = l “A ‘EW— {7:938— ‘22 (2) guberjggge Reliability (batman-rater variane removed) = 8 -. . 040 = :22 . + 3 . (3) Intermgge Reliabflitz (between-rater vanance left in) = 155.2? - (.56 + 24,22 = :220 2 \ 155.27 + (.56 + 21+.7T c-ITons‘oocific Dimension . ”'1 ch.»- Snocir ___‘ Raters A OJIJ. 3n330J5/U rJ/O 366br 3533333333333366h¥6 33333n33/0 350JOJ3333 121111221212221211111111111122221111111212111111 \ 111111122212211111111111111122121111111212111_11.1 11111112221221121111111111112Zllellllllellllll 36 3011 ml1 2 3a. 56 12 3b.. 56 7.1 2 38. 5/0 7.8 91 2 3:1. 1 2 3a. 51 2 3b.; 56 12 3h. 51 2 3h. 56 L) f.) o no Counselor 1 2 3 1+ 5 6 7 '11-»)fl \,\\ .-x.‘ (Spoc.-Ionspec. continued) Raters 31.1171 no Box/BOJOJQJQJOJ33333330J330J/0/Olo 33333333333564 33h. 33330.2«93/0 lllllllllllllllllllZ221111111111122211211111112 11111111111111111112221111111111112111111111112 11111111111111111112221111111111122111111111112 v.1ohms/'Urln/NBHWKJ/On/Oul2345/0711231234123v¢w56123412n94567n012o) . uUODSO ..) Counselor 9 10 11 12 3 14 U 16 N ’V ('3 ,‘ -.L.\)‘+’- (Sp ec . -'.Ionspec . continued) Sum w Re soonse Counselor I'Lr KJ336 333336 3336 3333OJ33333333333333333333OJBQJOJ333 22112111112111211llllllllllllllllllllllllllllll 11112111112111211lllllllllllllllllllll111111111 12112111112111lellllllllllllllllllllllllllllll 1.56781234512345123456789m12341234123456 78 9m1234 8 9 O 2 3 - 1 1 2 a 2 2 NA“ (Spec.-Uonspec. continued) Raters 3333333.”. 0J0)333333333333333333333333336)36/06/06 3333 1111111211111llllllllllllllllllllllll1222221111 11111111111111111111111111111111111111222221111 11111111111111111111111111111111111111222221111 .129245123123145/07123412n3hv56123n4567123Dr5/0120J4120Jn45 Rosno "e (u 2 0) OJ Counselor U 6 W 28 w 0 31 2 3 spec. continued) Sum Raters A Soec.-Xon ( Rosno so Counselor 33n93n333333333333335534 33333336 BBQ/333.4 333331.“. 3333 11111111111111111122121111111211111121111111111 llllllllllllllllll22111111111211111111111121111 11111111111111111111111111111211111111111111111 .6700123I4 5123456789101120J12a.Ju4512345612n)45/0 7Du.123hv 5/07.. w. % w my % W m (Spec.-Eonspec. continued) Raters Sum “D n933333333m)333333333333333-4U. 3333336 330J35/O/O Bax/366 3 lllllllllllllllllllllllllzlellll21111222111221 lllllllllllllllllllllllllllllllll21111222111221 111111111llllllllllllllllllllllll21111122111221 ”.1.— 2 0th rJ/Q l 2 3h». KJ/O 7.00 l 2 69.4 5,0 71‘9“ 3h“: 5/0 1 2 3h. 51.. 2 3h. 56 l 2 QJhW l 2 0.4.0,. 5 a CNS 9 q AUG 45 1+6 [4, Counselor M 1+ L‘y’ _ m ontinued) Snec.-Honspec. c ( Raters A 333355444666556 3663446664 3356.4— 1*1r1r1r9~9~9~erases?“9~9~9~9~1.9~9~1*9~9~9~9~9~9~1r1rososoé .llll2111122221212211122211122L . S'DOI'ISG T) ; Counselor 111—112111222122122111222111121— .l 2 3hr 51 2 3h. 56 71 2 3.4 56 71 2 3b,. 51 2 3D,. 56 1P9 50 5L 52 fl 1097 Sma12=LPl73 (1097)%12031:..o9 360 , 381 3% Sum fl? 401,u97 W x -l ,1! ('2'- (Spe c . -I.’onsp ec . continued) Analysis of Variance Source of Variation; df 1. Sun of Squares ES) Variance Counselor 41:23 - 1,203,402 = 109.41 109.41 = .351 Responses 312 3 939 312 Raters 2 1:01.1922 - 1,203,402 = 1.15 1,15 = .575 313 939 2 Error 621% 131.41 - (109.41 + 1.15) = 20.85 20.8 = .033 Total 938 1413 - 1,203,002 = 131.41 939 (l) Laterml Consistencv = .° - 0 = . 06 _.....— ......................u.. .2. 4 .351 (2) Enterjg' dole Rehamlitx (betV-Jeen-rater variance removed) = 1.351 "' 0033‘ = .26?! .35L +73-DT. 0337 (3) InterE' dgg Rehabilitv (between-rater variance left in) = 109.41 -(l.15 + 20,85) = .242 2 109.31 +ZI.IS + 20.85) l qtory Dir: ensien L 27110? Raters '3-2 Tone: at o: lor Yi‘h 14. .. 307,39Jhw. 3/06 5/0 3355333333333333333536 333n)333/0 OJ/D 0).ajnfixojejsrY 111121222211Gugllllllllllllvlll222111171111212111111 1111111111119~9~1l911111as911111111.1111111111111l1l1l1l119l1l7~1l111l11111l1191119~1l111111111l 111111222lelllllllllllllllllll21111111212111112 O S m mull Op 3.5. 5/0 1 Op 394. 5.6 71 2 3hr 5/0 700 91 2 31¢. l 2 3h. 51 ad 3h. 5/0 ql" 0w 43h“. 51L 7.. OJMW. rJ/D e. “7,. ‘4 ct. Couns ole 1'11 C‘ L.) Raters w—‘v fipl.-Xonorpl. continued) v“ “J. (.. Resnonso Counselor OJBOJNWDV 33330J334 33333336 33333333333336 3333b. 3333366 lll221111111lellllllelllllllllll2111121111122 llllllllllllllllllll211111111111112111111111122 llllllllllllllllllllZllllllllllllllellllllll22 «l23419~3v4567001234567123123,4120).“.5/0120Jv41234rJ/O70012OJ 10 ll 12 13 14 15 16 17 a (7") .. u. Waters Q A (Eqfl. . ~130nezqfl. . contilm 0d ) hv. 3335.0. 333333333333333333333333333330J33«Jaw/BBBBQJOJOJ 2111221111111llllllllllllllllllllllllllllllllll lllllllllllllllllllllllllllllllllllllllllllllll lllllellllllllllllllllllllllllllllllllllllllll e S m min». rJ/O 78123.451234 51 23h". 56 789.0212 3.412 AJbrl 2 3.4 5/0 78910.1 2 3.10.. M r o a. S 8 9 O a 2 3 .4 m l l 2 2 2 7 nu (Etcpl . -Honexpl. c ontinuod ) Raters finn D4 33333333333334933333333333333333333333n)/D 33h». SaJBnJOJ llllllllllllllllllllllllllllllllllllll211221111 llllllllllllllllllllllllllllllllllllll211121111 llllllllllllllllllllllllllllllllllllll211111111 1136 O $.12 31451 2 31 Ema/h... 5/071 2 0.2.41 2 3nd. 5612 3.“. 5/0 7.1 2 02.0,. 5/01 2 92.3. 1.” 2n9h~ r) Re Counselor 5 6 W 8 w % fl 3? 3 Tonoxpl . conti med ) l & q '1 ~ .5331 . .- ( } ‘ J Ra tars A 333333n33333333333354 3b. 333.”. 33/06 333333333330Jw3AJOw/493 llllllllllllllllllz212111leZlelllllllllllllll llllllllllllllllll21111111112211111111111121111 llllllllllllllllllllllllllll2lellllllllllZIlll 0 S n O m‘67nu1234512n3456789w1ZOJl23hW512345612345/078123.4rJ/07 Re. Counselor 5 6 37 38 9 Lo Y i ommLcmfimw) :2fl-o-l (w Raters Sum 1‘13 0 333333n3333330J3Q/33333314 333.“. 3333333464 3'46 56h. 3a3333 lllllllllllllllllllll21112111111112112222211111 llllllllllllllllllllllllllllllllll2111222111111 lllllllllllllllllllllllllllllllll22211212111111 0 $1 2 3.4 56 l 2 3.4 5/0 78 l 2 3.4 5/0 71 2 3.4 56 l 2 3a. 51 2 3.4 56 l 2 34 l 2 3.... 5 .0 D; Counselor M W #3 M 45 % 2;}? m ~r Lpl. continued) .4 4. ‘ml . -None P A44 (7 Raters A Re sponse Counselor 36 Surr12=3782 (1056)2=J.11 51 33.4 33.4Dvhw34u556u. 36/044466“ 33353; 1056 112112221222222122222222111211— 372 l1.1111111112221122111221111211— W 1111111111111211221112211lllll— W l23:45.1.234567123456712345123u5/O 49 5o 51 52 53 m X2=1290 #06 11.90 372, 320 394 Sum-2 (g) “51)2 (33:91 . -ITone:rpl . conti r11 ed) Analysis of Variance Source of Variatmu (11‘ Sum of Squares (g) Variance (82) Counflor Responses 312 2282 - 1,115,136 = 73.09 23.09 = .234 3 939 312 Raters 2 3:22.32 - 1.11513 = 1.94 1.0:1'. = .970 BB 939 2 Error 623% 102.392 -(73.09 + 1.94) = 27.39 2 , L = .0)th 2 Total 938 1290 - 1,115,13 = 102.u2 939 (1) Internal Comistency = .2 - .01% = .812 137+ (2) Lnterjuchje Reliabflity (between-rater variance removed) = 1235'; " 00141" = 0220 “2% + (3-1)(.Oifl1.) (3) ligaterjugge Reliability (betteen-rater variance left in) = 73.09 - (1194+ 27.139) = fl 2 73.09 + (17.91? + 27.39) Sun Raters A J T“feotive-Koneffeotive Dimension 966653998m599646 33r)33333645#9979666446 39/0 933533hw 322221333423n321311211111212233332221121323111111 322211332313322111111111211133232221121323112111 / 32222133330~3321211_211111222133232222221323112112 ..lOLOthrDélZAQhVKJ/anl234567891234123h¥5123h¥56123.45120.41456 Rosoonso Counselor 1 3 4 5 6 8 -‘9// -1 N: (Eff.-Uoneff. continued) Raters R% 646.46 35nd. 333354 33333mu 9.“. 33nd. 33.4 533Q8 97h”. 0J733333/060/ 2121211111112111111433111111121113332131111.1223 21212.1.21111111111113431111111111123211211111223 222221..092111122111113432112112211233211oalllqilZn/Ho) 9 S m $.1234123br5/0700123:4.5/071231234120J45/Dl23.410434567001104n) “m Counselor 9 10 ll 12 13 14 15 16 17 (Eff.-Soneff. continued) Raters no A 6 S n m. Res Counselor 78h. 300:4 3h. 339.455934554 333333346 33333333333333336 5 331132111131213111111111llll21111111111111.1111122 22.1.121111131123112211111111121111111111111.1112]. 23213112113222312222111111122111111111111111122 45670012«345123.4512345/0789012341234123hw5/0780/012a9hw l l 18 19 20 21 22 23 74 Noneff. continued) Raters "1"".9 ..JJ. .L . - ( , 644.4 5.4146 333333353333333333334 33355333398 999.4 p34 3 911111119211117211111111111111a21111111111111111111_11119211111192111111111.QJQJQJQJQJ92921111 921.92111.119292111111111111111111111.1111111_111111111111111111929211111111QJ02QJQJQJ11111111 221222121111111lelllllllllllllllndllll333331221 e S n O $.123br510a312345/O7l234123.456123456712345/01O~3410~31Urr3 @ L1 Counselor 25 6 27 28 29 30 31 32 33 \U. (Eff.-Uoneff. continued) Raters .4 3.4 31/14 3.4 33.4.4 3333338 936.4“. 35538 9.4.0. 33.4 37334.4 304.4. 333 21211112111111111133132211le2311.1111oJllllllqull 11111211111111111133121112213311111121111131111 ..111111—11112oalllllqignfillllllZlBQJfi/N2112121122132111 0 S n 0 W6 7.00 .12 3.0... 51 2 3a. rJ/O 7.8 9m. .12 31 2 3.4. 51 2 39...... 5/0 1 2 3.4 rJ/O 7.0m .12 3h. ..)/0 7 Q ‘9‘ a. n0 OJ Counselo 3’ 3 % W 9 40 (Eff . -::onefr . continued ) _ Raters 3333333333333333333a)3.4 3377.8 333333994” 58 99.4.4 38 9/0 lllllllllllllllllllllle23311111133112333121332 llllllllllllllllllllllll22211111133111333111232 llllllllllllllllllllllll323llllll33222233211332 30118 0 _...l_ 0.“ 31'». 5/0 1 2 0).“ 5/0 700 1 9H QJDr 5/0 71 2 3h”. ”..)/0 l 2 3.4 51 2 3.4 5/0 1 n4 3h». .1“ 9H QJhw. r.) Res Coun selor 41 m 43 1.1+ #5 % W .8 Sum _.“(X‘U Raters (Eff.-Uoneff. continued) 5565909860908996996770n956399£ 1* 1. 1+ 1* 29~223h¥ 332 3.411“. 3332332334432 21332— w v 1121 333323333332 3322 2 3331 21 332_ n5222333221423233233222343221332— ..123u.5123brr),07123u4567l234512345/0 Resoonso .01‘ (‘ COun ue] 49 5O 51 52 53 / Sum2=8738 (1496)2=223301~ (J 25 x2=2990 1496 747,288 470 508 1056 888 1046 (Sum)2 (Ef1.-Ionef . continued) Analvsis of Variance Source of Variation df.’ Sum of S_cn1ares (SS) Variance (82) Counselor 312 82:23: - 2,238,016 = 529.1+O 529.110 = 1.70 Responses 3 939 312 Raters 2 297,288 - 242371916 = 4.23 4.23 = 2.12 313 939 2 Error 624 606.73 - (529.uo+4.23) = 73.10 23:10 = .12 5E5 Total 338 2990 - 2,232,016 = 606.73 939 (l) Enternal Consistency = 1.70 - .12 = l.§§ = .922 1.70 l 70 (2) Intorjpdre Reliabiligz;(between-rater variance removed) = 1.90 - 1.2 = .814 1.70 + (3~17?112) (3) InterjudggeReliabiligz'(between-rater variance left in) = 529040 ”(4023 + 73010) = .892 2 529.Eb #(h.23 + 73.10) APPENDIX H STATISTICAL GLOBAL EATER ANALYSIS 0.1? THE SCORE OF THE OBJECTIVITY STUDY BASED UPON HOYT'S COMTATION OF REIIAHIIITY BY ANALYSIS OF VARIANCE .ll .1! l I I.'llnll]4llllul|ltl ~207— AffO ‘ Elmeasasauinnsea ta 6 A BETSL“""TF""T'1§fif"’ Counselors 1 8 8 8 2h 2 o 1 o 1 3 10 10 9 29 h 10 10 10 30 5 1+ 6 4 ll» 6 8 10 7 25 7 1+ ' u u 12 8 10 10 10 30 9 1o 10 10 3o 10 1o 10 1o 30 11 10 10 10 3o 12 o o o o 13 10 10 10 30 1A 10 10 1o 30 15 5 5 3 13 16 10 10 9 29 17 6 8 5 19 18 10 10 10 3o 19 6 6 6 18 20 10 10 10 30 21 7 7 7 21 22 1o 10 10 3o 23 10 10 10 3o 24 10 10 1o 30 25 10 10 10 30 26 10 10 10 3o 27 10 10 10 3o 28 10 10 10 30 29 10 10 10 3o 30 10 10 10 3o 31 1o 10 10 30 32 o o o o 33 8 9 7 22 10 10 10 30 35 1o 10 10 3o 36 3 7 3 13 37 10 10 6 26 38 8 8 8 22 39 10 10 9 29 #0 9 9 1o 28 41 10 10 10 30 #2 1o 10 10 30 43 10 10 10 30 an 7 7 7 21 C "-""5 .. . . r7 ‘ ~ _ F4 " \ ',. “ ‘) ‘ *“ ”«O‘, .- . . .. ..— —J 5 cu ., - ‘- -.- “.-- M. .. , ‘0'.- . ‘1‘. I .o. 4 7v“. - Ccoe w. .. _A r .-,. . .3 ' . 1 . I 1 f1." ’ . l‘ 1‘ l‘ . r, , ’ v . . . -Am! Affect-Oomtive (cone 1:.) A B C SUM Counselors 45 10 10 10 3O 46 7 7 5 19 47 5 3 3 11 48 4 4 4 12 49 8 8 8 24 50 3 3 o 6 51 4 3 3 10 52 4 4 4 12 53 _i .1 .2 .1: Sum 413 422 394 1229 Sum2=32,789 122924, 510,441 (3:ng 3693 3812 3478 (2410.983 (Sum) 170569 178084 155235 503389 was of Vagance Source of Variation (if Sum S ares Va 32 Counselors 52 m - 1.20.44 = 1430 27.50 3 159 Raters 2 M II 1' 20'“ = 7 3050 53 159 Error 104 1483 - (1430 + 7) = ' ’46 .44 Total 158 10983 - 9500 = 1483 (1) Int_____e_____rnal cons§stengx= 0 - = 2 06 I284 $501,427.50 (2 ) Enter-figs Lew (between-rater variance removed) 22 -44 £206:- ,253 27.50!' +' T335!‘.'E47 2%"3'5 (3) Inter 11" dge reliafillitx (between-rater variance left in) = 1430 - (23: 46) = 140:32'05 :- 1430 + (7 + 4-65 u.‘ I ' .0 . _.., .— 7 7‘ - ‘ my..." ‘ . ,.. .05." . .- ..u 1 . ‘ K ‘ 1 ‘ . r n “' 3 4 ‘~ ' _.n—‘ . - .‘nId . . I . I \ '1‘ . x o 1‘ .-»~ ‘.I‘ ' -- ’ . _-209- Understanding-Nommderstanding Mansion Scores Raters A B c SUM Counselors 1 5 7 7 19 2 o 1 o 1 3 9 10 8 27 4 10 10 10 3o 5 4 6 4 14 6 8 8 5 21 7 4 6 6 16 8 7 8 8 23 9 7 10 10 27 10 10 10 10 3o 11 10 10 10 3o 12 o o o o 13 10 10 10 3o 14 10 10 10 3o 15 5 3 3 11 16 10 9 9 28 17 5 6 4 15 18 10 10 10 3o 19 6 6 6 18 20 10 10 10 3o 21 7 7 7 21 22 10 10 10 3o 23 10 10 10 3o 24 5 7 7 19 25 4 8 6 18 26 7 10 3 20 27 10 10 10 3o 28 1o 10 10 3o 29 10 10 10 3o 30 10 10 1o 30 31 10 10 10 3o 3% o o o o 9 9 7 25 34 10 ‘ 10 10 30 35 10 10 10 3o 36 3 3 3 9 37 10 10 6 26 38 7 7 7 21 39 10 9 9 28 4o 9 7 10 26 41 10 10 10 3o 42 10 10 10 3o 43 10 10 10 30 “1+ 5 5 5 15 . , ‘ , _ . _ 4 _, . _.. , O’QJ- -~'~v " - - cu- .- n2 . .- 1. 0v irfi' . .‘1. non- -. . ooll' . I l‘ , . ~, *‘wo - vcv l w a. ’ . ) l x \ .. 1. 1.1. .9001" ’~'\ fi -210- Understanding-Nonnnderstanding (con ° t ) A B C SUM Counselors 45 10 10 10 30 46 5 7 5 17 1+7 3 3 0 6 48 4 4 4 12 1P9 4 6 6 16 50 0 0 - 0 0 SI. 4 3 l 8 52 4 4 0 8 53 .5. .2 .3. .1}. 375‘ 394 359 1128 Snag-428752 11282=1. 272.384 (311-112 3 3193 3428 3075 {@9696 (Sum) 140625 155236 128881 424742 Analysis of Variance Source of Variation df Sum of Squares (SS) Variance ($2) Counselors 52 2823 - I 1222 3 3841582 30.40 159 Raters 2 424242 - 1.222,:384 = 12 6. 00 53 59 Error 104 1694 - (1582 + 12) =100 .96 Total 158 9696 - 8002 = 1694 (1) Intemal consistengz = 30.40 - .26 = 2.44 = I268 30. 30. (2) Etegjgdge reliability (between-rater variance removed) =040- 6 =22.44=3211 310.40 "+" 23; 15!.93) 32.32 (3) Interjudge reliability (between-rater variance left in) = 1582 -(12 + 100) = 6 = 01 ----2 1125-): L 1§2 + 112 4» IE) - ; V , a. , .. .5 ~ - 1 ‘ s4 , ‘ I , ' .1 - ' v o- . ‘ . ‘ l . _ .1. .. _ . . , ‘ .‘ .\ , . I .‘ . .1 . ' ) .1 i . 1 ’ . 1 Q . b - . '1 .0 - ' . | g I . . \ .\ - ' V V . 1 ' . ._ “ ‘ 1 .- - X L . '.' I " 1 7 - V - I . . 1 4 _ - . 1 - ‘ , . . ' 1 , u w ‘ .. 4 ~ I; . 7 . ~ ‘ - \ . - w 1‘. ‘ . \ \ . ‘ -m. I“ - fi . -- .“y, r» _...fi‘ '1..~ . _ ‘ l , I . . , . ‘ x . >- ' .- ‘\ I .1 ~ f‘ '\ ... '0 2 I ' ’ ‘I . ..- a ' -- 4 a- . \ - ~ . ‘ ‘ - . L - r‘ ' . ‘ b n‘ \ "xr ’ .' ‘ . ' .. 1 I — . 4 '; ‘ . ’ - |\‘( , .4.’ . ~' \ '- . v . I n e V‘ I“ . .. ‘r. ,) 2 v . » .V ..1 J ..a» . ”..a .. ~-.-‘--o tn-Q. . '- ~‘O—l.-.oum‘bb-O. . A . . 8 A. row _~‘ \- - - . . . .‘Q— -. 5-. .s' ‘ 1 ' v-u ' ,. , . ~ \ . . n ‘ . I . . . . l . . - .-, _ \ ‘-‘ ‘J\/Lcu' ,‘o ‘. . u ~4'V "1 .. " § 1. '4. " “ g. C v. .‘ J-’ p- w...” a . , a h/ I -7. ‘0‘- ‘V" I < 4 c- I I i .. l r , ' x “' .- .o . 1 \ ,» \ .~ I . 3 1 .. \ 7‘ “. . . . _ . ,,. . . ~ . , m'v 7" u.“0- . . . .. . . "a; .7." 4 1 4 ‘_ ' A' . . a \ . . ‘ \ ' y 7 ‘ _ . ' ~r“!‘- ’1‘ “a. o ‘ IV' fi . 0' a u . 7 ’. . .’ ' . g ._ r L i . .- c- » ' , 400' ~ . ---o' - -r’v'w08' an . ‘ cc...-. 1 . o ‘- c 0 ml” . - o—- -- - A -- o - 3 o . ‘ ‘ ‘ ’ ‘ f . W .7 -. \' ,.-. r ‘ - . u v o . , 1.. , - . .. . ‘ ’ ~ . a . ' - . 7 ' ’ 4 .. k ' ' V . g- .-u'. - wv,-. . . A 1“ '1.. - on. - \' . .pwo. - - 1' ~ . - o . l‘ I ‘ ~ ‘, . . ' s ' a I. ‘ ‘ ‘ <2 ‘ o i "' n .v 1 ‘ . I1» . ' . ‘ “‘ 7 - ‘ z a, ' " . \ » ' I I . ." .. 1 ‘ . >3 ‘. . sI—OD ‘1 I - - u 1’1 . ‘ J . ' r I . p a ' .. . L .. .- .' ’~ 7 . . . . ‘ \ f _. ' I 1 , A . . , r, . . . I i ‘ 1 ‘1, ‘ _ . .A. ' 1 ' I Q . 3 ‘ -...'.. , . n ..a , o ‘.“'1 [..a-*1“ , u .‘ 1 ..a J , .‘ ‘ ~ _, . “..)-:4. n . ' ‘ “ 1» Ar 1 ' ‘ ‘ I 1211; Mic - Nonsmcific W Scores Raters A B C SUM Omnumflors 1 10 10 8 28 2 3 l 3 7 3 9 10 8 27 4 10 10 10 30 5 6 6 4 16 6 8 8 8 24 7 8 6 6 20 8 10 10 10 3o 9 10 10 10 3O 10 ”IO 10 10 30 11 10 10 10 30 12 0 0 0 0 13 10 10 , 10 3O 14 10 10 10 30 15 5 7 3 15 16 10 10 9 29 17 6 8 5 19 18 10 10 10 30 19 6 6 6 18 20 10 10 10 30 21 10 . 10 10 3O 22 10 '10 lo ,8 23 10 10 10 30 24 10 10 10 30 25 10 10 10 30 26 10 10 7 27 27 10 10 10 30 28 10 10 10 30 29 10 10 10 3O 30 10 10 10 30 31 10 10 10 30 32 O O 0 0 33 9 9 9 27 34 10 10 10 30 35 10 10 10 30 36 10 3 3 16 37 10 10 8 28 38 8 8 8. 24 39 10 10 9 29 40 10 9 10 29 41 10 1D 10 30 42 10 10 10 30 43 10 10 10 30 44 10 10 7 27 -212- Specific - Nonspecific (con't) . A B C SUM Counselors 45 10 10 10 3o 46 8 8 8 24 47 5 3 3 11 48 6 6 6 18 49 10 8 8 26 50 4 6 o 10 5|. 4 4 3 11 52 4 4 0 8 53 .8 _z .3 _18 _ 447 437 402 1286 $11m2=34836 12862=1.653,796 (311ng 4133 4011 3592 (12:11:36. (Sum) 199809 190969 161604 552382 * Wafiance Source of Va a on df Sum of S res (SS) W2) Counselors 52 21816 - 11.653ifl6 = 1211 23.90 3 159 Raters 2 552£2 - 1,633,226 = 21 10.50 53 159 Error 104 1335 - (1211 + 21) = ' 103 .99 Total 158 11736 - 10401 = 1335 (1) m cons2steng = 3:22; - :22 252 90 (2) Interme reliabiligz ( between-rater variance removed) 32.0- =22.1=‘882‘ 23.90 + (3.1“.99) 25% (3) M32 Egabihtz (between-rater variance left in) =12].1-(21+103) =11132 = ‘860 2 1335 12 + + 103 aa'~" . 7. -‘ .— 3 ~ ,, .. . o l“ u -v'- ~VC‘ » .L I." o On... .. ‘I". n ,‘3 3;" ‘—' . ‘J w , -— _... ___——- —-A—“—_ —_—_—————_ *1“ ._ a. -213. Exploratory _‘Wonexploratory Dimension- Raters .A B C SUM Counselors 1 10 10 8 28 2 4 4 3 ll 3 10 9 9 28 4 10 10 10 30 5 10 8 6 24 6 8 8 8 24 7 6 6 6 18 8 8 10 10 28 9 10 10 7 27 10 10 10 9 29 ll 10 10 9 '29 12 7 7 7 21 13 10 10 10 30 14 10 10 10 30 15 7 7 7 21 16 10 10 9 29 17 6 8 6 20 18 10 10 8 28 19 10 10 10 30 20 10 10 10 30 21 110 10 10 30 22 10 10 10 30 23 10 10 10 3O 5 24 10 '10 10 30 25 10 10 10 30 26 10 10 10 30 27 10 10 10 30 28 10 10 10 30 29 10 10 10 30 30 10 10 10 30 31 10 10 10 30 32 7 7 5 19 33 10 9 9 28 34 10 10 10 30 35 - 10 10 10 30 36 10 7 3 20 37 10 10 6 26 33 7 7 7 21 39 10 10 9 29 40 9 9 10 28 41 10 10 10 30 «42 10 10 10 30 43 10 10 10 30 44 10 10 7 27 . n o I I \ I\I\.. 6214- Ezmloratory - Nonexploratory (con't) A B C SUE Counselors 45 10 10 10 3O 46 5 8 8 21 47 5 3 0 8 48 10 10 10 30 49 10 10 8 28 50 10 9 l 20 51 6 4 1 11 52 6 6* 0 12 53 .2 .8. .9. .22 474 424 1379 Sum2=37613 13792-=1, 901, 641 971ng 4515 4402 3804 2x7: 12721 (Sum) 231361 224676 179776 635813 _f; 32237313 of variance Source of 2 Variation df Sum.o S uares SS) variance (S ) Counselors 52 32613 - 1,201,641 = 578 11.1 3 139 Raters 2 63§§13 - 1,201,64 = 36 18.0 53 159 Error 104 761 - (578 + 36) = 147 1.4 Total 158 12721 - 11960 = 761 (1) Internal consistenqx= ll 1 -11 4>= 2. .2: , 823 11. (2) Eter Edge rehabilitz (between-rater variance removed) = 11 l - 1 Llr 2,2 = (3) 11.12% reliability; (between rater variance left in) = 578 - (36 + 142) = 486 = ,632 2 7 578 + (33 + 1575 ' g or 1 u ., .'-v . ‘. ‘ ' V " ~ . .\r‘v I.'\,_'_».’ ‘K .. u A 3 .. - . a ‘| . u l \ C I . . - “-. ..-- won. - .4 . - .1 u- .. w a ‘ ‘ D ‘ u... o .0. . I."\\"“ -- o ‘ u u A . . -.. . A w -\ . 1 4 . new 1..» ”\AQ-» .10...- '4 '3. n..- ~. - . x ‘ 4 ‘ . . ' :4 ' 4 u 'C a 4 v - ~"‘C ~OI . u . . . . ‘ .. I... , . . . . \ «v‘ . - . a A? ' ~' " ha 7 a . . \IW" ‘ Q‘ ..oa--‘ on - . 1 B _. .. ..,. .. 4 . \' . ~~~ . I - ' . .. ..\- ”.- .25.“ Effective - Noneffectize Dimension Scores 3* 7A B C SUM Counselors 1 12 11 12 35 2 20 18 21 59 3 12 11 13 36 4 6 5 5 16 5 1.1 10 13 31" 6 13 11 11 35 7 11 11 11 33 8 8 7 6 21 9 8 6 6 20 10 11 10 9 30 11 9 7 8 24 12 10 10 10 30 13 6 4 4 14 14 8 7 8 23 15 10 8 10 28 16 9 9 ll 29 17 18 15 18 51 18 6 5 6 17 19 12 10 10 32 20 14 12 10 36 21 6 5 5 16 '22 4 4 4 12 23 10 10 10 30 24 6 5 6 17 2 5 9 7 7 23 26 5 5 4 14 '27 7 7 7 21 28 5 4 5 14 29 6 6 6 18 30 7 7 8 22 31 7 8 7 22 32 12 11 12 35 33 12 10 14 36 34 5 6 6 17 35 12 10 10 32 36 6 7 7 20 37 5 7 10 22 38 13 11 9 33 39 '11 9 10 30 40 11 9 7 27 41 6 6 6 18 42 8 8 8 24 43 7 7 7 21 -44 11 9 12 32 .~-..,. JO~O~ D‘ ~.,_-- I. O.‘ -216- Effective - Noneffective (can't) w—v (1) Internal consistencz= 48. 0 - 1.1 a 46. 9 48.0 48. O A B c SUI-i Counselors 45 5 5 5 15 . 45 13 10 11 34 4? 10 10 10 30 48 12 9 11 32 49 11 8 11 3o 50 19 20 23 62 51 18 19 19 56 52 14 13 17 44 53 12 .13. 11 22 3qu2 520 471 509 1500 am2=49114 15002=2, 2505000 (Sum 3 5856 4837 .5817 211-16510 ‘ (Sum) 270400 221841 259081 751322 __ . s of Va e 11 2 1 Source of 2 . Variation df Sum of Sguares $88) Variance {S ) Counselors 52 92114 - 2i250300 = 2220 48.0 3 59 Raters 2 22322 - 2,250,000 + 25 12.5 53 159 Error 104 2359 - (2220 + 25) = 114 1.1 Total 158 16510 .. 14151 = 2359 ‘ =a2.7'.Z (2) lutergg’ dge reliability (between-rater variance removed) = 48.0 - 1.1 = 363.2 = 48.0 + (34511.1) 50.2 (3) Inter E dge rehabilitx (between-rater variance left in) = 2220 - (23 + 114) 2 £3 2220 + (25 + 1147" 2359 = 20': = I211 ,,.‘ 4.1!- .1 .- .. v'va 4 v ,\ '- . . _ ‘ r‘ .L"\ . . | 1.. .‘ . Q . '1 I '- . ‘J . , i! l A "“« - .» . . , 7 K H» . . . -, I . - N,'.)4" 3". APPENDIX I STATISTICAL "t"-TEST ANALYSIS COI-‘iPAKING THE A-V SCALE SCORES OF THE OBJECTIVITY STUDY'S EIGHT PH.D. CANIIIDATE‘S WITH THOSE OF THE FORTY—F‘IVE 11.1. CANDIDATE -217- “1‘7“ Affect-Cognitive Dimension COGNITIVE CATEGORY Squared Deviation Cognitive Scores Deviation From Kean h.A. Candidates 017/Q/Q/5959999999993939799999999990/990/1990/5 mxn2n2no q,9.919191mwo1o1n691z391cJo11191919191919191919191Q5 919m8,9m 9M 11 11 11 /o 2 337730)3777737737373737777777777337737377. 16611114 . .2.111111112111116311911191111u111111111111111111116” .1111cm11 .11 oununonunwoonwnununununununuzjnuéunugunuornununonunvnonononvnununonunuqxnonunvo/ 1111 11111111 1111 11 11 11 1111111+111111111111 1111 11 11 l234567890fl23456789012345678901185934567390 l 1111111122222222223 333333 34 (Aff.-Ceg. continued) Cognitive Deviation Squared Scores From Mean Deviation R.A. Candidates 41 10 1.7 2.9 42 10 1.7 2.9 ZP3 10 1.7 2.9 44 7 -1.3 1.? Sum 373 329.8 N 45 ii 8.3 Variance (821) = 329.8 = 7.3 ‘45- Cognitive Deviation Squared Scores From Kean Deviation Ph.D. Candidates 46 7 2.2 4.8 [+7 3 -l.8 3.2 48 4 — .8 .6 49 8 3.2 10.2 50 3 -1.8 3.2 53‘ 3 '108 3.2 5- 4 - o8 '6 53 _j_ .2 .4 am 37 26.2 N 8 I; 4 8 Variance (82 2)= 26.2 = 6.3 1 .75.. t = ii . 352 = 8.3 - 4.8 =3.5 3.5 l 2 2 (213 + 6 .3 (153.1) I 1 s 1 + s 2 (11 + 1?) 2 2—-«- 2 degrees of freedom (df) = Il + 112- =45 + 8 - 2 = 51 significance at .05 level = 2.00 significance at .01 level = 2.66 nmrx Undorstqndingefionunderstanding»Dimension ITOEULIDEJETMIDIITG CATEG RY Nenunderstanding Deviation Squared Scores From Kean Deviation E.A. Candidates 1 7 - .9 .8 2 O -709 6201-“ 3 9 1.1 1.2 4 10 2.1 4.4 5 4 “309 1502 6 8 .1 .0 7 6 -1.9 3.6 8 8 .1 .0 9 10 2.1 4.4 10 10 2.1 4.4 11 10 2.1 4.4 12 O -7.9 62.4 13 10 2.1 4.4 14 10 2.1 4.4 15 3 .4.9 24.0 16 9 1.1 1.2 17 5 -2.9 8.4 8 10 2.1 4.4 19 6 ”109 3.6 20 10 2.1 4.4 21 7 - .9 .8 22 10 2.1 4.4 23 10 2.1 4.4 24 7 - 09 .8 25 6 -109 306 26 7 " 09 08 27 10 2.1 4.4 28 10 2.1 4.4 29 10 2.1 4.4 30 10 2.1 4.4 31 10 2.1 4.4 32 0 -7.9 62.4 33 9 1.1 1.2 34 10 2.1 4.4 3 10 2.1 4.4 36 3 -4.9 24.0 3 10 2.1 4.4 38 7 "' 09 .8 39 9 1.1 1.2 40 9 1.1 1.2 K.A. Candidates 41 42 1.13 14; [15 Sum II 351 Nonunderstanding Scores 10 10 10 5 .19.. 354 45 7.9 Scores Ph.D. Candidates 46 5 47 3 48 4 49 6 50 O 51 3 52 4 53 .2. Sum 30 N 8 3'62 3.8 =219-3L8 8.6 + °.O 2 df = 51 signiiicance at .05 level = si3nificance at .01 level - 2. 66 (Underst.-Eonunderst. continued) Deviation Squared From I can Deviation 2.1 4.4 2.1 4.4 2.1 4.4 -2.9 8.4 2.1 _4.4 384.8 Variance (S2 ) = 84.8 = 8.6 l ‘45" " Nonunderstanding Deviation Squared From Hean Deviation 1.2 1.4 - 08 .6 .2 .4 2.2 4.8 -3.8 14.4 - .8 .6 .2 .4 1.2 1.4 24.0 variance (822) = 24.0 = 3.0 8 %l 14.1 5.1:) f? "i c-I‘. on s be cij.‘ i c Dir-ten si on o" . '7 LJ?)CC:1..L 110.18 P 2171 F1 C CAT IIIORY Deviation Squared IIonspecifi 0 Scores Deviation From I-lean Ii . A . Candidat es O.”208:4000000000008080000000000000ZOOM/Wouoo 0000 000 21 2/00 62222:422B26262222222222224 20412 22 ..6h...466644446446464644444444 NA44/W44hW/nwhn/hwhwu. L6 1..21111£113l2121111111111113 LL5L..LL 0390686000000050606000000000000090030800 1 1 1111 11 1 1 111111111111 11 1 11 1234567890u23456 890E234567no90fl/23hfi5673JO 1 111111112 222222223 331333334 J; Nonspecific Scores H.A. Candidates 13. 10 MZ 10 43 10 1%! 10 1+5 .12.. Sm 389 N 45 Nonspecific Scores Ph.D. Candidates as 3 13,7 3’ #8 o be a 50 3 51 1+ 52 4 53 _6 Sum 02 N 8 332 5.3 t - 8.6 - 5,3 = 1.0 + KL 1 / 24 (115+ anJ df = 51 significance at .05 level = 2.00 sicnifieanee at .01 level = 2.06 k.) 2.2... > VA .79 (Lpec.-Uonspec. continued) Deviation From Kean Variance (821) = Deviation From Mean ILI I H NN MN 0 o o o o o o VWWWVVWV Variance (829) = .213. = 11.8 .23 Squared Deviation 7.0 °16.2 = 55 Squared Deviation 29.6 = 3.7 .2?_. Exploratory-Uonexploratory Dimension EOKEKPLORATORY CATEGORY Nonexploratory Deviation Squared Scores From Mean Deviation N.A. Candidates 1 10 .8 .64 2 4 -5.2 27.04 3 9 - .2 .04 4 10 .8 . 64 5 8 -1.2 1.44 6 8 -1.2 1.44 7 6 -3.2 10.24 8 10 .8 . 64 9 10 .8 .64 10 10 .8 .64 11 10 .8 .64 12 7 -2.2 4.84 13 10 .8 .64 14 10 .8 .64 15 7 -2.2 4.84 16 10 .8 . 64 17 6 -3.2 10.24 18 10 .8 .64 19 10 .8 .64 2o 10 .8 .64 21 10 .8 .64 22 10 .8 .64 23 10 .8 .64 2 10 .8 .64 25 10 .8 .64 26 10 .8 .64 27 10 .8 .64 28 10 .8 .64 29 10 .8 .64 30 10 .8 .64 31 10 .8 .64 32 7 -2.2 4.84 33 9 "‘ .2 0014’ 3’25 10 08 '64 3 10 .8 .64 36 7 -202 14'0811’ 37 10 .8 .64 3: 7 -2.2 4.84 39 10 .8 .64 40 9 - .2 . O4 (3xpl.-fionexpl. continued) Nonexploratory Deviation Squared M.A. Candidates Scores From Kean Deviation 41 10 .8 .64 42 10 .8 .64 43 10 .8 .64 44 10 .8 .64» [1'5 10 .8 064 Sum 414 95.20 N 45 1' . 2 _ _ k1 9.2 Variance (S l) — 22.20 — 2.1 5 Nonexploratory Deviation Squared Scores From Mean Deviation Ph.D. Candidates 46 8 1.0 1.0 47 3 -4.0 16.0 48 10 3.0 9.0 49 10 3.0 9.0 50 7 0 0.0 51 4 -3.0 9.0 52 6 -100 1.0 53 8 1.0 1.0 Sum 56 46.0 N 8 - 2 7.0 Variance (S ) = 46.0 = 5.8 1‘2 2 73-70 t = 2,2 - 7.0 = 2.2 = 2.2 = 8.5 .26 2.1 + 5.8 _l___ + 1 6.8 2 45 8 df = 51. significance at .05 level = 2.00 sigrifieance at .01 level = 2.66 ON “‘x / 1‘.." f o ctive-Z‘! onef f ective Dirtensi on Squared Deviation Deviation From Iloan Scores . Candi dates -. 1~' 24.88 828.24.08.6.80.80 820280008208002002088888 O O O O 3O.4464.6.40.47.67.4 07003047704374770900704744 1 4 41 112 916121 22 2211 126 88220028002228228ALBA/~82?»22228222220U22287x)22 1B226263 24N52947343255 2152553M 3572222 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7933282253.“.30351320230053703002052033833 1211212111 3112...212111111111111311121111 1234567890 L2 .456 7890fl234567890£o 3hfi56 7390 111111112 2222222230 330 333334 (Eff.-Honeff. continued) Deviation Squared Scores From fiean Deviation E.A. Candidates 41 10 -5.2 27.0 42 10 -5.2 27.0 43 10 -5.2 27.0 45 10 -5.2 22.0 Sum 682 1511.2 N 45 _. . 2 15.2 variance (3 2) = 15%;.2 = 33.6 X2 5 Deviation Squared Scores From Kean Deviation Ph.D. Candidates 45 18 —6.9 47.6 47 25 .1 0.0 48 22 ‘209 8014‘ 49 22 -2.9 8.4 50 33 8.1 65.6 51 29 4.1 16.8 52 28 3.1 9.6 53 .22.. -2.9 ..a-.4. Sum 199 164.8 11 8 X 24.9 Variance (52 ) = l .8 = 20.6 1 1 _.87—' t =2 24.9 — 15.2 = 9_.Z = Z2 = 4.2 Significance at .05 level = 2. 00 significance at .01 level - 2. 66 .n.‘ MP STDI X J DEPEIEDELZ’T -"t" STATISTICAL AI‘JALYSIS COERITATIOI‘S COI-ZPARII-IG THE IIOIGSUPm‘JIS-ED GRCIIP'S INITIAL AND TERI'II'JAL II-ETERVIEM SCORES Affect Categonz Eonsueervised Group (Client-Interrogation) AffC‘Ct SCOI'G AffBCt Score Difference Difference? Initial Interview Terminal Interview (3) (32) Counselor Enrollees F l 1 0 0 s 2 3 1 1 H O 0 0 0 I 2 3 l l J _;L_ 2 l 1 5“” 5’ Q ZD = 3 209-: 3 h 5 “ 2 2 x 1.2 1.8 (2D) = (3) = 9 dependent --t" =3 2 '5 = 3 = .6 = .6 = 2 n 1: .5 :EEC' n.) D .1..— UZDZ -4213} / .. 112m .. 1) 25255 degrees of freedom = N-l = 5-1 = 4 significance at .05 level = 2.78 Understanding Cateeogz Nonsupervised Group (ClientAInterrogation) Understanding Score Understanding Score Initial Interview Terminal Interview Counselor Enrollees F 8 8 G 2 0 H l O I 4 2 J 6 2 Sum 21 12 N 5 2 4.2 2.4 It can be seen that there was a decrease rather than an increase from the initial to the terminal ratinfis. Honsupervised Group (Client-Interrogation) Specific Score Specific Score Difference Difference Initial Interview Terninal Interview (D) (Dz) Counselor Enrollees F 2 4 2 4 G O l l 1 H 0 O 0 0 I l l 0 0 J _l__ .22.. ._1_. .1. Sun 4 8 4 6 h 5 a. 2 A .8 1.6 (4) = 16 dependent “t” = 4 = .8 = 8 = 2.2 5 .47 6-16 25 (11:4 significance at .05 level = 2.78 Ehploratory Categogx Nonsupervised Group (Client-Interrogation Exploratory Score Exploratory Initial Interview Terminal Interview Counselor Enrollees F 9 6 G O 2 H 0 0 I 5 2 J _.L .9. Sum 17 10 u 5 I 3.4 2.0 It can be seen that there was a decrease rather than an increase from the initial to the terminal ratings. Effective Category Konsupervised Group (Client-Interrogation) Effective Score Effective Score Initial Interview Terminal Interview Counselor Enrollees F 36 37 G 22 24 H 21 21 I 31 25 J 26 22 Sun 136 129 N 5 SE 27.2 25.8 It can be seen that there was a decrease rather than an increase from the initial to the terminal ratings. APPENDIX K DEPEIIDEITT - "t" STATISTICAL ANALYSIS COL-3UT1‘1TIONS COIJJMYIIEG THE SUPERVISED GROUP'S IHITIAL I‘d-TD TEm-II‘IAL INTERVIEJ SCORES fl no 1- _, 113.001. C9. teg’ll‘jf S1eervised Group (Counselor Interrogation) Affect Score Affect Score Difference Difference: Initial Interview Terminal Interview ' (D) (D3) Counselor Enrollees A 7 8 l 1 B O 4 4 16 C 4 10 6 36 D l 4 3 9 E l 10 2 81 Sun 13 36 5,1) = 23 £132: 143 L 5 2 2 A 2.6 7.2 GED) = (23) = 529 dependent ”t” = {_D = '5 = 23 z 4 6 = 4.6 = _; I} 2 -‘-- l. .1- S D .5 71.85 35 1:102 - (infi ‘/ 59.422 .. 522 11201 .. 1) 25 degrees of freedom = N - l = 5 1 = 4 significance at .05 level = 2.7 significance at .01 level = 4.6 5 O UnderstandingVCategogx Supervised Group (Counselor Interrogation) Understanding Score Understanding Score Difference Difference: Initial Interview Terminal Interview (D) (02) Counselor Enrollees A , 14 17 3 9 B 3 15 12 144 C 5 l4 9 81 D 7 5 2 4 3 _2_ 12 14 196 Sux 34 70 40 434 11 5 .: 6.8 14.0 (40)2 = 1600 dependent "t“ = .549 = 8.0 = 8.0 = 2.2 i 2.38 -* V5.70 51434) - 1600 25(4) (if : )3, significance at .05 level = 2.75 significance at .01 level = 4.60 Specific Category Supervised Group (Counselor Interrogation) Specific Score Specific Score Difference Initial Interview Teminal Interview (D) Counselor Enrolloes A 7 l7 7 3 1 l4 13 C 3 10 7 D 4 14 10 E 2 8 16 Sum 17 73 53 21 5 1: 3.4 14 6 (53)2 -_- dependent “t” = 53 = 10.6 = 10.6 = 6.1 .5 1.73 V 3.06 62 - 28 25 ‘ df = 4 significance at .05 level = 2 7 significance at .01 level = 4.6 Exploratory Cate 501.1 Supervised Group (Counselor Interrogation) Exploratory Score Exploratory Score Difference Initial Interview Terminal Interview (D) Counselor Enrollees A l5 l9 4 B l 16 15 C 6 l4 8 D 8 l4 6 E _§L_ 20 14 811121 36 83 47 11 5 x 7.2 16.6 (47)2 = dependent "t“ = 47 = 9.4 = 9.4 = 4.6 5 2.09 4.36 / 51522) - 2209 25(4) 01‘ = 4 significance at .05 level = 2.75 dignificance at .01 level = 4.60 Difference (I)2 ) N \)H l—’ O-P‘CN-‘w I FROG‘OQ Difference2 fiEfective Cotevonz Supervised Group (Counselor Interrogation) Effective Score Effective Score Difference Difference2 Initial Interview Initial Interview (0) (02) Counselor Enrollees A 44 54 10 100 3 25 49 24 276 C 29 52 . 23 52 D 33 42 9 81 E 26 6O 24 ll 56 Sum 157 257 100 2442 H 5 x 31 51 (100 )2 = 10, 000 dependent "t” = 100 = 20 = 20 = 4.2 __5 5.7 22.1 / 5(0002) .. 10,000 25(4) E E. {‘5 O (D - 93 (1' o O H H (D i II II (TN 0 O\ O »\.. Pun fl avg: ........ 3.5...”