AN EVALUATION OF THE OBJECTIVlTY OF AN
AUDIOMSUAL COUNSELOR SGALE WHICH COMPARES
THE BEHAVICIRS OF M. A. CANDIDATE COUNSELORS
WITH THE BEHAWORS 0F PH. D. CANDIDATE COUNSELORS

Thosis fot the Dogma of Ph. D.
MlCHiGAN STATE UNWERSITY
Gerald G. Griffin
196.6

 
  
 
   

LIB R A R:
Michigan 5-: ;‘: i
University

This is to certify that the

thesis entitled

AN EVALUATION OF THE OBJECTIVITY OF AN
AUDIO-VISUAL CWNSELOR SCALE WHICH
COMPARES THE BEHAVIORS OF M.A. CANDIDATE
C(IINSELORS WITH THE BEHAVIORS OF PH.D.

CANIEDATE CCII NSELORS
presented by

Gerald. G. Griffin

has been accepted towards fulfillment
of the requirements for

Doctoral degree in Philosophy

/ ’ ' I I I ' ‘
" J, . I/ , '1'" , ’ ’(‘é/{L |/ I
O ('1' "’ r ‘1, r I I k P '

Majoé profeséor

A33T£AC

3H SVALUATIOH OF THEL mMIfIf OF M
AJDIO-VISUAL COUI'S 3140.1 Soil... 'IIICEI CI) 33A £33
F13 BEHAJlC-BS Or i‘-'..A. CANDIDATE CCU 3110 L3
UfHTRBBJMWD$3m3HLD.C&DEMT3UL'CL$B

by Gerald G. Griffin

This 103 esrch examined three in H10 oth eses associated with the

A

‘

develop ent of an Audio—Visual Counselor Behavior Scale (A-. Sea 1:).

v Or 1
C.‘

:is A-; tca‘e was developed for the purpose of making available

an objective ins rur e‘.1t which would compare the response patterns

of gaster's degree candidates in Counseling an ad Guidance (L.A.
Candidates) with the response patterns of Doctoral de ;:ree candidates
in Couzsclin3 and Guidance (Ph.3. candidates). The A-Y Scale
consists of five forced choice, dichotor ous disens ions: the
Affect-Cornitive, Unders tanding-I onunderstane , Specific-
..ons1ocific, Sleoratoryl onexploratory, and Effective-Noneffective
ii: Lensions.

The research consisted of three separate studies. The Scale
Bdscrinination Study tested the hypothesis that the A-V Scale
would significantly discriminate the response patterns of Ph.s.
candidates from those of K.A. candidates on its five diaensions.

The Oojectiw itJ StudJ ested the hypofi mi that the A-V Scale
would obtain hi3h interjudge agreement when used for the ratin:
of counselor behaviors on audio-video tapes. The Sensitivity

Study tested the hypothesis that the A-V Scale is sensitive to

chan3e in the counselor behaviors meaw mred by its d'me mic

:3
a
9.)

a result of supervision which focuses on behaviors within these

Gerald G. Griffin
dimensions.

For the Scale Discrimination Study, three trained counselor
judges used the A-V Scale to independently rate 20 responses from
each of ten 30 minute audio-video taped counseling sessions.

Five of the counselors engaged in these interviews were K.A.
candidates. The other five counselors engaged in these interviews
were Ph.D. candidates. Each of the ten interviews was an initial
personal-social problem interview with a client.

The rating scores obtained by the two classes of counselors
on each of the five dimensions of the A-V Scale were statistically

compared using ‘ti-tests. There were significant differences
in the response scores of the two classes of counselors on all
five dimensions of the A-V Scale. The responses of the Ph.D.
candidates were more frequently rated affect, understanding,
specific, -xploratory, and effective than were the responses of
the L.A. candidates.

For the Objectivity Study, three trained counselor judies
used the A-V Scale to independently rate four minute seaments

from each of J: audio-video counseling tapes. Forty—five of these

tapes involved K.A. candidates and eight involved Ph.3. candidates.
Hoyt's analysis of variance method was applied to the

ating scores. Two types of interjudge reliability and internal

consistency coefficients were obtained. One was an index of

item objectivity and internal consistency, and the other was

an index of a global rater objectivity and internal consistency.

Item interjudge reliability coefficients of .89, .79, .76, .5},

_._‘

and .31, and global rater interjudge reliability coefficients

Gerald G. Griffin
of .95, .91, .39, .70, and .93, were computed for the Affect-
Cognitive, Understanding-Honun‘crstanding, Specific-Nonspecific,
Exploratory-Ionexploratory, and Effective-Noneffoctive A-V Scale
dimensions respectiv 1;. For these same scale dimensions, item
internal consistency coefficients of .96, .92, .91, .8l, and
.93 respectively, and global rater internal consistency coefficients
of .98, .979 oyb, .37, and .98 respectively were computed.

For the Sensitivity Study. ten HDEA Guidance and Counseling
Institute trainee volunteers were randomly assigned to two groups
of five; ne group comprised a supervised counselor-interrogation
group (Supervised Group), and the other group comprised a non-
supervised client-interrogation group (Nonsupervised Group).

Each member of both groups underwent six 30 minute video taped
counseling sessions with a college or high school student.

The Supervised Group focused on affective, understanding, specific,
and exploratory responses during interrogaticn 1112' the Eonsupervised
Group had no particular focus during interrogation.

The middle twenty responses of the first (initial) and
sixth (terminal) counseling sessions for each of the ten counselors
conprising the two groups were rated by three trained counselor
judges. These initial and terminal ratings within each of the
Supervised and Nonsupervised Groups were statistically conpared
using dependent *t"-tests. The results of these tests indicate
that‘for the Supervised Group there was a significant increase
in the rating scores, from the initial to the terminal ratings,
on the affect, understanding, specific, exploratory, and effective

dimension categories of the Scale's five dimensions. For the

Gerald G. Griffin
honsupermised Group there was no significant score increase, from
the initial to the terminal ratings, on these five Scale dimension

categories.

A3 EUALUATIOI CF THE OBJECTIVITY or a:
{DMD-VISUAL coma-max sous; macs comma
Tia BSHAVIORS OF n.A. CANDIDATE COURSELORS
41TH '33 BEHAVIORS OF PH.D. CAEDIDATE counssLoas
By

.. .. e.
Gerald G? Griffin

A THESIS

Submitted to
Kichigan State University
in partial fulfillment of the requirements
for the degree of

DOCTOR OF PHILOSOPHY

Department of Counselins, Personnel Services

L)

'and Educational Psychology

1966

' 1’j'O "' "I .fi'1 'fia'flfi
-1C;L.. 14.0 U..._J.L.l3

I .Jisa to e mress my sincere gratitude and appreciation to
my ajor Professor and Program Chain'nn, Dr. ITornan Kagan, not
only for providing me the opportunity to pursue this research
endeavor but also for t1 e very hel>ful support and consul ation
he provided towards its fulfilln nt. In addition, I an especial..y
grateful to Dr. Richard Rank, Dr. Bill Kell, and Dr. Bernard Cora-.1
for the scientific refinement they added to the research through
their critical evaluations and individual contributions.

Lg; sincere thanks 13 also extended to those many counselors

V

who helped in the develogI1ent of the A-J Scale, 1-: no participated
in the studies of t‘m'Ls research, and who devoted their time and
skills as judies. Particularly do I wis h to acknowledge nr

ti ude to Robert ..Ta.rd, Ifarbara Fuller, John Eiartsell, Joe
l-Lesanno, 31.13. 130300, Patricia Parisian, Robert Campbell, and
.511 Goluoel 1". I would like to 1::ake note, too, of my appreciation
to L3 -111 ..enc 1301. for t11e1::a11y secretarial services she rendered
in assoei . v.1. 1.. '..ith this resea1 ch.

Ffinaffly, to 1:1 wife, Patricia, 11.51311 to c::' rse s :i.y gratitu-ie
for her pat'.1_en e111; c139 )eration and uncierstaz'a‘infi; 1111:1511: the 11311118.

F 'L‘ " 1‘ "i .11 r ‘ .‘1 "‘ N ' . " '-.A —" e. 'l‘ ‘t-' '(' J‘ ‘ , 6 f‘
3.‘ 1.35. s roses.-c-1, 11.11.- uo extend rys . ..patu m. ». 1,0 111.1 for

. 1 . -.,:,. .m 11“,. . v .1...-:_-.-,_ ,,J.. ,1 v 1 “,1 . . . 0,. A ..-,,-. .2 . .
ENQUJ._.-;-Q £4110 1.3.1.1011.) {53-41.15, .11.,31513, 13.... 8119 U... tendent 3.11 ~.2.l.L_L~.«..‘.'A
I 'l“ 'L" -. u ' ‘-.. \r- .1 r‘v. ,o . . J" — J' of .

. -. x 4.-.»... .-l..:rc~::..1.-_on document.

r‘ v

. ..I,‘ V.
bfififl in»

I.

”or

III.

1V.

'1 ‘ Ty 1‘. \ ".1 ;!
A—La-‘l 1 .bk" _ -‘.J4 .

O O

T ' " -: > '3
JOEL--5 .L‘LVLJUF. 13: 'I' .
.3 JILC‘J‘JLCGL . .

H . T - . 7‘ 5‘. 1 .— .fi "1 ow 'fif‘
. . V \ .‘
n¢- AA‘*‘)iN) k)“ AtbhiLlJLl.)

w1'- '

rv“"-'|
L uJA,

V

.-W-‘,‘

1. .1)

1.:,.4;1".;".'.f COLICLULSICLIES AID IliFLICATIOITS
3

Bl BLIOILXE’I LI . . .

H.

1*

k'AGL‘.’

1’)\

\11
(7‘.

7l

C‘ ’3
13¢
’-
\,l
/ _’

LIST OF TABLLS

ABLE PA"

h
L?)

j.u. Comparison of the loan (K) Scores TGtVOOH the
Ph.D. and E.A. Candidates on Each of the
Five Dimensions of the A-V Scale . . . . 7?

§.~. Ito; Interjudce Reliability and Internal
Consistency Coefficients Computed from
the Scores of the Scale Diserininetien Study . 73

Kn
O
\J)
O

Iteu Interjudge ieliability and Item Internal
Consistency Coefficients Computed from the
Scores of the Objectivity Study . . . . 7%

r’ ' 7- »\ \ I fin'L ‘ ' '3/ ~ . ‘ . g . vv J-

3.4. gleshl “beer IAtGTJUL’Q ?elianitity and Internal
A - ‘av _ -v _ .— .a. “-0. ‘. 3 .0- ‘ “ .I- V, ‘n " J_ ‘r‘
~a-r/ .~.'.A.I(.ILII- I... :~."_'. ' . ‘.",(".:.‘J-(J Cf) ~k J .A. .. - \j’. . Due

,_ --, _ __-.-,'_2,, -¢-.'f.
'.,) “f x I, c..'. but/Jr, 4”...“ ..IV.£\x/ o o o o (J

j. . Conscrison of the Xeen (K) Scores Between the

En.2. and 3.3. Candidates on the Cognitive,
Honunderstandint, Fenspecific, and Ionexploratory

C togories and Effective-Noneffective

Di ension, of the A-V Scale . . . . . 7C

\J'x
Q\

. Leon (1) Scores of the Initial Interview Rating
for both the Konsupervised and Supervised

' ‘

:vogos on the Five Dimensions of the A-V Scale 77

5.7. Coxpgiidon of the Kean Scores of the Initial
and Terminal Interview Ratings for the
{ensupervised (Client-Interrogation) Group
gn tie Five Dimensions of the A-V Scale . . 8

5.3. Ce Sinison of Loan Scores of the Initial
and Terminal Interview Ratings for the
Supervised (Counselor-Interrogation) Group

on the Five Dimensions of the A-V Scale . /;

7v

Ll.

- O
;

5%.

LIST OF APPEYDICES

A Conpilation of Counr elor Video Veroal

Sehaviors as Observed on Audio-Video Tapes , .
Categonisction of Counselor Audio-Video

Zehavi01 3 According; to Behavior Categories , ,

Definitions and Illustrations of the A-V Scale's
Dimension Categories . . , , ,

Q 0
Clarifi ation of Frequently Occurring Rating
Problems and Questions , , . . . . .

oc‘ulstical t -Iest Inalfsis Cenparing the
”-7 Scale Dimension Scores of the Scale
Discrir' ztion Study’s Five Ph.D. Candidate'
cith Those of the Five K.A. Candidates . , .

Statistical Itex Analysis of the Scores of the
Scale Discrixi.nation Study Based Upon Hoyt's
Get utation of Reliability by Anal sis of

Valixnice , . . . . . . . . .

StatiLt ieal Ite Analysis of the Scores of the
0330( mti tjf Study 3ased Upon hoyt's Compute tion
Reliability oy Anal;si.s of Variance . . .

0

“Leo ;lieal Cloaal Rater Analysis of the Scores
of the Oh jectivity Study 3ased.Upon Hoyt's
Ce Jutation of Reliaoility by Analysis of

\VCLiO Ice . o O O O O O O 0 0

Statistical "tt-Tcst Analysis Comparing the
A-T Scale Scores of the Objectivity'Study's
Ifiqfit Ph.D. Candidates with Those of the

Forty-Five I.A. Candidates . . . . . .
sependent-“t” Statistical Analysis Computations

Conparing the Konsupervised Group's Initial

and Eerjinal Interview Scores . . . . .

3e enlont- t” Statistical Analysis Computatiors

C; paring the Supervised Group's Initial
and Terminal Interview Scores . . . . .

Ill

‘4

r3

"Nf

”’K

}_J

(_ \

K11

or]
r, L k;

[\ff‘

l‘v

CR; 12R

THE PROELBf

This research is primarily an objectivity evaluation of an
Audio-Visual Counselor Behavior Scale (A-V Scale) developed to
compare two classes of counselors at Hichigan State University.
«ssentially, Iaster”s degree candidates in Counseling and
Guidance (I.A. candidates) constitute one class of counselors
while Doctoral degree candidates in Counseling and Guidance
{;h.D. candidates) constitute the other class of counselors.
The A-V Scale consists of five forced-choice, dichotomous
dimensions which measure the extent to which each of these two
classes is characterized by affective, understanding, specific,
exploratory, and effective responses in initial personal-social
problen interviews.

In the remainder of this chapter the problem will be more
explicitly defined under the headings of Specific Problems,
definition of Terms, Delinitations, Basic Assumptions, vaotheses,

Heed for Study, and O~sanization of the Stu.y.

L.)

o
A“

Speci:_c Problems

 

The specific problems of this research are:

«L

1. Can the two defined classes of counselors be signi-

licantly differentiated on any of the dimensions of

the A—V Scale?

N
0

Ion objective is the A-V Scale for measuring these

th;ensions?

-1-

Ix)

-

how sensitive is the A-V Scale for indicating change
in counselor behaviors on its five dimensions as a
result of supemision which focuses on behaviors

within these dimensions?

Definition p_f_ Terms

 

The terms validity, objectivity, and reliability are defined

in the following manner:

 

l. Validity. The extent to which the A-V Scale does the
job it is employed to do. That is, the property of the
scale to differentiate the video-verbal behaviors of
ILA. candidates from the video-verbal behaviors of
Ph.D. candidates.

2. ijectivity. Interjudge reliability based upon Hoyt's
analysis of variance. This is the average reliability
of ratings based upon the average of interjudge correla-
tions.

3. Reliability. Reliability based upon the internal analysis
of the average ratings of judges employing Hoyt's analysis
of variance method.

gelimitations

 

The major dehmdtations of this study are:

l.

The video taped counseling; sessions employed in the
individual studies halting up this research, with the
exception of one objectivity study, were based upon
relatively small numbers of counselors (I'J'ith diverse

theoretical orientations) at I-‘Iichigan State University.

-3-

This was necessary because of the special problems,
xpense, and time involved in video taping, and

because of the limited number of counselors available

for the type of video taping required.

The counselor verbal behaviors studied were laboratory

behaviors in which both the counselors and clients were

aware that their counseling interviews were being video

taped.

Basic Assumptions

The basic assumptions upon which this research is based are:

l.

Counselor responses which are affective or cognitive,
understanding or nonunderstanding, specific or nonspecific,
exploratory or nonexploratory, and effective or non-
effective, are available on audio-video tapes for judges
to observe and measure by a rating scale.

Both H.A. and Ph.D. candidates in Counseling and
Guidance have a pattern and manner of counseling
associated with these behavior dimensions which are
consistent throughout their counseling interviews when
such interviews pertain to a client's personal-social
concerns.

Those A-V Scale dimension behaviors which are character-
istic of the counseling practice of Ph.D. candidates,

but are not characteristic of the counseling practice

of N.A candidates, are assumed to be learned by the Ph.D

candidates as a result of their training.

gmmtheses
The general expectations which this research examined are:
First_§ypgthesis. The frequency of affective, understanding,
specific, exploratory, and effective responses in initial
personal-social problem counseling interviews, as measured.
by the A-V Scale, will be significantly greater for Ph.D.
candidates in Counseling and Guidance than for M.A. candidates
beginning their practicum experiences in Counseling and
Guidance.
Second Hyppthesis. Based upon Hoyt's analysis of variance
method, the A-V Scale will have adequate objectivity for
differentiating between Ph.D. and M.A. candidate responses
on each of its five dimensions.
Third.Hypothesis. Counselor trainees who receive professional
supervision.which focuses on.exandning affective, understanding,
specific, and exploratory responses will receive higher
frequency ratings for these categories on the A-V Scale
than will counselor trainees who do not receive this pro-

fessional supervision.

Most instruments which have been developed to measure counselor
performance have had poor objectivity, and therefore, poor reliability.
In recent years the demands of an ever expanding, service oriented
society have created a tremendous press for counselors. These
demands have not only necessitated an increased need for adequate

counselor education progr-ms, but they have necessitated the need

for adequate counselor education programs WhiCh could accelerate
counselor training. However, the educational and counseling
professions have been seriously hampered in their ability to
experiment with various counselor accelerated programs because
they have lacked an objective counselor behavior measuring

instrument .

Organization 9_f_ thg M

A review of relevant rating measurement methodology, related
research, and common counseling elements are presented in the
following chapter. The development of the A-V Scale is presented
in Chapter III. The methodology for testing the three hypotheses
set forth in the present chapter is presented in Chapter IV.
The results of this research are presented in Chapter V. The

summary, conclum‘ons, and implications are presented in Chapter VI.

CHAPTER.II

FflfViEu' Ol‘ LITZU1L I):

The review of literature presented here is divided into
four areas, each with a summary: relevant general principles
in the construction and use of rating scales; methodological
considerations in the development of rating scales; previous
counselor behavior measurement; and common elements of
counseling pra tice that are important in their evaluation.
The terms *therapist”a nd couns elor are used interchange-

ably.

Construction and Use of Pating Scales

A joneral review of the value and use of rating scales,

and the important considerations associated with their develop-

5
:3
(+-
\O
:1:

‘e presented.in this section.
”...3 ratizg instrument generally consists of two com-

ponents; a descm ption of the behaviors to be considered and a
set of altCLnDtiVO res we ses fr on which the rater is to choose
for each person rated‘ (Ielmstadter, l96fi). A rating scale is
one o: the ;:est peHJul r techniques for describing hrs a1

characteris‘ics (Helrzsta dter, l96~z Tor3erson, 1953). A rating

scale, states el‘r

‘l' 5 "2 "\“ . " t" ‘ ”It “. “" "" ’- -!" ~ '. -'-.'
l\’ r us-.. 1.» v1“ .00 (‘L' (".2 . ‘0 i.'_ '.,.:

for dte Cl':ilfr-‘.‘.{; the w y in which others react to an individual

in question, and for this situation they are valid by definition.
Helmstadtcr (l964, p. 199) remarks, ”...if sufficient care and
trouble are taken, rating scales can be used to obtain highly
satisfactory quantitative descriptions of many behaviors which
cannot be studied by other measuring devices.”
Test construction is a never ending process (Tyler, 1964).
Rating scale development is a special form of this process.
In the initial stages of test construction there are no specific
rules or techniques rhich can tell a person exactly what to
measure in a particular situation. The researcher ultimately
decides what the measure will include based.upon his experience
and judgmental skill (Helmstadter, 196M). However, helmstadter
presents the following steps which a test constructor must
follow if his instrument is to be based upon sound construction.
1. The first and most essential step is a careful
delimitation and breakdown of the area(s) or trait(s)
involved.
2. Once the areas of knowledge and/or particular traits
which need to be included have thus been laid out,
he next stei is to list the behaviors which the
examinee should exhibit if he possesses the knowledge
or characteristics under consideration.
3. When the test items have been written, the next major
step is that of trying them out on an analysis group
of s1bjeets who are representative of the population

for which the total test is being prepared.

.’\

4. Kort, the items are to be organized into a resonably
administered format and an appropriate system for
scoring the test is to be develeped.

5. The last logical step in develOpnent is hat of naking
a final eval ation of the test, as revised and re-

organized, on a new analysis group.

{elmstaater (1964) points out that even with the best
possible psychological measuring instruments, errors will be
made. He mentions that what test constructors should do is,
"...to learn what sorts of error can occur; to develop procedures
for estimating this error; and, insofar as possible, to utilize
methods of test construction and administration which minimize
these errors."

When rating scales are used, errors arise from rater char-
acteristics and from the behavior selected for measurement.

Rater biases and constant errors of leniency (tendency to be
overly generous in descriptions), halo effect (rating a given
individual in the same way on all behaviors), and error of
central tendency (the avoidance of using extreme positions on

a rating scale) are errors which arise from rater characteristics
(Helmstadter, 1964).

One of the most recent innovations in the rating scale area
has been the development of a forced choice technique which has
been specifically designed to overcome these difficulties
(Highland and Berkshire, 1951). In a forced choice rating the

judge is required to consider not just one attribute, but several

characteristics all at one t‘me. he general format requires
that only one or a few of the behaviors listed in each item
be selected as applicable.

To reduce error ariSing from the behavior selected, Helmstadter

‘ 7

ting behaviors which can be dependably rated. He

23
E
C)
(i-
(.1
L“)
a
O
0

highest for those characteristics which are Specific entities.

Tie major logical objection to the rating scale approach is that
ratings are often based on only broad impressions about the
person being ‘ated. Therefore, for rating scales to be useful to
researchers ana workers in applied fields, numerical ratings

must in some way relate to relevant observable behaviors. Recent
studies (Gobhavt a.d Sort, 1958; Guilford, 1959; and Thorton, 19b3
s of specifically defined
characteristics obtained on carefully developed scales can have

S

validity, while judgments of broadly defined traits are often
susject to irrelevant influences.

Eel stadter (196%) offers four major techniques for the
improvement of ratings. First, rate only relatively Specific

‘ .

aly

characteristics which "...are readily discernible and have a hig
out meaning for a wide variety of judges.” Second, be
careful in the preparation of the rating scale itself. Select
behavior labels which "...are short enough to fit on the scale yet
sufficiently'unanbiguous to result in general uniformity of
meaning for all potential raters. Incluce a concise description

and a few examples along with each behavior label.” Decide

upon the best number of steps to be used. Five and eleven see;

-10-

..L 1

to be the o;mt1'al nunbor of steps above and below which lb does
not pay to proceed (Chazpney and 2;.rshall, 1939). After the
individual behavior scales have been developed, organis- then
into a complete racin in tvuzent. Third, “...use careful selection
and training of the judges who will do the rating.‘ Forth,

(_)

...have the judges make their ratings independent}r and then

{I

pool tzeir res ts.-

SUJBUaIfir

ratings cale consists of a description of the be
to be considered and a set of alternative resporses from Uzich
the rater is to choose for each person rate . It is the most
direct n asur currently available for determining the way in which
othe‘s react. Common errors of the rating approach, such as ater
biases, errors of leniency, halo effect, nd errors of central
tendency, are gelmer mil avoided by the forced choice technique.

If sufficient care and trouble are tagen in the devel lopment and

use of rating ins' oruxents t1 er can DO used to obtain 1 "h 3
satisfactory quantitativcc descriptions of 21an; behaviors which

_L ‘0 1 ‘

cannot be squaiec oy oth r measuring devices. Ratings are
improved by selection of traits for neasurewent which relate to

is and relevant observable behaviors; areful preJarat- n

(A

cale itself, including appropriate selection
and aese"i;tio; of behavior levels and proper organization of
individual behavior scales into a complete rating instrument;
careful selection and training of the judges who will do the

rating; and the peeling of the results of indeflaen eit judge

 

ratings.
Ratin" Scales and fiethodological Considerations
Ac lesion of the pertinent methodological considerations

presented in this section. These

51
U)
0
{a
i..J
O
P.
L)

related to rat 5.-
considerations refer o TOllflJl_le, objectivity, and validity,

each of Inlic:1 is assoc M1 ited with a different kin: of error, and

each of which represents a separate criterion for evaluating a

test ('eltsstesel, 196U).

.1th aza; test, relio.bi]-ity of the techniole end vali <15.t ' of

the results are two highly si'nificant rechodelogical considerations
(Tyler, 196%; Helnstadtcr, 1964; Ruck, 1963; Lindquest, 19Cl).

In the case of rating scales, which are us ed by a "roup of judges
to rate st..crs, a reliability consideration of ingortance is
that of interjudge agrcexcnt (Helxstidter, 1969; Ebel, 1951;
Hoyt, 1931). The central nethodelogical question for
scale, then, nay be stated: how use “11 (valid) and objective

J.

(degree of interjudge reliabi t7) is the scale for the surgese

Following is a general overview of reliability, objectivity,

and v: lidity considerations and their relation to measure: .ent

Selig 1b: 411g;
/n a o a

End 1(190)) en has 303 that a measuring dcv1ce cannot be

valid unless it is first of a1. reliable. The 1959 Glossarv

of 11,0.s1rez'10nt Tens," published by the Califorrfia Test .; tree-.13.,

/\/’)

defines reliability (in common rith others such as inch, 192)
and Tyler, 19C:) as the le ree to w a an individual would

obsain the sane score on test if the test were read irjstered

E.)

4‘, ‘n~- "1 ‘1. .A .1 (“'11. . (n v -:‘ ., .s 4- m -* .
to one sL .e incavjduaaq 1:113 “301 ssL.rj oi “easure .enL, Lerls

distinj"shtm three tvpes of reliability coefficients:

j ‘1’ n15: o 4‘ _fi (‘"‘ ”’1‘. H. o J; .-
. - A‘,‘ r -. | -,

4‘. COL-.. ..L.--C_L .ILL .. L) .JL.-_/:1..L.l. .1"
—v._~ -— W

 

A correlation between test and retest with sene period
of time intervening. The test-retest situation nay
be :ith tno ferns of the sane test.

 

 

 

 

2. Coefficient of eluivalence
u correlation between scores from two ferns javen ct
essentially the same time.

3. Coefigicient of internal cons is er 9:
A ‘easure ba.sed on internal ancljsi-sofd1ta obtained
on a sfnfile trial efa . tes Tore nro: inent of these

are the sWnlit- elf method and the Kuder-lichardson and

..l n

hogs analjsi s o; variance netheds.

fieli1yilit7, in each of these cases, is defined as “...the ratio Of
true score variance to observed score variance and is an index

of tle agount of 'ariable error (chance errors varyinfi from

° .1. .1.

person to person and fro: one tixc to tle next) in a sesc’

There are four classical ways of estimating rclia m; tr (best-
retest, parallel for , split—half, and finder-Richardson), all of
which may be expressed 11 n for-: analorous to the quotient of the
observed variance ninus the error variance to the o
variaace. P1ch of these procedures is basically different in

that '...it defines chat is meant by error in a slirhtly fferent

wag. cl-ss (ter (l‘/’) points out that the proce(lure for

'1 ’3

deter ining the test-retest correlation ...defines as variable
error any tine to tine fluctuations;'f the procedure for deter inins
parallel for; reliability ‘...defines as variable error both time
to time and fern to for; fluctuations;“ the procedure for deter-
Lining split-half reliabilit" "...defines variable error only as
fluctuations from one half of the test to the ne::..;'2 the procedure
for cote: inizg finder-Richardson reliability (Iuder and Richardson,
1937) '...defines variable error in terns of consistency of per-
forance fro:.; iter. to item."
Loyt (thl) has presented a procedure for estimating reliability
which defines a variable error in a sonewhat different way from
these other aggroaches, although the results are identical with
Kuder-Richardson reliability. Hoyt's final basic way of defining
what is n ant by variable error involves a slightly different
breakdown of observed score variance from what is usually presented.
According to Hoyt‘s formulation, ”...variation in the performance of
an individual fret iter to item is not considered error. Rather,
it is a real (non-error) difference, an intraindividual difference,
and one which should not be involved in the estination of reliability.’
Total variation observed, then, is conceived to be made up of three
coxoonents: 'true interindividual differences, intraindividual
differences, and error interindividual differences. According to
Hoyt a better definition of reliability is expressed by the quotient

h

of the true interindividual variance (error variance subtracted 1Wa~

*-

the obs rved variance minus the intraindividual variance) to the
aserved variance Linus the intraindivioaal variance. Using an

P O 1
l

1‘ "/V. (s ' ' Q’fi . A A \ r ‘1 ~ 7 V (w 4| 1 . —. ~-
anal;s1s o- «_riance procedure, then, lovt's 1e11aoil tj can

be estimated by

I‘__;] = 1-13 individuals - IZS residuals

175 individuals

I-IS individuals = mean square of deviations from the
il1rklvidual's means

l-LS residuals 2‘ z-nean square of the deviations left
over after individual and item variation
have been removed.

The reliability of rating scales is affected by the major
considerations for f ornulating an observational measurement. The
number of scale categories and the effect of the length of the
judged lin‘telial appear to influence reliability (Helmstadter, 1961+;
Bendig, 199%; Bendig and Sprague, 199+; Symonds, 1922). Reliability
of rating; scales is enhanced by clear and. ummbiguous directions
(Helrlstadter, 1961}; Benc‘ig, 1953), preciseness of rating criteria
(Helmstadter, 1960'; Rose, 191W), uniform scoring procedures
(I-Izmlin, 1951:»), use of adequate samples of observed behavior
(Helmstadter, 196+; Hardin, 199+), verbal definitions of scale
categories and illustrations of behaviors which fit various scale
categories (Bendig, 1953; Albee and Hamlin, 1950), and the
peeling of independent jucre ratings (Iielmstadter, 19’»; Gordon,
1924). The factor which influences scale reliability the most,
he: ever , is interju r“e agreement (IIelmstadter, 1961+; Ruch, 1963;

Benrlwi; ll55; 3011618 and Spraguea 1954)“

‘L.
.-bj]

O] ajectiw

lfielgastadtor (1964) defines objectivity,r as the extent to which

-13..

pers0111 errors have been avoided in rating. Thus, a measure "hick
is relatively free from pers sonal errors is referred to as' objective.
Per:;;<l errers refer to inca‘sirte‘eie‘ anon; several dif-ereni
s . I—Zel‘ :stadte‘.‘

_ 0 ~ 0 9 .1. o
.. I ..~ ,. --r1--V-- ‘1': '7' . - . (.._. ..7 .fi.” .. __. , ‘
“'.' “~).‘n »-‘(_.":/.a..--¢ U110 ”ti. .«2 J-.".\.3 Vfiv].. .KlAlU 01 -...-.h h'] V _-xl'

L.aues. ...an, c.c 'RL‘VLS observin enaccl' the sane rets onses
, 1.. . L) J

p.)

likely to record.different scores, because they can see the
perfor: nee on1v from their own po W1 ion or bias.’ Since such
fluctuation in scores is directly attributable to the judge doing

.1.

the rating, they are called perso onal errors. Interjudge reliability
is the indeztthich reflects t2 0 e:ctent to niich a measure is free
from this type of error. O't's arélysis of variance method

(Hoyt, 19Ql) is one which provides such an index. It provides a
Vailit; COeSficient which indicates the degree of consistenc;
mhieh is obtained when several different judges rate the sage test
or in ividusl.

Eel ”"lCuC" (195$) indicates that objectivit by is a special

form of rel-ia:ility. ”...Becausc tne fersonal biases of jud~es majinf

oservations fluctuate from time to time and from subject to
subject, the resulting personal errors represent a particular
type of variable error. Because of this, no new index other than

the index of reliability is nor:.ally needed to 9 es s an instm ieit

L.)
U)

in this respect.” How 'ever, Helmstadter mtr cs that when judges
are used to rate res :on ses, personal errors becone of su111,1_1t

magnitude to require special attention and therefore deserve a

special label. This is tne index of objectivity.

(D
}_J
C.)
O
; x“
O
L.)

lac: of objectivity is directly reflected as

states ...i11 order to be reliable, a measuring instruzzent

rust be ob jective, so that tt.o or 2‘.ore individuals ca n score

the subject'31es_)onses and obtain tile same result.” helmstadter
(196#) states, "...a separate index of ebjectivit;...:ay prove
valuable to a test constructor by suggesting a specific cause

of unreliability in an iml trum ent under development Other
things being equal, the greater the Objectivity of an instrument,
the greater its reliability will be (Ed mstadter, 196%).

Each (lTGS) suggests that the testrvny'to insure objec-
tivity in a measuring instrument is not to include iten1s which
Iaist be scored on the basis of subjective judglent The fewer
the r sponses which must be scored by subjective judgment, the
greater the probability two or more persons can score the same
responses and rat the same results. Rueh points out, though,
that in the measurement of human behavior peeple must act as
the measuring instrument, making it nearly impossible to obtain
objective measurement unizfluenced by personal perception.
However, one r 1y asm mne that through the use of professional
judges, those who are somewhat expert at what they are measuring,
this subjectivity'nav be minimized. This assumption is supp01¢ed
by Cgtt.ll'l(nj{fifml statement in 1903 that, "...there is,
however, no other criterion for a sun's work than the estimation
in which it is held by t1 1050 most cos'lpetent to judge."

Shapiro and Seerh a~t 19%?) recognise that perfect accuracy
is neithel necessary nor‘usua lly possible in observational
judgments. They make reco P“dfiblOlu the t are believed to

facil:taee reliability, c.;., techniques to el'vinate ”apparent

1n ervie"c or sins.“ These types of techniques to insure similarity

between the 1; tervi0U01s or judges have seen au)>lied an analysed

in various sec;Lal settings by neldnan, Hyman, and Hart (1951).
I.

The use of an interview or pretest in the formulation of a ratinr

scale has preven to be especially valuable by Rose (1947).

epersonal charactel of the judges appear to be
significant in observational analysis. Bendig aW1d Wrague (195/!)

found thct the individual rater reliability increased rith hi 3her
levels of education, and the va riance depended grea tly'upon the
length of the scale and the experience of the rater. In a
fellewaup study'using the sane rating scale, Bendig (1955) again
found that the educational level of the judm affected the
reliability. Here bias measures were found in the undergraduate
"lower education" judges.

Helzm tr.d oer (196%) declares that the careful selection and

J‘J.

training 01 ulOSO jud:es the will do the rating is one of the

most effective ways of insuring adequate results from the rating
scale approach to measurement. He indicates that the firsat rule

or who I’ll]. have the op aortunity to ma}: e the
necessary observations of the subject in pertinent s.tuatiens.

is important tO‘try to secure raters who are willing to
maze grud3 ents free of distortion. One of the first tasks in

tlm etraining of raters to use a scale apprepriately, notes
Helmstadtcr, is to convince they of the value of honest and accurate
ratings no It also helps to point out the types of errors which
raters are li. :cly to r:.l:e azd provide some .L!‘O tiezn 3for avoiding

.L‘
U; LC‘.‘ .

Ya lilit “c

Validity is the most important consideration in the con-
struction and use of all t31)es of tests (Kelmstadter, 196%;
Tyler, 1963). Rob abi ity is morelv a hes ste the er d of

validity (Ruch, 1963). A measuring device can not be valid unless

Lalrstadte" 105%) states that ‘...reasure:ent of bellavioral
Claractew astics is indirect. Zecause of this, one cannot be
con>letely co: tain that a test measures the precise characteristic
for which it U’C sdesigned. Thus, an error can be made in using
tests which is the sane for every individual to which the test is
aoplicl and the sane every'tine it is used. Such errors are
called constant errors. In test measurement, the problem of these
const nt or; ors is the problem of validity.” Validity is an
indication of the extent to which a test is not influenced Ly
constant err
Cronbach (lBQG) and others point out that validity may have
a varie OtJ o; xeanings. The the basic approaches to the deter-
mination of validity are logical analysisa 1nd empiric al analysis
( Zlossary oi Hcasurenent Terns”, 1959).
Content validity and it01 structure are t3Q)es of va.ljdit"
1v logical analy81s. Content validity refers to
"...he" '.e Ll the content of the test samples the subject matter

,

or situation .Dout wllic.1 conclusions are to be drawn.” This r'y

be determined the opinions of a jury of experts. Content
validity includes face validity, which invol a simple

evaluation of co: Itent adequacy; logical validity, in which it is

'1 1"\

U)
Q)

carefully deter.- ined I'FlIO'UlOI‘ the test behavior i . representa-
tive sample of the behavior in question; and factorial validity,
in Mulch the e:~;tent to which certain factors contlibute to total
test scores are determined by exmfining the many relationships
among; test-taking behaviors (IIelmstadter, 1961;). Item structure
includes corroborative evidence from item analysis supporting

the other elmracteristics of the test and item coztmposition
emphasizing conceptual clarity in the expression of items.

Data shoxing the relationship between test and some later
(eliterion) behavior is called empirical validity. Concurrent
validity and. predictive validity are types of validity determined
by empirical analysis (”Glossary of Ifeasurement Terms”, 1959).
Concurrent validity refers to how well test scores match measures
of contemporary criterion performance. Predictive validity
relates to how well predictions made from the test are confimed
by the data collected at a later time.

A type of validity in which both logical and empirical
methods any be used in its determination is construct validity,
validity in which the theory underlying a test is validated.
Cronbach (1957) defines construct validity as an analysis of the
meanil'z; of test scores in toms of psychological constructs.

Rank (196+) points out that this definition requires that in
order to establish construct validity, the test-m. {er must
1;;easuro a constluct which is tied into an appropriate theoretical
framework. EIeL‘zstadter (1962:.) states that construct validity

"...is evidence which seems to establish the test as a measure

of so: :0 11;.rpothcsized trait which may affect performance in a

variety of ways in rsny different kinds of situations." He

adds that ”...construct validity involves such diverse evidences as
group differences, perfornanee change as a result of experimental
:-anipula tion of certain variables, axltitrait-nultinethod matrices
of correlations, internal consistency in.ornation, and studies

of the tes -ta:ing proces.." Keehl and TacCorquodale (55) suggest
that no test has approached the ideal level of construct validity,
indicating that this is a difficult type of validity to establish
for a t st.

lelnstadtcr (193 ) feels that it is; rather naive to ask whether
a test is valid. Cronbach and heehl (1955) point out that 1
test is never rea’ly ”validated." Rather, a principle for
making certain kinds of inferences about persons who obtain
"ivcn test scores is verified or refuted. Information taat is
gathered to determinee z'netly fnat kind of infereimce can be
drawn from test scores is referred to as validity evidence.

The question should not be t"Is the test valid?” but “Is the
test valid for such and such?“

Ruch 1963) states that "...the validity of a measuring
instrument is the extent to which it measures that Vhich it was
designed to measure." Lindquest (l 961) points out that validity
y for a pal ticular purpose with the essential
question being how well does a tee t do the job it is employed
to do. Lindquest (1961, p. 622) states:

The validity of any test is its vali dity
as an indicator of individual differences
i

n some pa articular function among the
nezoel s of some specified group.

Ty]. er (193:) foo ls in e:ca:ining validity that the proper question
1: is "...;just what is it that this test does measure?"
rather that the old validity question of "...to what extent
does this test measure what it purports to measure?” He feels
that the content of the test 1;:ust be analyzed and many correlations
em: ined with different criteria in various groups before the
answer can be 131mm. 'l‘yler's view is that it is not necessary
for a psycholoList to have an exact definition of what he hopes
his test will measure at the beginning of his test developl 1ent.
Tyler (1963, p. 29) cor-azents:

...i.f he has a general idea about the character-

istic and its relationship to either theoretical

concepts or practical situations, his precision

in defining it will increase as he tries out his

test in series of separate research studies.

Bayroff, Haggerty, and Rundquist (195%) point out that
the use of rating scales and judges for the assessment of
segments of behavior carries with it certain problems and
‘ ~awbac:s. One of the most important problems seems to be the
obtaining of :i1a::'1.11ruz:1 rating validity. These writers note that
ratings are subject both to rater contamination (agreement of
ratings due to the .LO ct that the same person makes two evaluations)
and technique contamination (extent to which validity coefficients
are al 'tifici all inflated because of reseltzblance of technique).
They found that the most effective way of increasilg the rating
validity was to:
l. Avera e the ratings :tade by the raters.

2. have raters 1dcnt1f‘y their rating- s.

). .iavc the crucial. rating; completed early in the series.

In 19.11 1113:; stated that the rating f hmsmn character
was practicable "...if the rating is done under conditions

as rigorous as “no follm-Zing:”

1. If each fiml rating given a person is the average

oz- three independent ratings, each one made on a
scale as objectified as the man-to-nan comparison
t;po scale.

2. If the scales on which the ratings are made are
c01 :parable and equivalent, having been made in
conferences under the instruction of one skilled
in rating scale work.

9. If the ‘hrec raters are so thorourr 11y acquainted with
the person rated that they are conpetent to rate.

If validation evidence identifies a number of scale items
which sigfificantly distinguish between the criterion groups
obtained in one sampling, the test constructor may wish to try
them out on a second sample. This is known as cross-validation.
If the sale results occur on the second sample, the researcher
can have a fair degree of confidence that his finding will hold
up in subsequent studies (Helnstadter, 1961+). Cross-validation
is accomplished by trying out a previously developed and refined
scale on a new group that resembles closely the one for which

the scale will ultimately be used.

Slur-'13 r”;

Ijothodological pl‘OlDlOZ‘lS are the most prevelant questions
regarding; rati: 3. Two of the most highly significant mothedolofical
considerations concern the reliability of the rating technique and
the validity of the rating; results. The reliability of a rating

‘techzfirplc refers to its consistency as a measuring instrument.

Tirls indicates its relative freedom f ron variable error.
Personal errors, witch are due to the impressions and biases of
the raters, are a special class of variable error. They tel-{e
on sufficient zragnitude when rating scales are used. A special
index of reliability, known as objectivity, reflects the extent
to which personal errors have been avoided in ratings. Lack of
objectivity is one of the most common causes of unreliability.
The greater the objectivity of an instrument, the greater its
reliability will be.

Using pretests in the forrmlation of rating scales, developing
precise rating criteria, providing clear and unambiguous directions
and category definitions, providing examples of scale categories,
following uniform scoring procedures, providing adequate samples
of behavior to be rated, and pooling of independent judge ratings
are all techniques which reduce variable error and thus increase
scale reliability. Personal errors are reduced, and thus objectivity.
and reliability increased, when careful attention is given to
the selection and training of the judges who are to do the rating.
The educational level and knowledge of the judges seems to be of
particular importance. ‘

The validity of test results is the most important consideration
in the construction and use of a rating instrument. The problem
of constant error is the problem of validity. Information that is
gathered to determine exactly what kind of inferences can be made
from the test scores is referred to as validity evidence. The
major validity criterion of any instrument is how wen does the

instrument do the job it was e:‘::ployed to do. The validity of

rating results is increased, according to expert opinion supported
by some experimental studies, by averaging judge ratings, having
judges identify their ratings, and having the crucial ratings

completed early in the series.

Counselor Behavior heasurement

 

Some of the most relevant research efforts over the past
years to objectively measure various aspects of counselor behavior
are reviewed.in this section. A brief historical overview of
counselor measurement efforts preeeeds a discussion of the signi-

ficant counselor behavior scales related to the present research.

Histogz

Laswell (1929) reported the need for more adequate recording
of the data of the analytic hour if such interviews were to be
used for research. In subsequent articles (1935, 1936) he
reported the usefulness that electrically recorded verbatim.data
could have for this purpose. Symonds (1939) suggested to
psychologists that phonographic recordings might provide the
objective data for the study of the interview process. Rogers
(1942) described the ways in which such data might be used for
training and research, and Covner, one of his students, wrote
a series of articles (1942, 194h) spelling out the details
of the rationale and technique of phonographic recording.
Subsequent research under the general supervision of Carl Rogers
validated the assumptions regarding the use that could be made

of verbatim data.

-25.

It was recognized by early investigators that in order
to conform to the rigors of the more objective approach
characteristic of psychology, it was deemed necessary to
translate the raw data of the interview into a form that could
be amenable to statistical analyses. Lasswell (1938) devised a
system of categorizing what the patient said during the psycho—
analytic intervieW'by designing codifications for the patient's
positive and negative attitudes towards himself, towards others,
and towards the therapist. A system was developed by Porter (1943)
for the codification of what the therapist said. Porter's system
of categories was refined by Snyder (1945), who added categories
regarding the statements made by the patient and, in part,
studied their interaction.

Other attempts to objectively describe the activity of
the therapist followed the early efforts of Lasswell, Porter,
and Snyder. In 1948 Keet codified the statements of the therapist
to differentiate between expressive and interpretive activity.
Finesinger (1948) classified therapeutic activity into three
levels of activity: low, moderate, and marked, while Dittman
(1952) classified therapist activity into questions reflecting
feeling, clarifying feelings already expressed, clarifying
feelings somewhat less surface, and depth interpretations.
Similarly, Carnes and Robinson (1955) categorized therapeutic
activity into clarifications, tentative analyses, interpretations,
and urgings. In 1953, Collier focused on a continuum of activity
level, while Harway, et. a1. (1955) and Rausch, et. a1. (1956)

focused on a dimensional analysis of interpretive activity.

D

Signiiieant Counselor Behavior Scales

The earliest validated counselor-rating scales of any

Significance are Rogers' Relationship Inventory Scale (1962)

and Andersons' Interview Rating Scale (1962). Rogers' scale

3

is one whi a per .zits the client to rate the counselor. It
eonsi sts of three sectioxm which focus on the type of relation-

Q

ship provifcd by the counselor. The first section consists of
72 items thich describe possible positive and negative feelings
cr‘ at'i'vles trhich the counselor may have toward the clie mt;
e.g., f‘-at tines he feels contempt for re." The client is asked
to rate on a four point scale, ranging from, ”I strongly feel
this is true, to ”I strongly 1801 that it is not true," the
extent to whiehh e feels each of these 72 items app ply'to 's
can melor as he relates to the client. The second section of
Rogers' scale consists of two questions asking the client to
rate, along a seven point scale ranging from extremely dis-
ied toe ompletelys ti. sfied, how he feels about the
relationsnio wifi1the cou11selor. The third section consists of

eight sentence completions concerning the counselor and his

counseling; 0.3., ”the counseling seemed very .*

 

Although the Rogers' Relationship Inventory Scale has demonstrated
some utility for counselor eva aluation by clients, its major
linitation is ‘nat it is tied too closely to the theoretical
concepts of Rogers' wn counseling viewpoint. It focuses upon
ulnatuu, Positive Regard, Congruence, and Unconditionali y.

Thus, it contains no questions related to effective or ineffective

use of counselor questions, confrontations, interpretations, and

”7
-.u -

other counselor behaviors not emphasized as important by Rogers.
Also, the scale depends entirely upon the subjective impression

of the client doing the rating rather than upon objective measure-
ment of counselor behaviors.

Andersens' Interview Rating Scale is an attempt to provide
an operational definition of ideal rapport, defined in terms of
effective communication in counseling interviews. It consists of
50 items that are based upon a consensus among a select group
of counselors and clients concerning the meaning of rapport.
Eighteen of the items refer to client behaviors and attitudes;
e.g., “the client distrusts the counselor," while the remainder
refer to counselor behavior; e.g., "the counselor is very
patient." The items are described in terms of specific behaviors
land attitudes, each of which are rated on a five point scale
ranging from ”never existing" to "always existing." The 50
items are evenly divided.between items representing high and low
rapport. The Pearson correlation between the ratings of the
counselors and those of the clients, based upon median scores,
was 0.98.

Anderson and Anderson have demonstrated that a core of
agreement among clients and counselors can be reached concerning
the nature of ideal rapport, using the Anderson Interview
Rating Scale. The effectiveness of the scale as a measure of
rapport in counseling has also been supported by other inde-
pendent investigators. However, Andersons' scale limits itself
to an effectiveness of communication concept and does not

draw broadly upon other counselor behaviors such as aspects

~23-

ased upon vie al perception and confrontation. Like Rogers'
scale it relies more Upon svb' t'v 'mpr ssiens than uoon
u..—, J. .u.. _... 9.3801633“.A ea.) 1
objective observations.

. 0 .0 L.

as part of a atienal Institute of Lental Health project,
Rioeh, et. al. (l, 3), developed a scale for rating counselors in
interviews which used outside profes smouals rather than clients
as judges. Zioch's scale was uuch simpler than either the
Rogers or .inderson scales. It consisted of tno Slall sectiozs:
one for rating the counselor on his general counseling competency
during the interview and one for rating the counselor on auto—
criticisn. The interview section consisted of nine global
rating s 0.3., “re spect for the patient, " and the section on
autocriticisn.consisted of five global ratings; e.s. ”shows

° V

at.areness of major neakness or weaknesses,“ both of which were

marked on a five pei.t scale from poor throusl excellent.

no aiocn Scale was applied to eight 40 year old mothers

who had one year of training in the project's two year Hental
health Pregran of individual and group procedures designed to

train them to do psychotherapy'under lixited conditions. Four
professional judgesr rated thes e counselors on each gloeal din ension
using the five point seal». is a group, the eight counselors
receiveda :mcan score above 3.0 on each dixension. In addition,
the clients of these counselors were cliuiw Ml eval1ated to be

fUnctioning better. These results vere considered positive.

a?" _ a1" ‘1‘) '1‘ .
.4

" ' ' "V I“ J. 'L . V (- J' As ‘v N r.
.03 enough caese tentative results Laj suggest so 0 utilitr

I

'r‘ - " a - \ 1 r‘ .'L ‘ -a .x..-' D C 1-,, u 5,. -
lo: bfl‘ AHOC. seaae, lbs use seeus lihited ior caree reasons.

H. .J- .‘ . '0 1 o ,'.L' 1 - ‘ '0‘ ‘H J. h ."fi
sires, no rellasilicf stur: es 1:e1e reporcel for the sc.13

Second, the scale is too general and too global, and thus does not
permit rating of specific counselor verbal behaviors. And third,
like the other scales reviewed, it relies upon subjective
impressions.

During the 1963-6@ academic year the Interpersonal Process
Recall (IPR) Project members develOped the Counseling Process
Effectiveness Scale to measure counselor trainee effectiveness.
IPR is a new technique in stimulated recall methodology designed
to secure the maximum effect from the replay of video tapes in
counseling through emphasis on the visual element during recall.
This technique was developed during the course of investigation
into the dynamics of empathy at Michigan State University's
College of Education by Kagan, Krathwehl, and Miller (1963).

The Counseling Process Effectiveness Scale was developed by IPR
project m-nbers after an unsuccessful search for a suitable

scale. It consisted of 33 behavioral and.feeling tone items,

a global rating, and a question asking the judges to write adjectives
or phrases which.described the trainee.

The 33 behavior items, e.g., "the counselor paces the interview
inappropriately," were each rated on a four point scale with
one end of the four point continuum corresponding to ineffective
and the other end corresponding to effective. A few of the items
were taken and modified from Andersons' Interview Rating Scale.
The global rating was adapted from Rioch's scale and.was scored
on a representation of the normal curve with baseline divided
into eight equal segments. The judges were asked to rate each

counselor in comparison with counselors of comparable experience

mm
- WU-

in order to establish an absolute rather than a relative judgment
so the general quality of the counselor as a product could be
evaluated. On the third part of the IPR project's scale the
judges were asked to indicate the main factors, in order of ,
importance, vhich led to their particular rating cu‘eaeh counselor.
As part of the development and progress of the IPR project,
Ward (1965) used the IPR technique to compare the effects of
three different methods of education on counselor trainee
effectiveness as measured by the project's scale. Fifty-four
hichigan State University Counseling and Guidance trainees were
divided equally into three counselor education treatment groups:
a video recall, an audio recall, and a control group (standard
counselor training). Ezch of the groups had three thirty minute
counseling sessions with high school students. The video recall
group had each of its counseling sessions video taped and replayed for
recall purposes with an interrogator. The audio recall group had
each of its counseling sessions audio taped and replayed for recall
purposes with an interrogator. The control group had its counseling
sessions audio taped but received no stimulated recall sessions with
an interrogator. At the end of this training each of the 54
counselors held a counseling session with the same coached client
(a college student actr-ss). This session was audio-video taped
and used as the criterion for evaluating treatment effects.
Hard employed three professional counselor educators s
judges to rate each of the 5% counselor's criterion tapes for

counselor trainee effectiveness using the Counseling Process

"v
-/-_-

Effectiveness Scale. For each criterion tape they rated two
separate four minute segments. One four minute segment was
rated early in the interview session (the first to the fifth
minutes) anc. the other was rated later in the interview session
(the ninth to the thirteenth r-damtes). Using this scale Ward
found no sigfificant differences between the three groups in
counseling ability as rated by ‘he three professional counselor
educator judges. Ward found that the three professional judges
did not rate the 51+ counselors participating in the study in the
same way; their ratings often differed markedly from each other,
and these differences were found to be statistically significant.

Based upon a two-way analysis of variance, the range of
interjudge agreement between any two judges ratings for the first
four I':Iil‘fll’t0 seguents was .31 to .46 with a mean of .36. The
range of interjudge agreement between any two judges ratings for
the second four minute segments was .52 to .61 with a mean of
.56. These low objectivity findings suggest that the Counseling
Process Effectiveness Scale may have questionable utility.

Kelz (1966) constructed the Counselor Effectivenes Rating
Instrument, a measure for assessing counselor trainee effectiveness.
This rating instrument consists of eight major categories to
appraise counselor effectiveness in interview situations. The
eight categories of the scale are: (1) Appearance, (2) Expression,
(3) Relationship, (1+) Commnication, ( 5) Knowledge, (6) Perception,
(7) Interpretation, and (S) Ten-dilation. The subdivisions of
these categories represent personal (relationship) and technical

(professional) old-fonts of counselor effectiveness which are

I)”
.‘-

- (L
"

\

rated on a five point scale ranging from unsatisfactory to out-
standing. Accor "1 g to Kelz, this scale was developed by collecting
effective counselor characteristics from several references and
from discussions with experts in the field of counselor education.
Iielz selected those characteristics f or rating which he considered
to be relatively independent of the nature of the problem and the
1’:10'U10Cl of counseling.

The counselor subjects used by Kelz to test his scale were
tl'firty members of a I‘Iational Defense Education Act Guidance and
Counseling Institute conducted at the Pennsylvania State University
d111‘ing the academic year 1960-61. Standardized interview
situations were developed using student actors who, it was felt,
cmld meet the standards considered necessary for the portrayal
Of high school counselees. Two interview situations, each em-
pha sizing problems commonly faced by secondary school counselors,
V'Tere developed. One situation focused on personal factors
f-‘J- cilitating counseling relationships (interview S) and the other
eP-lfphasized technical proficiencies such as the integration and
£3“Iblalication of data (interview P).

During the study a panel of six trained professional judges
from the Permsylvania State Urfiversity observed, via closed-
circuit television, two 15 minute interviews (interviews S
{11343 P) for each of the thirty counselor trainees. Ratings

of counselor effectiveness using .Kelz's scale were made indepen-
dently by the judges during and directly following their ob—
SE>J:'vat:°'-.ons of the interviews. The correlations of interjudgo

a‘LII‘COI‘ECITt on interview S ranged from. .23 to .77 Earth a mean of

-33-

.53. This conpared to a range of .21 to .65 on interview P, with
a mean of .455. The average correlation between ratings given
by pairs of independent raters was . 50. The average agreement
of individual judges with the pooled judge ratings was .78 for
interview S and .73 for interview P. The judgments of counselor
effectiveness were correlated with the rankings of the counselor
trainees made by counseling practicum supervisors to establish
the validity for the ratings. The correlations obtained were .42
and .28 for interviews S and P respectively.

Although Kelz's Counselor Effectiveness Rating Instrument
ajppears to be useful for assessing the counseling proficiency
of counselor trainees, it lacks favorable test validation and is
characterized by low objectivity, and thus, low reliability.

Stitching from the global and four to five point continuum
counselor rating scales, Rank and DeRoo (1965) developed a forced
Choice, check-off Counselor Response System (CBS) Scale for the
IT‘Leasurement of each counselor statement along six dichotomous
dLimensions as follows: (1) Affective-~Cogrdtive Content, which
indicates whether or not reference to affect is present in a
C=<Junselor response; (2) AffectiveuCognitive Change, which deals
"‘fith gross changes in feeling level between a counselor response
and a preceding client statement; (3) Content-Follor'I-Shif t, which

C1eels with whether or not there are changes in the general topic
of discussion between the client's preceding statement and the
Counselor’s response; (1%) Present vs. Past or Future, which
indicates whether the counselor refers to or focuses upon some-

thing in the past, the present, or the future; (5) Restrictive-

31.1., 177.1ici1 deals ”with the extent to 1-:11ich the counselor
li::dts or per its freedorz. of e::1ression by the client; and
(6) Client--Other Refercnt, which indicates whether the client
or so: 1eone else is the primary referent of the counselor's
:r‘GSponse. The 3’31: di11ensio1 18 are theoretically derived,bu out they

have not oeen based exclusively upon any one counseling theory.

3

.110 CBS Scale was designed to describe, but not evaluate, the
verbal statertents of counselors during interviews, and is part
of a larger system currently being developed for examining both
counselor and. client respens es and their inter-relationshi 1115 The
rating evaluation of each counselor's verbal response involves
realjng six dichotomous judgments, one for each dimension. For
adequate evaluation, any given judge is designated to rate only
t‘LTO of the six dinens ions. Thus, a minimum of three judges-are
used to rate a counselor's verbal statements.

1302200 (1965) applied the CBS Scale to audio tapes of eight
cioctoral candidate counselors at the end of their practicum
8and to a group of 30 advanced NDE‘A Guidance and Counseling
Institute candidates at I-‘ichigan State Urdversity at the end
01‘ the 1963-65 academic year. Using trained judg es, DeRoo
C>lotained a range of reliability ratings on the scale's six
c3:111ez1sionsfr'..:o .75 to .99 with marginal reliability resulting
Oh the Rest1ictive--E:_, ansive dimension. Thus, in irrit' a1

.JiDliCJtiOl'l to audio taoes, Rank and DeRoo have demonstrated
Very f‘avorab-..e interjudge objectivity and reliability across
the scale's six dimensions

or the counselor rating scales developed to date, Rank and

C‘I.’
'JJ'

Deloo's 013 30.10 aooears to be the nest obgective, it is “no

t.
‘1
.)
O
, 1
E
’3'
0

nos; specific in the counselor echo -viors i
one of the least tied to any one couns eling t1eoqy; and i
r the '2 st cozzparative si' 1' >licity and ease of use by raters.
‘escriptive rather than an evalLative

‘-J
L}.
a

l
f.
‘

because its dimensions do not neces s

|_)

the counselin

r negative characteristics for

it pay be limited at this stage of

I:>c3haviors it r .3 ,
for evaluatin; counselor educatieh

(1 CV e101 tent ..)
“rims and counselor trainee progress, and may have greater

’n research as an ’nstvunent to identify counselor

1lress A“

" _0 w Q - o
’ 0". 00-10. 1210?.

(lfej) h s demonstrated the tthe co11dition 01

0.1)

nulzs presentation of naterialt a rater has a s b

O
<+
1a

-1 ~
(“J-.1111. O -

l‘4

cijJTOCt upon his rati;1gs. he presented audio, visual, ant
of a nuxDer of Ward's criterion tape seguents to
'tLl iree separate groups of four judges. Each segment was rated

L‘fi‘I \oeaLl" 1‘ ti 1e "udges of each group on each of fourteen counselor
J

”31'12’dwie,‘ along a four 1oint coz1tinuun r-n i1”

,
’KDlriablés; e.g.,
'e:ri’xremel;,rT scored 1': J. T'L'ie

tion

1

inpreceptible” (scored 1), to
conai

‘1.“
{“Krerage inter-rater QSTOOHGHD value for thee audio-viM1al
r'e inte1-rater agreenent for the visual consition

‘K? A .2. _ - . A

‘1€1s .7u, the averab

‘»‘ ‘- (n . ,3 .'-'1 -- a ’ —' ' - ~ ‘1‘

‘ 31s .Lw, and he average value for th audio condition 1;'as ,uq,
1 p flS‘to

the visual node of presentation lca
he" inter—rater agreement than he audio mode because it he;

' 17"" value.

1 ~ .
«m ”71‘8",“ ~' ‘3"
‘J.'. _ L.) kl... .L ,-~o~.. L)

 

terature indicates that

H.

‘ revieV'of the l

’) c
-J("_

velopmental studies of the past have failed to produce an
objective, and thus reliable, instmtcnt for evaluating counselor
behaviors. flthougl'x a majority of these scales appear to have
good face validity, they lack“ objectivity. Agreement among
judges using these scales is poor.

Although attempts to objectively measure counseling inter-
view data date back to 1929, counselor rating scales did not
become validated until the develo; ment of the Rogers' Relationship
Inventory Scale and the Anderson Interview Rating Scale of
3.962. Both of these scales had some research validation, but
Rogers' scale was too closely. tied to his own counseling
‘vi ewpoint, and the Anderson scale did not draw broadly upon
various types of counselor behaviors, nor was it designed to
Duke use of viwal cues available now through video tapes. Both
S cales presented rather cumbersome rating assignments based upon
f Our or five point scale continumtzs that were rated according
to the subjective impressions of clients. These two scales,
hits the Idoch Scale, the IPR Cmnseling Process Effectiveness
scale, and Kolz's Counselor Effectiveness Rating Instnment, which
f (allowed in development, are, for the most part, global type
rating scales.

The IELOCh scale, developed after the Rogers' and Anderson

6'2

cales, uses professional judges rather than clients as raters.

This

P.

s also true of the Counseling Process foectiveness Scale
and the Counselor Effectiveness Lia-ring; Instrmient which followed
‘1‘

‘ -, a V o o '!
me 3100.» scale in development. Follox‘nng the fonaat of tne

v) .
hOgors" and .mderson scales, these latter two scales present

Jo‘ - ‘- - t. r! (3 “fir-A \« A ‘ ‘ 1" V ‘- .‘ '-":‘
rather cumbersoue ratin as Signnents sod u)on fear or live point

scale continuuxs. In addition, they are not very objective and

.13

o: h“ .1 .1 r! ‘ "“1 r 1 r v f‘." fi ‘ r1 . ‘
rellaolc scales, and they suffer frox vclf meal valilatioa

Of “"7 the counselor rating sea les revie"ed £10 the

L..-'- . .

czounselor measureficnt literature, Rank and DeRoo's Counselor

' ' - ‘J' u‘ fl A ~ ~ “’- . ‘3‘ w ~ 4‘ ‘~ 3 J' . ‘ ‘ "
files pens SJSUCJ uCulC appears so oe the hose ooJecoiJe and

zausuable scale for neasur :3 coxm selor behaviors. Unlike the

CJLimn'scal es ernnined, it is least biased toward any one coun—

:5celin' “~001‘23 it is not ve lue laden; it deals with specific

f
\_)

:111d obs rv: Talc oelavi rs ratl er tn; iglobala and im ”erred
<3<3nstructs' and it presents a rating assign: ent ch aracterized
'13:; sixplicit; and ease of ere ‘tion However, the CRS Scale

fiL:3 designed to describe, rather than evaluate, counselor verbal
IJehaviors. It neasures certs mi counselor behav’ ors and

{Ittenpts to avoid placing a value as to their desirability in

(counseling.

T't‘ *3 pos‘ible e: :ce: tion of the IPR Counseling Process

’bJ-A.

LEffectiveness Scale, and Kelz”s Cou.ns elor Effectianess P iting

:IW1S crunent, all ‘ escales reviewed were developed primarily to
I:easurc audio taped counseling behaviors and are unao‘e to talce

dvrnoe e of visual ucs available on video tapes. Visual cues
ppecr to enhan cc interjudge arfi elent and thus add to the

C37je“tiV13y of a counselor scale.
" Lselifi; Elevents Co: on to Various Counselinr Practices
“his part 3’ the review of the research was done only aftel

9 video taped Verbal Couyrelfi_n: behaviors were

collected and tested in an effort to deterziine the cormon counseling
elements or dimensions characterizing these behaviors. This
review attez-zpts to specify those elements of counseling which
are common or relevant to all counseling practice, regardless of
counseling orientation. suvmarizing the research data on counseling, 1
“the "Review of Fxhlcatimlal Research" (April, 1963, pp. 181L-185)
concludes that the trends in the data of counseling research
incticate that experienced counselors probably produce better
counseling results than novice counselors and that most experienced
counselors choose their 1:1ethods to fit the counseling task rather
than a stereotyped counseling orientation. Fiedler's findings
(1.950, 28) suggest that the better trained therapists of different
Schools agree more with each other than they agree with less
‘bI‘ained therapists within their own school. Fiedler's findings
(1950, 29) also suggest that expert therapists of diff eling
Orientations formed similar relationships with patients. In
Cliscmssing ccnnseling process Cartwright and Lerner (1963)
point out that frequently studies which compare the respective
Jpercentages of successfully treated cases suggest that some
common elements exist among the various approaches to counseling,
I‘egamiless of theoretical orientation.
Truazc and Carlchuff (19610 mention that an examination of
the mar; foztmlations designed to describe successful and
effective counseling reveals such common elements as the

.‘E‘ ellorfing:

re

1. The abi 'ty 0

the counselor to accurately understand
the client.

"f3

_Jy-

. The ability of the counselor to communicate this

understanding to the client.

j. The ability of the counselor to be able to accept the
client while being integrated, mature and genuine within
the counseling or therapeutic relationship.

CPhey'note that these elements of the psychotherapcutic relation-

:3Irip are aspects of the therapist’s behavior and are common
lea]_enents in a wide variety of psychoanalytic, client-centered,
.gzrqd.ec1ectie approaches to psychotherapy and counseling. These

:z‘€>curring themes are emphasized by psychoanalytic theorists such

sans; Alexander 1948), Forenczi (1930), Schafer (1959) and Halpern

aizdxi Lesser (1960); Client-centered theorists such as Dymond (1949),

e712n4rard (1959), Rogers (1951) and Snyder (1961); and eclectic

1:11130rists such as Rausch and.Bordin (1957), Strunk (1957), and

S trupp (1960).

Truax and Carkhuff (1964) indicate that another aspect of

131343 counseling process commonly pointed to by these approaches

IJJCIS been the client's exploration of his feelings, his values,

11EELS relationships, his fears, and other concerns. Host of

-tiflese approaches, they add, consider the client's self-exploration

‘tLCD be one of the central happenings resulting from the process

c3’1."psycl-lotherapy. They conclude this self-exploration is

1Tlc>st likely to take place when the counselor communicates

EL(aceptance and understanding to the client. Based upon their

<3vrn research and that of others, Truax and Carkhuff 1964)

ifeel that the findings of importance to a theoretical model

fOr evaluatiig psychotherapy were that patients who explored

t«leir feelings and concerns were and more throughout

psychothera; y also demonstrated the greatest constructive
personality change, while patients who underwent very little
self -e:-.jfl.oration throughout the course of parchotherapy showed
little constructive personality change.
x‘uiother crucial variable for succesle psychotherapy
implied by the previous practitioners of analytic, client-
contered, and eclectic theory has been that of concreteness.
They all 2:39.50 references to specific problem areas and affect
and regard abstract interactions as non-therapeutic and even
defensive. True}: and Carldmff (1964) note that, "...perhaps
one of the most useful learnings is that the cases lfigh in
understanding, high in patient intrapersonal exploration, and
17:1. gh in case outcome seen heavily loaded with specificity or
concreteness of interaction." They conclude that this seems
‘to serve three important functions: 1
First, by enmring that the therapist's response
does not become abstract and intellectual and, thus,
more earlotionally removed from the patient' 5
feeling and experiences.

Second, by forcing the therapist to be more
accurate in his understanding of the client.

Third, by influencing the client- to attend with
Specificity to problem areas and emotional
conflicts.
.Smmmry
A review of significant counselor behavior literature strongly
Suggests that the counseling practice of experienced and trained
Counselors, regardless of theoretical orientation, is characterized

by corrion elements. Specifically, these coiugzon elements are that

-"r’..'--

\

e:q)erienced and successful counselors (l) sensitively and
accurately understand their clients and comrIunicate this

understanding to then; (2) canmnicate acceptance to their
clients in a genuine and integrated manner; (3) penrit and
encourage self-exploration by their clients; and (1!) deal

specifically with the important concerns of their clients.

A“: I ~ fife ‘3 -I'P
‘J.-J..LJ.LA14l‘b .L .L

31‘1“." ‘fl (xv-«fivvfi m
HIV WA.) 314 ..‘J-c—bel ..JL-‘L

The Si: ingor steps followed in the dcvele'.):1ent of the 1‘1-7
Scale (Eigv‘c .., page 1‘3) are oi scu ed here. These steps we e:

‘

1. Sb ‘erea ti on and co:".pilation of O.‘.‘.’.’::LO-VZ‘..C‘1.CO taps .
co .‘.. so -.or behaviors.

2. C0. .30, OL'ILzation of 5110 observed a_1.1d_-'Lo_~.;;,100 taped
CO~b‘\:1\JO--Ol‘ 1301111112103".

3. Organ." nation of categories into a scale format.
I, .- " ‘ ‘5‘
If. Jimj_):Lj..0 4‘

e: t'
couns elir

ion of tae scale to c- -e reti1.; of am‘io-yfi .lee
1g? tapes

5. Definition of scale co.;'1e1'1sions.
I1. Develo 1.:ent of a scale scoring S‘fs‘tej‘.

,v-n‘ - 1r

5 a V
1-10 --.-. Scale as developed for the purpose of co :parxu ' tge

.A

essence 35"”[301’113 of 11.1‘1. cand. dates t-I'Lth those of Ph.D. ca.:e1";fi.e"t .

T: “ ""3 'n 4°1\ ‘: DI' fie . "\ t‘aé-H. '7 " N fiL\-‘ ‘7‘ "- J': I’ ‘3
peev-u.Lc)Lz O... L: .O (LLL-_01311CO 3.1.. -.-L‘LL."1:; 1.». I‘C.:...IO.L1. 4.0 C.-UOCQO~U-LU-;
° , .Lj.,..'- .l.‘. .. . '1 '. ,. m J.‘ .... l‘ -L‘ -- 1 . " . . ‘ -. .
is Lucio the co17.-1:,.,_1.inu -.aote “1.53 0.. ..ae t. 0 Classes 01 coanselers
~'. ‘° 1"" -- ‘V- .1 ‘9 .... ..,‘ 1 V.. 1-. . ‘. ° . J- '- . J-
alll Ci-.. -.-‘. _.-.1e scale (_.C‘."Jlf)73‘.;~.. 1:. axis research is an _-l
-: ‘r.-»‘ J" . I“ v “\ 1 ‘.\ . J‘ . - 'c --- ~ ‘ . J-
to .LCAU-.V1-—:, 3-1.. OwJGCol.’91J . oasure Specific an; relevan.

. “'14.... . 'n‘l- . .3 ° f. . .. .L . . 4.1
coun:.e.-z-..g v0.3.1...oles ..aicn see. to disc-1" 1.:in..te 1e“:'.een LA-C

.1

.-~,. ‘ ‘ I'\ . '.- 1‘ -I > A. r‘. N h ,‘. ‘ J. ‘
001.12130.5.1.1.‘\J l)Cll.'.V-J-Ol‘ Oi _'...-.. .-.: 1;. l..D. C..Y‘.-.Q.Q.0.L,OS.

 

The first step in the development of the A-V Scale 7.11:: to

observe the "“L~'O-".’:-‘LC‘O ta )ed counseling be} aviors of Loth "

1... , . ..1- 1,..." - , ,. ~ - .
1-10 \lCVOlOHI-O-:b of this scale .10..» eased prinaril upon (31133.3.
- 1
1 1,-

U .sC.’.LU.

’b".
-* 3/4,-

-43.

348 moyémm mSmmzooo >4. 1... H 93mg

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

D a... . ..
n
. nod mo
9“
2H
2H
d—
m—
in
«H
6..
4H
C—
a
a
N
c
m
d
r
a
a
H N n q Joule w gum « 8 30mm we a u
@333: «.5303: 1.3“an— .ruov§.na show:
1.53 1:02
Inga £85323
umuonwom peace—50 ,
III "3.5 Boo—.35.." "out;

35m 95.53 umzommnm MOH>$fin MQHMHBO >14

 

 

n: ot-
cl .
..
.
t' I
'4
"' I
U

.'.al I.

n...

 

Ll
--.-,--

‘

05111; 7.0: "Les 0.1.6. “11.3. canafidates. Audio-13-1100 counseling; tapes

0

of t-1ese 17.13 classes 0'1-

,

counselors, ave -3- :‘1-11010 at "he 13301135111 State
1 Diversi- C0---ere 0f 311110.11; 3.:01 010 013.5crve:.1 The audio-"1130
9
001121303111; tapes 1030 of 0011-.selors 0131331113 out personal-social
11:001-310.: 1.1-0.1310 1.13-1.11 clients at IicEfi-;;an 51:0. be University.
.' l f. I .L p L] w ~ 0 I‘ J.
thee 0 03:10:31.1? 3-:-“01s. 1110 1:13., 17:13 11110 .01“ 0.10 1:101131710rs o- .11.:

V ~ ( .0 I. 1 ‘0‘ l‘ .. C ‘_L .0 _‘ 'Q a ‘ _f‘
I‘D-.3. 00.11-11.10- and 0110 13.01, 1.218 1.0.00 101“ the 0011a.-.0 :5 0...

l J
,- 1
0

$3.... cr11--.1..f_.1;cs (11301.3.-- :1, p. 101). For each list, the

Toe-'10 1.13 .3;- Irere 00‘ 1313.108. under 03.1.1101: an Affective 11011151115 01' :1

'-J

-.H: J.-." -r ‘ ‘.‘, .- ,. . ”J. N J. . ° .
1307,11 . 0 3.0.1..- u. ”3011-1301013 spate-e111,: 1. 1:1.01-r 010 1 ~1~ed t0 the
e-gien'Us 3005:1113 or alibi-Javier: were 1' sted 11111101: the Afif‘ective

L -
\_) cool-

--\l~. kt.--

’V‘ A " '. ' '~ - - ‘fi .1 ~ 1 Q-s v '. q ‘r- r‘ “- "

1.0.1-131.“ CO‘U.:..'ZO--OI‘ state .10 .‘t5 .;-3-I'.c s e 0-1 s0 leer. my 13030101100
v~a’.~ ’L>~. ‘, . J‘ 1 -t ' ’*u ~v - .1 . .n - . -

to or 1:000 -.=-0:'-0-1 0f tzlc 0.1.10; 10's 3001111955 .030 113.011 11.111 0

'- n ,-.v.--, , 1 -.-,, 1. . .- '1 - "J. . . .0 °..: n11...

t;1e .0;,.~1.-- 0 ‘ e set-113.11). 1.0 1.1.0 1.1.0:. more p11? sed 01 01.1.1.0 1 ,

(10130121911170 c..'1.ss:'1-f..01‘.cie11s 0f 1’30 1011:1033 ‘1 0:51; fr0311(,‘.1‘bl" 07"“5‘Vn‘

'L - . 'f‘ ‘w n ‘ "‘ \~-'\r~ .1 - ~
1,0 00 031cm.1*0r;1;01'3-t10 0- 03.01. 0-11-5.5: 0- 00111.5--0111.

f‘nJ‘ ..'- VA ”6"". r“ ' 2.“ ’YNr‘n'n0rf‘t’J
-'I(I 10,--.-.:€.-1J-1.‘..)JA C ‘J.'.O '_ .'._).,_ .‘ z A
A __A._

 

 

. ' , w 1 n .- ". '1 . .
..10..0-.3 . '30 Ts. 1e . .0u-.se_-0r 3.1.13 3.::

w1‘. ‘ m-.. - 1 . 4: , - 4.x. .. a
-..e 1.11---.~.-'1.0‘ 0133033.---5201 co.._'_):1..1-e:1f0: 0110 1.1.1.1.-..-

VV \“—. - u 'Ir‘~\nr‘_ J‘ 'L‘.‘ (V -\ v1‘ ‘31 A" f ‘A ’L‘. 71‘ T‘ fl‘flf—V (‘AJ‘ \f‘ C‘\
.oCIJ. t.) CO. .N/I.... «_o'-.‘. L‘O UA.O;)('I CO. 3... L‘v‘sas ‘ .L b-10 - LAQJ. CgL’L-egl.»LLL\J'/U . 3.0

.. '-.- _,-,-1.'1 - 47.1.0“ -. 1- ., 0 ‘I .. n ., ‘ ° . - m1
1101030315140 .-..-.er1:-1ce 00051. -0 013-00111, 11:01.1 “0111-55 00 1"11‘18021. 1-10
.1.-- . .0 ~ .1 . . -'~ - _ 4.1 ~» 1 .- -, - . 4-1.

W905) 0~ 9'3— C‘V—L‘ 35 Closer-1133-11; 0-110 1mm. 43-11. .8 1.030 01:13:30

-.‘ . ‘S V" ‘ .‘l-t J“ (- 6' 'c‘ A‘-"‘ v ‘- ‘-. '3“ «u. <. J“. ‘\ .s ,-
...1:Lc.. .030 --ssexl .1031: free-110.101,; 101.113 0110 (309.11-01.10 seeds-11”.

w- .'--'..- ,.'. J.“ .!.,_ _‘n V j - ‘ I‘. 0-“ 7 ‘ '1 ~ , ' 3 1.
.1-11 CO.-.0--1.0\5, _..3 0' )1“?- OJ. 00.10.131.011) ..SCI’IL; 1" f‘ 110 911...). 001111-1100?

.l.- A o
~-,_J

l

M9 x -.p~' "-.. .A1‘ I. J.- . —.—.. n -. 0 , . ¢ .“ . .- '1 -~—.*. . .3. .
.110 00. .41.---0.-...-5~- 0. 0 10.50 03501'V.-11.1-011s Wis .11- .- 1.111 . 1 1 1;. 0 ~

V
-145...

\

were those which were most frequently listed under the Affective
heading. It appeared that the two Classes of counselors were
apparently differentiated on the basis of affective and cognitive
responses.

Because the Affective and Cognitive categories encompassed
a broad range of counseling behaviors, additional categories
were sought for finer categorization of the compiled counselor
behaviors. The review of the literature (Chapter II, pp. 37-41)
suggested that responses which were "understanding," " specific,"A
and "exploratory" were common and important counseling behaviors
characterizing the practice of many counselors of diversified
orientations. These three types of behavior designations appeared
useful and relevant as categories for classifying the compiled
counselor'behaviors.

Based primarily upon the clinical definitions of these
counselor behavior designations from the literature, an effort
was made to classify the compiled counselor'behaviors as to
whether they were understanding, exploratory, and/or specific.
Briefly, understanding is defined as the counselor's ability
to communicate to the client the fact that he knows what the
client is feeling or'talking about; exploratory is defined as
the counselor’s assistance and skill in helping the client to
explore his feelings, values, and other concerns; and specific
is defined as the counselor characteristic of being concrete
and getting to the core of the client's problem areas.

A majority of the behaviors compiled for Ph.D. candidates

seemed to appropriately fit under one or more of these categories

(indendi: I, a}. 135-137). A majority of the beha.iors co piled

A - "‘ 3 ' 3 r- fir ‘ '3'. I‘ ‘v‘ '-- ' "A ‘ . J‘ W- 9‘. ‘ ‘ v" "
Io: t..c __,.... 1.1 J. Lat tes and .-et see. so apropr'iately 4.-.: 111-103?

A

$1)
(i
O
r 3
F I
b
C)
O
E}
)3
O
O
O
I.)
C a
9
0'
W
o
(3')
0
CO

. O . A '1 ‘ ' ' ’1 A. .
w ~*er, bf uSinn -nonunee -ou““k‘“L,

n ' fl: ' . 1 -.., . --,- . (..- A. J. 0 .. .I.‘ \
WORSUOCi_lC,' “f” me o Mlo~ato beha.j r (as gnaeio-u, ..a
‘O \ ‘ . J“ " J“ '.‘ ‘1 ‘ - 1 o - ' fl t 3 -" ‘ - ' . ‘ . I l - I N
‘ -., -.v.-x., O. '- 'J ‘ '31 ’3 CU. u 10 . o... s . I r (H b) L)’
v :- ~ ' \'- ' v. " -. ' 3: \ " "‘3 -"-.~ - 3 ‘ . >3: - - fl ‘ - \ 1 “(N ‘1’ l,/\\ l I- ‘v‘, n" -
()L'r') a. .‘. 'J ~‘.. .1.\-KI A) ~ ' 0-...” I. - _.. -.K} A .‘1- ;.fl~_3_‘.-_l_,. ~-.\L1 . .."--\.."-.:. I.) / . ~ .6 -.- \ _L _.-
., - .. U l - ’ a s. ’

13_u:;-;wfr;l;h.".. ‘a;inee as the coan selor's failure to

‘13:, -’\"'\'. Win 011 .f“ 11:1 \fi‘“ 3 4") +1.“ 1 . Y‘t J‘lxq'i‘ \o '~fir-v"rrw‘ v7r‘.")"L t“(‘\
lx'CV -i.'.s1v «L C‘)- _.Ak‘ls -(w UL! Ll Vné'xl PhlolOoA UA‘LHU .\I :3-..) tolLLskJ ‘1--

-,-"‘ - ~ 3. , ~ -\r- e. 'L - 4' . a r . w‘. ‘
leze one in -uslmng; none;;eloraeony is defined as the counse o3n r’s

:7. -.. .v. w .1 .- -4 '1 x.- .. - a...
faila-a uO aeup .ie clie ant to enplore “l3 feelings and concern“,

W

and nonspecific is eefinee as the eouzselor characteristic of

being vague ang.keeping clear of the central corcorns of the client.

 

:13 .-. "I 0

a I'M A '0 . . v .n . r” .‘ v “ 'L f ' f‘
Lao e-. “b categories of affective, he itive, bnoerseanding,
an>rn ntazy, Specific, orii<nrstanding, Honexploratory, anl

qv o no ‘ V ' ‘\ PJ-

. 1 ,... . ,‘ J- ,. ..L - n o. - .. J..-
. O.’1$.‘OC-°.-..:LC L13 ‘Oill‘Ou LO LJC Sgblirlf’ C‘LIOI'3 iOI‘ lllCC 3.2361. ‘0

tien int

. . '3 - "A- J“‘- - A . ° - -!-‘ ~ .- 1

a scale fos: "or three reasons: First, they seened to dis crininhso
L1 . . . :- v a ..a ._,. :- .. w: 1 -
tae ras30nses of _.n. candidate from taose oi 11w. u. ,anaieates

a ..a 4.3 .- . 3 - . _. .. 4» H- , - M .- ----. .r\ '1 -
second, eney seemed to once pass tne gr at “flJOTltf 0. con sel r

0030? i :tivo 30h viors compiled. Ana third, they seemed to refer

1‘ 3* ~3~n- ° ‘10 '4‘: '3 ‘1 (1 s a“ -'~ ' q-lfi- ‘ °- 3 t 3
b0 lJ!~-,l‘.'.-'J,:LO.LL_‘ ‘.:-...1-L~i COL»... 1'30 I‘D. ullOl QAJLIC.L_f3-CQ.L—~d' QOLLQJO. cll‘Kl
‘\ 1 1 " ’0 1‘ . '. .. ’3. ‘
OL/‘JDC L: ' v0-3..a' .L‘-'.Oll ._ .OQ.
an -.a ..- ‘ a ,. . 4-1 '1 An o . ~ :1 . .. m
lllC 110- .v 33004.83. 1.3.3 ullC? CCQOCvlon O... 3 SOC-.10 fortifit. 4L lO;'. Co
v ' ‘ Ix " “ '.V ‘ fl “ A. . .. . J- V. J- . . -. 1
“as noodoi Halon COLlu :3rosez t tllose categories in the ”OSD Girdle
"’3 .L .,, _,A .1- _ p n . 1 .L‘ .1- w - ..,.. .. . w , . ,3
It .as eageceed cloranane snap Ph.D. cam13m_dates moalo be elc‘act131n3H
by affect, KLN- .‘3.~SLJ&:‘.1A;(»“j- 1'1", C§';}eCif'.C 9nd C3")---Ol“" 1631“" rns:\onsng 3111.:
\J A.

. A n ‘,(~ 1' r c ‘ W I ‘ 3’ ' - ’3
- _..). Ct~15r-L'.‘x.‘.k.'.'\J\/‘S ~01-U~VL UC’ 0-33:1.0 40.1,. Z(%CL 1);],- (“04.31.11' ’3, EIO'TJ 'WCT _01‘()+U(.V1' '-_
--~ -' "3 n A 3 ~m - 1" w J' - -v 1 -
1101113.;OCJ... in , . ..11L'. .1O;1(3.3.1;)l 3.0.31.0 LU respo‘; 117. ,5, .

.47-

but effective rating form possible. Rank and DeRoo's (1965)
dichotomous dimension, forced choice CR3 Scale format offered the
most promise fo meeting this need because it was simple and seemed
capable of objective judge ratings.

It appeared that the eight categories could be satisfactorily
redo into four forced choice dimensions. A counselor response
could be either affective or cognitive, understanding or nonunder—
standing, specific or nonspecific, or exploratory or nonexploratory.
The eight categories, therefore, were made into the Affect-Cognitive,
Understanding-Honunderstanding, Specific-Nonspecific, and
Exploratory-Honexploratory counselor behavior dimensions. These
four dimensions'uere adapted to a forced choice 5 ale format

similar to that in Figure l, pace 43.

A plication 9£_the Scale 22 the Rating

A

g; Audio-Video Counseling Tapes

In a series of rating sessions, the four counselor behavior
dimensions Here'used for the rating of audio-video tapes in-
volving I.A. candidates, Ph.D. candidates, and, in some cases,
counselors with the Ph.D. degree. The counselors on these tapes
were engaged in personal-social problem interviews with clients.
The judges used for rating the tapes were Ph.D. candidates in
Counseling and Guidance. They were provided with category
definitions and illustrations as presented in Appendix B, pp. 1?f_ifi

During these rating sessions some of the tapes were used
for clarification, modification, and expansion of category

0 OJ..

definiuions based upon judge feedback. The judges employed for

.148-

!

these rating sessions were in agreement that the categories of
the four dimensions appeared to differentiate the Ph.D. candidates
from the K.A. candidates on a rather consistent basis. Their
frequency ratings for the affect, understanding, specific, and
exploratory'dimension categories were consistently higher for
the Ph.D. candidates than they were for the FLA. candidates.
These counselor categories, as measurement items for this type
of scale, seemed most meaningful to these judges as counselors
rating other counselors. They felt the dimension categories
referred.to important aspects that were readily discernible and
consistent in the counseling behaviors of M.A. and Ph.D. candidates.
During the ratings it was sometimes observed that verbally
a counselor could be rated affective, specific, and/or exploratorya.
presenting on.paper a positive counseling impressionpawhereas in
fact his nonverbal manner could.be so rigid and mechanical as
to have little, if any, positive effect on the client at all.
It was judged that this should be taken into account by the
ratings. Thus, the definition and use of a more global effective-
noneffective dimension was discussed.
It was then decided to incorporate such an Bffective-Noneffective
dimension into the scale. The Effective-Noneffective dimension
is defined in terms of counseling progress (see the next section).
Effective rated reSponses are scored either 3 or 4 and the
ineffective responses are scored either 1 or 2.
The scale with the new dimension was applied to an analysis
group. Five judges were used as raters. Again, the results

suggested that the scale had good discrimination power for

differentiating: between the responses of ILA. and Ph.D. candidates.
Also, the high interjudge agreement obtained suggested that the

scale had favorable objectivity.

Definition 9_i_‘_ Scale Dimensions

 

The final form of the A-V Scale consisted of the Affect-
Cognitive, Understanding-Nonunderstanding, Specific-Nonspecific,
Exploratory-Nonexplorato2y, and Effective-Noneffective forced
choice dichotomous dimensions as illustrated by the scale format
in Figure 1, page 43. The definitions for these dimensions are
based primarily on those given in the counseling literature
reviewed (Chapter II, pp. 38-LL0), clinical judgment, and the
feedback received from judges who had used the scale to rate

counseling behaviors on audio-video tapes.

Affect-Cogétive Dimension
The Affect-Cognitive dimension indicates whether a counselor's

response is in reference to some affective or feeling aspect of

a client's verbalization (an effective response) or whether his
response omits ans,r such reference to affect or feeling in the
client's verbal commnication (a cognitive response). Basically,
an affective response refers back to client emotions, feelings,
fears, moods, desires, urges, impulses, fantasies, dreams; and

any ideas, attitudes, beliefs, and convictions which are based
upon such referents as these. A cognitive response, in contrast,
is devoid of any basic concern, reference, or recognition of client

emotions, feelings, moods, etc., which may be implied in the

-50-

client's cornmnication. The counselor responds at a non-feeling
level or asks a question or makes a statement concerning cognitive
or content material. Such cognitive responses are more formal,
informative, matter of fact, or on the order of a general cognitive
discussion, .and are altogether on a different plane from responses

which make affective contact with the client.

Understanfirgg—Hormnderstanding Dimension

The Understanding-Nomnderstanding dimension deals with how
well the counselor's response communicates to the client the
fact that he understands, or is seeking to understand, the client's
basic concerns. Understanding responses indicate that the
counselor understands the basic underlying affective or cognitive
concerns of what the client is talking about, or is trying to
obtain from the client sufficient infomation to enable him
to develop an understanding of the client's underlying concerns.
Nomnderstanlflng responses are those which indicate that the
counselor lacks an understanding of the basic underlying affective
or cognitive concerns of what the client is talking about, or
are responses which indicate that the counselor is making no
attempt to obtain appropriate information from the client which
would enable him to develop an understanding of the client' 5

underlying concerns .

§pecific-I»Ion§peaific Dimension
The Specific-Nonspecific dimension indicates whether the
counselor is dealing directly with the client's basic concerns

by confirfmg his response to a concrete (specific) aspect quite

relevant to the presented problem (a core response), or whether
the counselor passes over dealing directly with the client's
basic concern and gives a more general, or nonspecific, response
not to :mvlevant to the presented problem (a peripheral response).
A specific response, then, indicates that the counselor is
handling directly and honestly the basic problem presented
ezqylicitly or implicitly by the client. The counselor's remarks
are keyed to the central aspects of the problem which the client
is feeling and cormninica'ting. A nonspecific response, in contrast,
is a pelipheral type response which indicates that the counselor
is not dealing directly and honestly with the basic problem
presented explicitly or implicitly by the client. Rather, the
counselor's response stays away f rom the client and his problem,

or at best handles the problem at a superficial, surface level.

Ebploratoxy-ZIoneaqaloratory Ilimension

The Emloratory-Zionexploratory dimension indicates whether
the cmelor's response permits, encourages, or elicits further
exploration on the part of the client of any aspect of his
problem areas or affective concerns. An exploratory response is
one I-Ilfich first, indicates that the counselor has recognized
some portion of the client's basic problem or concern and,
second, has made some verbal reference of this back to the client
in such a way that the client is free from any defined or
"limited" structure in his response, and is thus free to respond

to any degree and depth to the counselor's reference that he may

choose. The counselor's whole manner creates an atmosphere

-52-

f or further exploration. A nonexploratory response, in contrast,
indicates no recognition of the client's basic problem or concern
and no attempt to search them out further. This is a response

which also structures or limits the client's response.

Effective-Ileneffective Dimension

The erctive-I‘Joneffective dimaision is based upon the judge's
professional impression as to how appropriately, from a counseling
vantage point, the counselor's response deals with the counselor's
verbalization and thus contributes to the development and maintenance
of counseling progress. Counseling progress is generally defined
as concentration of discussion on basic problems and feelings
of immediate concern to the client such that there develops
for the client an increasing undistorted awareness of their
influence and consequence on his behavior and relationships
with otherS. In this regard, the judge decides globally...
taking into consideration the counselor's total manner and
behavior, including nonverbal—the effect the counselor's
response will have, or could have, toward permitting or en-
couraging the client to move in a direction of counseling progess.
If the judge evaluates the counselor's response as effective he
can either check a 4 or a 3 under the heading "effective." A
rating of 4 indicates that the judge's professional impression is
that the response is about as "good" or the "best" possible in
terms of counseling progress. A rating of 3 indicates that the
judge's professional impression is that the response is effective

towards some counseling progress but it was not the "best", that is,

-53-

a more effective response or communication could have been given.
If the judge evaluates the counselor's response as neneffective
he can either check a 2 or a 1 under the heading "noneffective."
A rating of 2 indicates a response which contributes in no way
to counseling progress (sort of neutral) while a rating of l, in
addition to this, indicates also a complete lack of understanding
or concern regarding the client's problem situation, or is a
response which is definitely detrimental to counseling progress.
Further elaboration and illustrations of these dimension

category definitions are presented in Appendix C, pp. 111-125.

Scoring System For T112
A-V Counselor Behavior Scale

The unit for evabaation by this scale is the counselor-
client unit. A client verbalization is emitted or elicited and
the counselor's response to it is then rated on all five dimensions
of the scale. One forced choice rating is made independently for
each of the four specific counselor behavior dimensions and a
fifth rating is made evaluating the effectiveness a». ..oneffective—
ness of the counselor's response. Generally, each client-counselor
unit is judged independently of preceding units, although the
congruence between the direction and level of counseling movement
by the counselor with the direction and level of counseling
concerns the client is attempting to present must contimally
be kept in awareness when making rating judgments.

The marmer of scoring the effective—noneffective dimension

has already been described (pageSZ). The scoring for the other

four cbhtzezzsiozis is rather simple. For each counselor response
rated, a score of l is assigned to each cognitive, nonunderstanding,
nonspecific, and nonexploratory category marked, and a score of 2
is assigned to each affect, understandim, specific, and explora-
tory category lizarked. Then all the responses for a particular
counselor have been rated, total dimension scores are obtained by
51';j;13 “L7 5-11lVlrFCJ. seoswxzrii thc;i:'o catmxyarii(;:w3 rudrxra'
each dimension. .:lzen the test administrator is interested in
the frequency ratings for particular categories (e.g., the frequency
of affect, understanding, specific, and exploratory responses within
a group, or between groups, of counselors) rather than total
dimension scores, then only a score of l is assigned to the

category each time it is marked.

The A-V Counselor Behavior Scale x-ras developed in six

steps. These steps were:

1. The observation and compilation of the audio-video taped
counseling behaviors of Ph.D. and ILA. candidates in
Counseling and Guidance at Liichigan State Universi 3'.

22. Tao search for appropriate categories by which to
categorize those compiled counselor behaviors. Affect,
Understanding, Specific, and E‘qoloratory categories
seemed to appropriately classify the compiled behaviors
of the Ph.D. candidates. Cognitive, l‘fonunderstanding,

lTonspecific, and IIonez-tploratory categories seemed to

appropriately classify the compiled behaviors of the 11.11.

candidates.

The organization of these categories into a dichotomous
(Extension, forced choice scale format. The dimensions
of the scale were the Affect-Cognitive, Understanflng-
Itommderstanding, Specific-Nonspedfic, and Exploratory—
IIoneiqnloratory counselor behavior dimensions.

The application of the scale to the rating of audio-
video counseling tapes. This was to test how well
judges could differentiate Ph.D. candidates from 16.1%.
candidates using the scale's four dimensions. The
judges were able to differentiate these two classes

of counselors on the scale's four dimensions. Based
upon these rating sessions a fifth Effective-Noneffective
dimension was added to the scale. Category definitions
x-rere also clarified, mocflfied, and expanded.

The defizfition of scale dimensions.

The development of a scale scoring system.

I STIIODOL OCI

_.', <;(J-~¢;;;;;j.o;q of the methodologies of the three studies

to test the hroothese set forth in Cna :t) or I ' 3 er-—:

L) ' V J. A

Q ‘ «1-,
/
4.

center; 1101‘0. 1 {'15 chapter is divided into three sections; a
discussion of ' 1e . ‘zethoC. ol 03/ of tie Scale Di: scir-i nation StudJ,
a ri’.scus:.;;"~.on of the method logy of the ijectivity Study, and

a discussion oi the zzetl‘xodologr of the Sensitivity Study.

1‘ J-‘ n (‘4‘ 9 r1“—

.. .— n. Hnmvh-«f‘qu-4 'n .
ll. .3 .3CCL-.O ..z...-J‘4J. .‘.. »-~. AK~ U...‘3.- 5." bit ‘1

 

O

00. 13-10 :1 rpothe: is that t.1e frer iency of

\ .g._o’ __ f _ ~ , 0‘A "_‘ o o .vqfi 1 ' n4 '
air OCbl‘v'C, U..1LO£St€l.luLll ..pechfic, e plorator; , and ef: active

in
I ~ 9 o o I Q I 0 _g- 0 .‘ .
reseozmes in initial personal—somal problem counseling 1 ibul“.’1<3..:

will be si .if‘fcantlr greater for Ph.D. c.11detes t-1a‘1 for

‘ - a ‘ ‘ C

-..-;. oancicaoes beginring their pra cticur: expelienc es . The

;:'-.ethedolo;f desipned to test tl-‘is hypothesis is presented here
H

under the} headings of Counselor Subjects, Judges, Judge Trail‘clng,

1 ‘ocedure, and Design.

 

(‘01 ole? Lilfl' cots
Th Halo-video tapes of five 11.91. candidates (3, E, F, Ii,

and I) and five Ph.D. candidates (.91, C, L), G, and J) were used

I . :‘1':
My . 1.

in this stu 1e 21.3. candidates were undergoing their firs
counseling practicum experiences. The Ph.D. candidates had
completed their graduate practicum training. All ten counse elo1
underwent an initial personal-social problem interview with a

. («4

high school . cadent of approrirntely 30 rinutes in length. ~'-_‘hese

/

-57-

ten couns1eLL1; interviews were recorded on audio-video tapes.

Thel ive counseling 5e381ons involving th ue P11. D. candidates
were specially audio-video taped for this study. Ieit-1er t 1ese
counselors nor their clients knew the cm rpose for which these
tapes were to be used. They were simply asked if they would

volunteer for a regular personal-social problem counseling

~eneral couns lor

session which would be audio-video taped for Q
’tra1in;Lnr rese .rch Tur1oses. The clients were all volunteers

and were ass' gncd to the counselors on a random bas;s.

The five counseling sessions involving the X.A. candidates
were taped earlier as part of another research project. However,
none of these tapes had ever before been used in the d(velow ant

and evalu ation of the A-V Scale.

"'1"

1nrce qu cs ratei the audio-video tapes of these ten
counselim1se,ssions. Two of the judges were l;61ic1i:an Stnte
Universioj IDLA Guidance and Counseling Institute graduates. In:
third ans a Zester’s Fraiuate in Guidance and Counseling. .“ess
shrei jILQOS had not seen any of’the tapes prior to t.

session,nor did they knor'an; of the ten counselors the; were

i ‘ Q 'L — -'-
asned b0 rese.

w ‘ . ’P-an’ '- n
Jim/3 _J....3.*‘J.D.

 

\

All three judges had been trained in the use of the A-Y antlé.

3a ‘1 judge was given a copy of the scale, the dine 1sion definitions
and illrs m‘ations (Appendix C, p.33j,), and the counselor audio-

video behavior categories and illustrations (Appendix B, p.1fi' )

.1.

for send“. Following this, a training session was arranged in
which the judges were allowed to use the A-V Scale for practice
ratings of counselor responses on audio-video tape. Before
beginning the practice ratings, the use and rating assignment

of the A-V Scale, the dimension categories, ard.the technicalities
of the audio-video operation and the audio-video playback were
discussed. The value of honest and accurate ratings was em-
phasized, as well as he independent scoring of each dimension
according to category definitions.

1"“

inc pilot use of the A-V Scale for practice ratings of
audio-‘ideo counseling tapes was executed.under supervision,

and the results of these practice ratings were discussed. This
clarified the meaning of category definitions and was very
helpful in develOping a uniform standard among all the judges.
Counselor responses illustrative of all the scale category
behaviors were observed and rated independently by the judges.
Their ratings were then compared, followed by a discussion over
differences in ratings, dimension definitions, scale use, and
important cues used in scoring. A tabulation of the answers to
some of the most frequently occuring rating problems that
required clarification by the judges, including some of the most
camaonly shed questions about the scoring of dimension categories,

is oresented in Appendix D base 126.
1 ) l, l.)

A4.

F

J‘th this kind of training it was expected that judges would
tend to develop a CQEHOH frame of reference and understanding of
the scale dimensions and use of the scale. This traininy, provided

.J

the jué'es have a counseling background and have thoroughly studied

r,
~ _7/)_

..

the dimension category definitions and illustrations, requires
approximately one to two hours. It is expected that hfs judge

trainins is an integral part of the use of the A-V Scale.

Q -v

Procedure

The audio-video tapes of the ten counselors were presented
to the three judges, in nixed order, by audio-video replay.
Twenty respons " from the latter part of each tape were rated.
Each counselor response was identified by number and then rated
independently by the judges. The replay of the tapes was
controlled by a stop—start switch, which was used to step the
replay of any tape whenever a judge felt that the counselor

P

responses were too rap‘d for adequate handling.

Desipn
For each counselor response rated cognitive, nonunderstanding,
nonspecific, or nonexploratory, a score of l was assigned to the
respective category. For each response rated affect, understanding,
specific, or exploratory, a score of 2 was assigned to the
respective category. For each of the Affect-Cognitive, Understanding-
Nonunderstanding, Specific-Nonspecific, and ExploratoryaNonexploratory
dimensions, the total dimension score was the sum of its two
category scores. Each counselor response was rated either 1, 2,
3, or 4 on the Effective-Honeffective dimension. The total score
for this dimension was the sum of the l, 2, 3, and k ratings.
These were the scores used for making statistical comparisons.
All rating scores were averaged across the three judges to

give one rating per response for each of the five dimensions of

f
-09—

the scale. The total score on each dimension f or the ILA. candidates
was corizparcd to the corresponding total dimension score for the
Ph.D. candidates. Significant differences, based upon "UT—tests,
were detentined at the .05 level of confidence.

Indices of item interjudge reliability and internal consis-
tency were also established by applying Hoyt's analysis of variance
method to the unaveraged rating scores for each individual

counselor response ,

Ifethodolgjr 2;; the Objectivity Study

This study tested the hypothesis that the A-V Scale will
have adequate objectivity for differentiating between ILA.
and Ph.D. candidate responses on each of its five dimensions.
The me’lfllodologr designed to test this hypothesis is presented
here under the headings of Counselor Subjects, Judges, Procedure,

and Design.

Counselor Subjects

Fifty-three counselors were observed in this study. Forty-
five of the counselors were ILA. candidates (1 through #5) under-
going their initial counseling practicum experiences. Eight were
Ph.D. candidates (1+6 through 53) undergoing their initial doctoral
practicum work. These 53 counselors had an initial personal-
social type interview with the same coached client. These interwieI-rs
were previously audio-video taped under controlled conditions as

S Q r 1
part of another research study (hard, 1965).

lThis study is reviewgd in Chapter II, pages 30-31.

1

J11le"0.

\r

Three judges rated the video tapes of these 53 counselors.
One judge was near completiOn of his Ph.D. in Counseling and
Guidance; one was a 1965 hichigan State University NDEA
Guidance and Counseling Institute graduate; and one was a Master's
graduate in Guidance and Counseling. These judges had not seen
any of the 53 counselor tapes prior to the rating session, nor
did they know to which of Ward's three experimental conditions
these counselors had been assigned. The three judges were
thorough 1 fariliar with the dimensions of the A-V Scale. They
had received training in the use of the Scale with audio-video
tapes as described under judge training in the previous section,

pages 57-590

Procedure

The second half four minute selected segments of ward's
53 video counseling session tapes were presented by video-replay
to the three judges in the same random order which they had first
been presented to the three judges for rating in'Ward's study.
All counselor verbal responses of each four minute segment were
rated independently by the judges using the Scale. The video
replay of the 53 counseling tapes was controlled by a stop-start
switch, which was used to step the replay of any tape whenever
a judge felt that th audio-video counselor responses were too

rapid for adequate handling.

Des1rn

For each response rated cognitive, nonunderstanding, nonspecific,

/

-l (7‘)“,-
I

O

or nonexploratozg' a score of l was assumed to the respective
category. For each response rated affect, understanding, specific,
or e:c1fl.orately, a score of 2 I-ras assigned to the respective category.
Each counselor response ms rated either 1, 2, 3, or 1+ on the

Sf f e ctive-Nonef f e ctive dimension .

Heyt's ana- ‘ sis of variance method was applied to the rating
scores for computation of interjudgo reliability (objectivity)
ane izrterml consistency (mm-imam lil-celiheed of reliability).1
Two different amflyses were made. One was an item analysis of
the individual counselor responses as rated by the three judges
on each of the A-V Scale's five dimensions. The other was a
global rater analysis based upon the total score obtained by
each counselor on each of the scale's five dimensions. The
item objectivity was ccmputed from the actual scores for each
counselor response, while the global objectivity was computed
from proportionate scores based upon the total cognitive,
nemnderstanding, nonspecific, and nonexploratery category scores
for each commeler.2 The scores for each of these categories
were totaled, then divided by the number of. counselor reSponses
rated, and then multiplied by ten to give the proportionate score
for each category. Proportionate scores for the Effective-

I~Ioneffective dimensions were computed in a similar manner.

1 -
Fonzulae for heft's method are presented in Appendix F, page 139,.

2Because the counselors in Ward’s study differed in their rate
of verbal responding to the client, the number of responses
available on each counselor's four minute tape segment was not
necessarily equal. There was a small range. Therefore, to
equate for this discrepancy so that the total scores for, each
counselor could be compared statistically, proportionate scores
were used.

/r‘\

-~) )-
i

'71"
-b

no total proportionate scores for the Effective-fieneffective
dimension and the four categories mentioned were also averaged
across the three judges. These scores were used to compare the
K.A. candidates with the Ph.D. candidates. Significant differences,
based upon “t”-tests, were determined at the .05 level of con-

fidence.

 

Yethodelogy'2£_the Sensitivity Study

This study'tested the hypothesis that counselor trainees who
receive professional supervision which focuses on examining
affective, understanding, specific, and exploratory responses will
receive higher frequency ratings for these categories than will
counselor trainees who do not receive this professional super-
visiez. The methodology for testing this hypothesis is presented

here under the headings of Counselor Subjects, Judges, Procedure,

and Design.

Counselor Subjects

The counselor subjects were ten of the thirty National Defense
Education Act HDEA) Counseling and Guidance Institute enrollees
attending Eichigan State University'for the 1964-65 year who had
volunteered for this study. The subjects were randomly assigned
to two groups; five to a professionally Supervised Group (A, B, C,
D, and E) and five to a hensupervised Group (F, G, H, I, and J).
These 3 EA Institute enrollees were assumed to have had similar

intelligence, education, and high school counseling experience,

as well as identical K‘E; nstitute instruction and group therapy

(I.
-L)~r—

class ezq1erience during the 19613-65 year of Institute training

at t11e1Uriversi t3 '

Three jud‘;es 1: ere orployed to rate the video tapes of these
ten counselor enrollees. One judge had a I'astor's degr- e in
Counseling; and Guidance and t-1e other two were near cozzpletion
of the Ph.D. degree in Counseling and Guidance. The three
judges had no knowledge as to which of the two groups--Supervisod
or lTonsupervis od--' th e counselors they were rating belonged. The

three judges -1: d received training in the use of the A-V Scale

..2. 4-3 who . . 1 .:.
\.J_ a v9.L-\'lo-V'1UOO p.110

(.1

as described under jud e training in

the first section of this ch aptcl‘ , pages 57-5/-

1 roceaure
The“; counselor 1::e.:ber in ..)oth groups underwent six separate,
tl‘i1 jur‘inute “die-vi deo taprd counseling interviews with
volunteer high school and college stu:.ont clie 11st assigned to the;
on a rand-e: basis frozs: an available pool. Zer-izors of the Zen.
sup mised Cred. 1 w ricod in two-nan counselor-client interrogator
teams. Daring any given counseling session, one team 1‘:,;:bor
would serve as the counselor with the client and the other team.
111e1‘1ber foul? 1. late serve as the interrogator with the client.
Hiring the next counseling session the two tear: nem‘oe s rould
ozzehang e roles. Over the six sessions each team member served
three times as the counselor and 'Etrree tires as the dim

intorrogator. The fifth ;:e;:‘oer of the I‘Tonsuporvised C-rov p alternated

with an Institute 21011110;e not participating in the study. Each

LToz1su;)o:.'v.'1-sod"-.~1 ouo counseling interview I. as followed by a one
‘ 1' ~4‘ °--“ -~-- ° (V av. M
nour clieao-inccm ogation sessioh.
Iic;::‘oers of ‘he Supervised Group wor‘.:od independently of
one another and had e. on of their coun seling interviews followed

D

by a 110 hour counselor11terrogetion session. One stair? :‘-:e:".‘oer,
1:11.11 :3. cl:°:.;ical be ckgroun , was the ccxurselor-interroga’r :
(mpervisor) for all five trainees following each of their six
video taped counseling interviews.

Throughout each Ionsupervised Group counseling interview, he
ate“ tea.» :'-'.e1‘:1‘eer observed and listened f roz'z
behind a one wry :irror. Following this, the client was interro-
the video r;e)lay of the interview while the counselor
tear: nether observed and listened to the cliez 1t-rntcrroga ation
from ‘oelind the one Ira . :irrer . Throuj1out the audio-video
recall, the c.ie11tinterrorator tom 1 ..e.1‘oer, by his cuesti ens ana
contents, con: ‘ucted the recall session in a manner he felt was

.1.

I’LOC; ' appropriate for an interrogator. During this client-recall

session, "he audio-video replay was controlled by a stop-start
switch, and either the client or the client-interrogator tea)“

veneer could Stop the replay at any point he Inshed in order

: a (rues tion.
7.111i11f‘ the counseling in rterviews ‘oetI Teen a cli enta nd a
on 101 'isec‘ {roe 1 counselor .1: 1.1‘oer, the supervisor observed ans.
121.1» so Ired to tlis session fros-z behind the one way mirror.
he interview, the counselor, rather than the client, was
a a

i1.terr ;_;c.'1f.ee. oy the supervisor during the audio-video repday

of the counseling session. Either the counselor or supervisor

I x

-‘_J-'_ 1—

could step the audio-video replag),r at any point he wished to

'1: a question or pulse a com-tent.

The genera procedure followed by the supervisor was to

pursue

l.

'\
1
0

k1.)
o

1'1

5.

1

r-"aduc1.lly a-1o. subtle;r the following with the counselor:

The extent to which the counselor felt he was dealing
with the affective concerns of the client.

The extent to wlrich the counselor felt he was dealing
with the real and specific concerns of the client rather
than superficial considerations.

The extent to which the counselor felt he was communicating
to the client the fact that he understood the real

concerns of the client.

The e:-.cent to which the counselor pennitted and encouraged
the client to e:1:plore his real concerns.

1.11e counselor's opinion of the effect that a lack of

1.110 behaviors in 1 through 1+ on his part would have upon
the client and their counseling progress.

The counselor's opinion of the effect that an emphasis

of the behaviors in 1 through 1+ by him would have upon

U

the client and their counseling pr eress.

.L

11cm1ragez1on1. of the counselor to consider the possible
counseling use of these types of behaviors in his subsequent

.9- _.. - o --. -‘OLj‘ _o ‘3- .
1.11.1.0. 1:10.11. ..-.11.11 clients.

'1‘.“ J-’ V o g

..1 these behaviors '-;.‘ere ea.11n1as:1.zed in ts1e :mbsequent

'J.

.0 . J. .; 0 ‘ v(‘ -’ V . ‘ - _.‘.o _o . 1 (‘ _..
.1--11,c1*vie.1s b1 the counselor, an e.-r..1.1.nat.1.o.1 of the ceII:1selo.1's

U

1- .'

evaluation of their effectiveness 1' or the client was made.

The counselor's effect on clients when he lacked those

behaviors was compared with his effect on clients when

he c:g;phasised these behaviors. The counselor could

observe on audio-video tape the effect that these behaviors,
or the lack of these behaviors, had on the client.

Des d upon these observations, he was then left to

judge for hinself the value these behaviors had for
cmnseling.

Upon coz'zpletion of the six counseling sessions by the 1=1e121bers
of both the lionsupervised and Supervised Groups, the riddle
twenty,r responses of the fi “st and sixth sessions 1‘ or each
counselor :‘.:er-.ber Herc replayed and rated using; the A-V Scale.
There was, then, a before and after comparison for each counselor
of both the Zfonsupervised and the Supervised Groups. The first

sess:Lon rated for both groups was an initial interview with a

client. The sixth session rated for both groups of counselors

was the second consecutive interview with their last client.
The ratings for the initial and terminal interviews were

made over two different rating sessions; one was held for the

initial interviews and one was held for the terzrinal interviews.

' Design

For each counselor response rated affect, unders ending,
specific, and e:~:1fl.oratory, a score of l was assigned to the
respective category. On the Effiective-Zioneffoctive dimension,
each counselor response was rated eith )r l, 2, 3, or 3%. These

were the scores used for 1,:aling; statistical coztparisons. Thus,

the higher ".310 score on each (linensiou, bile greater the indication

/(‘\
-...| I-

that a cow.u'1selor's responses had bee 1 rated affective, under nding,
>ecific, explorato 'y, and effective.

ra'inQJs were avora red across the three jud'res to give

one rating per response for the Effective-Ebneffective diir1ension
and each of the four categories mentioned. The initial and
ten~1inal ratings for both the Supervised and Nonalpervised Groups
were cozfzpared for siQLificant differences on all five dimensions

of the A~V Scale. The dependent "t"-test was the statistic used

JW‘

for: 12.1:111 3 131 1s comparison at the .05 level of confidence.

Summary

The no thodologies of the Scale Disczirfination Study, the
Objec 1.°vit3, 3111'? , and the Sensitivity Study were discussed in
this chapter. These three studies tested the three hypotheses
set forth in Chapter I.-

The 1:1ethodolegy of the Scale Discrimimtion S‘bidy was designed
to test the hypothesis that the frequency of affective, understanding,
specific, exploratory and effective responses in initial personal-
social problem counseling interviews will be significantly rrreater
for Ph.D. candidates than for I..A. candidates ber'inning their
practicum e:q>eriences. Three trained counselor juwes used the
A-V Scale to rate 20 responses from each‘of ten 30 minute audio-
video 1.2 .ped co1.;1seli11g interview sessions. These ten taped counseling;
sessions involved five 11.. . degree candidates and five Ph.D.
candida tes 0:131.ng in personal-social probl e.1 interviews with

clients The rat'ng scores of both groups were coar1pared on the

five (L1? .01 ions of the A-'.' Scale by means of ”t”-tests. Heyt's

analys s f variance 1. cthod '1 'as .121 0 applied. to the ratinr scores
to establisn an ind-0:: of iten objectivity.

The Iet 1odolor;y of the Oojectivity Stu :13] w .2. designed to
test the 11;:101.th3.8 that the 311-! Scale will have adequate objectivity
for Ch..1.1.0“8‘.’11.--.-t1LnU between ‘12.!1. and Ph.D. candidate responses
011 each of its five dj.1:1.s1e1 ion .3. Three trained counselor judges
used. the 1‘.—‘1' Scale to rate four minute segm- nts from each of 53
audio-video couns cling tapes. Forty-five of these tapes involved
12.11. candidates and eight involved Ph.D. candidates. All 53
counselors were engaged in 21 i1r'1 1.3al intervicw with the same
coached client. Hero's analysis of variance was applied to the
rating; scores to establish two sets of objectivity and internal
consistency coefficients. One set was an index of item objectivity
and internal consistency and the other a global rater objectivity
and internal consistency. The rating scores of the 21.11. candidates
were co-zzpared Ui th those of the Ph.D. candidates on the five
dimensions of the A-V Scale by means of "t -tests.

The methodology of the Sensitivity Study was designed to
test the hypothesis t‘ at counselor trainees who receive professional
supervision '1I1I121- ch focuses on examining affective, understanding,
specific, ande ::11'L0ratory responses wil 1 receive higher
frequency ratings for these categories on the A-V Scale than
Iill c0111.." lor trainees 1.7110 do not receive this professional
training. Ten I1DEA Gui (121100 and Counseling: Institute trainees
partic:'1.p.1.ted in the study. Five of thes e counselor trainees

belonged to 2. Supervised Group which received profe ..::°101-12.l interro-

ga. on .1; :.;ea;1s of 2.11dio-v1deo time re 112.3; of their counseling

interviews. These interrovation sessions evaluated the effect that
affective,'understanding, specific, and xfiloratory responses

woulc have on the counselor trainee's clients. This group received
six of those interrogation sessions. The other five counselor
trainees belonged to a Eonsupervised Group who also were exoosed

to the audio-video tape replay of their own counseling sessions.
However, their interrogation sessions were carried out with the
Client, rather thanmthenselvcs, by a colleague team nefiber.

These interrogation sessions pursued no particular emphasis

during the audio-video replay. The pre and post tapes of the

five trainees belonging to each group were rated by three counselor
judgeS'using the A-V Scale. For each group, the rating scores of

the pre and post tapes were statistically conpared.using dependent

”t "t-tC Sts .

..v

o‘

The neonats of the:

Chapter IV are repo; 130d here

three sections; '1 discus

Discri":'Lnation S'bidy, a discu

I " ~v-f‘ov—a‘

_ -.. 41.4-1.1.1“)

sion 01 the rcsu

now-‘7 mqfi~j
‘1-—..4 L4- ‘1

1r

'3'“wa mr“
Luddabu

("'1

VJ.

ezthod ole. ice.-- procedures described in

J‘ J

'11.

of the Scale

ssion of the romlts of the Objec-

 

tivity Study, and a (U. sous on of the results of the Sens trusty
301103“.
Results of the Scale Discrinination Studg-Jr
The hypothesis tested by the Scale mscriifizmtion Study was

that the frequency of affect,
and effective responses
than for 7.2.5.. candidates.

which woul CL r01. ”lect on the so

next section.

will be greater

understanding, speCific, exploratory

for Ph.D. candidates

1lso, objec"c.ivit3r data 1' s collected
eond hypothesis discussed in the

 

Anajsis of Sea- le mSCl'lI tinat ion Data
Table 5.1, page 72,511.111r1rizes the statistical "t" amiys's
of Appendix: E, p. 132. The analysis presented in this appendix
compares, on 00 ch 0-“ the five dimensions of the A-V Scale, the
scale see; 1 s of the five Ph.D. candidates with those of th
\
11.91. cant’aeates. it can be seen from inspection of this Table

-w .l. .0“. . >0 ‘ ‘3‘ ,L: ‘ I ._
that t.- 1e.‘o is a spa c.1_st:1.eall‘;
(51.10 :zio:1s 1 steel one scores
a J,“ ' l‘ r - '1... '1 ’.‘ I. {’1‘.‘
01. the . ._-.. 01111.1.(L1tes. 11-0

significant difference on all five
of the Ph.D. candidates and these
responses of the Ph.D. candidates

I“

-70-

were nere frequently rated affect, understanding, specific,
exploratory, and effective than were the responses of fine
L.A. candidates.

Table 5.1 -- Camparison of the Kean (X) Scores Between

the Ph.D. and X.A. Candidates on Each of
the Five Dimensions of the A-V Scale

" - Y! A _ ‘ ‘Ar'\
(3 Judges, 5 Ph.D. counselors, 5 h.a. counselo-s/

 

Aff.- Underst.- Spec.- Exnl.- Eff.-
Cogg . l'onunderst . lTonspec . Hone

 

r’n

P:1.D. J

35-2 3&2 35.4 34.11 58.8

1.0410 X 2.3—0? 23-00 20.2 20.2 37.].

t _ ratio 5.9** 13.4** 7.nx* v 5*» 5.9**

 

necessary: t .05 = 2.31
necessary: t .91 = 3.36 **significant at .01 level

Aff.-Cog. = Affect-Cognitive

v

Undorst.—Tonunde:st. = bnderstan‘ing-Ionunderstandin;

ll

3pcc.-Tonsoec. Specific-Ionspecific

'IL...
.2. 7

l . -'. $011022} 11 . = Ezploratommlionexgfloratory

I
-
‘

Eff.-Ioncff. = H fective-honeffective

These findings support the hypothesis tested. They suggest
that the dizcnsions of the A-V Scale are able to discininate
between the responses of these two classes of counselors. These

results, then, indicate that the scale has validity for this purpose.

fauljsis 0-; lnterjgdge Reliabilit" Data

noyt's analysis of v.riance method for estinating reliabi_ity

was applied to the scores of the five Ph.D. candidates and the five

‘. l -.n "\'I‘ A‘L '3 a: - "1 . a '1‘ Q— m, ‘ Q r‘ - 1‘ \' 4‘. ~A 1
11.1.. 051.1... ’ .uCS l‘c. uCU. 4.1."; lat-LI) Solid-u, . .L..lO Oil-ta all t CO. L11.tC. LIV-LL

_ ~. -J‘) .3, .v.1”o ‘ J. J-‘ o n . '1-- o o ‘ -- ,: 0
1113-1 ‘...~'-C.. 1,-.1-1. SUE-L113 1c.l zealdsis 157130.306 are presented in

O ‘ U

..a-.1 , . 1- - . . ° .2. .' ..1 -- . '1.-.. 1.,.-.1..2. ...--
“3901-1-1.“ F, 3. 130. .-1e 1.1.111 llluOl"Jllk.‘.i_;G 1elaz1- 01.1111”. (oe IeCL. 1:31.)

and. itez: 3419013211 consiste: 1c, coefficients catputed 3;; t.1-'1.s

. n1,_‘_0 ‘ - _ n _‘ 1 n J_1 _po. 1. _
.11111-5111-3 arc :e1ortec‘. in Table 5.2 or eacn 01 1,110 -ILJO oi. on: 1.111s

the -‘.-'1" . Scale .

out r A

- .1,->--c d... -- Ite;- Intern-16150 Deli-'1 oilit " nc- Interzwl
eons-.stencJ Coefficients Cor.a 1 to c. fro:
the Scores of the Scale lecrizixmtion Stir-Ty

\.,/

k3 jmlg; cs, 10 counselors

.i‘f.- 'Jnderst.- Spec.- Expl... :f.-
00;. ITenunderst. Ions sec. Honor-“pl.

I; 1te‘.‘d'ud{_;e
hells-Lilli); .792 .816 .817 .753 763

. ens:- stenc; . 919 . 930 .929 .903 .906

 

These rez‘mlts suggest that perhaps the dimensions of the
11’

31-. Scale have suite favorable objectivity and reliability

l o
Q‘21‘CCUVLZE1D'P‘ .

lesfl-ts f the glijectivitl at 11‘"

 

 

o .1. c \ N g

‘L‘he h;‘ goth-cs l .1 tested 13;; the Objectivity Study was that the

a. - ficn-ln 1" 1'.1 all "“.‘Il‘l‘ .2"- ‘9‘.‘ .1317 ‘0 "
-.- g L) [‘1.-V -1-—¢.- ALHV C u“~0&;v“v~ U0 0 JJOC U1 n J J. or

7 .L

’1 1ere1t~acin~
between ZS.-‘.. and. $11.3. candidate r- sponses on each of its five
dizcnsions. ..;0 ob ccti‘vit; Stud; also served as a type of

— ‘ nvfi ‘ "‘-1 ,-—.~ 'r' ‘ '9" - n“ . . ~-~. ~ “‘1‘ fl
cross-...1.-1..-e.1-ac.s.-1-1 r v.10 Scale Discm1.i.1s. ion .1 cues,

-72-

Jnal;si:‘ of Intezjudge Reliability Data
Hoyt's analysis of variance method for estimating reliability

was anal-iod- to the scores f or the f orty-five 17.3.. and eight
Ph.D. ca Iii-dates rated in this study. Two sen. rate sets of
inter judge reliability and internal consistency coefficients
were conjmte-J. by this analvsis. The first set of coefficients
conputed were the iten interjudge reliability and internal
consist no; coefficients based upon a comparison of individual
counselor resuonse ratings. “nose coefficients are reported
in Table 5.3 for each of the five dimensions of the .-V Scale.
The Statistical analysis upon which the computation of these
coefficients is basedzis presented in A3);>endix G, page 165.
.. a -- Iter- Interju dgge Reliability and Item Internal

Consistency Coefficients Computed from the

Scores of the Objectivity Study
(3 judges, 53 counselors)

 

 

Affu- Underst.- Spec.- E:Cpl.- Eff.-
Cog. Lionunderst. lionspec. l-Ionexpl. lionef f .

Inter j'u dfje
«chant ‘12; .5390 .792 . 763 . 590 .8124.

Internal

Consistency .961 .919 .906 .312 .92

She seco 1d se ‘ of coefficients computed U ere the global rater
interjudgc reliabiliy and internal consistency coefficients based
upon a conparison of the total dimension scores for each counselor
rated. Chose coefficients are reported in Table 5.@, page 75,

for each of the five dimensions of the i-V Scale. The statistical

-75-

analysis upon."hich the COHputation of these coefficients is
based is presented in Appendix H, page 206. The internal
cons istenc; coefficients reported refer to the naiimum likeli-
hood estimate of reliability.
Table 5.M - Global Rater lnterjudge Reliability and Internal
Consistency Coefficients Computed from the

Seer s of the ijectit'ty Study
; Judges, 53 counselors)

 

 

 

iff.- Underst.- Spec.- Expl.- Eff.-
Cog. Ionunderst. Ionspec. YONGKDl- Ioneff.
Interjuaje
10113.2 . . 353. . 3:1 .335 . 697 .934
:anC r ‘16.].
Consistenc: ,th .903 .,59 .373 .977

 

 

It can be seen unen inspection of Table 5.3 that the item
interrud;;e relia oiliw .nd the maximum estimate of item reliability
(internal consistency) coefficients are rather high for a counselor
rati.g scale. Although loner, these coefficients compare favorably
with those obtained from the ratings of the Scale Discrimination

o I,

Study'reported in Table 5.2. The results reported in Ta ale 5.w
indicate that thee e coo:fic:m nts are lligher "hen the total or
glooal scores for each dirension are compared.

nese findings support the hypothesis tested. They suggest
that the lizensions of the L-V Scale have favorable objectivity
for the rating of Ph.D. and X.A. candidate reSponses. This hi Mr-

Objecti'ic” is accompanied, as :‘ould be er)ected, by a hiSh

Irujsit1c>s ui4r to o: reliability.

l-
\J

finalisis f Jata Cogparins Ph.D. with X.A. Candidates
Table 5.5 summarizes the statistical “t” analysis of
1“» ”H .,. " .4...“ P1" _ .Q -84“ . F .2. ,1 a t“ :1 c‘ .h‘.‘r
_gppenci. -L, e 217. 1-10 011 ,y sis presentet 1n .1...) appenoa-..
compares the proportionate scores of the eight Ph.D. candidates
:ith those of the forty-five I.A. candidates on the Cognitive,
-2'onunderstann‘ing;, Nonspecific, a nd Nonexploratory categories of
the 1—7 Scale. A co: :parison of the proportionate scores of the
Lifective-Koneffective dimension is also made between the two
classes oi counselors.
-aele 5.5 -- COMPCTiSOH of the Kean (X) Scores Between the
$11.3. and 11.31." at -10 1'da ates on the Cognitive,
Nonunderstanding, Ionspecific, and Ionexploratorv
Categories, and Effective- onoffective

binension, of the A-V Scale
(3 Judges, 8 Ph.D. counselors, 45 7.3. counselors)

Eff.-

 

 

 

Cog. Uonunderst. Honspec. Honexpl. Noneff.
311.0. 3; 3.8 5.3 7.0 21.1.9
,1 -: ’3 O ‘3. / 9 l 2
1.0.0. as do.) 70/ vex) 90k 50
t - ratio 3.5** 14.l** ll.8** 8.5** h.9**
necessary: t .?5 4 2.00
necessarf: t .Ol = 2.66 *asi'nificant at .01 level

Inspection of Table 5.5 discloses that there is a statisticallv
signifi ant difference at the .01 level between the scores of the
%1 D. candioct es and the scores of the K.A. candidates on all
five scale dimensions. The responses of the I.A. candidates were
3

more frequentlv'ratcu cognitive, nonuneerstanding, nonspecific,

nonexploratozy and less effective than were the responses of the

-77-

Ph.D. candidates. These results support the finc‘inr's of the

x.)

Scale 7 .):_sc i‘111ation Study reported in Table 5.1, and again
L

10 five «’51.: 1e11sions of the 1’1—V Scale are able to

discrbimte bettreen the responses of 23A. and Ph.D. candidates.

Results of the Sens'tivi v Stud;

 

 

The Sensitivity Study testeo the hypothesis that counselor

trainees 17;: receive 111110 ‘essional s:u)ervi sion .rhich focuses on

1

53125111115 affective, unnerstandinb, specific, and exploratory
responses trill receive higher frequency ratings for these categories
on the -‘-.-'."S ‘cale than will counselor trainees v.10 do not receive
at. s are; es siornl training.

Sable 5.6 reports the i1fitial.eans of the Lionsu >erv1 sed
Group 11th t_1e initial means of the Supervised Group on the five

‘0 l '1‘

eigensions oi the A-! Scale before interrogation sessions.

1a‘olo 5.6 -- Iloan (3'?) Scores of the Initial Interview Rating

for both the lfonmpervised and Supervised Groups
the Five Dimensions of the A-‘J Scale

(3 judges, 10 counselors)

 

.f f . - U11d.erst.- Sp ec.- ficpl .- E’.‘ .-
Cog;. lionunderst. 1 'son 1300. 101102-2131. Zioneff .
-fons‘ 1 10”"! so: .2 1.2 13.2 .3 3.’% 27.?

Taole 5.7,:1age 73, margarizes the statistical comparisons of‘
4-1 -,_.;.!-- '1 ..., 1 ,. _m a 4.1..» (M. b " ‘u.
14-10 1-1.1.10.-- _-eaa scores Cl LLO IT:on :upervis ed 11. cup 01010 n n-

supe‘misod client interrogation with the tenzinal mean scores of

-78-

05810115.) .

o
D
(.7

_9‘1 9 _.. 1"- 5.,“ ' _1 a ‘~f' -L' ‘_ «L . ‘ 0
this are”; 14119511“ the g1ons u)ervised clienti :1te1rogatio.

N

The data axe conputations upon which this analysi

L1
F! n
m
0‘
,1:
m
(D
Q.
’1)
“3
CD

prese nLeCZ in Anuendix J, page 228.

'1‘,hle 5.7 -- ConDarison of the Kean Scores of the Initial
and Terminal Interview Ratings for the
‘Jonsupervi sed (Client-Interrogation) Group
on the Five Dimensions of the A-V Scale
(3 judges, 5 counselors)

 

 

Aff.- Underst.- Spec.- Expl.- Eff.-

Cog. Ionunderst. Honspec. Ionexpl. Ioneff.
Llicial '2 1.2 3.2 .3 3.4 27.2
Toninal ‘7 1."; 2.4; 1.6 2.0 25.3
t - rntio 2.5 2.2

 

necessary: t .05 = 2.73

The comrarison of the mean scores of the initial and terminal
ratings of Table 5.7 show that there were no significant changes

on any of the five dinonsions of the A-V Scale. Since the means

for the Understanding-Tonunderstending, Exfflol atory; onexploratory,

'1)

.nd Effective-Soneffective dixensions decreased rather than
increased iron the initial to the terminal ra.tings o, no t-tests
were neeesSarf. It was apparent that these three dimension scores
had not chanfied in the nocessarv directioz.

AAJHO 5.3, page 7?, summarizes tae statistical conparison of
the initial :ean scores of the Supervised Group before supervised

-‘

counselor into: rogation with the terminal mean scores of this
group folloz'ng the supervised counselor interrogation sessions

The data and co:1nutations upon thich this analysi sis based are

-79..

presented in A} m aniX II, 1X1? 0 232-

Table 5.8 .- Conparison of Ilean Scores of the Initial
and Ter‘timl Interview Ratings for the
Supervised (Counselor-Interrogation) Group
on the Five extensions of the EI-V Scale
(3 judges, 5 counselors)

 

 

Aff.- Underst.- Spec.- Expl.- Eff.-
Cog. , Ronundel st. Nonspoc. Nonexpl. Honeff.

 

Initial I 2.6 6.8 3.4 7.2 31.0

 

Tonia-1a]. Li .2 4.0 14.6 16.6 51.0
t - ratio 3.3? 3.3” 6.l** 1.5.6“ 111.21'
-leCOSSC-ll‘ff: t .05 = 2.73 *dmificant at .05 level
necessary: t .Ol = 1.5.60 MSiSnificant at .01 level

The comparison of the mean scores of the initial and tcniinal

fl r1“

8 0:; mole .85 shot: that there were stati sti cally significant

.0

changes on each O.‘. the five dimensions of the .»\-V Scale. :3.

co: :;_>arison of thes fi ICLings I'ith those reuorted in Table 5.7
suggest that the supervised counselor interrogation Irith counselor
trainees, Irhica focuses on o:ca::°1.:1ing affective, understanding,
specific, and ez-qfloratom' counselor responses, incre-.ses sigzri-
ficantly the froquency of counselor trainee responses I-lilich are

rated affective, understanding, specific, and er .1loratory. These

ed.

(4‘

findizrs "u”:uort the 1‘1'not1.esis tes

'1
“\‘1\~‘QW'P

N’s.) L. .L-L‘
—

rue results of the Scale Discrimination Study, the Objectivity

StucL, a nd tne Sensitivity Study are reported in this chapter.

The findings of the Scale Discrimination Study suggest that
the five dimensions of the A-V Scale significantly discriminated
between the responses of Ph.D. candidates and those of M.A.
candidates. The responses of the Ph.D. candidates were more
frequoztly rated affect, understanding, specific, exploratory,
and effective than were the responses of the K.A. candidates.

The results also suggest favorable objectivity for the A-V Scale
in making these discriminations. Item interjudge reliability
coefficients of .72, .82, .82, .76, and .76, and item internal
consistency coefficients of .92, .93, .93, .90, and .91, were
Obtained for the Affect-Cognitive, Understanding-Honunderstanding,
Specific-Zionspecific, Exploratory-Nonexploratoxy, and Effective-
honeffective dimensions respectively.

The findings of the Objectivity Study suggest that the
A-V Scale has.favordhle objectivity, accompanied by favorable
11121111111111 reliability estimates, for the rating of Ph.D. and M.A.
candidate responses. Item interjudge reliability (objectivity)
coefficients of .89, .79, .76, .59, and .81, and global rater
interjudge reliability coefficients of .95, .91, .89, .70,
and .93 were computed for the Affect-Cognitive, Understanding-
1~§onunderstanding , Sp emifi c-I’onspecifi c , {1:201 omtory-IIonexfloratory,
and Effective-Ioneffective A-V Scale dimensions respectively.

For these same scale dimensions, item internal consistency

(maximum est'uate of reliability) coefficients of .9 , .92,

.91, .81, and .93 respectively, and global rater internal consistency
coefficients of .98, .97, .96, .87, and .98 respectively were

computed. The findings of this study also support the findings

(N '1
—c1-

of the Scale Discrimination Study that the A-V Scale significantly
discmtdnatcs between the responses of Ph.D. candidates and those
of ILA. candidates.

The findings of the Sensitivity Study suggest that supervised
counselor interrogation with counselor trainees, which focuses
on errandning affective, understanding, specific, and ezqnloratoxy
counselor responses, increases significantly the frequency of
counselor trainee responses which are rated affective, understanding,

specific, and exploratory.

CHAPTER VI

SUI'I'XRIL, CONCLUSIONS HID II-ZPlICATIOZTS

This chapter consists of three sections; a summary of this
research, the conclusions of this research, and.a discussion of

the implications generated by this research.

Sunnarv

This research examined three hypotheses associated with
the development of an Audio-Visual Counselor Behavior Scale
(A-V Scale). This A-V Scale was developed for the purpose of
making a. vai-11ble an objective instrument which would compare
the res pens patterns efI aster's degree candidates in Counseling
and Guidance (K.A. candidates) with the response patterns of
doctoral degree candidates in Counseling and Guidance (Ph.D.
candidates). Its development was based upon the clinical
observation of counseling behaviors on audio-video tapes
followed by a categorization of these behaviors according to
constructs clinically evaluated to be relevant and descriptive
of H.A. and Ph.D. candidates. The A-V Scale consists of five
forced choice, dichotomous dimensions. These dimensions are
the If feet-Cognitive, Understanding-Nonunders t: nding, Speci fic-
honspeeific, 111lorat01 - one: :ploratory, and Effective-Noneffective
(131821"? ens .

The three hypotheses examined by t}1is research are:

Firs t I guxat es1s

The frequency of affective, undersea 1nding, specific, exploratoflfi

nd effective responses in initial personal-social problem

counseling interviews

L1

as measured by the A-V Scale, will be

9

C1
-Jo
)
..J
H)
H.
0
P3
:3
Ca
IJ
\"

gseator for Ph.D. candilates than it will be for
M.A. candiiates beginning their practicum experiences in Counseling
and Guidance.
Second Hypothesis

The i—? Scale will have adequate objectivity for differenti-
ating between Ph.D. and K.A. candidate responses on each of its
five dimensions.

(1" o ~1_" _L1 ‘3 ‘
1h1rd.1n;x)miosis

 

Counselor trainees who receive professional supervision
which focuses on examining affective, understanding, specific,
and exploratory counselor responses will receive higher frequency
ratings for these categories on the A-V Scale than will counselor
trainees who do not receive this professional training.

The Scale Discrimination Study tested the first hypothesis.
For this study, three trained counselor judges used the A-V
Scale to independently rate 20 responses from each of ten
30 minute audio-video taped counseling sessions. Five of the
counselors engaged in these interviews were M.A. candidates.

The other five counselors engaged in these interviews were Ph.D.
candidates. Each of the ten interviews J38 an initial personal-
social problem interview with a client. The judges had not seen
the tapes prior to the rating session, nor did the; know the
counselors they were to rate.

The rating scores obtained by the two classes of counselors

on each of the five dimensions of the A-V 3cale were statisticallv

eoapared using ”t -tests. There were significant differences
in the respons scores of the two classes of counselors on all
five dimensions of the A-V Scale. The responses of the Ph.D.
cmxdidates were more frequently rated affect, understanding,
specific, exploratory, and effective than were the reSpenses‘of
the M.A. candidates. Hoyt's analysis of variance method was
applied to the rating scores. Item interjudge reliability
coefficients of .79, .82, .82, .76, and .76 were computed for
the iffect-COgnitive,‘Understanding-Ionunderstanding, Specific-
Nonspecific, Exploratory-Honexploratery, and Effective-Noneffective
dimensions respectively. For the same dimensions, item internal
consistency coefficients of .92, .93, .93, .90, and .91 respectiVely
Jere obtained.

The Objectivity Study tested the second hypothesis. For
this study, three trained counselor judges used the A-V Scale
to independently rate four minute segments from each of 53
audio-video counseling tapes. Fortybfive of these tapes involved
K.A. candidates and eight involved Ph.D. candidates. All 53
counselors were engaged in an initial interview with the same
coached client. The judges had not seen the tapes prior to
the rating session, nor did they know the counselors they were

to rate.

Hoyt's arnlysis of variance method was applied to the

l

item objectivity and internal consistency, and the other was

an index of a global rater Objectivity and internal consistency.

{‘3’

—\7J )—
a

Interjudf;e reliaeilj.ty refers to objectivity while internal
consistency refers to the maximum likelihood estimate of reliability.
Item intorjudge reliability coefficients of .89, .79, .76, .59,
and .Ol, and global rater interjudge reliability coefficients
of .95, .9l, .39, .70, and .93, were computed for the Affect-
Cognitive, Understanding-Nonunderstanding, Specific-honspecific,
Tvnlor.torr-1oaexnloratorv and Bffective-Honeffeetive A-V Scale
dimensions respectively. For these sane scale dimensions, item
internal consistency coefficients of .96, .92, .91, .81, and
.93 respectively, and global ra ater internal consistency coefficients
of .98, .97, .96, .87, and .93 respectively'werec mr‘.puted. The
dimension Hoe es of the fortyAfive N.A. candidates were statis-
tically compared'with those of the Ph.D. candidates by means
of "t -tests. There were significant differences in the response
scores of the two classes of counselors on all five dimensions
of the A-V Scale. These findings supported thos e obtained by
the Scale Discrimination Study which tested the first hypothesis.
The Sensitivity Study tested the third hypothesis. For this
study, ten UDEA Guidance and Counseling Institute trainee
volunteers were randomly assigned to two groups of five; one group
comprised a supervised counselor-interrogation group (Supervised
Group), and the other group comprised a nonsupervi sed client-
interrogation group (Eonsupervised Group). Each member of both
groups undeleent six 30 minute video taped counseling sessions
with a college or high school student. Following each counseling
session 00 ch ne:1ber of the Supervised Group received a one hour

audio-video recall couns 0101-int errogation which evaluated the

out," 11’-

effect that affective, understanding, specific, and exploratory
respons- 3 would have on the counselor tra inee's clients during
the coulseling interview. Following each counselin ng session
each member of the N01asupervised Group either observed through

a one view: .ir ror while a colleague team member interrogated his
client foro one hour, or he interrogated his colleague team

member's client for one hour while his team member Observed the

interro ation. There was no particular emphasis to this client

The middle twenty responses of the first (initial) and
sixth (terminal) counseling sessions for each of the ten counselors
comprising the two groups were rated by three trained counselor
judges. r1hose judges had no knowledge as to which of the two
groups each of the ten counselors belonged. These initial and
terminal ratings within each of the Supervised and Honsupervised
Groups were statistically compared using dependent "t"-tests.
The results of these tests indicate that for the Supervised Group
there was a significant increase in the rating scores, from the
initial to the terminal ratings, on the affect, understanding,
specific, exp101~atory, and effective dimension categories of the
Scale' 3 five dimensions. For'the honsupervised Group there was
no significant score increase, fran'the initial'to the terminal

ratilgs, on these Mi Scale di 1e sion categories.

Conclusion

 

In drawing conclusions based on the findings of this research,

it is necessary to consider certain variables which ight have

limited or influenced these findings. First, there were the
major delimitations outlined for this research in Chapter I.
This refers ‘

I
.9.

)rinarily to the fact that the behaviors studied in
this research were laboratory behaviors and that the number of
counselors Observed in two of the three studies of this research
were relatively'srmdl, Second, there'was the fact that the

)
counselors and judges used in this research were primarily
associated with bflchgan State University}, Third, there was the
possible Operation of special factors which could have influenced
the outcome of tne findings. This would include the operation
of such variables as rater bias, the overtraining of judges,
the convenience and desirability of selected tapes, and the
nonverbal conveyance of rating attitudes resulting from ju r“es
rating in the 8 me room.

Konetheless, within these limitations, the findings of this
research appear to warrant the following concbusions about the
hypotheses presented in Chapter I:

l. The data of this research indicates that the frequency
of affect, understanding, specific, exploratory, and
effective responses in initial personal-social problem
counseling interviews, as measured by the five dimensions
of the A-V Scale, is significantly greater for Ph.D.

candidates 1n Counseling and Guidance than it is for

yr

1.A. candidates beginning their practicum experiences in
Counseling and Guidance. Therefore, when used for the

purpose of discriminating between the responses of
1
Their training, however, was diversified and acquired at various
other universities.

{I (W

-'.,,S ‘-

.1

those two classes of counselors, the A-V Scale appears

to have adequate validity.

a. The data of this research suggests that the A-V Scale
has high interjudge reliability for discriminating
between the responses of H.A. and Ph.D. candidates
on each of its five dimensions. Therefore, the A-V
Scale seems to have adequate objectivity. Compared
to the low objectivity findings reported for most other

counselor evaluation scales, these findings appear most

favorable.

The data of this research suggests that counselor

\J)
o

trainees who receive professional supervision which
focuses on examining affective, understanding, specific,
and exploratory responses will receive significantly
higher frequency ratings for these categories on the

A—V Scale than will counselor trainees who do not

receive this professionally supervised training.

Therefore, the A-V Scale seems to have adequate sensitivity
for measuring the subsequent change in these category
behaviors which are.associated with special counselor

training.

inplications

This section is divided into two parts; a discu351on of the
implications for counselor education and the implications for

future research.

Implications for Counselor Education

Affective, understanding, specific, and ermioratory counselor
behaviors seen to be associated with Ph.D. candidates in Counseling
and Guidance but do not seem to be associated with H.A. candidates
beginning their practicum experiences in Counseling and Guidance.
This would suggest that these counselor behaviors are associated
with, and are perhaps relevant to, the counselor training process.
If such an indication proves valid, then special emphasis on
these counselor behaviors during training can perhaps accelerate
counselor education. The A-V Scale would then possibly become a
potential evaluator of this type of accelerated counselor
education.

The affect, understanding, specific, and exploratory dimension
categories may be central to effective counseling or they may
have nothing to do with effective counseling. This we do not
know. However, if it can be satisfactorily demonstrated that
these category behaviors are significantly related to effective
counseling, then this would indicate that these types of behaviors
are relevant to the counselor training process. If this be the
case, the A-V Scale can serve as a cumulative progress report
of a counselor trainee's change in counseling behaviors as he
progresses through the various phases of his counseling develop-
ment. Hhether or not these category behaviors are significantly

related to effective counseling will have to be substantiated
by further research.

luring the course of the rating sessions which were a part

of the three studies compiising this research, a new value and

:w n

I

use of the J‘s-V Scale was suggested. IIDBA Institute and other
I-LQSCOP:S levol counselors who served as judges said they understood
more fully the importance of affective, understanding, specific,

L

and ez-zlfloratomr responses on the part of the counselor because
they had been required to focus on and evaluate clearly defined
aspects of the counselor's verbal behavior. This may have given
them a very useful and meaningful set, one which they had not
acquired before, for evaluating counselor behaviors. As a result
of their rating experiences with the A-V Scale, the importance

.0

of affective, understanding, specific, and exploratory reSponses
for effective counseling became more apparent to them. As a
result, some of these counselors reported more of an emphasis on
these counseling behaviors in their own counseling practice.
They also reported that through the use of the Scale in rating
audio-video tapes they learned more about what is "good" counseling
than they had ever learned in their Easter level practicum
experiences.

Could it be that the use of the A-V Scale itself for rating
purposes by counselor trainees would be a valuable supplement
to regular counselor trailing pro gram procedures in developing
basic counseling understanding and grot-rth as well as serving as
a means for providing a relevant model of counselor behaviors
with which to identify? Is it possible, then, that by using
the A-‘J Scale to rate experienced and inexperienced counselors,
the Scale can contribute directly to the education of counselor
trainees by providing exerci ses for focusing on essential

counselor behavior as well as providing realistic opportunities

for observing the effects affective, understanding, specific,
and exploratory responses, or the lac!r of hen, has on clients
in real cmu'zseling intervieX-J situations? Could it be that such
experiences would accelerate counselor education and give a
sound counseling perspective early in the training period? The

answers to these questions can only be acquired through further

research with the Scale.

Iraalipajti0 is for Future Research

Further development and evaluation of the A-V Scale is
needed before definitive conclusions can be made as to its
characteflstics and value. The implications for the types of
researeh'v- which seem most promising for the A-V Scale are:

l. Cross-validation of the A-V Scale on audio-video taped
counseling interviews at other universities using
other trained judges at each of the universities.

2. Ecmating two groups of counselors A and B. Train group A
along the dimensions of the A-V Scale. Train grcup 3
along some other counseling dimensions. Rate both
groups as to global effectiveness and er mine whether
group A is rated higher.

Establishment of more intermediate counselor levels of

LI)
0

e:;)e;ience and. training and evaluating the Scale's
sensitivity to differentiate these levels.

1+. Applying the Scale to the rating of counselor trainees
in various counselor education programs and observing

J

how satisfactoxily it detects change in their counseling

behaviors as they progress through the programs.

Eva nation of the Scale's value for accelerating counselor

kn
0

educational growth as a result of counselor trainees

using the Scale themselves in counselor rating sessions.

0

 

ill ‘ll‘ollll‘lr'!

 

9.

10.

12.

I)
/.

Bibliography

Albee, G. 7. and Harlin, R. K. Judgment of adjustment
f_on irawings: the applicability of rating scale
ethods. J. Clinicalfifsvchol. 1950, 6: 363-365.
Salerrnder, F. Fundarentals of Pch hoanalvsis. ETew Yorlm
.‘-.Ol" con, 19113.

American Educational Research Association. ReViGU'Of
Educational Res :earcl., Guidance, Counseling, and
Personnel Services, April, 1903, pp. 184-185-

Anderson, R. P. and Anderson, G. U. Development of an
instrument for measuring rapport. Eprsonnel and
Guidance Journal. l9o2, 41: 18-24

Bayroof, A. G., Haggerty, H. 3., and Rundquist, E. A.
validity of ratings as related to rating techniques
and conditions. Personnel PsychologZp 1954, 7:
0a 11?
J-""“./ 0

Bondig, A. I. The reliability of self-ratings as a function
of the amount of verbal anchoring and of the number of

categories on the scale. J. Applied Psychol. 1953,

37: 38-41

Bendig, A. 7. Reliability and the number of rating scale
categories. J. Applied Psychol. 1954, 38: 38-40

Ihmxhgh A, V. Rater experience and case history judgments
of adjustxmelt J. Clinical Psychol. 1955, ll:
127—132.

Bendig, A. J. and Sprague, Janine. Rater experience
a.nd the reliability of case history ratings of
adjustment. J. Consult. Psychol. 1954, 18: 207-211.

California Test Bureau. A Glossary of Measurement Terms,
Konterey, California, 1959.

Carnes, E. F. and Robinson, F. P. The role of the
client talk in the counseling interview. Educ.
and PSLChOl.LeaSXt.19M8, 0: 635-644.

 

Cartwright, R. D. and Lerner, B. Emuathy need to change

&

andi Drovenent in.psychotlleraoy. J. Consxlt:_

A

Pmrchol... 19L)3, 27: 133-1145.

 

Cattcll, J. L. Cited by floodworth, Robert S. and Harold
Scholsoerg. Experiuental sychology, Revised Edition.
Net Yer : Henry Belt and Company, 1954.

in
_.Q‘)-
r/

‘9

(113411111033 73. and Earsl1z111,:;. Optinal refinement of the
rating; sca ael c. J. fixnplied Psychol. 1939, 23: 323-331.

Collier, R. H. A scale for rating; the responses of the
therapist. J. Consult. Psychol. 1953, 17: 321- 372C.

Coxr:1cr,2. J. Studies in phcncgraphic recording; of verbal
tclial: I. The use of phonographic recordings
in counseling practice and researcl . J. Consult.
§_S*':'chol. 19132, 6: 1019-113.

Cori-nor, B. J. Studies in phone rap 11:7
tie ial: 11:. The cord. pletene es
counseling intervie on reports. J.

"115', 30.1ul-203.

0 recording of verbal
s and accuracy of
Gen. P3; hol.

 

 

 

 

O .a‘ 9 n so 0
1... CO‘J‘ 10;, s. u. DoucLleS'll‘l ahonor'ra 1111c reco ‘dirgs of verbal
:aterial: II". Ifritten reports of interviews.
J. Implied ngchol. 19m, 28: 39-9-3.
'I, f‘- u o c
:3. 1.. ' ‘anach, L. J. Response sets and test valichty.
Zane. and Ps:/‘(:l1ol...ea"1_t. 191-16, 6: 475.1; 34.
20. Cronbach, L. J. Essentials of Pshchclcggcal Testing;
230:: Ioriz: Ila- rpc cr 3110s., 1960.
.1 I Q Q - - o o o
2*" Crone: .en, L. and Loehl, P. Construct validity in
‘jsychological tests. Psychol. Bull. 1955, 52:
35(1-302.
at)
"“' DeP oo, .,'. II. A study of the relationship between selected

 

5.4. Dittz‘an, A. T. The in
a

verbal behaviors of counselors and variables of the
IIoltm-‘an in}: blot test and the Rolze ach dog; Katie :1
scale. Unpublished doctoral dissertation, 19C5,

9'.

-ichi'an State Univcr sity.

er personal proces in 1 svchotherap":

4- f‘

nU L)

developnent of research neth od. J. Ab onorm. and Sec.
..n
(-

Psychol. 1952, 2 9" """

LlJLhL, 1‘1, .

21L. 1);. 10nd, R. .7. see 1e for the 1:1easurencntoer1pathic
ability. J. Consult. Psvchol. 191.9, 3: 127-133.

 

535- 3001.311. Estimation of the reliability of ratings
fif‘f'

.s eqoletrika, 1951, 16: 407_h2.
LL). Feld‘"g.ny Jo Jo, Zb’i‘fln, Hg, and £13. 11% C. .. A fiol d
"131153 05 intend. ever effects on the quality of
3‘1?va (Lita. 3.101.210 Cpizfion 5118'le bellJ— L951, 55

principle of rel-.:.-t1.o and neocartharsis

 

 

23. 17:5, 001101", F. E. The concept of an 1.10 al therapeutic
relationship. J. Cansult. Ps jchol. 195C, 14:
.- 19—..11-5.

29. 13‘fi. cdler, I‘. 3. A ceg.11a1*i son of therapeutic relations hips
in psychoanalytic, nondiroctive, and Adlerian therapy.
J. Co1m11t. P sychol. 1950, 11-4: 1"’6-LH5.

 

30. mmsinger, J. 3. Psychiatric interviewing: I. Some
11121116111105 and procedures in insight therapy.
11102?. J, Parchiat. 19"?, 105: 137—195.

31. F3. “3111:, G. II. On the history of the objective investigation
of the process of psychotherapy. J. Pslchol. 1961,

(‘1
Duo 09-950

52. Fl“;“3.1’112, G. II. and Sweetland, A. A study of the process
of 1 sychotherapy. J, Consult, Pachol. 1952, 26:
135—138.

33. Ge‘ohart, G. and Hoyt, D. Personality needs of under and
over achievers. J. Implied Psycho; 1958, 2: 125—128.

31+. Guilford J. Personalitv. New York: I~IcGraw-Hill, 1959'.

35- Gordon, Kate. Group judgments in the field of lifted
weights. J,_Eb:per. Psychol. 1924, 7: 398—1100.

36' Hal ..).)ern, II. and Lesser, L. Empathy in infants, adults,
and. psychotherapists. Psychoanal. Rev. 1960, 27:
32-42.

37 ‘ Hagz‘lin, II. 2:. The clinician as judge: implications of
a 50110 es of studies. J. Consult. Psychol. 1954, 18:
233-7338

38. I-Ianzay, I3. I., Dittman, A. '1‘.. Raush, H. L., Bordin, E. 8.,
and 75”].01", D. The neasure2:ent of depth of 1111301 are-
ta 131011. J. Consult. Psychol lo 55, 19: 247-253.

39. I-Iel'astac tor, G. C. Principles of Psychological Measurement.
Lew York: Appleton-Century-Crofts, 193+.

1.50. Highland, R. and Berkshire, J. A methodological study of
forced-choice oerf or111ance ratings. f 3108. Bull.
51-9 IL‘EIC. San Antonio, Texas, 1951.
41. Hoyt, C. "est diability ob‘taincd 13; ans 1731:: of variance.
. 12:51.. 111,191, 131-10

’0‘

0.

a

Q 1'
,_

5- r:

ZIQ. 0:71), C. Cit -. by Ebel, R. 7’ easuringj Educ Lona}- Acifievezzert.
o f
315101-2001": Cliffs,Ie1.r Jers 0/: Pre1fiMcIIau,j-91>5.

 

 

‘ (In... Iii .II 1'

 

 

 

L'53. J <9 izrofec‘u, S. I-thou relationship versus mmimlotion
:1 counseling on I. ‘szcfilotherany. J. Ind. Psychol.

‘Jo

I
.L

gnu 15; 17¢.179.

‘..//,

'._J

M'- - “”5 5.1:, :39 I“L"‘at11'~'ohl, D. R. , and 13.3.10: 3., 3:;i mlotcd

rCCSiI ill Lhera)v u Jinn VidCO tape--a C386 Sbldj.
n / I‘ A ,-

J- CWUL’O—lnii P810110» 1:03, la: 37-443.

 

'5. 110 ct, C. 3. Two verbal techniques "n a. pinioturec ounsolinf
situation. P3101101. 1301106: 1*. 1M3, 6-2: 110. 2954'.

 

..a

do. 2:03.23, J. If. The c‘i.c'v'olop:.:ez’lt and ovanotion of o. “109.571.1208
01‘ coansolor effectiveness. :E‘-'C.-'1 Journal, 1366 in
;)~'J.::.’..

y'r -,__- - . . o ’ 0 fi' f‘ '

~.-/. :_1_. (2310:, 0., . Clotucu, 17.0.1121 ileln, .—.. 33131121."; fro-r:
"1‘.-’3 reqcordcc‘. tnormy inter-vi ow: a coazlmrative study
of r‘iiforont segment lengths . J. Consult. Psycho?“

L} .L" Q - " A . .
LC: ' L 2'»: no.‘(, )Ir9-3/7.

a ‘4,

 

 

I./‘, Y’— Q n .' . . . a
m -'-‘z.1-o_ :9 .:. 2.911 “a cr‘ontson, . The theory of :s5:-::o.tlo.1 o;
o 9‘ _ o . ._ o 7 _ 0 'fi /
ooso .n‘ello311_z..:,'. 133.1323- strike, 19?", 2: l5l-l-0.
H '1 M —- 0 '
W. Q? or? 0.7.1, .1. .J. The Tl‘OQl on; of adec ate we- 3011.13.13};
records: a. torso )0351. -.‘W‘. J. 1 .lrciiot. 1:329,
f‘v’ V
_f :0 103’7-1OL\6.
so ..
.-., ‘l ’1' - ." F
2 0 .-.o.ss.-o.-l, :1. D. Jel‘bol re; erence cs and ‘ 1:, s"! ..ologic .1
changes Curing the :sychoan a1 rtic i..terv1 e1 2.

'_:-:‘.1J..-.n..1ff commnicotion. Psionoonol. Rev. 1:335,

73 'P) . :1— 0-211'.

tip—v u

"'1
wk . lassvoll, 1:. 3. Cert/9.51.21 1.1‘032105. tic c212 n es during
tzt'vol psychoanalytic) i: tervietfi‘ . P yl‘choanal.
v“’\/ A '
box. 19-0. 23: 2.9-273.
C'A _

.24. uélSSITOJL, II. D. 1 provisiono 11 classification of spool
' -otr' 1938, 1: 197-20%.

 

 

 

 

”f! 3 . 1 ‘3. r"" 1“ vI‘ I ~ ‘~ » .0 h _ _L ‘ 0
DJ. ILLJCIfl-uso, 11.. :. D081“1’1 and analysls of iw5el'."_.lef1oi: 31_
~ - x .w- '*t " . . .,- . 22"
:svcwolo' , 2.21-: .‘;.C:dC£T.lO.1. Boston: ZIourolmon .-j.:..‘2.n
p'A Oran
CO..1 93.11;" ,lDJ-J, (.>L;.
6': e ' d- r‘.‘ “ ~71 - w~1 7'. m wn‘
1‘7. 4.3..“1 :11 Lou- ‘ J, u. 2' o 4.10“ U...C’-.;~ . .LaSU.C. -Lu 4. {$3.13. 8:1: ,
-o ‘ y ‘( A V _.. . 3, 1 Av/fl
”1110011151. 1: 301'f6:_‘2.1-ta1 £101-35»- "1-0 . , 2-1-...

55. 1.90.17 ’ P. 3' L LLLL CihCCorguodgle, K- "3”t1~’..;;1 voliJit)’
of'fi..-1";or"~rltii1~r; constaxcts. In: 27. ii. 123‘7.

"V‘- 1 a. m‘ r $.- v' " I ‘- _.. . f,
(its 0 163 3.7. wor’ . -.o.. 1.01%., -~2.cr..:.ll... .

I ..an ..z

/ ‘fi '0
5;. Porter, _.. ..., Jr. Tho C‘Lovcloortont and evaluation of o.
re of c 121:: 03221;; intervi '.-.‘ procedures: I.
The devaflogztent. Educ! and Psycho}... Yeast. 19:":3,
’2; . fl 5.1 ”A

--... - ->—a\l.

 

 

 

llama all?! . Yul.

 

P\
_f -
I

A

“7. FOP-"1301‘, 3. H., Jr. The clevelowzzent 5.11.1.1 evaluation 0:

 

J 5. :0:sure of counseling; and 1v1lcr'vie1jarocec‘mres.
I. The 01.101.115.13bion. Educ :1an P5301101. 2705.330.
19’33 3: “15-233.

3’]. 27:13.12, 2‘1. C. The 5.53””1'110 01" com selor-treinee 110110011121 ons
of fairinrviox-I protocols before and a; “ter an inten sive
procticmzt 031015102100. Doctoral dis sorts cion, 12361:,
Ynivcrsity of 13.112100 01351..

59. 31* 1.;1‘2:, 31. C. :1 .d 30.05, '..'. II. Counselor response system
of the ‘ Dollavim interaction description syst0
Pager read at 1:110 ALGA Convention, 1-':'1-m1051.:1olis,

,I‘V ‘3 -‘ “ vv _ 3.‘ ‘ ‘ ‘ g -, — v. .0. ‘ ‘
1,3. -«Z‘. 612.. -;. --., 0110-1306131», 313101‘, 3. .11 1114-0-13, J.,

 

o- . . U, A ‘ ‘
“5.1.0,, .I., -fon‘i.” .3. 8., Jitt. man, A. 1‘. 51.1.0
22:.” s, ‘I. r. A w. 011010.111 3.119.111.1320 of depth f
iz'x'borgrctation. J. Consult. P1591331 :11. 195-5, .730:
."'T‘ 3"
"/-' ..
:1 T: r. - .- -- - . .. .. -,
1-0. «‘ ~1L:,-;, --.. and zon‘in, .3. .2. ..':1.:r th in pers onelita

(10110101321011: 51.:.;‘-. in lt-S;r010t11ore‘.".3’. PS.','Cl‘iE‘~._t- 1957’
An , O a
.10. ’1, 291-263.

 

 

’m --

‘J; ‘4 ‘ "' 'fi‘- q c Q’- F‘H .\ ‘

L . “—-OC-I, . o, ..b JCS, C. , Fl..‘-‘1.:l, A. 11., JSQ1~I‘J.'&J',, ..). SQ,
-0..5.n, II. 3., 9.1.6. 31‘ - oer, 3. Lotions-l Institute 01‘
2.012351 Health pilo and study in *z-oiningj- cental 110911211
059.11.:30--0rs. The 300:1. J. ’thhopzjgcriot'g. 1903,
"X’I. ’ A ’flr)
/" u L-J- /I o

,

03 '\ r0 ' 0 r‘ '3 1 . r. ‘ w '3 -, 1‘ ' ‘v V,

- flowers, 0. a. CO‘II’ISanA :1.-v10. ;s 0.05110121- 1 . Boston,
«.1
22011le ”on I-‘ifflin C0: 1.1.1157, 3.;102.
(a

“‘- -.0 0.5, C. R. 1:3. ectr'ically recorded interwi ears in
"41111011. ‘ 1;; I)S;’C.100.-Or"lf)0uulc intelwiwss. Inter. J.

3:. $05077.) 4,01" )j. LU. l;’11’o ’ 12: [4‘29 "35.

 

p. a
1.1
rh

'-

a o

Rogr-‘crs, C. 21. Client-centered TILCI‘QEY. Corbridge,
O - ‘.O Q ‘ ' A
21'.’0:0ice Press, 193i.

/ f‘ f‘ Q: Q . ‘.

60 11032118 C. 3. opo-.sor: Lat-lieu, P. on - curse-11d, J. P.
311“ 1035313131 05 31‘0“? 0011113011an t0 m‘osequent ac;1 1610.210

L‘Clrf011?2--co a" £110 00110;. level. C001erativc research.

7“

b
Project no. 037194.? 3fl-sconsin :syc-xiatric Institute,

 

‘ 3
v o - .n _ r r 70 ‘* " F (V 5 _ /
bits-versit; of .4 sconsin, maison, ..2isconsin, 1202.
«37.3000, ZI. Inte11w.101.:1'_ng to test for valic-ity and
rolicoility. I--tern. J. of Ouizio or and .‘Lttifiude
20.130130?" 9377, 1.: 1031—101.
/ v. o -‘ H .
113. Mich, 17.3. I‘syc‘1010'12' and life Chicago: ocott, gorcsrzan,

 

5.:6. :30. ‘ iz-tth edition, 19?}.

69.

72.

75.

7'

3:11:15 3.. 0. Is the rating of human character practicable?
J. 3:13.10. Psichol. 19? , 12: 1:25-1:33 and LIBS-501.

 

s clxafcr, R. Generative empathy in the treatment situation.
Psvehoanal. Quart. 1959, 23: 34252-373.

Silapiro, S. and Eberhalt, J. C. Interviewer differences
in an intensive interview survey. Intern. J. of
Ozzizfion and Attitude Research. 1947, 1: 1-17.

 

Srvder, 3!. U. An investigation of the nature of non-
directive psychotherapy. J. Gen. chhol, 1945,
33: 193-223.

SIUClOl‘, 2.7. U. The Psyghotheraoy Relationship. New York:
I-La.c:j.11an, 1901.

Ste 3311, Joseph. Responses to hypothetical counseling situation
as a prediction of relationship orientation in school
counselors. Unpublished doctoral dissertation,
University of Wisconsin, 1963.

"C’I'unlg 0., Jr. Expath : a review of theory and research.
Psychol. Newsletter. 1957, 9: 47-57.

76. Sta—11px), H. H. Hature of psychotherapist's contribution

to the treatment process. Ardh. Gen. Psychiat, 1960,
3: 219-231.

77- Szntloncls, P. II. On the loss of reliability in ratings due

to coarseness of the scale. J. Laser, Psychol. 1922,
7: 1956-461.

78' Syztzonds, P. N. Notes on rating; J. Applied Psychol.

19259 9: 188-1950

79- Sjfi-Londs, P. Ii. Research on the interviewing process.

J. Echlc._Esxc1mli 1939, 30: 356-353-

80. Thornton, G. The effect upon judgments of personality
traits of varying a single factor in a photograph.
g. Soc. Psycho; 1943, 18: 127-1118.

81. Thornton, G. The effect of wearing glasses upon judgments
of personality traits of persons seen briefly.
J. igflied Ps;~[_chol. 19134, 28: 203-207.

 

C3
(‘0
o

Torgerson, 2']. Theory and Methods of Scalian Ifew York:
John Kiley and Sons, 1953.

83. Truax, C. B. and Carklmff, R. R. Theory and research
ha

in counseling and psychotherapy. 11pm: Journal.
Day, 19:31:}, 880-335.

 

and Ifiea mrez‘c nts . Engl evood Cliffs ,

32:;rfler, L. 2. Test /
u L a 1 [_
: ren ul 8-;13Jl, Inc . , 19e3 .

“0*: Jersey

85. E'fajdcer, H. II. and Lev, J. Statistical Inference. New York:
Zlolt, Rinchart and 'Jinston, 1953.
‘-.7'::2.3_3, G. 1‘.. and Johnston, J. A. Counselors look at
therzsolves on video tape. J. lenselinLPsychol.
1963, 10: 232-236.

CO
C\
o

r .

87. ZFaZ‘d, R. Educating counselors by video-tape using the
interpersonal process recall technique. Unpublished
electoral dissertation, 1965, Inchigan State University.

33. 312:: ’tson, P. D. and Kanter, S. 8. Some influences of an
expci'uental situation on the psychotherapeutic process:
a report based on 1&4 treatment interviews, of the
reactions of a patient and therapist to observation,
recording, art‘s. physiological measurement. ngchosoztz.
lied” 1956, 18: 457—470.

89. ~ Testnan, N. C. A comparison of three modes of rating
counseling interaction. Unpublished doctoral
dissertation, 1965, University of L'Iisconsin.

90- lander, c., Almad, F., Bandnra, p... and Han, L. Dcwendency
of patients psychotherapists' responses and aspects of
psychotherapy. J, Conmlt. Psychol. 1962, 26:
129—134.

91- 2:013, R. and l-Iurray,
personalities .

. A. An experiment in judging

Lof Psychol. 1937. 3: 345-365.

C453

APPENDICES

-100-

APPENDI X A

A WI'IPILATIOI‘J OF COUNSELOR VIDEO
VERBAL BEHAVIORS AS OBSERVED

ON AUDIO-VIDEO TAPES

-l Ol—

VIDEO VERBAL BEHAVIORS CHARACTERIZIKG MASTER'S CANDIDATE COUNSELORS

Cognitive
l o Sllperfi 0131
2. Clumsy structuring of interview
3. Nonverbal behavior indicates anxiety and discomfort with client
4. Prestructures client's reply
5. Distant from the client emotionally
6. Avoids client problem areas
7. Reflects or rephrases cognitive aspects
8. Expresses no feeling about client
9 0 V9. gu 9
10. Moves client away from problem area
11. Questions cognitive aspects of client's verbalization
12. Makes general comment in response to client's verbalization
13. Agrees with client's cognitive verbalization
14. Repeats client's comments verbatum
15. "Yeah," "um huh," "I see," "right,”
16. Comments on room
17. Makes judgmental comment concerning client or client's
verbalization
18. Adds additional cognitive content or meaning to client's
verbalization in restatement effort
l9. Leads client in specific cognitive direction through
suggestive comments
20. .Lectures or sermonizes to client
21. Philosophizes to client
22. Conducts monopolizing conversation
23. Advises
24. Offers cognitive evaluation of client's tuation
25. Pleads for information
26. Excessive Verbalization response to client cognitive content
27. Anxiety laughter
Affective
1. Inaccurately reflects feelings
2. Inaccurately interprets feelings
3. Reflects ambiguity
4. Leads client in irrelevant affective direction through suggestive
comment 5
5. Adds additional affective meaning to clients verbalization in
reflection effort
6. Excessive verbalization response to client affective response
content
7. "Yeah," "um huh," "I see," "right,"

Video Verbal Behaviors Characterizing Ph.D. Candidate Counselors

Cognitive
1. Questions cognitive aspect of client's verbalization
2. ”Yeah," "um huh," "I see’n "right,"
3. Reflects cognitive aspect of client's verbalization
4. Reconstructs or simplifies cognitive presentation
Affective
1. Responds at the same emotional level as the client
2. Free from discomfort with client
3. Questions client's feeling
4. Appropriately reflects feelings
5. Appropriately clarifies feelings
6. Appropriately interprets feelings
7. Asks personal-intimate questions
8. Discusses client's problem areas
9. Reassures client concerning affective aspect of Verbalization
10. Offers affective evaluation of client's situation
11. Questions to clarify more specifically general feelings
expressed by the client
12. Expresses to client the feeling which client's Verbalization
elicits from him
13. Expresses his feeling about client
14. Suggests related affective area for discussion
15. Points out to client his reluctance to discuss an affective
area
16. Points out to client his refluctance to discuss more deeply
his feelings
17. cqeah," nun huh," "I see," "right,"

APPEE-IDIX B

CATEGORY. ZATIOI‘J OF COUNSELOR
AUDIO-VIBE.) BEHAVIORS ACCORIIING

TO BEHAVIOR CATEGORIES

B.

CCU ZISESLOR VIDEO VEZBAL BEHAVIOR CATEGORIES AND IILUSTRATIOI-IS

Affective

This categorv includes all counselor responses which make
reference to some affective or feeling aspect of a client's
verbal communi cation.

Illustrations

1. Responses which enter the feeling frame of reference
of the client, which deals with expressed concerns,
needs, and wants of the client as expressed or
implied.by the client.

2. At the same emotional level as the client. Emoting
with the client on his feeling level.

3. Responses which reflect, deal with, clarify, or’interpret
the feeling of what the client is saying.

Understanding

The counselor's ability to communicate to the client the fact
that he knows what the client is feeling or talking about.
Counselor communicates an understanding of client's feelings
to the client.

Illustrations
l. Counselor reflects client feelings accurately

Counselor attempts to express in fresh words the essential
attitudes (not so much the content) expressed explicitly
by the client. The counselor attempts to mirror the
client's attitudes for his better selfaunderstanding

and to show the client that he is being understood by

the counselor.

2. Counselor clarifies feelings in a specific, non-vague,
rial-iner e

The counselor clarifies what was implicit in what the
client said; he reflects relationships or meanings for
the client which are suggested or implied in the feelings
or ideas brought about by the client's actual responses.

3.

5.

-106-

Counselor interprets feelings appropriately.

Interpretation refers to any remark which interprets,
analyzes, explains, or draws inferences. Counselor
presents to the client an hypothesis about reliationships
or'meanings of his attitude behaviors, which he had

not considered before, for his consideration.

Counselor's verbal responses and accompanying behavior
demonstrate primarily an interest in and comfort with

the client and thereby conveys counselor acceptance to
the client.

a. Counselor's attention and concentration are on the
client.

b. Counselor's verbal responses demonstrate that he is
following what the client says (e.g. "yes," "I-Mim,"
”un huh," "I see," etc.)

o. Counselor gives indications of approval which seem
to give reassurance and encouragement (e.g. Rgood,"
"I feel that's a fine idea.") These may be positive
confirmations by the counselor of the adequacy of
client behavior, attitude, feelings, etc., as alluded
to in the verbalizations of the client (e.g. "you
are a capable person;" "you can.be consisten ;"
"you can be reasonable;" "you can be organized:"
"you can feel better.").

d. Counselor's nonverbal manner is free of discomfort
and anxiety (there is an absence of continuous
shifting in the chair, looking away from the client,
or continuous movement of arms, hands, and legs).

Counselor questions client's feelings generally or

questions to have client clarify more Specifically

the general.feelings which he has expressed. This

is a request for general elaboration and expression
of feelings.

Counselor expresses his feelings about the client or
the feelings which the client's verbalizations have
elicited from him. This includes:

a. Statements of counselor's feelings about the client
as a person.

b. Statement of counselor's feelings about something
the client has said, done, or thought.

0. Statements in which the counselor's feelings are
expressed as being his own.

C.

D.

-197-

Specific

Counselor's interaction with client is characterized by
concrete (specific) responses concerning the client's
problem.areas and emotional conflicts.

Illustrations

l.

2.

3.

Counselor proposes or suggests that the interview be
devoted primarily to the objective of exploring the
client's problem areas and emotional conflicts.

Counselor's responses reflect, or are directed toward,
the core of the client's remarks rather than their
peripheral aspects.

Counselor's responses move the interview discussion
in the direction of the client's problem areas and
emotional conflicts or bring the client's discussion
back to these topics.

Counselor's responses are relevant to the client's
prdblem areas and emotional conflicts rather than
being irrelevant as characterized'by abstract,
intellectual, lectural, extraneous, etc., type of
responses.

Exploratory

The counselor's assistance and skill in helping the client
to explore his feelings, values, and other concerns.

Illustrations

l.

2.

3.

Counselor suggests possible exploration of affective
area referred to either*explicitly'or implicitly by
Client 0

Counselor points out to the client his reluctance to
explore an affective area.

Counselor asks personal-intimate question of client.

Counselor'is willing to explore with the client, in a
nondefensive manner, the import of his (i.e., the
counselor's) behavior traits which the client observes
in their relationship.

F.

-lCIS-

 

Cognitive

This category includes all counselor responses which omit

any reference to affective or feeling aspects of a client's

verbal communication.

illustrations

1. Response is distant from the client emotionally.

2. Superficial reSponse which passes over emotional
implications of client's verbalization.

3. Not with the client emotionally.

4. A response which is not on the feeling level of the
client.

5. A response which does not enter the client's emotional
frame of reference-it is at a different level from
the client's feelings.

6. Own feelings or apprehension preventing counselor
from making emotional contact with the client.

7. Clumsy structuring of interview.

8. Anmering factual questions.

9. Repeat of client's remarks verbatum.

10. Responses devoid of any reference to client affect-
feeling, emotion, mood, attitude, etc.

ll. matter of fact responses, informative responses,
general and superficial discussion responses.

Nommderstanding

Counselor does not communicate an understanding of the
client's feelings to the client.

 

illustrations
1. Counselor empres uses confusion as to cognitive or

2.

affective meaning of client's verbalization.

Counselor's responses show evidence that he has pre-
judged the client and thereby has classified,
stereotyped, or categorized the client.

G.

3.

7.

Counselor's responses inaccurately reflect the client's
feelings.

Counselor's responses of clarification of client's
feelings are attempted in a vague, non-specific
manner.

Counselor’s responses encourage or reflect ambiguity.

Counselor's responses in restatement or reflection
add unwarranted additional affective or cognitive
content meaning to client's verbalization.

Counselor responses reflect the cognitive aspects of
the client's verbalization when affective aspects are
central. ,

8. Counselor repeats client's comments verbatum.
9. Counselor pleads for information.
10. Counselor verbal and nonverbal responses indicate a
lack of interest in, and discomfort with, the client.
a. Counselor looks away from the client rather than
concentrating on him.
b. Counselor does not appear to be following what
the client says.
Nonspecific

Counselor's interaction with client is characterized by
nonconcrete (nonspecific) responses concerning the
client's problem areas and emotional conflicts.

Lllu strati ens

l.

3.

Counselor's reSponses reflect, or are directed toward,
the peripheral aspects of the client's remarks rather
than toward their core meanings.

Counselor responses are related primarily to the
cognitive content of the client's verbalizations and
deal only superficially with the affective content.

Counselor responses are irrelevant to the client's
problem areas and emotional conflicts. They are
characterized by abstract, intellectual, and
extraneous responses.

a. Counselor encourages expansion of superficial
cognitive discussion.

.11 C...

b. Counselor intellectually debates cognitive or
affective aspects with client.

0. Counselor remains silent and simply permits
client to wander in his verbalizations.

d. Counselor is excessive in his response to client
cognitive or effective content.

H. 15 onexplorat cry

The counselor's failure to help or permit the client to
explore his feelings, values, or concerns.

Illustrations
l. He—structures clients reply

4.. Counselor monopolizes conversation, philosophizes with,
or lectures to client.

3. When client presents affective content, counselor
response does not encourage further exploration of
this content.

1+. Counselor's responses move the interview discussion
in a direction away from the client's problem areas
or emotional conflicts

3. Counselor makes judgmental or evaluative responses to
client’s Verbalizations.

6. Counselor's responses are characterized by advice-
giving and question-answeling.

7. Counselor's responses are structured to specifically
lead the client in definite cognitive or affective
directions rather than permitting the client to .
initiate these directions.

APPENDIX C

DEFINITIONS AND ILLUSTRATIONS OF

THE A-V SCALE'S DDIETJSION CATEGOKEES

—llZ-

TIT OF EVMHATIOI‘I FOR THE A-V CCDNSELOR BEI-IA‘JIOR SCALE

The urit for evaluation for the A-V Counselor Behavior
Scale is the counselor-client unit. A client verbalization is
emitted or elicited and the counselor's response to it is
then rated on all five dimensions of the scale according to
the dimenfion mtegory definitions. One forced choice rating
is made independently for each of the four specific counselor
behavior dimensions and a fifth rating is made evaluating the
effectiveness of the counselor's response. Generally, each
client-counselor unit is jxdged independently of preceding
units, although the congruence between the direction and level.
of counseling movement by the counselor with the direction
and level of counseling concerns the client is attempting to
present must continually be kept in awareness when making
rating judgments. The scoring of all dimension categories
must be based upon concrete and observable behavior and is
not to be based upon what a judge may believe could possibly

be occurring within the counselor.

13112313810113 OF THE A-V (XDUITSFLOR BEHAVIOR SCALE

The Cognitive-16f ective Dimension

This dimension indicates whether a counselor's response is
in reference to some affective or feeling aspect of a client's
verbal communication (an affective response) or whether his
response omits any such reference to affect or feeling in the
client's verbal communication (a cognitive response). These
categories are more clearly defined as f ollows:
A. Affective Regponses

Affect refers to emotions, feelings, fears, moods, desires,

urges, impulses, fantasy, dreams and any ideas, attitudes,
beliefs, convictions, etc., which are based upon such re-
ferents as these. A counselor affective response is one
which makes reference to any of this type of affective
material on a feeling level regardless of how congruent it
is with that being expressed by the client. An affective
response is not determined by the extent to which the coun-
selor's manner or communication is affective, but rather, it
is determined solely by the fact that the counselor responds
throughcomnent (reflection, clarification, interpretation,
etc.) to underlying affective elements, or to what he implies
are underlying affective elements, in the client's verbal
and nonverbal expressions as they are revealed in his inter-

actional discussion with the counselor.

B.

.114-

Examples:

1. “You feel quite inadequate in these situations."

2. "You act that way because you are angry."

3. "You feel better when she mothers you."

4. "How do you feel when your parents argue?"

5. "Maybe that's not what you want at all."
Coggiclve Responses

A counselor cognitive response is one which is on a non-
affective plane in that it is devoid of any basic concern,
reference, or recognition of client feelings, emotions,
moods, attitudes, or any other affective aspects which may
be implied in the client's communication. Such a response
passes over any affective implications inherent in the
client's remarks. The counselor responds at a cognitive,
non-feeling level or asks a question or makes a statement
concerning cognitive or content material. Such cognitive
responses are more formal, informative, matter of fact, of
a general discussion order, and socially acceptable oriented,
and are altogether on a different plane from responses which
‘mske emotional contact with the client through discussion
of feelings, attitudes, etc. Cognitive responses include
pnswering factual questions, "busy" talk, discussing triviali-
ties, repeating client remarks verbatum, exclusive talk about
third parties without affective reference back to the client,
any any general educational, intellectual, vocational, con-

versational etc., type discussions.

II.

.115.

Examples:

1. "Yes, that seems to be a logical step to take."

2. "Your father was an attorney, huh. Well that seems
to give you a good background."

3. "How are you feeling today?" (If intended in a
general sense)

4. "I think your performance in your science course
would support a decision to enter medicine."

5. "So you want to enter graduate study. I think

that ' 8 nice. "

The Understanding--Non-Understending Dimension

This dimension relates to how well the counselor's response

communicates to the client the fact that he understands, or is

seeking to understand, the client's basic problem. These

categories are more clearly defined as follows:

A. Understanding Responses

These are responses which imply that the counselor

understands the basic underlying affective or cognitive

concerns of what the client is talking about, or is trying

to get enough information from the client (through appropri-

ate reference to what the client is feeling and expressing

and are appropriately timed to the significant cues given

by the client.

Examples :

1.

"These feelings make it difficult for you to get

along with others."

'I I‘

. .
I
s 0“
4 .
. V .
”I
l
'7:
o. a-l
J .3
. ..

I .
‘3
51". r
2

\v'.

As.‘

:
0' "‘
.9
_.-
'
.
I
. ,L .
.4 a"7
_‘ .4.,'
,. -.
a
;|'
I

4'“

B.

-116-

2. "In other words, you feel inadequate and self-
conscious in these situations."°
3. "You really want to be a big man."
4. “Could you tell me more specificallywhat you mean
by this odd feeling about your brother?"
5. "Could you tell me more about that?"
Non-Unders tandigg Resgnses
These are responses which imply that the counselor
lacks an understanding of the basic underlying affective
or cognitive concerns of what the client is talking about,
or are responses which indicate the counselor is making no
attempt to obtain apprOpriate information from the client
from which he may derive an understanding of the client's
underlying concerns. Non-unders tending responses are those
which make inappropriate reference to what the client is
feeling or expressing, are inapprOpriately timed to sig-
nificant cues, or are responses which are made after several
significant client clues have either been ignored or entirely
missed.
Examples:
1. CL: "When he said that, I just turned red and
clutched my fists."
CO: "Some people don't say nice things."
2. CL: "When I showed my mother my grades and she

said they were acceptable, but I should be

 

4
fl. .
.
. . . .
a
.
.
..c
I:
II
0
: v
I. l
.
. 1
v 0
. v . u . r
v , All,
i
L
..
x I
w‘.
. .
.

 

 

III.

.117-

doing better, I felt like quitting college and
telling her to go to hell."
C0: "What would you do if you quit?"
3. CL: "Sometimes I have queer feelings about my parents."
CO: "I see."
4. CL: "When Ivan put his arm about~,me, I started
crying."
C0: "Ybu.mentioned that Ivan likes the classics."
5. CL: "I'm.so self-conscious in a crowd.- (CO: Silence).
I just can't speak to groups because of my nose
(CO: Silence). My brothers always made fun of
my nose (CO: Silence). My aunt said once I looked
like an eagle. Boy, I hated her guts (CO: Silence).
I just don't like my family."
C0: "When you go to college you can get away from
your family.
The Specific-~Non-Specific Dimension
This dimension indicates whether the counselor is dealing
directly with the client's basic problem by confining his re-
sponse to a concrete (specific) one quite relevant to the pre-
sented problem (a core response), or whether the counselor
passes over dealing directly with the client's basic problem
and gives a more general, or non-specific response not too
relevant to the presented problem (a peripheral response).

These categories are more clearly defined as follows:

r
1
a
A
.A.
\
.x
I
.
.u.
.\.
up .
I.
r.
~

.
.
I.
I 0
II.
a
.
. u 1
.
.
.
x.
J.
J
»
.s
.4
.
..
.

A.

i118-

Specific Resppnses

These are core type responses which indicate that the
counselor is handling directly and honestly the basic
problem presented explicitly or implicitly by the client.
The counselor's remarks are keyed to the central aspects
of the problem which the client is feeling and communica-
ting. The counselor thus responds to the underlying con-
cern, feeling, or meaning--the primary message--of the
client's remarks such that his responses zero in and are

congruent with the level of affect or concern being ex-

~pressed by the client.

Specific responses are those which influence the client
to attend with specificity to his problem areas or emotional
conflicts and concerns. They are relevant to these problem
areas or emotional conflicts and concerns, and move the
interview discussion in this direction or bring the client's
discussion back to these tepics. Specific responses are
usually directed at underlying affect, but may consist of
cognitive responses if awareness of affect is not essential
to the working through of the counseling prOblem.

Examples :
1. CL: "I've gotten all A's this year and I still feel
lousy. "
CO: "You're sensing that obtaining such high grades
is not such a satisfactory solution to your

feelings of inadequacy as you thought."

a.

 
 

. _ \I
. .. .
. .
' .. a
on, hm p -
.bJ !
\. . 1r. .
a . \- ..|..
.
. y
.1 Q1...
. .034
.f
.
. o
a .
J
. , . h. , .
. ‘e A
. .
‘
~ — c'
,
. . . .
I I.
. p1
- u.
. a .
r
.A . .. .
u a
.
J
4.
7‘
v
» l
- r
x, . r
.r

.34 , .

I
V
L
p
5
.-
.
I. h
a
‘l.

.
.-
.-
. s a
.I.
.1
VI.
. . .
. fl
« I.
.vlls
I
n
V..
I
0
.na
.
.
.. .
o
I,
1's

(C
.d
a“
, ..).
.
..
a ,
e
..
v
~
.
| ..)
..
If;
I.
.
|\
4. .
o.
.. A
on

V.

B.

2. CL:

3. CL:

4. CL:

CO:

5. CL:

CO:

‘ .119-

"It's a common thing for men to have mistresses
in EurOpe. ‘Why do they have to make such a
fuss about it over here?"

"What's that got to do with your sexual problem?"
"Hell, what do I care about being on.time, my
boss never says a good word about my work and
he didn't do a thing about getting me that
raise." 7

"When your boss doesn't continually praise and
take care of you, you become angry and take

it out on him by being late."

"When I try to make love to my wife anymore,

I just can't perform."

"Ybu still look on your'wife as your mother."
"It's my twenty-second birthday tomorrow. I
hate growing old." ‘

"It's more fun being a little girl, then you

don't have to face these adult responsibilities."

Non-Specific Responses

These are peripheral type responses which indicate that

the counselor is not dealing directly and honestly with the

basic problem presented explicitly or implicitly by the

client. Rather, the counselor responses stay away fromnthe

client and his problem, or at best handles the problem at a

superficial, surface level. These responses indicate that‘

the counselor has missed or ignored cues which reveal the

t

‘ ' ' V: ‘ 7t
‘ '1
6
L J ~
v! .‘
)Wf , , . .
‘.,‘.,.),..,,7
‘. \flj". ‘
. ":1: I
, ' r" :
w. ‘, l.
..
:r
a! ,- 'h‘
.
'Jf‘ ' T‘f'
. Tut. (l‘ '
. r .
T‘“ r .‘ - ..a"
’ ‘. A. o
'. 2‘ '3 H'.’

I A
o‘

a
s
..l
‘.-.
‘-‘
.

~l

(an

a . ,
,‘ , s r' ' ,‘»‘ ,,
. L ‘1‘ L 4‘
)
.} ‘ ‘1.:
.
.2 ‘
.1 . '
a .v‘
- q 0‘»,
. _
.
‘ ,
4 -‘ _ } .I
.
\ \ ~‘ "
.
V ‘1‘ ‘4
’ --
fx " ,C r
s! J“
A ., ..
.., .. ‘
-3 .’ a. .A.
‘ . .l
‘v '. Vy‘
, .
o
l ' ‘
..-.
~': .
o . _ "
If:f.¢( . _ ‘1
- . H
, . 3" ~“ -.
. ‘~ a ‘l
,l 5
O
' I
. — ~ ~ .
J. . , z.’ .
.
. _ ,- .-
"A{l‘.f‘ 1.
.
‘ ~I. ‘0‘ . l-D
x= . J T. '
. .
.. . .. .
I L - a .~ ..
.- f ..a - ~Av
‘I ‘3 (153: _'.‘
v
-ta
, 'A f.
-.‘ J“\- "J

, .
_.j l
. l'
a u‘
I o
0 va-
. i .
... l " '
-. . ,\
. If .‘
, ‘,‘: .
.
'5
« .‘.L
' s‘a , ~
n ‘\
..
L W'
. _ . \f-
, | .
‘ ck.
..
.. J ‘ ‘, ‘- _.
. ,n +3.. 3J'». .1
_....- ....-- _..--. _...-. . ..
. ‘1"..‘1 5,.“ _... . . . ‘,‘
s. I s. .. I .J
‘ 7
a, '. ., r
.: ' l’_ L ‘ .
, < 4.- r \ , .
. p‘; .
.
. r-.~’ .
_. n .. .
3.. . . , .«zu' ~ .1

, e e
I' '- F
' ~ .J.
. , ,

;120.

client's main problem areas. In such responses the caunb
selor concentrates on aspects of the client's Verbalizations
that are of less importance than the main message which the
client has attempted to comunicate. The counselor's re-
sponses refer mostly to cognitive generalizations or super-
ficial cognitive specifics rather than their underlying
concerns, and thus such responses are not congruent with the
level of affect being expressed by the client.

Non-specific responses are those in which the counselor
becomes abstract, intellectual, or extraneous rather than
confining his remarks to the specifics of the client's
problem areas or emotional conflicts. Such responses tend
to be irrelevant to, or deal inappropriately with, these
problem areas or emotional conflicts and thus move the inter-
view discussion in a direction which avoids them. Such re-
sponses encourage expansion of superficial cognitive content;
leads to debate with the client over cognitive or affective
concerns; are typified by excessive response verbalization
and verbal monopolizing on the part of the counselor; and
are illustrated by all sorts of lecturing, sermonizing, and
philosOphizing to the client by the counselor.

Examples:
1. CO: "I've gotten all A's this year and I still feel
lousy."
CO: "I wouldn't feel so bad if I were you. There

are a lot of kids who would give their eye-teeth

 

J. L.,

 

IA

7.

 

2. CL:

CO:

3. CL:

C0:

4. CL:

CO:

‘-121-

for those grades. Feel thankful and count

your blessings."

"It's a common thing for men to have mistresses
in Europe. ‘Why do they have to make such a

fuss about it over here?"

"Well you know it's really not the Christian
thing to do. Anyway, it's wise not to develop
relationships which are frowned upon by our
society. Have you considered locating in Europe?"
"Hell, what do I care about being on time, my
boss never says a good word about my work and
he didn't do a thing about getting me that
raise!"

"Don't you think though, that consistently
being on time will be to your advantage in the
long run? It looks much better on your record
and shows a sense of maturity and responsibility.
Your boss is probably a very busy man. I'm sure
he will make some comment on your work in the
future. And, don't be so disappointed about the
raise. A lot of people have to get by on a lot
less than you're making."

"When I try to make love to my wife anymore, I
just can't perform."

"I imagine that's very frustrating to your wife."

v \

WING.)

‘I".
1

.
.1
.3: .
A ‘-

515.";

.
c;£ ‘r
. .
If.
.
.
I ..r
.‘;_l
. .

V’I‘",

- ' I
' I
.
.
A
a
.
. .
.
S
.
.
‘
. l
v
I . I
I I
O l'
.
.
U,
. .W
. .
.
\u ,
‘
I

‘. ‘
.
.
A.
a
w
. ,1
v

...a

-122-

5. CL: "It's my twenty-second birthday tomorrow. I
hate growing old."

C0: "My lands, child, that's not old. You're real
young yet. You have your whole life ahead of
you."

IV. Explorative - NondExplorative Dimension
This dimension concerns the extent to which the counselor's
response permits, encourages, or elicits further exploration on
the part of the client of any aspect of his problem areas or
affective concerns. These categories are more clearly defined
as follows:
A. Egploratopy Resppnses
Counselor exploratory responses are responses which
first, indicate that the counselor has recognized some por-
tion of the client's basic problem and, second, has made
some verbal reference of this back to the client in such
a way that the client is free from any defined or ”limited"
structure in his response, and is thus free to respond to
any degree and depth to the counselor's reference that he
'may choose. In addition to the verbal response, the counselor's
whole manner creates an atmosphere for further exploration.
An exploratory response, then, is one which concerns some
aspect of the client's basic problem and at the same time
permits and encourages the client flexibility and freedom

in his response. Such responses are often open ended and

V.
. t
.
I v.
.
.w ' it
. a
_. ..
._
‘ a. .1
.[
In
..IJ .1
I.
.3 ok
9......
. . ..« .
. 7..
.
A K:
.14
A o
. I.
.. 4 .
. \J "u.
4. u \
)u .
. i.
.
I.
.1
IA
. ...
. ‘ a.
u”; ,.
v
.7} 4 A .
. ..
_. . . »v ’A
I.

..a
0!).

,,._
A.
.IJ.
.. -

 

.
.
r
. I
.‘
r
.0.
..
.
..
v
f
..
H.
.
.,.
LI;
5‘.
r»
J
.
f

B.

’3
.12)-

allow the client to explore his own feelings and to expand
upon them.
Examples:

1. "It seems that your anger is really directed toward
your father."

2. "Perhaps your need for such high esteem has some-
thing to do with your choice of medicine as a
career?"

3. "Maybe we should discuss a little more these strong
attitudes about sex."

Non-Exploratogy Resflnses

Counselor non-exploratory responses are responses which
indicate no recognition of the client's basic problem, and
thus no attempt to search them out further, or responses
which structure or limit the client's response. A "yes"
or "no" answer or a "pat" answer is often implied by such
responses. Non-exploratory responses are those which give
the client little opportunity to explore or expand, or to
express himself freely.
Examples:

1. "What is your average in English so far this year?"

2. "That's a nice place to study, isn't it?"

3. "What was your uncle's middle name?"

a

C.

I
J Ir
,7
'l'
.-
.
",1.
-\
-‘ .'¢'
_ .. .
.‘I"
‘ r g‘ - '
v‘ ‘ ‘
, t
. J 0 .
, ,__ .. “_...- .--... --v- ,...- «4....-- . —
. , .. 00'
.
r " “' L)
. . _. .
O
.' ‘._,
64's.
I o l‘
;_L..
‘
, ,.. .
, .s. , .
. ; “ r v.
’ -- . : . . "(‘7 F ‘
“.2 u. . ..1:. :3 ..~ J
I. .I. . ‘ '
' “ ' n .a .I
J. Li. ..:_:. ., " -
a
l‘ C
' . .- , s . .~
..l!‘ [,1 .L; ', Y.‘ —i
l '. > I p

 

 

-12

COUNSELOR RESPONSE EVALUATION

This evaluation is made independently of the four counseling
dimension ratings and is a dichotomous rating whereby the coun-
selor's response is marked either effective or non-effective.

This rating is not based upon an evaluation of the client's response
to the counselor's remarks but, rather, is based solely upon the
judge's professional impression as to how appropriately, from a
counseling vantage point, the counselor's response deals with the
counselor's verbalization and thus contributes to the development
and maintenance of counseling progress. Counseling progress is
generally defined as concentration of discussion on basic problems
and feelings of immediate concern to the client such that there
develops for the client an increasing undistorted awareness of their
influence and consequence on his behavior and relationships with
others. In this regard, the judge decides globally--taking into
consideration the counselor's total manner and behavior, including
nonverbal--the effect the counselor's response will have or could
have, toward permitting or encouraging the client to move in a di-
rection of counseling progress.

If the judge evaluates the counselor's response as effective
he can either check a 4 or a 3 under the heading "effective." A
rating of 4 indicates that the judge's professional impression is
that the response is about as "good" or the "best" possible in terms

of counseling progress. A rating of 3 indicates that the judge's

a
a].

W

.

.11.

I.

, \‘
. , I
~ -.«

v "-

 

.. ~ ‘ ‘ _ . "

._ - l ,

v - ,

y ~ ‘ ~
r , ' ’ .
' ..I
.
t' . .. .
. .\ .'

 

 

. I V v

. »' L K '

.

I . .j A I . r)

.| A’ p - I i ‘ L \ .

. . ‘ I ‘v N

u 3 ~ g (I. _' ,. ' ‘ . l, .' ‘~ ‘

I

, ~ ( . ‘ ‘.

‘ . a; .J ‘ ' ‘ ' ‘ .

. A . . .

. ' I y I. ‘ .1 ‘

if ”4 .-. .J. .v I. s“,

. l ’ ‘

- ' J, l

1 ‘l. .N'(,'. ..’

f . 4 .I ~

. ' .1..\ - ‘ "I.

0' \“’

. 'J ; “

.. - : -,-i f (\ (j-.-,',v~\ , VA >-‘-

x' d- .‘ -- ‘ v .-‘ ‘J. ..a , ' ev .‘
. ‘ ' I
A ‘1‘ x '
d v
\v
. w
'9

 

‘...4

.
l
.l,
e

professional impression is that the response is effective toward
some counseling progress but it was not the "best," that is, a

more effective response or communication could have been given.

If the judge evaluates the counselor's response as non~effective

he can either check a 2 or a 1 under the heading "nonveffective."

A rating of 2 indicates a response which contributes in no way to
counseling progress (sort of neutral) while a rating of l, in
addition to this, indicates also a complete lack of understanding

or concern regarding the client's problem situation, or is a response

which is definitely detrimental to counseling progress.

APPEIDI K D

CIARIFI CATIoi-I OF FREQUENTLY ocwsrcme

RATING PROBLE‘I-TS AND QUESTIONS

-126.

Right Side and Left Side Scoring Tendencies
For a given counselor response, judges should be particularly
cautioned against rating the first dimension and then marking all
the remaining 'nensions to follow suit: e.g., marking the first
dimension cognitive, and then the remaining dimensions nonunder-
standing, nonspecific, nonexploxatory, and noneffective, or
marking the first dimension affective, and then the remaining
dimensions understanding, Specific, exploratory, and effective.

Each dimension.is to be scored independently according to

category definitions.

Possible Counselor Dimension Scoring Patterns
According to the dimension category definitions, the following
scoring patterns are possible:

1. A response can be cognitive and still be understanding,
specific, and/or exploratory.

2. A response can be affective without being understanding,
. specific, and exploratory.

3. A response can be affective and understanding without
being specific and exploratory.

4. A response can be understanding and specific without
being affective or exploratory.

5. A response can be affective, understanding, and exploratory
without being specific.

A reSponse, however, can never be exploratory without being

understanding.
Recuggipg Judge Qgestions Concernipg Dimension Category Definitions

 

Although it is not possible to anticipate the nature of all

judge questions concerning category definitions, the important

..l'

(0

recurring ones are presented here along with the answers of

clarifi cation:

c1)
so

&)

L;

"If the counselor demonstrates affect himself in his response,
is tlfis scored affect?"

"1'0, not necessarily. The main criterion is whether the
counselor is referring back to some affective aspect
presented by the client. The important thing is what he
is referring back to, not his manner in referring."

7 fhat if the counselor ma (95 reference back to affective
aspects or :u 1plica“ ions in the client's remarks which are
not justified b3, the client's remarks or manner. Is this
scored affect?“

"Yes. However, if you feel the affective reference is
inappropriate you would also score the response nomnderstanding,
nonspecific, nonexploratory, and probably noneffective."

‘Uhat if the counselor refers to or discusses the feelings
and concerns of third parties which are in no way related to
the concerns or involvement of the client. Is this scored
ffect?"

"No, such a response is scored oogrdtive. To be scored
affect, (Bantam of feelings and concerns of third parties
must be directly related to the client and tied in with

the feelings and concerns of the client."

"Are all questions by the counselor to be considered as
seeking understanding?"

“No. The question has to be related to clarifying for
the counselor some aspect of major concern, explicitly or
implicitly implied by the client. Random questions asked
by the counselor from the 'top of his head' to alleviate
satiety, to continue a cognitive discussion unrelated or
leading; away from the client's rain concerns, or which in
any way soon unrelated or inappropriate to what the client
is trying to equress, are scored nonunderstanding."

”If the counselor fails to respond to several important
affective client leads but finally responds to a lesser
type of client lead, is this scored nonunderstanding?”

I] In

"3"OS . cc

/-7¥ /'130..

::If the couns selor nail

nt ins a cognitive discussion with the
client when the clie1 t'
we

remarks are implying more affective
nonundcrstanding?”

concerns, is this see

n
L».
5

”Yes. Jhencver a counselor maintains a cognitive discussion
0"

er festive concerns, his res sponses are scored nonunderstanding.

"Ehen is a cognitive response on the part of the counselor
scored undcrs handing?”

”Jhenever the cognitive content to which the counselor is
responding seems to be the important concern of the client
and is not a coveraup for underlying affective prOblems or
concerns. This also applies to the specific and exploratory
categorieS. Respon cs are not scored understanding if it is
obvious that affective problems are of more concern to the
client than c0gnitive content, but the counselor continues on
a cognitive plane. In such cases the counselor's response
is also nonspecific and nonexploratory."

"Can a counselor's response be specific without being
understanding?"

"Yes. A client can present point blank to the counselor a
central concern of his and the counselor may comment on it
without demonstrating any understanding of its significance
or may comment on it is such a‘waj that there is no further
pursuit of it."

"flhat is the main criterion for specific?"

"When the counselor's response stays at the same emotional
level as the client and.it is able to zero through the client's
verbage right to his central concerns without being side-tracked
by less important affective or cognitive concerns.“

"If a counselor's response is such that it is possible that
the client could respond in any way that he chooses, is that
response scored exploratory?“

“he. For a counselor's response to be scored exploratory it

is essential that the counselor's response not only permit the
client to respond to arw'length or any depth that he chooses,
but it must also reflect that the counselor has some under-
standing that there is some aspect of real concern to the
client and his response is thus encouraging further elaboration
of this concern. This concern of the client may be vague

and still not specifically identified by the counselor, but he

“2-
w

-131-

understands there is some kind of concern there and is seeking
further orflorntion or clarification of it.

”Iln not quire clear as to the difference between an
effective response scored 4 and one scored 3,"

“1n effective response scored 3 is one in which the counselor
is appropriately with the client and.is helping him to
express more clearly his feelings. An effective response
scored H-is tl's also, but in addition it is one which
demonstrates rather exact appropriateness and timing such
that the response vividly reflects back to the client the
essence of what he is feeling.“

APPEI‘IDI X E

STATISTICAL "t"-TEST ANALYSIS COMPARING
THE A-V SCALE DIMENSION SCORES OF THE SCALE
DISCRII-‘INATION STUDY'S FIVE PH.D. CANDIDATES

WITH THOSE OF THE FIVE M.A. CANDIDATES

Affect-Cognitive Dimension

Ph.D. Candidate Deviation Squared
Counselors Scores From Yean Deviation

29 -6 2 38.hb'
no 1+.8 23 . 01::
2. 2 4.81;
8 . 614-

8

C4 f.) U G L‘-
VV »-’\/V
k»)
S
o

2.8 2. 2.81;
Sun 176 74.80
3 5

.1— 35.2 variance (821): ZQ.8 = 14.6
5

 

 

 

K.A. candidate Deviation Squared
Counselors Scores From Yean Deviation
(B) 26 4.8 23.024.
(E) 2 -1.2 1.1m
(F) 20 —1.2 1.1%
(H) 20 -1.2 1.144
(I) :20 -1.2 1.44
.1U1 106 28.80
' 5
§% 21.2 variance ($22) = 28.8 = 5.8
5
t: 331-332 = 35.2-21.2 2 14.0 = 14.0 = 6.2

 

 

 

 

 

"'—

0 2 “14.6 + 5.8 (g) in].
S l + S 2 %: +-;EV> 2 b

A - ‘4

0.1

degrees of freedom (df) =

»4

~ v o _ _ o
.l + ;.7 - t1. "" lo - 2 "' v

w

9”.
Lu._/

1
3.36

significance at .05 level
significance at .01 level

firm-fl

Understanding-Konunderstanding

Ph.D. Candidate

 

 

Counselors Scores
(1.) 31;,
(C) no
(D) 40
(G) 37
( ) A0
Sun 191
5
ii 38.2
1:.A. Candidate
Counselors Scores
(B) 24
(E) 20
(F) 21
(n) 20
(I) 20
Sun 105
E 5
3% 21.0
't = 38. 2 - 21.0 = l .2 =
V 5.8 + 2. (é) V1.6
(if = 8

.gLeni"ieaneo at .05 level
sj,3nifieance at .01 level

K») ‘~J)

VON

J

Dimension

 

 

Deviation Squared
From Kean Deviation
-4.2 17.64
1.8 3.24
1.8 3.2M
-1,2 1.44
1.8 3.2a

28.80
9
Variance (8” ) = 28.8 — 5.8
1 r
J
Deviation Squared
From Kean Deviation
3.:) 9.0
-1.0 1.0
0.0 0.0
-1.0 1.0
-l.0 1.0
12.0
variance (322 )= ___9_= 2.4
5
12.2 = 13.&
1.28

H

O\

 

Specific-Ionspecific Dimension

Ph.D. Candidate Deviation Squared
Counselors Scores From Kean Deviation
(A) 27 -0.0 70.56
(J) “v 3.6 ..a.
Sun 177 105.20
v 5
L1 35.4 Variance (321) =10§. = 21.0
“ 5
I-I.A. Candi<.‘.ate Deviation Squared
Counselors Scores From Kean Deviation

21 .8 60

(3) .

(E) 20 -.2 .04
(F) 20 -.2 .04
(H) 20 -.2 .04
(I) 2 -02 col-L
Sun 101 .80

? 20.2 variance (322) = .8 = .2

Kn

t = 35.0 - 20.2 = 12.2 = 15.2 = 7.4

 

 

 

2.05
21.0 + .2 2 4.2
2 '3
df = 8
significance at .05 level = 2.31
significance at .01 level = 3.36

'7
-.L

A f
. I
3) Q-

wloraeo"‘— 'onozrp101atory Dimension

 

 

 

Ph.D. Candidate

Counselors Scores
(A? 27
(C) L50
(D) 37
(C?) 38
(J) 40
Su. 182
7:1 36.4

L.A. Candidate

Counselors Score
(3) 21
(E) 20
(F) 20
(H) 20
(I) 20
Su. 101

5

31, 20.2

t.:364-20.2=16.2=

\/ 221+ + ,2 (3) 4.7
5

CT)

df
significance at .05 level

significance at .01 level

 

 

Deviation Squared
From Kean Deviations
-9.4 88.36
3.6 12.96
.6 .36
1.6 2.56
3.6 12.26

117.20
variance (821) = 11 .2 — 23.4

Deviation Squared

From.Mean Deviations
.8 .64
-.2 .04
-.2 .04
’02 .0“
-.2 .04
.80

Variance (S72) .8 = .2

5

Effective-Yoneffective Dimension

Ph.D. Candidate Deviation Squared
Counselors Scores From Kean Deviation
(:1) 1158 —lO.3 116. 6L;
(C) 62 3.2 10.26
(D) 58 - .3 .6@
(s) 60 1.2 1.44
W .611. 7-2 .2322.
Sun 29% 180.80
1 5
ii 58.8 Variance (521) = 180.8 = 36.1
Deviation Squared
X. A. Candidate Scores From Dean Deviation
Counselors
(3) AZ 6.9 2a.01
(‘3) 37 "' cl .0].
(F) 3]- “601 3702]-
(x) 38 .9 .81
(I) 28 .9 .81
Sun 186 62.85
- 5
‘2? 37.1 Variance (52?) = 62.85 = 12.4
' 5

t = 58.8 q_37.1 = 21.2 = 21.2 = 6.9

3.12 _—
2 5

 

 

df = 8
significance at .05 level = 2.31
significance at .01 level = 3.36

APPENDI X F

STATISTICAL ITEI AI‘EALYSIS OF THE SCORES
OF THE SCALE DISCR'U-E'LNATION STUDY
BASED UPON HOYT'S COIIPUTATION OF

RELIABIIITY BY ANALYSIS OF VARIANCE

..'L"" 2,-

‘.

39ft's Statistical Anallgis for Determining haximum Likelihood Estimate

of Reliabilig (Internal Consistenc ) and Interjudre Reliability

 

 

QQ. Sum of Sguar es garlangg
K ,C 6’? ? 9
Counselors d—l SSR = gig .. @_ S R = DDR /R-1
(ROMS) C RC
Raters C 5 9 £5 2
1 ‘L S ‘ - H ‘-
(Co—wn > C-l 58¢ _.LRiL 1.19.. S c 95c/c-1
..\ - n V " 2 4"
error (2~l)(c-l) So. = SST - as. - SSC 8 a = Saw/(a.1)<c.1>
‘36 fifc
Total RC-l SST = $2 X2 - 6 x 2
RC

(1) Kaximuvaikelihood Estimate 2£_Reliability (Internal Consistency)

The reliability of the average ratings or the total score on
the test. 2 2

rtt = 00 R - 53 E

 

2

SS R

(2) Entorjudge Reliability

The average reliability of all the raters or items when between-
rater variance is to be removed. This is done when decisions
are based on averages of complete sets of ratings from all
observers or ratings.

 

SS
E
r " SS -
tt R _Cd
SSR + SSE

(3) Interjudg e Reliability

The average reliability of all the raters or items leaving in
the between-rater variance. This should be done when decisions
about pupils are based on a single score or rating which may
be given by different raters.
830 + 331

a

 

R ..
C-l
89R + (SSC + SSE)

'1 J! I".

“1" U '-

-4-

.nitive Dimension

£3

Affect-Co

 

Sum

 

ters

 

Response

Counselor

QB/OQJQJ/O/OBQJ/O/O 531J/O 36.4.46

1121129~1122111212112

112112n41122211212122

2121122119~221_1210~212

O

1234 ..)/0700901 Add)

fry/on/nonynu
l 11.11 11192

1

A

nu. 3h». 334/0536 3.414554 (.2333

11111221121111111111

21111122121112222111

11211122121222212111

1234567890HZAJQVJ/O7m090
1 111111112

Up

K8 /0 /O /Q /0

22222

9... 2 n4 on A4

Op 0... 7w fig 2

1212.02 5

C

Sum

4m-
Raters

 

 

(Aff.-Cog. continued)

6/0/06/O/O/O/O/O/O/D/U/O/O/O /036/O/033/D/O/0u45/O/O/D/Oh~.hw.3/O BOJQJWrBQ/aJQJQ/B

0L2222226¢20422222 21222112221222221112 1111111111

2222229~9~2222222 2122211222222227~2212 111211—1111

2220.~2229~o/~?~22222 2120~2112221122221112 1111111111

8
S
n
MUHCU 750 D!) O _.l n/H 3
a 1L11

\.

a) nu 11n4 anw :J/o Ozno n/nu
1 9w 1*

4,

(.267.
1.11

n09
11

-

700 O/ O l 9‘1 thw fJ/D n/OU 0/0 1 2 l
1112 111 3

Re

Counselor

(Aff.="og. continued)

Raters

Sum

 

.l.

 

3333333333

1.11111111111111111.

1111111111111111111.

1.111111111111111111

m
m
0
PH 23hr 5,0 73 90
MW 111111112
an

Counselor

Nu. 3333.33a330J333333333

llllllllllllllllllll

119~11111111111111111111111.111111111111

1111111111111111111111111111111111111111

l 2 3:4 5/0 700 Q/

10

6 5.4666 3.4 5366/0/06

Oglln/undnéllZlOLA/gn/JZZ

OLZZZOLZlZZlA/ezn/uA/LZ

29~19~221111n¢0~d¢29~

l 2 34 126 73 0;, O l 2 3h. 5
l l l l l l

a}

l

-l

(Aff.-Cog. continued)

(1
01.173”!

Jhtors

 

 

Counselor

55/0 (0. 6

22222

04 0;. ad Ad 2

11717~?~ol

e
S
n
0
.01/0 700 9 O
"s1_111111o
»O.
. .

a333330JOJ33333333a33333

111117.11111111111111

11111111111111111111

11111111111111111111

l 2 9).” rJ/o 7.3 9 O 2 3b. 56 78 90
l «lwlqlwfixulalfllulfin/w

30-41/02?)OJ333A)aJ33¢J30\/n)33nfl/

llllllllqlmlwllllllllllwl

1111111111111111111_11111111111111111111

lllllfilmlllnlullfilllllwl1uqlm1

l 2 a211,”. 5-70 700 m7 0 l 2 3.1... rJ/O 7. a H, O,
“I.

.71“ 1A.!“ 1 1L 1.” .1-..

11 7.

l_

O
n a

_1/1':

AT. ri-

\

(Aff.-Cog. continued)

 

 

 

 

 

Raters
l 2 _3 Sum
Counsrz-CLor .{csponse
J l 2 2 2 6
2 2 2 2 6
3 l 2 2 5
£1, 2 2 2 6
5 2 2 2 6
{3 2 2 2 6
7 2 2 2 6
8 2 2 2 6
9 l 2 l 4
10 2 2 l 5
ll 2 2 2 6
l2 2 2 2 6
13 2 2 2 6
14 1 2 2 5
15 1 1 2 4
16 2 l 2 5
l7 2 2 2 6
13 2 2 2 6
l9 2 2 2 6
20 2 .2. .1. .2
Sumz 281 292 276 849 Sum2=3975 (849)E=720,
Sum 443 476 428 g x‘=134-7
(Sum)2 240,401
Source of
Variation df Sum of Squares (SS) variance (82)
Counselor 3212.- 220,801 = 123:7(ssR) 122.2 = .62(32R)
ReSponsos 199 3 00 ‘ 199
Raters 2 240,401 - 720,801 = .7(ssc) ‘1__ = .35 (32C)
200 00 2
Error 398 145.7 - (123.7 + .7) = 21.3(383) 21. = ,05
39’
Total 599 1347 - 720,801 = 145.7(33 )
OO ‘ T

 

 

(1) Internal Consistency = .62 - .0 = .212
._ _..:2:F_J§

(2) Intorjgdgg Reliability (between-rater variance removed)

= '62 " 005 = 07,2
.52 + TB-T) (.175)

 

(3) Intgrfiud 2 Reliability (between-rater variance left in)

= 127.7 - (~Z + 21.3 = .729

2

 

127.7 + (.7 + 21.3)

:7 1"“.1
x) \ I. ,

ndingrDixension

n
cl

ugLonunderst

Understand'n~

 

Raters

 

Sun

OJ

 

02189

J11 2 024 5/0 73 90 l 2 o3 .

(

‘
0..

R

Counselor

46/0 556652706 5336.4 55.45

1202292722212721112a42212

lZZlZZlenLZleleln/NZ

20920212929~1020L211212211

.HN. .r
1:..Iulwl_l.l_ql~

64.4 333.4 55533333)...Y 3333

2.111111111111119222111111111_2211111111

211111221111111111ull

222l11122211111113_17*

123.0(5/0780/

lO

,0 6 IO /0 2.0

2 2 0p Ad 0;.

972 2 22 d;

7” 9H 2 n4 n4

1 2 c.,.hw r)

(Undersb . -Zfonundcrst . cont inu ed )

Sum

2)

 

 

O
S
1

.0.

3

.. ..a

S

I
am
TI.

01‘

1:“
0
OJ

C011 n

/O/O/O/O/O/O/O/O/O/O/O/O/D/O/O

22222229~2222222

n4n4222222222n4222

A.L222229~22222222

go 7.8 90 l 9 9:74 .1340 7.8 O/ O

111. lljxllaimnina

/O/O/O/D/O/O 566666666656 5/0

22222222222222222222

2222n42122222222222169

222220n2922222222219~2a~

1 Op 31.0.4. 5/0 7nu Q/ n

9
19

U

OHZBbrS/nv?
1 111111

1.8

{.4

c3 3 a) .4 3, 3 3 23 a) a)

lailn/“llwilll

llllllllll

lqilllllalflwlnav.‘

l 0‘, 3-4. rJ/U 7‘09 0/ AU
1“

Pg

-12!

(Underst.-Honunderst. continued)

Raters

9~

 

1

3333333333

11111111111111111111

1111111111

 

11111111111

dosgonsc
ll
12
13
l4
15
m
W
16
D
20

Counselor

363333333333333QJ3333

12111111111111111111

12111111111111111111

12111111111111111111

l 2 3.4 5/0 7nUQ/m

1 02 3b. ”..)/0 7nv 0/

6/0/7046/0 rJCJ/O 55/06/020

2221220204202729~2292

222222192219227~29~

22219~Og 0....12nalnd2922

01 0” 3nd,. .LJ
11.!“ 1.1.11”

er; .-

-l

inuod)

J.
v

(Undorst.-Konunderst. con-

Raters

9n”
5,. A, .

1L.

 

66/066

 

22222
22222
22222
G
S
O
Dvonfooo/no
311.117.?"
p
u.

‘

ounselor

'1
J

(

3333a BBBBQ/BBBQJAJQJBBOJOJ

1111111111111111111111111111111111111111

1111111111111111111111111111111111111111

1111111192111111111111111111111111111111

.l. 9~ 91/4 CJ/U 7.0UO/O 23 Y5/O
an“ 1 l l

333333333a233BnJGJfiJOJAJBQJ

1111111111111111111111111111111111111111

llllllllllllllllllll

111111111111111111111111111111111111111.

f 3/0 7nU 0’0

123:“. 5/0 700 0/0 3rd w
l 1!— 7. l .1. w; A,”

2
.|_ l

-1 171,0-
—— - /

(Caderst.-Ionundorst. continued)

 

 

Waters
1 2 _1 Sum
Counselor Rosuonse
J l 2 2 2 6
2 2 2 2 6
3 2 2 2 6
a 2 2 2 6
5 2 2 2 6
6 2 2 2 6
7 2 2 2 6
8 2 2 2 6
9 2 2 2 6
10 2 2 2 6
1.1 2 2 2 6
12 2 2 2 6
13 2 2 2 6
14 2 2 2 6
15 2 l 2 5
l6 2 2 2 6
l7 2 2 2 6
8 2 2 2 6
l9 2 2 2 6
2 .2. ..a. .2. .6...
Sum.) 297 290 299 886 Sun12=4318 (886)2=781+, 993
Sam; 491 470 U97 25'X2=1H53
(Sum)2 261,710

 

Source of
Variation df Sum of Sguares(SS)_ Variance ‘82)

Counselor 1+ 18 - 8b 96 = 131.0 1 1.0 = .658
.25.. 2'36%" .2_._.

 

 

 

Responses 199 199
Raters 2 231,710 - 234,996 = .3 .2__ = .15
200 00 2
39'
Total 599 1458 - 284,926 = 119.7
' 00
(l) lgyornal Consisteugg = .658 - .046 = .220

(2) Intorjuige Reliability (between-rater variance removed)

= .653- .ous = .316
0 6—58 + (3‘1)(o 0746)

 

 

(3) Intorjugge Reliabilitg (between-rater variance left in)

= 131.0 - (.3 + 18.u) = .811
2
131.0 + (13 + 1835)

 

Raters

 

CifiC-llonspocific Dimension

 

?)0

__g

r1
.3

35643663366433534434 unnflJq/333353333333330J3 6/06/06

11221221122111212212 lllllllllellllllllW+ 22222

12211221122211111111 11111111111111111111 22222

192211221122111..le111 2211.1111211111111111 20~220N

 

e
S
n

0
Pl 2 3h. 56 78 90 1233. 5/0 78
MW 1.1.11.1qlalwigl

MUN 12 3.456 789 1231:.er
r

10

Counselor
A
B
C

(Spec.-Uonspec. continued)

Raters

Sum

 

66/06/0666/0/06/0666

20L22222227~22222

222222222222222

 

20422222220~9~222n4

Rosnonoe
6
7
8
9
O
2
3
'+
5
6
7

Counselor

6566/043u/O66/D/0/D/D/Ouu56

921.9292921.121292v272929292929212129292

29.2222le222222222le

229~22112222205222110~04

l 2 3.4567890 1.72314 56 73 90
l... ljfillllez

33?;3333nfl/033

1111111111..—

1*111111111

1212121.1_1212121212

1 09 «3.4 5/0 7.00 gm

«nu

-152-

(Spoc.—Fonspoo. continued)

Raters

Sum

 

 

esponse

I)
-~

Counselor

3333333333

11111111111111111111

1111111111111.111111

1.111111111111111111

11
12
13
15
16
17
18
l’)
20

33333333q/OJ3333333333

1111111.111.11111111111111111_111.111111

llllllllllllllllllll

llllllllllllllllllll

11ohqynw:JAUnroonynunu02QJthJAovlnoo/no
1 111111112

6564/Oubr 55.4 5/0 5/06

21212122091229~22

9fi9292929211117292119292929202

ZZZlZlelOQlZlOun/u

l 2 3U. 5/0 7nU 0/

AU

0
l

u

2 6/.
1171

hr

1

p.)
l

7 L”?
””‘J/H.

(Spoc.-Uonsnec. continued)

Raters

 

 

Counselor

Response

6/06/06

22222

22222

222292

17
18
1

20

16

32’30J3333333333333333

111.1111111111111111111111111111111111w1

11111111111111111111

11111111111111111111

123+rJ/0 78 O/OJRU

ll

33333333333333.333333

111111111111111111111

111111111111111117111

lllllllllllllllwiflllll

1231.». 5/0 78 0.,OJZ3L15
11 1111

l {3
17
1d
19
2 O

(SDOC.-Ionspoc. continued)

 

 

 

 

 

Raters
l 2 3 Sum
Counselor Response
J 1 2 2 2 6
2 2 2 2 6
3 2 2 2 6
4 2 2 2 6
5 2 2 2 6
6 2 2 2 6
7 2 2 2 6
3 2 2 2 6
9 l 2 2 5
10 2 2 2 6
. ll 2 2 2 6
12 2 2 2 6
13 2 2 2 6
14 2 2 2 6
15 l l 2 4
l6 2 2 2 6
17 2 2 2 6
3 2 2 2 6
19 2 2 2 6
20 .2. .2. .2. .6.
Sum. 270 280 281 839 Sum2=3897 (839)2=703,921
31m" 1+3!» 1:40 1:43 g X72131?
(Sum )2 234,645
Source of
Variation df Sum of Squares(SS) ‘ Variance (82)
Counselor QQQZ - ZOE3221 = 125.80 12§.8 = .632
RGSponSGS 199 3 00 199
Raters 2 224,645 - 0 21 = .03 ‘;92’ = .015
200 00 2
Error 398 143.8 - (125.8 +.03)=17.97 12.92 = .04
3 0 5
Total 599 1317 - 20%;.921 = 143.80
00
= .222

 

(1) Internal Consistengl = .632 - .045
. 32

(2) Interjudgg Reliability (between-rater variance removed)

= .6 2 - .045 = .01
.632 + (3—1)(.045)

(3) Enterjudgg Reliability (between-rater variance left in)
= 125.80 - (.03 + 17.97) = .812

2
125.80 + (.03 + 17.977

Sum

ry Dimension

A

Foncxnlorato
Raters

 

7?:933101‘atory-1

Ll

U‘5ébkpUr/O/D33/0/0433.5h~..4h~.3v4 “43333335433333.430233 5/0/05/0

110227.2le22111.222le 11111111221111121111 292929.72

1.2211221122le111111 lllllllqillllllllllll 12222

 

02221102211nan/2117.21.11.11 227_llllllellllllW.ll 22212

, O 1:" 0.. OJU. r)

12..-.45678901234567390 1234567890uo,.34567 O
1111 1 1 1 1 1,.

l «I. at” l a!“ «in OH 1 .|_ _.I

18
A

R0 sponsc

Counselor
A
B
C

'J
.I,u

\ u
_j»

_l‘ '

Raters

 

n:91 . -l Tone:-:p1 . continu ed )

.‘

LL:

(

/O/D/.U/O/O/O/O/O/O/O/O/U6/D6

642222220u220~2222

222222222222222

2229.. u26¢n¢7~229~9~29~

 

8
8

RI.
1

O .
0.6 780 0/0 2 oJuUr wag/O 7mg n90
MW 1711~lllfinnlql~ng
.3“

lor

Counsc

666664446 56666664 556

222221120222222221222

22717~0921120922222229~12

22222121212222221122

l 2 3.4 4&6 7.8 90 u 2 6 78. my .
_l

ther
1 111111

20

QIBBQ/BBQ/QJBB

1111111111

1.1.1.1.1.1.1.1.1.1.

1111111111

1 2 3b,. r36 7.8 9m.

E

-157-

(Exp1.-Ionexpl. continued)

Raters

Sum

 

 

Response

Counselor

3333333333

1111111111

1111111111

1111111111

33333333333333333333

11111111111111111111

11111111111111111111

11111111111111111111

1234567890u234567890
1 1111111 2

666.46 5:4 5555/0 56/0

222122227~7~22222

222221122127~2022

202212~21112121_022

1 2 «)4 56 78 90 2 «)4 ..D
11 ..111 «11

(Exp1.-Nonexpl. continued)

Raters

 

 

66/0/06

22222

22222

22222

wcsponse
m
N
18
D
2

q

Counselor

33333333333333333333

11111111111111111111

11111111111111111111

11111111111111111111

1234567890fl23 567890
1 11 1111112

M

H

33333333333333333333

11111111111111111111

11111111111111111111

11111111111111111111

11.253n4p3/0n(oon7nu
1n

'7 f‘
— 5‘:-
._ /

(Exp1.-fionoxplo Continued)

 

 

 

 

Raters
l 2 3 Sum
Counselor Tesponse
J 1 2 2 2 6
2 2 2 2 6
3 2 1 2 5
4 2 2 2 6
5 2 2 2 6
6 2 2 2 6
7 2 2 2 6
G 2 2 2 6
9 1 2 2 5
10 2 2 2 6
11 2 2 2 6
12 l 2 2 5
13 2 2 2 6
14. 2 2 2 6
15 1 1 2 u
16 2 2 2 6
17 2 2 2 6
3 2 2 2 6
19 2 2 2 6
.2. .2. .2. .62.
Sum 276 278 291 845 Sum2=3933 (845)E=714,.25
81111120 428 434 473 g X“=l3 3 5
(Sum)“ 238,141
Source of
Variation df Sum of Squares (58) ~ variance (32)
Counselor 3223 - 14 02 = 121.0 121.0 = .608
ReSponses 199 3 00 199
Raters 2 238,141 - 214,025 = .70 :22, = .350
200 00 2
Error 398 145 - (121.0 - .7) = 23.3 22.? = .059
39

Total 599 l - 14 02 = 145
600"2

(1) Inter-mil Consistengy = .608 - .252 = .903
‘ .608

 

 

(2) Interjudgg Reliability (between rater variance removed)

=.608-.059 =. {'6
.603 + (341711059)

(3) Intcrjudgc Reliability (between rater variance left in)
= 12.1 - (.2 22.01) : 07 22
121 + (.7 + 23.37

0

-160-

Effective-Koneffeetive Dimension

Raters

 

 

 

301186

‘2‘) .
“Cb

Counselor

67n0 77mO/6/099/0550/78 7/0 7.

91nd «)33hW3n/123G222 233332 3

01.227123322332113271222

91. n)3r/~ 2339;. 20/322 230.1371 2 9..

7.1013675/07n0O/O 0* GJ 315/0 7nd 0/0
111.1. ;

1.11.13

A

8 76/0667688666/067/0555

222220921332229~23229~2

21?.727.71711Jq/91919171912191o2929.0192

a1,n1.n2 02 02 on 02 n2 51.51.02 02 02 n2 o2 o2 02 11 11 11

lOLOJhWKJ/O7nUO/0n21/ r
l 1111.

17
13
19
23

11'
15
16

an

10
10
10

9
10

QJ o) 3 3 «J

A) OJ 3 a.) 3

1..“ 9” OJ 71”. r)

Pg

—j. 6]_-

.f.-Koneff. continued)

-fifl
0'
(1

Raters

Sum

 

 

34 34 3444 34 3444 3
44 3333433333334
34 34 344 33333333
8
s
m
.016 7.3 90 1. 2 31.1. rJ6 78 90
MW 31711111111116“
F
R

Counselor

9990067799902u996889
11 ll

33OJ332 2 BBAJOJU.“ 3332 333

333.4422 2333344332323

330J3n32 32 91/333414 332 2 33

123456789

10

5666 333/0/0/0

22221111626291

22221111112202

loan/10111191912

l 2 QJU. 5/0 7nU 9m

Wu

(Eff.-Uoneff. continued)

Raters

Sum

 

 

)OnSB

Ros;

Counselor

66666/06/0/06

9wn49~0L29~2227~

9;.0420420422204

n4 «4 2 2 2 Ad on Ad ad 0L

nwcjnwnwnwnwnwzJAchnwhw:chAuchjgucjnw

O~2111112221111222211

1*2222222211122211222

111111112227~220u22221

.l 9” 0.4M- EJ/J 7nd Q/

10
ll
1?
13
L

5
6
l7
9
O

90/0/7987000/0u00 9

OLOO
111
3330.." 33333334 330)

333332 2 332 3.4 bf 33

3332 OJQJn/w 2 3n3néhw «QM/w. a)

l 2 a)” 5,0 78 9 O u 2 n.2,»- r)

(Eff . -Z?onoff . continued)

 

Sum

3

 

9990/9

33333

33nJOJ3

023 3.?J3

l.

”.os‘mnso
16
17
n
J
19
20

Counselor

6556/0 5.451266666666665

2222211110L2222222222

2112042221222220u9~2221

22222212122220~222222

al— 2 3h”. 5/0 78 O/

10

64433655466/0/055666/05

21211210412222227~227~2

2111122212222119~229~1

2211122122222220una2nd?"

O

l 2 34 5/0 78 9O 1 2 3.4 5,0 7.8 9
1.1. 1L1;11._llqi~l

I

‘4

A

(3:f.-uoneff. continued)

 

 

 

 

 

 

 

 

Raters
l 2 43 Sum
Counselor Response
J 1 4- 3 3 10
2 4 4 3 11
3 3 3 3 9
1.3, 1+ 3 3 10
5 14, 14, 1+ 12
6 3 3 4 10
7 3 4 4 ll
8 3 4 4 ll
9 3 3 3 9
10 3 3 3 9
11 3 4 3 10
12 3 3 3 9
l3 4 3 4 ll
14 3 3 3 . 9
l5 2 2 3 7
16 3 3 3 9
17 3 4 4 ll
8 4 4 3 ll
19 4 4 4 12
20 .3. .3. L .2.
am 476 474 491 1441 Sum 2=11381 (1441 )2=20764: 1
31117122 12 54 1252 1351 i X -3857
(Sum) 692,333
Source of
variation df Sum of Squares £§Sl_, Variance {82)
Counselor n381 - 2 076 481 = 332.9 322.2 = 1.672
Responses 199 00 199
Raters 2 6926222 - 2 076 481:. . = .4’0
2 O 00 _‘g— )
Error 393 396.2 - (332-9+.9) = 62.4 62.4 = .157
T t 1 "99 3357 2,0E6,% 1 396. 2 39‘
o a 3 ~ < - =
(1) Internal Consistency = 1.672 — .157: 906
1.672

(2) lgfierjugge Reliability (between-rater variance removed)

:1 672 - 0;? = 9 6
1672 + (B-lW 1577

(3) Inter’1d39 Reliability (between-rater variance left in)
= 332.9 - 1,9 + 62.4) = .260
’)
L,
333'? + (19 +“525¥7_

 

APP EITDI X '3

STATISTICAL ITEI AIL-XLfSIS 0‘7 TH

Ls)

SCORES OF THE CBJSCI‘IVITY STUDY
BASED UPON HOL’T'S COIIPU'I‘ATIOI‘I OF

RELIABILITY BY AI-IJ‘LLYSIS OF VARIAI‘TCE

.165-

/ ,"
~130-

:ensien

Affect-Cognitive 313

 

Raters

Sum

 

 

Resoonse

Counselor

6 3.333a9/O/O/O/O EKG/C 302:4 BBQ/33333333366 551w. 333333/0/0/0 OJQJOJQJOJQ),

62 11 11 11 11 11 62 62 62 62 62 62 62 11 11 62 11 11 11 11 11 11 11 11 11 11 11 11 62 62 62 62 62 11 11 11 11 11 11 62 62 62 11 11 11 11 11 11

921111111111929292921192921111111111111111111111111111119291111111111111111111929292111111111111

91111112220,2912211111111111111122221111111292le1111

1.11.01 OJHW rJ/O 1.“ 2 QJIIW 5/0 71 2 314 5/0 ?8 O/l 2 «314 1.2 31H». 51 2 «)4 5/0 1 2 3'4 r311 0201/14 ,rJ/O

 

.i‘i Ill}...

-1157-

)

Aff.-Coy. continued

(

Raters

SL1 37‘:

A

 

 

1’186

0330)0J3333333333333333333333333333356533.4 3333333/C

1111111111111111111111111111111111111111111111111111111111111111119272921111921111111111111192
11111111111111111111111111111111111111111111111111111111111111111111929211111111111111111111o2

1111111111111111111111111111111111111111111111111111111111111111119292111111111111111111111192

o
.91123412345,07812345/07112312341234561234112345,.D78123

S
x}

R(

Counselor

9

10
11
12
13
14

.-Cog. continued)

Raters

«4.2

A

 

 

1) Denise

Re

Counselor

145336 BGJBQJAJ/O 30.236 33333333333336333333333014n3336331130)

7292111192111111111192111111921111111111111111111111111192111111111111111111111111111111111111

1111111192111111111192111111921111111111111111111111111192111111111111111111111111111111111111

1162111162111111111192111111921111111111111111111111111192111111111111111111111111111111111111

rind. 5/0 700 1 Ad OJrUr 51 2 3b. 51 2 31“.? 5/0 7.8 90 l 2 3h. 1 9N 3n. 1 2 3.4 5/0 78 90 l 01 1.14...

_.l_ l

19
20
21
22
2

24

18

. continued)

{r
L)

(Aff.-C01

Raters

Sum

 

 

A

8
S

n
m

1 51
S

Re

Counselor

33n233333333333330233333n1JOJ333333333333336/O/O/O5/0 333

11111111111111111111111111111111111111111111111111111111111111111111111111119.9292929292111111
1111111111111111111111111111111111111111111111111111111111111111111111111111929292921171111111

1111111111111111111111111111111111111111111111111111111111111111111111111111329272929291111111

1234512312345671234123.4463/0123456711234561123412345

on QJ
OJ 3)

25
26
27
2:3
29

\ 30
31

’3‘ £1

 

 

(Aff .-Cog. contimed)

Resoonso

Counselor

Big/333333333333330J3/O53H. 3334333633n)33h~. 3,3n_JA},n)QJSn.).OJn«J.GJ

llliLllLillLllllllZZlo,~1112111211111211_L.LLlLLLlL
ililLllilllllllllllelllllllllelllllillllRHLLLL

Llilll.LilllllLllllZ21illllilllelllllLiiLiRiiLL

to 7.3 i 9., 3a. .51 2 3a.. ..)/0 73 9mm 1.2qu 2 3.4 51. 2 oJflv 5/0 1 2 374 5/0 7.3 l 2 3 ml 5 , 7.

34
3
30
37
8
9

continued)

‘1‘
(a.

(Eff .-Co

Raters

Sum

A

 

 

333n33333330J3333333333330J3/0/D333333.06OJOJDrSr01033.3/Q/O/O

llllflflkLlllLVLlllllllllllW_l22111l11221.i22ZXlLLn/NZB

.lllllll.iill_llllllllllilllg21111112211.12221iL222

«L.L.L.L1L.;.;1L1u1+nuI*1*1*I*1*1*1*1_1*1*1*1*1*1*obvslu1*1u1.1*1.9~9~1*1*1*1*Quasi Lwin4n¢o~

01180

Jul 2 43):“: 5,0 o... 2 3d. 5,0 700 l 2 3E.5/O 7i 2 «3%... 5,40 l 2 3.4 51 2 3H“. CJ/O ..lm 2 ”3.4 l 0;. 3U: 5

7 Q
has.

Counselor
’L
42
43
44
[+5
46

WI.
1;

(Aff.-Cog. continued)

Raters

 

1’1

3330J.fu/O/Or)5./OU.105663rO/O 30J36/O/OOJ3366/0—

lli.inp9‘.2n/~OLQ~9~29~22127~lll2092111222—

llLlné9~20~12129~9~217~7~lllZZn/klllGun/~2—

 

la!”I.loan/Hallnan/ulOLWLA/NZHLZn/Jlll2221.11222“

“l 2 3&1 VJ]; 9w nQU. 5/0 7.!” 2 02.4 r.) ,0 71* 2 3M. 5i 2 34 5/0

Loo)onse

CouheeLor
pg
50
51
52
53

12566¢1

7

2].)~

Sum2=u321 (11

367 383 1121

371

‘ 475 523

457

9‘.

'7‘.

1)

419,019

(Sum)2

(Aff.-Cogj. continued)

Analysis of Variance

 

 

Source of ,
Variation __ df Sum of figuares LBS)» Variance 158)
Counselor
Responses 3L2 412i - l32§oaofll = 102.0(SSR) 102.0 = '330(SZQ)
3 939 312 ‘
Raters 2 412019 .. lazgeaénl = 6.5(ssc) 9&5; = 3,4(520)
313 939 2
Error 624 146.7 - (l32+6.8) = 7.9(ssE) L2 = .013
6:34
Total 938 was - 1,256,641 = 146.7(ssT)

939

 

(l) Internal Consistengy =_;330 336013 = .g%% = .961

(2) Intcrjudgg Reliability (between-rater variance removed)

= O I- .01 = .820
.330 + 3-1 .013

(3) Interjgdgg|ReLiabili§1 (between-rater variance loft in)
= 132.0 - 56,8 + 2.22 = .820

132.0 + (as: 7.9)

 

-17! L—

Uonundcrstanding,Dimension

1' 3
L111 57-
A

\

Understanr

 

Raters

Sum

 

A

 

Response

Counselor

((3 3/0 3.0.1 3/0 066506435333333333333/06 5/0434 333QJ/Ohw..b joJ/O uJO)...L-

Quin/will?“222222210¢llllllllllll229~22121ul1

21.XllizzZZlZlel.lllllll.l—lllLZZIlellll

2l2i2l2222222112llllllllllllZZZlellll

.i 9., c.,).HLr rJ/Q I. 2 3h”. ..)/0 7i 09 BR. .126 7,00 91 2 3h. IA 2 3a. 5.1. O,” 9J9: 5/0 .1

LP

6

a
.

..L3_i9w_l‘_l?wntull

.|*n/NlleO‘._ll,Ll

|_n/HVu./.ill0.flm,lwom

2 3h”. 5|" 3 n) M. K7?

. Ill-IIIIIIIIII |.|l|l|..1|
III

.erst . -ITe:1underst . continued)

Raters

 

 

01130

..)}..Uv. O-/3n)330J33u4/.33BQJOJOJOJOJ/O/O IO 3330J3333331/IQ /O 53nw/50JQJOJAJOJNW. CJ/O

llll

lllllllllllllllz2

zilllliilllludthLIlfi/vllwlil_2i?

lllllllllllllllllll222lllllllllllflZZIlu/llllll22

l/ll

Pl 9“ H) u.

I‘
u.)

"j
“

elor

S

U/

lllllllllllllllZZlelllllllll2211llllllll22

L 2 OJDr ..)/D 78 l 2 3:4 5/0 7.1“?”

”2.1.. Oh 3a. 1 05.. 3a. 5/0 i 2 u) M. 1L0“ nigh”. 50 7mu All a,“ Q

i3
14
i5

st . ~130nunderst . contimed)

‘5

(Uncle

Raters

h». 5336 333336 3336 33333333333336 333333333333333364

221121111121112111111111lllllellllllllllllllZl

 

 

lllllellllellellllllllllllellllllllllllllZl

l211lellIlellellllllllllllellllllllllllll22

4567001234512345123456789m1234l23.41234567009m1234

mmom

Cmmdw

18

m

20
21
22
8
2#

-l"/ 7-

(Underst. . -l-Ionunderst . c ontimled)

Raters

m 4.4/0 314. 53nd. 33333334333333333333333333333366/066 nfl/BQJH/

C 2.].2112121111....11211llllllllllllllllllllzzz22111...,"i

BIlllelllllllllllllllllll11111111111111222221111

A l221Zlelllllllllllllllll1.111111111111222221111

 

 

123.451231234567123.4123hw561234567123456123412017.,wa

Response

Couns elor
25
26
2'7
28
29
30

_um

(Underst.-Honunderst. continued)

Raters

Sum

 

 

kwmw

mmnmflor

33h. 33333333333333366 34.43333366 333333533311». 353333

11211111l11111111122122111112211111121111111111

llllllllllllllllllz2111111112211111121112121111

llllllllllllllllllz2111111112211111111111121111

%
N
w
w
m

3%
35

~179-

(Underst . ~33 onundor st . continued )

Raters

333333333333333333333333666 33333366.“. 34/06/04 33/0/06

11111111111111111111111122211111122112222211222

llllllllllllllllllllllllzz211111122111222111222

 

 

llllllllllllllllllllllll22211111122211222111222

123456123456781234567123.45612345123fi561234123h~5

Respo ¢‘e

1&7
1&8

1+2
43

Counselor
1+1

 

wL.

(Undorst.-Nonunderst. continued)

Raters

 

Sum

 

Rosoonso

(3cn1rn3eflaoz-

3.4/03666/0/0/0/0/05/0/03/0/Ohwn4hW/066U.33/06/0—

ll2122222222222122222222211222—

120u122222222122120ulll222111222—

:1—2345123hw5671234567l2345123456

Ly)
5C)

7
a9

@937 (1183>2=1399u»

404 1183 3111,12:

385

394

Smn

é x2=1671

529 586

556

466,677

(amwz

 

 

1 (3'1
-

_...-Vl- .-

(Underst . -llonunderst . c ontinu ed)

sis of Variance

 

Source of

 

 

Variation di‘ Sum of ngares (SS) Variance L52)
Counselor
Responses 312 £221 - 1,322,182 = 155.27 1 .2 = .498
3 939
Raters 2 I+66,6zz - 1,322,162 = .56 :16. =.280
313 939 2
Error 621} 180.60 - (155.27 + .56) = 24.77 24 = .040
Total 938 167.1. - 1,322,482 = 180.60
939

 

(l) Intemal Consisten = #8 - 040 = 1+ = l
“A ‘EW— {7:938— ‘22

(2) guberjggge Reliability (batman-rater variane removed)
= 8 -. . 040 = :22

. + 3 .

(3) Intermgge Reliabflitz (between-rater vanance left in)

= 155.2? - (.56 + 24,22 = :220
2
\ 155.27 + (.56 + 21+.7T

 

c-ITons‘oocific Dimension

.
”'1
ch.»-

Snocir

___‘

 

Raters

 

A

OJIJ. 3n330J5/U rJ/O 366br 3533333333333366h¥6 33333n33/0 350JOJ3333

121111221212221211111111111122221111111212111111

\

111111122212211111111111111122121111111212111_11.1

 

11111112221221121111111111112Zllellllllellllll

36

3011

ml1 2 3a. 56 12 3b.. 56 7.1 2 38. 5/0 7.8 91 2 3:1. 1 2 3a. 51 2 3b.; 56 12 3h. 51 2 3h. 56

L)

f.)
o

no

Counselor
1
2
3
1+
5
6
7

'11-»)fl
\,\\
.-x.‘

(Spoc.-Ionspec. continued)

Raters

31.1171

no

 

Box/BOJOJQJQJOJ33333330J330J/0/Olo 33333333333564 33h. 33330.2«93/0

lllllllllllllllllllZ221111111111122211211111112

11111111111111111112221111111111112111111111112

 

11111111111111111112221111111111122111111111112

v.1ohms/'Urln/NBHWKJ/On/Oul2345/0711231234123v¢w56123412n94567n012o)

. uUODSO

..)

Counselor

9
10
11
12

3
14
U
16
N

’V ('3 ,‘
-.L.\)‘+’-

(Sp ec . -'.Ionspec . continued)

Sum

 

w

 

Re soonse

Counselor

I'Lr KJ336 333336 3336 3333OJ33333333333333333333OJBQJOJ333

22112111112111211llllllllllllllllllllllllllllll

11112111112111211lllllllllllllllllllll111111111

12112111112111lellllllllllllllllllllllllllllll

1.56781234512345123456789m12341234123456 78 9m1234
8 9 O 2 3 -
1 1 2 a 2 2 NA“

(Spec.-Uonspec. continued)

Raters

 

3333333.”. 0J0)333333333333333333333333336)36/06/06 3333

1111111211111llllllllllllllllllllllll1222221111

11111111111111111111111111111111111111222221111

 

11111111111111111111111111111111111111222221111

.129245123123145/07123412n3hv56123n4567123Dr5/0120J4120Jn45

Rosno "e

(u
2

0) OJ

Counselor
U
6
W
28
w
0
31
2
3

spec. continued)
Sum

Raters

A

 

 

Soec.-Xon

(

Rosno so

Counselor

33n93n333333333333335534 33333336 BBQ/333.4 333331.“. 3333

11111111111111111122121111111211111121111111111

llllllllllllllllll22111111111211111111111121111

11111111111111111111111111111211111111111111111

.6700123I4 5123456789101120J12a.Ju4512345612n)45/0 7Du.123hv 5/07..

w.

%
w

my

%
W
m

(Spec.-Eonspec. continued)

Raters

Sum

“D

 

n933333333m)333333333333333-4U. 3333336 330J35/O/O Bax/366 3

lllllllllllllllllllllllllzlellll21111222111221

lllllllllllllllllllllllllllllllll21111222111221

 

111111111llllllllllllllllllllllll21111122111221

”.1.— 2 0th rJ/Q l 2 3h». KJ/O 7.00 l 2 69.4 5,0 71‘9“ 3h“: 5/0 1 2 3h. 51.. 2 3h. 56 l 2 QJhW l 2 0.4.0,. 5
a

CNS 9

q
AUG

45

1+6
[4,

Counselor
M
1+
L‘y’ _
m

ontinued)

Snec.-Honspec. c

(

Raters

 

A

333355444666556 3663446664 3356.4—

1*1r1r1r9~9~9~erases?“9~9~9~9~1.9~9~1*9~9~9~9~9~9~1r1rososoé

.llll2111122221212211122211122L

 

. S'DOI'ISG

T)
;

Counselor

111—112111222122122111222111121—

.l 2 3hr 51 2 3h. 56 71 2 3.4 56 71 2 3b,. 51 2 3D,. 56

1P9
50
5L
52
fl

1097 Sma12=LPl73 (1097)%12031:..o9

360 , 381

3%

Sum

fl?

401,u97

W x
-l ,1! ('2'-

(Spe c . -I.’onsp ec . continued)

Analysis of Variance

 

Source of
Variation; df 1. Sun of Squares ES) Variance
Counselor 41:23 - 1,203,402 = 109.41 109.41 = .351
Responses 312 3 939 312
Raters 2 1:01.1922 - 1,203,402 = 1.15 1,15 = .575

313 939 2
Error 621% 131.41 - (109.41 + 1.15) = 20.85 20.8 = .033
Total 938 1413 - 1,203,002 = 131.41

939

 

(l) Laterml Consistencv = .° - 0 = . 06
_.....— ......................u.. .2.
4 .351

(2) Enterjg' dole Rehamlitx (betV-Jeen-rater variance removed)
= 1.351 "' 0033‘ = .26?!
.35L +73-DT. 0337
(3) InterE' dgg Rehabilitv (between-rater variance left in)
= 109.41 -(l.15 + 20,85) = .242
2
109.31 +ZI.IS + 20.85) l

qtory Dir: ensien

L

27110?

 

Raters

'3-2 Tone:

 

at o:

lor

 

Yi‘h
14. ..

307,39Jhw. 3/06 5/0 3355333333333333333536 333n)333/0 OJ/D 0).ajnfixojejsrY

111121222211Gugllllllllllllvlll222111171111212111111

1111111111119~9~1l911111as911111111.1111111111111l1l1l1l119l1l7~1l111l11111l1191119~1l111111111l

 

111111222lelllllllllllllllllll21111111212111112

O
S
m
mull Op 3.5. 5/0 1 Op 394. 5.6 71 2 3hr 5/0 700 91 2 31¢. l 2 3h. 51 ad 3h. 5/0 ql" 0w 43h“. 51L 7.. OJMW. rJ/D
e.
“7,.

‘4
ct.

Couns ole

1'11

C‘
L.)

Raters

w—‘v

 

 

fipl.-Xonorpl. continued)

v“
“J.

(..

Resnonso

Counselor

OJBOJNWDV 33330J334 33333336 33333333333336 3333b. 3333366

lll221111111lellllllelllllllllll2111121111122

llllllllllllllllllll211111111111112111111111122

llllllllllllllllllllZllllllllllllllellllllll22

«l23419~3v4567001234567123123,4120).“.5/0120Jv41234rJ/O70012OJ

10
ll
12
13
14
15
16
17

a (7")
.. u.

Waters

Q

A

 

(Eqfl. . ~130nezqfl. . contilm 0d )

hv. 3335.0. 333333333333333333333333333330J33«Jaw/BBBBQJOJOJ

2111221111111llllllllllllllllllllllllllllllllll

lllllllllllllllllllllllllllllllllllllllllllllll

lllllellllllllllllllllllllllllllllllllllllllll

 

e

S

m

min». rJ/O 78123.451234 51 23h". 56 789.0212 3.412 AJbrl 2 3.4 5/0 78910.1 2 3.10..
M

r

o

a.

S 8 9 O a 2 3 .4
m l l 2 2 2 7
nu

(Etcpl . -Honexpl. c ontinuod )

Raters

finn

D4

 

 

33333333333334933333333333333333333333n)/D 33h». SaJBnJOJ

llllllllllllllllllllllllllllllllllllll211221111
llllllllllllllllllllllllllllllllllllll211121111

llllllllllllllllllllllllllllllllllllll211111111

1136

O
$.12 31451 2 31 Ema/h... 5/071 2 0.2.41 2 3nd. 5612 3.“. 5/0 7.1 2 02.0,. 5/01 2 92.3. 1.” 2n9h~ r)

Re

Counselor
5
6
W
8
w
%
fl
3?
3

Tonoxpl . conti med )

l
&

q

'1 ~
.5331 . .-

( } ‘
J

Ra tars

A

 

333333n33333333333354 3b. 333.”. 33/06 333333333330Jw3AJOw/493

llllllllllllllllllz212111leZlelllllllllllllll

llllllllllllllllll21111111112211111111111121111

llllllllllllllllllllllllllll2lellllllllllZIlll

 

0
S
n

O
m‘67nu1234512n3456789w1ZOJl23hW512345612345/078123.4rJ/07

Re.

Counselor
5
6
37
38
9
Lo

Y
i

ommLcmfimw)

:2fl-o-l

(w

Raters

Sum

 

 

1‘13 0

333333n3333330J3Q/33333314 333.“. 3333333464 3'46 56h. 3a3333

lllllllllllllllllllll21112111111112112222211111

llllllllllllllllllllllllllllllllll2111222111111

lllllllllllllllllllllllllllllllll22211212111111

0
$1 2 3.4 56 l 2 3.4 5/0 78 l 2 3.4 5/0 71 2 3.4 56 l 2 3a. 51 2 3.4 56 l 2 34 l 2 3.... 5

.0

D;

Counselor

M
W
#3
M
45
%
2;}?
m

~r

Lpl. continued)

.4

4.

‘ml . -None

P
A44

(7

Raters

A

 

 

Re sponse

Counselor

36

Surr12=3782 (1056)2=J.11 51

33.4 33.4Dvhw34u556u. 36/044466“ 33353;

1056

112112221222222122222222111211—

372

l1.1111111112221122111221111211— W

1111111111111211221112211lllll— W

l23:45.1.234567123456712345123u5/O

49
5o
51
52
53
m

X2=1290

#06 11.90
372, 320

394

Sum-2

(g) “51)2

(33:91 . -ITone:rpl . conti r11 ed)

Analysis of Variance

 

 

Source of

Variatmu (11‘ Sum of Squares (g) Variance (82)

Counflor

Responses 312 2282 - 1,115,136 = 73.09 23.09 = .234
3 939 312

Raters 2 3:22.32 - 1.11513 = 1.94 1.0:1'. = .970
BB 939 2

Error 623% 102.392 -(73.09 + 1.94) = 27.39 2 , L = .0)th

2
Total 938 1290 - 1,115,13 = 102.u2

939

 

(1) Internal Comistency = .2 - .01% = .812
137+

(2) Lnterjuchje Reliabflity (between-rater variance removed)
= 1235'; " 00141" = 0220
“2% + (3-1)(.Oifl1.)

(3) ligaterjugge Reliability (betteen-rater variance left in)

= 73.09 - (1194+ 27.139) = fl
2

73.09 + (17.91? + 27.39)

Sun

Raters

 

 

A

J

T“feotive-Koneffeotive Dimension

 

966653998m599646 33r)33333645#9979666446 39/0 933533hw

322221333423n321311211111212233332221121323111111

322211332313322111111111211133232221121323112111

/

32222133330~3321211_211111222133232222221323112112

..lOLOthrDélZAQhVKJ/anl234567891234123h¥5123h¥56123.45120.41456

Rosoonso

Counselor
1
3
4
5
6
8

-‘9//

-1 N:

(Eff.-Uoneff. continued)

Raters

R%

646.46 35nd. 333354 33333mu 9.“. 33nd. 33.4 533Q8 97h”. 0J733333/060/

2121211111112111111433111111121113332131111.1223

21212.1.21111111111113431111111111123211211111223

 

 

222221..092111122111113432112112211233211oalllqilZn/Ho)

9
S
m
$.1234123br5/0700123:4.5/071231234120J45/Dl23.410434567001104n)

“m

Counselor

9
10
ll
12
13
14
15
16
17

(Eff.-Soneff. continued)

Raters

 

no

A

 

6
S
n

m.

Res

Counselor

78h. 300:4 3h. 339.455934554 333333346 33333333333333336 5

331132111131213111111111llll21111111111111.1111122

22.1.121111131123112211111111121111111111111.1112].

23213112113222312222111111122111111111111111122

45670012«345123.4512345/0789012341234123hw5/0780/012a9hw

l l

18
19
20
21
22
23
74

Noneff. continued)
Raters

 

"1"".9
..JJ. .L . -

( ,

644.4 5.4146 333333353333333333334 33355333398 999.4 p34 3

911111119211117211111111111111a21111111111111111111_11119211111192111111111.QJQJQJQJQJ92921111

921.92111.119292111111111111111111111.1111111_111111111111111111929211111111QJ02QJQJQJ11111111

221222121111111lelllllllllllllllndllll333331221

 

e
S
n

O
$.123br510a312345/O7l234123.456123456712345/01O~3410~31Urr3
@

L1

Counselor

25

6
27
28
29
30
31
32
33

\U.

(Eff.-Uoneff. continued)

Raters

 

.4 3.4 31/14 3.4 33.4.4 3333338 936.4“. 35538 9.4.0. 33.4 37334.4 304.4. 333

21211112111111111133132211le2311.1111oJllllllqull

11111211111111111133121112213311111121111131111

 

..111111—11112oalllllqignfillllllZlBQJfi/N2112121122132111

0
S
n

0
W6 7.00 .12 3.0... 51 2 3a. rJ/O 7.8 9m. .12 31 2 3.4. 51 2 39...... 5/0 1 2 3.4 rJ/O 7.0m .12 3h. ..)/0 7
Q

‘9‘
a.

n0

OJ

Counselo
3’
3
%
W
9
40

(Eff . -::onefr . continued )

_ Raters

 

3333333333333333333a)3.4 3377.8 333333994” 58 99.4.4 38 9/0

lllllllllllllllllllllle23311111133112333121332

llllllllllllllllllllllll22211111133111333111232

 

llllllllllllllllllllllll323llllll33222233211332

30118 0

_...l_ 0.“ 31'». 5/0 1 2 0).“ 5/0 700 1 9H QJDr 5/0 71 2 3h”. ”..)/0 l 2 3.4 51 2 3.4 5/0 1 n4 3h». .1“ 9H QJhw. r.)

Res

Coun selor

41
m
43
1.1+
#5
%
W

.8

Sum

_.“(X‘U
Raters

 

(Eff.-Uoneff. continued)

5565909860908996996770n956399£
1* 1. 1+ 1*

29~223h¥ 332 3.411“. 3332332334432 21332—

w

v
1121 333323333332 3322 2 3331 21 332_

 

n5222333221423233233222343221332—

..123u.5123brr),07123u4567l234512345/0

Resoonso

.01‘

(‘

COun ue]
49
5O
51
52
53

/

Sum2=8738 (1496)2=223301~

(J

25 x2=2990

1496
747,288

470 508
1056

888

1046

(Sum)2

(Ef1.-Ionef . continued)

Analvsis of Variance

 

 

 

Source of
Variation df.’ Sum of S_cn1ares (SS) Variance (82)
Counselor 312 82:23: - 2,238,016 = 529.1+O 529.110 = 1.70
Responses 3 939 312
Raters 2 297,288 - 242371916 = 4.23 4.23 = 2.12
313 939 2
Error 624 606.73 - (529.uo+4.23) = 73.10 23:10 = .12
5E5
Total 338 2990 - 2,232,016 = 606.73
939

(l) Enternal Consistency = 1.70 - .12 = l.§§ = .922
1.70 l 70
(2) Intorjpdre Reliabiligz;(between-rater variance removed)

= 1.90 - 1.2 = .814
1.70 + (3~17?112)

 

(3) InterjudggeReliabiligz'(between-rater variance left in)

= 529040 ”(4023 + 73010) = .892
2
529.Eb #(h.23 + 73.10)

APPENDIX H

STATISTICAL GLOBAL EATER ANALYSIS
0.1? THE SCORE OF THE OBJECTIVITY
STUDY BASED UPON HOYT'S COMTATION

OF REIIAHIIITY BY ANALYSIS OF VARIANCE

 

.ll .1!
l I I.'llnll]4llllul|ltl

~207—

 

 

AffO ‘ Elmeasasauinnsea
ta 6
A BETSL“""TF""T'1§fif"’

Counselors
1 8 8 8 2h
2 o 1 o 1
3 10 10 9 29
h 10 10 10 30
5 1+ 6 4 ll»
6 8 10 7 25
7 1+ ' u u 12
8 10 10 10 30
9 1o 10 10 3o
10 1o 10 1o 30
11 10 10 10 3o
12 o o o o
13 10 10 10 30
1A 10 10 1o 30
15 5 5 3 13
16 10 10 9 29
17 6 8 5 19
18 10 10 10 3o
19 6 6 6 18
20 10 10 10 30
21 7 7 7 21
22 1o 10 10 3o
23 10 10 10 3o
24 10 10 1o 30
25 10 10 10 30
26 10 10 10 3o
27 10 10 10 3o
28 10 10 10 30
29 10 10 10 3o
30 10 10 10 3o
31 1o 10 10 30
32 o o o o
33 8 9 7 22
10 10 10 30
35 1o 10 10 3o
36 3 7 3 13
37 10 10 6 26
38 8 8 8 22
39 10 10 9 29
#0 9 9 1o 28
41 10 10 10 30
#2 1o 10 10 30
43 10 10 10 30

an 7 7 7 21

C
"-""5 ..

. . r7 ‘ ~ _ F4
" \ ',. “ ‘) ‘
*“ ”«O‘, .- . . ..

 

..— —J 5 cu ., - ‘- -.- “.-- M.
.. , ‘0'.- .
‘1‘. I
.o. 4 7v“. - Ccoe w. .. _A r .-,.
.
.3 '
.
1
. I

1

f1."
’
.
l‘ 1‘ l‘
. r,
, ’ v
. .
.

 

-Am!

Affect-Oomtive (cone 1:.)

 

 

 

 

 

A B C SUM

Counselors

45 10 10 10 3O

46 7 7 5 19

47 5 3 3 11

48 4 4 4 12

49 8 8 8 24

50 3 3 o 6

51 4 3 3 10

52 4 4 4 12

53 _i .1 .2 .1:

Sum 413 422 394 1229 Sum2=32,789 122924, 510,441
(3:ng 3693 3812 3478 (2410.983
(Sum) 170569 178084 155235 503389

was of Vagance
Source of
Variation (if Sum S ares Va 32
Counselors 52 m - 1.20.44 = 1430 27.50
3 159
Raters 2 M II 1' 20'“ = 7 3050
53 159

Error 104 1483 - (1430 + 7) = ' ’46 .44
Total 158 10983 - 9500 = 1483

(1) Int_____e_____rnal cons§stengx= 0 - = 2 06 I284

$501,427.50
(2 ) Enter-figs Lew (between-rater variance removed)
22 -44 £206:- ,253
27.50!' +' T335!‘.'E47 2%"3'5
(3) Inter 11" dge reliafillitx (between-rater variance left in)

= 1430 - (23: 46) = 140:32'05 :-

1430 + (7 + 4-65

u.‘

I

' .0 .
_.., .—
7 7‘

- ‘

my..." ‘ .

,..

.05."

 

 

. .-

 

..u

 

1 .
‘ K
‘ 1
‘ .
r
n “'
3
4 ‘~

 

 

 

' _.n—‘
. - .‘nId
. .
I
.
I \
'1‘
.
x
o
1‘ .-»~
‘.I‘ '
-- ’ .

_-209-

Understanding-Nommderstanding Mansion Scores

 

 

 

 

Raters

A B c SUM

Counselors
1 5 7 7 19
2 o 1 o 1
3 9 10 8 27
4 10 10 10 3o
5 4 6 4 14
6 8 8 5 21
7 4 6 6 16
8 7 8 8 23
9 7 10 10 27
10 10 10 10 3o
11 10 10 10 3o
12 o o o o
13 10 10 10 3o
14 10 10 10 3o
15 5 3 3 11
16 10 9 9 28
17 5 6 4 15
18 10 10 10 3o
19 6 6 6 18
20 10 10 10 3o
21 7 7 7 21
22 10 10 10 3o
23 10 10 10 3o
24 5 7 7 19
25 4 8 6 18
26 7 10 3 20
27 10 10 10 3o
28 1o 10 10 3o
29 10 10 10 3o
30 10 10 1o 30
31 10 10 10 3o
3% o o o o
9 9 7 25
34 10 ‘ 10 10 30
35 10 10 10 3o
36 3 3 3 9
37 10 10 6 26
38 7 7 7 21
39 10 9 9 28
4o 9 7 10 26
41 10 10 10 3o
42 10 10 10 3o
43 10 10 10 30

“1+ 5 5 5 15

. , ‘ ,
_ .
_ 4 _, . _.. ,

O’QJ- -~'~v " -
- cu- .- n2 .

.- 1. 0v irfi'

 

.
.‘1.
non-

-. . ooll'
.
I
l‘
, .
~,

*‘wo
- vcv

l
w
a.
’ .
)
l
x

\

..
1.
1.1.
.9001"
’~'\

fi

-210-

Understanding-Nonnnderstanding (con ° t )

 

 

 

 

A B C SUM

Counselors

45 10 10 10 30

46 5 7 5 17

1+7 3 3 0 6

48 4 4 4 12

1P9 4 6 6 16

50 0 0 - 0 0

SI. 4 3 l 8

52 4 4 0 8

53 .5. .2 .3. .1}.

375‘ 394 359 1128 Snag-428752 11282=1. 272.384
(311-112 3 3193 3428 3075 {@9696
(Sum) 140625 155236 128881 424742
Analysis of Variance
Source of
Variation df Sum of Squares (SS) Variance ($2)
Counselors 52 2823 - I 1222 3 3841582 30.40
159
Raters 2 424242 - 1.222,:384 = 12 6. 00
53 59

Error 104 1694 - (1582 + 12) =100 .96
Total 158 9696 - 8002 = 1694

 

(1) Intemal consistengz = 30.40 - .26 = 2.44 = I268
30. 30.

(2) Etegjgdge reliability (between-rater variance removed)

=040- 6 =22.44=3211
310.40 "+" 23; 15!.93) 32.32
(3) Interjudge reliability (between-rater variance left in)

= 1582 -(12 + 100) = 6 = 01
----2 1125-): L

1§2 + 112 4» IE) -

 

 

 

     

 

     

   

 

 

 

; V
, a. , .. .5 ~ - 1
‘ s4 , ‘ I , ' .1 - '
v o-
. ‘ .
‘ l . _ .1. .. _ . . , ‘ .‘ .\ , . I .‘ . .1 .
' ) .1 i . 1 ’ . 1
Q . b - . '1 .0 - ' . | g I .
. \
.\ - ' V V
. 1 ' .
._ “ ‘ 1
.- - X
L . '.'
I "
1 7
-
V
- I
. . 1 4
_ - .
1 - ‘
, .
.
' 1
,
u w ‘
..
4 ~ I;
. 7
. ~ ‘
- \
. - w
1‘. ‘ .
\ \
. ‘ -m. I“ - fi .
-- .“y, r» _...fi‘ '1..~
. _ ‘ l , I . . , . ‘ x . >-
' .- ‘\ I .1 ~ f‘ '\ ... '0 2 I ' ’
‘I . ..- a ' -- 4 a- . \ - ~
. ‘ ‘
- . L - r‘ ' .
‘ b n‘ \ "xr ’ .' ‘ . ' ..
1 I — . 4 '; ‘ .
’ - |\‘( , .4.’ .
~' \ '-
. v .
I n e V‘ I“
. .. ‘r. ,) 2
v . » .V ..1 J
..a» . ”..a .. ~-.-‘--o tn-Q. . '- ~‘O—l.-.oum‘bb-O. . A . . 8 A. row _~‘ \- - - . . . .‘Q— -. 5-.
.s'
‘ 1 ' v-u '
,. , .
~ \ . . n
‘ . I . . . . l . . - .-, _
\ ‘-‘ ‘J\/Lcu' ,‘o ‘. . u ~4'V "1 .. " § 1. '4. " “
g. C v. .‘ J-’ p- w...” a . , a h/ I -7. ‘0‘- ‘V" I < 4 c- I I i
..
l r , ' x “' .- .o
. 1 \ ,» \
.~ I . 3 1 .. \ 7‘ “. .
. . _ . ,,. . .
~ . , m'v 7" u.“0-
.
. . .. . . "a; .7." 4 1
4 ‘_ ' A' . . a
\ . . ‘ \ ' y 7 ‘ _ . '
~r“!‘- ’1‘ “a. o ‘ IV'
fi .
0' a
u
.
7
’. . .’ '
.
g ._
r
L i
. .- c- » ' ,
400' ~ .
---o' - -r’v'w08' an . ‘ cc...-. 1 . o ‘- c 0 ml” . - o—- -- - A -- o -
3 o
. ‘ ‘ ‘ ’ ‘ f .
W .7 -. \' ,.-. r ‘
- . u v o . , 1.. , - . .. . ‘ ’ ~
. a . ' - . 7 ' ’ 4 .. k ' ' V .
g- .-u'. - wv,-. . . A 1“ '1.. - on. - \' . .pwo. - - 1' ~ . - o
. l‘
I ‘ ~
‘, . .
' s
' a I. ‘ ‘ ‘
<2 ‘ o i "'
n .v 1 ‘ . I1» . ' . ‘ “‘ 7 - ‘ z a, ' " . \
» ' I I . ." .. 1 ‘ . >3 ‘. .
sI—OD ‘1 I - - u
1’1
. ‘ J
. ' r I . p a '
.. . L .. .- .' ’~ 7
.
. . .
‘ \ f _. '
I 1 , A .
. , r,
. . .
I i ‘ 1
‘1, ‘ _ . .A. '
1 ' I Q . 3 ‘ -...'.. , . n
..a , o ‘.“'1
[..a-*1“ , u .‘ 1
..a J , .‘ ‘ ~ _, . “..)-:4. n
. ' ‘ “
1» Ar 1 ' ‘
‘ I

 

1211;

Mic - Nonsmcific W Scores

 

 

Raters

A B C SUM

Omnumflors
1 10 10 8 28
2 3 l 3 7
3 9 10 8 27
4 10 10 10 30
5 6 6 4 16
6 8 8 8 24
7 8 6 6 20
8 10 10 10 3o
9 10 10 10 3O
10 ”IO 10 10 30
11 10 10 10 30
12 0 0 0 0
13 10 10 , 10 3O
14 10 10 10 30
15 5 7 3 15
16 10 10 9 29
17 6 8 5 19
18 10 10 10 30
19 6 6 6 18
20 10 10 10 30
21 10 . 10 10 3O
22 10 '10 lo ,8
23 10 10 10 30
24 10 10 10 30
25 10 10 10 30
26 10 10 7 27
27 10 10 10 30
28 10 10 10 30
29 10 10 10 3O
30 10 10 10 30
31 10 10 10 30
32 O O 0 0
33 9 9 9 27
34 10 10 10 30
35 10 10 10 30
36 10 3 3 16
37 10 10 8 28
38 8 8 8. 24
39 10 10 9 29
40 10 9 10 29
41 10 1D 10 30
42 10 10 10 30
43 10 10 10 30
44 10 10 7 27

-212-

Specific - Nonspecific (con't)

 

 

 

 

. A B C SUM
Counselors

45 10 10 10 3o

46 8 8 8 24

47 5 3 3 11

48 6 6 6 18

49 10 8 8 26

50 4 6 o 10

5|. 4 4 3 11

52 4 4 0 8

53 .8 _z .3 _18 _

447 437 402 1286 $11m2=34836 12862=1.653,796
(311ng 4133 4011 3592 (12:11:36.
(Sum) 199809 190969 161604 552382
* Wafiance
Source of
Va a on df Sum of S res (SS) W2)
Counselors 52 21816 - 11.653ifl6 = 1211 23.90
3 159
Raters 2 552£2 - 1,633,226 = 21 10.50
53 159

Error 104 1335 - (1211 + 21) = ' 103 .99
Total 158 11736 - 10401 = 1335

 

(1) m cons2steng = 3:22; - :22 252
90

(2) Interme reliabiligz ( between-rater variance removed)

32.0- =22.1=‘882‘
23.90 + (3.1“.99) 25%

(3) M32 Egabihtz (between-rater variance left in)

=12].1-(21+103) =11132 = ‘860
2 1335
12 + + 103

aa'~"

.

7.

-‘

.—

3
~
,, ..

. o
l“
u

-v'- ~VC‘
»
.L I."

o On...
..

‘I". n
,‘3 3;"
‘—' . ‘J
w ,

 

 

 

 

-— _... ___——- —-A—“—_ —_—_—————_

*1“

._ a.

-213.

Exploratory _‘Wonexploratory Dimension-

 

 

Raters

.A B C SUM

Counselors
1 10 10 8 28
2 4 4 3 ll
3 10 9 9 28
4 10 10 10 30
5 10 8 6 24
6 8 8 8 24
7 6 6 6 18
8 8 10 10 28
9 10 10 7 27
10 10 10 9 29
ll 10 10 9 '29
12 7 7 7 21
13 10 10 10 30
14 10 10 10 30
15 7 7 7 21
16 10 10 9 29
17 6 8 6 20
18 10 10 8 28
19 10 10 10 30
20 10 10 10 30
21 110 10 10 30
22 10 10 10 30
23 10 10 10 3O
5 24 10 '10 10 30
25 10 10 10 30
26 10 10 10 30
27 10 10 10 30
28 10 10 10 30
29 10 10 10 30
30 10 10 10 30
31 10 10 10 30
32 7 7 5 19
33 10 9 9 28
34 10 10 10 30
35 - 10 10 10 30
36 10 7 3 20
37 10 10 6 26
33 7 7 7 21
39 10 10 9 29
40 9 9 10 28
41 10 10 10 30
«42 10 10 10 30
43 10 10 10 30
44 10 10 7 27

 

.
n
o
I
I
\

 

I\I\..

6214-

Ezmloratory - Nonexploratory (con't)

 

 

 

A B C SUE
Counselors
45 10 10 10 3O
46 5 8 8 21
47 5 3 0 8
48 10 10 10 30
49 10 10 8 28
50 10 9 l 20
51 6 4 1 11
52 6 6* 0 12
53 .2 .8. .9. .22
474 424 1379 Sum2=37613 13792-=1, 901, 641
971ng 4515 4402 3804 2x7: 12721
(Sum) 231361 224676 179776 635813
_f; 32237313 of variance
Source of 2
Variation df Sum.o S uares SS) variance (S )
Counselors 52 32613 - 1,201,641 = 578 11.1
3 139
Raters 2 63§§13 - 1,201,64 = 36 18.0
53 159
Error 104 761 - (578 + 36) = 147 1.4
Total 158 12721 - 11960 = 761

 

(1) Internal consistenqx= ll 1 -11 4>= 2. .2: , 823
11.

(2) Eter Edge rehabilitz (between-rater variance removed)
= 11 l - 1 Llr 2,2 =
(3) 11.12% reliability; (between rater variance left in)
= 578 - (36 + 142) = 486 = ,632
2 7
578 + (33 + 1575

 

 

 

'
g or 1 u ., .'-v . ‘. ‘ '
V " ~ . .\r‘v I.'\,_'_».’ ‘K ..
u
A
3
..
- .
a
‘|
. u
l
\
C I
. .
- “-.
..-- won. - .4
. - .1 u- .. w
a ‘ ‘ D ‘
u... o .0. . I."\\"“
-- o ‘
u u A . . -.. . A w
-\ .
1 4
.
new 1..» ”\AQ-» .10...- '4 '3. n..- ~. - . x
‘ 4 ‘ .
. ' :4 '
4
u
'C
a
4 v - ~"‘C ~OI . u
. . .
.
‘ .. I...
, . . . .
\ «v‘ . - .
a
A? ' ~' " ha
7 a . .
\IW" ‘ Q‘ ..oa--‘ on -
.
1 B
_. .. ..,. .. 4 . \' . ~~~
. I - ' . ..

..\-

”.-

 

.25.“

Effective - Noneffectize Dimension Scores

 

3*

 

 

7A B C SUM

Counselors
1 12 11 12 35
2 20 18 21 59
3 12 11 13 36
4 6 5 5 16
5 1.1 10 13 31"
6 13 11 11 35
7 11 11 11 33
8 8 7 6 21
9 8 6 6 20
10 11 10 9 30
11 9 7 8 24
12 10 10 10 30
13 6 4 4 14
14 8 7 8 23
15 10 8 10 28
16 9 9 ll 29
17 18 15 18 51
18 6 5 6 17
19 12 10 10 32
20 14 12 10 36
21 6 5 5 16
'22 4 4 4 12
23 10 10 10 30
24 6 5 6 17
2 5 9 7 7 23
26 5 5 4 14
'27 7 7 7 21
28 5 4 5 14
29 6 6 6 18
30 7 7 8 22
31 7 8 7 22
32 12 11 12 35
33 12 10 14 36
34 5 6 6 17
35 12 10 10 32
36 6 7 7 20
37 5 7 10 22
38 13 11 9 33
39 '11 9 10 30
40 11 9 7 27
41 6 6 6 18
42 8 8 8 24
43 7 7 7 21
-44 11 9 12 32

.~-..,.

 

JO~O~ D‘

~.,_--

I. O.‘

 

 

-216-

Effective - Noneffective (can't)

 

 

 

w—v

 

(1) Internal consistencz= 48. 0 - 1.1 a 46. 9

48.0 48. O

A B c SUI-i

Counselors

45 5 5 5 15
. 45 13 10 11 34

4? 10 10 10 30

48 12 9 11 32

49 11 8 11 3o

50 19 20 23 62

51 18 19 19 56

52 14 13 17 44

53 12 .13. 11 22

3qu2 520 471 509 1500 am2=49114 15002=2, 2505000

(Sum 3 5856 4837 .5817 211-16510 ‘
(Sum) 270400 221841 259081 751322

__ . s of Va e 11 2 1

Source of 2 .
Variation df Sum of Sguares $88) Variance {S )
Counselors 52 92114 - 2i250300 = 2220 48.0

3 59
Raters 2 22322 - 2,250,000 + 25 12.5
53 159
Error 104 2359 - (2220 + 25) = 114 1.1
Total 158 16510 .. 14151 = 2359 ‘

=a2.7'.Z

(2) lutergg’ dge reliability (between-rater variance removed)

= 48.0 - 1.1 = 363.2 =
48.0 + (34511.1) 50.2

(3) Inter E dge rehabilitx (between-rater variance left in)

= 2220 - (23 + 114)
2

£3

 

2220 + (25 + 1147" 2359

= 20': = I211

,,.‘
4.1!-

.1 .- ..
v'va
4
v
,\
'-

 

 

 

. .
_ ‘ r‘
.L"\ .
.
|
1.. .‘ . Q
. '1
I '-
. ‘J .
, i!
l A "“« -
.» . .
, 7
K
H» .
.
.
-,
I
. - N,'.)4"
3".

 

APPENDIX I

STATISTICAL "t"-TEST ANALYSIS COI-‘iPAKING
THE A-V SCALE SCORES OF THE OBJECTIVITY STUDY'S
EIGHT PH.D. CANIIIDATE‘S WITH THOSE OF THE

FORTY—F‘IVE 11.1. CANDIDATE

-217-

“1‘7“

Affect-Cognitive Dimension

 

COGNITIVE CATEGORY

Squared

Deviation

Cognitive
Scores

Deviation

From Kean

h.A. Candidates

017/Q/Q/5959999999993939799999999990/990/1990/5

mxn2n2no q,9.919191mwo1o1n691z391cJo11191919191919191919191Q5 919m8,9m 9M
11 11 11 /o 2

337730)3777737737373737777777777337737377.

16611114 . .2.111111112111116311911191111u111111111111111111116” .1111cm11 .11
oununonunwoonwnununununununuzjnuéunugunuornununonunvnonononvnununonunuqxnonunvo/

1111 11111111 1111 11 11 11 1111111+111111111111 1111 11 11
l234567890fl23456789012345678901185934567390
l 1111111122222222223 333333 34

(Aff.-Ceg. continued)

 

 

 

 

 

Cognitive Deviation Squared
Scores From Mean Deviation
R.A. Candidates
41 10 1.7 2.9
42 10 1.7 2.9
ZP3 10 1.7 2.9
44 7 -1.3 1.?
Sum 373 329.8
N 45
ii 8.3 Variance (821) = 329.8 = 7.3
‘45-
Cognitive Deviation Squared
Scores From Kean Deviation
Ph.D. Candidates
46 7 2.2 4.8
[+7 3 -l.8 3.2
48 4 — .8 .6
49 8 3.2 10.2
50 3 -1.8 3.2
53‘ 3 '108 3.2
5- 4 - o8 '6
53 _j_ .2 .4
am 37 26.2
N 8
I; 4 8 Variance (82 2)= 26.2 = 6.3
1 .75..
t = ii . 352 = 8.3 - 4.8 =3.5 3.5
l
2 2 (213 + 6 .3 (153.1) I 1
s 1 + s 2 (11 + 1?) 2
2—-«- 2
degrees of freedom (df) = Il + 112- =45 + 8 - 2 = 51
significance at .05 level = 2.00
significance at .01 level = 2.66

nmrx

Undorstqndingefionunderstanding»Dimension

 

ITOEULIDEJETMIDIITG CATEG RY

Nenunderstanding Deviation Squared
Scores From Kean Deviation
E.A. Candidates

1 7 - .9 .8
2 O -709 6201-“
3 9 1.1 1.2
4 10 2.1 4.4
5 4 “309 1502
6 8 .1 .0
7 6 -1.9 3.6
8 8 .1 .0
9 10 2.1 4.4
10 10 2.1 4.4
11 10 2.1 4.4
12 O -7.9 62.4
13 10 2.1 4.4
14 10 2.1 4.4
15 3 .4.9 24.0
16 9 1.1 1.2
17 5 -2.9 8.4
8 10 2.1 4.4
19 6 ”109 3.6
20 10 2.1 4.4
21 7 - .9 .8
22 10 2.1 4.4
23 10 2.1 4.4
24 7 - 09 .8
25 6 -109 306
26 7 " 09 08
27 10 2.1 4.4
28 10 2.1 4.4
29 10 2.1 4.4
30 10 2.1 4.4
31 10 2.1 4.4
32 0 -7.9 62.4
33 9 1.1 1.2
34 10 2.1 4.4
3 10 2.1 4.4
36 3 -4.9 24.0
3 10 2.1 4.4
38 7 "' 09 .8
39 9 1.1 1.2
40 9 1.1 1.2

K.A. Candidates

41
42
1.13
14;
[15
Sum
II
351

Nonunderstanding
Scores

10
10
10

5

.19..
354

45
7.9

 

 

Scores
Ph.D. Candidates
46 5
47 3
48 4
49 6
50 O
51 3
52 4
53 .2.
Sum 30
N 8
3'62 3.8
=219-3L8
8.6 + °.O
2
df = 51

signiiicance at .05 level =
si3nificance at .01 level - 2. 66

(Underst.-Eonunderst. continued)

Deviation Squared
From I can Deviation
2.1 4.4
2.1 4.4
2.1 4.4
-2.9 8.4
2.1 _4.4

384.8

Variance (S2 ) = 84.8 = 8.6
l ‘45" "

Nonunderstanding

 

Deviation Squared
From Hean Deviation

1.2 1.4

- 08 .6

.2 .4

2.2 4.8

-3.8 14.4

- .8 .6

.2 .4

1.2 1.4

24.0

variance (822) = 24.0 = 3.0

8
%l 14.1

 

5.1:) f?

"i c-I‘. on s be cij.‘ i c Dir-ten si on

o" . '7
LJ?)CC:1..L

 

110.18 P 2171 F1 C CAT IIIORY

Deviation Squared

IIonspecifi 0

Scores

Deviation

From I-lean

Ii . A . Candidat es

O.”208:4000000000008080000000000000ZOOM/Wouoo

0000 000
21 2/00 62222:422B26262222222222224 20412 22

..6h...466644446446464644444444 NA44/W44hW/nwhn/hwhwu.

L6 1..21111£113l2121111111111113 LL5L..LL
0390686000000050606000000000000090030800
1 1 1111 11 1 1 111111111111 11 1 11
1234567890u23456 890E234567no90fl/23hfi5673JO

1 111111112 222222223 331333334

J;

Nonspecific

Scores
H.A. Candidates
13. 10
MZ 10
43 10
1%! 10
1+5 .12..
Sm 389
N 45
Nonspecific
Scores
Ph.D. Candidates
as 3
13,7 3’
#8 o
be a
50 3
51 1+
52 4
53 _6
Sum 02
N 8
332 5.3

t - 8.6 - 5,3 =

1.0 + KL 1
/ 24 (115+

 

 

anJ

df = 51
significance at .05 level = 2.00
sicnifieanee at .01 level = 2.06

k.)

2.2...

> VA

.79

(Lpec.-Uonspec. continued)

Deviation
From Kean

Variance (821) =

Deviation
From Mean

ILI I

H NN MN

0 o o o o o o
VWWWVVWV

Variance (829) =

.213. = 11.8
.23

Squared
Deviation

 

7.0

°16.2 =
55

Squared
Deviation

29.6 = 3.7

.2?_.

Exploratory-Uonexploratory Dimension

 

EOKEKPLORATORY CATEGORY

Nonexploratory Deviation Squared
Scores From Mean Deviation
N.A. Candidates

1 10 .8 .64

2 4 -5.2 27.04

3 9 - .2 .04
4 10 .8 . 64

5 8 -1.2 1.44

6 8 -1.2 1.44
7 6 -3.2 10.24
8 10 .8 . 64

9 10 .8 .64
10 10 .8 .64
11 10 .8 .64
12 7 -2.2 4.84
13 10 .8 .64
14 10 .8 .64
15 7 -2.2 4.84
16 10 .8 . 64
17 6 -3.2 10.24
18 10 .8 .64
19 10 .8 .64
2o 10 .8 .64
21 10 .8 .64
22 10 .8 .64
23 10 .8 .64
2 10 .8 .64
25 10 .8 .64
26 10 .8 .64
27 10 .8 .64
28 10 .8 .64
29 10 .8 .64
30 10 .8 .64
31 10 .8 .64
32 7 -2.2 4.84
33 9 "‘ .2 0014’
3’25 10 08 '64
3 10 .8 .64
36 7 -202 14'0811’
37 10 .8 .64
3: 7 -2.2 4.84
39 10 .8 .64
40 9 - .2 . O4

(3xpl.-fionexpl. continued)

 

 

 

 

Nonexploratory Deviation Squared
M.A. Candidates Scores From Kean Deviation
41 10 .8 .64
42 10 .8 .64
43 10 .8 .64
44 10 .8 .64»
[1'5 10 .8 064
Sum 414 95.20
N 45
1' . 2 _ _
k1 9.2 Variance (S l) — 22.20 — 2.1
5
Nonexploratory Deviation Squared
Scores From Mean Deviation
Ph.D. Candidates
46 8 1.0 1.0
47 3 -4.0 16.0
48 10 3.0 9.0
49 10 3.0 9.0
50 7 0 0.0
51 4 -3.0 9.0
52 6 -100 1.0
53 8 1.0 1.0
Sum 56 46.0
N 8
- 2
7.0 Variance (S ) = 46.0 = 5.8
1‘2 2 73-70
t = 2,2 - 7.0 = 2.2 = 2.2 = 8.5
.26
2.1 + 5.8 _l___ + 1 6.8
2 45 8
df = 51.
significance at .05 level = 2.00
sigrifieance at .01 level = 2.66

ON “‘x /

1‘.." f o ctive-Z‘! onef f ective Dirtensi on

 

Squared

Deviation

Deviation

From Iloan

Scores

. Candi dates

-.

1~'

24.88 828.24.08.6.80.80 820280008208002002088888
O O O O
3O.4464.6.40.47.67.4 07003047704374770900704744
1 4 41 112 916121 22 2211 126

88220028002228228ALBA/~82?»22228222220U22287x)22

1B226263 24N52947343255 2152553M 3572222
. . . . . . . . . . . . . . . . . . . . . . . . . . . .

7933282253.“.30351320230053703002052033833
1211212111 3112...212111111111111311121111
1234567890 L2 .456 7890fl234567890£o 3hfi56 7390

111111112 2222222230 330 333334

(Eff.-Honeff. continued)

 

Deviation Squared
Scores From fiean Deviation
E.A. Candidates
41 10 -5.2 27.0
42 10 -5.2 27.0
43 10 -5.2 27.0
45 10 -5.2 22.0
Sum 682 1511.2
N 45
_. . 2
15.2 variance (3 2) = 15%;.2 = 33.6
X2 5
Deviation Squared
Scores From Kean Deviation
Ph.D. Candidates
45 18 —6.9 47.6
47 25 .1 0.0
48 22 ‘209 8014‘
49 22 -2.9 8.4
50 33 8.1 65.6
51 29 4.1 16.8
52 28 3.1 9.6
53 .22.. -2.9 ..a-.4.
Sum 199 164.8
11 8
X 24.9 Variance (52 ) = l .8 = 20.6
1 1 _.87—'
t =2 24.9 — 15.2 = 9_.Z = Z2 = 4.2

 

 

Significance at .05 level = 2. 00
significance at .01 level - 2. 66

.n.‘

MP STDI X J

DEPEIEDELZ’T -"t" STATISTICAL AI‘JALYSIS
COERITATIOI‘S COI-ZPARII-IG THE IIOIGSUPm‘JIS-ED GRCIIP'S

INITIAL AND TERI'II'JAL II-ETERVIEM SCORES

Affect Categonz

 

Eonsueervised Group (Client-Interrogation)

 

 

AffC‘Ct SCOI'G AffBCt Score Difference Difference?
Initial Interview Terminal Interview (3) (32)
Counselor
Enrollees
F l 1 0 0
s 2 3 1 1
H O 0 0 0
I 2 3 l l
J _;L_ 2 l 1
5“” 5’ Q ZD = 3 209-: 3
h 5
“ 2 2
x 1.2 1.8 (2D) = (3) = 9
dependent --t" =3 2 '5 = 3 = .6 = .6 = 2 n
1: .5 :EEC'

 

n.)

 

 

 

D .1..—
UZDZ -4213} / ..
112m .. 1) 25255
degrees of freedom = N-l = 5-1 = 4
significance at .05 level = 2.78

Understanding Cateeogz

Nonsupervised Group (ClientAInterrogation)

Understanding Score Understanding Score
Initial Interview Terminal Interview
Counselor
Enrollees
F 8 8
G 2 0
H l O
I 4 2
J 6 2
Sum 21 12
N 5
2 4.2 2.4

It can be seen that there was a decrease rather than
an increase from the initial to the terminal ratinfis.

 

Honsupervised Group (Client-Interrogation)

 

Specific Score Specific Score Difference Difference
Initial Interview Terninal Interview (D) (Dz)
Counselor
Enrollees
F 2 4 2 4
G O l l 1
H 0 O 0 0
I l l 0 0
J _l__ .22.. ._1_. .1.
Sun 4 8 4 6
h 5
a. 2
A .8 1.6 (4) = 16
dependent “t” = 4 = .8 = 8 = 2.2
5 .47

 

 

6-16
25
(11:4

significance at .05 level = 2.78

Ehploratory Categogx

Nonsupervised Group (Client-Interrogation

Exploratory Score Exploratory
Initial Interview Terminal Interview
Counselor
Enrollees
F 9 6
G O 2
H 0 0
I 5 2
J _.L .9.
Sum 17 10
u 5
I 3.4 2.0

It can be seen that there was a decrease rather than
an increase from the initial to the terminal ratings.

Effective Category

Konsupervised Group (Client-Interrogation)

 

Effective Score Effective Score
Initial Interview Terminal Interview
Counselor
Enrollees
F 36 37
G 22 24
H 21 21
I 31 25
J 26 22
Sun 136 129
N 5
SE 27.2 25.8

It can be seen that there was a decrease rather than
an increase from the initial to the terminal ratings.

APPENDIX K

DEPEIIDEITT - "t" STATISTICAL ANALYSIS
COL-3UT1‘1TIONS COIJJMYIIEG THE SUPERVISED GROUP'S

IHITIAL I‘d-TD TEm-II‘IAL INTERVIEJ SCORES

fl

no 1- _,
113.001. C9. teg’ll‘jf

S1eervised Group (Counselor Interrogation)

 

 

 

 

 

 

 

Affect Score Affect Score Difference Difference:
Initial Interview Terminal Interview ' (D) (D3)
Counselor
Enrollees
A 7 8 l 1
B O 4 4 16
C 4 10 6 36
D l 4 3 9
E l 10 2 81
Sun 13 36 5,1) = 23 £132: 143
L 5 2 2
A 2.6 7.2 GED) = (23) = 529
dependent ”t” = {_D = '5 = 23 z 4 6 = 4.6 = _;
I} 2 -‘-- l. .1-
S D .5 71.85 35
1:102 - (infi ‘/ 59.422 .. 522
11201 .. 1) 25
degrees of freedom = N - l = 5 1 = 4

significance at .05 level = 2.7
significance at .01 level = 4.6

5
O
UnderstandingVCategogx
Supervised Group (Counselor Interrogation)

Understanding Score Understanding Score Difference Difference:

 

 

 

Initial Interview Terminal Interview (D) (02)
Counselor
Enrollees
A , 14 17 3 9
B 3 15 12 144
C 5 l4 9 81
D 7 5 2 4
3 _2_ 12 14 196
Sux 34 70 40 434
11 5
.: 6.8 14.0 (40)2 = 1600
dependent "t“ = .549 = 8.0 = 8.0 = 2.2
i 2.38
-* V5.70
51434) - 1600
25(4)
(if : )3,
significance at .05 level = 2.75
significance at .01 level = 4.60

Specific Category

 

Supervised Group (Counselor Interrogation)

 

 

Specific Score Specific Score Difference
Initial Interview Teminal Interview (D)
Counselor
Enrolloes
A 7 l7 7
3 1 l4 13
C 3 10 7
D 4 14 10
E 2 8 16
Sum 17 73 53
21 5
1: 3.4 14 6 (53)2 -_-
dependent “t” = 53 = 10.6 = 10.6 = 6.1
.5 1.73
V 3.06
62 - 28
25 ‘
df = 4

significance at .05 level = 2 7
significance at .01 level = 4.6

Exploratory Cate 501.1

Supervised Group (Counselor Interrogation)

 

 

 

 

 

Exploratory Score Exploratory Score Difference
Initial Interview Terminal Interview (D)
Counselor
Enrollees
A l5 l9 4
B l 16 15
C 6 l4 8
D 8 l4 6
E _§L_ 20 14
811121 36 83 47
11 5
x 7.2 16.6 (47)2 =
dependent "t“ = 47 = 9.4 = 9.4 = 4.6
5 2.09
4.36
/ 51522) - 2209
25(4)
01‘ = 4
significance at .05 level = 2.75
dignificance at .01 level = 4.60

Difference
(I)2 )

N

\)H l—’
O-P‘CN-‘w

I
FROG‘OQ

Difference2

fiEfective Cotevonz

 

Supervised Group (Counselor Interrogation)

 

 

 

 

Effective Score Effective Score Difference Difference2
Initial Interview Initial Interview (0) (02)
Counselor
Enrollees
A 44 54 10 100
3 25 49 24 276
C 29 52 . 23 52
D 33 42 9 81
E 26 6O 24 ll 56
Sum 157 257 100 2442
H 5
x 31 51 (100 )2 = 10, 000
dependent "t” = 100 = 20 = 20 = 4.2
__5 5.7
22.1

 

/ 5(0002) .. 10,000
25(4)

E
E.
{‘5

O

(D
- 93
(1'

o

O
H
H
(D
i
II II
(TN
0

O\
O

»\..
Pun

fl

avg: ........ 3.5...”