IN FO RM A TIO N T O USERS

This material was produced from a microfilm c o p y of the original d o c u m e n t. While
the m ost advanced technological means to p h o to g ra p h and repro d uce this d o c u m e n t
have been used, the quality is heavily d e p e n d e n t upon the qu a lity of the original
su b m itte d .
The following explanation o f techniques is provided to h elp you u nderstand
markings or patterns w hich m a y appear on this re p ro du ctio n .
1 .T h e sign or " ta r g e t " for pages a p p a re n tly lacking from th e d o c u m e n t
photographed is "Missing Page(s)". If it was possible t o obtain th e missing
page(s) or section, th e y are spliced in to th e film along w ith adjacent pages.
This may have necessitated cutting th ru an image and duplicating adjacent
pages to insure y o u com plete c o n tin u ity .
2. When an image on th e film is o bliterated with a large ro u n d black m ark, it
is an indication th a t th e pho to graph er suspected th a t th e copy m ay have
moved during e x p o su re and thus cause a blurred image. You will find a
good image of the page in the adjacent frame.
3. When a map, draw ing or chart, e tc ., was p a rt of th e material being
photographed th e photographer followed a d efin ite m e th o d in
"sectioning" the m aterial. It is c u sto m a ry to begin p h o to in g at th e up per
left hand corner o f a large sheet a nd to continue p h o to in g from left to
right in equal sections with a small overlap. If necessary, sectioning is
con tin ued again — beginning below th e first row and continuing on until
com plete.
4. T he m ajority of users indicate th a t th e textual c o n te n t is of greatest value,
however, a s o m e w h a t higher quality rep ro du ctio n co u ld be m ade from
"p h o to g ra p h s" if essential to the u n d erstand ing of th e dissertation. Silver
prints of " p h o to g ra p h s " may be o rd e re d at additional charge by writing
the O rder D e p a rtm e n t, giving th e catalog n u m b e r, title, a u th o r and
specific pages you wish reproduced.
5. PLEASE NOTE:
received.

S o m e pages m a y

have in distin ct print. Filmed as

Xerox U niversity M icrofilm s
300 North Z eeb R o ad
Ann A rbor, M ich ig a n 48106

73-29,749
MILLAR, John James, 1943THE GRADUATE ADMISSION PROCESS IN TWO
BEHAVIORAL SCIENCE DEPARTMENTS AT MICHIGAN
STATE UNIVERSITY.
M ichigan S ta te U n iv e r s ity , P h.D ., 1973
E ducation, higher

U n iv ersity Microfilms, A \ E R Q \ C o m p a n y , A n n Arbor, M ic h ig a n

©

C o p y r i g h t by
JOHN JAMES MI LI AR
1973

Till; GRADUATE ADMISSION PROCESS

IN TWO BEHAVIORAL

SCI UNCI: DEPARTMENTS AT MICHIGAN STATE ON I VCRS ITY

By

John James M illa r

A 11IUS I S

Subm itted to
M ichigan S ta t e U n iv e rsity
in p a r t i a l f u l f i l l m e n t o f th e re q u ire m e n ts
fo r the degree of

DOCTOR OF PHILOSOPHY

Department

of A dm inistration

1973

and H i g h e r

Education

ABSTRACT

THE GRADUATE ADMISSION PROCESS IN TWO BEHAVIORAL
SCI E NCE DEPARTMENTS AT MICHIGAN STATU UNIVERSITY

By

John James M ill a r

Tlie p u r p o s e
(1)
the
in

To d e v e l o p a n

level

decision

estim ating
(3)

to

the

In a n
future

in the

substantiate
level

of
if

one kind o f

effort

for

current

the

graduate
the

for

use

graduate

in

determ ining

student

in

process

value of

population

in

im portance

average and

10 p e r f o r m a n c e

adm ission
data

developed

is

o f adm ission

school, data

students

school

adm ission

process

for a behavioral

Performance

the

set

for

useful

o r program.

the

graduate

the

of

performance.

situation

118 g r a d u a t e
graduate

representation

estim ation

to d ete rm in e

performance

adm ission

collected

the

procedure

a quantitative

To d e t e r m i n e

i n more t h a n

of

three-fold:

studied.

To d e v e l o p
and

s t u d y was

evaluation

o f performance

the d e p a r t m e n t s
(2)

to

of this

was m e a s u r e d

categories.

for 5 predictors

science

who m e t

Data

the
by

criteria

department

population
the g ra d u a te

on t h e

employed

were

conditions.
grade-point

performance

.John .James Mi I 1 n r

categories

were

collected

triadic-com parison
ability

on

each

variate

scalc--a

category

Th e a c t u a l

from th e

analysis

forced

for

adm ission

departm ent
choice

placem ent

decision

o f mean d i f f e r e n c e s

analysis

the

predictors

to p re d ic t

scores provided

the method

for

as p e r c e i v e d

g ro u p s on

those

differences
Ibis
ment

ability

performance

wa s

seven

the

ndm ission

population

R esults

in

the

students

one

rating.

one

rating

was

receiving

department

of

rater.
in

linear

performance
probability

a m ulti­
group
regression

category
of succcss--

Tour a d m i s s i o n

were

placement

significant

group

evident.
science

fo r a sample o f

one

were as

differences
rating

reliability

depart­

147 s u b j e c t s

follow s:

on t h e

performance

and s t u d e n t s

on

students

receiving

criteria
more th a n

who r e c e i v e d m o r e t h a n

regression

analysis

for predicting

indicated

n strong

predictive

placem ent

decision

R = .56.

P redictive

reliab ility ,

predicted

and a c tu a l

group placem ent

those

the

level

a

r *= . 4 4 .

The m u l t i p l e

I:o r

the

M ultiple

the

the

of

conditions.

first

Intcrrater

of

significant

another behavioral

predictors

b y means

employed

where both

predictions

T h e r e w e r e no s i g n i f i c a n t
between

developing

in

by

determ ine

expected

level--for

replicated

was

criteria.

categories

and s i g n i f i c a n t

study

using

w h o mot

11 p e r f o r m a n c e

to

on t h e

dcfincd

com parison

a subs.ample s e l e c t e d

differences
of

faculty

performance
(p<.05)

and

evident,

relationship

in which
exists

on

the

scores,

categories

group d iffe re n c e s
a strong

based

found

actual

relationship

agreement

was
to

the

r

w ith

between

= .70.

contain

significant

adm issic

significant

prediction

between p r e d i c t e d

was

placem ent

J o h n Jame s M i l l a r

and

the

probability

having high

o f a subject

in

the

to

process

analysis

evident
of

viously

r e v i e w e d by

in

a significant
a successful

these

the

and

as

possible

decision-m aking

to

research

improve

the

in

first

indicate

that

and

the

data

ability

is

of

by

A

the

the

by

replacing

approach w ith

an

a d d i t i v e - p r o b n b i 1i t y

faculty

to

factors

program.

current

to

the

per­

pre­
report

provide
achieving

Therefore,
adm ission

patterning
model.

to

intellectual

stu d e n t's

input

process

possible

of w ritten

opposed

graduate

adm ission

of

the p ro b ­

have been

effectiveness

w o rk e r as

the

three

departm ent.

it

a measure

regarding

Multi pi

categories.

The p r o b a b i l i t y

estim ate

level

on

o th e r measures

a teacher.

performance

prediction

as h a v in g h ig h

i.e .,

im portance
five.

the

in

was n o t

original

performance

where

be o f

group placem ent

clearly

faculty;

reliable

the

in

performance w ith

areas

the

to

analyzed

three

study

found

a significant

being perceived

a productive

and p ro m ise

added

between p r e d ic te d

perform ance

as

in

categories

student

in

and p ro m ise

and were

resulted

subject

graduate

variables

as

adm ission

were

ception

is

of

predictors

The r e s u l t s

it

point

TWo a d d i t i o n a l

f a c u l t y was

breadth

the

replication

of the

document

at

a com plete

relationship

ability

faculty

available,

four performance

strong

by t h e

was

adm ission

regression
the

group d i f f e r e n c e s

second departm ent

possible.

perceived

ability.

As n o m e a s u r e o f
the

being

of

ACKNOWLFUGF.MHNTS

I would
fo r his
He h a s

a

through
to

most

like

to

able

assistance

long

countless

honored

I also

trials

and

Ur.

passing

Cl.

interest

I am d e e p l y

him and

thank

you

Ra y m o n d

to

W alter

F.

.Johnson

my d i s s e r t a t i o n

advisor.

new r e s e a r c h e r s

I count

m yself

follow ed

com pletion

indebted

to

the

for

a special
for

of

faculty

their

his

as

honored

by many more

o t h e r members
Ur.

contributed

w ith

McSweeny,

more

than

a

study.
of

the

participation

in

this

to

o f my

M arycllen

members

thanks

able

the

Franktnann,

this

the

U epartm ent

Ur.

assistance

Thomas
in

the

1..

Uepartm ent
study.

1

Conner o f

development

of

the

of

design.
in

debt

assistance

To t h i s

next

ever express.

to

in

the

the

secretarial

collection

gentlem an,

of Sociology

days

in

for

four years

adm inistration

of

of

the w r i t e r

I have worked w ith

ment

the

llr.

leading

be

to

express

I am a l s o
their

] w ill

all

to

for

of

S t a m a t a k o s , who h a v e

would

research

I know

Ur.

and S o cio lo g y

the

history

as

tribulations.

a sincere

Psychology

Sociology

and

com m ittee:

Louis

like

and c o m m e n ta r y

to

persons.

express

dissertation

my a p p r e c i a t i o n

and d i s t i n g u i s h e d

have worked w ith

equally

express

Ur.

staff
the

in

this

study.

owes much more

titan

he

A.

Faunce o f

a n d we h a v e b e e n

that
iii

departm ents

for

W illiam

data

both

departm ent.

through

From B i l l

the
up

could
U epart­

and down

Faunce,

I have

learned,

to weigh
thanks

arguments

finish

sons

things,

carefully.

to

I shall

think

before

acting

miss w orking w ith

and

him;

Bi I I .

F inally,
to

a mo n g o t h e r

to

my w i f e

a n d who h a s p u t

Stephen,

who t h i n k s

. J o y c e , w h o h a s b e e n b e s i d e me f r o m s t a r t
up w i t h
when

my m o o d s

I become

Daddy a n y m o r e ,

a n d M i c h a e l , who j u s t

1 am m o r e

g r a te f u l- - they

than

cntcrpri s c .

arc

the

and q u i r k s ;

a Doctor

got

here

reasons

and

T cannot

for

the

behind

big
the

to

my

he h is
finish,
entire

TAR LI; OF CONTENTS

Page
L I S T OF T A B L E S .........................................................................................................................

vi i i

L I S T OF F I G U R U S ...................................................................................................................

xi

L I S T OF A P P E N D I C E S .............................................................................................................

xii

INTRODUCTION

...............................................................................................................................

3

STATEMENT OF THE P R O B L F . M .......................................................................

5

P u r p o s e o f t h e S t u d y ...........................................................................
N e e d f o r t h e S t u d y .................................................................................
O verview
........................................................................................................

5
5
14

CHAPTER
I.

II.

REVIEW OF PREVI OUS S T U D I E S

.................................................................

G r a d e - P o in t Average as P erfo rm an ce C r i t e r i o n . .
G ra d c -P o in t Average a n d /o r F a c u lty R a tin g s
a s P e r f o r m a n c e C r i t e r i o n ..........................................................
Performance C rite rio n o th e r than G rade-Point
A v e r a g e a n d / o r F a c u l t y R a t i n g s ..................................................
S u m m a r y ...................................................................................................................
III.

RESEARCH DESIGN

15
16
IK
25
54
56

Selection o f
t h e S a m p l e ..............................................................
S election o f
t h e P r e d i c t o r s ..................................................
Selection o f
t h e C r i t e r i a ........................................................
C onstruction
and P r e - t e s t i n g o f th e T r i a d i c C o m p a r i s o n S c a l e .................................................................................
C o lle c tio n and P rep aratio n o f the P re d ic to r
D ata
..............................................................................................................
C o l l e c t i o n and P r e p a r a t io n o f th e C r i t e r i a
D a t a ...................................................................................................................
D a t a A n a l y s i s P r o c e d u r e s ................................................................
S u m m a r y ..............................................................................................................

v

56
57
40
45
50
52
55
58

TABLE OF CONTENTS

(C o n tin u ed )

CHAPTER
I V.

Page
THE DEPARTMENT OF S O C I O L O G Y ...............................................................
C u r r e n t A d m i s s i o n P r o c e d u r e s ....................................................
P o p u la tio n an d Sample S ize
..........................................................
Triadic-C om parison R esults
.
T e st f o r Group D if f e r e n c e s
.......................................................
P rc d ic to r-P e rfo rm a n c e Category R e la tio n sh ip s
. .
M ultiple R egression A nalysis R esults
.............................
R e l i a b i l i t y o f th e P r e d i c t i o n liq u atio n s on
t h e B a s i s o f G r o u p D i f f e r e n c e s ........................................
P r e d i c t i v e A c c u ra c y Based on D ic h o to m iz e d
G r o u p s ..............................................................................................................
P r e d i c t i v e A c c u r a c y i n Terms o f G roup P l a c e ­
...........................................................................
ment P r o b a b i l i t i e s
T h e P r o b l e m o f A f f e c t ...........................................................................
G r a d u a t e GPA a s a P e r f o r m a n c e C r i t e r i o n .......................
E f f e c t s o f t h e A c t u a l A d m i s s i o n D e c i s i o n on
the G rad u ate Program
.....................................................................
Comparison o f T hree P r e d i c t i o n P ro ced u res . . . .
Summary o f R e s u l t s
.........................................................

V.

DEPARTMENT OF P S Y C H O L O G Y .....................................................................
C u rren t Adm ission P ro ced u res
. . . . . . . . . .
P o p u l a t i o n a n d S a m p l e S i z e .........................................................
T r i a d ic Comparison R e s u lts
..........................................................
D eterm in atio n o f Group D iffe re n c e s - F in a n c ia l
A s s i s t a n c e ..................................................................................................
D e te rm in a tio n o f Group D iffe re n c e s - C u rren t
S t a t u s i n P r o g r a m .................................................................................
D eterm in atio n o f Group D iffe re n c e s - O v erall
E valuation
..................................................................................................
M ultiple R egression A nalysis R esults
.............................
A d d i t i o n o f T w o P r e d i c t o r s ..........................................................
T e s t F o r G r o u p D i f f e r e n c e s ..........................................................
P rc d ic to r-P e rfo rm a n c e Category R e la tio n sh ip s
. .
M ultiple R egression A nalysis R esults - 7
P redictors
..................................................................................................
P r e d i c t i v e A c c u r a c y B a s e d on D i c h o t o m i z e d
G r o u p s ..............................................................................................................
P r e d i c t i v e A c c u ra c y i n Terms o f Group P l a c e ­
ment P r o b a b i l i t i e s
...........................................................................
Summary o f R e s u l t s
.................................................................................

vi

59
59

61
62
<>3
64
65
70
75
R0
82
84
86
no
02
«>B
95
97
97
98
103
105
107
109
Ill
112
113
115
116
118

TABLE OF CONTENTS

[C o n tin ued)

CHAPTER
VI.

Page
SUMMARY OF’ MAJOR F I N D I N G , CONCLUSIONS AND
POLI CY I M P L I C A T I O N S ......................................................................................

Research

121

D e s i g n Summary ...............................................................

121

M ajor F in d in g s - Departm ent o f Socio logy
. . . .
M ajor F in d in g s - D epartm ent o f Psychology
. . . .
C o n c l u s i o n s .............................................. .........................................................
P o l i c y I m p l i c a t i o n s o f t h e P r o b a b i l i t y Model
. .
F u t u r e R e s e a r c h P o s s i b i l i t i e s ....................................................

123
124
123
125
129

B I B L I O G R A P H Y ............................................... ..... .........................................................................

131

A P P E N D I C E S ....................................................................................................................

vi i

1 3E

L i . S t 01 - t a b l e s

TABLE
4.1

PAGE
M u ltip le R egression R esults - P re d ic tio n o f
A d m i s s i o n D e c i s i o n ..... ......................................................................................

f>S

M u ltip le R egression R esults - P re d ic tio n o f
T r i a d i c - C o m p a r i s o n R e s u l t s .....................................................................

f>8

4.3

Com parison

...................................

72

4.4

D ichotom ized Group R e l a t i o n s h i p b etw een A ctual
a n d P r e d i c t e d A d m i s s i o n D e c i s i o n P l a c e m e n t .......................

74

D ich o tom ized Group R e l a t i o n s h i p b etw een Actual
and P r e d ic t e d Perform ance C ateg o ry P lacem ent:
O r i g i n a l i t y i n R e s e a r c h ................................................................................

77

4.2

4.5

4.

4.7

4.8

4.9

4.10

4.11

4.12

(>

of

A dm ission

Group P la c e m e n ts

D ich o to m ize d Group R e l a t i o n s h i p betw een A ctual
and P re d ic te d Performance C ategory Placem ent:
F . f f c c t i v c n e s s o f W r i t t e n R e p o r t s ....................................................

77

D ichotom ized Group R e l a ti o n s h i p b etw ee n A ctual
and P r e d i c t e d Perform ance C ategory P lacem en t:
P ro m is e as a P r o d u c t i v e R e s e a r c h Worker
. . . . . .

78

D ic h o to m iz e d Group R e l a t i o n s h i p b etw ee n A ctu a l
and P r e d i c t e d Perform ance C ategory P lacem ent:
O v e r a l l E v a l u a t i o n .................................

7R

P r o b a b i l i t y o f P e r c e i v e d S u c c e s s B a s e d on
G r o u p P l a c e m e n t .......................................................................................................

80

R e l a t i o n s h i p o f A greement on Group P la c e m e n t
A c r o s s S e l e c t e d P e r f o r m a n c e C a t e g o r i e s ...................................

82

D icho tom ized Group R e l a ti o n s h i p Between A ctual
a n d P r e d i c t e d G r a d u a t e G P A .....................................................................

85

A c tu a l Group P la c e m e n t o f an A d m itte d S tu d e n t
S u b s a m p l e .........................................................................................................................

87

vi i i

L I S T OF TABLES

(C o n tin ued )

TABLE
4.13

I1ACL
C u r r e n t S t a t u s o f S u b s a m p l e Who E n r o l l e d .
F a l l 1 9 7 1 ............................................................................................................
-

Performance

4.14

Admission

..............................

89

4.15

Comparison o f P r e d ic t io n P ro c e d u re s f o r O v e r­
a l l E v a l u a t i o n C a t e g o r y ...........................................................................

91

A ssistance

Placem ent

5.1

Financial

5.2

S u b j e c t G ro u p in g Based on C u r r e n t S t a t u s in
P r o g r a m - F a l l 1 9 7 1 ...............................................................................
E valuation

C ategories

S hifts

88

5.3

O verall

Groups

5.4

M ultiple

5.5

N atural

5.6

M ultiple

S.

7

5.8

P ro b a b ility o f Perceived
P l a c e m e n t ............................

Regression
Science

............................................................

......................................................

A nalysis

R esults

Coding C a t e g o r i e s

Regression

A nalysis

-

104

. . . .

5 Prcidictors

106
.

......................................................

R esults

-

7 Predictors

D i c h o t o m i z e d A g reem e n t o f S c o r e s on S u c c e s s f u l
M a r g i n a l P l a c e m e n t ....................................................................................
Success

100

Based

.

108
Ill
114

or
116

on Group
117

6.1

P r e d i c t e d Group P lacem en t o f N o n -A d m itted A p p lic a n ts
- S o c i o l o g y .........................................................................................................1 2 7

B.l

M u ltip le R eg ressio n A n a ly sis D ata - D epartm ent
o f Sociology:
Adm ission P lacem ent D e c isio n . . . .

142

M u ltip le R e g re ssio n A n a ly s is Data - D epartm ent
o f Sociology:
G raduate G r a d e -P o in t Average

143

B.2

. . . .

B.3

M u ltip le R eg ressio n A n a ly sis D ata - Departm ent
o f Sociology:
O r i g i n a l i t y i n R e s e a r c h ............................. 1 4 4

B.4

M ultiple

R egression

o f S o c io lo g y :
B.5

A nalysis

D ata

-

Departm ent

E f f e c t iv e n e s s o f W ritten R eports . .

M u ltip le R egressio n A n a ly sis D ata - D epartm ent
o f Sociology:
Prom ise as a P r o d u c tiv e R esearch
W o r k e r ......................................................................................................................... 1 4 6
ix

145

LIST

or TABLES ( C o n t i n u e d )

TAB LI:
B.6

C .l

C.2

C.3

C.4

C.S

PAGE
M u l t i p l e R e g r e s s io n A n a ly s is Data - Department
o f S o c i o l o g y : O v e r a l l E v a l u a t i o n ...............................................

147

M u l t i p l e R e g re s s io n A n a ly sis Data - Department
o f Psychology:
G raduate G ra d e -P o in t Average.

149

. . .

M u l t i p l e R e g re s s io n A n a ly sis Data - Department
o f Psychology:
M astery o f Fundam ental Knowledge.

.

ISO

M u l t i p l e R e g r e s s io n A n a ly s is Data - D epartm ent
o f Psychology:
E ffe c tiv e n e ss o f W ritten Reports.

,

1S1

M u ltip le R egression A nalysis
o f Psychology:
Prom ise as
search W o rk e r

Data - Department
a P r o d u c t i v e Re­

M u l t i p l e R e g re s s io n A n a ly s is Data - Department
o f Psychology:
O v e r a l l E v a l u a t i o n ........................

x

152

153

LIST

or F I GUM'S

FIGURE
4.1

PAGI:
Group L o c a t i o n s

on

the

Continuous

xi

Scale

.......................................

71

L I S T OF APPENDICES

APPENDIX

PAGE

A

Triadic-C om parison

........................................................

136

H

M u l t i p l e R e g r e s s io n A n a ly s is Data D e p a r t m e n t o f S o c i o l o g y ........................................................................

141

M u l t i p l e R eg ress ion A n a ly s is Data D epartm ent o f Psychology
..................................................................

148

C

Rating

Scale

xii

/

INTRODUCTION

Whenever
results

a rarely

journal

mcnt

to

o f a research

answer to
the

I begin

study,

answered

article

usually

deals

form at,
w ith

why

researching

o r how t h i s

the

problem

in q u e s t i o n .

for

doing whatever

to

follow .

rend

it

However,

n journal
I am a l w a y s

question.

this

research

w ill

question

justification

but

from t h e

myriad o f p ro b lem s

vast

studied?
this

In t h e e v e n t

question,

problem

1 plan

investigated

On e o f
vise

how d i d

the

programs.
on t h e

that
to

in

duties

and c o - o r d i n a t e

the

other

of certain

c o n t i n u u m d e s i g n e d by

of

standard

com m ittee

me h i s

current

caliber

than the

the

1970

was

applicant

paragraphs

is

not

choose

also

occupational

this
to

be
in

o f how t h e

status

is

to

and e v a l u a t e s

itself.

During

c a m e t o my o f f i c e

applicant

1

problem

interested

adm itting persons

pool.

the

selected.

criteria

corm ittee

that

the
in

on

a ju stificatio n

pool

was

I t wa s h i s

the
to

to

super­

our graduate

com m ittee which s c r e e n s

1971 o n e member o f t h i s
finding

is

in

state

research

have been o r ou g h t
are

the

n problem worth

now a b r i e f h i s t o r y

process

the

statem ent

readers

thesis

finding

opening
is

I am i n t e r e s t e d

which

We e m p l o y a t h r e e - m a n

basis

problem

be discu ssed

o f my c u r r e n t

the

the

r e s e a r c h e r come t o

present
this

the

the

fa m ilia r w ith

im proved on p r e v i o u s

logically,

the

nhout

I f you a r c

y o u know t h a t

that

reporting

curious

then

research

is

nrticle

all
them

applicants
along

a

W inter o f
discuss

clearly
opinion

of
that

w ith
a

low er
on

all

criteria
tim e

last

y e a r ’s

I had n o t

seen

applicants
enough

finding

intelligently,

attem pt

to

check

his

A pproxim ately
the

com m ittee

come t o
of

the

see

superior

sidered!

W hile

the

second

the

p ro b lem was
At

it
to

him self

was
the

for

application,

interpreted

developed

com m ittee

set

of

adm ission

ferent

pretations,
how w e r e

that

our

what

they

the

adm ission

is

the problem o f

was

in need o f

the

the

applicant

data

there,

looked

at

agreed

data

his

make an

on

pool

fact,

data

in
the

our

can
and

first

member,

to

over

had

60S. o f

read

adm it

that

com m ittee.

w ith

the

each

or

reject.
a pre­

A three-m an

sumo e v a l u a t i o n
at

by

there.

to by

standards

significantly

dif­

data.

were s u b je c t

to

such

the departm ent

programs?

interpretation

departm ent

the

criteria

stu d e n t's

we c o n ­

date

same

was

were

later

the

this

a review

criteria

painfully

arriving

also

a

com m ittee

the

he had

applications

all
at

member o f

A fter

we h a d p r o c e s s e d

still

students

answers

1971

claim o f

in

em ploying

regarding

to

the

seems

pool.

a n d made a d e c i s i o n

were

of

the

pool

A three-m an

it

"hard"

type

some

applicant

discussion

had

perform ing

of

problem w hich

that

a n d who w e r e

interpretations

Given

that

discuss

a second

It

opinion

standards

data,

my o f f i c e .

his

year.

contmi t t e e t w h o s e m e m b e r s
the

to

at

I would

e l a p s e d when

applicant

of

this

that

in

substantiate

applications

A three-m an

member t h a t

current

nevertheless
of

applicants

the

1970

a review

tim e

the

o n e - h a l f h o u r had

member d i d

the

current

Since

findings.

me r e g a r d i n g

clearly

the

I assured

presented

applicants

of

w e r e mu c h b e t t e r .

Wh a t

is

drastic

adm itting

the

the

adm ission

F inally,

process

resolve?

The

departm ent

the w rite r

had

a

thesis

and

relationship

program perform ance?
in

inter­

in

a

question

problem .

for

/

CHAPTER I

STATEMENT OE TUT PRORLTM
Purpose o f

the

Study

The p u r p o s e o f t h e
Cl)

To d e v e l o p

an

study

evaluation

o f perform ance o f

the

is

three-fo Id:

procedure

current

graduate

for

use

in

student

determ ining

population

in

the

level

a given

departm ent.
C2) T o d e v e l o p
sion

and t o

estim ating
(3)

substantiate
level

than

Need

for

the

if

the

were

Using th e

estim ation

m inutes

in

process

cost

m illion

in

adm ission

adm ission

developed
process

1969-1970 academic y e a r ,
that

over

processed

given

and assum ing

the

the

in

data

item

1.

developed

is

set

deci­
for

useful

in

o r program.

7 m illion

by A m e r i c a n

1969-1970 Michigan S ta t e

a base,

that

that

review ing
the

the

each

faculty

time

alone.

alo n e such

educational

research

applications

It

to

are

o f Graduate

fo r graduate

universities.

average

spend

reach

at

faculty salary
this

university,

an a v era g e

a decision,

would ap p ear th a t

population

deserves

than

is

of ten

the

o f h ig h e r education

an e n t e r p r i s e

3

and

studied

member w i l l

an a p p l i c a t i o n

Council

U niversity

departm ents

faculty

the

colleges

American i n s t i t u t i o n s

o f human r e s o u r c e s
from t h e

the

as

of

of

Study

estim ated

programs

value

one k in d o f s i t u a t i o n

During
Schools

the

representation

o f performance

To d e t e r m i n e

more

as

a quantitative

in th e

adm ission

$33,6
utilization

more a t t e n t i o n

currently

evident.

4
Of m o r e
problem o f
g a to r's

determ ining

survey

appear

to

titude

test

tion

and

At t h i s
teria

im portance

use

the

it

term s

being
have

is

the
been

the

amount

the

results

grade
the

grade

an e x t r e m e l y
situation,
ble
an

to

view

m eeting

what
each

The
the

departm ents

level

set

is

of

grade-point

the

represent

also

on

to

arrive

a continuous

studies

to

lim ited

look

at

each

of

have

been

utilized.

to

as

cum ulative

scale,

in

usually

the

the

gradc-point
as

a measure

requirem ents.

represents
sets.

the

has

By u s i n g
stu d e n t's

for

this
past

tasks

m eeting o f

to

there

determ ine

in

average,
that

which

It

have

is

ability

perform ance
perform ed

requirem ent

sets,

possi­

to

assum ption,

be

4

educational

contribute?
stu d e n t's

cri­

meet
the

in
at

the

the

some r e l e v a n c e .

graduate

average

o f the

As o n e o f t h e

successful

the

average

W hile

personnel

number o f

these

4 w ith

G iven

context

adm ission

total

to

gradc-point

program.

and

graduate

0

to

of quality.

gradc-point

places.

an a t t e m p t

a specific

the

recommenda­

a brief

a knowledge o f th e

of

of

ap­

decision

requires
that

average,

this

m e a s u r e d by

to

investi­

at

two d e c i m a l

conducted

achievem ent

letters

a n d ho w t h e y

referred

out

gradc-point

candidate,

take

carried

the

the

of

of

criteria:

feasible

knowledge

of

of

to

grade-point

faculty

kinds

interview s,

they

grade

average

undergraduate

in

can

requirem ent

graduate

graduate

usually

lim ited

unspecified

grade-point

m ajority

Average,

earned

the

the

academ ic

was

is

literatu re,

statem ents

interpretation

tim e
the

lim it,

are

faculty
Based on

a placem ent

of

of

applicant.

follow ing

significan t

use

the

o f what

upper

efficient

of

seems

G rade-Point

the

quality

some c a s e s

point

average,

the

scores,

in

in

of

than

programs

the

last

appear

two y e a r s

to
of

be

prim arily

interested

undergraduate

work.

In

doing

it

is

his

this

during

work

point
but

the
this

in

is

an

Follow ing

this

in

those

he

applying

where

portion
ogy

at

the

the

has

choice;

taken

graduate

as

to

M ichigan

pool.

course

(For

U niversity

graduate

majors

sociology.

allied
This

behavioral

being

previous
in

the

work

reselling

adm ission
from th e

the

top

applicant
if

the

the

the

two y e a r s ,

subject

w ill

The

in

the

less

these

sots

field.

is

appear

the

area

gradc-

f o r which

academic

work

in

an

a small

Department

applicants

to

the

were under­

four percent
in

pro­

of S ociol­

ISO a p p l i c a t i o n s

applicants

average

were

in

sociology.)

on t h e

basis

considerable

the

norm ally

departure

(4.00),

test

the

of

the

decision.

in

tho

m ajor

a p p lic a n t's

admit

probability
aptitude

in

of

im portance

in

the

chance

this

field
is

the

a p p lic a n t's

A review

departm ents

<3.30

o f academic

function

greater

is

The i n t e r p r e t a t i o n

grade-

would

represent

the

should be o f

a favorable

one o f the

grade-point

area

it

the

s tr o n g minors

of

the

of

decision.

continuum

receive

w ith

in

work

rem aining

comparison

grade-point

of the

pools

the

fields

adm ission

decision?

application

that

case,
in

How d o e s

science

The

of

of

graduate

processed over

N inety-six percent

the

traditional

1971-1072

program.

m ajority

gradc-point

the

that

requirem ent

step,

who p u r s u e

graduate

in

next

W ithin

had no p r e v i o u s

State

meet

an u n d e r g r a d u a t e

number o f s tu d e n t s

applicant

the

the

content

cum ulative

school.

assum ption

Therefore,

to

relevant

logically
the

m ajor.

ability

of the

than

on t h e

performed

the

only of h is

courses
to

student

mastery

they have

of

the

of his

im portance

disciplines,

program o p e r a te s

assum ption

point

area

areas

of his

o f more

is

period

a measure not

hopefully

that

graduate

of

scores

<3.00

is

his

four years

approxim ately

test

grade-point

that

study,

and

graduate

of

indicates
fo r the
2/25.

no e a s i e r

last

f>
than

the

interpretation

a numeric

assessment

one-tim e-only
design
the

of

test

point

this

study

scores,

It

to

supplem ented

or

has

test

by

very

small

Test

scores

decisions
G raduate

the

usually

not

adm ission

with

test
of

are

the

considered

discussion
According

in

w ill
to

focus

Educational

designed

to

candi d a t e s :

assist

in

graduate

to

his
is

regarding

a very high
the

producing

a very

purposes
t h e y bo

on

large
is

or

not

adm isssion

Exam ination

A ptitude

test;

for

As n o n e o f

Graduate

and

relevant

the

M iller

Record

Service,

A ptitude

school

the

the

subject

departm ental
Analogies

test,

Exam inations.

the

valid:

ignored.

graduate

Testing
the

adm ission

being based

that

grade-

in

employed th e

on t h e

Record E x am in atio n ,

time

to

authors

of

and Advanced t e s t s
fellow ship

the

that

prior

companies

alone n o r should

test.

study

G raduate

those

Advanced t e s t

A nalogies
this

decision

for decision

o f the

tost

ma y l e a d

would i n d i c a t e

Record

Record E xam ination

and t h e

c o m m i t t e e members

This

a

between

Wh a t k n o w l e d g e h e h a s

other

scores

u s e d most

tests,

the

is

preparing

The a v e r a g e

fa m ilia r w ith

adm ission

used

to

relationships

aptitude

lim ited.

scores.

tests

Graduate

M iller

is

c o n d u c t e d by

n o t be

tests
the

on

m arginal

score

b a s e d on

re se a rc h e r' s experience

w ith

the

ability

reviewed p r i o r

com m ittee.

conversations

should

report

this

The t e s t

academic

studies

tests

dependence

and th e

units

been

and a p titu d e

The t h r e e

to

average.

achievement o r

The r e s e a r c h

achievem ent

The

these

low p r o p o r t i o n

score.

area

the

experiences

very

stu d e n t's

tended

knowledge o f

appointm ent

the

usually

c o m m it te e member i s

their

of

grade-point

perform ance.

average.

faculty

of

the
are

selection

of

7
The A p t i t u d c T e s t i s a t h r e e - h o u r t e s t o f
general s c h o la s tic a b i l i t y at the graduate
level.
I t measures th e b a s ic verb al and m ath­
em atical a b i l i t i e s th a t a student has acquired
o v e r many y e a r s . . . .
Included in the t e s t a re
v e r b a l r e a s o n i n g q u e s t i o n s and r e a d i n g co m p re ­
h e n s i o n q u e s t i o n s drawn from s e v e r a l f i e l d s ; and
quantitative-m athem atical questions th a t req u ire
a rith m e tic reasoning, the so lu tio n o f alg e b ra ic
problem s and th e i n t e r p r e t a t i o n o f g r a p h s , d i a ­
grams and d e s c r i p t i v e d a t a .
The A dvanced T e s t . . . m e a s u r e s m a s t e r y and c o m p r e ­
hension o f m a te ria ls b a s ic to g raduate study in
m ajor f i e l d s .
S u b je c t m a t t e r s p e c i a l i s t s from
c o l l e g e s arid u n i v e r s i t i e s , a p p o i n t e d i n c o n s u l t a ­
tio n w ith the various p ro fessio n al a s s o c ia tio n s ,
p la n and d e v e lo p each t e s t w i t h . . . a s s i s t a n c e from
t h e !:TS t e s t d e v e l o p m e n t s t a f f .
In d e v e l o p i n g
e a c h t o s t , an a t t e m p t i s made t o i n c l u d e q u e s t i o n s
t h a t r e q u i r e r e a s o n i n g w ith and a p p l i c a t i o n o f t h e
b a s i c c o n c e p ts and p r i n c i p l e s o f th e s u b j e c t . (1 4 :9 )
One o f t h e
to

the

problems

sentence

test

corporated

the

has

into

tests

is

developed

interpreting

d e a lin g w ith

ho w m u c h o f t h e

these

in

score
test

prcvnlent
in

abilities

represents

ta k e r's

persons

K xam ination.

Any s t u d e n t w i t h

program which

ma y r e s u l t

students

may r e a c t

exam ination
real

adversely

resulting

in

acquired
learned

the

for passing

perseverance

a high
to

a test

test

can

score.

which

a thriving
the

is

As t h e

establish

in­

use

of

business

G raduate

Record

a preparation

C onversely,

the on e-tim e-o n ly n a tu re
score

relates

o v e r many y e a r s - -

ability

nation,

Test

good

of

which u n d e r e s t i m a t e s

the
their

ability.
TTte s t a t e m e n t

score
the

in

A ptitude

academic b e h a v io r?

throughout

preparing

the

provides

subject

to r w ith

an

area.

these

a re a knowledge.

above I n d ic a te s
indication

of

the

The p r o b le m o f

scores,
In t h e

resulting
opinion

that

the

Advanced T est

a p p lic a n t's

"cramming” i s

knowledge o f
also

in o v e r e s tim a tio n
of the

of

investigator,

a fac­
the
the

subject
lack

of

any

8
significant
the
to

attem pt

sim ilarity
graduate

o f the

test

the

report

i n c l u d e d on

in

adm ission

function

the

of the

time.

a careful

fall

the

The n e x t

test

fiftieth

or,

criterion

from each

the

which

candidate.

in

on t h e
require

If the

G raduate

and

the

thought

value

In

plan

exam o v e r

Fxam ination

w ill

over

the

a p p lic a n t's

sm aller

percentile

being

reviewed

for

a retake

their

the

of

form.

pattern

W ithout

epistles

applicant
are

some

a method

developed

from

program,

then

the

such

scores

autobiography

developed

in

most

percentile.

the

There

extensive

a statem ent

these

sixtieth

statem ent

also

if

a

usually

average;

the

a

score

studied,

a fairly

o f wading th ro u g h

the

is

departm ents

application

device.

score

the

some c a s e s ,
is

Advanced T e st

taken

application

o f c a n d i d a t e s who w e r e s u c c e s s f u l

dure

the

7f>0,000 s c o r e s

the

of his

program has

form

useful

basic

process?

latter

scores

statem ents

a highly

lim it

and

Record

from

key p h rases

ma y b e

determ ine

raw p e r f o r m a n c e

of pulling

this

to

Ibis

grade-point

manner.
w ill

Test

who h a v e

derived

on r e c o r d

which he o r she p r o v id e s
g ra d u a te programs

score.

99th p e r c e n t i l e

the

docs

adm ission

These p e r c e n til e

and th o u g h t f u l

who t a k e

the

A ptitude

the

the p ro b a b ility

applicants

in

persons

is

th e same manner as
the

to

" . , . m aterials

test

in d iv id u a l's

For

percentile

greater

below

used

F.xnmi n a t i o n

(14,17).

range between

score,
in

the

period

in

of

the

Advanced

and a p e r c e n t i l e

period

to

departm ents

decision.

performance of a l l

A ptitude T e st,

the

Record

representation

year

the

exam inations

a raw s c o r e

a specified

three

the

these

b a s e d on t h e

o f graduate

t h e i r program c o n te n t

Graduate

norm ative

part

study"

Mow a r e
Both

of

on th e

a system atic
is

proce­

indeed one o f

considerable

onus.

Hie m a jo r ity
those

accom plishm ents

e x p e r ie n c e which
cently

relates

tional

sim ilar

to

field

statem ent

which

o f such

statem ents

reader.

the

statem ent,

the

three

must be

sion-m aking
the

letter

in

be

function

what

I heard

place

of

is

even

the

list

academic h o n o rs ,

is

previous

publications

then

and r e ­

follow ed w ith

"Make a s t a t e m e n t

a

of your educa­
.I n c l u d e

fields

of

interest,

if

it

with

m eetings,
to

both

of

of

any,

w ithin

a good d e a l o f

gamut o f

phone

to

that

o f the

to

do

The

the

m erit

in

this

to

im portance

It
in

situation

is

been

ment w here

fam iliarity
the

w ith.

investigator

is

p r o c e e d e d by

other s t a t e ­

error.
process

in what
of the

of d e c i­
'flic

it

author.

em ployed,

is

it

also

of

it?

difficult

a document w r i t t e n

experience

or

seems to

Who w r o t e

extrem ely

B a s e d on t h e

im portant'

says

letter

som eone y o u h a v e n e v e r met b y someone y ou h a v e n e v e r met
you h av e no

of

o f w eight on

statem ent.

be not

the

the

im portance

and elev en

applicant

knowledge o f

or

the

interpretation

I know t h e m ?

faith

and

which h a s

calls

appears

say.

The

a good deal

recom mendation

re a d e r's

o f them o r

the

reader

a statem ent

nine

recommendation

the

the

considerable

letters

neglects

of

run

r e v i e w e r may p l a c e

sim ilar
of

statem ents

l a n g u a g e may b e e m p l o y e d .

evaluation

subject

The p l a c e

to

to

i n

such

varies

While

two c o u r s e s ,
ments

expect,

the w ritte n

the

Have

candidate

training,

listing

follow ing:

the

o f _________________ ______________________________. ’'

in

a

the

This

ask
i.e .,

academic

your special

As o n e w o u l d

of

to

and c a r e e r o b je c t i v e s

this

ways

usually

deemed w o r t h y ,

read p u b lic a tio n s.

question

in

o f programs

at

about
someplace

the d e p a rt­

extrem ely

risky

10
to

pi ace

never

a Rood d e a l

faith

in

a letter

met by s o m e o n e y o u h a v e h e a r d

whose p r o f e s s i o n a l
n e v e r met!
last

of

and

In e s s e n c e ,

serves

have been

opinion

to

on

the

confirm

developed

in

the

the

about

a b o u t - -w hose work

field

letter

w ritten

of

someone you

you have

you r e s p e c t - - a n d

rccom nendation

cither

positive

review

of

the

is

or negative

read,

whom y o u h a v e
usually

read

feelings

four previously

which

m entioned

cri tc ri a .
Although none o f
the

applicant

m erit

if

that

the

in

to

differing
not

meaningful

have

interview er

the

be

used

Tins
the

candidate.
to

allow

method

test

for

of

this

it

appears

different
first

from t h e

im pressions

review ing

candidate,

the

case

serve

as

as w e ll

interview

basic
one

gather data

providing

to

to

confirm

m aterial.

of

the

to

the

all

of

of

the

of

interview
to

knowledge

o f questions

should

device,

independent

a practical
lim ited

interview s

who a p p e a r s

of

experience

from h i s

already

also

reliability

colleagues
is

o f reconm endation.

A candidate

m aterial

n decision.

feelings

the

would have

no p r i o r

gathered

has

require

for

applicant

letters

is

method would be

From t h e

evidence

function

ascribed

serve

as

technioue

responsibility

that

w ith

a screening

points

the

a prepared set

interpretation.

and h e a rs a y

is

An a l t e r n a t e

in terv iew er to

the w ritte n

elnborate

to

this

interview er(s)

insure

data

the

w ill

feel

make u s e o f

f a m i li a r w ith

the

conduct

investigator

the

study

On e m e t h o d

It

decision,

that

manner.

interpretations.

than

adm ission

that

this

docs

com pletely

inform ation.

the

in

investigator

addition,

In t h i s

could

adm ission

In

prepared

provides

the

the

included

a system atic

m aterial.

interview er,

of

units

i n t e r v i e w e r ( s ) must be

of questions

subject
the

interview ,

conducted

a p p lic a n t's
a set

the

no
V isual

developed by
weak w i l l

r e c e i v e mu c h d i f f e r e n t
With
sion

few e x c e p t i o n s ,

decision

process

of

and

determ ining

programs.

a rather

pessim istic

feels

not

appears
the

attem pt

opinion

com pletely

review

difficulty

stems

u t r a v e l l e r w ith

an

from t h e

that

is.

where

that

is

he i s

to where he w ants

are

poses
if

the

forced

to

study

adm ission

In t h e

to

go.

related
this

continue
is

to

process

represent

to

best

that

cannot

The l i t t l e
led

o f m onotonic

academic

This

on

at

is

really

that

no i d e a

that

he

are

ability.

but

and
he

he d o es

symbols

in

the

direction

in

adm ission

regarding
to

is

at

continuous

still
the

least

numbers

w hi1

pur­

and th e n

see

intended.

as

one adm ission

W hile th e

kn ow

better

decisions,

view w ith

sure

from w here

standing

the

going

not

is

for

like

not

get

a key

docs

f o r which

is

On e o f

for

The

arc

however,
is

process

person

pace I

investigator

achievem ent

each

how t o

traveller,

to

reader

the

com mittees

go b u t

the

and p r e p a r e d

interpret

h ig h e r value

quality.

the

a quicker

employed

incident
the

to

adm is­

re a d e r w ith

which

o f his

adm ission

he h a s

provide

achievem ent data

e a r l i e r has

notion

positive

the

the

adm itted

quite

com mittee given

an i m p l i e d m o n o to n ic q u a l i t y - - a
low er v a lu e .

arc

the

adm ission

departm ents,

He k n o w s w h e r e he^ i s ,

Finally,

adm ission

of this

is

fact

left

to

strong.

constitute

for possible

In most

He k n o w s w h e r e h e w a n t s
either.

which

the whole p ro c e s s

a ma p o f s y m b o l s h e

where

they

quality

good p e o p l e

no key p ro v id e d .

the

of

who a p p e a r s

m a te ria l inputs

interpretation

application

is

than

the

justified.

that

there

off

arc

a candidate

i n v e s t i g a t o r ma y h a v e

The i n v e s t i g a t o r

to

than

candidate

The

t o work g iv e n

field.

these

th e means o f

graduate

is

treatm ent

there

i

good as

a

com mittee

skepticism
employed to

m easures,

what

proof

12
d o we h a v e
are

that

derived arc

the

situations

functionally

score

of

1460 a n d

B has

a GRF s c o r e

from w h ic h

identical?

3.4 7 g r a d e - p o i n t

these
If

candidate

average

from sch o o l

Y,

A more q u a l i f i e d

for

graduate

program C than

at

not

known.

specific
still
in

given

for

w ith

program?

two q u e s t i o n s

he

follow ing
decides

w ill

to

grow?

to

grow

A fte r doing
three

he

can

The a n s w e r

situation

probabilities

provides

for other

programs.

the

and in

on

how d o e s

the

he

and

of success

us w i t h

a qualified

We a r c n ow f a c e d

Insufficient

these

is

to

kinds
things,

of

has

data

a plot

of

for

a

if

or not

provided

conditions

they

check

tom atoes

he w i l l

to

w ith

have

succinctly

by t h e

tom ato

necessary
plots

grown

tom ato

tom atoes

of

for
land

tom atoes

"experts"

as

grow.

probably

be i n

possession

of

facts :
(1)

Ibe

conditions

(2)

Some o f t h e

may b e

neighbors

mixed r e s u l t s .

adequate
have

for yielding

grown to m a to e s

in

l a n d on w h i c h

t h o s e w h o own t h e

area
can

very

h e know w h e t h e r

ascertain

can ask

he

expressed

inform ation

general

Finally,
the

the

plants

Secondly,

study

individual

can read

fared.

opinion

I f an

tom atoes,

raised

his

how t h e y

their

the

a common r e s p o n s e :

annlogy.

growing.

surrounding
and,

on

B?

candidate

relationship

research

purpose o f th is

F irst,

company w hich
tom ato

and

program,

a monntonic

inform ation

and n no

candidate

is

answer.

The c e n t r a l
the

into

Previous

some p r o g r a m s

m eaningful

F o r any g i v e n

he t r a n s l a t e d

provide m eaningful

the

yes

criteria

a GRF

X and c a n d i d a t e

3.17

is

A has

from school

1290 and

point

quantities

average

of

this

num erical

good tom atoes.

a n d met w ith

13
(3)

The

"experts"

think

,rb c e f s t c a k "

may d o b e t t e r

than

other

brands.
The c o n c l u s i o n
does
is

h e know t h a t

only

logical

but

do n o t

carry

it

admit

ability,

data

admit

everyone.
which

the

tised

student

resenrch

their

attem pt

differ

what

far!

If

done
on

to

to

him?

provided

to

in

tells

criteria

is

that

' Hi e p r o b l e m

of

graduate

b a s e d o n how w e l l

w h ic h h e was
When t h e

there

are

false

negative

in

the
the

be

student

three

is

the

best

types

process.

have

its

are

Pre­
in

W h e r e we

already

"growing

one o f q u a l i t y

performed

Ihe

o n how

fared

situation.

is

of

of infor­

clim ate.

an e v a l u a t i o n

has

successful

inform ation

programs

grow,

of success.

u s h ow o t h e r s

programs

then

to

sim ilar

their

start,

no program

adm ission

us witli

graduate

needed
the

in

a somewhat

analogy

is

w ill

probability

two o f

There

tom atoes

decide,

provides

What

a student

if

to

analogy,

these

know

needs

the

not.

from t h e

im probable;

highest

set

obvious

to

But

and sec w hat h ap p en s.

is

the

data

He d o c s

latter

process

this
use

appears

know i f

lias

date

prow?

you want

The

t o grow to m a to e s .

tom ato p l a n t s

analogy

currently

the

grows b e s t ?

decision
to

in

from th e

tom atoes."

to

are

has

vious

this

able

actually

bach program

student

outlined

set

w ill

surc--plnnt

too

R e fe rrin g back
m ation

h e may b e

I f you want

your program,

can

be

outcome o f

y o u grow them.
in

that

tom atoes

o n e way t o

The

is

of

in

the

the

c o n t r o l -adm ission

program

adm itted.

decision

two t y p e s

made t o

o f e r r o r which

decision

had he o r she been

is

error;

adm itted.

the

admit o r

reject

can o ccu r.
applicant

The s e c o n d

an a p p l i c a n t ,

The

first

type

is

the

wh o w o u l d h a v e s u c c e e d e d

error

type

is

the

false

positive

14
decision;
in

the

the

the

applicant

program.

adm ission

that

The

were n o t

adm itted

Repeating

(2)

of

level
(3)

performance

of

can he

of

performance

To d e t e r m i n e

if

more t h a n o n e k i n d o f

false

this

the

positive

for

study,

those

the

successful

evaluation
error

of

given

applicants

graduate

of the

or

item

1

process

in

who

of

proposes:

determ ining

student

adm ission

in

estim ation

investigator

for use

representation

situation

not

under review.

developed

the

involves

collected

current

value

as

of the

procedure

a quantitative

a n d wh o i s

study

programs

the

the

adm itted

this

term s

purpose

substantiate
of

the

of

an e v a l u a t i o n

To d e v e l o p

and t o

in

data

to

the

( 1 ) To d e v e l o p
level

focus

decision

no perform ance

who was

the

the

population.
adm ission

data

set

for

decision

estim ating

.
doveloped

is

useful

in

program.

O v c r v i cw

In
their

im portance

follow ed
was

the n e x t

taken.

findings

or,

for

study w ill

be

considered.

the

design

of

the

the

problem s

The

for each

ITi e l a s t
data

to

this

a n d some o f

interpretations

the

various

by a c h a p t e r o n

selected

turns

chapter

of

rem aining
unit
the

chapter w ill

analysis

chapters

selected
data

types

w ill

in

involve

this

area

This w ill

study:

cncountored

and t h e

and
be

How w h a t w a s
as

w ell

an

analysis

im plications

as

as w ell

used

wrong
of
as

the
other

uncovered.
deal

w ith

and a d i s c u s s i o n

that m atter,

of studies

w hether o r not

policy

o f what
a next

recom m endations b a s e d on
the next
step

is

steps

should

be,

even w arran ted .

CHAPTER I I

REVIEW OE PREVI OUS STUDI ES

In a l l

areas

of

before.

In

graduate

performance

the

areas

have
pages

order to

of

to be

the

format

has

the

same p r e d i c t o r

the

sole

based

studies

criterion.

criterion.

The f i n a l

varies.

of t h e i r

on p r e v i o u s

to

work

choice

rely

The

group

a

in

findings.

15

on t h e

the

ascertain
which would
the

follow ing

th a t have been

conducted

the p a st
in

types

twenty

this
of

years.

period

with

criterion

sets

placed

into

three

major

criteria.

The

first

group

graduate

faculty

in

of

grnde-point

w hile

rating

of

average

still

using

the

subject

the
as

a

are th e i n n o v a t o r s and t h e c h o i c e o f

W ithin each

appearance

over

second group,

employs

discussed

same

of

to

problem area s

be

are

who h a v e g o n e

a review

undertaken

unchanged

and t h e

those

study,

research

reviewed

on t h e

which

average,

order

of

graduate

studies

grade-point

criteria

studies

variations

The

classifications

as

in

always

rcoccurring

remained e s s e n t i a l l y

employed.

of

work was

m ajor a re a s

success

The

consists

and th e
The

are

the p r e s e n t

prediction

d ealt w ith.

represent

there

maximize

strength

on p r e d i c t i n g

being

research,

group,
the

the

studies

literature

are

reviewed

in

a s many o f t h e m b u i l d

16

G rade-Point
Borg
ion,

(17),

collected

Record
tion

Average

using
the

Exam ination

of

as

.36 and

Performance
grade-point

Verbal
for

C riterion
average*

and Q u a n t i t a t i v e

175 g r a d u a t e

.37 w ith

the

Verbal

Borg n o t e s

value

seriously

graduate

range o f

the

grading

Chase,

c t . a 1.

M astery T e s t,
portion
tion

of

(either A or

English

D ifferential

method w ith
and

personal
last

that

about
of test

Test,

on

attendance

race,

for

by

sex,

criter­

Graduate
C orrela­

A bility

scores

discrim ination

given

all

the

restricted

cases).

scores:

and th e

A ptitude T est;

data

from t h e

education.

the

reduced

B in

(8 ) combined a s e t

C ooperative

the

institution

GPA i s

in

performance

and Q u a n t i t a t i v e

were o b ta in e d .

the

the

scores

students

respectively
of

as

Concept

Numerical

the

M ultiple

experience,

1013 s u b j e c t s

A bility

to

C orrela­

previous

predict

graduate

GPA.
The i n t e r a c t i o n
scores

were n o t

m ultiple
is

Subgroup
N a tiv e White
N a tiv e Negro
Men
Women
Total

significant.

correlation

presented

effects

inputs

o f the

personal

d a ta w ith

A breakdown o f th e
fo r each

subgroup

all

test

most s i g n i f i c a n t

and t h e

total

group

below.
S ignificant

Predictors

jl

C ooperative English
C ooperative English
Concept M astery
C ooperative English

reading
reading

Gex p re ssio n
Gv o c a b u l a r y

Reading

GD i f f e r e n t i a l

A ptitude

.45
.52
.30
.64

C ooperative English

Reading

GD i f f e r e n t i a l

A ptitude

.48

* For the re s t o f th is c h a p te r the phrase "g rad e-p o in t average" w ill
b e a b b r e v i a t e d a s GPA.
T h i s i s a commonly a c c e p t e d a b b r e v i a t i o n
f o r th e e x p re s s io n o f th e term .
Furtherm ore, i t is both e a s i e r to
w r i t e and e a s i e r to read .

17
The a u t h o r s

found

ma y c o n t r i b u t e
their

to

finding

result

of

efficiency
lected

the

departm ents.

felt
the

the

G raduate

graduate

grade-point

do t h a t

the

average

at both
the

they

do b e s t .

Stordahl

(43)

in

A nalogies
The n e t
graduate

Test

result

at

(16)

the

last

levels;
people

over

GPA a l o n e .

across

Ihe

authors

analysis.
it

Given

would seem

than

the

with

statistical
b a c h e l o r and

predictors
in

of

the

GPA a n d t h e

GPA

R = .67.

This

usually

continue

enrolled

doc­

study
to

in M a s te r's

com bination o f M ille r

two y e a r s
.04

569 s u b j e c t s .
.69

o f com pletion

correlation

an im provem ent o f

in

the best

o f undergraduate

adage th a t

used a m u ltip le

was

rather

col­

and Q u a n t i t a t i v e

scores,

a s t u d y o f 534 s t u d e n t s

Scores w ith

for

regression

point

previous

GPA f r o m u n d e r g r a d u a t e

Alexakos

the

a com bination

which

degree programs,

found t h a t

Tests

from 0 t o

Verbal

design

a

the p r e d ic tiv e

A ptitude

a s a m p l e o f 90 s t u d e n t s

education

confirm

w ith

respectively.

study

in

courses

,18

in

degrees

would a p p e a r to

and

the

test

m aster's

Education

.19

for

aptitude

(2 2 ) u s in g

deal

GPA r a n g e d

o f the

Herbert

ap p e a r to be

av era g es by d e p artm en ts
to

which

range.

Exam ination

of

method.

in

to

fu tility

lies

program were

do not

the

problem

toral

Record

functions

Of more i m p o r t a n c e was

an a t t e m p t

were

indicates

test

grade-point

correlation

subjects

study

the

in

scores

M ultiple

interrelationship

that

graduate

(27)

of test

all

in

low c o r r e l a t i o n s

grade-point

The c o r r e l a t i o n

for

the

and Walsh
of

overlap

low c o r r e l a t i o n s .

restricted

Graduate

scores

the

that

the

Madaus

considerable

undergraduate
the

prediction

(M ultiple

a s t u d y o f 92 M a s t e r ' s

GPA.

R «* . 4 5 )

candidates

using

of

18
undergraduate

GPA a n d

tho

as

predictors

found a

relationship

it

may b e

significant

could be

done,

formance

in

Newman
found

Riven

terms
(31)

a m ultiple

Exam ination

deviation

GPA.

for

However,

a m ultiple

not worth

ful

a sole

avenue o f

highly

criterion
study.
in

performance p ic tu re

of

range

not

the

research

far

from r e s o lv e d .

G rade-Point
In
students

Average
Webb

enrolled

C ooperative

of

plus

p <'.05.

a waste o f

to

predict

tests

Frankly,

tim e.

Wh a t

graduate

the

score

the

Advanced

reports

of

students

.18 between

that

the

range

indicate

in

per­

psychology
Record

Psychology with

size

the

for the

in

Graduate

test

indicates

.18 would

of

graduate

that

each

and

the

to

of

the

standard

correlation

Rrade-point
that

is

un-

criterion.

Newman's

from

concern

is

a small

Furtherm ore,

effect

the

Faculty

is

not

indicate

the

and

part

parts
w hile

actual

Ratings

1 94 7 - 1 9 4 9 :
score,

grade-point

success

summation o f t h e s e

collected

test

graduate

o f p e rfo rm a n c e by t h e

review ed s t i l l

and/or

the

school

grade

picture.

(49)

English

that

measure

appear

results

1951,

t o be

graduate

This

any t o t a l

does

of 66

can be s t a t e d

restrictive

of providing

.213 w ith

A ptitude

effort.

In summary i t
age as

test

restriction

the

Exam ination

GPA?

a study

R of

of

relationship,

predicted

by

Record

appears

The a u t h o r

the

the

it

correlation

effcctcd

is

the

A ptitude

tho grad u ate

but
this

of
in

Graduate

student

falls

as

total

far short

technique,

problem

is

Performance C r i t e r i a

follow ing

C ooperative

the

is

restriction

measurement
that

a m eaning­

o f the

the

aver­

data

on 492 g r a d u a t e

General

undergraduate

GPA.

Test

score,

Tl i e c r i t e r i a

19
em ployed wore
ment.

R esults

ratings

than

correlation
a better
than

graduate

GPA a n d

indicated

between
was

indicator

a higher

test

very

scores

strong,

of

faculty

the

correlation
and

stu d e n t's

between

graduate

Webb d o e s

feel

of overall

GPA.

that

perform ance

achieve­

test

scores

and

W hile n e i t h e r
the

in

ratings

graduate

are

work

grades.
King

and

llcsco

(25)

the

G raduate

Record

graduate

GPA a n d p a i r e d • c o m p a r i s o n

form ance.

tween

of research

Exam ination

Hie c o r r e l a t i o n

K en d all's
across

success

conducted

predict

Tnu a s

this

departm ents.
the

ratings
The

rating;;

A ptitude

and

latter

GPA.

ness

of

the

were

in

engineering

em ployed
Record

for

placem ent

in

1/3,

faculty

scores

and n a t u r a l
(36)

predictors,

all

A ptitude

success

was

com parison
lower

M ultiple

1/3,

and

the

o

to

of

portion

The

and

rather

given

that

science

either
or

faculty
others

of

were

overall

per­

was

o f perform ance

(p^.05)

was

combined

were

the

011

119 s u b j e c t s

not

found b e ­

ranking

of

significant.

dram atic

negative

the

of

bulk

of

the

48 g r a d u a t e

com binations

of

U ndergraduate

rating

in

a

scores

to

skew­

subjects

departm ents.

a study

Scores

for

U niversity

criteria

ratings

1.98

results

in

their

an e q u a l i t y

ratio

the

Purdue

of

the

th e program ;

students

G raduate

GPA.

The

stu d e n t's

current

i.e .,

10

top

^>,

etc.

correlation

ratings

to

and N cilso n

Exam ination

criterion

m iddle

due

quantitative

seven

assume

Verbal

ma y b e

Test.

method em ployed

A critical

at

from

faculty

The Q u a n t i t a t i v e

result

Robertson

fellow s

A ptitude

did n o t

test

a study

produced

com bination

an

R of

.44

(p < . 0 S )

o f mean s c o r e

on

the

between
G raduate

20
Record

hxam ination

Science

courses.

lations

but

use

as

the

dents

w e r e made
ties:

"degree

field;

knowledge

niques

in

tual

breadth;

constituted
the

As
pooled

all
into

coses.

of

m astery
of,

and

overall

GPA,

various

the

study

by

not

and

Robertson

1955

1958.

on an

-point

and
at

that

11

to

use,

express

corre­

w arrant

of

on t h e
and

their

ranged

various
for

w riting;

intellec­

oneself

in

the

level

all

Record

H all

(35)

subjects,

the

a

from

were

range
.08

on

in

any o f

the

predic­
GPA f o r

W hile

d ifj

all

com pari­

the

predictor

accepted

percentage

GPA.

im prove
23

95R

.41.

compared.

on

dnta were

333 t o

and g r a d u a t e

or graduate

data

of

to

levels

the

i n an a t t e m p t t o
collected

Fxnm ination
Test.

w ith

GPA

included

A nalogies

c u t - o f f on

faculty ratings

general

G raduate

variables

were o b ta in e d

mean

quali­

tech­

basic

rankings

groups

five

research

on d e c r e a s i n g

and N e ilso n

of

ratings

the

cut-off

the

.0 1

The

in

M iller
for

958 s t u ­

knowledge

G raduate

the

r a is in g the

im pact

scale

The p r e d i c t o r

rejected
the

and

for

o f g rad u ate w ork."

com binations

values

had g r e a t e r

increasing

between

obtained

effect

projected

Robertson

docs

(48)

available

sample

was n o t e d

in

to

scores

were not

sons,

than

and N atu ra l

significant

b y Tul l y

quality

Advanced T e st

significant

variables

produced

relationships

fundam ental

criterion.

ferences
it

of

ability

mean c r i t e r i o n

different

also

obtained

and a b i l i t y

field;

data

assess

also

advisors

'Ihe c o r r e l a t i o n s

To
tors,

were

m ajor

another

Test,

those

g raduate wort

undergraduate

A ptitude

of

GPA i n M a t h e m a t i c s

tools.

to

by t h e

the

com binations

strength

ratings

adm itted

undergraduate

O ther

prediction

Faculty

plus

on

a

recent

previous
graduate

21
psychology
both

students

studies

was

An a d d i t i o n a l
exam ination

a

score

score

Record

and

(N = 4 2 )

the
w ith

com prehensive
ratings

success

for

Koadcn

collected

rating

was

to

of

thesis

consisting
Test;

W hile
the

research.
graduate

of

criterion

receive

was

the

the

in

Ph.D.

com prehensive

and a

com bination

score,

M iller

th eir

of

correlation
scores

was

assessm ent

degree.

graduate

an

.54

Test
.32

(N = 1 9 )

between

w ith

faculty

.73.

advisor

Tho p u r p o s e

GPA w h i c h

mean

r of

o f M aster's

from the m a jo r

their

the

A nalogies

GPA p r o d u c e d

correlation
The

of

provided

candidate

for

171

of the

no

accounting

of

test

scores

correlations

were

undergraduate
the

Ohio

and th e

S tate

undergraduate

produced.

GPA;

the

The b e s t

W atson-G later

Psychological

Test;

GPA,

a

predictor
C ritical

produced

a

.40.
study

is

do p r e s e n t

"Careful
program o f

applicants
In

of
and

this

authors

in

com pleted

a variety

correlation

to

study

undergraduate

ratings

the

Ihc

perform ance.

o f m ultiple

Thinking

A ptitude

exam ination

faculty

SO.

capacity

a w eighted

scores.

(30)

augment

on

follow -up

on

ratings

exam ination

original

subjects.

junior-senior

and

Using

the

Exam ination

who h a d

set,

in

some

fncul ty

the

rating

index b ased

candidates

series

faculty

and com prehensive

Owens

the

augment

criterion

A selection
G raduate

to

not

an

mu c h

im portant

consideration
study

different

of

insight
the

than
for

specific

is

essential

before

(29)

developed

a factor

its
the

predecessors,
direction

of

future

requirem ents

of

a

a w ise

selection

of

can be m ade."

1969,

M ehrabian

analysis

o f the

13

22
variables

used by t h e

departm ent.
year,

six

variance

With

factors
in

1.

the

Test

adm ission

a sample o f

committee

266 c a n d i d a t e s

were o u t l i n e d

decision

Score

to

for

which

accept

Percentiles

or reject.

- G raduate

M iller
Grade

I’o i n t

3.

Research

Average

-

O rientation

UCLA p s y c h o l o g y
the

for

1968-1969
.75 o f

These

Record

the

factors

arc:

Exam ination

and Advanced T e s t

A nalogies

Sophomore
-

for

accounted

Apti t u d o

2.

the

and

Test

and S e n i o r y e a r s

Previous

experience

and

letters

of

r e c o m m e n d a t i on
4.

Grade

Point

Improvement

-

Increase
first

5.
6

in

last

two y e a r s

over

two y e a r s

Sex

.

M athem atical

Training

-

Number o f m a t h e m a t i c s

or

logic

courses
The
test

adm ission

score

performance

in

candidates.

tative

use

of

l o a d e d e x t r e m e l y h e a v y on t h e

score,

the

and

last

of current

increased

faculty

Record

factor

GPA a n d

evaluation

that

two y e a r s

does

grade-point

school

for

a w eighted

Exam ination Advanced T e s t ,

graduate

scales

criterion

were c o l l e c t e d

indicated

sophistication

rating

analysis,

a faculty

potential

correlations

G raduate

predictor

With

of research

M ultiple

A ptitude

the

of graduate

term s

o f the

the b est

the

th e worth o f

consisting

bination

decision

factor.

To d e t e r m i n e
measures

committee

of
79
c o m­

Q uanti­

a v e r a g e was

performance.

o f th e methodology
appear to

produce

employed,

a stronger

23
r e l a t i o n s h i p between measures
present

level

problem s.
point

o f performance.

The m o s t p e r v a s i v e

average

rating

of past

on t h e

a student

rating

he h a s ,

The d e p e n d e n c e o f

these

There

arc,

problem

is

in

all

Another c r i t i c a l

problem

terms

the

time

In o r d e r

to

rater
develop

useful

results

solved

in

lines
this

of

the

are

in

variables

stantial

variations

under discu ssio n

lack

is

studies

o f adequate
In

O ther than

arc
of

in

the

the
done

the

replication

is

graduate
their

the

o f Performance

The p u r p o s e

of th is

in

to

grade-

member i s

student.

accurately
both

in

same s a m p l e s u b j e c t .

which w i l l
must

yield

first

some

be r e ­

past

and/or

choice

In most

twenty

The s t u d i e s

in

of predictor or

realm o r

kind

Faculty

two c o n t i n u i n g

prediction.

usual

its

Average

represent

choices.

they

cases

uncovered
years.

are

sub­

the study

in

the

re­

In some c a s e s

the

indeed u n fo rtu n a te .

1951 a m o n o g r a p h p r e p a r e d b y

The P r e d i c t i o n

the

manner.

only one o f
in

the

analysis

reviewed

of

usual

on

reliability

that

outside

that

reliability

G rade-Point

area of

of

faculty

graded

rater

data

and p l a u s i b l e

innovators

criteria

the

effective

the

influence
If a

su b ject's

s o me r e o c c u r r i n g

im possible

and r a te r s

two c l a s s i f i c a t i o n s

research

is

is

problem o f r a t e r

C riterion

first

section

view o f

an

a workable

Performance
Ratings
The

across

the

likelihood,

two m e a s u r e s

and t h e

however,

o f performance.

assess.
of

performance

Kelly

C linical

s t u d y was

and

Fiske

Psychology

to ev alu ate

(13)
first

the various

titled
appeared.
procedures

24
employed in

training

clinical

psychologists

f o r work

A dm inistration

installations

and t h e

predictive

procedures

later

in

the

for

success

both

in

V eterans

validity

training

of

program

these

and th e

profession.
The s t u d y was
involved

in

objective
tests;

test

was

perform

components:

of

in

had

O ther
1.

2.

clinicnl

did

of

tests;

tests,
tests.

o f an

socio-em otiona1
diagnostic

two i n - d e p t h

in ter­

The c r i t e r i o n

psychologists

clinicnl

R ater

of

arc

employed

expected

to

be elim in ated w ith

relation

acceptable

to

consist

clinical

skills.

of

three

functional

the

authors

separate

of diag­

these

measure,

the

criterion.

skills

While

objective

a common c o r e w h i c h

inter-

refer

to

com petence."

as b e i n g
role

the
no

seems

single

im portance were:

a small
w ith

satisfactory

accom plishm ents,

a g r e e m e n t was h i g h e s t

occupational

sig n ifican tly

no

function,

and s o c i a l

identify

results

Only

indicate

no c o m p l e t e l y

was e x p l a i n e d

ra te r's

study

terms

and t h e r n p y ;

"perceived

This

the

Intellectual

correlntions
as

consisted

i n t e r e s t s - - and c l i n i c a l

four p ro jec tiv e

which

techniques

198 t r a i n e e s

a VA i n s t a l l a t i o n .

criterion,

criteria

of

and

year period w ith

intelligence

of situation-series

functions

Results

nosis

The p r e d i c t i v e

of values

consisting

in

a five

b a tte r y - - containing

and a v a r i e t y

those

The

program.

and m e a s u r e s

procedures
views

the

conducted over

d u e a s much t o

as

to

proportion
criterion
loss

on t h e

the

ability

of the

the
of

function

most

component.
of

the

the ra te e .

objective

measures,

of predictive

intellectual

tests

of these

ability.

correlate
tests

could

25
3.

Standard psychological

predicting

future

performance

test

as

scores

the best

do

about

as w e ll

professional

at

staff

members

assessm ent.
4.

The p r e d i c t i o n

training

is

generally

of

relative

more a c c u r a t e

standing

in

than

prediction

the

tho group

com pleting

of

training

outcom es.
5.

The

and t h e

relationship

number o f v a r i a b l e s

ment b a se d

on

credentials

objective

test

profile

Webb

( S O)

conducted

grade-point
academic

and

ability

from g r a d u a t e
progress

the

research

of

Academic

the

W riting

and

and

and

the

Three

the p r e d ic tio n

m arkedly n o n lin e a r .

good

Genernl

of

of

as

recommendation)

all

predictors

departm ents
C ulture

using

test

C ooperative

criterion

to

measures

Achievement

courses);

Research

(m echanics,

plus

combined.
undergraduate

develop

English

ratings:

Expression

Assess­

test

were

an
as

developed

(GPA p l u s

(ratings

diction

o f work

and o r ­

papers).
the

than o f

research
ability

criterion

than

dictor of

the

criterion.

is

across

faculty

follow ing

achievem ent

criterion
of

a study

accuracy

letters

as

in n o n - r e s e a r c h

Webb d e v e l o p e d
Hj

almost

predictor.

courses);

ganizations

(grades,

predictor

grades

ratings

is

the

employed

C ooperative

achievem ent

a w riting

in

between

the

w ill

o f the

a better

research

criterion
w ill

be

hypotheses:

be

than

criterion
the

a better

achievem ent

achievem ent

predictor
and

criterion

criterion

than

of

achievem ent

a b e tte r predictor

expression

predictor

o f the

criterion.

o f the

research

and a b e t t e r
the

pre­

expression

Using
accepted
w arrant

a m ultiple

correlation

as no sam p le p r o v i d e d
rejection.

This

was

s t u d y was w e l l

constructed

appear

s o me

to have

In an a t t e m p t
instructions

to

factor

analysis

ficant

rating

real
to

in

a predictor,

the

authors

found

and d i s c r i m i n a t o r y

factor.

It
set,

their

and

of

the

insure

a buffer

discrim inatory,

for

the

In r e c e n t
school

utility

there

of graduate

of this

S tarting

use o f such
the

and d i d

criteria

(44)
and

two

found

factors

two s i g n i ­
and g e n e ra l

was

A ptitude T est
factor

the

w ith

be

scales

for

as

less

excellence

of

personality

scales

.50.
scores

to

nature

essentially

rating

through

employed a

excellence

given

are

personality

has

been

school

little

procedure.

w ith

as W eb b 's

the p r e ­

variables

irrelevant.
the

needed

academic

to

be

included

judgem ents.

U nfortunately,

d o u b t on t h e

people

years

records

success.

rating

that

to

design

academ ic-scientific

relationship

However,

are

scale

the

that,

such v a r ia b le s

factor

Kinnane

tho p e r s o n a lity

than

areas

occurrence

criterion

Exam ination

s h o u l d n o t o c c u r and
order to

all

problem o f d e f in in g

conclusion

a significant

in

the

dim ensional

Record

reliable

is

of

academ ic-scientific

G raduate

in

were

benefits.

The i n t c r c o r r e l a t i o n

Using th e

dictor

terms

potential

o f an e i g h t

data

hypotheses

a most u n f o r tu n a t e

S uzicdclis

factors:

personality.

sufficient

avoid th e

raters,

techn iq u e both

candidates

has been

done

The one s tu d y

to

as
to

review

the

a predictor
determ ine

encountered

does

high
for

the
cast

some

records.

statem ent;

earning doctorate

a movement

degrees

do a s i g n i f i c a n t
appear to

be

proportion

gifted

on

of

the basis

of

a

27
high

school

Kcnick
His

grade-point

( 3 .3 ) c o l l e c t e d

results

"risks"

indicate

on t h e

the

month p e r i o d

on

to

the

utility

the
in

the

a nine

Of t h e

106 s u c c e s s f u l

com bination
appears

to

V ariables
between
taken
this

as

bo

which

first

factor,

you have

to

adm it

com pleted

the

all

to

is

7 were

an d Huber

(42),

doctorate

in

purpose

is

the

64 p e r s o n s

employed

the

best

arrive

in a s tu d y

of

in

of

a

success.

to be u n s u c c e s s ­
41 w e r e

finding

incor­

that

discrim inant
to

scholarly

age;

at

the
function

work.

number o f y e a rs

An i n h e r e n t

least

the

wh o h a d e a r n e d

probability

are:

to

of

receiving

o f MA; a n d n u m b e r o f

in o r d e r

clinical

were

candidates

factor

candidates--at

the
as

predicted

graduate work.

that

answer

degree.

commitment

starting

for

students

the

Of i n t e r e s t

this

as

over a five

compared w ith

the

to a s c e r t a in

factor of

BA a n d

sem ester o f

however,

S trieker

a

contribute

com pletion of

in

to

classed

R cnick's

defined

were

which produces

doctorate.

a firm no.

64 u n s u c c e s s f u l

successful.

related

.39!

of success,

candidates,

the

ratings

a design

attained

of

scores?

and r e c o m m e n d a t i o n s .

f o r m was
is

test

sample were

106 e d u c a t i o n

not

function

of variables

the

predictor variables

out of the

classed

of

and

recipients

teacher

y e a r period

tw enty

discrim inant

of

developed

predictors

linear

rectly

rating

ratings

grades

(28)

s a m e p e r i o d who h a d

However,

school

records

A sample

over

269

school

and G ill

A pproxim ately

ful.

for

over half of

a standard

degree.

doctorate

teacher

intra-reliability

appropriate

graduate

data

of high

o f high

M arascuilo
finding

such
that

basis

Furtherm ore,

average,

credits

problem with

a prediction,

for re g istra tio n !
37 p e r s o n s

psychology,

who h a d

developed

a prediction

28
set

of

15 i t e m s .

A ptitude

and Advanced

undergraduate

GPA.

months betw een
tion

Seven were

o f the

the

o f minimal
(2)
graduate
(3)

of

value

as

The b e s t

This
to

using

the

other
were

third

Graduate

eight

were

graduate

year

in

Record

Exam ination

derivatives

GPA a n d t h e

the

of

the

number o f

pro g ram and

comple­

the

Pearson

Graduate

Record

Exam ination

a predictor of e ith e r

correlation

method

courses.

relationship

psychology

grades

inconsistency

a review

by

Reyes

of

- . G 2 was

in

in

oral

grades

and Clark

variability

were

GPA w a s

the

under­

However;

and t h e

apparent

com binations

criterion.

p re d ic to r of graduate

psychology

intra-individual

m ultiple

results:

single

An i n v e r s e

undergraduate

jected

the

GPA i n

on

exam ination.

follow ing

(1) All

the

end o f th e

A 19 x 19 m a t r i x
produced

tests,

The c r i t e r i a

the

oral

based

as

(3*1),

grades

earned

found

to

exist

exam ination

between

criterion.

a p r e d i c t o r was

sub­

Their hypothesis

that

is

a w orkable m oderator

*

variable

was

not

up o v e r

hold

later

grades

better

than

in

grades

rejected.

sim ilar

time

grades

predictions

showed th a t

for

consistency

dim ension.
consistent

for students

docs

The p r e d i c t i o n
students

w ith

high

of

was no

inconsistency

received.
e t.a l.

w hether g ra d e -p o in t
criterion.

(54)

in

inactive

designed

average o r

T h e ir sample

doctorate

had been

results

any s i g n i f i c a n t

from e a r l i e r

W illiam s,

the

The

for

and

longitudinal

graduation

included

education

a

wa s

33 s u b j e c t s

study

determ ine

a more s i g n i f i c a n t
who h a d

51 p o s t - c o m p r e h e n s i v e

a 21 m o n t h p e r i o d .

to

M ultiple

received
subjects

correlation

who

29
indicated

that

producing

a correlation

cant

the

predictors

M aster's

This

of

factors

at

th e same

severely

criterion

.65

were

program

received.

graduation

wap c l e n r l v

(p ' . 0 1 ) .

related

However,

to

prior

institution

lim its

the

a better
the

most

attendance

where

the

use o f such

choice,
signifi­

in

the

doctorate

was

a prediction

device

f o r ma n y i n s t i t u t i o n s .
As w i t h
latter
not

other

group has

large

fact

the

be s p u r i o u s .

places
results

one

in

on

in
the

now t u r n s

sound

discussed

throughout

m entioned,

the

area

designs

set

which he hopes

in

selection

on

linear

replication
[K elly

accepting

and

the

sample

those

Piske

both

this

regression

for

(13)

positive

a group o f s t u d i e s
resolving

proceeding

In most

question
to

process

variables.

w ithin

a specific

In t h e

review

If

pages.

cases,

as

crosses
the

s o me o f

graduate

and t h e

justify

to be confounded g iven

uncontrolled
stay

to

for

the

o f design.

the program

has

of

m ajor problem with

for

subject

design

position

considers

the

based

The a b s e n c e o f
of

reviewed,

In some s t u d i e s

correlations

terms

approaches

size
ma y

is

in

studies

which

a n d Webb

(50)1

and n e g a t i v e

faith.

■The r e v i e w

in

research

some p r o b l e m s .

and high

showed p ro m ise

two

is

investigator

reoccurring

problems

been p re v io u sly

a r e no c r i t e r i a

appropriate.

a study

the

performance p re d ic tio n

researcher

possible

program to

the

As h a s

there

program

which

develops

when th e

criterion

set

number o f i n t e r v e n i n g

and

to be

the

available

a criterion

furtherm ore,

lines

m eaningful

maximize t h e

u tility

lies

it
of

needs
the

to

criterion

s e t.
o f performance

prediction

the number o f s tu d ie s

30
containing

the

follow ing

A fter speaking
applied

to

the

select

and p r e d i c t o r
tion

process!

find

relevant

logical

to

It

predictors
point

needed

"policy

capturing"

is

(23)

not

of

The f i r s t
graduate

the

to

"garbage

the
The

latter

c e s s was

term ed Normative Judgement

order

reach

the

A least

predictor

squares

( 2 ) *n*e v a r i a n c e

group

is

actual

selec­

criterion

to

The
used to

question.

generation

studies

but

being

arc

a
employed

somewhat

capturing"
in

11 ) 6 8

the

item s

of his

A nalysis
joint

Houston

approach

on e a c h

expression

the b e st

,

approach

by

to

subject
policy.
the

policy

determ ine
(N = 3 0 )
This

author.
is

the

pro­

In

achieved,

the

wore ta k e n :

s e t was

sim ilar variance

the

predictors

follow ing

judge

(1)

the

program in

current

f o r each

steps

from

the

were

appeared

o f 22 p r o f i l e

a p o in t where

in

previously

predictors

what p r e d i c t o r s

lin e a r regression

in

generate

run.

a "policy

performance

(N = 2 0 )

been

long

the

in

has

surprising.

approach.

employing

a m ultiple

to

im portance

follow ing

to

applied

can" p re d ic to r

to

program.

this

school

establish

relative

to

proceeds

m eaningful

rejection

quite

of predictors

study

not

approach

study

developed

is

is

or

specific

representative

to

a set

would seem t h a t w o rk in g back

selection

the

that

students

What i s

w ithin

fact

i n c o n s i s t e n c y was

c o m b in atio n s which w ere n o t

starting

determ ine

logical

regression

for

cnch

ju d g e 's

combining o f

computed.
for

all

coefficients

judges
are

is

compared

grouped

and th o se w ith

and a v a r i a n c e

for

this

computed.

(3) The o v e r a l l

loss

in

predictive

efficiency

when N i s

replaced

31
by t h e b e s t

N-l

(4)
sents

set

of

Steps

the

best

raters

1 *3 a r c

joint

policy

indicate

that

is

also

2
until

repeated
for a ll

computed.
R^

raters

is

with

computed*

as

little

This

error

repre­

as

possible.
R esults
same p o l i c y
with

the

as

profile

ttoscoc
A nalysis
the

as

In

m ultiple

data

a criterion

to

employed

determ ine

Record T.xamination

results

2 31

were

the

A ptitude

doctoral

. . .
,
and Advanced l e s t

of
two

faculty

dictors

(53)

members.

groupings

Both g r o u p i n g s

Normative
predictive
Test

recipients

Judgement
validity

and th e
a n d 21

of

Advanced

dism issals

Graduate g ra d e -p o in t
„
.. j
1
Received d egree
N ormative Judgement A n aly sis*
*.
w
Faculty ratin g s* *

in

Q -sort Technique

12 f a c u l t y

analysis

the

a study
to

Using

for

.75

of

the

The i n v e s t i g a t o r

is

assuming
the

33 d o c t o r a l
the

14 v a r i a b l e s

and awarding

one m a jo r problem w ith

of

determ ine

em phasizing

accounted

.39
.34
.60
.32

_

. ^.
.
A ptitude

ployed th e

judgement

the

C riterion

n
.
.
R eco rd I-.xamination

c t.a l.

the

essentially

achieved:

_
,
.
Graduate

W illiam s

of

tho

Predictors

v

using

It = . 9 3 .

(37)

a study o f

follow ing

was

judges were

correlation

and Houston

G raduate

Test.

the

all

of

the

"Judgemental

fo r each

different

recipients

variance

A nalysis"

subject

predictor

em­

he

found

variables.

between

the

pre­

degree.

three

t h a t b y now t h e
studiesju st

reader

review ed.

hasr e a liz e d
Hie t e c h n iq u e

* 16 j u d g e s r a t i n g 30 s u b j e c t s o n a 5 - p o i n t s c a l e o f p r e d i c t o r
p r o f i l e s o t h e r th a n G raduate Record Exam ination S co res.
** 16 j u d g e s r a t i n g 1 0 d e g r e e r e c i p i e n t s whom t h e y k n e w o n a
1-10 ranking o f p r o f e s s io n prom ise.

the

.32
in

use

is

achieved

employed
is

an
In

1968,

graduate

in

faculty

w h i c h was

the

design!

rating

used

in

What i s

criteria.

the

actual

being

However,

selection

o f the

The s t u d y

representing

Score,

six

were:

was

produced by th e

w hich was

the
is

conducted

G raduate

predictor

to

(m ultiple

linear

Score,

Record

b a s e d on

that

(12)
Graduate
under the

w ithin

22

academic d i s c i p l i n e s .

U ndergraduate

A ptitude

(judgement

grade-point

G raduate

average,

Gradu­

Record Q u a n t i t a t i v e

Advanced T e s t ,

"common" s e n s e

discipline),

Judgem entally w eighted

approach

and s t a t i s t i c a l l y

to

im portance o f

w eighted

totals

regression).

The c r i t e r i a
six

success

Board re v ie w e d 6 p r e d i c t o r c o m b in a tio n s

Record Verbal

A ptitude

Lannholm, Marco and S c h r a d e r

school

Board.

The p r e d i c t o r s

w ithin

the

a monograph by

Exam ination

departm ents

totals

point

unknown q u a n t i t y .

auspices

ate

of

of prcdictors

on p r e d i c t i n g
Record

th e wrong

a refinem ent

com bination
still

at

employed w ere q u a l i t y

categories

reflecting

level

current

ratings

o f performance

placem ent w ith in

the

pro­

gram.
All
size

in

data were not
some was

The r e s u l t s
poor across

work

Hie
study.

of predictive

strongly

and an

follow ing

in

all

departm ents

and the

sample

sm all.

departm ents

The s t u d y
future

very

available

efficiency

as w e l l

as

recommends

emphasis
points

across

ranged

a minimum s a m p le

m entioned

in

to

disciplines.

on d e p a rtm e n ta l

are

from e x c e l l e n t

size

of

85 f o r

studies.
the

summary o f t h i s

33
(1)

U n d e r g r a d u a t e g r a d e - p o i n t was

12 d e p a r t m e n t s
(2)
the

but

poor

In g r o u p s

where

predictor

in

4 of

5 others.

undergraduate

judgem ental ly w eighted

than

total

grade-point

proved to

be

was

available

a better

predictor

GPA a l o n e .
(3)

Tn g r o u p s

c a lly w eighted

(4)

of

total

judgem entally

proved

com bination

statistical

jecting

Dawes'

to d e v ise

strapping",

school

statem ent

by

of

centered

is

linear

predictors--a

is

adm ission

higher

four

ofthese.

problem

o f determ ining

selection
(9).

the

the

that

decision

considered

eight

predic­

at

indicated

com m ittee's

in

the

wns c o n d u c t e d

R esults

applicants

around

the

He r e f e r s
at

the

job

to

th e Oregon

stu d y 's

a

would have

w ithout

re­

docs
this

as

it

w ell

process

Research

data

of predicting
itse lf,"

from th e

monotonic

value

at

least

that

as
as

is
the
"boot­

Institute.

p r o o f o f the h y p o th e sis

of the

com mittee

is

hypothesis

method t h a t

com bination

a better

derived

only over

only

adm ission

nn a c t u a r i a l

interest

the

s ta tis ti­

applicants.

developed

do

case

available

Dawes

the

a term

w ill

one

the

were

actual

question.

a sim ple

than

the

in

Of p a r t i c u l a r

considers

is

in

(N> 6 0 )

which

in

the

of the

55 p e r c e n t

human j u d g e m e n t

"that

in

w ith

Institute

work

scores

be u s e f u l

dealt

any a d m i t t e d

possible

test

sim ulation

screened out

size

total.

to

used

Research

sample

showed an a d v a n ta g e

w eighted

A s tu d y which

Oregon

sufficient

The A d v a n c e d

departm ents

tor

in

a fair

the

performance
(9:4).

quality
as

adm ission

of

good as

in

com mittee
graduate

The p r e c e e d i n g
the

any

adm issions
low er v alu e.

34
Therefore,
to

do as

since
such

the
as

linear

w ell

as

actuarial

of

also

we c a n

of

to

further

meet

contends

this

to

random e r r o r

actual
one

principles

predictors.

related

or near

the

assum ption,

the

error

that

variables

than

judgem ental

zero

ought

by e x t r a n e o u s

related

system atic

the

function

b e more v a l i d

from th e

w o u l d s um t o

then

actual

Dawes

should

In o r d e r

that

the

affected

system atically

and u s e

considerable

not

deviations

assume

members,

of

itself.

sim ulation

that

are not

interpretation

is

method is

com m ittee.

assume

com mittee

judgement

the

the

in q u e s tio n
however,

representation

the

fatigue,

decision
must

a

to p r e d i c t o r

zero

assum ption

If,

across

o f the

the

sim ulation

value.

S u mma r y
This

discussion

prediction
a result

of

of

of

those

studies

doubt

as

to

judgem ents

or

data

available

of

techniques

significant

sim ulated

process

a predictor
relationship
substantial
judgem ent

success.

validity

success

in

in

or

of the

indication
the

past.

Most
to

or

and

as

to

the

r e s e a r c h e r and

of

this

provide

is

are

the

data.

In

is

their

findings.

some

actual

criterion

enough

Therefore,

of

inform ation

the

fact

that

a higher co rrelatio n al

m eaningless

predictive

the

there

s o me s p e c i f i e d

has

in

problem s

com binations,

involved.

judgem ent

these

do a p p e a r

interpretation

a sim ulation

actual

inherent

dealing w ith

results

judgements

persons

of

the

between p r e d i c t o r

the

the

in

problem s

th e program do not

com bination
than

the

school

analysis
where

f o c u s e d on

correlations

performance
the

kind

the

S ignificant

about

graduate

the

lim itations

has

unless

accuracy o f the

we h a v e
actual

35
As s t a t e d
study

is

three

(1)
ing

in

the

the

chapter,

the

purpose o f

this

current

fold:

To d e v e l o p
level

first

an e v a l u a t i o n

o f performance

procedure

o f the

current

for

use

in

graduate

determ in­

student

popula­

tion .
(2)

To q u a n t i f y

the

param orphic s e le c tio n
of this

data

in

( 1 ).

i tern
(3).

in

developed
of

the

various

in

process

situations

study
to

if

turns

achieve

assum ptions
pieces

the

to

develop a

level

of performance

developed have

the

as

value

developed

any u t i l i t y

and program s.

now t o
these

to

the

decision

purpose o f assessin g

procedures

a discussion

purposes.

and r a t i o n a l e

needed

com m ittee

for the

determ ining

To d e t e r m i n e

different
This

set

adm ission

o f the

This w ill

underlying

construct

the

the

design.

research
include

design

consideration

selection

of

the

CHAPTER IT f

RESEARCH DESIGN

Selection

of

the

In o r d e r
investigator

to

Sample
achieve

felt

in c l o s e l y

related

(1)

the

use o f

school

stated

necessary

to

disciplines

same K i n d s

candidates

and,

of

purposes

find

who m e t

Testing

the

inform ation

An i n t e r v i e w
adm issions

and S o c io l o g y
of these

in

representative

stated

sim ilar

other

to

inform ation

the

the

had been

as

Departm ents

Canadian

U niversities,
F all,

the

used

students

the

the
departm ents

criteria:

selection

enrollm ent

of

that

graduate

would

recommended by t h e

member r e s p o n s i b l e

location
the

each

(1) w i th

department

Fall,

a minimum o f tw o t e r m s
36

in

Educa­

the

in

the

graduate

Psychology
the

Each

selection
departm ent

the process
same k i n d s

the p re v io u s

five

degrees

adm itted
1970,

and

to

was

of

years.

c a me f r o m a p o p u l a t i o n

undergraduate

1967 t h r o u g h

study.

used

and t h a t

least

( 2 ) who w e r e

resulted

fo r the

inform ation

for at

for

o f A nthropology,

U niversity

two d e p a rtm e n ts

graduate

and c o m p le te d

faculty

that

The s a m p l e w i t h i n

the p e rio d

the

a t M ichigan S ta t e

departm ents

study,

(12:7).

w ith

process

the

follow ing

in

(2 ) h a v e a g r a d u a t e

Service

of

a minimum o f t h r e e

a sample o f s tu d e n ts a t the le v e l

provide
tional

it

the

of

from American o r
graduate

work

for

( 3 ) who e n r o l l e d

o f coursework.

The c h o i c e

37
of

these

of

the

three

sample

investigator

as

adequately

interpret

countries,

and t h e

in

two o f

derived
select
two
in

the

his

the

of

had

the

The n a t u r e
w hich

in

each

m ajor

records

The

considerable

on

third

until

bearing

inability

beyond

these

students

the

the

on

student

criteria

to

1967

was
did

not

of

one

com pletion

the

to

two

prior

characteristic

departm ent

advisor

the

c o m m itte e members

transcripts

com plete

that
or

adm ission

meaning o f
of

d e t e r m i n e d by

to

be

or

used

were
and

P redictors
of

the

currently

could be

study

stated

that

adm ission

decision:

planned

em ployed

expressed

partm ents

the

(1)

The A p t i t u d e

[2)

The U n d e r g r a d u a t e

Test

in

required

the

selection

in q u a n t if ia b l e

follow ing

of

the

the

of predictors

process

term s.

inform ation

G raduate

use

Record

by

All

was

the

depart­

three

em ployed

de­
in

their

Exam ination.

Grade

Point

Average

for

C3) T h e U n d e r g r a d u a t e G r a d e

Point

Average

in

fourth

the

third

and

years.

discipline

or

related

(4)

A statem ent

(5)

At

least

A discussion

of

of

letters
these

tion

of questionable

the

science

interest

that

were

the

two

social

indicated

of

the

departm ents.

fact

This

as

was

study.

S election

ment

the

com mittee

term s.

w ell

lack

three

from th e

characteristics

of

their

goals

by

the

candidate.

recom m endation.
w ith

statem ent

u tility .

thesis

in

courses.

predictors

c a n d id a te 's

in v e stig a to r's

or

courses

and

Further

com m ittee

members

and

of each

letters

of

conversation
review

departm ent

recommenda­
w ith

members

o f some p r e v i o u s

38
research

in

this

m eaningfully
those

process
which

units

discussing

com m ittee

in

in

led

one

the
to

norm ally

order

to

correct,

the

the

the

is

an

not

that

grades

if

arc

grade

th is

a qualitative

devised.

Dawes

on

(9)

as

to

C olleges.

In
(1)
by

raw

the

Score

score.

of

a rough
by Cass

at

the

predictive
of

the

true.

to
the

and

M ichigan

had

i n d e x was
data

G raduate

of

faced

based

to be
Record

on

to
had

method
achieve
to

this

that

for

collected

be
prob­

adm issions

Com parative

indicated
adopted

practice,

grades.

this

institutions

R im baum 's book,

logically

reputation"

In o r d e r

also

know

assum ption

S tate

influence

process.

to

actual
the

S tate

In

w hile

In

the

undergraduate

M ichigan

a "national

index o f q u a lity
and

of

at

statem ent,

Oregon had

Dawes’ r e s u l t s
and

earned

for

indicated

member o u g h t

undergraduate

work

responsible

faculty

equivalent

adm ission

adm ission

f a c t o r was

o f knowledge

determ ine

the

application.

one w ith

to

Therefore,

the

com pletely

not

futile.

A com parison

latter

lack

is

his

some u t i l i t y

TTie V e r b a l
the

rated

summary,

a

the

in

selectivity

has

be

not

rating

developed

cedure

to
on

the

This

made t o

lem and h a d

American

a sixth

review ing

com parison

attem pt

study.

person

the

person

any

members o f

the

assum ption.

school

com parison

w ith

gradc-point

based

was

from t h e

and w ith

that

would he

equivalent

probably

A decision
of

removed

school.

the

o f data

predictors

appear

is

conclusion

two d e p a r t m e n t ,

this
the

the

type

follow ing

about

com parison

made

these

made by

does

were

other

perform

som ething

this

departm ent

g rad c-p o in t w ith
is

led t o

quantify

two d a t a
In

area

this

this

Guide
pro­

study.

were:

Exam ination

as

expressed

39
(2)

The Q u a n t i t a t i v e

as e x p r e s s e d by
(3)

the

Score o f th e

as

e x p r e s s e d by

T h e GPA i s
(a)

computed

Assign

C=2,

D el,

tion

reports
(b)

G rade-Point

Average**

a 0 to 4 sca le
as

use

carried

for

the

third

two d e c i m a l

and

fourth

places.

follow s:

a number to

Fe 0 o r

Record E xam ination*

raw s c o r e .

The U n d e r g r a d u a t e

year

Graduate

the

each

num eric

letter
grade

grade,
if

i.e .,

that

is

A=4,

how t h e

B=3,
in stitu ­

grades.

Record

the

number o f c r e d i t s

o r hours

earned

f o r each

grade

recei ved.
(c)

Sum e a c h

grade

column t o

determ ine

the

number o f

credits

per grade.
(d) M u ltip ly
tion

assigned
(e)

grade

that

D ivide

points

in

*

for

grade

to

credits

(d)

and ro u n d o f f

the
in

per

determ ine

sum o f a l l

T h e GPA e a r n e d

discipline

total

the

number r e p r e s e n ts
(4)

the

grade
to

g ra d e by t h e

num eric

designa­

grade p o in ts.
rows

in

(c)

two d e c i m a l

by t h e
places.

sum o f

all

This

GPA.
undergraduate

two d e p a r t m e n t s

courses

or social

of

the

science

graduate

courses***

school
for

the

For th e re m a in d e r o f t h i s t h e s i s th e G raduate Record Exam ination
w i l l b e r e f e r r e d t o a s t h e GRE.
T h i s i s an a c c e p t e d a b b r e v i a ­
t i o n f o r t h i s t e s t and i t s u s a g e i s w id e s p r e a d .
** F o r t h e r e m a i n d e r o f t h i s t h e s i s
the phrase "grade-point average”
w i l l b e r e f e r r e d t o a s GPA.
( F o r r a t i o n a l e c t . C h a p t e r 2 p . 16)
*** F o r p u r p o s e s o f t h i s s t u d y s o c i a l s c i e n c e c o u r s e s a r e d e f i n e d
according to the requirem ents fo r a b a c h e lo r's degree in the
C ollege o f S o c ia l S cience a t M ichigan S ta t e :
S courses in S ocial
S c ie n c e c h o se n from th e D ep artm en ts o f A n th ro p o lo g y , Econom ics,
Geography, P o l i t i c a l S c ie n c e , P sy c h o lo g y , and S o c io lo g y .

40
D epartm ent

of

A nthropology

two

decimal

places

(5)

A quality

the

b ach e lo r's

the

follow ing

1971

degree

is

b a se d on

the

Row, New Y o r k ,

(4);

(1).

scale

and t h e

cific

departm ents,
are

other

to bo

in which

C riteria

The c e n t r a l

question

the

of the

what w ill
The

he

6

carried

(3 ).

scale

which

awarded

computed

in

adm ission
program :
or

firs t

she
item

(3);

The
for

predictors
in

by

Bim baum .
Dawes

(5);
(2);

contribution
study

w ill

this

(9:22)

very

not

of

this

provide

specific

evi­
rating

scale.

further
were

the

1970-

rating

between

and

a n d M.

values

the

im plied

analysis

Col l e g e s ,

selective,

the

agreement

discussed

Cass

selective,

conversation

collected

data

and

analysis

w ith

tested.

of

the

spe­

th e y were em ployed.

of

the

selection

data

set

relates

If

a

be a s k e d
that

highly

selectiv ity

Amcri c a n

b y .1.

1.)

co m m ittee's

the

in

is

of

data

to

inform ation,

analysis

amount

in itial

departm ent

m ation

to

num erical

(6 );

U niversity

adm ission

the

of

item

adm issions

Gui d c

selective,

method o f
the

however,

S election

in

4 scale

institution

1

of

1970)

given

(For r e a d e r

the

regarding

These,

very

S tate

dence

study

w ere

Most s e l e c t i v e ,

(♦),

A fter

Com parative

categories

to

a

rating

and

predictor

described

e x p r e s s e d on

(Harper

M ichigan

a 0 to

manner.

indicated:

m ents

as

as

by

undergraduate

book,

m entioned,

the

the

the

selective

for

of

in

T heir verbal
os

expressed

and co m p u ted

rating

The i n d e x
developed

as

candidate
to

comes

is

to

process
the

is

how t h e

specific

adm itted

to

the

infor­

require­
program ,

do?
to mind

regarding

what

ma y b e

asked

o f the s tu d e n t
In C h a p t e r
sure

as

the

appear

wa s

concluded
to

process

which

this

study

examine

asked o f

all

of

of

thesis.

population

was

not

com pletion

doctoral

any o r

criterion

to

GPA.

p a r t i c u l a r mea­

other

to

Graduate

its

other

measures

relationship
performance

unwise.

thesis,

student

m eeting

period o f

In t h e

tasks

too small

student

are

com ple­

o com prehensive exam ination

indicated

these

the

in

to meet

three

that

departm ent

t h e number o f

the

any

study

specified

legitim ate

time
sample

requirem ent.
Given

easier
w ill

the

by t h e

of all

relationship

are norm ally

and c o m p l e t i o n o f t h e

size

and i t s

expressed

the use o f t h i s

However,

tasks

of

that

reasonable.

of a m a ster's

a review

as

exclusion

w o u ld be e q u a l l y

O ther

persons

com pletion

the

selection

measures

tion

it

course

a criterion

did not
to

II

is

that

question

require

the s tu d e n t
volved

insufficient

perhaps

to

do?
this

issue

centuries.

rational

one

to

therefore
definition

remain

have been

the

this

purposes

subject

o f the

departm ent prepared

statem ent
for

of

inclusion

issue,

to

teach

which have r e ­

of considerable

the

for decades,

greatly

enjoy

investigator

of that
than

task

is

and w i l l

an o p e r a t i o n a l

study.

educational
in

the q u estio n

academic w orld

im possibility

of this

answer th e

program p u rp o rt

differences

the

to

then

i n v e s t i g a t o r would

resolve

realize

student,

the

the

throughout

available

c o n t e n t w i t h p r o v i d i n g no more

f o r the

A review

the

Wh a t d o c s

While t h i s

finally

enough to

asked o f

The p h i l o s o p h i c a l

and d is c u s s io n s

being th e

is

restatem ent:

around

w ritings

o f what

e v i d e n c e was

the

objectives

graduate

catalog

which
led

each
to

42
the

conclusion

under study
or

that

is

faculty

tw o-fold:

which w i l l

these

society;

this

what

to

student

sent

the

prior

That

of

to

Given
assess

the

pletion
if

in

every

point

in

work

these

studied

is

ought

to

say,
the

level

student

in

these

to

However,
individual

tasks

variable

tim e

intervals

at

the

degree.

To ask

all

same p o i n t w o u ld p l a c e

able

It

the

in

a n d how i t

that

to

discipline

their

can

be
of

to

or

some a t

assumed

w hich

With

of his

the
is

same

a working

to

which

research.
one

satisfactory
possible
at

this

a disadvantage

obtain

vocational
of

common g o a l

perform

to

com­

a certain

variations

the

repre­

field.

how d o e s

task

of

perform

students

term s

tasks,

nature

reach

the

m anipulation

further

conducting

measurement

to

in

form ulate q u estio n s

specific

the

perform

skilled

and

clarification
for

question

do t o

discipline.

can

the

each

the

a specified

discipline

some b o d y

perform

requires

ceiving

occupation

about

must be

o f performance

training
at

this

program the

symbols

student

A direct

asked

training.

t o be

facility

of

programs

occupation--college

studies

graduate

have

further

some

tasks?
is

the

the

the

existence

student

conduct

understand

provide

graduate

students

ro ad and w r i t e ) .
to

of

academic

and n u m e ric

term s

stu d e n t's

his

of

accom plished

he has

these

an

the

knowledge b a s e .

student

verbal

know how t o

the

be

the

function

(2 ) t o

present

of

a specific

college

as

O bviously,

of

that

knowledge

to

and

for

The

teach

definition

functions?

(he h as

can

its

does

understanding

to

referred

expand

Given
becomes;

to

purpose

training

member.

(1)

body o f knowledge
contributes

essential

vocational

university

appears

the

the

these
of re­

same task

o v e r which th e y

43
h a v e had no
beyond the

control.
powers

each

program t o

sure

of the

If the

the

faculty
is

to

to

ask

em ployers
the

future

a

of

was

his

task

aspects

of
of

therefore

ten
the

be

to

task!

to

possible,

require

all

Therefore,

as
of

judge

it

is

students

an

in

indirect

judgement

th eir

o f the

mea­
faculty

tr a in in g - - appears

the
it

.a s t u d e n t ,

performance

assum ption

of

graduate

members w e re
members

as

though

those
is

inclined

usually

require

knows h i s

subject

area;

of

research

the

questions

members

level

of

is

appenred

to

his

training
best

as
of

a pro­

S pecifically,

indication

ability

ability;

kind

which

ask.

some

compared

to

of

communicate

and a b e s t

guess

a "teacher-scholar. ”
the

literature,
for

expressed

f o r t h e i r work

vocational
in

one s t u d y

which

providing

inform ation

in
is

term s

The

categories

nature
a rating

of

the

to

that

his

representing
graduate

devised
on

concerns

Lannholm,

follow ing

instrum ent.

had

o f em ployer

indebted

and hopes

them j u s t i c e .

performance

em ployed

perform ance

what

characteristics

does

the

and w ith

The i n v e s t i g a t o r

(12)

instrum ent

consists

oriented,

perform ance

encountered.

their

Given

current

review

and S chrader

were

and s u p e r v i s e d

asked

faculty

set of probable

graduate

a task

perform ancc--the

of faculty

prom ise

In t h e

a

be ju d g ed ,

candidate

effectively;
at

arc

faculty

em ployer

how w e l l

such

conducted

being v o c a tio n a lly

spective

such

investigator

task

a judgem ent?

questions

if

replacem ent.

e x a c tly what
reach

this

stu d e n t's

reasonable

to

of

com plete

m e m b e r s wh o h a v e
a

liven

Marco

adaptation
criterion

of

set

m easurable

program

and

could

44
C riterion
1.

Set:

M astery

of

fundam ental

of

relevant

knowledge

in

the

stu d e n t's

general

field.
2.

M astery

3.

Intellectual

4.

O riginality

5.

Industry

6.

E ffectiveness

o f w ritten

7.

E ffectiveness

o f oral

8.

Prom ise as

a

productive

9.

Promise as

a

teacher.

10.

H aving
explanatory

for

the

mentoer,

term s

was

no tim e
sion

on o r
In

vant

the

expressed

in

the

for

the

as

of the

explained.

have
scale

as

the

presentations.
research

worker.

to

each

category

involved

in

the

clarification

or

further

the

raters.

of

of

plus

and

against

ten

did

in

established.

set

can

that
into

the

a

further
anyone

of

stated

as
these

that

request

at

expan­

categories

c r ite r io n o fthe
the

This

GPA,

criterion
strengths

be m easured
a prior

legitim ate

rele­

samem an n er as t h e

relative

point,

community

definition

perform ance

com puted

self-

categories.

additional

w hich

data

set

of
the

now b e e n

criterion

study

would be

academic

can be

perform ance

set

achieve

It

this

the

term s

adm ission

In o r d e r
of

reports.

name o f

execution

same

to be

placem ent

no

the

graduate tra in in g

GPA,

weaknesses

that

a criterion

undergraduate
used

application.

clarification

summary,

to

research.

a person

provided

during

in

evaluation.

assumed

faculty

skills.

breadth.

and

O ver-all

research

and,

set

is

and

possibly,

phase--the

measurement

device

45
to

ascertain

must

first

the

be

C onstruction

and
of

and t h e

of

some o f

relevant

1.

Raters

2.

In

appears

have
to

the

that

the

the

train in g

is

outcomes

plus

to begin

these

student

perform ance--

of
of

the

to
the

the

more a c c u r a t e

consideration

of

the

the

(I-dwards

m iddle

tech­

(2),

G uilford

facts:
points

of

the

scale

points.
levels

the

optim al

technique

approach.
conclusion
standing
than

the

of personality

(44)

Scale

on s c a l i n g

follow ing

extrem e

relative

im portance

scales,

the

performance

of

literature

the

mark

statem ents

Kinnanc

Triadic-C om parison

rating

produced

prediction

and

the

paired-com parison

generally

by S u z i d c l i s

of

a tendency

com parison

Adding t o
(13)

(6)],

selection

to be

the

construction

and T o rg crso n

opposed

on g r a d u a t e

Pre-T esting

(3),

as

opinion

com pleted.

A review
niques

faculty

of
in

the

buffer
felt

nnd

group

prediction

investigator

instrum ent

K elley

com pleting

of

scales

Fiske

training
indicated

adequately

fo r m easuring

the

armed

perform ance

categories.
The o r i g i n a l
Schrader
w ith

(12)

consisted

a provision

happened

to

be

by

a group o f

in

question.

tendency

the

sm all

for

the

to

group

of

as

select

research

reached

called

the

for

by

for

m iddle

results

was

Lannholm,

each

no o p p o r t u n i t y

The i n s t r u m e n t

who h a d

planning

constructed

S categories

selecting

case.

raters

As c u r r e n t
the

instrum ent

to

a single

and

characteristic
observe

designed

a concensus

Marco

to

if

that

be

answered

decision

rater

o f the

responding

to

ratee

the

g r o u n d w o u l d b e much s t r o n g e r b a s e d
in

the

area

of

the

"risky

shift"

scale,
on

phenomena

A6
in

sm all

the

group

original

lack

research.

5 categories

of observation

tions

Therefore
to

Poor,

decision

7 categories,

possibility.

underlined):

the

The

M arginal,

By a d d i n g

the

spreading

ratings

categories

was

increased.

among s u b j e c t s ,

as

the

compared

to

the

categories

these

across

C oncom itantly,

a

made

to

allow ing

Below A v e r a g e ,

O utstanding.

of

still

rating

A verage, S uperior,
possibility

was

expand

for

were

Average,

the
(addi­

Above

two c a t e g o r i e s
the

finer

m iddle

five

discrim ination

three-category

spread,

would

he

achieved.

w ith

The q u e s t i o n

o f what

created

real

some

ideal

for

the high

"boni

loser"

for

the

would

a definite

end

or,

to

be more e x a c t ,

had

to

be p ro v id e d

K elly

and

cision
ison

Fiske

be

rated.

At

this

his

(1.1)

point,

Paired

of

prior

assum ptions

the

and

peers

design

requires

on t h e

need
that

to
the

to

to
the

w ithin

right

the

all

ju d g e d by

compared w ith ,

program ,

into

as

the

the

other

paired

a serious

that

subject

each

question.

(1)

N is

that

conclusion

developed

be met;

the

Therefore,

be

be

of

r n t e c was t o be r a t e d

defined

in

made

a per­

expected

was

was

be

notion

categories.

R ecalling

each

trait

total

be

subject

status

was

leads

three

to

selected

nnd h i s

could

subject

rater.

that

rater

scale

it

the

whom t h e

study

subject

(2)

to what

concerning

com parison

other

and

m iddle

course

the

every

subjects

the

specify

peers

This

in

for

was made t o

w ith

than

to

the

end o f t h e
end,

as

c o m p a r i s o n was

If

low

up

statem ent

rating

problem s.

ceived

subjects

the

that
of

the

of
the
in

manageable

to

com parison.
problem .

compared w ith

To b e e f f e c t i v e
rater

compar­

subjects

technical

be

de­

can

size.

rate

two
all

B ased on

47
previous
such

attem pts

attem pts

.0001

that

sam ple

in

would

the

thought

of

(or h e r s e lf )

the

this

is

to

be

w ith

these

carried

out,

please

the

order to

departm ent

inform ation
it

the

and

ratings

ratings

rating

to

the

populations,

would
The

the

for

a

triadic-

rater

The

this

Instructions

of

the

him self

triadic-

anchor

a com parison
how

the

im possible.

by

select

of

entire

Furtherm ore

action.

To s e e

the

of

level

respectively.

chosen

to
by

rate

possibility

course o f

follow ed

to

seemed

the

results

significance

manner).

sample

logical

refer

a

departm ent

paired

anchor p o in ts.

inform ation

study

was

circle

in

this

to

as

on

to

from

review

population

w ith

most

points

o f each
process

H ating

sample
is

in

was

o f the

rater

fensiblc.

Due t o

D epartm ent

of

unit
of

popula­

for

this

rate

basis

sent

to

relationship

and

a

fairly

e f­

knowledge

was p o s s i b l e .
all

faculty

one year.

perform ance

subjects

Sociology,

gathering

members h a d p r i o r

f o r more t h a n

all

the

return

a day-to-day

list

be

Because

departm ent

six

the

logical

a high

population

departm ent

requested
on th e

both

the

interaction

the

the

a pre-test.

as

subject

required

relationship

achieve

tim e

was

m ethodology would

selected

conducting

o f the

who h a d b e e n
member was

and

if

existing

turn-around

A list

then

the

reflections,

rated

see

w ould bo e a s i e r

ficient
of

some

in v e stig a to r's

this

other

number o f

a

at

the

A.

However,
in

given

a forced-choice

subject

tion

any

a self-selected

appeared

sample

A ppendix

in

students,

would he u n a b le

9,312

considerable

com parison w ith

for

(or

on

prediction

raters

45,582;

coding

inform ation

for

com binations,

13,806;

com parison

allow

manner

require

A fter

gather

faculty

this

theoretical

to

Each

members
faculty

categories

whom h e

felt

he

and

48
could

rate

(return

on

rate

a minimum o f
of

77%)

the fa c u lty

member

that

the s u b je c t

list

population!
w hich
need

this
for

10 r a t i n g s

at

the

also

is

per

for

rating
was

the
to

scales
to

in

in

review

those

feedback

A rating

rating

However,

verbal

categories

''b est"

subject

"w orst"

as

It

be most

increased.

o f the

design

faculty
of

the

possibilities

im petus

geared

to

an

indicated

instrum ent

I,

to

to

the

average

that

a pre-test.

the

raters

for

rnting

who h a d

5 was

noticeable
H appily,

and

difficulty

that

subject

S o me r a t e r s

w hile

they

felt

were

that

they

to

be

to

the
be

could

reluctant

their

was

men­
list
20

letter

com plexity o f

w ith

were

to

a b i l i t y to

understanding

characteristics
felt

an

number o f

proved

in

reasons

a cover

the

the

this

step

Appendix A ),

that
as

was

population

indicated

assumed

This

the

the

(sec

A p p e n d i x A)

was

construction

containing

considerable

a "w orst"

"poor".

was

minimum d i f f i c u l t y

on w h ich

and

the

added

Sociology

for

members

t h e r e was

The m a j o r i t y

of

package

sheet

faculty

of

design

instructions

indicated

the

C onsidering

4 percent

showing b u t

lists

students

average

only

provided

of

Departm ent

would

w ith

design

process

te n o r more s t u d e n t s .
instructions

5.

the

adequacy.

the

the

(see

task.

the

of

the

triadic-com parison

be accom plished

the

a

rater,

step

only

A lthough

departm ent,

sent

rate

represents

previously.

the

a poor

of

mean n u m b e r o f

118 nam es, t h e

this

conducted

tioned

contained

A review

r a t e was

Frankly,

The n e x t
the

could

fam iliarity

minimum p o i n t

subject

he

the

than

study.

of

felt

that

more

showing

the

categories.

indicated

whom

member i n d i c a t e d

two

ratings
false

as

the
seven
rated.
select
to

class

" w o r s t " was

in

49
fact

"below

average"

and no

" p o o r " w a s muc h s t r o n g e r
est

subject

required
the

ought

of

the

im ply

subject

possible,

student

of

scale,

what

verbal

better

is

the

than

trait
is

instrum ent

to

allow

by N p o i n t s

characteristic

his
in

point?

Building
the

o f where

term
the

low­

triadic-com parison
order to

achieve

rating

category

labels

define

It
to

w ill

of

o f the
the

rater

do was

select

that

was

is

ability

all

subject

that

indirect

did
the

in

than

logic

on

it

that

other

terms w hich

re­

appeared

whom h e p e r c e i v e d
to

the

level

was r e q u i r e d .

s u b j e c t whom h e p e r c e i v e d

point

X2 o n

of the

scale,

a

"Some

In e s s e n c e

level

not
pur­

If

the

ability

the

measure of

c a n make?

better

down

on t h e
that

to

that

statem ent,

su b je c t's

points

violation

performance.

scale."

the

in

be av oid ed

an

rater

" XI

all

a place­

t h e m on a s e v e n

this

categories

sample,

be r e c a l l e d

and r a t e s

from

imply

categories

student

the

to

to

establish

the

end p o i n t s

had

the

set

o f the

and t h a t

designed

of

follow ing:

a definition

ability

is

comparison w ith

flect

had to

of the

in

As t h i s

statem ent

perception

T h e n why n o t

Rather than

were

on a s e v e n - p o i n t

subjects.
this

the

self-selected

criteria

sim plest

others."

giving

that

the

description

X number o f s t u d e n t s

can be expanded

most

the

instrum ent.

p e r f o r m a n c e on t h e

selects

rater

a

categories

outside

the

rating

rater

this

Given

u l a r g e r s a m p l e was n e e d e d .

pose o f this

are

r a t e r ’s opinion

distinctions,

rating

purpose o f

extent

connotation

revised.

The o r i g i n a l
ment o f the

the

The

o f ma x i mu m v a r i a b i l i t y

d e sire d performance
to be

than

to be p l a c e d .

anchor points

w ould have

lower.

What
to

have t h e

that
the

have
least

the

50
ability

from h i s

would be r a t e d
fore,

the

seventh
scale

using

first

point,

While
gories,

it

"most

A -

on t h e

did

serve

same

we c a n

of

to

another scale
so

that

The n a t u r e
for

obtain

each
on

of

This

and a

discussion

further

Collection

and

pulled
staff

by t h e

on a s e p a r a t e

the

All

i n v e s t i g a t o r who c o n f i r m e d
major o r s o c i a l
tion.

At t h i s

the

another

results.

science
time

the

and t h e
a

possibilities.

problem

by

one r a t e r

the

scaling

are

com bination

sim ilarity

is
of

w ill

be

Predictor

for

all

subjects

was

and

in

placem ent

any t r a i t
rater

a m atter
a sliding

best

measure

inter-rater

analysis
forthcom ing.

the p o p u la tio n were

the

then

scores

the

Data

departm ent.

GPA a n d r e c o r d e d
follow ing

and t h e

data

the

test

creates

the

of

m aterial

is

a measure o f

the procedure

Scores

the
on

equal

procedure

cate­

concerning

s u b j e c t by a n o t h e r

scale p o in ts

in each

verbal

Sim ply s t a t e d ,

t h e same

the

placement

problem o f

all

staff

able"

II)

GRE A p t i t u d e

sheet.

thus

seven

one s u b j e c t

of

records

clerical

recorded

with

was p i m i n e d as p a r t

Preparation

The a c a d e m i c

p ro v id e d with

subject- rater

reliability.

There­

is

on

base-line

;uichor p o i n t s .

Th e r a t e r

resolved

for

as

s a m p l e members

"least

Scale

rating

other

term ed,

accentuate

scale

All

s c a l e was

level

scale

the

trait

of chance.
base-line

ability

second

of

the

Hating

least-m ost

that

on

sample.

two s u b j e c t s

able".

the

interpretation
of the

these

point

of perceived

(See A p p en dix

with

s o 1f - s e l c c t c d

The

1a s t - t w o - y e a r s

turned
a n d GPA;

the

clerical

over

to

GPA

the

computed th e

undergraduate

i n f o r m a t i o n wa s a l s o

institu ­

recorded:

5]
].

Department

of Sociology

Graduate

GPA

D ecision

o f Admissions

5 categories
mit

and o f f e r

assistance;
Type o f
ship
(3)

-

(1)

admit

alternate

[4)

financial
or

Conunittc

A lternate

Department
Graduate
Typo o f

financial

(3)

admit

adm it;

and

(5]

(2)

received

Teaching

(4)

Program o r o t h e r m in o rity
2.

candidates

status;

assistance

status;

all

and o f f e r

m arginal

Trainocship;

-

none;

or

and

program

-

were

placed

assistance;

and o f f e r no

provisional

5 categories

in

(2)

financial

adm it.
-

(1)

Fellow ­

Research

A ssistantship;

(5)

O pportunity

Lqual

ad­

funds.

o f Psychology

GPA
financial

assitance

received

- C ategories

same as

Sociology

3.

Area o f

specialization

Current

status

Department
G raduate
Typo o f

in

-

program -

o f Anthropology

GPA
financial

assistance

received

- C ategories

same

as

Sociology
Score

on

departm ental

The r a t i o n a l e
of

the

for the

analysis

section
All

for

collection

itse lf

and w i l l

exam ination

of
be

the

additional

discussed

in

the

dnta
data

is

a function

analysis

each d ep artm en t.

d a t a were

previously

qualifying

be

the

then

converted

investigator.

to

the

quantifiable

term s

outlined

52
At t h i s
and

point

the

rechcckcd

d a t a were hand

for errors.

number f o r p u rp o s e s

C ollection

and

20 r a t i n g

(because o f th e
list
the

included
raters

covering
the

scale

in

purpose

and

faculty

also

A com pletion
follow -up
the

letters

th a t

a three-digit

of

tim e o f

to

staff

for the

Psychology

in

the

that

at

least

faculty

during

to

the

on

population

to

assist

area),

departm ent

s t u d y was s e n t

all

the

ra te r's

for

department

specialization

department

m ailed

instructions

and a

indicating
all

faculty

one y e ar.
leave

period o f

The

and th o s e
the

study

and

process

and

institutions.

two weeks was

were s e n t

at

the

allowed

end o f

this

for this
period

in

two o f

departm ents.

one-man r e s e a r c h

rating

p h a s e was

team c o u ld

carried

function

out

separately

more e f f i c i e n t l y

as

under

arrangemen t .
As e a c h

rating

p a c k a g e was

a r a t e r number and each
for that
the

list

the

the

other

Each d e p a r t m e n t a l
the

population

students

of

who h a d b e e n o n t h e
at

sheets

Data

the

Chairman o f

im portance

accepted p o sitio n s

assigned

containing

Department

those

from th e

p a c k a g e was

C riteria

name a n d a r e a o f

m e m b e r s who h a d b e e n w i t h
rating

the

the

o f the

locating

letter

of

package

subject

wa s

key punch

identification.

scales,

size

on t h e

Much s u b j e c t

Preparation

The r a t i n g
rating,

of

coded

student

scales

profile

returned,

s c a l e was

subject.

At t h e

were hand s o r t e d by the

scale

of

all

ratings

assigned

the

scales

the

com pletion

subject

assigned

three-digit

of the

assigned su b ject

f o r each

were

return

number
phase,

number and a

n u m b e r was

prepared.

S3
The p r o f i l e
ceived

for

scale

each o f

the

The n u m b e r o f r a t i n g s
p la ced on

consists
ten

the

mean s c o r e

of

performance

categories

for

received

the p re d ic to r

of

and t h e

keypunch

mean s c o r e s

sheets

according

ratings
that

wore
to

re­

subject.

then hand

the

subject

number c o d e s .
S P E C I A L NOTE:
m issing

predictors,

A review

of

subjects

m issing

these

figure

the

two p r e d i c t o r s

w ith

cases

process

contam inating

the

use o f th e

other

inform ation

us t o

his

actual

Data A n a ly s is

the

for

the

the

for

mean

for

subject,

is

level

the

it

was

M

was

decided

categories
to

the

is

small

statistically

=

for

number

of this

remote.

population,

fN

those

possibility

analysis

on t h a t

resolved.

number o f

a departm ent

Given

the

be

highest

me a n s c o r e

available.

statistical

to

predictor

this

category

the

had

problem o f

com m ittee,

GRI; t e s t

and g iv e

overall
the

indicated

the

Further­

g iv e n no

the

"best

guess"

predictor.

Procedures

detail

requires

the

research

problem

facto

sheets

in v e stig a to r's

perform ance

The o r i g i n a l

post

on

data

category

each p re d ic to r

more,

in

any

f o r whom n o s c o r e w a s

o f such

lined

for

population

in

the

a b s e n c e o f GRI: s c o r e s ,

t h e mean s c o r e

departm ental

students

the

data

Upon c o n s u l t a t i o n
to

In p r e p a r i n g

plan

called

below.

The s t e p s

com pletion

of

all

and d esig n

research.

for a nine-step

A ll

are

prior

being

o f the

process

successive
steps

as

d e a lt w ith
research

out­
step

a prerequisite.

The

is

that

is

each

here

in

which

centered

m anipulations

in

the

in

ex

study

loft).

are

the

each
not

placem ent

criterionsubject

selected,
school

highly
the

secondary

ex p o s t

different

A nalysis

Steps:

(1)

Determ ine

by c o m p u tin g

to

im proper

of m ajor

the

the

to

of

principle

applying

to

one

this

of

random ization

control

to

the

next

and d a t a

of

in

research

reliability

intraclass

noted

of

the

steps

of

v ariability

of

the

interval

class

correlation

is

computed by t h e

step

p arti­
and

action

was

In

minimize
approach

the

com ple­

sound,

was

a number o f
be

conducted

inform ation.
to be

and

Ihe s t e p s

follow ed.

In

unsuccessful
in

the

and

the

respective

unit.

raters:

class

total

to

and d i s c u s s e d

correlation

d eg ree o f hom ogeneity o f each

to

logically

had

useful

analysis

be

w hile

I n some c a s e s

any

proposed plan

hi o r d e r

successive

com binations

gleaning

is

U nfortunately,

step,

accom plish.

department

the

im portance.

the

and

to

graduate

at

interpretation.

the

taken w ill

the
an

choice

therefore

represent

fo r each

self-

lacks

outlined

chapter

arc

research

approaches

steps

The p o p u l a t i o n s

facto

to

alternate

fixed

tests.

prior

cases where

are

f o r many s t a t i s t i c a l

reviewed p r io r

those

respective

them selves

o f enrolling
the

set

provides

difficult

below

then,

data

it

is

one s t e p

reality

variables

of m isinterpretation,

analysis

of

adm ission

initial

choice

lacks,

powers

possibility

tion
in

h a v e made t h e

susceptive

the

the

The i n d e p e n d e n t

The s t u d y

essence,

to

they

inherent

on

fu rth e r m anipulation.

and th e

cular.
the

to

o f w eights

This

was

which p r o v i d e s
[rater]

scale,"

accom plished
"a m easure o f

relative

B lalock

(1).

fo llo w in g method:

to

the

The i n t r a -

where
MS^ = M e a n S q u a r e

Between

MS *= Me a n S q u a r e
w
^

W ithin

N
(2)

<= A v e r a g e N u m b e r o f C a s e s

Determine

who r e c e i v e
rating:

is

ducted
then
(3)

are

than one

computing

based

on t h e

computed w ith

performance
present

there

more

A fter

categories
F -test

if

category.

then

the

for both

significant
rating

and t h o s e

nunber of
the
If

m ultiple

ratings

a significant
regression

im portance

analysis

p r o g r a m known a s

vides

a step-down

results

of the

F-ratio

test

program p io v id e
also

(4)

to

determ ine

between the
ten
this

placem ent

performance
step

is

(3)

to

accounted

decision

categories

the

of the

and th e

d e te rm in e which

the

performance

(1 o r

>1)

an

. OS f o r e a c h

difference

is

found to be

w ill

this
the

for

have

t o be

difference

con­

exists,

unit.

step

the
is

adm ission

a m ultivariate

FINN p r o g r a m w h i c h p r o ­
o f mean v e c t o r s .

an a n s w e r to

is

o n ly one

at

analysis

for

subjects

set

of equality

o f w h a t P, m e a s u r e s
1
Repeat s t e p

level

o f each p r e d i c t o r to

The p r o c e d u r e

variance

received

a single

com mittee d e c i s i o n :
of

who r e c e i v e

I f no s i g n i f i c a n t
as

between

f o r each o f

significance

sub-sam ples.

the

differences

t h e mean s c o r e

t h e s a m p l e may b e t r e a t e d
Determine

Per C ell

in

the q u e s tio n :

The

"how much

P . . . . P.,?"
2
N

degree of in te rre la tio n s h ip
adm ission

G raduate

criteria

GPA:

clearly

com m ittee

and t h e

The p u rp o se o f
differentiate

56
the

sample

and w a r r a n t

. 1 0 was u s e d
(S)

as

C onstruct

sample

a linear

adm ission

of significant
step

from th e

scries.

The p ro g ram
U nrestricted

is

of

the

placem ent

A gricultural

Squares

level

of

step.
adm ission

decision

relationship
decision

obtainable:

a m ultiple

Least

this

o f the

prediction

for this

called

strength

A significance

for

regression

the

and t h e

is

study.

cu t-o ff point

to a s c e r ta in

predictors
level

the

further

as

between

w ell

for

the

the

as t h e

The p r o g r a m e m p lo y e d

experiment

regression
and p ro v id e s

S tation

analysis
the

BASTAT

routine

follow ing

infor­

m ation :
a)

Simple
the

correlation

for

all

b)

A nalysis

o f variance

for the

c)

M ultiple

correlation

coefficient

d)

Regression

c)

b eta w eights

f)

P artial

g)

R

2

order

in

coefficients
fo r each

for

regression
and

equation

variance

each p r e d i c t o r

predictor

variable

variable

deletes

and th e

of the

predictor

linear

rule

correlation

for

the

actual

expected

set.

The v a r i a n c e

the

The r e g r e s s i o n
that

between

coefficient

value

correlation,

hypothesis

entered

coefficients

relationship

variable

variables

program

The m u l t i p l e

order

m atrix

( R)

value o b ta in e d

from th e

2

(R ) p r o v i d e s

stronger

analysis

none of th e

the

linear

The

zero
criterion

com bination

the strength
la rg e r the

of

the

zero

relationship.

o f variance provides
variations

the

for the

least-squares

zero o rd e r re la tio n s h ip .
the

represents

of scores

a te st of the
from th e r e g r e s s i o n

57
line

is

due to

this

hypothesis

ticu lar

the
is

F -test,

by o n e
all

they

statistical

correlation

between

independent
can.
in

itself.

Any u n u s u a l

an e r r o r

in

the

must be

have

should

the

"m ultivariate
not

a direct

determ ination
evidence

to

test

the

check

all

of

the

amount

can be e x p l a in e d
have

explained

measure
the

the
of

and

program

the
run

a result

follow ing

fixed

of

and

all

Y distribu­
each

variables
the

both

to

two a s s u m p t i o n s

a

analysis

coefficient

of variance

variable

provide

regression

the

assump­

X variables

Furtherm ore

analysis

of these

be p j^.05.

could be

that

Although

regression

par­

form at.*

other

assum ptions

presence

on

shifts

o f homos c c d a s t i c i t y .

these

that

distribution

about

w ill

this

com bination

distribution.
of

this

variables

between

requires

a normal

conducting

a measure

other

program

For each

2
(R ) a n d t h e

support

the

of

level

values

placem ent

distributed

normal"

the

regression

property

be n o rm a lly

in

must be

is

a reliab ility

predictor

effect:

Y distribution

tions

is

in

coefficient

significance
In

significance

after

3 provides

linear

F -test.

o f agreement

d a ta placem ent

The u s e o f

the

and expected

variable

analysis

tions

actual

The e x t e n t

step

The s t a t i s t i c a l

d eterm in e d by

the

The p a r t i a l
of v a ria tio n

predictors.

of

may p r o v i d e
in

the

sample

distribution.
(6)

Determ ine

adm ission
least

*

decision

squares

decision

the p re d ic tiv e

as

fo r each

program u sin g

the

predictive

validity

of

criterion:
the

original

variables

for

the p re d ic to rs
this

is

also

predictors
each

of the

and t h e

done w ith
and th e

the

adm ission

11 c r i t e r i a .

A more d e t a i l e d e x p l a n a t i o n o f t h e l e a s t s q u a r e s r o u t i n e
a v a i l a b l e in th e M ichigan S ta te U n iv e rs ity A g r ic u ltu r a l
E x p e r i m e n t S t a t i o n STAT S e r i e s , D e s c r i p t i o n N u m b e r 7 . L S ,
" C a l c u l a t i o n o f L e a st S q u a re s " (M imeograph).

is

58
The i n f o r m a t i o n
lined
(7)

in

step

process
analysis
(8)

as

the

to

how t h i s

section

em ployed

dealt

w ith

is

in

the

same m a n n e r as

the

to

on p e r c e i v e d

be s i g n i f i c a n t

accom plished

out­

is

in

success
step

discussed

in

(4):

those
The

in

the

data

w ithin

the

sig n ifi­

departm ent.
of

categories
to

agreement

found

extent

decision

than

of

f o r each

the

perform ance

regression

extent

categories

D eterm ine

cant

was

(5).

D eterm ine

perform ance

obtained

prediction
from th e

ascertain

if

possible

actual
one

decision

method

is

and

more

the

linear

appropriately

other.

Summary
Sum m arizing
of

im portance

test

th eir

graduate

to

the

discussion

the

selection

effectiveness

program

in

outlined.

by G r a d u a te

GPA a n d

faculty

a self-selected

for

categories.
the

bight

prediction

a detailed

this

process

predicting
G raduate

chapter,
were

of

is
the

to

steps
be

rating

students

have

been

conducted

results

in

each

w ill

be

ton

outlined

and t h e

in

the

by

the

perform ance
as

study

departm ent

to

measured

com pleted
on

units

and p la n s

perform ance

performance

sample o f

procedural

S predictor

defined

graduate

a triadic-com pnrison

analysis

analysis

in

t o how
w ill

unit

present

studied.*

* I t is w ith c o n s id e ra b le re g re t th a t the in v e s tig a to r re p o rts
th e lo ss o f one d ep artm en tal u n i t to th e s tu d y ; th e Departm ent
o f A nthropology.
H ie tr ia d ic - c o m p a r is o n r e t u r n r a t e , a f t e r
3 f o l l o w - u p a t t e m p t s , r e p r e s e n t e d 4 o f 17 r a t e r s a n d o n l y 2 7
o f 96 s u b j e c t s .
Given th e d a ta a n a l y s i s used in t h i s s tu d y ,
no m e a n in g f u l a n a l y s i s o f t h i s d e p a r tm e n t c o u l d b e c o n d u c te d
a n d t h e d e p a r t m e n t was rem oved f r o m f u r t h e r c o n s i d e r a t i o n i n
th e study.

CHAPTER

T ill:

Prior
to

to

the

fam iliarize

graduate
Since

the

com m ittee

study

the

relative

taken

by

Current

prised

of

three

by

the

Chairman

by

the

A ssociate

to

contact

Committee o f

faculty
of

by

the

Chairman

member.

the
the

uncontam inated

predictor

is

at

its

to

convert

terms

decision

for

[1967-1970).

in

variables,

each

the

Departm ent

the

order
the

should

of

the

The o v e r - a l l

departm ent

and

applicant;

and s e e

that

decision.

to

steps

be

review ed.

individually

applicant

review s

since

is

sent

receives

is
to

59

it

the

is

the

the

the

is

is

his

co-ordinated

next

which

receives
com m ittee

decision
to

term

r e s p o n s i h i 1i t y

adm ission
his

com­

a one-year

applicant

returned

three

n o member o f

for

process

and re c o r d s

The s h e e t

application

Sociology

answer q u e s tio n s

bach member o f

sheet.

of

appointed

data;

rating

and th e

Thus,

study

m issing

application

ment o f f i c e

this

appropriate
procedures

study

regarding

review s

appropriate

the

quantifiable

departm ent.

of

the

of

m e m b e r s wtio a r e

notification
each

of

o f the
arrive

seems

decision-m aking

period

into

it

Procedures

applicants

presented

to

results,

the

tho

decision

com m ittee

Adm ission

the

purposes

effects

’Hi e A d m i s s i o n

are

during

stated

adm ission

the

of

read er w ith

adm issions

one o f

W :P A R T M !:N T 0 I : S O C IO L O G Y

analysis

the

IV

the

on

depart­

com m ittee

separate

and

com m ittee

relatively

has

any

prior

60
knowledge
members

regarding

have

reached

three

decision

there

is

the

not

made
the

in

Hach

term s

The
first
the

is

garding

ment

is

tial

and

the

to

part

to
is

in

term s

internal
never

final

to

university
of

D uring
adm issions

the
the

continuity

* For

of
of

has

for

during

adm it.
This

or

A dm issions

point,
among

is

the

adm it,
re­

The s t u d e n t

The

financial

inform ation

which

the
is

is

place­

rem ains

to

application

O ffice

this

categories*

made a v a i l a b l e
the

he has

two p a r t s ,

decision

five

sheets

decision.

of

assistance.

made,

confiden­

student.

returned

inform s

the

ap­

decision.

studied,

1967-1070,

re-appointed
three

of

for

two y e a r s .

the

interpretation

category

the

decision

application

consists

one o f

only.

been

d e p a rtm e n t's
span

in

the

At

If

members ,

decision

consensual

If

adm it/not

G raduate

tim e

final

Chairm an.

all

the

three

and t h e

com m ittee

w ith

review s

decision

w ith

three

discuss

financial

discussed

application

colleagues.

to
a

When a l l

A ssociate

com m ittee

of his

placem ent

com m ittee were

served

the

reject.

decision

decision

One m e m b e r s e r v e d
members

the

possibility

is

the

at

adm it o r

an

plicant

arriving

w hether

only

to

the

among t h e

may c h o o s e

com m ittee

When a
the

to

decision

members

prior

the

notified

agreement

returned

colleagues.

decision,

forwarded

adm issions

second

of his

c o m m itte e member t h e n

of

com m ittee

them selves

are

clear-cut
is

decision

a separate

sheets

application

included.

the

description,

the

This

see

some m e m bers

on a y e a r
four years

to
and

the

application

C hapter

III

p . 51.

the

year basis.
two o t h e r

should have p rovided

of

of

data

reasonable
and

allow ed

61
for

the

c s t a b 1i s h m e n t

candidates
In

the

reviewed

in

judged to
rate

could be

the

Sociology

Department

On t h e

unusually

and

this

the

each
they

of

unacceptable.

on

high

point.

It

fund

over-adm it

not

to

was n o t

t h o s e who e n r o l l e d w o u l d b e

that

this

assum ption

detrim ental

Population

C hapter

III.

students
enrolled.
at

least

ducted.

the

valid

since

of

the

support.

the

the

adm itted

a d m itte d would not

actually

of the

that

percent

the b e t t e r

discussion

com m ittee were

contention
ten

adm ission

c o me

Therefore,

graduate

program

enroll.

further­

quality

results

and t h e r e f o r e

it

students
w ill

who h a d

he shown

contributed

in

a

program.

and Sample S iz e

predictor

procedure,

adm its would

was n o t

manner t o

The p o p u l a t i o n
whom a l l

the

their

jeopardizing

more,

In

the

a 75 p e r c e n t

do so w i t h o u t

only

funds.

of

surface

t h o s e wh o w e r e

since

received

all

25 p e r c e n t w e r e

t h e members o f t h e

was

afford

policy

a third

which

400 a p p l i c a t i o n s w e re

o f which

approxim ately

y e a r many o f
could

against

judged.

d e p a rtm e n t would o n ly

because

structure

approxim ately

appears

students

a norm ative

1967-1970 p e r i o d ,

be

questioned

of

the

of graduate
d a ta were

A fter

who r e c e i v e d
Of th e

s i z e was

of

prior

to

the

10,
the

for

study

is

to

105.

but

In t h e
only

1 had not

term th e

rating

11 8 f o r

outlined

triadic-com parison

by any r a t e r ,
all

the

and coded as

reduced

no r a t i n g

rem aining

fo u r term s

collected

com pletion

sample

students

in

rating

group o f

13

3 were c u r r e n t l y
been

enrolled

process

was

for

con­

62
Triadic-C om parison
The

faculty

usable

m aterial

called

from

indicated
subjects
tually
the

to

population
return

III,

the

rntcrs

' Hi e a v e r a g e

the

t h e minimum

sub-sam ples

on each

significant

differences

category.

in

should

were

be

prediction

of

by

to

the

in

be

the

rated

apparent
on

the

the

to

rate

However,

when

and p re se n te d

w ith

had

called

for

each

1 rating

.44
lack

for

performance
of

yielded

overall

o f consensus

as

continuum ,

the

subject

between

intraclass

the

for.

75 r e c e i v e d

categories
The

expec­
three

sample

and

many
ac­

in v e stig ato r’s

mean d i f f e r e n c e s

com putation

as

expectation

level.

re­

gathering

about

received

of

be

Id,

10 p e r f o r m a n c e
.05

w ill

was

30 r e c e i v e d

the

It

able

study.

com parison

of

returned

m ultiple

these
no

correlation

evaluation
to where

the

two t h i n g s
regression

equations:

the

size

number o f
expected

large

y on

of

the

in

prediction

discrepancies

between

a given perform ance

standard

deviations

in

the

the

category
actual

per­

rating.

A reduction
the

the

significant

at

bo p l a c e d

expected

formance
(2)

level

ratings

the

A substantial

to

for

a coefficient

a c tu a l y and th e
due

for

Because o f

subject

appear

rating

1 05 s u b j e c t s ,

method r e s u l t e d

not

21

inform ation

responded beyond

a v e ra g e number o f
Of

72 p e r c e n t .

number o f s u b je c ts

An I: - t e s t

(1)

the

of

prelim inary

necessary

>1 r a t i n g .

to

did

29 o f w h i c h

was

rate

that

raters

perform

above what

3.

a

believed

m aterial,

The
was

for

faculty

asked

tim es

rater

Chapter

as

tation.

R esults

m ultiple

correlation

efficiency

coefficient.

given

(1)

as

measured

63
Test

for

G r o u p Pi f f c r e n c c s

The m u l t i v a r i a t e
between

the

4 compared
.10

level

These
sion

adm ission
to

for

groups
the

2)

U ndergraduate

3)

Q uality
tion

Score

the

follow ing

categories

on

5)*

for

f o r mean d i f f e r e n c e s

adm itted

significant

predictors

the

based

on

subjects

differences

a step-dow n

GKF

R ating

(group

at

the

F -test.
p

.001

p <■ . OS

of U ndergraduate

Institu­
p < . oor>

variables
for

should

predicting

same

groups

perform ance

of

categories

Grade

2)

O riginality

3)

[Effectiveness

4)

Promise

as

5)

O verall

[Evaluation

in

inform ation

the

resulted

G raduate

a

therefore

analysis

1)

From t h i s

yielded

test

GPA

The m u l t i v a r i a t e
between

decision

follow ing

Verbal

equation

o f variance

1-3,

1)

three

analysis

Point

contribute

decision
variance
in

of

the

test

on

the

most

the

Reports
Research

follow ing

W orker

conclusion

can be

on

the

F -test:

Research

Productive

regres­

com m ittee.

differences

step-dow n

the

differences

Average

of W ritten

to

adm ission
f o r mean

significan t

based

the

p

< .01

p

< .00 5

p

< .01

p

< .10

p

< .19

tentatively

stated :
a)

The G r a d u a t e

among g r a d u a t e
of

GPA ma y p r o v i d e

students

then

a more

perceived

accurate

probable

given

discrim ination
its

restriction

range.
b)

*

The two

perform ance

categories

F or th e r a t i o n a l e in s e l e c t i n g
a n a l y s i s s t e p 3, p a g e 5 5 .

dealing

this

w ith

grouping,

research

see

data

and t h e

64
w riting

category

provides
all

could be

a better

evaluation

assessm ent of

The f o l l o w i n g

tion,
this

with
list

d ic to r's

is

using

cient

list

2.

to

provide

4.

5.

the

the

predictor

regression
w ith

than

that

the o v e r­

score;

industry

and r

r

com paring

possible

tellectual

breadth,
r

and

intellectual

r = .33 promise

as

by

presenting

on each

pre­

a

in p r e d i c t i o n

possible

the m u ltip le

product-moment

in

research

coeffi­

and

** - . 1 1

a productive

* The s i g n i f i c a n c e l e v e l s
p <.001 i f r >.32.

as

relevant

research

application.

= . 24 e f f e c t i v e n e s s

M a j o r GPA;

prom ise

provides

= .28 m astery o f

industry
r

= <.01

R ating;

list

in

correla­

category.

score;

GPA;

Q uality

indication

r = .26 o r i g i n a l i t y

= -.03

U ndergraduate

and t h e

and a p p l i c a t i o n .

GRH Q u a n t i t a t i v e
and r

The p u r p o s e

improvement

best

variable

a product-m oment

an

The

analysis
the

performance

GRK V e r b a l
= .10

read er w ith

amount o f

coefficient

ports

each

and w e a k n e s s e s .*

for

skills
3.

presents

categories.

for a given

r

factor"

R elationships

10 p e r f o r m a n c e

the m u l t i p l e

1.

"research

performance

m e a s u r e d by

point

correlation

actual

a

as

stm gths

reference

Category

relationship,

the

into

category.

Prcdi ctor-Perform ancc

range of i t s

combined

of w ritten

re­

breadth.

a teacher

promise
research

bto p < . 0 5 i f

as

and r

«= . 0 9

in­

a teacher

and r

=

<.01

worker.

r >.19;

p <.01

if

r >.25;

65
The a c t u a l
as

adm ission

follow s:

and

r =

decision,

.34 o r i g i n a l i t y

in

o f com parison,

research

and r

= .05

correlates

industry

application.

M ultiple

R egression

When t h e

A nalysis

5 predictors

combined and a n a ly z e d
the

fo r purposes

actual

presented

decision
in

Table

Results

e m p lo y e d by t h e

using m u ltip le

of

the

4.1

adm ission

indicate

adm ission

com mittee were

regression

analysis

com m ittee,

the

a significant

to p red ic t

findings

predictive

as

relationship.

TABLE 4 . 1
M ultiple

M ultiple

R egression

C orrelation

C oefficient
Regression

R esults

- Prediction

o f Admission

C oefficient

R «= . 5 6
2
R“ = . 3 2

o f D eterm ination
A nalysis

o f V ariance

F-test

The s i z e
tion

of

tween
the

m ultiple

a statistically
adm ission

degree

ginal

p<,0005

of the

the

given

D ecision

of

significant

data

linearity

the

correlation

set

2

R .

ly m onotonic r e l a t i o n s h i p
and i t s

im pact on

The v e r y

strong

significant

F -test,

a measure

for

squared

differences

This

the

level

relationship

that

decision

for the

mean i s

an i n d i c a ­

decision.

value o f each

that

is

relationship

indicates

actual

determ ining

from t h e

actual

in t h i s

fo r the

variable

predictive

and the

present

value o f

coefficient

of

accounted

However,
is

mar-

a stric t­

predictor
is

not

Regression

none

be­

the

present.
ANOVA

sum o f t h e

f o r by th e

predictors,

66
leads
cant

to

the

factor

conclusion
in

the

actual

on t h e b e t a w e i g h t
coefficients,
the

the

prediction

2.

Q uality

3.

GPA i n

4.

GRE Q u a n t i t a t i v e

5.

U ndergraduate

variables
expected

the

this

that
It

more c o n c e r n e d w i t h

is

as

the

Q uality

10 p e r f o r m a n c e
m ultiple

of

test

most

to

Institution

f o r mean d i f f e r e n c e s ,

it

found

courses

in

the
of

in

this

the

relationship

m ajor arc ta k en
adm ission

how t h e

has

done in

applicant

the

in

the

U ndergraduate

categories,

is

these

to be

during

courses
the

relationship

Institution

of

com mittee

p ro g ram to which

product-m om ent

that

A possible

interrelationship

the

courses

appears

have exchanged p la c e s.

would a p p e a r t h a t

the

regression

phenomenon.

order of contribution

GPA

C onsidering

R ating

correlation

Score

The s t r e n g t h

graduate

applying.

the p a rtia l

Based

coursework

ma y b e

,65).

given

a sig n ifi­

co m n ittee members.

and

ranked in

is

Score

Sociology

two y e a r s .

nature

levels

o f GPA p r e d i c t o r s

(r =

the

R ating o f U ndergraduate

to

of

of

data set

follow s:*

GRE V e r b a l

explanation

last

predictors
as

adm ission

decision

1.

two t y p e s

the

significance

score

R eferring back
the

that

is

o f the

same

applicant
of

the

p r e d ic to r w ith

im portance

of this

represents

an e x a m p le

o f the s u p p re s s o r v a r ia b le

is

a measure t h a t ,

in

the

the

The s u p p r e s s o r v a r i a b l e

variable

the

the

w hile having

* A ctual r e g r e s s io n c o e f f i c i e n t s , b e t a w eights, p a r t i a l c o r r e l a ­
t i o n s a n d s i g n i f i c a n c e l e v e l s ma y b e f o u n d i n A p p e n d i x B.
For a d e f i n i t i o n o f t h e p r e d i c t o r v a r i a b l e s , s e e C h a p te r I I I
pp. 38-40.

67
no r e a l

relationship

tributes

significantly

specified

value

variables

in

is

of

the

the

equation

criterion

the

from i t s

for

alone,

correlation

b a s e d on

com bination

sets,

of

the

the

including

student

adm ission

the

to

raters.

The p r o c e d u r e s

common k n o w l e d g e
placem ent
casual

conversation

the

financial

line

first

im ply t h a t
faculty

Sotting

in

the

of significant
from t h e

and p o s t- h o c
placem ent
of

stu d e n t's

members

the

Regression

students

for

of the

4.2 p resen ts

the

results

discussed

in

fo r Table

decision

as

a predic­

availability

the

the

Both

clearly

create

faculty.

student
the

the

indicate

a base­
This

"ought

would
the
to be".

level,

evidence

triadic-com parison

rating

scores

com m ittee

8 o f the
o f the

at

the

meaning o f

faculty.

to

the

to

Conm ittce were

im plied

used

above.

.05

adm ission

found

m entioned

p e r f o r m a n c e ma y b o j u d g e d b y

ANOVA F - t e s t

was

five

ra te r questioning
is

set

previously

data

its

and t h e

com parison w ith where

prediction

other

in

Admission

decision

actual

com bination o f th e

adm ission

the

known a n d u n d e r s t o o d b y

im pression
the

o f the

the departm ent

apparently

that

Table

in

the

an u n ­

com m ittee d e c is io n

actual

v a r i a b l e o f g r a d u a t e p e r f o r m a n c e was

faculty

w ith

con­

(1:54).

predictor variable

reason

when s t a n d i n g

m ultiple

the placem ent

a sixth

principal

tor

to

11 c r i t e r i o n

added as

The

the

resulting

In p r e d i c t i n g
each

to

decision

performance

analysis

in

the

and the

categories.
same m an ner

4.1.*

* A ctual re g re s s io n c o e f f i c i e n t s , b e ta w e ig h ts, p a r t i a l c o r r e l a ­
t i o n s a n d s i g n i f i c a n c e l e v e l s may b e f o u n d i n A p p e n d i x B.

68

TABLE 4 . 2
M ultiple

Regression

Results

Performance

-

Prediction

o f Triadic-C om parison

G r a d u a t e GPA
M astery o f

fundam ental

M astery o f re le v a n t
O riginality

in

knowledge

research

of w ritten

E ffectiveness

of oral

os

a productive

Promise

as

a teacher

O verall

Evaluation

of prediction
dictor

the
is

reports

reports

Promise

Although

data

research worker

in Table

evident,

criterion

o f these

results

with

was a d d e d t o

the

i m p r o v e m e n t was p o s s i b l e .
relative

worth

o f the

the

This

member o f t h e
statem ents

and r a t e

and m a rg in a l.
variable,

adm ission

seventh

a stratified

in

the

c o m m i t t e e was

to maximize
sample o f

. 34

.12

P <.02

. 37

.14

P <.02

. 39

. 15

P <.01

.43

.18

P <.003

. 36

.13

P <.03

. 35

.13

P <.04

.45

.21

P <.001

. 39

.15

P <.01

that

a high

i s weak.
committee

v a r i a b l e was

statem ent

degree
the p re ­

A fter

a dis­

a seventh

a measure of

of goals

and i n t e r ­

fo llo w in g manner.
asked

th e m on a t h r e e - p o i n t

In o r d e r

P <.0005

to d eterm ine w hether c o r r e l a ti o n

c a n d id a te 's

The m e a s u r e was o b t a i n e d

.25

r e la tio n s h ip between

adm ission

r ^

a

. SO

variables

equation

C .^

indicates

linear

and t h e

variable

ests.

the

4.2

variables

cussion

the

sk ills

research

E ffectiveness

Regression
ANOVA

R

Category

R esults

to

read a sample o f

scale--superior,

the e f f e c t

50 s t a t e m e n t s

Each

of the

average

statem ent

wa s u s e d .

The s a m p le

69
consisted

of

those

subjects

triadic-com parison
remaining

sample

split)

dichotomy
tical

to

tests,

ences

these

this

the

actual

coefficient
the

significance
be the

case

the a d d itio n
from t h e

in

coefficient

m ultiple

found.

in

the

statem ent

equation.

correlations
o f no

the

sta tis­

f o r mean d i f f e r ­
differences

of w ritten
(N = 5 0 )

for

was

total

v a r i a b l e was

As t h i s

also

de­

correla­
sample

o f no

proved

to

11 c r i t e r i o n

im portance

con­

to

m ultiple

the

on

reports.

predictors

the

for a ll

a

a n d 24 m a r g i n a l

sample

of that

in

conducting

5 adm issions

range

was

in

While

f o r each

placed

significant

for the

of the

variable

raters,

effectiveness

the

( a 1/3-1/3-1/3

The i n t e r - r a t e r

of variance

decisions.

prediction

to Table 4 .2 ,

of the

used to

what

least-squares

(R) w o u l d

indicate

predict

o th e r hand,

question

from th e

and was

sets,

dropped

study.

results

On t h e

the

conducted

statem ents

resulted

analysis

w ithin

was

sample s iz e

analysis

adm issions

the
the

in

selected

group,

three

26 a c c e p t a b l e

p r e d i c t o r w ith

was

were

the

and n a t u r a l l y ,

of this

R eturning

could be

in

contribution

to

of

sufficient

regression

using

(R = . 5 2 ) ,

differences

two groups

score

A m ultiple

term ine

for

o f each

of goodness-of-fit

The m u l t i v a r i a t e

between

ducted

frequency

A com posite

resulted

t h e GRI: v e r b a l

the

test

.79.
allow

statem ents.

tion

expected

and no s i g n i f i c a n t

a g r e e m e n t was

"anchor p o in ts"

(N = 1 7 ) .

a chi-square

rater

as

(N = 3 3 ) a n d a r a n d o m s a m p l e

set

Based on t h e

selected

the

of predictive

conclusions
analysis?

that

a

the d e c isio n

coefficient
accuracy.

can

now b e d r a w n

The m u l t i p l e

lin e a r com bination
of the

adm ission

o f determ ination
It

is

also

evident

from

correlation
of variables

com m ittee.

2
(R ) r a i s e s
that

in

all

the

70
but

two o f

However,
for

the

the

criterion

data

from t h e

mean d i f f e r e n c e s

cant

prediction

given

that

all

sets

is

lead

m ultivariate
the

always

For those

among g r o u p s

exist,

Furtherm ore,

are

are

is

these

questions

R eliability
Pi f f e r c n c c s

of

In o r d e r
com parative
The

actual

to

answer th e
which

decision

categories

of

were

results

in

of the

combined
an

aspect

that

actual

and e x p e c t e d

clusive

groups.

scale

was

this

divided

into

U nfortunately,

the

of

the

to

sample

prediction

and

analysis

Equations

create

on

break

problem ,

groups.
bulk

chance

in

the

of

on

the

Basis

the

must

com m ittee

four

for

regression

these

results?

be­

It

Group

as

of a

be

resolved.

provided
the

the

us w i t h

bottom

fifth

places

both

into

parts

to

therefore,

an

four m utually ex­

triadic-com parison

achieve

were h ig h ly

m iddle

two

cate­

analysis

four equal

the

placem ent

The

point

in

group placem ent?

group.

seven

located

the

differences

of

first

scales,

scales

fo u r groups

in

thut

the

the

place

problem

has

groups

size

continuous

a sig n ifi­

categories

group

actual

test

now t u r n s .

groups

adm ission

that

same

For measurement purposes

scores

must b e made to

the

significant

above q u e s tio n s ,

m ultiple

attem pt

at

occurred.

variance

conclude

differences

N of questionable

second

To r e s o l v e

the

delineates

subjects.

is

the

P rediction

design

5 categories

gory

the

that

by

to

of

has

some p e r f o r m a n c e

where

is

significant

what would be e x p e c te d

to

in

categories

tw een

analysis

essentially

how a c c u r a t e

there

prediction

investigator

probable

g ro u p means

distribution.

significant

four

distinct

unbalanced w ith

two g r o u p s .

Frankly,

71
this

should have been

solute

scale

trying

to

w ith

get

regression
standard

the

logical

on

the

deviation

for

deviation

above

the

placed

mean.
below
on

deviation

the

the

given

the

to

action

was

and

scale

through
For

greater

than

located

t h e mean a n d t h r o u g h

mean t o
located

one-half
at

a more

1 presents

than

to b u ild

Locations

1

on

the

one-half

moan.

deviation

of

the

3

< ------- * ■

1/2

SD

x

i
i
i
SD

FIGURE 4 . 1

standard

Group
below

group

Scale

4

*

*

1/2

the

between o n e - h a l f

Continuous

2

the

perform ance

scale.

Group

by

o n e -h a lf standard

a diagram

is

mean a n d

each

the

standard

ab­

w hich

provided
the

set.

Group 2 was

an

possible!

therefore

continuous

of

technique

t h e mean a s

achieved

at

com bination

a measurement

criterion

placed

Figure

continuous

was

each

above

4 was

mean.

o f the

mean.

from th e

Group

close

This

1 was

of

course o f

analysis.

group

was

as

basis

category,

standard

results

everyone

The o n l y
four groups

anticipated

SD
>
*
i

y

3
the

deviation

locations

72
The d e l i n e a t i o n

o f groups

a more even d i s t r i b u t i o n
the

purpose o f

ing

subjects

this

this

at

of

analysis

the

through

is

to

appropriate

decision

to

presents

the

the

predicted

above method

th e number o f s u b j e c t s
determ ine

point

method o f g r o u p d e l i n e a t i o n
Table 4 .3

the

in

appears

relationship

decision

in

the

the

achieves

p e r group.

As

accuracy o f p la c ­

distribution,

then

reasonable.
of

the

term s

actual

of the

adm issions

number o f

subjects

p e r group.

TABLE 4 . 3
Comparison o f Admission

A c t u a l Group*
ic e m e n t

Group P la c e m e n ts

Predicted
1
2

Placem ent
4

N

1

17

20

4

2

43

2

5

11

9

4

29

3

2

8

4

7

21

4

1

1

3

7

12

N

25

40

20

20

105

Number o f

Returning
that

Group
3

the

decision

to

the purposes

of this

study,

Subjects

it

w ill

development o f a param orphic r e p r e s e n ta tio n
is

delineation

one o f
of

the prim ary

concerns.

a num eric e x p re ssio n

This w i l l

sim ulating

the

p e r Group

be r e c a l l e d
o f the

require

adm ission
the

adm ission process

* Tliis d o e s n o t r e p r e s e n t t h e d e c i s i o n c a t e g o r i e s 1 - 5 p e r s e , t h e
numbers r e p r e s e n t w h e re t h e s t u d e n t w o u ld b e p l a c e d on t h e co n ­
tin u o u s s c a l e (see F ig u re 4 . 1 ) as t h e r e s u l t o f t h e ad m is sio n s
d ecisio n placem ent.

73
decision

fo r each

show t h a t
of

the

this

subjects

creating

an

assum ption

dent

from the

adm ission
design

The m a r g i n a l

to be

would p l a c e

the

at

param orphic

is

is

student
1 or

opinion

the

and i t s

2 means

placem ent
groups

that

f o r which

"successful"

ought

can be

assistance

and th o se

(3 an d

4)

can be

reconstructed

o f each

in

as

to be.

4.3

four group

The p e r c e n t
.37.

It

only

of

would

appear

statem ent e v i­

improve t h e

a re-evaluation

consequent

effect

ns

Given

as

recommended
f o r which

"m arginal".

a dichotomous

either

that

of

the

on g r o u p

the

to

a dichotom y.
(1 a n d

On t h i s

successful

the

the kind o f

granted,

2)

assistance

table

represents

a placem ent

should be

interpreted
is

candidate

co m m ittee as

assistance

groups

can bo c l a s s e d

candidate

adm ission

financial

categories

Table

necessary.

of the

candidate

the

the

In o r d e r t o

The a d m is s io n p la c e m e n t on any g iv e n
collective

is

representation,

committee placem ent

possibilities

diagonal

tim e.

at

totals.

agreement

in

Complete agreem ent

subjects

m arginal

this

totals

reached.

all

intercept

o f minimal

analysis

o f the

yet

identical

l o c a t e d on

that

case.

p o in t has

two d e c i s i o n s

intercepts

quality

subject

in

group

then

the

These

can be

is

not

basis,

classed

recownended

Table

4.3

b a s e d on t h e p l a c e m e n t
or m arginal

category.

D i c h o t o m i z e d Group R e l a t i o n s h i p Between
A c tu a l and P r e d i c t e d A dm ission D e c i s i o n P lacem en t

Predi cted
Success ful

Predi cted
M arginal

Actual
SO

18

12

20

Success ful

A ctual
M arginal
Percentage

The e x t e n t

of

agreement

is

.70.

To d e t e r m i n e

as

a measure o f p r e d i c t i v e

differences
of

the

to

right

the

was

This
puted

using

the
the

reliability,

in

As p e r f e c t

(N « 1 1 8 )

diagonal;

being

significance

conducted.

population

population

the

thus,
each

case

the

(summating

in

on t h e

Z-test

for the

formula:

left

o f the

in

to the

extent o f

a 2 test

to

extent

right

diagonal)

agreement

would p la c e

two c a t e g o r i e s

left

Group

of proportional

reliab ility

distribution

category

follow ing

the

one o f th e

a chance

o f Subjects

on t h e

all
left

would b e h a l f o f
right
of

diagonal.

a g r e e m e n t was

com­

7S
w here:
Pj

= the

sum o f t h e

agreement

proportions

(left

= the

sum o f t h e

differing

proportions

(right

P2 =

to

right

diagonal)

left

diagonal)

to

* 50

q 2 = . SO
Nj = S a m p l e n
N2 = P o p u l a t i o n
The

statistical

R esults
extent
p

N

of

<.00S.

of the

agreement

adm ission

the

dichotom izntion
a question

1)

a strong
2)

data

set

the

param orphic

as

success-

equation

for

measurement

w ill

have

been

be

legitim ate
in

to

the

this

follow ing
point,

the

alterna­

pages.
follow ­

made:

correlation
the

a

explored

presented

coefficient

adm ission

(.56)

com m ittee

between

placem ent

the

indicates

relationship.
which

have

rating

of

the

an

im pact

coranittee

in

the

decision

undergraduate

param orphic
are

institution,

GRE v e r b a l
and

GPA

coursework.

The p r e d i c t i v e

reliab ility

agreem ent between
is

the

into

expectation

as

score,

decision

a chance

the

ju stified

adm ission

placem ent

than

show

be

of the

J)

differences

placem ent

of

p<.05

W hether o r not

The p r e d i c t o r s

sociology

at

significantly.

representation

in

set

increased

and s i g n i f i c a n t

quality

better

adm ission

analysis

and

was

has

which

The m u l t i p l e

adm ission

far

reliability

can

the

determ inations

the

the

level

for proportional

being

decision

Summarizing
ing

as

dichotom y,

the

is

Z tost

By c o n v e r t i n g

m arginal

tive

significance

.70.

the

defined

predicted

as

a percentage

decision

and t h e

o f group
actual

76
Predictive

Accuracy

One o f
the

the p u rp o s e s

adm ission

data

perform ance.
tion

in

B a s e d on D i c h o t o m i z e d G r o u p s

set

Using

term s,

of this

study

is

in p re d ic tin g

the

dichotomous

table

the

an a t t e m p t

w ill

be

to

determ ine

level

made t o

of

the

graduate

value

of

program

format

w ith

some m o d i f i c a ­

see

this

purpose

if

can be

accom plished.
In

Chapter

III,

a measure o f the
scale

for

outlined
those

the

fa c u lty 's

Figure

4.1.

g r o u p s which

fall

in

successful"

(3 and 4 ) w i l l

be

on t h e

follow ing p ag es.

in

of variance

has

in

added

point.

test

Table

prediction

indicates

g r o u p s which

4.1,
has

that

the o v e ra ll

category

by the

faculty

in

however,

To t h e s e

for

reasons
the

fall

triadic

decision

adm issions

manner
labels,

labeled
the

mean

The p r e d i c t i o n

the

tables

presented

p r e s e n t e d which

all
the

be

but

m ultivariate

group
four
to be

analysis

differences

the

are

investigator

discussed

variables

showed

2 performance

at

of actual

com parison w ith

com bining the 5 p r e d i c t o r
com m ittee.

the

below

relationship

group placem ent
o f the

in

2) w i l l

e v i d e n c e was

4 .8 show

on t h e

used in

significant

categories.

through

be

occurred

only f o u r

4.5

(1 a n d

as

The c o n t i n u o u s

successful-m arginal

mean

As m e n t i o n e d b e f o r e ,

Tables

placem ent

to

the

4.4 w ill

was p r e s e n t e d

be grouped

"perceived m arginal".

Table

R e fe rrin g back

categories.

above t h e

labeled

presented

significant

Employing

scale

of perform ance.

category w ill

and th o se

labels

present

perception

each p erfo rm a n c e

"perceived

that

triadic-com parison

a future
group

the p red icted

and t h e

placem ent

77

TABLE 4 . 5

D ic h o to m iz e d Group R e l a t i o n s h i p s Between A c tu a l
P re d ic te d Perform ance C ategory Placement: O r i g i n a l it y

and
in Research

Predicted
M arginal

Predi cted
Successful
P e r c e i ved
35

14

21

30

Successful

Perceived
M arginal
Percent o f

Subjects

in

Group

E x te n t o f Agreement = .65
Z test
p <.05

TABLE 4 . 6

D i c h o t o m i z e d Group R e l a t i o n s h i p s Between A c t u a l an d P r e d i c t e d
Performance Category Placem ent:
E ffe c tiv e n e s s o f W ritten Reports
Predicted
M arginal

Predicted
Successful
P e r c e i ved
37

17

16

30

Successful

Perceived
M arginal
Percent
E x te n t o f Agreement * .6 7
Z test
p <.05

o f Subjects

in

Group

78

TABLE 4 . 7

D i c h o t o m i z e d Group R e l a t i o n s h i p B etw een A c t u a l a n d P r e d i c t e d
formance Category Placem ent:
Prom ise as P ro d u ctiv e Research

Per­
Worker

Predicted
M arginal

Predi cted
Successful
Perceived
33

16

22

29

Successful

Perceived
M arginal
Percent o f Subjects
Extent

in

Group

o f Agreement = .62
p < .05

TABLE 4 . 8

D i c h o t o m iz e d G roup R e l a t i o n s h i p Between A c t u a l and
P re d ic te d Performance Category Placem ent: O v erall E v alu atio n
Predicted
M arginal

Predicted
Success ful
Perceived
30

15

21

34

Successful

Perceived
M arginal
Percent
E x t e n t o f A g r e e m e n t *= . 6 4
p <. 0 5

o f Subjects

in

Group

79
The d i c h o t o m i z e d
regarding
existing

the

tables

validity

of the

group d if f e r e n c e s
categories

to

data

adm ission

ment

regarding

seen

as being

this

dichotomy

in

a

function
as

the

4.5-4.83,

At t h e

to

create

The

proportion

for

a given

the

successful

predicted

of

students

.51.

W ithin
to

adm ission

and not

same t i m e ,

m arginal

probability
F inally,

the
is

of

.30
as

are

results

in

group

regression

differences.
prediction

in Table

perceived

successful.

this
4.8,

by

point.
overall

the

Therefore,

score places

regarding

The c o n v e r s e

from
him

X being

decision

estim ate

from t h e

him i n

probability

situation

a probability

for

of

the

estim ate

of

perform ance.

3/S

notion

best

3/5.

may p l a c e

the

cessful-m arginal

of

successful

group

performance

illustrate

a p p l i c a n t whose p r e d i c t e d

successful

scores

this

category

The d i f f e r e n c e
predictor

predicted

employ

reasons.

meaningful
to

agree­

to

student

obscure

dichotom y,

know t h e m b e s t ,

category

performance

regarding

to

in p a rt

strong

developed

fo llo w in g example s e rv e s

who c l a i m

successful

serves

a successful-m arginal

decreased.

faculty,

two c a t e g o r i e s

due

is

follow ing

fo u r groups

are

For the

of

fo r the

the

determ ining

extent

placem ent

chance.

evidence

the

a program would n o t be v a l i d
of

for

program.

graduate

evaluation,is

the

graduate

predict

By c r e a t i n g

2/5

equation

w h e r e mean d i f f e r e n c e s

(Tables

of

supportive

method t o

scales

The

the

successful-m arginal

The c o l l a p s i n g

is

set

additional

prediction

w ithin

fo u r performance
the

present

2/5
in

indicates

the

successful

of

the

for

appropriate
is

adm ission

this

th a t w hile

sasple

is

actual

classification,

approxim ately
decision

X 's

1/2.

being

seriously

a suc­

constrained

80
given
that

that
all

all

subjects

were

were p e rc e iv e d

adm itted.

by t h e

It

adm ission

must

then

com m ittee

be

as

assumed

capable

o f being s u c c e ssfu l.

Predictive

Accuracy

A more
are

in Terms o f Group P la c e m e n t

rational

p e r c e iv e d by

the

group placem ent

is

more s u c c e s s f u l

than

m ent by t h e
ject
or

others.

os

capable
o f the

Therefore,

process,

faculty--on

appear

as

an e x p r e s s i o n

perceived

the

differences
needed to

com nittcc

regression

x being

two by

assu m p tio n would be t o

what

is

assert

to e x is t?

belief
based
the

answer th a t q u e stio n

performance

The t a b l e
plus

that

w hile

all

of being s u c c e s s f u l, the
that

some w i l l

on t h e

placed

in

categories

below p r e s e n t s

equivalent

data

be

group p la c e ­

probability

successful--m eaning
those

P robabilities

for

of

sub­

group one
where
the

group

data

the

GPA

criterion.

TABLE 4 . 9

Probability

of

Perceived

Success

Based on G roup

Placement

P r e d i c t e d Group Placem ent
4
3
1
2
Actual

Group P la c e m e n t

O riginality

in

E ffectiveness

1 or

2

Research
o f W ritten

Prom ise

as

a Productive

O verall

Evaluation

G r a d u a t e GPA

is

Presentation
Research

Worker

.75

.61

.31

. 36

.89

.66

.41

.27

.80

.58

.36

.36

.67

.58

.36

.20

.88

.70

.48

. 33

Percent o f subjects per
g r o u p whom f a c u l t y p l a c e
in groups 1 o r 2

81
TTie t a b i c
the p re d ic te d
the

docs
group

student w ill

There
into

are

equation,

in

be

unlike

recognized as

other

categories

subjects
the

faculty

all

four

in

groups

have

distributes

the

fo r the

predicted-actual

into

O riginality
as

of these

and

in

in

sam ple

group

referred
is

to

as

the m a jo rity of
t h e me a n

categories

even manner
of

from

across

the

prob­

m a t r i x was c o n s t r u c t e d

the

between
three

E ffectiveness

Research

perfor­

o th e r hand,

relationship

Research,

relationships

w ith

the

the

1 on t h e

interpretation

of clarification

taken

on x t o

Therefore,

On t h e

correlation

that

A prediction

scores

a fairly

the

are

which must b e

the perform ance
in

in

a Productive
is

3.

higher

th a t placem ent.

table.

in

the

chances

ignored.

placed

placem ent

For purposes

one c a te g o r y which

The m a t r i x

2

a product-m oment

and P ro m ise

are)

of this

is

fact

To a s s i s t

table,

ports

the

awed by h i g h

received

ability

categories

in

that

s m a ll--th e average N = 7,

categories.

categories.

not

better

always

inform ation

is

subjects

the

is

who a r e

found

the b e l i e f

m aintaining

interpretation

humans,

number o f p e rso n s

score

support

placem ent, th e

the

p o in t where a l l

the

fact

some c o n s t r a i n t s ( t h e r e

account

mance

in

relevant

performance

o f W ritten

Re­

Worker were combined
the

presented

research
in Table

category.
4.10.

82

TABLE 4 . 1 0

R e l a t i o n s h i o p o f A g r e e m e n t on Group
Placement Across Perform ance C a teg o ries
Research
Category

Admission
Deci s i o n

Overal 1
E valuation

Admission D e c isio n

Research
O verall

Category

.56

Evaluation

.44

84

. 38

.44

Graduate
All

GPA

correlations

are

significant

at

.53

p<.01

The P r o b l e m o f A f f e c t
Focusing
appears

that

performance
tablished
between

the

on

strongest

categories

evaluation

where

adm ission

to

the

The s t r e n g t h
performance

decision

over-all

performance

b ased on

formance

category

as c r e d i b l e

in

the

overall

sound re a so n

evaluation

for this

In C h a p t e r
design o f th is

III

be

D ecision

column,

achieved

in

the

have b ee n

es­

of

and o v e ra ll

between o v e ra ll

of

the

an a p p r o a c h
There

that

on

research

as a c t u a l
in

evaluation

inference

combined

is,

three

correlation

category

indicates

a review

category.

the

it

per­

prediction

fact,

a rather

phenomenon.

one o f

s t u d y was

academic perform ance

can

relationship

placem ent

is

Admission

group d iffe re n c e s

combined r e s e a r c h
compared

the

prediction

by o t h e r m e a su re s.

the

and t h e

our attention

the

that

item s,

principles
in

order to

personality

m entioned
insure

related

for

the
item s

guiding th e

accuracy of
m ust be i n c l u d e d

83
to

serve

as b u f f e r s .

in

the

an

"affective
role.

gory

and

variate

its

this

category
at

As t h i s

sponge"

located

at

is

not

accounted

the

.05

level

derivation

the

is

of the

The p e rfo rm a n c e
is

that

conducted

categories,
overall

therefore

categories

the

as

the p a r t

presented

of

in

any o t h e r

industry

are

level.

is

levels

of the

obtained

to be s t r o n g e r in

Further

support

Table
that

and e x p e c t e d .

4.10.

and

F inally,

the

as

the

is

category
input

a buffer
tests

in o t h e r
(.44)

on th e

R ater

agree­

o th e r performance

the placem ent

socio-psychological

category.

assum ption

category

the

one

different

expected.

o f Table 4.2

step-

signifi­

assumed

bettor

evaluation

the

measurem ent,

obtained

fo r the

group

was

correlation
than

All

somewhat a p p r o p r i a t e .

to have

R esults

univariate

categories,

sponge" is
appears

the

measured in

intuitive

cate­

m ulti­

But,

category,

assumed

overall

test

as

assumed

the

and a p p l i c a t i o n

the p re d ic tio n

the

In

scale.

performance

"affective

intraclass

levels

evaluation--im plies
w arranted

be

.9S

the

d e p e n d e n t on

category,

analyzed.

significant

ships

the

on

evaluation.

that

evaluation

can

role

given

in

c a te g o r y which

of overall

indicate

for

significant

phrase

the

provided

have

rem arkable.

at

included

and a p p l i c a t i o n

h ow muc h o f w h a t

fo r the

other

industry

same p o i n t

heavily

to also

f o r mean d i f f e r e n c e s

for determ ining

category

the

categories

specifically

appears

significant

the

was

was q u i t e

o f variance

c a t e g o r y was

compared to

ment

performance

and a n o t h e r

effectiveness

analysis

down F t e s t ,

role

ten

The o n e p r o v i d e d was

means w ere

cant

the

t r i a d i c - c o m p a r i s o n , one s c a l e

this

F for

In

of a buffer

evident

in

interrelation­
nam e--overall

factors

are

both

84
G r a d u a t e GPA a s

a Performance

In C h a p te r
as e i t h e r
were
was

the

sole

reviewed.
the

II,

studies

p red ic to r set.

ship

present

addition

the

adm ission

To
data

in

decision

given
this

results

of the

group

delineate
given

representing

mean d i f f e r e n c e s

sub-group

conflicting

same m a n n e r a s T a b l e s

.26,

that

the

the

1 and g r o u p 4.

inform ation

presents
point.

4.5-4.8,
groups.
one

studies

G r a d u a t e GPA

R =

relation­
.50.

In

b a s e d on t h e

relationship

is

t h e g r a d u a t e GPA e a r n e d

<.01).

problem o f range

grade

that

(p

GPA
set

these

a strong

test,

and

regarding

existence

4.2 in d ic a te s

in

between

placement

actual

in

relationship

adm ission

grade p o in t

to

problem s

a strong

Table 4.11

deviation

a criterion

show t h a t

sample

to

in

placem ent,

the

point

a factor

re-occurring

Table

G r a d u a t e GPA.

is

as

the G ra d u a te

the D epartm ent o f S o c io lo g y with

results

p re s e n t between
f o r any

or

absence o f a s ig n if i c a n t

and t h e
is

w hich had em ployed

criterion

One o f t h e

C riterion

the

has been

restriction

relationship

The

table

using th e

is

minimum a b s o l u t e

of

the

for the
predicted

constructed

mean and t h e

The r e s t r i c t i o n
standard

found in

in

the

standard

o f r a n g e phenomenon

deviation
difference

from

t h e me a n i s

possible

between

85

TABLE 4 . 1 1

D ichotom ized Group

R e l a t i o n s h i p Between A c tu a l
G r a d u a t e GPA

and P r e d i c t e d

Predicted
Successful

Predicted
M arginal

44

17

14

25

Perceived
Success ful

Perceived
M arginal
Percent
Extent

4.8

reveals

grade point
predictors
coefficient
tomy,

ability

this

that

above t h e

is

in

the

same m a n n e r a s

61 p e r c e n t o f t h e
sam ple mean.

s u p p o r t e d by t h e
predicted

successful
of

table

Group

subjects

employed

have

The r e l a t i o n s h i p

When t h e

successful

whereas

.40 o f b e i n g
groups

variance

as

in Table

formance

is

quite

subjects

again

for

4.9,

this

have

predicted

the

correlation

for

a probability

differentiated

the p r o b a b il it y
group

m arginal

on t h e

the
of

dicho­
.80

have

department

the

appears

basis

a prob­

o f mean

range o f s u c c e s s fu l

1 <= . 8 8 a n d g r o u p 4 "

B ased on t h e e v i d e n c e p r e s e n t e d ,
a criterion

probability

a

successful.

are

large;

success

for

earned

between

a n d G r a d u a t e GPA e x p r e s s e d b y t h e m u l t i p l e

Subjects

o f being

as

in

o f Agreement e .69
p <.001

A breakdown o f
Table

o f Subjects

use o f

the

ju stified .

per­

.33.
G r a d u a t e GPA
However,

the

86
data p resen te d
the

in

Table

group placem ent

and t h e

clearly

o f the

m arginal

regarding

agreement

group placem ent

category
tion

4.10

intercept

agreement

indicates

performance

this

is

categories

relationship.
across

the

sim ilar

to

G r a d u a t e GPA r a n g e

conclusion
of actual
as

that

faculty

grading

a performance

measures

that

There
garding
device

for

dealing
their

effect

of

o f the
Tliis

is

not

a review

the

all

case.

the

the

4.9).

the

This

to

table

for

of

are

summed a c r o s s

the

actual

below

the

collapsing

g ro u p in g which
o f the

creates

adm ission

of
an

be

to

G raduate
that

the

a function
GPA

alternative

the

and

t h e i r w orth

actual

the

P rior

decisions

re­

as
to

and

G raduate

indicates

Program
performance
the

m ajority

c a t e g o r y mean.

groups

into

would

a suc­

distribution.

indicate

that

this

case.

R ecalling

that

approxim ately

75 p e r c e n t

a

appropriate.

artificial

practices

answered y et

evaluation

rows

the

not

candidates.

on th e

the o v e r a ll

scores

leads

is

available.

degree

D ecision

probability

to

given

equations

a discussion

GPA s h o w a

categories

use o f

w hich

of

intercorrela­

the

appear

are

questions

adm ission

of

performance

docs n o t

prediction

The

G raduate

range

(Table

category

on a n o t h e r p e r f o r m a n c e

not w arranted,

Admission

received

cessful-m arginal
However,

the

dichotom ous

may b e d u e

the

with

between

one perform an ce

p r o g r a m p o p u l a t i o n w o u ld be

Actual

subjects

not

more v a l i d i t y

o f the

the percentage

fact

is

questions

on t h e

on

Therefore,

some i m p o r t a n t

these

the

In t h e
category,

criterion

determ ining

w ith

E ffects

practices.

strength

for

perception

dem onstrate

are

the

fo u r groups

relationship

intercept

Furtherm ore,

factors

the

the

of all

applicants

were

87
adm itted because
act

as

this

the

only

the

total
this

for

control

b e l i e f would

can

of

the

cover

appear

applicant

three

com m ittee

believed

r e g a r d i n g who a c t u a l l y

three

study.

the

adm ission

to be

years
pool

Table

order.

o f the

for

4.12

year period

in

came,

presents

period

destroyed

the

would

an e v a l u a t i o n

U nfortunately,

four y ear

196 7 was

funding

in

this

prior

adm ission

the

to

of
review

study

the

placem ent

as

start
data

1968-1970.

TABU: 4 . 1 2

A ctual

Group P lacem en t o f

an A d m i t t e d

Adm itted

Student

Subsample

U nrolled

Percent of
Adm itted
Who U n r o l l e d

Group
Placem ent *
1

29

9

31

2

58

16

28

3

66

19

29

4

61

14

23

N

214

58

27

It
not

is

noted

appear to

This

is

from th e

recorded

guaranteed.

*

the

correct

e x p la in e d by

gathered
both

be

that

total
given

the

fact

running

This

being

that
that

tally

and m anaged t h e
the

number o f

we h a v e
the

data

f o r each

tally

case,

enrolled

the

only

A c tu a l P lacem ent by th e adm ission
j e c t s dropped from th e sam ple.

n sample

of

fo r Table

year

sheets

applicants

the

and as

does

105 s t u d e n t s .

4 .1 2 were
many

actual

individuals

com mittee w ith

individuals

count

is

not

included

group

5 sub­

flR
from t h i s

sheet were

s h e e t was

in

folder.
in

the

ten

m issing y e a r,

discussion,
actual
Table
and t h a t
for

com plete

Furtherm ore,

subjects

the

all

were

4.12

the

Therefore,
to be

is

as

for Table

37 w h ic h w o u l d

inaccurately.

of prim ary
that

which

all

adm itted

financial

A check

o f the

indicate

adm itted

that
of

only
this

rather

than

candidates

the

is

do n o t

alm ost

lower groups

the

actual

assistance

adm ission
enroll

records

that

contain

does

two

not

appear

com m ittee had presumed.

what was

for th a t

these

enroll

equal

stu d en t population.

of

who d i d

students

their

means

the

as

in

tally

sim ilarity
the

students

actually

enroll

since

in

a control

the

interest.

U nfortunately,

availability

on

presents,

g r o u p w h ich do

percentage

set

For purposes

it

over-represented

strong

1971?

is

picture

indicates

this

data

a p p ro x im a te number o f

percentage per

For those
Fall

the

four groups.

are

the

recorded

count,

adm ission

ag reem en t w ith what was

1967,

however,

more s t u d e n t s ,
groups

th o s e whose

their

status

term provides

the

as

of

data

4.13.
T ABU; 4 . 1 3
Current

Status

Group
P l a c e m e n t * Number

o f S u b s a m p l e Who E n r o l l e d ,
(see Table 4.12)

enrolled

D o c t o r a t e No L o n g e r
Completed
Fnrolled

Fall

1971

Percent
Enrolled or

1

9

2

1

6

33

2

16

11

0

5

69

3

19

11

0

8

58

4

14

10

0

4

71

N

58

34

1

23

60

*

See

footnote

p.

87.

S till
Completed

89
The p r o b l e m w i t h
t h e wrong g ro u p o f
students

in

group

re v e rte d back

to

cost

of education

departm ent
ing

did

While

mcnt

technique,

th o most

decision
53 are
4.14
ful

itself.

and

seen

the
as

presents

w ith

in

the

adm its

and

above

the

versa)

overall

these

students

awarded

a doctorate

such

to

at

Given

awards,

due

the
hold­

as

can

reason

to

believe

artifact

of

the

be

found

is

74.

m arginal

o f placem ent

m easure

in

the

actual

Of t h i s

group

adm its.

Table

shift

(success­

b a s e d on a c o m p a r i s o n o f t h e

evaluation

the

students

on whom b o t h

agree
21

give

by an

argument

t h e number and p e r c e n t a g e
and v ic e

o f the

19(>7-1970 p e r i o d .

underrated

decision

cases

enrolled.

$0 , 0 0 0

The num ber o f s u b j e c t s

param orphic

ten

accompanies

convincing

some

com pleted

presented

fact

successful

to m arginal

decision

not

last

currently

during

two a r g u m e n ts
is

and in

Of the

arc

column,

F u r t h e r m o r e , most

an c s t i m a i c d

w ithdraw ing

tho sam ple

sample

three

receive

the p e rc e n ta g e

fellow ships

none o f them h a v e

that

the

in

leaving!

a l lo tm e n t which

not

the

is

U niversity.

and o n ly

fellow ships

is

1 w ore on

a n NDEA f e l l o w s h i p ,
departm ent

tabic

students

the

this

this

adm ission

category.

TABLE 4 . 1 4
Admission

Admit

Placement

Percent

Shifts

o f Group

Percent

of

to

M arginal

Evaluation

M arginal

Admit

Successful

Performance

Number

Shift
Successful

-

21

40

20

to

Evaluation

6

6

Sample

90
From t h i s

strong

assum ption
overall

o f the

All

perception
regard".

the

of

sample

negative

Comparison
It

current

a direct

be
of

these

P rediction

contention

the

representation
the

w ill

proceeding

predictive

accuracy

agreem ent between
shown t o
faculty

is--to

superior
weakest

w ill

one o f

w hile

of the
is

in

is

the

not

faculty

"minimum h i g h

this

perception

adm ission

clearly

be

that

practices,

evident.

a param orphic

can be u s e d

applicant.

to

more a c c u r a t e
of

the

repre­

predict

the

Furtherm ore,

actual
than

to

significant

been

than

the

this

actual

d e c isio n w ith

the

either

decision

of

probability

actual

decision

and

current

performance

is

the

adm ission

category--in

and

and t h e

and

The e x t e n t

predicted

decision

o f agreement

group placem ent has proved

An i m p o r t a n t
performance
practices?
term s

categories

analyzed:

probability

and p r e d i c t e d

reliable.

extent

The

the

performance

discussed

be s i g n i f i c a n t .

what

the

the

the d e te rm in a tio n

in

has

perception

be s ig n if ic a n t

swered

study

o f the

pages

its

to

that

the

the

alone.

performance

between

perception

that

fact,

be more a c c u r a t e

successful

been

sample

dichotom ization

indicate
in

the

Procedures

com bination

for

has

the

practices

decision

status

in

faculty

result

of this

adm issions

and th a t

In t h e

of

is,

ma y n o t

pred icto r variables
standing

sample

can bo draw n

performance

decision

of

conclusion

the

param orphic

o f m arginal

ev id en ce would

influence

of

n downward s h i f t

a result

student

o f Three

is

sentation

being

o f tho

the

The

of

prevalence

evaluation

supported.

o f the

indication

question
category

still

approach

To a n s w e r t h i s ,

of sig n ifican t

unan­

group

the

91
diffcren ces--o v eral1 evaluation,
dictive

strength

of the

w ill

be

used to

follow ing p re d ic tio n

1)

the

actual

placem ent

2)

the

param orphic placem ent d e c isio n

3)

the

performance

compare

the pre­

procedures:

decision

decision

for

the overall

evaluation

category

TABLE 4 . I S

Comparison

of

Prediction

Placem ent

Paramorphic
Decision
Evaluation
Decision

Decision

Placement
Category

Tho e x t e n t

of agreement
placem ent

decision

from chance

from e a c h

other.

nor

is

both

significantly
as

significantly

different

confuted w ith

the

an i n t e r e s t i n g
tiation
all

assum ptions

three

*

2

3

4

.58

.58

.53

.48

.31

.59

.68

.45

.35

. 31

.64

.67

.58

.36

.31

actual

are

better

procedures

For d e s c r i p t i o n

the

of

placem ent

finding

category

than chance
techniques

this

test

itse lf
at

the

on g r o u p p l a c e m e n t
to

group

see

support

(group

page

and

different
extent

of

and
.05

level,

2-test. *

the

h a v e been made t h r o u g h o u t
baseline

decision

significantly

differences

o f success based

that

not

from the o t h e r

and i m p o r t a n t

P ro b ab ility of Perceived
as S u c c e s s f u l Based on
Group Placement

The e v a l u a t i o n

proportional

The p r o b a b i l i t y

E v a lu a tio n Category

1

for the

th e param orphic

agreement

f o r O verall

Successful-M arginal
E x te n t o f Agreement

Procedure

Actual

Procedures

74.

presents

group d iffe re n ­
the

study.

4) p r o b a b i l i t y

In
figure

92
is

identical.

procedure
is

not

which

A reading

indicates

that

significantly
the

m ajori ' .

the

actual

top

three

o f the

as

the

a true
the

of

first

than

chance,

both

procedures

groups w ith

category

defining

is

probability

becomes

the

Table

4.10,

d ic to r variables
decision

presented

itse lf

of

the
the

is
or

the

as

It

the

two

a group

groups

can be c o n c l u d e d
between

decision

does n o t

in
that
the

o th e r group ov er

groups

the

the

than

tim e.

in

fact

the

inference

better

the

Furtherm ore

adm ission

jump b e t w e e n g r o u p

other

in

Table

4.9

that

and
for

technique

the param orphic

significantly

between

adm issions

a superior

evaluation
w ithin

. SO p r o b a b i l i t y .

equation

Table

actual

is

and t h e

of success

Returning

obtained
in

decision

probability

noticeable

clear that

then

com bination

placem ent

representation.

inform ation

for

any

decision

success w ith in

differentiate

like

differentiate

somewhat b e t t e r

actual

decision.

chance

clearly

data

is

placem ent

a particularly

When t h e

in

does not

decision,

throe

param orphic
it

fall.

param orphic placem ent

each

figure,

decision

the

of

from chance

subjects

placem ent

category

the

row f o r

probability

adm ission

perform ance

the

the

g r o u p s - - a n y one g r o u p

In b o t h

in

the

different

placem ent

In o t h e r w o rd s,
function

across

to

the

1 and g ro u p

extent

the

two t e c h n i q u e s

considered.

4.15
the
all

is

correlational

either

representation

evaluation

compared w ith

categories
w ith
the
is

the

data

reviewed

the

pre­

adm ission

highly

plausible.

Summary o f R e s u l t s
1.
ferences

On t h e
test

applied

basis

of the

to the

m ultivariate

performance

analysis

category

data

2

o f agreement

fo r predicting

decision
to

placem ent

o f mean d i f ­
obtained

93
from th e

triadic

com parisons,

group d iffe re n c e s
those

categories

An e x a m p l e o f
w ritten

this

the

faculty

w riting

at

category

member w i l l

reflects
This

variatc-F

ratio

category

assum ption
as

of

the

subjects

should

be weak.

.79!

on t h e

extent

2.
indicates

of

at

can b e

factor

agreement

A review
that

oral
is

as

of

categories.
reports

probability
the

in

this

substantiated

the

Given

.63
seen

support

that

effective­

promise
few o f

have been observed

factor
this

of

point

this
is

a uni-

and a m u l t i v a r i a t e - F

ratio

in

o f the

a di chotomi z a t i o n

a teacher

c a t e g o r y which
of

the

uni-

The

s u p p o r t e d by t h e

differentiation
to

w ritten

category.

fewer s t i l l

o f 45 p e r c e n t
of

for

category.

and

that

su b je c t's

judgement

than

.03

further

yields

sample

and

of

a

an

.75.

o f the p re d ic tiv e

a prediction

raters.

The p r o b a b i l i t y

effectiveness

Evidence

prom ise

successful-agreem ent
overall

levels

the

significant

more

for

s u p p o r te d by a com parison o f t h e

taught

then

Further evidence

fo u r groups

opinion

data

actually

category

ratio

low er and th e

te ach er performance

rater

variate-F

is

fo r the

by a n y

a high

program.

much

the

of w ritten

graduate

is

teaching

is

to

performance

subm ission

there

significant

a com parison o f th e

the

significance

have

in

a sample o f

substantive

a productive

the

then

inferential

.33

seen

available

reviewed

assum ption

to

is

that

m easurable

have

in

presentation

data

are

effectiveness

require

requirem cnts,

opinion.

ness

courses

concluded

population

can be

and o r a l

some p o i n t

o f an o r a l

graduate

statem ent

effectiveness

fulfill

the

the

can be

where s u b s t a n t i v e

As m o s t g r a d u a t e
to

in

it

equation

com binations

consisting

o f the

available
actual

clearly

94
decision
tive,

and th e

5 p red ic to r variables

U n d e r g r a d u a t e GPA, S o c i o l o g y GPA,

Institution)

is

a param orphic
3.

superior to

actual

The p r e d i c t o r

of W ritten
O verall

adm ission

yield

a

fairly

faculty

as

o f U ndergraduate

decision

alone

decision

and

or

decision.

com bination o f the

Promise

Tho p r e d i c t o r

successful

as

to

actual

reliable
raters

estim ate

on t h e

in

of

the

four s ig n if i­

Research;

a Productive

by t h e

offering

of

TV*o o f t h e

faculty

student

three

1 above.
the

estim ate
defined
training

determ ine

population

listed

Research

E ffectiveness
Worker;

and

for the

adm ission

fo r determ ining
as perfo rm in g
program.

Psychology,

the

at

o f a given

the

four

now a p p e a r t o

o f performance

types

adm ission

a given
a high

data

provide

s u b je c t’s
level

of

purpose

on w hich

set

perceived
(based

four

study w ill

discussed
in

for the

in

point

num erical
of success--

the

graduate

D epartm ent

usefulness

of

some s i m i l a r i t y ,
focus.

can

conbination

probability

the

achieved.

graduate

a reliable

analysis

programs w ith

the

fo r the

ability--in

regarding

point

o f the

d if­

te c h n iq u e which

of categories

third

the

groups

have been

an e v a l u a t i o n

decision,

across

subject being

f o r each

the

procedure

significant

placem ent

I n C h a p t e r V,

probability
central

level

those

provides

above.

provides

the

also

assistance)

purposes

for

The v a l u e s
actual

across

financial

The t r i a d i c - c o m p a r i s o n
used to

com bination

the p ro b a b ility

performance c a te g o rie s

of

that

and Q u a li ty

categorios--O riginality

Reports;

ferentiation

w ith

the

GRE Q u a n t i t a ­

Evaluation.

4.

be

of

g r o u p p l a c e m e n t by t h e

performance

on t h e

either

representation

the p re d ic to r v a ria b le s

cant

(GRE V e r b a l ,

the
is

CHAPTER

THE

Cu r r e n t

Admission

The
is

one

sity.

DEPARTMENT O F

of

graduate

of

the

largest

In

the

four year
per

o f such

50 p e r

The a d m i s s i o n

procedures

the O ffice

identical

with

Sociology.
point

in

the

Department

where

w ithin

the

tions,

including

each
of

the

role

interest

w ill

be

adm itted
Across

applicant

hiring

interest

group

w ith

and

the

is

the

to

the
of

an a v e r a g e
600

whose
in

retention

of

This

In k e e p i n g w i t h

this

by w h ic h

groups,
95

the

faculty,

the

co-ordinatec
arc

D epartm ent
process

of

is

of

occurs

program

interest
func­

carried

on

an o r g a n i z a t i o n

department

serving

organizational

the

univer­
600

of departm ental

how many a p p l i c a n t s

the procedures

are

functions

areas

creates

the

of

The g r a d u a t e

o f nine

The b u l k

the

programs.

the d ecision-m aking

consists

Psychology

applicants,

Psychology

review ed.

of

w ithin

graduate

counterpart

chairm an o f

interest

functions

Chairman,

divisions.

determ ines

and

in

Department

From t h e s e

o f psychology.

o f a Dean.

each

review ed.

his

the

study,

Departm ent

of

Psychology

the

of m ini-departm ents
in

in th e

the A sso ciate

discipline

aren o f

of this

adm itted

The m a j o r d i f f e r e n c e

the

these

time sp an

functions

to

departm ental

y e a r were

of

the

at

in

students

y e a r were re v ie w e d .

approxim ately

through

PSYCHOLOGY

Procedures

adm ission

applications

V

to

their

structure,
area

applications

number o f

faculty

w ill

be

who w i l l

96
participate

in

seven.

W ithin

1967 t o

1970

m ajority

of

student

the
each

assistantships,

direct

of

means

usually
and

the

of

assistantships

group

on

the

depend

new s t u d e n t s .
departm ental

from

w ealthier

interest

by

A fter
Chairman
m ent,

those

The

to

types

top

interest

departm ent

support

grants

their

Those

of

the

also

to

graduate
and

groups

pool

includes

who h a v e

not

been

w ith

Chairman

research

candidates

provide

total

from

w ithout
teaching

of

those

new
students

selected

for

groups.

a scries

concerning

a decision

separate

adm its.

30 n e w

However,

groups

to

to

make o f f e r s
sc.

one

adm itted

training

per

from

students

form o f

the

ranges

number o f

in

requiring

funding

adm its

adm it

various

students
the

the

not

control

therefore

support

or

groups

departm ent

for

admit

fro m n o new

interest

support,

to

interest

ranged
the

independent

decision

to

interest

of discussions
the

adm ission

treat

areas

the

was

the

process

departm ent

made on t h e

in
as

the
one

basis

the

psychology
unit

of

and

the

rather

A ssociate
depart­
than

as

follow ing

inform ation:
(1)
cation
on

an

The a v e r a g e

w ithin

an

applicant
(2)

Each

interest

interest
grades

the

adm it

than

(3)

Ma n y o f

one

interest

in terests.

the

group

and

faculty

g r o u p was

were d is c u s s e d

undergraduate
adm it-not

number o f

to

felt

school

members

three

who

reviewed

and d i f f e r e n c e s

achieve

a consensual

that

used

it

attended

as

the

factors

of

an

opinion

decision.

GRE A p t i t u d e
in

appli­

arriving

test,
at

decision.
faculty

group

in

in

term s

the departm ent
of

their

are

teaching

active

in

more

and r e s e a r c h

97
(4)

Ma ny o f

tho

selection

of

com m ittee

members.

(5)

five

courses

Given

procedure
of

is

the

of

a....n

appropriate

Population

data

be

as

into

were

was

available
a sample

dic-com parison
the

actual

not

enrolled

of

procedure

number o f
to

from t h e
usable
W hile

the

clearly

the

areas

in

the

selection

of

guidance

from which
students

period

those

achieve

the

probability

adm itted

of

this

groups.

sample

Therefore,

aggregate

in

study

W hile

sufficient

combined.

least

yielded

the

com bination

size
the

was

at

was p r o b a b l e ,

com bination

seen

as

the

most

approach.

f o r whom a l l

predictor

Due t o

the

return

was

not

below)

it

or

im portance

the

procedure

rate

of

possible

re­

the

to

tria ­

determ ine

o f number o f

term s

procedure.

R esults

of

sets
below

students

147.

isolates
rating

students

The t r i a d i c - c o m p a r i s o n

(see

rating

Departm ent

O iairm an

rating

number o f

*

the

for

219.*

T riadic-com parison
in

the

graduate

size

Triadic-C om parison

members

for

group

Sample S iz e
of

in

and

design

a departm ental

m ethodological

and

program

easily

The p o p u la tio n

sulted

to

interest

number o f

sizes

groups

could not

groups

the

groups

sample

interest

their

measurement

interest

non-significant

others

for

developed,

the

o f some

students cross

the

were
the

of

packets

Psychology.

D epartm ent o f
returned

for

preconceived

f o r wh o m r a t i n g

For a d efin itio n

of

the

were d is tr ib u te d

a

A fter

Psychology
return

return

sets

one

were

of

of

letter
18

26 p e r c e n t .

o f 45 p e r c e n t ,

available

study p o pulation

68 f a c u l t y

follow -up

a total

rate

rate

to

see

page

provided

36.

the
a

98

l e g i t i m a t e sample s i z e
ment o f S o c i o l o g y ,

for th e

total population.

As in t h e

t h e a v e r a g e number o f r a t i n g s

Depart­

per each f a c u l t y

member was h i g h e r t h a n t h e e x p e c t e d number r e q u i r e d

for the

In

subjects

rated

sample

subject

the

per

psychology

each

rater

is

The a v e r a g e
was

3.

Of t h e

>1 r a t i n g .
sam ples
cant

of

Sociology,

based

score

a given

those

resulted

in

a

lack

of

from t h a t

Group

of

the

See page 62.

the
in

the

process

This,

- Financial
in

the

w ithin

concerning

no

sub-

sig nifi­

correlation,
than

As

in

one

the

between
in

a

of

.46

Departm ent

subjects

a substantial

score

and

turn,

the

w ill

expected

affect

e v a l u a t e d . **

A ssistance

Departm ent

procedures
the

two

coefficient

in

Departm ent in

the psychology

o f placem ent

actual

these

more

result

procedure being

Sociology

I I I , page 55,
coefficient.

the

category.

D ifferences

adm ission

assigned

type

of

w ith

category.

should

between

perform ance

At n o t i m e

**

received

intraclass

difference

received

yielded

reliab ility

perform ance

ratings

a n d 78

between

subjects

rater

each

categories

The

of significant

efficiency

See Chapter
correlation

level.
on

manner.

*

,05

discrepancies

The g r a d u a t e
differs

the

on

1 rating

differences

agreement

of

large

D eterm ination

the

at

the

predictive

received

10 p e r f o r m a n c e

number o f

the

69

of

evaluation

on num ber

for

subjects,

f o r mean

(N = 7 8 ) , *
overall

number o f

com pleted

l: - t e s t

of rate r

the

average

number o f r a t i n g s

differences

rating
for

The

the

lb.

147

on e a c h

measure

departm ent,

study.

of

Psychology

a highly
are

adm itted

com putation

significant

adm itted

group w hich

of the

candidates
exists

intraclass

99
in

the

sociology

placem ent
of

in

successful

evaluation
of

the

equal

to
of

justify
the
of

training

H ealth

at

contains

of

adm ission
determ inant

a

successful

the

establishm ent

adm ission

to

the

recorded

at

the

received

the

at

the

level

train in g

special

financial

tim e

assistance

for
It

arc

then t h e i r
level

not

who w e r e

year,

of

the

on

to

all

the

The

the

positions

basis

Psychology
C linical
o f M ental

graduate

clinical

study

in

interest

candidates.

who w e r e
faculty,
were

of

second

Institute

adm itted

clinical

and

attend
the

C linical

the

grant

a predictor

study,

candidates

These open

as

a N ational

of

The

departm ent.

otherw ise

determ ined

this

practice

those

the

of supporting

traineeship

expectation

grant.

of

study.

educationally

not

in

category,

opportunity

not w arranted.

the purpose
the

by

usefulness

is

the

equal

fund

who c o u l d

awards

was

to

assistance

from

adm its

designed

obtained

first
of

as

financial

received

had

a one y e a r

of

had

group

award

the

Thus

require

removed

period

Grant

end

be

students

se,

the

the

group

from

does
tim e

During

interest

the

of

to

<10 s t u d e n t s

Psychology.

ing

program.

the

the

a significant

a student

use

is

graduate

C linical

At

grant

these

per

the

T raining

to

that

sue)) d i f f e r e n c e s

not

classed

As

program.

Psychology

of

the

disadvantaged

academic m e rit

sub-sam ple

is

that

at

12 s t u d e n t s

and were

school.

perform ance

recalled

technique

sample had

opportunity

culturally

of

measure

be

received.

sub-sam ple

fellow ships

w ill

differences

was w h e t h e r o r

type

graduate

prediction

The o n l y

two s u b - s e t s

The

Departm ent

the

In o r d e r

first

Sociology

group

adm ission

and

It

perform ance w ithin

of

plausible

program .
of

the

process.

then

not

perform ­

were removed
used

to

100
fund

the

lack

of

next

support

distribution
interest
fore,

group

difference

for

applicants.

w ithin

this

sample

given

group would

cause

in

top

clinical

interest

on

adm ission

the
was

the

of

financial

employed

in

the

Departm ent

of

four

categories

representing

Psychology,

each

the

invalid

There­

from t h e

regarding

sample.

awarded

at

rem aining
assistance

subject

support

the

this

category.

assistance

that

an

increase

95 s u b j e c t s

four p o s s i b i l i t i e s

the

large

create

on t h e

As

of

could

removed

financial

collected

was b e l i e v e d

financial

sample.

one

the

s u b s a m p l c was

type

It

group

total

Inform ation
of

adm itted

the

the

point

of

was

the

in

the

were

placed

in

possibilities.

TABLE 5 . 1
Financial

Category

A ssistance

Financial

C ategories

A ssistance

1

fellow ship

2

teaching
ship

3

a lte rn a te fo r fellow ship
ass is t a n t s h i p

4

no su p p o rt

It
of

the

in

sire

is

evident

sample
of

receiving
ferences

receiving

the

com bined

support,
on

more,

these

being

placed

that

the

it

or

is

no s u p p o r t

assistant-

SB

or

highly

0

unbalanced w ith

assistance.

receiving

between

would

23

14

financial

5 predictors

the

research

table

groups

Number o f S u b j e c t s

traineeship

was h y p o t h e s i z e d

differences
in

the

or

Received

provide

support

that

these
an

category.

Given
w ith

the

groups

group

for

significant

not

mean d i f ­

do e x i s t .

explanation
If

difference

the

significant

85 p e r c e n t

the

F urther­
subject

differences

101
did

exist

cant

on t h e

differences

evaluated
for
to

this

in

a

that

point

differences

at

support

those

and

predictor

to

the

also
the
to

the

the

group

students

same

any

effect

of

financial

the

the

sociology
of

Hvidence

subjects

departm ent

faculty

assigned
to

perception

sig n ifi­

of

ability

2)

test

results
those

not

for

show no
students

receive

mean

differences

significant
who

support

received
on

the

5

is

of

above on

not

an

the

test

between

category.

support

for

mean

differences

10 p e r f o r m a n c e

differences

perform ance

are

variance

categories

those

groups

The h y p o t h e s e s

received

on

the

at

relating

predictor

and

rejected.

Therefore,

the

financial

acceptable

replacem ent

for

the

assis­

adm ission

category.

Psychology

replication

given

on t h e

adm ission

of

psychology

of

of

who d i d

significant

variables

Departm ent o f

ness

inability

between

as

in

category

the

groups

level

An e x a c t
the

Sociology,

o f variance

analysis

no

placem ent

categories

of

term s

1 and

level

in

perform ance
tance

perform ance

sig n ifi­

variables.

resulted
.05

those

that

program.

analysis

.10

was h y p o t h e s i z e d

Departm ent

in

in

it

on

the

category

the

The m u l t i v a r i a t e
com paring

is

placem ent

in

4 compared

exist

of the

hypothesis

The m u l t i v a r i a t e
(group

variables

also

analysis

adm ission

change

later

would

the

latter

m arginal

cantly
at

predictor

the

of the

Sociology
that

is

a group

decision

is

not

not

variables

of

utilized

However,
for

procedure

possible

placem ent

departm ent.

predictor

evaluation

in

the

adm itted
in

the

D epartm ent
subjects

adm ission

an e v a l u a t i o n

determ ining

em ployed

of

of
based

procedures

the

perform ance

in

effective­
in

the

102
program

and

a

retest

com plished

by

predictors

w ith

categories.
validity
the

adm ission,
sociology

at
the

shifts

predictor
The

independent

in

(Table

of

for

the

the

to

produce

as

expressed

significance,
for

a given

prior

on

that

perform ance

in

a com parison

ment

of

knowledge

of

tim e

used

m ultiple

performance

w ithout

would

establishing

of

an

to

w hich

by

from p r e d i c t o r

to

the

that

the

is

to

cannot

analysis

regarding

the

be

The

perform ance

for

of

of

this

differences
in

of predictive
of

variance

interpreted

group

differences

problem

categories

the

com bining

level

properly

of

the

stability

aid

analysis

existence

An e x a m p l e

in

of

unlikely.

study

regression

point

evident

in

score.

at

triadic-

group

the

category.

the

arc

highly

regression

five

predictive

of

expected

the

of

interpolate

this

the

ac­

perform ance

a measurement

category,

two o f

tim e

indicate
is

be

compare d i f f e r e n c e s

the

placem ent

over

interpretation

at

to

ten

triadic-com parison

4.14)

the measures

predictors

ability

probable

group

differences

reason

establish

differences

the

on

to

of

of

based

the

loss

is

still

of

the

it

score

can

on e a c h

alternative

tim e

analysis

score

expected

this

w ith

the

the

procedure

of

A lthough

differences

probability
of

actual

in

entry

com parison.

such

the

The u s e

of

the

a com parison

results

point

of

is

from

evident

the

D epart­

Sociology:

Hie

originality

in

correlation

of

.39 w i t h

The p ro m ise

as

a teacher

of

a predictive

,45 w i t h

knowledge

of

group

research
a

level

category

of predictive

category

resulted

significance

differences

resulted

it

is

level

in
of

therefore

in

a m ultiple

significance
a m ultiple
.001.
logical

<.01.

correlation

Given
to

no p r i o r

state

103
that

the

than

research

ences
mean

predictors

as

expressed

is

not

more

originality.

differences,

above

arc

by t h e
would

accurate
However,

originality

in

research

category

significant

at

the

level.

F -test
.63

for

the

prom ise

as

assessing
existence

analysis

that

accurate.

.001

the

m ultivariate

indicate

com pletely

for

the

The

statem ent

univariate

indicates
On t h e

a teacher

of

group

te a c h e r prom ise
o f group

differ­

variance

test

for

presented

F-test

for

the

differences

other hand,

category

is

the

point

the

univariate

significant

at

the

level!

line

If

all

os

indicated
prediction

ficant
starts,
this

the

subjects

as

variables

of

significance

in

research

in

the

prom ise

is

always

the

adm ission

data

D eterm ination

in

set

data

of

the

mean

same

in

a teacher

for

m easure
in

of

that

predicting
the

the

to

category,

Thus

-

of

the

Current

a

signi­
subject

The u s e
of

of

the

The

pre­

level

the o r ig i n a li ty
groups

are

accuracy

distribution
the

then

findings.
for

regression

o f where

strength

subject

that

u tility

Group D i f f e r e n c e s

test

the

on t h e

subjects.

m eaningless

distribution.

regarding

all

differences

indicates
the

as

same

possible--regardless

would r e s u l t

category

points

is

or near

an e v a l u a t i v e

on

different

m eaningful

at

end-point

category

dictor

ore

of

at
the

provides

predictor

variables.

S tatus

the

in

Program

The o p e r a t i o n a l i z a t i o n o f an independent measure o f group
d i f f e r e n c e s in th e p sych o lo gy sample was accom plished by p la c in g
each s u b je c t in one o f fo u r groups r e p r e s e n t in g
ment in the graduate program.

le v e l o f a c h ie v e ­

Table 5 .2 p r e s e n ts th ese grou p s.

104

TABU: 5 . 2

Subject

Grouping

Category

Based on C u r r e n t

Current

R eceived M asters
n o t to continue

4

W ithdrew
degree

those

subjects

of

these

factors

do

variables

there

ferences

com paring

is

1 to

level

set

had withdraw n
The

lack

of

from

to

at

received
the

significant

do r e s u l t
o f the

possibly

a p erso n 's
to

1971

research
and

3 and

11 p e r f o r m a n c e

between
subjects

im pact

of

for

conducted

categories.

the

With

the

two

program

were e v i d e n t

differences

at

the

on

groups

any o f

to

for

the

the

the

mean d i f ­

differences

and

a

to w ithdraw

.1 0 no s i g n i f i c a n t
degree

that

tan talizin g

test

4 was

that

indicate

the

too

a

those

decision

of variance

groups

and
to

in

idea

exist

evidence

im portant

analysis

and t h e

of significance

could

opportunity

group

w hich

an o f f - s h o o t

sufficient

5 predictor v ariab les

had

15

program req u ire m en ts

seemed t o o

A m ultivariate

group w hich

com pleting

groupings

contribute

ignore.

the

9

variations

from g ra d u a te w o r k , th e
predictor

four

who c o m p le te

W hile

to

and e le c te d

problem was

predictor

who do n o t .
host

prior

Fall

Number o f S u b j e c t s

111

3

significant

-

enrolled

C urrently

distribution

Program

12

2

serious

the

D octorate

R eceived

of

in

Status

1

The c h o i c e

Status

the

minimum l e v e l

between
which

5 predictors.
of

acceptance

105
(p ^ . O S )

was

also

found

to

be

categories.

An e x p l a n a t i o n

basis

sm all

tor

of

is

the

inclined

(ranging
dicate

that

the

analysis

size

and

program

A ll
pendent

the

of the

for

the

11 p e r f o r m a n c e

decision

of

to w hether

be

has

to

perform ance

made on

the

the

investiga­

step-down

F

categories

or not

significant

was

made

to

score

category.
create

represents

group

each

in

the

in­

student

regardless

made

four c a te g o r ie s

o f the m u ltip le

mean
to

of

abandon t h e

current

O verall

against

is

four groups.
able"

It
and

scale
w ill

was
be

a score

not

status

for

w hich

to

inde­

compare

perform ance

com pleting

pages

not

differences,

tho

for

and

truly

do e x i s t .

" a r t i f i c i a l ” group

rating

for

preceeding

received

all

E valuation

group

The n e c e s s i t y

differences

were

group d i f f e r e n c e s .

categories,

the

of

regression

test

of

-

was n o t

differences

a measure o f

prediction

subject

The 1 - 7

"most

use

a m easure

fruitless.

create

the

p re d ic to r equations

follow s--significant
p ro o f that

the

develop

been o u t l i n e d

out

the

was

as

of

Group D i f f e r e n c e s

of the

were

three

of

the perform ance

categories

to

values

due

the

can be

the

for

results

attem pts
of

in

allow

categories

strength

actual

findings

that

2.06)

size

to

the

D eterm ination

as

11 o f

However,

program would n o t

sample

significant

cess

on a l l

involved.

p red icto r differences

from th e

sufficient

the

to

for these

sizes

believe

case

size.

As

in

to

from 0 . 6 7

w ithdrew
sam ple

sample

the

this

can be

sum m arized

significant

Therefore,

differences

the

based

the o v e r a l l

pro­

w ith­
decision

on t h e

perform ance

divided

into

four equal

parts

recalled

that

a score

7

of

1 represents

"least

of

able".

10(>
Tabic

5.3

presents

the

groups

based

on t h i s

procedure.

TAB LI: 5 . 3

O verall

Actual

Group

Number o f S u b j e c t s
49

2

5.59

- 4 . 10

35

3

4.09

- 2.00

43

4

2.59

-

20

the

of d istrib u tio n

type
it

rater

is
of

to

of

likely

x when h e

that

gories

to

the

the

is

interpretive

rating

high.

tool

in

to

use

the

level

the

would

for

in

the

as

it
an

con­

the

If

that

characteris­
com posite
rater

one

performance

is

a

cate­

over-all

not p o ssib le

independent

measure

program.

evaluation

o f the

the

indicate

" 7 " on t h e

Ilterefore,
groupings

in

x.

nine

analysis,

assum ptions

a total

of

the

selecting

overall

analysis

of

the

regarding

asking

provides

regression

Research

scales

opinion

groups

meet

characteristics

of his

performance

possible

groupings

a " 7 " on e i g h t

quite

into

for m ultiple

analysis?

overal1-evaluation

su b je c t's

subjects

a question

probability
is

of

on th e

y is

category

consider

It

these

change h i s

answers

subject

than

evaluation

of the

that

regression

a summary s t a t e m e n t

feels

necessary

and a d m i n i s t r a t i o n
not

1.00

classifying

state

for m ultiple

struction

tics

method

possible

required

to

Range

- 5.G0

this

or

Score

Groups

7.00

Although

is

Evaluation

groupings

significance

levels

as

an

for

the

107
regression

equations.

dependent

measures

As t h e

rather

significant

differences

In e s s e n c e ,

the

What n e e d s

to

fo r a given
range

of

gories.

the

scores

these

for

in

the

an

the

perform ance

2.

Intellectual

3.

O riginality

4.

Industry

5.

E ffectiveness

6.

Prom ise

actual
is

scores

the

rem aining

nine

it

same
cate­

can

be

assumed

evidence

of

true

from a l l

that

group

subjects

being

in

above

of variance
o f the

the

range

the

test

significance

f o r mean

differences

mean d i f f e r e n c e s
of

level

test

group d if f e r e n c e s

fundam ental

indicate

on th e

knowledge

breadth
in

research

as

of w ritten

a productive

A nalysis

regression

significant

in

worker

R esults
analysis

differences

evident

reports
research

three

of

variance

attributable
o f the

to

F -test

the

performance

at

for

was

and A p p l i c a t i o n

R egression

were

than
is

of

categories:

M astery

the

the

level.

sim ilarities.

category

resulting

outlined

results

sim ilarity

of

prediction

artifact

results

Hie

on

of

distribution.

analysis

<.0005.

group

do a p p e a r

of significant
and n o t

evaluation

that

differences

1.

variables

overall

range

inter­

test

unusually high

the

at

level,

an

that

set

S etting

at

is

m ultivariate

M ultiple

set

be d em o n strated

the

follow ing

must be

w ith

m easures» the

significant

To a c h i e v e

significant

independent

find

same p o i n t

p

now d e a l i n g

to

discrim ination
the

is

is

actual
If

than

result

g r o u p on

the occurence

at

net

analysis

the

predictor
categories.

.05

108
Tabic

5.4

presents

performance

the

m ultiple

regression

results

for

these

categories.

TABLE 5 . 4

M ultiple

Performance

R egression

A nalysis

- 5 Predictors

2

R

R

GPA

. 33

.11

.006

M astery o f Fundamental
Knowledge

.29

.08

.034

M astery o f
search

.28

.08

.047

G raduate

Category

R esults

Relevant
skills

re­

A com parison o f

the

results

on p a g e

exists

in

only

knowledge.
ficance

graduate

the
that

the

5.4

is

table

psychology
of

adm ission
quite

(4.2)

for

set--GRE

5.4

w ith

a m arginal

that

and th e

significant

prediction

mastery

performance
partial

t h e mean

of

is

level

fundam ental

category

correlation

predictor

the

difference

the

signi­

coefficients

GPA i n

under­

courses.
the

lack

on t h e

noticeable
the

The e v i d e n c e
data

only

Table

category;

w ithin

beta w eights

in

that

one perform ance

The e f f e c t
time o f

107 i n d i c a t e s

Furtherm ore,

of

indicates

results

R e g r e s s i o n ANOVA
F test

to

verbal

o f a m easurable

m ultiple
when t h i s

sociology
this

correlations
table

is

generated

compared w ith

in

at

the

Table

a sim ilar

departm ent.

p o in t would

score,

group d if f e r e n c e

indicate

GRE Q u a n t i t a t i v e

that

Score,

the

adm ission

undergraduate

109
GPA f o r
and
is

the

last

a rating

of

the

institution

o f no s i g n i f i c a n t

current

performance

extensive

opinion

and th e
cided
tion

in

and th e

regarding

evidence

performance

to

return

to

these

members o f v a r i o u s

the

assu m p tio n which

psychology

5 predictors

adm ission
factors

the

the

sim ilarity
finds

the

data

set

adm ission
Therefore,

o f Psychology

w ith

the

psychology

investigator

between

groups

arc

previously

being

was

additional

factors

by

courses

nil

to

it

an assum p­

Chairman

who p a r t i c i p a t e d

in

of

was d e ­

discuss

A ssociate

used

the

that

disciplines.

interview ed
collected

date

seem t o

a s s u m p t i o n was

Record E xam ination
logic,

the

m athem atics

emphasis

was

the

some

and
adm is­

that

true

Advanced

on t h e

additional

im portance
The

in

concensus

Psychology

two
test

th e number o f

other n atural
science

l a b o r a t o r y work

While

and t h a t

Board and
or

t h e members

im pact

adm itted.

quite

to

follow s:

be o f more

the n a tu ra l
on

as

indicates

was

were o f i n t e r e s t - - t h e

in

presented

ma y h a v e

a candidate

The i m p o r t a n c e o f

f lu c tu a te w ith

to

and w o u ld

this

G raduate

com pleted

investigator

evidence

w hether o r not

o f opinion

given

the

department

decision,

determ ining

to

interest

the

B a s e d on

the

and n o t i n g

of

o f Two P r e d i c t o r s

The

the

predictors

perception

in both

unacceptable.

Departm ent

level--

process.

A ddition

of

perform ance,

relationship

the

faculty

members

departm ent

categories

the b ach elo r

psychology program.

faculty

sociology

at

determ ining

graduate

w ith

student

attended

for

the

o f no u s e f u l

regarding

sion

value

conversation

departm ent

u n d e r g r a d u a t e GPA i n p s y c h o l o g y c o u r s e s ,

two y e a r s ,

in

science

f a c t o r seemed
the

interest

110
group.

The A d vanced

im portance

to

procedures

with

Advanced

all

Psychology

of

the

t e s t was

the

pleted

student

application
factor

in

adm ission

The a c t u a l
146 s u b j e c t s
the

test

scores

reported

score

was

in

the

Service.

resolved

Chapter

available,

the

III:

each

for

the

subject
subject.

A ssociate

student

have

reports

on

in

Table

the

those

the

for

science

faculty

was

sent

adm ission.
courses

com­

adm ission

member a s

collected

to

the

the

same m anner a s

For th o s e

all

cases

f o r the

a

fo r the

d e p a r t m e n t by

courses
of

the

assigned.

c o m p l e t e d was t a l l i e d

undergraduate

who u s e d

and

the

were p l a c e d

in

A ptitude

CN “ 2 0 ) w h e r e n o

transcripts

j u d g e d more a p p r o p r i a t e

faculty

application

for the

s a m p l e was

s o me d i s c r e p a n c i e s

Subjects
5.5.

in

m e t h o d was

indicated

transcript.

outlined

and

taking

The p r o b le m o f m i s s i n g o r n o n ­

science

review ing

This

Chairman

experiences

on t h e

by

the

slip

mean s c o r e

The number o f n a t u r a l
for

to

adm ission

departm ental

Advanced t e s t

notification

Testing
was

the

o f the

decision.

on

from t h e

Educational

existent

score

available

that

department

and n a t u r a l
on

considerable

A check

revealed

of the

reported

and t h e r e f o r e

the

contacted.

m athem atics
was

seemed to be o f

Chairman

a requirem ent

logic,

the

persons

A ssociate

Th e n u m b e r o f
by

test

this

by th e

factor

as

between what

actual

amount

their

the

recorded

1 o f 6 categories

as

Ill

TABLE 5 . S

N atural

Category

Science

Coding

Number o f C o u rs e s

1

0

2

C ategories

Completed

Number o f S u b j e c t s

3

24

4 -

7

76

3

8 -

11

30

4

12 -

15

9

5

16 -

19

a*

6

20 +

Although
ing

two

it

was

categories,

categories

as

the

this

study

is

to

ment

should

cants

the

chances

Test

possible
<7 a n d

data

evaluate

be made which

for

faculty

greater
o f being

the

decision

the

could

was

In a d d i t i o n ,

to

variable
record

creat­
the

purpose

of

of predictors

then no

adjust­

between

appli­

differences

the

conversations

w ith

p r e d i c t o r would i n d i c a t e

courses,

six

As t h e

mask t h e

this

this

sample.

strength

number o f

the

better

the

a

the
feeling

c a n d id a te 's

adm itted.

Group D i f f e r e n c e s

Using the o v e r a l l
ferences

dichotom ize

for

the

regarding

the

6

to

>7,

cards

on t h e p r e d i c t o r s .

psychology
that

-

developed

of variance

test

ferences

the

at

evaluation

earlier

in

this

category
chapter,

f o r me a n d i f f e r e n c e s
.10

level

for

the

measure o f group
the

resulted

follow ing

d if­

m ultivariate
in

significant

predictors:

analysis
dif­

112
U ndergraduate
Psychology

GPA i n

Advanced

Number o f n a t u r a l

Prom th e
appear
With

the

the

ought

to

be

the

the

analysis

w ill

tion

group d if f e r e n c e s

in

Chapter

provides
in

psychology

indicating
results

arc

those

of

this

reported

the

as

in

valid.

group

Table

5.3

the

significant

Group D i f f e r e n c e s
the

-

identifica­

on b o t h p r e d i c t o r v a r i a b l e s
the

is

on w h ic h

c h a p t e r . Given

now f o l l o w

would

significant

reported

in

it

faculty

categories

conducted

Category

analysis

and p e rfo rm a n c e
form at

used

the

value

GRE V c r b n l
=

relative

strength

of

the

m ultiple

correlations.

r

.17 M astery

Promise

and r

U ndergraduate

for each

the

score;

.08

correlation*

of

GRE Q u a n t i t a t i v e

ports

R elationships

th e product-m om ent

an i n d i c a t i o n

skills
3.

focus

chapter w ill

range o f

and r
2.

the

test,

I V.

assessing
1.

to

regression

section

Prcdictor-Porform ancc
The

me a n d i f f e r e n c e s

test

Evaluation

this

p <.03

The p e r f o r m a n c e

O verall

categories,

courses

this

m ultiple

p <.06
p <.09

o f two p r e d i c t o r s

mean d i f f e r e n c e s

of

courses

score

presented

im proved.

probability
by

of

assum ption

differences,

test

science

results

addition

psychology

as

score;

= .04
GPA;

and r = <.01

* The s i g n i f i c a n c e l e v e l s
p < . 0 0 1 i f r >. 3 2 .

future

fundam ental

reference

knowledge

a teacher
r

=

.16 M astery o f r e l e v a n t

Intellectual
r

of

for

predictor

»= . 0 8

are p

breadth

E ffectiveness

M astery o f

<.05

research

relevant

if

r

of w ritten

research

>.19;

p <.01

re­

skills

if

r

>.25;

113
4.

Psychology
and

5.

r = .03

Q uality
and

6.

GPA; r = . 2 3 M a s t e r y

r

Industry

Rating;

ledge

and r

r = .11

scores;
=

.16

knowledge

and a p p l i c a t i o n

~ .03 M astery o f

Advanced t e s t

of fundamental

E ffectiveness
relevant

r =

of

w ritten

research s k ills

.3 3 M astery o f

Promise

as

reports

a teacher

fundam ental

know­

- O riginality

in

Research
7.

N atural
r

M ultiple

Science

= =.02 M a s te ry
course

inverse

relationship

Regression

groupings

evaluation
tionship,
the

as

predictors

= -.15

fundam ental

coefficients

A nalysis

w ith

knowledge

-

from t h e

e x p re s s e d by th e
analysis

for the

actual

The s t r e n g t h

vnrianco,

of variance

due

Setting
to

the

the

F test

predictors

at
are

N atural
an

categories)

is

ft

2

.05

present

is

seven p re d ic to rs

score
of
=

for

the
.13.

f o r measuring

the

the

7 Predictors

constructed

R = .36.

of

indicating

10 p e r f o r m a n c e

coefficient

is

a t e a c h e r and

(All

are n eg ativ e,

the

R esults

Promise as

on t h e s u m m a t e d mean d i f f e r e n c e s

level.

ferences

r

correlation

category

regression

.008

of

Science

The m u l t i p l e
and th e

courses;

the

overall

linear
The

rela-

F test

the e ffe c t

significant

of

at

level,

significant

in

follow ing

the

of
the

the
dif­

perfor­

mance c a t e g o r i e s . *

* The a c t u a l r e g r e s s i o n c o e f f i c i e n t s , b e t a w e i g h t s
c o r r e l a t i o n s c a n b e f o u n d i n A p p e n d i x C.

and p a r t i a l

114

TABLE 5 . 6

M ultiple

Regression

A nalysis

R esults

-

7 Predictors
R e g r e s s i o n ANOVA
F -test

2

Performance

Category

G r a d u a t e GPA
M astery o f
ledge
M astery o f
Ski 11s

Fundamental
Relevant

of

E ffectiveness
sentations

o f Oral

O verall

W ritten

the

Advanced 'le s t
strength

of

categories
also
in

the

increased

part

linear
range

.37

.14

p <.003

.33

.11

p <.025

.33

.11

p <.022

.30

.09

p =.05

.35

.13

p <.008

in

three

both

tables.

relationships
Table 4 .2 .

two p r e d i c t o r s

the

seven

reported

for

Given t h i s
psychology

discussed

above

two p r e d i c t o r s ,
increases

predictors

w h e r e mean d i f f e r e n c e s

predictors.
the

o f Table

The s t r e n g t h

variance
the

analysis
can be

2
[R )

sociology

sim ilarity,

the

performance

The a d d i t i o n a l

categories

e x p r e s s e d by

in

the

fo r the

as

strength

of

relationship

relationships

regression

addition

results

courses,

the

the

regression

Science

to

as

the

the

attributable

in

the

p <.007

N atural

th e number o f

categories

of

that

and t h e

linear

reported

.15

5.6 w ith

conclusion

score

p <.007

Pre­

a Productive
Worker

to

.13

.38

Evaluation

leads

.36

Re­

A com parison o f Table
5.4

R

Research

E ffectiveness
ports

P rom ise as
Research

Know-

R

the

are

of
in

have
are
the
the

same

performance
increase

through

the

interpreted

of

the

addition

t o mean

115
that

the

loss

sion

analysis

probability
equivalent

the

has

of
to

significant

P redictive

of

been

that

dictors
is

to

perform ance

group

section

if

in

representations

in

analysis

the

actual

placem ent
in

the

of this

chapter

is

significantly

the

com bining o f

division

of the

the

performance

four

category

procedure

docs

between

m ultiple

for

extent

the

categories
analysis

of

where

indicate

provide

significant

For a d e f i n i t i o n

of

seen

the

of

Z -test

see

in

as

the

- O verall
from

the

Evaluation

create

of

Thus,

for

the

and

the m u ltip le

a

end-point,
agreement
a Z -test

fo u r performance

may b e p o s s i b l e .

Chapter

appro­

from t h e

to

extent

step

chance.

a fruitful

perception.

test

prediction

The n e x t

4.1)

that

seven p r e ­

resulting

figure

conducted

mean d i f f e r e n c e s

for

indicate

the

score

different

(sec

and f a c u l t y

a g r e e m e n t * was
the

actual

a good m easure

regression

should be

analysis.

categories.

categories

" s u c c e s s f u l - m a r g i n n l " d i c h o t o m y was n o t
the

b a s e d on

Group Dif fere nc es

defined

the

reported

interpretations

this

of

Groups

score

of

regres­

category

sociology

Dichotom ized

as

Although

a given

the

the

significance

param orphic

some p e r f o r m a n c e

the

the

in

in

regression

im possible

Therefore,

categories

of

decision

performance

Based on

prediction

determ ine

priate

o f the

Accuracy

placem ent

offset.

successful

The m u l t i p l e
a m arginal

adm issions

IV p .

74.

regression

110

TABLE 5 . 7

D ichotom ized Agreement o f Scores
on S u c c e s s f u l o r M a r g i n a l P la c e m e n t

Performance
M astery

Extent of
Agreement

Category

Effectiveness
Reports
P ro m ise as
Worker
O verall

is

supposed

most
to

of the

P redictive

faculty

group

1 or

where

the

of the

agreement

p <. 05

.60

NS

.61

p <. 05

.69

p

evident

encouraging

function

given

performance

categories.

in

presents
as

term s
the

of

p red icto r variables

differences

from t h e

this

performance

4.1)--in
account

mean as

an a c t u a l
those

for

category

o f the

student

P robabilities

of subject

as

<.0005

evaluation

representation

probability

Figure

overall

Group P lac em en t

successful--defined
(see

the

that

a com posite

group 2

ANOVA F - t e s t .

in

as

Accuracy

Tabic 5 .8
by t h e

.61

a Research

Evaluation

category

all

level

o f W ritten

The p l a c e m e n t

for

significance

o f fundam enta1

Knowledge

is

Z-tcst

X being perceived
placem ent

performance

a significant

e x p r e s s e d by t h e

in

categories

proportion

regression

117

TABLE 5 . 8

Probability

of

Perceived

Success

Based on

of

Fundamental

Effectiveness
Prom ise as
Worker
O verall

of

Placem ent

P r e d i c t e d Group
1
2

A c tu a l Group P la c e m e n t
of 1 or 2
M astery

Group

Knowledge

W ritten

a Productive

R eports

Placem ent
4
3

.72

. 76

.53

. 46

1.00

.73

.57

.50

. 88

.65

. 46

.29

.87

.76

.40

.44

Research

E valuation

Percent o f s u b je c ts p e r group
when f a c u l t y p l a c e i n g r o u p 1 o r
A reading
reveals
1 and

of

for

group

columns.

not

a f f e c t e d by
the

point

number o f
more

group

of

the

sample

of

groups

2 and

his

variation
This

each

of

predicted

success

variation

an e x t r e m e l y

whore

mean

is

is
the

other

who a r e

or

in

fact

8-18

and in

group

mean,

perceived

in

the

which

analysis.
to

the

is

is

4 being

two g ro u p s
to

by

group placem ent.

the

the

group

N in

equation

any one

sm all--thc

those
is

variable
Thus

the

deviation
range

of

or
N

8-13.

The m a j o r i t y

w ithin

o n e-h alf standard

The p r o b a b i l i t y
that

for

suppressed.

be expected

conclusion

successful

is

on

one-h alf standard

direction

located

of

a prediction

inform ation

placem ent

probabilities

low s c o r e

either

3 lead

predicted

all

high

group

a function

in

regression

o f being
to

of

for

As m e n t i o n e d p r e v i o u s l y ,

1 being

deviation

4.

subjects

from th e

in

tion

columns

considerable

two

to

the

given

the

differences

a s u b je c t’s

faculty

u tiliz a­

are highly

between

chances
related

2

1 18
The r e l a t i o n s h i p
m e n t on t h e
actual

placem ent

stro n g w ith

S u mm a r y o f

at

The

the

time

ment,

to

score

the

a measure o f

ndm ission

financial
allow

placem ent

for

an e x a c t

predictors,

for

mean p r e d i c t e d combined

the

decision

is

very

collected

adm itted

in

the

replication

of

GRE A d v a n c e d

and d a t a

for these

candidates,

sociology
the

study

depart­
in

the

an

attem pt
on t h e

to

to be

of

and num ber

im portance

predictors

were

measure group d if f e r e n c e s

trindic-com parison

c a t e g o r y was

statistical
present

used to

p

in

overall

the

place

test

level,

the o th e r perform ance
to

found

test

in

the

collected

subjects.

The m e a n d i f f e r e n c e s

found

decision

decision

performance

were

the

group d if f e r e n c e s

were

In

ferences

and

categories

courses,

received

ings.

three

science

sample

3.

in

of

of

Two a d d i t i o n a l

ndmission

other

category

departm ent.

of natural

all

on t h e

absence

would n o t

2.

E valuation

place­

= .89.

the

psychology

high

r

a s u b j e c t ’s p r e d ic te d - a c tu a l

R esults

1.

sim ilar

on

O verall

between

the

the

to

determ ine

evaluation

categories.

as

in

evaluation

four group­

was

set

at

an u n u s u a l l y

if

the

group

dif­

c a te g o ry were p re s e n t

Six o f

difference

actual

overall

subjects

significance

<.0005,

be of s im ila r

for

the

the nine

categories

overall

evaluation

the

category.
4.
tional

M ultiple
predictors

ment d e c i s i o n

regression

analysis

do a c c o u n t

variable

as

the

for the

indicates
loss

strength

of

of the

that

the

the

two a d d i ­

adm ission p la c e ­

linear

relationships

119
were

sim ilar

for

those

performance

categories

significant

in

both

departm ents.
5*

When

predictor,
rating

the

the

number

received

in

contrary

to

decision

process.

6.

the

to

ment betw een
the

. OS

is

all

to

science

be

system

ANOVA P - t c s t

for

and

three

and o v e r a l l

evaluation

(Table

5.6).

probability

ful

(placem ent

in

the

subject

directly

(Table

m easurable

score

ceived

ns

tionship
The

the

of
1 or

5.8).

differences

dicted

8.

group

extent

a subject
group

being

2)

by

For th e

4 perform ance

the

found

the

to

research

su b je c t's

exist,

probability

of

The

summary

evident

in
of

statem ent
the

the

D epartm ent
probability

perform ance

is

between

two d e p a r t m e n t s

the

therefore

perform ance.

in

point

group

predictors
of

as

agree­

success­

who know

predicted

the h ig h er
subject

of

to

Sociology

factor

successful,
as

7 presents

indicating

what

the

(see

approach

in
the

group
which
pre­

being

per­

to

some r e l a ­

Table

4.9).

predicting

some s i m i l a r i t y

constitutes

is

w orker,

categories

the

finding

M astery o f

faculty

to

actual

significant

perceived

the

a

successful.

replication

graduate

a productive

related

were

greater

the

the

four categ o ries:

as

placem ent

indicated

as

the

interest

of

prom ise

is

the

to

'ihis

g r o u p p l a c e m e n t was

the

used

related

actual

know ledge,

The

in

s u m m a tcd mean d i f f e r e n c e s ,

predicted

is

categories.

functioning

fundam ental

7.

courses

inversely

performance

regression

the

level

found

ten

belief

Where t h e

contributed

at

number o f n a t u r a l

acceptable

120
9.

S ignificant

jects

as

allow

the

of
the

the

indicated
adm ission

su b je c t's

adm ission

by

the

set.

between

probability

com m ittee

future

data

differences

of

to d e v e lo p

performance

level

the

four

perceived

success

a m eaningful
in

the

groups

would

estim ate

program

of

from

sub­

CHAPTER VI

SUMMARY OF MAJOR F I N D I N G S ;

Research
In
to

Design

an

to

the

(GRE V e r b a l

Score,

of

determ ine

perform ance

employed by

years

collected
1)

for

of

3)

1)

work,

the

Sociology

GPA i n

an

of

data

in

the

adm ission
for

five

criteria

predictors

adm ission

process

Score,

GPA f o r

the

sociology

courses

and an e s t i m a t e

undergraduate

in stitu tio n

students

who m e t

undergraduate

degree

last

two

a tte n d e d ) were

the follow ing c o n d itio n s:
from an American o r

U niversity.

Were

adm itted

1967

through

to
Fall

Who h a d e n r o l l e d

graduate

work

for

the

period

Fall

Term

Term 1 9 7 0 .
and c o m p le te d

a minimum o f two t e r m s

coursework.

Performance
GPA a n d

school,

GRE Q u a n t i t a t i v e

Had e a r n e d

of

im portance

graduate

118 g r a d u a t e

Canadian
2)

in

the

Departm ent o f

of b a c h e lo r's

theq u a lity

I MP L I CAT I ONS

Summary

effort

future

CONCLUSI ONS AND P OLI CY

in

graduate

10 p e r f o r m a n c e
M astery

of
of

school

M astery

3)

Intellectual

4)

O riginality

Relevant

Knowledge

Research

Breadth
in

measured by

the

G raduate

categories:

Fundamental

2)

was

Research
121

Skills

in th e S tu d e n t's F ie ld

122
5)

Industry

6)

E ffectiveness

ofW ritten

7}

E ffectiveness

of

8)

Prom ise

as

a P roductive

9)

Prom ise

as

a Teacher

10)

O verall

E valuation

D ata on
faculty

of

actual

m ultivariate
group

by

of

rating

on

the

success--defined

on

those

ferences

seven

predictors

m entioned
course

was

perform ance

placem ent

147 g r a d u a te

11 p e r f o r m a n c e
to
the

choice

for

method

categories

a sub-

prediction

em ployed

in

in

the

the

in

science

who m e t

for

both

had

linear

perform ance
the

each

cate­

probability
group

group

dif­

occurcd.
of

Psychology w ith

procedure

psychology

study

a

M ultiple

sig n ifican t

Departm ent

courses

in

significant

developing

psychology;

the

determ ine

lcvel--for

adm ission

substituting

and n a t u r a l

to

em ployed

expected

ability
where

was

criteria.

predict

perceived

replicated

students

decision

o f mean d i f f e r e n c e s

GPA; GRE A d v a n c e d T e s t

m athem atics

from t h e

rater.

as

previously,

collected

category

and s i g n i f i c a n t

The s t u d y

were

on cacti

the

perform ance

Worker

s c a le --a forced

provided

of

Research

categories

predictors

scores

Presentations

triadic-com pnrison

the

analysis

Reports

Oral

adm ission

differences

regression
gory

of a

ability

selected

The

A pplication

perform ance

by means

com parison
sample

the

and

course

(the

GPA f o r s o c i o l o g y

and number o f

com pleted)
conditions.

five

for

logic,

a sample

of

1 23
M ajor F in d ings

-

Departm ent o f

No s i g n i f i c a n t
who r e c e i v e d
any

of

the

The

one

mean d i f f e r e n c e s

rating

The m u l t i p l e
and

predictive

the

of

Score,

rating

d e c isio n , is
use

of

the

work was

have

The

actual

o f the

found

for the

decision

reliab ility ,

five

predictor

indicates

signi ficantly

placem ent

undergraduate

to

defined

the

as

predicted

to

GPA a s

a strong

to

decision

the

param orphic

are

GRR V e r b a l

institu tio n

a n d GPA i n

the

extent

decision

of

and

a g ree m en t on

the

actual

where

previously

evaluated

of

the

those

decision
on

were

categories

the

or

data

found

set

by

the

criterion

was

four perform ance

exist
data

for

only

in

regarding

rater.

and th e

actual

superior

the param orphic

performance

to

substantive

5 predictors

form an a d m i s s i o n

adm ission

a perform ance

be not w arranted.

group d if f e r e n c e s

been

group placem ent
For

coefficient

contribute

G raduate

com bination

decision

rater

.70.

perform ance

ab ility

a m easure o f

adm ission

between

S ignificant
those

on

(p < . 0 5 ) .

placem ent

reliab ility ,

group placem ent

graduate

rating

courscwork.

P redictive

The

the

>1

( R ** . 5 6 ) .

which

representation

received

subjects

.44.

adm ission

relationship

quality

=

found b e tw e e n

who

categories

correlation

The p r e d i c t o r s

sociology

r

were

subjects

correlation,

a coefficient

variables

and

11 p e r f o r m a n c e

intraclass

produced

Sociology

to

decision

placem ent
either
in

the

predicting

categories.

categories

where

group d iffe re n c e s

124
and

significant

ship

was

found

the

probability

the

faculty.

prediction
to

M ajor Findings

exist

of

-

any

of

the

The

one

to

intraclass

w hich
for

evidenced

3 of

score.

the

level

Psychology

for

3 of

ability

in

differences

of

is

the

and n a t u r a l

existing

criterion

4 perform ance
and

evident
subject

between
being

able

by

for

the

on

for

to

be

the

of

r

=

adm ission

relationship,

were

to

subjects

>1 r a t i n g

a coefficient

science

those

exist.

4

perform ance
predictors

regression

courses

data

equivalent

The

Psychology

.46.

analysis

Advanced

test

taken.

significant

at

the

.05

categories.

was

found to

be

o f no m e a s u r a b l e

group d if f e r e n c e s

where

prediction

predicted
as

and i t s

use

w arranted.

categories

perceived

and

<.05).

(P

differences

found

was n o t

significant

as most

who r e c e i v e d

contribution

GPA c r i t e r i o n

placem ent

fo u n d between

relationship,

4 perform ance

determ ining

were

produced

group

was

relatio n ­

Psychology

categories

reliab ility

a perform ance
For the

GPA,

the

perceived

a predictive

significant

a strong

group

coefficients

Sociology

significant

The G r a d u a t e

ship

indicate

4 perform ance

P redictive

as

correlation

where

being

categories

correlation

Departm ent o f

categories

predicted

and s u b j e c t s

11 p e r f o r m a n c e

(7 p r e d i c t o r s )
the

subject

evident,

mean d i f f e r e n c e s

rating

The m u l t i p l e
set

between

Departm ent o f

No s i g n i f i c a n t
who r e c e i v e d

the

were b o th

are

both

significant

present,

group placem ent
most

able

by

the

a strong

and t h e

group
relation

probability

faculty.

125
Conclusions
The
ment

results

the

level

of

the

o f graduate

the p e rc e p tio n s

of

the

this

is

an

perception

testable

as

study

accurate

no d i r e c t

measure

at

the

same

evaluated

by

all

faculty

perceptions

arc

perform ance

given

Pi ine

function

arc

defined

a

as

formance
faculty

real

of

any s t u d e n t

of

the

the

graduate

the

adm ission

a measure

any one

of

aspect

program by
is

opinion.

no m o re

to

a measurement o f
works.

W hether

is

not

graduate

each

docu­

subject

w ork,
and

The

triadic

com par­

legitim ate

measure

of

available

As

no evidence
fact

of

available.

opportunities

in

of

data

student

it

is

firs t

ment w ish

to

or

by

possible

o f the

question

that

to

the

than

in

w hich

the
has

the

d iscibeen

contrary,

the

opinion

to

employ

an

significant

of successful
can be

com bining

process

increase

of

per­
any

the

and r e l i a b l e

perform ance

developed

directly

adm ission

adm ission

the

the

from
w ith

com mittee

from

of

current

additive-probability

the
pattern­

model.

Model

m u st be a n s w e r e d

the p ro b a b ility

in

data

efficiency

by r e p l a c i n g

Probability

which

a

m ade by t h e

approach w ith

Im plications

provide

attainm ent

ability

decision-m aking

o f variables

do

The e s t i m a t e

itse lf

Therefore,
adm ission

factors

program .

data.

The

is

w ith

perform ance

the

given

possible

assessment

faculty

s tu d e n t's

adm ission

Policy

of

perform ance

a more t h a n

future

is

member.

estim ate

ing

however,

it

student

on

in

that

w i t h whom t h e

members,

real,

The p r o b a b i l i t y

the

point

that

is

student

faculty

com pleted

ison

indicate

model

in

is:
the

Does t h e
adm ission

depart­
decision

126
or ra th e r,

use

In e i t h e r
1)

th e model

case

the

of

the

graduate

Given

the

model

sh o u ld not

standard

of this

model

then

is

it

other

of faculty
two y e a r

2)
those

results

w ill

period

GPA h a d

undergraduate
as

the

that

to

the

to

model

as

not

is

the

GPA o u g h t

necessary

of

the

not

to

Therefore,
be

inform ation

removed
to

not

use

the

a
group

each

group.

into

assumed

scores

or

of

take

and

account

that

last

design

two

given

that

undergraduate

test

scores

from t h e

justify

in

end o f

for

study

the

the

triadic-com parison

may b e

test

the

adm its,

level

the

did

It

the

that

p resen t level

w i t h l o w GRl: s c o r e s

excluded.

a

at

tim e.

categories,

of success

adm itted.

in

o f m arginal

developed

for

decision?

point

Assuming

conducted

a function

candidates

one

probability

determ ine

contribution

been

at

Therefore,

be

the

a cross-sectional

performance

probability factor

th e model

already

the

constant.

who w e r e

of

d e v e l o p e d on

indefinitely.

ought

on

must be o b s e r v e d .

population

for

assume

order

significant

check

a low er p e rc e n ta g e

rem ain

the

candidates

the m a jo r ity

set

in

to

in

and

GPA i n

was

be em ployed

The p r o b a b i l i t y

lack o f
years

model
student

perceptions

differences

cautions

deviation

invalid

groups

an e v a l u a t i v e

follow ing

The p r o b a b i l i t y

sample

as

such

and

adm ission

a decision

the

data
is

not

avai 1a b l e .
The

second q u estio n

what t h e
placem ent
all

effect

as w ell

applicants

standard

of

regarding

screening
as

the

a fifth

all

the

utilization

applicants

predictive

validity?

group placem ent,

d ev iatio n below

the

w ill

category

located

mean,

of

the

h a v e on

With
at

defined

the
and
as

model
the

is

group

addition
less
the

than

of
one

non-adm it

127
category,

must

To t e s t

this

who w e r e n o t
the

sam ple

their
then

be

the

accepted

of

in

to

adm ission

the

five

group placem ent
are

presented

model.

screening

the

characteristics,

added

results

to

question

predicted

dicted

added

63 c a n d id a te s

of

Sociology

a n d who met

were p la c e d

in

the

to

predictors

in

applicants,

D epartm ent

decision

on

all

the

placem ent.
to

estim ate

overall

Table

model

6.1

This

placem ent

this

evaluation

determ ine

sam p le's

category.

was

pre­
The

below,

TABLE 6 . 1

Predicted

Group P la c e m e n t o f N o n -A d m itte d
Sociology
Group

Category
A dm ission
O verall

Placem ent

E valuation

and i s

regression

tow ard

to

be

expected.

5 percent

of

the

as

successful

o f most

of

the

in

sample

0

7

6

24

26

0

3

3-1

22

4

the

mean

However,

the

* R e p r e s e n t s <1 s t a n d a r d
adm itted stu d en ts.

in

the

subjects

Table
table

a reasonable

sociology
to

5*

2

sam ple have
the

Placem ent
4
3

1

Number o f

The

A pplicants,

program.

third

and

deviation

6.1

is

does

F inally

below

the

quite

evident

indicate

chance

fourth

(N a 6 3 )

at
the

that

being

only

recognized

addition

categories

category

serves

mean

for

] 28
to

increase

decrease

the

the

measure

category

probability

this

sam p le's

b a se d on

observed

able

the

for
This

N*s

probability

factor

actual

relative

to

the

model

could
of
the
of

the

be

relevant

mean

a decision

to

To b e

is

not

on

your application
unethical.
can

be

im pact
There

In

im proved
in
are

known w h i c h
the
as

are

high

F inally,

the

such

formance;

it

does

Thus, the

purpose

and

then

up t h e

placed

who d i d

categories

of

common d e c e n c y

not

would

in

place

all
above

would be n o t i f i e d

o f num eric

in

criterin

such

being

is

degrading

the

ease

w ith

m anipulated

were

discussed

w hich

above

criteria

not

measured

in

is

fact

not,

to

graduate

able

that

to

utilize

3 in

students
provides

10 o f

in

some o f

the

ever having

test
and
is

this

perform ance.

It

scores

their
m aximal.

model
ma y b e

these

num erically
the

approach.

and p o s sib ly

one o u tlin e d
and

an

w ithout

the

model

of

group

this

adm ission

as

ability

the

avail­

I Zv o r y a p p l i c a n t

five

chapter,

GPA's

com m ittee

e v id e n c e d by

probability

To s e t

probability

of

a human

do c o n t r i b u t e

adm ission

adm its

by

first

other

the

estim ate

com m ittee?

Any a p p l i c a n t

basis

a procedure
also

way t o

qucstion--should

and e x p e n se .

adm ission

lack

read

and

best

or

adm it.

the

the

no p o s s i b l e

the

accom plish

tim e

three

a certain

rejected

of

an

for

increase

Therefore,

is

adm ission

to

categories.

category

There

in

is

m ajor p o lic y

the

amount

classified

there

perform ance.

third

replace

a m inim al

as

can n e i t h e r

sample.

com puter program n e c e ss a ry
require

it

frequencies

non-acccptcd

leads

only;

the

eyes

of

no more

than

an e s t i m a t e

m arginal

faculty.
of per­

can n o t

guarantee

good p e rfo rm a n c e .

such

a model

is

replace

to

that

intangibles

it

not

not

the

adm ission

129

com m ittee

but

to

num erical

adm ission

an

a p p lic a n t's

is

clearly

the

of

the

estim ate
all

the

this

is

6/10,

the

w ishes

Research

been

develop b e tte r
for

the

" I f we c o u l d
program,
for

being
and
What

There
alone.

on w i l l

s till

that

as

and t h e

As

in

members

foresight

if

to which

require

applicant.
faculty

Knowing

However,

decision

the

frequency,

the

from a th o u g h tf u l

adm itting

best

reading
the

of

past,

t h e wisdom o f

of

Isaiah.

Ibis

them w e l l .

of

ad m issio n 's

and

developed

selection

on

m entioned.

we w o u l d n ' t

could

the

resulting

of Job,

measures

intangible

practices.

2/10,

is

com m ittee.

on o b s e r v e d

o f a program re - e v a lu a tio n

measure

tangible

based

of

P ossibilities

The n e c e s s i t y
already

or

interpretation

adm ission

current

she

o f the

patience

inv estig ato r

input

4/10

available

require

Solomon,

has

on

f a c to r he o r

m aterial

fo r the

success,

o f a human b e i n g

w ill

future

of

improvement

applicant

side

criteria

chances

an

a more m e a n in g fu l

provide

Of more
academic

need

these

of

measures

else".

aspects

and im proved.
aspects

As o n e

of

the

to

is

to

the

need

w hich p ro v id e

to

the

degree
instrum ents

human b e i n g

community
the

the

member s t a t e d :

instrum ents

university

contribute

basis

M easuring

M easuring

the

a bi-annual

faculty

and m o t i v a t i o n

anything

intangible

im portance
achievem ent

decision.

commitment

on

are

for

also

constantly

tangible
exist.

im provement o f

the

process?
are

Each

a candidate

over sixty
departm ent

for

graduate
adm its

evaluation.

departm ents

graduate

What d o

at

students

they

use

in

this

university

and i s
their

therefore
adm ission

130

data s e t ?

What do they e x p e c t as s u c c e s s f u l performance in t h e i r

programs?

Will the p r o b a b i l i t y model as c o n s t r u c t e d provide u s e f u l

inform ation f o r them?
F i n a l l y , t h e p r o b a b i l i t y model i t s e l f i s not p e r f e c t , i t needs
f u r t h e r t e s t i n g and improvement.

I b i s i n v e s t i g a t o r p lans t o

experiment w it h i t again i n at l e a s t one o f the o r i g i n a l study
departments.

Any o th e r i n v e s t i g a t o r s who would l i k e to p a r t i c i p a t e

in the refinem ent and perhaps r e b u i l d i n g o f t h i s model are most
welcome to t h e f i e l d .

The graduate programs in t h i s country occupy

the most p r e c a r io u s p o s i t i o n they have for d ecad es; more s t u d e n t s ,
l e s s funds and fewer j o b s .
product w i l l be a p p r e c ia te d .

Any h e lp to improve the q u a l i t y o f the

BIBLIOGRAPHY

BIBLIO GRA PH Y

Books
B la lo ck , Hubert M., J r .
(New York, 1972).

Social S t a t i s t i c s .

McGraw-Hill

Edwards, A. L. Techniques o f A tt it u d e S e a l e C o n s t r u c t io n .
A pp leton-C cntury-C rofts (New York, 19S7).
G u ilf o r d , J . P. PsychoiiKitric Methods.
York, 1954).

McGraw-Hill (New

Lavin, D. E. The P r e d i c t i o n o f Academic Performance.
R u s s e l l Sage Foundation (New York, 1965).
Meehl, P. E. C l i n i c a l v s . S t a t i s t i c a l P r e d i c t i o n .
s i t y o f Minnesota Press (M inneapolis, 1954).
Torgcrson, W. A. Theory and Method o f S c a l i n g .
(New York, 1956).

U niver­

John Wiley

Monographs
Bum s, R. L. Graduate Admission and Fellow ship S e l e c t i o n
P o l i c i e s and Procedures Part 1. Educational T e s t in g
S e r v i c e ( P r in c e t o n , 1970).
Chase, C. I. c t . a l .
P r e d i c t i n g Success f o r Master* s
Degree Students i n Ed u cation . Bureau o f Educational
S t u d i e s and T e s t i n g , Indiana U n i v e r s i t y (Blooming­
t o n , 1964).
Dawes, R. M. Graduate Admissions: A Case Study. Oregon
Research I n s t i t u t e T e chnical Report 1 0 - 1 , U n i v e r s i t y
o f Oregon (Eugene, 1970).
Dawes, R. M. and D i l l e r , R. D. The P r e d ic t i o n o f Boot­
s t r a p p i n g and a Method o f AmalgamationT Oregon Research
I n s t i t u t e T ech n ical Report 10-6, U n i v e r s i t y o f Oregon
(Eugene, 1970).

132
11.

Lannholm, G. V. Review o f S tu d ies Employing GRE Scores
i n P r e d ic tin g Success in Graduate Study: 1952-1967.
Graduate Records Examinations S p e c ia l Report 6 8 - 1 ,
Educational T e s t i n g S e rv ice ( P r in c e t o n , 1968).

12.

Lannholm, G. V. , Marco, G. L. and Schrader, W. B. Coopera­
t i v e Studies o f P r e d i c t i n g Graduate School S u c c e s s .
Graduate Record Examinations S p e c i a l Report 6 8 - 3 ,
Educational T e s t i n g S e r v i c e , ( P r in c e t o n , 1966).

13.

K e l l y , E. L. and F i s k e , t), W. The P r e d ic t i o n o f Performance
i n C l i n i c a l P s y c h o lo g y . U n i v e r s i t y o f Michigan Press
(Ann Arbor, 1951).

14.

___________ . Guide t o th e Use o f GRE Scores in Graduate
Admissions 1 9 7 1 - 1 9 7 2 . Educational T e st in g S e r v i c e
( P r i n c e t o n , 1971).

A rticles
15.

Alexakos , C. E. " P r e d i c t i v e E f f i c i e n c y o f TVo M u l t i v a r i a t e
S t a t i s t i c a l Techniques in Comparison with C l i n i c a l
P r e d i c t i o n s " . J. Ed. P s y c h o l . , 5 7 : 2 9 7 - 3 0 6 , October
1966.

16.

A l e x a k o s , C. E. "Graduate Record Examination: A p titu d e
T e s t s as S cr e e n in g Devices f o r Students f o r t h e
C o lle g e o f Human Resources and Education at West
V i r g i n i a U n i v e r s i t y " . C o l. fj U niv. , 4 3 : 3 4 2 - 7 , Spring
1968.

17.

Borg, W. R. "GRE A ptitud e Scores as P r e d ic to r s o f GPA f o r
Graduate S tu d e n ts in Education". Ed. 5 P s y c h o l . M. ,
2 3 :3 7 9 -8 2 , Summer 1963.

18.

Chase, E. B. "Study o f Undergraduate Records o f Graduates
from Huntor C o l l e g e who l a t e r earned D o cto ra tes" .
J . Exp. Ed. , 2 9 : 4 9 - 6 0 , Spring 1960.

19.

Crane, D. "Social C la ss O rigin s and Academic S u c c e s s :
The In flu e n c e o f Two S t r a t i f i c a t i o n Systems on Academic
Careers". S o c . Educ. , 4 2 : 1 - 1 7 , Winter 1969.

20.

Hackman, J. R. e t . a l . " P r e d i c t i o n o f long-term S u c c e s s in
D octoral Work i n Psychology".
Ed. 5 Psych ol. M. ,
3 0 :3 6 5 -7 4 , Sumner 1970.

133

21.

H eiss , A. M. "Berkeley d o c t o r a l s t u d e n t s p r a i s e t h e i r
academic program". Educ. Rec. , 4 8 : 3 0 - 4 4 , Winter
1967.

22.

Herbert, D. J .
"A P r e d i c t i v e Study o f q u a l i t y p o in t a v e r ­
age in Graduate Education Courses". J . Educ. Res. ,
6 0 : 2 1 8 - 2 0 , January 1967.

23.

Houston, S. R. "Generating a p r o j e c t e d c r i t e r i o n o f
graduate s t u d e n t s u c c e s s v i a normative judgement
analysis".
J . Exp. Ed. , 3 7 : 5 3 - 8 , Winter 1968.

24.

I r v i n e , I). W. "M ultip le p r e d i c t i o n o f c o l l e g e gradua­
t io n from p r e - a d m is s io n data". J . Exp. E d . , 35:
84-9, F a l l 1966.

25.

King, D. C. and Bcsco, R. O. "Graduate Record Examina­
t io n as a s e l e c t i o n d e v i c e f o r graduate research
fellow s".
Ed. 6 P sy ch o l. M. . 2 0 : 8 5 3 - 8 , Winter 1960.

26.

Law, A. " P r e d ic t io n o f r a t i n g s o f s t u d e n t s in a d o c t o r a l
t r a i n i n g program". Ed. 6 P sych ol. M. , 20:84 7-51 ,
Winter 1960.

27.

Madaus, G. F. and Walsh, J . J . "Departmental d i f f e r e n ­
t i a l s in th e p r e d i c t i v e v a l i d i t y o f t h e Graduate
Record Examination a p t i t u d e t e s t " .
Ed. 6 P sych ol. M.
2 5 : 1 1 0 5 -1 0 , Winter 1965.

28.

Mnrascuilo, L. and G i l l , G. "Measuring the d i f f e r e n c e s between
s u c c e s s f u l and u n s u c c e s s f u l d o c t o r a l s t u d e n t s in ed u ca tio n " .
C a l i f . J. E d . , Res. , 1 8 : 6 5 - 7 0 , March 1967.

29.

Mehrabian, A.
"Undergraduate a b i l i t y f a c t o r s in r e l a t i o n s h i p
to graduate performance: graduate p sych ology program".
Ed. fi P s y c h o l. M. , 2 9 : 4 0 9 - 1 9 , Summer 1969.

30.

Owens, T. R . , and Roaden, A. L.
" P r e d ic t in g academic s u c c e s s
in m a s t e r ' s degree programs in e d u c a t io n " . J . Educ. R es. ,
6 0 : 1 2 4 - 6 , November 1966.

31.

Newman, R. I.
graduate
1968.

32.

Payne, D. A . , e t . a l . "Another c o n t r i b u t i o n t o e s t i m a t i n g
s u c c e s s in graduate s c h o o l : a se arch f o r s e x d i f f e r e n c e s
and a comparison between th r ee degree t y p e s " . Ed. 6 P s y c h o l .
M., 3 1 : 4 9 9 - 5 0 3 , Summer 1971.

"GRE s c o r e s as p r e d i c t o r s o f GPA f o r p sy c h o lo g y
students".
Ed. G P sy c h o l. M., 2 8 : 4 3 3 - 6 , Summer

1 34

33.

R c n i c k , T. I'.
"A rc h ig h sc h o o l r e c o r d s i n d i c a t i v e o f s u c c e s s
a t th e d o c to ra l le v e l" ?
.1. E d u c . M e a s , , 3 : 3 3 - 4 , S p r i n g
1966.

34.

Reyes, R. and Clarke, R. B. "Consistency as a f a c t o r in p r e ­
d i c t i n g grades".
P c r s . fj Guid. J o u m . , 4 7 : 5 0 - 5 , Septcnijer 1958.

35.

Robertson, M. and H a ll , F. " P r e d ic t io n s u c c e s s in graduate
study". J. Gen. P s y c h o l . , 7 1 : 3 5 9 -6 5 , 1964.

36.

Robertson, H, and N e i l s o n , W. "The GRE in the s e l e c t i o n o f
graduate s t u d e n t s " . Amer. P s y c h o l . , 1 6 : 6 4 6 -5 0 , 1961.

37.

Roscoc, J. T. and Houston, S. R. " P r e d i c t i v e v a l i d i t y o f GRE
s c o r e s f o r d o c t o r a l s t u d e n t s in ed u ca tio n " . Fd. G P s y c h o l .
M. , 2 9 : 5 0 7 - 0 9 , Summer 1969.

38.

S a rb in , T. R. "The l o g i c o f p r e d i c t i o n in Psychology".
P sychol. Rev. . 5 1 : 2 1 0 - 2 8 , 1944.

39.

Sharp, L. M. "Graduate study and i t s r e l a t i o n to c a r e e r s :
the e x p e r ie n c e o f a r e c e n t cohort o f c o l l e g e grad u a tes".
J. Hum. Rs. . 1 : 4 1 - 5 8 , F all 1966.

40.

Sorenson, G. and Kagan, D. " C o n f l i c t s between d o c t o r a l candidates
and t h e i r sp onsors". J . Higher Ed. , 3 8 : 1 7 - 2 4 , January 1967.

41.

Spaeth, J, L. " A ll o c a t io n o f c o l l e g e graduates to graduate and
p r o f e s s i o n a l s c h o o l s " . S o c . Educ. , 4 1 : 3 4 2 - 9 , F a ll 1968.

42.

S t r i e k e r , G. and Huber, J. T. "Graduate record examination
and undergraduate grades as p r e d i c t o r s o f s u c c e s s in
graduate s c h o o l" . J . F.d. R es. , 6 0 : 4 6 6 - 8 , July 1967

43.

S t o r d o h l , K. E.
" P r e d ic t in g grades in M aster's degree programs".
J. Educ. Meas. , 4 : 1 1 9 - 2 2 , Summer 1967.

44.

S u z i e d e l i s , A. and Kinnane, J. F. "Faculty judgements o f
graduate s u c c e s s : a f a c t o r i a l study". Amer. P s y c h o l . ,
1 6 : 1 8 1 -4 , 1961.

45.

T r a g e s s e r , E. e t . a l .
"Do academic p r e s s u r e s n e c e s s a r y for
admission to graduate and p r o f e s s i o n a l s c h o o l s produce
i n t e l l e c t u a l grants and s o c i a l m i s f i t s " ? Col1. 6 U n i v . ,
4 4 : 3 1 9 - 2 1 , Summer 1969.

46.

Thorndike, E. L. "Fundamental theorems in judging man".
J . Appl. P s y c h o l . , 2 : 6 7 - 7 6 , 1918.

1 35

47.

Tucker, A. and S lo a n , L. "Graduate school t a l e n t : s o l d to
the h i g h e s t bidder". J. Higher Ed. , 3 5 : 1 2 -1 8 , January
1964.

48.

T u l l y , G. h. "Screening a p p l i c a n t s for graduate study with the
a p tit u d e t e s t o f the graduate record exam inations".
C o l l . f, U niv. , 3 8 : 5 1 -6 0 , F a ll 1962.

49.

Webb, S. C. " P r ed icto rs o f achievement in graduate s c h o o l" .
J. Appl. P s y c h o l . , 35 :26S-71, 1951

50.

Webb, S. C. " D i f f e r e n t i a l p r e d i c t i o n o f s u c c e s s in graduate
s c h o o l" .
J . Ed. Res. , 50: 4 5 - 5 4 , 1956.

51 .

Wegner, E. L.
"Some f a c t o r s in o b t a i n i n g p ostgr ad u ate ed u cation" .
Soc. E d u c .. 4 2 :1 5 4 -6 9 , Spring 1969.

52.

Wcrts, C. E. and Watley, E. J .
"Analyzing c o l l e g e d e f e c t s :
c o r r e l a t i o n vs r e g r e s s i o n " . Am. Ed. Res. J o u m . , 5:
5 8 5 -9 8 , November 1969.

53.

W illia m s, J. D . , e t . a l .
"Judgement a n a l y s i s fo r a s s e s s i n g
d o c t o r a l adm issions p o l i c i e s " . J . Exp. Ed. , 3 8 : 9 2 - 6 ,
Winter 1969.

54.

W illia m s, J . D . , e t . a l .
"A l o n g i t u d i n a l study examining
p r e d i c t i o n o f d o c to r a l s u c c e s s : g r a d c - p o in t average as
c r i t e r i o n o r graduation v s , n on-graduation as c r i t e r i o n " ?
J . Ed. R es. , 6 4 : 1 6 1 - 4 , December 1970.

55.

Wright, C. R.
"Success or f a i l u r e in ea r n in g
Soc. E d u c ., 38 :7 3 -9 7 , F a l l 1964.

graduate degree".

APPENDICES

APPENDIX

A

TRIADIC-COMPARISON RATING SCALE
I n s t r u c t i o n s and Rating Sheets

136

TRIADIC-COMPARISON

I n s t r u c t i o n s for Rating
1.

Read the n i n e items l i s t e d on the r a t i n g s h e e t , as t h e y w i l l
provide you with needed inform ation f o r the f o l l o w i n g s t e p s .

2.

Attached i s a l i s t o f t h e graduate s t u d e n t s who c o n s t i t u t e
the r e se a r c h population f o r t h i s deportment. P le a s e review
t h i s l i s t and c i r c l e t h e names o f a l l s t u d e n t s w ith whom you
have had s u f f i c i e n t c o n t a c t to respond t o a t l e a s t fo u r o f
the items on the r a tin g s h e e t .

3.

From the l i s t o f names you have c i r c l e d in Step 2 , s e l e c t th e
one s t u d e n t whom you f e e l b e s t f i t s t h e "most able" category
on item 10, "Over-All Evaluation".
P l e a s e c i r c l e t h e number
7 in row 10 on that s t u d e n t * s r a tin g s h e e t and p l a c e an "x"
in the lo w er right-hand c o m e r o f t h e s h e e t .

4.

From the l i s t o f names y o u have c i r c l e d i n Stop 2 , s e l e c t the
one s t u d e n t whom you f e e l b e s t f i t s t h e " l e a s t a b le " catego ry
on item 10, "Over-All E valuation".
P l e a s e c i r c l e t h e number
1 in row 10 on that s t u d e n t ' s r a tin g s h e e t and p l a c e an "x"
in the lo w e r l e f t - h a n d c o m e r o f the s h e e t .

5.

Complete item s 1 through 9 on the r a t i n g sh e e t f o r t h e i n d i v i ­
duals i n S te p s 3 and 4 by c i r c l i n g t h e number in t h e appropriate
response column for each item .
NOTE: Item 10 "Over-All Evaluation" i s n o t the Mean o f items
1 through 9 .
It i s a s e p a r a t e item ancT w h i le i t may come out
os the mean you should n o t complete t h e r a t in g s h e e t s u sin g
t h i s assum ption.

6.

Place t h e s h e e t with an "x" in the lower l e f t - h a n d c o m e r on
the l e f t s i d e o f your d e s k . Place t h e s h e e t with an "x" in
the lower right-h an d c o m e r on the r i g h t s i d e o f y ou r desk.
P lease be s u r e the s t u d e n t ' s name i s on t h e s h e e t .
Now s e l e c t
any remaining student from the names you c i r c l e d i n Step 2
and p l a c e h i s sh e e t d i r e c t l y in f r o n t o f you. The arrange­
ment s h o u ld look as f o l l o w s :

□

□
137

□

Using the sh e e t o f the student who rated "Least Able" on
item 10 and the s h e e t o f the stu d en t who rated "Most Able"
on item 10 as r e f e r e n c e p o i n t s , respond to item 1 for the t h i r d
st u d e n t by completin g the f o l l o w i n g statem en t:
"Compared with "John Most Able" whom 1 r a ted as ”x" on
item 1 and with "John Least Able" whom I rated as "y" on
item 1, t h i s stu d en t would r a t e ________ on item 1.
C i r c l e the a p propriate number in the response column for
item 1. Consider the i n t e r v a l s from 1 through 7 as being
equal i n t e r v a l s .
Repeat Step 7 f o r items 2 through 10; p l e a s e make sure you
s u b s t i t u t e the r a t i n g s you gave "John Least Able" and ".John
Most Able" on each item i n t o the r a t i n g state m e n t before
you respond.
Repeat Steps 6 through 8 f o r a l l remaining s t u d e n t s whose
names you have c i r c l e d in Step 2.
A f t e r completin g a s h e e t for each s t u d e n t , s e t them a s id e for
at l e a s t one day and then review Steps 6 through 9 to se e i f
you s t i l l agree wi t h your o r i g i n a l a s se ssm e n t.
Check the s h e e t s to s e c th a t each has the s t u d e n t ' s name.
While the p l a c i n g o f your name on the sh e e t i s o p t i o n a l , I
would remind you t h a t t h e s e s h e e t s w i l l be co n sid e r e d con­
f i d e n t i al and w i l l be used f o r re sear ch purposes o n l y .
P l a c e a l l the m a te ria l ( d i r e c t i o n , l i s t , r a t i n g s h e e t s ) in
t h e e n c l o s e d , s e l f - a d d r e s s e d en v e lo p e and d e p o s i t i t in the
ma i l . Thank you f o r your a s s i s t a n c e .

1 39

TRI ADIC-COMPARISON
Rating Sheet I
Name o f Student________________
Name o f Rater
______________
(option al)

UJ

UJ

CD

o

2

UJ

UJ
UJ

PC

o

2
1. Mastery o f fundamental
knowledge in h i s gen eral
field .
2. Mastery o f r e l e v a n t
r e sea r ch s k i l l s .
3. I n t e l l e c t u a l breadth.
4. O r i g i n a l i t y in r e s e a r c h .
5. Industry and a p p l i c a t i o n
6. E ffectiven ess o f w ritten
reports.
7. E f f e c t i v e n e s s o f o r a l
presen tation s.
8. Promise as a produc­
t i v e re sea r ch worker.
9 . Promise as a t e a c h e r .
10. OVER-ALL
EVALUATION

CD
5

-r.

o
■ /:
i— i

a:

o

»—i

2

o

=£

UJ
0

S

2

■<

0£
UJ
O.
rs
WJ

2

3

4

S

6

2

4
4
4

5
S

6

6

2

3
3
3

S

6

2

3

4

5

6

2

3

4

5

6

2

3

4

S

6

L
CD
C

2

o

NO OPPORTUNITY
TO OBSERVE

P l e a s e read the i n s t r u c t i o n s c a r e f u l l y b e f o r e you attempt t o respond
These r a t i n g s are in te n d ed to show r e l a t i v c s t a n d i n g s .
In s o f a r
as p o s s i b l e the r a t i n g s should r e v e a l the c h a r a c t e r i s t i c s in which
a s t u d e n t i s r e l a t i v e l y s t r o n g and t h o s e in which he i s r e l a t i v e l y
weak in comparison with your s e l f - s e l e c t e d sample.

140
T R 1 A D I C - COMPARI SON
R atin g

Sheet

II
Name o f S t u d e n t ___________________
Name o f R a t e r
(o p tio n al)

___________________

P le a se read the i n s tr u c tio n s c a r e f u l ly b e fo re you a tte m p t to re sp o n d ,
T h e s e r a t i n g s a r e i n t e n d e d t o show r e l a t i v e s t a n d i n g s .
I n s o f a r as
p o s s i b l e t h e r a t i n g s s h o u ld r e v e a l t h e c h a r a c t e r i s t i c s in w hich
a s t u d e n t i s r e l a t i v e l y s t r o n g and t h o s e in w hich h e i s r e l a t i v e l y
weak i n c o m p a riso n w i t h y o u r s e l f - s e l e c t e d s a m p le .
NO
MOST OPPORTUNITY
LEAST
TO OBSERVE
AB
ABLE
1. M a s t e r y o f f u n ­
d a m e n ta l know ledge
in h is g e n e ra l f i e l d .

0

2. M aste ry o f r e l e ­
vant research s k ills .

0

3. I n t e l l e c t u a l
b read th .
4. O r ig in a l it y
research.

in

0

5. I n d u s t r y and
ap p licatio n .

0

6. E ffectiv en ess
of w ritten re­
p o rts .

0

7. E f f e c t i v e n e s s
o f o ra l p resen ta­
tio n s

0

8. P r o m is e as a
p ro d u ctiv e re ­
s e a rc h w orker.
9. P ro m ise
teach er.
10.

as

OVER-ALL
EVALUATION

a

0

APPENDI X B

MULTIPLE REGRESSION ANALYSIS DATA
D epartm ent o f S o c io lo g y

141

TABLE B.l
M u lti p le Regression Analysis Data - Department o f S o c i o l o g y
Admission Placement Decision

M ultiple Correlation = .56
Variance
= .52
Regression ANOVA p <.0005

P r e d ic to r

Regression
C oefficient

Beta
Weight

Beta S i g n i f i c a n c e
Level

P a r t ia l
Correlation
Coefficient

Rk

Delete

Constant

10.3221

GRE Verbal

- .0 03 3*

-.28

<.003

-.29

.25

GRE Q u a n tit a tiv e

- .0 0 1 5

-.14

<.139

-.15

.30

Undergraduate GPA

- .3 2 6 0

-.11

<.352

-.09

.31

S o c i o l o g y GPA

- .9 4 2 2

-.29

<.018

-.24

.28

Quali ty Rating

-.1878

-.24

<.010

-.25

*

The n e g a t iv e s ig n s are a fu nc ti on o f gr ou p in g-- the admission d e c i s i o n i s numbered
from 1 as the top score to 5 as the bottom sc ore whereas the p r e d ic t o r s are scored
high t o low.

TABLE B.2
Multi p le Regression Analysis Data - Department o f So ciolog y
Graduate Grade-Point Average

M ultiple Correlation = .50
Variance
= .25
Regression ANOVA p <,0005

P r e d ic to r
Constant

GRE Verbal

Regression
Coefficient

Beta
Weight

Beta S i g n i f i c a n c e
Level

Partial
C or re lation
Coefficient

,
R“
Delet e

3.0294

.0001

,03

<.601

.03

.25

- .0 0 0 5

-.20

<.043

-.20

.22

Undergraduate GPA

.0134

.02

<.869

.02

.25

S o c io lo g y GPA

.2515

.35

<.008

.26

.19

Quali ty Rating

.0276

.16

<.110

.16

.23

-.23

.21

GRE Q u a n t i t a t i v e

Admission Placement
*

- .0 53 6*

-.24

<.023

The n e g a t iv e s ig n s f o r the Admission Placement in th e Appendix t a b l e s do not imply an i n ­
v e r s e r e l a t i o n s h i p . They are a fu nc ti on o f group numbering, i . e . 1 to N while a l l other
p r e d i c t o r s are N to 1.

TABLE B .3
Multi p le Regression An alysis Data - Department o f S o cio lo g y
O r i g i n a l i t y in Research
M u lti p le Corr el ation = .39
Variance
= .IS
Regression ANOVA p <.011

Pr e d ic to r

Regression
Coefficient

Constant

2.8200

Beta
Weight

Partial
Beta S i g n i f i c a n c e Corr el ation
Level
Coefficient

R“
Delete

GRE Verbal

.0017

.15

<.184

.13

.14

GRE Q u a n tit a tiv e

.0002

.02

<.874

.02

.15

Undergraduate GPA

.0768

.02

<.847

.02

.15

S o cio lo g y GPA

.3402

.10

<.456

.0 8

.15

Quality Rating

.0220

.03

<.794

-.03

.15

-.2480

-.2 5

<.032

-.2 1

.11

Admission Placement

TABLE B.4
M u ltip le Regression Analysis Data - Department o f S oc iology
E f f e c t i v e n e s s o f Written Reports

M ultiple Correlation = .43
Variance
= .18
Regression ANOVA p <.003

Pr edic to r
Constant

Beta
Weight

Beta S i g n i f i c a n c e
Level

Partial
Correlation
Coefficient

2

R
Delete

2.2249

,0026

.24

<.027

.22

.14

- .0 0 2 2

.22

<.035

-.21

.14

Undergraduate GPA

.2706

.09

<.449

.08

.18

S o c io lo g y GPA

.5205

.17

<.205

.15

.17

Quality Rating

.0039

<.01

<.959

.01

.18

-.1316

-.1 4

<.201

- .1 3

.17

GRE Q u a n tit a ti v e

Admission Placement

145

GRE Verbal

Regression
C oefficient

TABLE B.5
M ultiple Regression Analysis Data - Department o f Sociology
Promise as a Productive Research Worker
M ultiple Correlation = .35
Variance
= .13
Regression ANOVA p <.037

P r e d ic t o r

Regression
C oefficient

Constant

1.4691

GRE Verbal

Beta
Weight

Beta S i g n i f i c a n c e
Level

P a r t ia l
C orr elatio n
Coefficient

R
Delete

.0017

.14

<.226

.12

.11

<.0001

<.01

<.997

<.01

.13

Undergraduate GPA

.0846

.0 3

<.844

.02

.13

S o c io l o g y GPA

.6175

.18

<.212

.1 3

.11

Qu al it y Rating

.0181

.02

<.842

.02

.13

-.1735

-.1 6

<.162

-.1 4

.11

GRE Q u a n tit a ti v e

Admission Placement

TABLE B.6
M ultiple Regression Analysis Data - Department o f Sociology
Overall Evaluation
M ultiple Correlation = .39
Variance
= .15
Regression AXOVA p <.012

P r e d ic to r

Regression
Coefficient

Constant

.7081

GRE Verbal

.0023

.1 7

<.129

.15

.13

GRE Q u a n t i t a t i v e

- .0 0 0 8

-.06

<.550

-.06

.15

Undergraduate GPA

-.0111

-.01

<.981

-.01

.15

S o cio lo g y GPA

.9956

.27

<.055

.19

.12

Qu al it y Rating

.0466

.05

<.625

.05

.15

-.1432

- .1 3

<.268

-.1 1

.14

Admission Placement

Beta
Weight

Beta S i g n i f i c a n c e
Level

P a r t ia l
Corr el ation
Coefficient

R"
Delete

appen d ix

c

MULTIPLE REGRESSION ANALYSIS DATA
D cpnrtm cnt o f P sy c h o lo g y

148

TABLE C.l
M ultiple Regression Analysis Data - Department o f Psychology
Graduate Grade-Point Average
M ultiple Correlation = .36
Variance
= .13
Regression ANOVA p <.007
Beta S i g n i f i c a n c e
Level

r

P r e d ic to r

Regression
C oefficient

Constant

2.9618

GRE Verbal

<.0001

.01

<.907

.01

.13

GRE Q u a n ti t a ti v e

.0001

.03

<.732

.03

.13

Undergraduate GPA

.0132

.02

<.834

.12

.13

Psychology GPA

.0891

.15

<.170

.12

.12

Quality Rating

.0285

.18

<.029

.18

.10

Advanced Test

.0006

.18

<.080

.14

.11

-.0145

-.07

<.409

-.07

.12

Natural Scien ce
Courses

Delei

tifrl

Beta
Weight

P a r t ia l
C orr elat ion
C oefficient

TABLE C.2
M u lti ple Regression Analysis Data - Department o f Psychology
Mastery o f Fundamental Knowledge

M ultiple Correlation - .38
Variance
= .15
Regression ANOVA
p <.002

P r e d ic to r

Regression
Coefficient

Beta
Weight

Beta S i g n i f i c a n c e
Level

Partial
Cor re lation
Coefficient

R2
Delel

Constant

.3720

GRE VeTbal

.0003

.02

<.859

.02

.15

GRE Q u a n ti t a ti v e

- .0 0 0 5

-.04

<.680

-.04

.15

Undergraduate GPA

- .6 0 9 0

-.17

<.111

-.13

.15

Psychology GPA

.8959

.24

<.023

.19

.11

Qu al it y Rating

-.0 2 0 6

-.02

<.792

-.02

.15

.0065

.31

<.002

.26

.09

-.0983

-.08

<.356

-.08

.14

Advanced Test
Natural S cien ce
Courses

TABLE C.3
M ultiple Regression Analysis Data - Department o f Psychology
E ffectiv en ess o f Written Reports
Multi p le Co rr el atio n = .33
Variance
= .11
Regression ANOVA
p <.025

P r e d ic to r
Constant

Regression
Coefficient

Beta
Weight

Beta S i g n i f i c a n c e
Level

P a r t ia l
C orr elatio n
Coefficient

^
R
Delete

.6296

GRE Verbal

.0006

.04

<.735

.0 3

.11

GRE Q u a n tit a tiv e

.0001

.01

<.909

.01

.11

-.4 9 5 4

-.12

<.247

-.10

.10

Psychology GPA

.8734

.21

<.049

.17

.08

Quali ty Rating

.0792

.07

<.368

. 08

.10

Advanced Te st

.0045

.19

<.059

.16

.08

-.1848

-.14

<.123

-.13

.09

Undergraduate GPA

Natural S cience
Courses

TABLE C.4
M ultiple Regression Analysis Data - Department o f Psychology
Promise as a Productive Research Worker
M ultiple Correlation = .30
Variance
= .09
Regression ANOVA
p = .05

P r e d ic to r
Constant

Regression
Coefficient

Beta
Weight

Beta S i g n i f i c a n c e
Level

Partial
Correlation
Coefficient

r
1

Delei

.3810

-.0003

-.02

<.870

-.01

.09

.0001

.01

<.948

.01

.09

-.5 7 1 7

-.13

<.236

-.10

.08

Psychology GPA

.8866

.19

<.075

.15

.07

Quality Rating

.0432

.04

<.662

.04

.09

Advanced Test

.0056

.22

<.034

.18

.06

-.2237

-.15

<.098

-.14

.07

GRE Verbal
GRE Q u a n ti tati ve
Undergraduate GPA

Natural Scien ce
Courses

TABLE C.5
M ultiple Regression Analysis Data - Department o f Psychology'
Overall Evaluation
M ultiple Correlation = .35
Variance
= .13
Regression ANOVA
p <.006

P r ed ic to r

Regression
C oefficient

Beta
Weight

Beta S i g n i f i c a n c e
Level

Partial
Corre lation
Coefficient

R2
Delel

Constant

.6848

GRE Verbal

.0006

.03

<.754

.03

.13

GRE Q u a n ti t a ti v e

.0001

.01

<.951

.01

.13

Undergraduate GPA

- .8 0 0 7

-.20

<.056

-.16

.10

Psychology GPA

1.0500

.26

<.015

.20

.09

Quali ty Rating

.0177

.02

<.836

.02

.13

Advanced Test

.0052

.23

<.025

.19

.09

- .2 1 5 9

-.16

<.065

-.16

.10

Natural Scien ce
Courses