:1 .d
fig; ..

”Ms
hm 8%!330“ 2312

.hl

28W.

812‘,

SE9, ¥ ‘3;
. “l l 1

 

 

 

 

 

THE EVALUATION OF A TEACHER TRAINING PROGRAM

BY

Kim Kanaga

A DISSERTATION

Submitted to
Michigan State University
in partial fulfillment of the requirements
for the degree of

DOCTOR OF PHILOSOPHY

Department of Communication

1978

(“$90900

Accepted by the faculty of the Department of
Communication, College of Communication Arts and Sciences,
Michigan State University, in partial fulfillment of the

requirements for the Doctor of Philosophy Degree.

{Difyctor of Dissertation

:’ / .
Guidance Committee: £23—~4\<E§£;1& , Chairman

ficmw, Lu,
ADM <3J M
yDawn 1. Meg

 

ABSTRACT
THE EVALUATION OF A TEACHER TRAINING PROGRAM

BY

Kim Kanaga

The evaluation of instruction has traditionally been a
controversial issue among educators. While there are several
plausible explanations for this situation, one important con-
tributing factor certainly appears to be the lack of adequate
procedures for assessing instructional performance. Of the
current practices in instructional evaluation, students appear
to be the most promising source of information because they
have much more exposure to the instructor in the classroom
setting, they are recipients of the teaching process and their
numbers are usually of sufficient size to minimize biases re-
sulting from individual differences. Student rating forms are a
quite widely used means of formally gathering evaluative infor-
mation. However, they only indirectly tap into the teaching-
learning process by examining various underlying components
which are assumed to be a reflection of teaching effectiveness.
A more appropriate and seldom used means of determining teach-
ing effectiveness is to examine student learning. Learning is,
after all, the goal of teaching.

A cognitive-mediation oriented approach to student learn-
ing appears to be the most fruitful for the purpose of evalu-
ating instruction. From such a perspective, learning can be

defined as the evolution of meaning. It would follow that the

Kim Kanaga
function of teaching would be to develop consensually shared
meanings for the basic concepts that define a Specific do-
main of instruction. Metric multidimensional scaling pro-
vides a precise and reliable set of procedures for assessing
the meanings for such a set of concepts over time. The ex-
periment reported here was designed to provide an initial
test of these procedures.

The current study evaluates a training program for
teaching assistants by examining the teaching effectiveness
of a group of trained assistants in contrast to a group of
assistants that were not trained. Student learning and per-
ceptions of actual classroom performance were used as the
criteria for the evaluation. Data was gathered from student
volunteers at four different times during the term.

Prior to discussing the results of this study, sever-
al problems should be pointed out. The particular data set
used here is really not appropriate for the analyses that
were performed on them. Specifically, every available stu-
dent was asked to participate at each data collection point.
The samples were thus not randomly drawn but the data was in-
stead correlated to the extent that some subjects partici-
pated in more than one data collection. Moreover, the data
was correlated within time intervals since some courses had
more than one teaching assistant. As a result, the multiple
responses by students in these courses were not independent.

The analyses performed does, nevertheless for illustrative

ACKNOWLEDGMENTS

I wish to express my sincere gratitude to a number of
people whose contributions and support were instrumental in
the completion of this dissertation. Above all, I would
like to thank my advisor and dissertation director, Dr.
Joseph Woelfel. His knowledge, guidance, and understanding
have been an inspiration to me. For these things I am deeply
grateful. Most of all, I am grateful to Joe Woelfel for
having the flexibility to be there when I needed him and to
otherwise allow me to find my own way.

Thanks are also due to the other members of my super-
visory committee. Dr. Erwin Bettinghaus, Dr. James Phillips
and Dr. David Ralph. Their assistance always seemed to come
at the right times. In particular, Dr. Bettinghaus and Dr.
Ralph provided the departmental approval for the study to be
conducted and backed it through some occasionally difficult
times. For this, I am indebted. It is also with a great
deal of appreciation that I thank Dr. Larry Barker, my ad-
visor for both my bachelors and masters degrees. It was his
initial help and encouragement that guided me in the direc-

tion which ultimately led to this dissertation.

ii

The success of this study was dependent on the co-
operation of a large number of people. All of the Com 100
instructors cooperated in allowing me to come into their
classrooms to gather data. The teaching assistants very
courageously permitted their classroom effectiveness to be
examined. The student respondents put several hours into
filling out questionnaires. In actually collecting and cod-
ing the data; several undergraduate assistants were extremely
helpful. In particular, Donna Paquette, Marianne Mnich,
Cliff DeMarco and Ilene Benison put a great deal of time and
energy into this study. Rick Holmes provided valuable com-
puter assistance and Mrs. Ruth Langenbacher did her always
excellent job in preparing this manuscript. Without the
effort put forth by all of these people, this study would
not have been possible. To each one of them, I am very
grateful.

I also owe a great deal to those peOple who have
been closest to me, my family. My parents, Kent and Phyllis
Kanaga, have always had confidence in me and have been sup-
portive of me in the things that I've done. My wife Mary
and son Jeremy have put up with my absense and frustrations
over the past several years. I cannot express what their
love and understanding has meant to me. To them, I dedicate

this dissertation.

iii

TABLE OF CONTENTS

Chapter Page
I INTRODUCTION . . . . . . . . l
Purposes for Evaluating Instruction . 5

Current Practices in the Evaluation
of Instruction . . . . . 7
Administrator Evaluation . . . 8
Colleague Evaluation . . . . 10
Self Evaluation . . . . . 11
Student Evaluation . . . . 12
Organization of the Report. . . . 16
II THEORY AND MEASUREMENT. . . . . . 18
Learning Theory . . 19
Measurement of Teaching Effectiveness . 28
Advantages . . . . . . 38
Limitations . . . . . . 40
III EXPERIMENTAL METHODS AND PROCEDURES . . 42
Undergraduate Teaching Assistants . . 43

Guidelines for the Function and
Evaluation of the Undergraduate
Teaching Assistants in Communica-

tion 100. . . . . . . . 45
The Teaching Assistants Training
Program . . . . . . . . 47
Session One . . . . . . 47
Session Two . . . . . . 48
Session Three. . . . . . 49
Session Four . . . . . 49
The Introductory Communication Course . 50
Measuring Instrument . . . . . 51
Research Design . . . . . . 55

iv

Chapter Page

IV RESULTS. . . . . . . . . . 59
Precision of Measure . . . . . 64
Manipulations . . . . . . . 66
Student Learning . . . . . . 69
Instructional Performance . . . . 81
Adjunct Analyses . . . . . . 94

Course Context . . . . . . 95

Instruction. . . . . . . 99

Summary of Results . . . . . . 102

V DISCUSSION, RECOMMENDATIONS, AND SUMMARY . 104
Discussion . . . . . . . . 104

Future Research. . . . . . . 112

Summary of Report . . . . . . 116

APPENDICES . . . . . . . . . . 118

A: Questionnaire . . . . . . . 118
B: Spatial Coordinates Matrices for

the Experimental and Control Groups

at Four Points in Time . . . . 127
C: Descriptive Statistics of Students

from Experimental and Control

Group Sections . . . . . . . 143

REFERENCES . . . . . . . . . . 144

Table

LIST OF TABLES

Sample Sizes per Section for the Experi-
mental and Control Groups at Four Time
Intervals. . . . . . . . .

Coefficients of Variability (V = 100
(S/X)) O O O ' O O O O O 0

Means and Standard Deviations for the
Variability Coefficients of Each Concept

Mean Distances, Standard Deviations, and
Sample Sizes Among Pairs of Content Concepts
for Experimental and Control Groups at Time
one 0 O O O O O O O O 0

Mean Distances, Standard Deviations, and
Sample Sizes Among Pairs of Content Concepts
for Experimental and Control Groups at Time
Two 0 O O O O O O O O 0

Mean Distances, Standard Deviations, and
Sample Sizes Among Pairs of Content Concepts
for Experimental and Control Groups at Time
Three. . . . . . . . . .

Mean Distances, Standard Deviations, and
Sample Sizes Among Pairs of Content Concepts
for Experimental and Control Groups at Time
Four 0 O O O O O O O O 0

Mean Distances, Standard Deviations, and
Sample Sizes Among Pairs of Instructional
Concepts for Experimental and Control
Groups at Time One . . . . . .

Mean Distances, Standard Deviations, and
Sample Sizes Among Pairs of Instructional
Concepts for Experimental and Control
Groups at Time Two . . . . . .

vi

Page

61

65

69

71

72

73

74

82

83

Table Page

10. Mean Distances, Standard Deviations, and
Sample Sizes Among Pairs of Instructional
Concepts for Experimental and Control
Groups at Time Three . . . . . . . 84

11. Mean Distances, Standard Deviations, and
Sample Sizes Among Pairs of Instructional
Concepts for Experimental and Control
Groups at Time Four . . . . . . . 85

12. Mean Distances, Standard Deviations, and
Sample Sizes Among Pairs of Content Concepts
for the Control Group Across Four Points
in Time . . . . . . . . . . 96

13. Mean Distances, Standard Deviations, and
Sample Sizes Among Pairs of Content Concepts
for the Experimental Groups Across Four
Points in Time . . . . . . . . 97

14. Mean Distances, Standard Deviations, and
Sample Sizes Among Pairs of Instructional
Concepts for the Control Group Across
Four Points in Time . . . . . . . 100

15. Mean Distances, Standard Deviations, and
Sample Sizes Among Pairs of Instructional
Concepts for the Experimental Group Across
Four Points in Time . . . . . . . 101

16. Spatial Coordinates of First Eight Dimensions
for Control Group at Time One. . . . . 127

17. Spatial Coordinates of First Eight Dimensions
for Experimental Group at Time One . . . 129

18. Spatial Coordinates of First Eight Dimensions
for Control Group at Time Two. . . . . 131

19. Spatial Coordinates of First Eight Dimensions
for Experimental Group at Time Two . . . 133

20. Spatial Coordinates of First Eight Dimensions
for Control Group at Time Three . . . . 135

21. Spatial Coordinates of First Eight Dimensions
for Experimental Group at Time Three . . . 137

vii

Table

22.

23.

24.

Page

Spatial Coordinates of First Eight Dimensions
for Control Group at Time Four. . . . . I39

Spatial Coordinates of First Eight Dimensions
for Experimental Group at Time Four . . . 141

Descriptive Statistics of Students from
Experimental and Control Group Sections . . I43

viii

Figure

1.

LIST OF FIGURES

Plot of the Communication-Social Science
Paired Comparison for Experimental and
Control Groups Across Four Points in Time

Plot of the Communication-Good Paired
Comparison for Experimental and Control
Groups Across Four Points in Time . .

Plot of the Communication-Me Paired
Comparison for Experimental and Control
Groups Across Four Points in Time . .

Plot of the Teaching Assistant-Leadership
Paired Comparison for Experimental and
Control Groups Across Four Points in Time

Plot of the Teaching Assistant-Instructor
Paired Comparison for Experimental and
Control Groups Across Four Points in Time

Plot of the Trace (Total Variance) from the

Coordinates Matrix for Experimental and
Control Groups Across Four Points in Time

xi

Page

75

76

79

88

89

107

CHAPTER I

INTRODUCTION

The evaluation of instruction has traditionally been
a controversial issue among educators (Doyle, 1975; Fenker,
1975; Kent, 1966; Lewis, 1975; Miller, 1972)- While most in-
structors would probably agree that there are potential ad-
vantages in carrying out such evaluations, Gray (1969) points
out that they are usually less than enthusiastic about having
their own classroom performance examined. This is particu-
larly the case in higher education where the reasons for this
lack of enthusiasm are quite evident. At the college level,
very few instructors receive any sort of formal training
(Doyle, 1975). In the absence of such training, instructors
are left to either develop their own teaching skills or model
their teaching after instructors from which they have had
courses. The underlying assumption is that the instructor
has, for certain courses, the necessary substantive knowledge
and should be able to convey that information to students
without much difficulty. The resistance to the evaluation of
instruction may, then, be partially due to the instructor's
insight into this deficiency in their own instructional

training.

2

Another issue contributing to this controversial sit-
uation is the current status of the reward structure within
institutions of higher education. Retention, promotion,
tenure, and pay raises at many major universities tend to
be based more on scholarly productivity than on instructional
achievements (Astin and Lee, 1966; Doyle, 1975; Fenker, 1975;
Hammond, Meyer, and Miller, 1969; Luthans, 1967). Even when
teaching is taken into consideration, it is not unusual for
the quantity of contact with students to be emphasized
(Astin and Lee, 1966). Committee membership and other such
responsibilities, in and of themselves, are usually not
weighted heavily in the reward system. They do, however,
involve working with peers and as such would be expected to
elevate their importance to faculty members. Moreover, con-
sulting and other outside activities typically provide sub-
stantial incentives. It would appear, then, that university
systems are not structured to promote quality instruction.
Instead, the emphasis on other responsibilities would seem to
lower the priority placed on developing and maintaining
superior teaching skills.

The lack of enthusiasm among instructors may, in fact,
reflect their dissatisfaction with the outcomes resulting
from current evaluation practices. Hence, instructors may
see potential advantages to evaluating classroom perform-
ance, but have yet to see any of those advantages realized.
Although administrators may now be at least looking at evalua-

tion forms, they still appear to have little faith in them

3
(Astin and Lee, 1966; Lewis, 1975). Furthermore, the infor-
mation provided by present evaluation forms is of limited
value to instructors who wish to use them for assistance in
improving their teaching skills. Typically, these forms rate
the instructor on various criteria (Genova, Madoff, Chin, and
Thomas, 1976). The instructors are then informed as to how
they rate in comparison with other instructors. In some cir-
cumstances, there is no explicit indication of whether the
individual instructor's position is more or less effective
than the other instructors, but merely how much they differ
from other instructors. Even if specific strengths and
weaknesses were precisely pinpointed, these forms rarely
contain any recommendations for how specific teaching skills
may be improved. Thus, the skepticism surrounding instruc-
tional evaluation may be, in part, due to the failure of
current evaluation procedures to produce any meaningful
results.

This latter point suggests that at least some of the
problems associated with instructional evaluations may be
found in the instrumentation rather than the lack of concern
among instructors and administrators. An extensive survey
conducted by the American Council on Education (Astin and
Lee, 1966) revealed that systematic measuring instruments
are, in fact, not a widely used means of evaluating classroom
performance. Instead, they found that by far the most fre-
quent sources of information are the subjective judgments of

department Chairpersons (85.1%) and deans (82.3%) followed

f’

4
distantly by scholarly productivity (43.8%) and the informal
opinions of colleagues (48.9%) and students (41.2%). As
Gustand (1961) put it, ". . . the most frequently cited
sources of information can be described simply as hearsay."

The extensive use of such imprecise procedures may,
very well, be an indication of the quality of the measuring
instruments that are currently available. Doyle (1975) and
other contemporary educational researchers (Blackburn and
Clark, 1975; Kauffman, Hallahan, Payne, and Ball, 1973; Lewis,
1975; and Miller, 1972) have suggested that current evaluation
procedures are inadequate. This conclusion was also reached
by the Commission on Academic Tenure in Higher Education
(1973). In their report, they recommend that "professional
associations and research centers continue and expand their
efforts to develop more sophisticated conceptions of teaching
effectiveness and more reliable methods of evaluating teach-
ing . . . ." The research reported in this paper is such an
undertaking. More specifically, the experiment reported here
was designed to provide an initial test of a new set of pro-
cedures for evaluating instruction.

The remainder of this chapter will first discuss the
purposes for evaluating instruction. The various types of
instructional evaluations that are currently in use will then
be critically examined. One specific type, namely student
evaluations, will be singled out as the most fruitful means

of evaluating instruction. The problems in this area will be

5
thoroughly reviewed. The chapter will then conclude by lay-
ing out the organization for the remainder of this disserta-

tion.

Purposes for Evaluating Instruction

Evaluations are generally conducted to examine some
program or activity. More specifically, they attempt to de-
termine the extent to which some objective or objectives are
being accomplished. The primary concern for instructional
evaluations is the assessment of the instructors success in
maximizing student learning of course material. While there
are a wide variety of specific purposes for which the infor-
mation obtained from such assessments may be used, Doyle
(1975) points out that they typically fall within the follow-
ing four basic categories: administrative decisions, self-
improvement, criterion for research, and advising students.

Administrative decisions can be aided by the meaning-
ful evaluation of instruction. As in most organizations,
someone or some group within the educational system has the
responsibility for the distribution of internal rewards and
punishments. Since instruction is a primary responsibility
of most faculty members, precise information regarding their
effectiveness as instructors would seem to be a highly de-
Sirable and useful commodity for those who have to hire, fire
and promote. Considering trends in student enrollment and in
available financial support for higher education, this infor-

mation should become even more important in the future.

6

Evaluations can also be useful to the instructors
themselves. Scholars of communication have long recognized
the value of feedback in modifying inappropriate or ineffec-
tive behaviors (Barker, 1971; Mortensen, 1972). Friedrich,
Galvin and Book (1976) discuss several cases in which syste-
matically manipulated feedback such as head nods, smiles,
verbal praise, etc. was successfully used to change specific
instructional behaviors. While such direct behavioral modi-
fications are beyond the sc0pe of instructional evaluations,
feedback provided by evaluations can be used to help instruc-
tors identify instructional skills that need improvement.

The third purpose for evaluating instruction that Doyle
mentions is in the area of educational research. Since the
ultimate goal of educational research would presumably be to
better understand the educational process so that improvements
can eventually be made, it would seem that a means of pre-
cisely determining how effective instructors are in their
classrooms would be of great value. That is, meaningful in-
formation regarding the teaching-learning process could, if
used constructively, be beneficial to the field of education.

Finally, Doyle points out that students and academic
advisors have recently taken an increased interest in obtain-
ing information from instructional evaluations. In most edu-
cational institutions, the only formal information provided
for students and their advisors is information regarding the
content that will be covered in various courses. Any infor-

mation about specific instructors is usually obtained through

7
informal and often inaccurate or incomplete channels. The
manner in which an instructor presents course material can
be an important determinant of how much student learning will
occur. As the consumer of the educational process, it is not
surprising to see students concerned with obtaining informa-
tion about prospective instructors and their styles of in-
struction. Seeking more formal and precise information does
not seem unreasonable and a more concentrated effort to do
so should not be unexpected.

Most evaluation instruments currently in use were
developed to serve one or possibly two of the four purposes
mentioned. While even the most precise and thorough evalua-
tion procedures may not be able to adequately fulfill each of
these purposes, an attempt in this direction should still be

made.

Current Practices in the Evaluation of Instruction

There are probably as many different instructional
rating instruments as there are colleges and universities
which employ them. Each one purports to tap the most essen-
tial characteristics of teaching effectiveness. It would
follow that if these instruments are, in fact, measuring the
same phenomena, there should be some commonality among them.
While some of the instruments are similar in appearance and
claim to be measuring the same dimensions (Doyle, 1975), the

items used to tap these dimensions are quite different (Genova

8

et: al., 1976). Since the items are not comparable from in-
strument to instrument, it is quite possible that even if they
were examining the same dimensions, they may be focusing on a
different range of those dimensions. Still other instruments
are quite diversified in terms of the source and type of in-
formation they obtain. Consequently, a meaningful discussion
of current evaluation instruments as a whole is questionable.
Instead, these instruments can be more clearly examined with-
in the context of the following four categories: administra-
tor evaluation, colleague evaluation, self evaluation, and

student evaluation.

Administrator Evaluation

 

In most cases, it is ultimately the administrators re-
sponsibility to evaluate the teaching performances of faculty
members. The administrator is typically asked to make an
overall assessment of teaching effectiveness as well as of
other faculty duties. While more specific information is
probably desirable, the more crucial issue in administrator
evaluation is to determine on what basis such general assess-
ments are made. Most administrators are unable to actually
observe the classroom behavior of their faculty. The reason
for this boils down to a simple matter of priorities. Although
the evaluation of faculty teaching is an administrative concern,
it is only a portion of the responsibilities delegated to a
dean or department chairperson. Consequently, administrators

do not have the time necessary to thoroughly evaluate each

9
faculty member. They must, instead, rely on other sources
of information.

Even if time permitted, the utility of an overall as-
sessment based on the observations of any single individual
is questionable. That is, direct observational ratings of
this sort are based on individual perceptions which are at
least partially determined by past experiences. While ex—
plicitly stated criteria for making ratings may minimize this
potential source of bias, it is difficult, in the absence of
other information, to determine if the criteria have been
appropriately applied by any one individual. Several possible
problems are noteworthy. Administrators faced with such a
task may be partial to certain teaching styles which would
not necessarily render other methods as less effective but
would merely be an indication of personal preferences. It
could also be difficult for them to determine the extent to
which the instructor is facilitating or impeding student learn-
ing of course materials. What is clear and understandable to
the administrator may be vague and incomprehensible to the
students. What may seem as disorganized to the administrator
may instead provide the students with the flexibility needed
to get them more involved in class projects. Another poten-
tial source of bias may result from a difficulty in assessing
teaching performance independently of other components which
comprise the faculty members' overall contribution to the

department and/or college (Blackburn and Clark, 1975). More-

10
over, these and other individual biases could result in a
lack of uniformly applied standards which, in turn, may re-
sult in serious morale and/or legal problems.

This is not to say that these biases will occur or
that they are by any means intentional. But even if adminis-
trators were aware of their biases, they may not be able to
eliminate them and in attempting to do so may cause biases in
other directions. It would appear, then, to be to the admin-
istrators' advantage to seek out the most precise, reliable,
and valid information at their disposal and to utilize that

information in evaluating teaching effectiveness.

Colleague Evaluation

 

Colleague evaluations tend to parallel administrator
evaluations in both purpose and problems. These evaluations
are primarily used to provide administrators with information
regarding the performances of faculty members. While some
peer rating forms (Genova gt_al., 1976) ask for summary infor-
mation, others (Blackburn and Clark, 1971; Miller, 1972) allow
colleagues to make more specific assessments and as such these
instruments may also be used to assist instructors in improv-
ing their teaching skills.

There are several problems which accompany this type
of evaluation. Faculty members are not immune to the biases
that administrators are subject to when directly observing
teaching behavior. In fact, they are probably more prone to

bias. Faculty members have more personal contact with one

11

another and would themselves be in a position to subsequently
be evaluated by those faculty members whom they must evaluate.
However, Genova e_t_i1. (1976) points out that faculty members
rarely visit each others' classroom for the explicit purpose
of assessing teaching performance. As a result, their eval-
uations must also be based on other sources of information.

A more serious issue resulting from peer evaluations
is the impact that they may have on the faculty. Eble (1970),
as well as others (Fenker, 1975; Miller, 1972), has found
strong Opposition to colleague evaluations. Fenker (1975)
reported that resistance was so strong that his attempt to
collect colleague evaluations had to be abandoned. This sug—
gests that, for at least administrative purposes, the cost of

such information may outweight the benefits.

Self Evaluation

 

As evidenced by the lack of attention in the literature,
formal self evaluations are relatively rare in higher educa-
tion. Moreover, the research that is available (Blackburn and
Clark, 1971; Centra, 1972) tends to suggest that instructors
inflate the assessments of their own teaching skills. If, on
the other hand, faculty members were able to accurately assess
their own effectiveness as a teacher, it is questionable
whether individuals should be put in the position of providing
the information that would warrant their own dismissal or
would call for a promotion or pay raise. It may be desirable

to think that professionals could and would make such self-

12
judgments. To make such an assumption is, however, unreal-
istic and would establish a potentially detrimental reward
system.

Formal self evaluation may however, be used as a means
of improving teaching skills. Having instructors seriously
reflect on their teaching behaviors could be quite helpful to
some instructors; particularly those interested in improving
their teaching ability but not knowing how to proceed. Self
evaluation could provide them with some needed direction.
Under these conditions, additional information from other

sources would also be desirable.

Student Evaluation

 

Student rating forms are by far the most widely used
means of formally gathering evaluative information for each
of the purposes previously mentioned (Doyle, 1975). That is,
student evaluations have been used to: (1) provide informa-
tion for administrative purposes, (2) provide diagnostic
feedback for instructors, (3) provide data for educational
researchers, and (4) provide information for students and
their advisers.

The most profound issue regarding student appraisals
of teaching effectiveness is a debate over the students'
ability or competence in making such judgments. While admin-
istrator and colleague evaluations involved assessments by
individuals with at least comparable experiences and training,

the opponents of this type of evaluation argue that students

13
are not qualified to make such assessments (Bryant, 1967;
Hildebrand, 1972). Supporters of student evaluations counter
by arguing that as the consumer of the instructor's efforts,
students are in the best position to actually know the ef-
fectiveness of the instructor (Astin and Lee, 1967; Grush and
Costin, 1975). The issue is, however, contaminated by the
limitations of the evaluation procedures that are presently
in use.

Most educators would probably agree that student re-
sponses based on learning are desirable. But current instru-
ments fail to actually measure student learning as a criterion
for determining teaching effectiveness. Instead, they typi-
cally require students to indicate, on some sort of forced
choice scale, the extent to which a given statement accurate-
ly characterizes a specific instructor. These statements are
intended to reflect the various underlying components which,
when taken as a whole, constitutes teaching effectiveness.
Students are asked to make ratings on such characteristics
as the instructors' flexibility, motivation, rapport, teach-
ing style, and interaction patterns (Doyle, 1975). Extensive
reviews of empirical research by Doyle (1975) and Costin gt
31. (1972) show rather convincingly that students are quite
capable of making such assessments. Although these results
are still occasionally questioned, it becomes more crucial
to determine the extent to which the components are related

to the actual learning that takes place in the classroom.

14
In the absence of direct measures, it is assumed that, col-
lectively, the dimensions tapped by the measuring instrument
are a reflection of the teaching-learning process. The issue,
then, becomes as Rippey (1975) points out, not a question of
whether or not students should be used for the purpose of de-
termining teaching effectiveness, but rather a question of
how to best utilize this potentially valuable source of in-
formation.

In addition to the problems created by the nature of
the information that present evaluation procedures obtain
from students, they also suffer from several methodological
shortcomings. All of the instructional evaluation instruments
that are either reproduced or described in the literature are
subject to at least one of the following limitations:

1) All relevant criteria which prospective
respondents use to evaluate instruction
must be known in advance and/or appro-
priately captured in the measuring in-
strument. Statements characterizing
specific instructional qualities must
relect these criteria. The lack of
agreement regarding what are the essen-
tial criteria and how they are to be
tapped suggests that current instruments
may not measure all of the criteria by
which instruction is evaluated.

2) Respondents are typically required to
assess the instructor separately for each
of the criteria that are employed in the
instrument. Since these criteria are
seldom independent of one another, obtain-
ing additional information regarding the
interrelationships among these criteria
would provide a more complete picture of
the evaluation process.

 

3)

4)

5)

6)

15

There is a general lack of precision of
measurement in instructional evaluation
forms. These instruments extensively
utilize some sort of Likert and/or seman-
tic differential type scales which, at
best, can be considered ordinal measure.
The necessary precision of measurement

is dependent on the phenomenon under con-
sideration. Ambiguous conceptualizations
of teaching effectiveness make this issue
unclear. However, unless evidence is
available to suggest that ordinal scales
would be adequate, more precise measures
should be sought.

Instructional evaluation instruments are
rarely designed with over-time analysis in
mind. Virtually all of the instruments
that are in use are set up as ex-post-facto
measures. If the teaching-learning rela-
tionship is a dynamic one, which few educa-
tors would dispute, it should be examined
across time. End of the term measures only
provide a static and potentially misleading
description of a complex process. Moreover,
they come too late for instructional modi-
fications to be made.

Criteria that are explicitly stated in

the form of statements concerning specific
instructional qualities runs the risk of
being quite easily distorted by respondents.
Administrators, colleagues, and students all
have vested interests in the effectiveness
of the instructors they are asked to eval-
uate. Consequently, the ability to bias
their responses should always be minimized.

In order to determine just how effective an
instructor is, what constitutes effective
instruction must first be identified. While
much attention has been focused on what
criteria ought to be incorporated into an
instrument, there is usually no provisions
made for determining some Optimal level of
performance for each of these criteria.
Without an explicit standard for comparison,
interpretation of any results would be at
least somewhat questionable.

 

16

In summary, it has been pointed out that assessments
of teaching effectiveness have been used to provide informa-
tion for administrative purposes, for the improvement of
teaching skills, for educational research, and for counseling
students. As sources of information, it was suggested that
administrators and colleagues are unable to invest the time
necessary to thoroughly evaluate teaching effectiveness and
are highly susceptible to a number of biases. Self evalua-
tion was found to be impractical. Students appear to be the
:most promising source of information because they have much
Inore exposure to the instructor in the classroom setting,
-they are recipients of the teaching process, and their numbers
Eire usually of sufficient size to minimize error resulting
from individual differences. However, until a precise means
of measuring student learning is developed, their potential
value will not be fully realized.

It seems quite clear, then, that if the formal evalua—
tjxon.of instruction is to play a vital role in the education-
a1 lprocess, the teaching-learning relationship must be the
focnis of theoretical deve10pment and a measurement system.not

suffering from the limitations of current procedures must be
devised.
Organization of the Report

The present chapter provides an introduction to the

problem under investigation as well as a specific direction

 

17
in which a solution to the problem will be pursued. The
remainder of the report is separated into four additional
chapters. In Chapter II, a theoretical framework and mea-
surement system are developed. Chapter III contains a de-
scription of the experimental methods and procedures used
in the study. The results of that study are then presented

in Chapter IV. Chapter V concludes with a discussion of the

study, summary, and recommendations for future research in

-this area.

 

CHAPTER II

THEORY AND MEASUREMENT

In pointing out that "the ultimate criterion of good
teaching is student learning," McKeachie, Lin, and Mann (1971)
are not making any startling revelation. They are merely
stating what educators have known for a long time. Learning
is, after all, the purpose of teaching (Tyler, 1958; Cohen
and Brawer, 1969). It has, however, rarely been used as a
determinant of teaching effectiveness because traditional
measures of learning are imprecise (Kauffman, Hallahan, Payne,
and Ball, 1973) and not conveniently applied in the educa-
tional environment. Before turning attention to the theoreti-
cal under-pinnings of learning measures, it would be informa-
tive to first discuss the means that have been used to measure
student learning when such attempts are made. One means of
getting at learning has been to tap into the grading system
(Costin fl. , 1972). While grades are intended to differ-
entiate levels of student learning, they do not necessarily
reflect the instructors' teaching effectiveness. The assign-
ment of student grades may also indicate the quality of stu-
dents in the course, the difficulty of the course material, as
well as the instructors' standards for assessing student per-

f0nuance. For example, a final exam in which all students

18

 

19
correctly respond to all items does not necessarily mean that
the instructor was maximally effective. It would probably
suggest that the instructor was unable to construct a test
that adequately discriminated between various levels of stu-
dent learning. Conversely, an instructor assigning abnormally
low grades may be very effective but rigorous. Standardized
achievement tests have also been used to determine student
learning (Gessner, 1973). However, they often lack flexibil-
ity and are not available for all subject matters. The most
:rarely used means of measuring student learning has been to
Inerely ask students what or how much they have learned. Even
tflmough researchers (Doyle and Whitely, 1974; Frey, 1973) have
ftrund that students' responses to these types of questions are
positively related to final exam scores, their accuracy in

making such direct judgments is somewhat questionable. More-

ovexr, their responses are quite easily distorted. Although
these latter two types of measures have shown some promise,
they are still not without serious limitations. This lack of

adequate measures of classroom learning can be traced to the

theories from which the measurement systems were derived.

Learning Theory

The most influential theories of learning (Bolles,
1975; JHilgard and Bower, 1975; Pittenger and Gooding, 1971;
Snelbecker, 1974) have been behaviorally based in the tradi-

tions (of Thorndike, Pavlov, and Skinner. These theories

20
generally define learning as a relatively stable change in a
behavioral pattern resulting from repeated exposure to some
external stimuli (Snelbecker, 1974). They were comprehen-
sively designed to encompass learning in all animals and as
such were specifically tested on rats, pigeons, and monkeys
with the express purpose of generalizing the results to
humans. That is, it was felt that if other animals adhere
to these principals of learning, humans, despite their cog-
nitive potential, must also be subject to them.

To use such a theoretical framework in formal educa-
tion would require specific behavioral outcomes for all course
material to be identified and measured. That is, learning
could only be detected by observing whether the appropriate
response followed exposure to some specified evoking stimuli.
Consequently, an assessment of teaching effectiveness must
take into account the specific student responses that would
be expected to result from exposure to course material. More-
over, procedures for making observations of these responses
would have to be established. While this may be a desirable
goal, an evaluation system based on this information must
either assume comparability of outcomes and measures, or make
some allowances for differences. In other words, it may be
difficult to compare the teaching effectiveness of a typing
instructor whose students have improved their typing efficien-
cy with an art history instructor whose students have become

able to identify specific paintings and sculptures with a

21
calculus instructor whose students have increased their
ability to solve differential equations. Differences in
specific student responses which reflect learning may prevent
any meaningful comparison of the instructor's effectiveness.
A more serious problem has to do with the incompatibility of
the behavioralist learning theory and the typical way courses
proceed. Course material is usually covered continuously by
topics. Once taught, topics are not frequently repeated.
Consequently, the repeated exposure to stimuli, which is usual-
ly a condition for measuring learning, is quite often not
present. Moreover, behavioral learning theories typically
call for the response to follow immediately after exposure to
the stimulus. In the classroom situation, it is not uncommon
to have weeks go by between the time certain course material
is covered in class and the time that students are expected
to utilize that information. What happens to the information
during this time between stimulus and eventual response is
not easily accounted for by the strictly behavioral perspec-
tive. It seems clear, then, that while these theories may be
applicable in other contexts, they are of limited utility in
evaluating classroom instruction.

A more appropriate theoretical orientation can begin
to be developed by first focusing attention on the work of
Hebb (1958, 1960),Mowrer (1960a, 1960b) and Osgood (1957,
1963). Hebb (1958) contends that there exists thought or

mental processes "which themselves independent of immediate

22
sensory input, collaborate with that input to determine
which of the various possible responses will be made and
when" (p. 107). In other words, these mental processes med-
iate responses to incoming stimuli. What is being suggested
here is essentially a two-stage activity. Stage one in-
volves the internal interpretation of external stimuli. In
stage two, the interpretation serves as an internal stimulus
for a given response. This internal interpretation is what
Osgood (1957) refers to as "meaning" which is gained from an
association made between specific stimuli and the responses
which they have elicited. Put another way, Mowrer (1960a
asserts that, "External events or stimuli influence and de-
termine behavior, not in a direct, physicalistic sense, but
through the intervening action of the meanings they have ac-
quired for the individual as a result of prior experience,
i.e., through learning, either immediate or vicarious" (p.
309). It is further suggested that because certain stimuli
are linked to similar responses, these stimuli also become
associated with one another. New stimuli can then be learn-
ed by being associated with other stimuli that already have
specific responses attached to them.

In a study investigating the associations between var-
ious stimuli, Noble (1952) found the existence of chain as-
sociations. That is, one stimulus is associated to another
which in turn is associated with another and so on. In other

words, the internal response to an external stimulus can

23
serve as an internal stimulus for yet another internal re-
sponse and this process can go on indefinitely or until the
ultimate external response is made. Theoretically, if the
chain were long enough, the resultant external response need
only be remotely related to the initial stimulus. Neverthe-
less, this indicates that rather than internally possessing
an enormous number of unrelated associations, stimuli are
probably incorporated into an overall structure and are at
least indirectly related to one another.

This idea of an internal structure appears to be quite
consistent with the position taken by Tolman (1932) and the
Gestalt theorists (Koffke, 1935; Kohler, 1947; Wertheimer,
1959). From a more cognitive perspective, they viewed learn-
ing as the acquisition and organization of information about
the environment. In other words, their emphasis was on man's
ability to see meaningful relationships and structure (Snel-
becker, 1974). Tolman (1932) did not, however, abandon be-
haviorism. Instead, he also theorized that cognitive pro-
cesses intervened between external stimuli and overt behavior.
Stimulus information is incorporated into an individual's
cognitive structure until such a time as it is called into
play by some activity. Thus, the relationship between ex-
ternal stimuli and behavioral response is indirect and not
necessarily immediate.

In order for the cognitive structure to become useful

in the assessment of student learning, it must first be

24
determined how the structure is organized. What is the basis
by which associations are made? What is the criteria for
relating stimuli to one another in the cognitive structure?
Bruner (1966) suggests that stimuli are categorized as con-
cepts or objects of an individual's experience. Objects are
associated with one another when they belong to the same
category. Categories are established according to certain
essential distinguishing characteristics or what Bruner refers
to as criterial attributes. "To categorize is to render dis-
criminately different things equivalent,1x>group the objects
and events and people around us into classes, and to respond
to them in terms of their class membership rather than in
terms of their uniqueness" (Bruner, 1966, p. 1). Thus,
learning involves knowing: (I) what attributes are avail-
able, (2) when they should be applied, and (3) how they should
be applied. Successful learning occurs when stimuli or
objects are appropriately categorized.

In the absence of more specific information, a cate-
gorical system may be sufficient to explain the organization-
al pattern of the cognitive structure. However, finer dis-
,criminations are usually possible. Individuals are able to
identify and respond to unique entities in the environment.
For example, the major means of transportation that is usual-
ly found in the garage is not just a car, but a given make
and model produced in a certain year by a specific automobile

manufacturer. The use of additional properties or attributes

25

such as color, rust spots, dents, mileage and license plate
numbers would eventually identify a single car. Maintenance
and use of that car is likely to depend not only on the cate-
gories that it belongs to but also on its idiosyncrasies.
In other words, as more information becomes available, the
categories in which an object can be placed become narrower
and narrower until such a time as the object becomes a cate-
gory in and of itself. The object is then differentiated
from everything else and can be responded to according to
its unique and/or categorical characteristics. Thus, objects
gain their meaning by their placement on attributes relative
to other objects.

Woelfel (1974) has further suggested that the infor-
mation extracted from the environment is organized into a
pattern of similarities and differences among the concepts
that are symbolized by‘words in the vernacular language.
Each concept is uniquely defined by its interrelationships
with all other concepts. Changes in the meaning of concepts
are cognitive processes and result in modifications of the
overall cognitive structure. For example, speech, as an
academic discipline which serves as a categorical domain of
meaning, is defined according to how it is differentiated
from art, biology, business administration, history, physical
education, psychology, and all the other academic disciplines.
While, at one time, speech was strongly associated with the

humanities, it has begun to be more strongly associated with

26

the social sciences. As a result, the cognitive structure
of those in the academic community that are aware of the
changing emphasis in speech-communication departments would
be expected to be experiencing a reorganization in which the
location of speech within the structure is undergoing change.

Given this foundation of cognitive processes, learning
can be defined as the evolution of meaning. This definition
portrays learning as a continuous process where the meanings
for concepts are initially created by setting observations
into correspondence with specific symbols and are subsequent-
ly molded by the accumulation of concept-relevant information.
Since attitudes, beliefs, and values all essentially involve
relationships among concepts, changes in the meanings of con-
cepts would appear to be of fundamental importance to these
processes. In other words, learning, in addition to its
function of evolving specific concept meanings, is also the
basis for the development and modification of an individual‘s
attitudes, beliefs, and values. It is in this latter sense
that meanings will be examined. Rather than investigating
meanings for individual concepts (Osgood, Suci, and Tannen-
baum, 1957), meanings for concepts relative to one another
will be emphasized.

In terms of formal education, learning must be con-
sidered in conjunction with teaching. The function of teach-
ing is typically to develop consensually shared meanings for

the concepts that define the domain of instruction. In

27
contrast, teaching is also occasionally used to broaden stu-
dent perspectives. Whether the intent is to develOp individ-
ual or collective meanings, all courses are usually set up
around some overriding theme or goal. The relationships be-
tween the concepts that make up this theme constitute the
basic foundation on which the course rests. Within this
framework, specific course objectives may be dealt with by
a number of different teaching techniques. The success of
students in meeting these objectives is, of course, important.
Just as important, if not more so, is the extent to which the
students have properly internalized the meanings of the basic
concepts on which the course is based. In this respect,
teaching effectiveness can be seen as the progress of the
students, as an aggregate, in learning the interrelationships
between the concepts which define the overall structure of
the course.

A potential limitation of a cognitive perspective is
its generalizability to all learning situations. In particu-
lar, the utility of an evaluation system based on a cognitive
conceptualization of learning may be called into question in
skill or performance oriented courses. The primary objective
of a calculus course, for example, would probably be to
teach students to actually calculate differentials and inte-
grals. An evaluation based on the student's ability to make
these calculations would be a more direct and probably more

desirable measure of teaching effectiveness. On the other

28

hand, a good grasp of the course material should also be re-
flected in the development of specific meanings for these
operations. The distinction appears to be one of knowing how
to perform a specific activity versus understanding when it
is appropriately performed. Both need to be learned and
neither is precluded when cognitive processes are considered
to intervene between external stimuli and resulting behaviors.

It should also be explicitly pointed out that this
theoretical framework is not strictly cognitive. That is,
'when students provide information regarding their cognitive
structures, they are in essence behaviorally responding and
.it.is assumed that those responses reflect the classroom
sstimuli that they have been exposed to. Rather than focusing
c1n.ultimate behavioral outcomes which vary from course to
cnaurse, the emphasis is on the mediating cognitive processes
VVTliCh it is assumed that all students must use to interpret
Eixld assign meaning to stimuli. The result is that the be-
llaavioral responses required of students, regardless of their
<=<>urse, is completely standardized. Moreover, it is assumed
illlat the cognitive structure serves as the basis upon which

other behaviors are made.

Measurement of Teaching Effectiveness

One of the principle objectives to early cognitive
c>riented theories of learning was the lack of adequate meas-

urement of cognitive processes (Snelbecker, 1974). It was

 

29

assumed that since cognitions are unobservable, they could

not be measured with any degree of success. Ironically, this
argument can now be turned around and used in support of a
cognitive perspective of learning. It is not that cognitive
processes are now observable but rather that useful procedures
have been devised which allow these processes to be accurately
measured. Metric multidimensional scaling provides a means of
assessing cognitive processes in a manner that has been found
to be precise and reliable (Gillham and Woelfel, 1976; Woel-
fel, 1977). Moreover, a multidimensional analysis as is to
Joe described here satisfied at least some of the limitations
c>f measuring teaching effectiveness that are found in tra-
ciitional evaluation instruments.

Multidimensional scaling is based on the fundamental
<2<1ncept of psychological distance (Helm, Messick, and Tucker,
(£1959). That is, the perception of difference between stimuli
(>1: objects is the basis of the measurement scheme. Woelfel
(21972) suggests that differences "among objects (whatever

tillose objects may be) may be represented by a continuous num-
t>€sring system such that two objects considered to be complete-
343? identical are assigned a paired dissimilarity score or
distance score of zero (0) , and objects of increasing dis-
Similarity are represented by numbers of increasing value."
UD<> utilize the set of real numbers in such a manner, a rule
Inll-let be established for setting the numbering system into

‘=<>rrespondence with perceptions of difference.

30
Stevens (1951), Torgerson (1958) and others (Campbell,

1928; Suppes and Zines, 1963) have stated that a rule for
quantifying distance or difference must stipulate an arbi—
trary standard difference to which all other differences are
to be compared. Such a rule is provided by Einstein (1961):

For this purpose (the measurement of distance)

we require a 'distance' (Rod S) which is to be

used once and for all, and which we employ as a

standard measure. If, now, A and B are two

points on a rigid body, we can construct the

line joining them according to the rules of ge-

ometry; then, starting from A, we can mark off

the distance S time after time until we reach B.

The number of these operations required is the

numerical measure of the distance A B. This is

the basis of all measurement of length.
To use Einstein's rule for measuring the perceived differences
among objects, all that needs to be done is to arbitrarily
stipulate that the difference between any two objects is some
designated distance and that this distance is the standard of
comparison for all other pairs of objects. This can be accom-
plished by wording a question in the following form:

If the difference between a and b is‘u units,
how different are x and y?

Responses to this type of question would be ratio measures
of the perceived differences between pairs of objects.

When this ratio rule is applied to N concepts, N(N-1)/2
non-redundant paired comparisons are possible. Completing all
such pairs produces a NxN symmetric matrix D for an N concept
domain. This matrix, then, represents the overall pattern of

differences among the concepts in the domain. Woelfel (1972)

further states that:

31

. the definition of an object or concept is

constituted by the pattern of its relationships
to other objects, the definition of any object

may be represented by an 1 x n vector, d

11' d12’

d13' . . . , dln' where dll represents the dis-

tance or dissimilarity of object 1 from itself

(thus d

11

=0 by definition), d12 represents the

distance or dissimilarity between objects 1 and

2, and d

In
lst and nth objects.

represents the distance between the

Similarly, the second ob-

ject may be represented by a second vector, d21,

d22' d23'

d

n’ and the definition of any

set of concepts or objects may therefore be
represented in terms of the matrix

d
d

11'
21’

dnl’

where any entry dij

d
d

d

12'

22'

2n'

, d
, d

1n
2n

, d
nn

represents the dissimilarity

or distance between i and 1.

Since the primary concern in evaluating teaching ef-

fectiveness is with aggregates of students, potential unre-

liability resulting from individual differences in responses

is minimized by averaging dissimilarity scores across the stu-

dents in a specific course.

This procedure yields a means

distance matrix D which represents the average dissimilarity

among concepts which constitute the domain of instruction.

This matrix is transformed into a centroid scaler products

matrix B (Young and Householder, 1939) which when factored

 

32

(Jacobi, 1846) results in a cartesian coordinate system of
orthogonal dimensions or axes. A rectangular matrix F is
then constructed with the dimensions as columns of the matrix
and with the rows representing the projections of the con—
cepts on each of the dimensions. These procedures, which are
described in much more detail by Torgerson (1958), Woelfel
(1972, 1974, 1977), Woelfel and Danes (1977), and Serota
(1974), essentially map the structure of the domain.

Cognitive processes such as learning can be represent—
ed by comparing a series of these spatial configurations
gathered at several points in time. The coordinate systems
generated at each point in time can be rotated and translated
into a least square best fit with one another (Woelfel, 1977;
Woelfel and Saltiel, 1974). In other words, this procedure
is used to establish a common frame of reference from which
changes in the meanings of objects or concepts can be observed.
Changes in the spatial location of the concepts over time is
then interpreted as motion through the space and as such can
be mathematically expressed as velocities and over multiple
time periods as accelerations. This enables at least portions
of the learning process to be precisely assessed over the
length of instruction.

The implementation of this multidimensional scaling
technique for the purpose <3f evaluating teaching effective-
ness is straight forward. What first needs to be done is to

identify the relevant concepts that will be scaled into the

33
space. Since these concepts define a particular domain of
meaning, they must be topic or course specific. Moreover,
these concepts must describe the basic nature of the course,
not evaluate it. In other words, they must constitute the
overall framework on which the entire course is based. Such
concepts typically make up the general course objectives or
outline. For example, one of the primary goals of an intro-
ductory communication course is to provide students with some
sort of definition of communication and a notion of how the
study of human communication can provide them with some use-
ful information. From this objective, the following key con-
cepts can be extracted: communication, process, information,
valuable, meaning, social science, humanities, and me (repre-
senting the self-concept, see Woelfel and Danes, 1977). A
second goal is typically to differentiate several types of
communication systems. Depending of course, on
the systems that are explicated, the following concepts might
emerge: communication, information processing, interpersonal
relationships, group interaction, organizations, mass media,
social change, persuasion, meaning, leadership, self-concept
development, efficiency, and socialization. A course in re-
search methods, which covers more technical information could
be described by concepts such as: science, theory, measure-
ment, precision, mathematics, function, causality, experiment-

al design, reliability, and validity.

34

A predictable pattern of interrelationships among the
selected concepts should develop over the duration of the
course. Assuming the course materials were being appropriate-
ly taught, the meanings given these concepts by the students
would be expected to begin to converge on the projected mean-
ings provided by the course objectives. Going back to the
example of the introductory communication course, several
changes would be expected to occur. Leadership, for instance,
may not be intuitively thought of as a communication construct.
If it is effectively taught as such, a stronger association
between communication and leadership would be expected to
emerge. Assuming a strong relationship between cognitive
processes (i.e. attitudes and beliefs) and behaviors (see
Liska, 1975; Cushman, 1977) the subsequent communicative be-
haviors of the students should become more salient to them
when they are placed in a position of leadership. If the
course objectives are attained, the students would also have
begun to more fully realize the value of communication.
Since concepts associated together have been found to converge
on each other in the multidimensional spatial configuration
(Woelfel, Cody, Gillham, and Holmes, 1976), the perceived dis-
similarity between communication and valuable should decrease.

Doyle (1974) has suggested that a useful criteria for
evaluating instruction is the stimulation of student interest
in the subject matter of the courses that they are taking.

This can be assessed for all courses by examining the

35
relationships between the concepts which constitute the in-
structional domain and the concept "me" which as previously
mentioned represents the students' self-concept. Increased
student interest would be reflected by the student's identi-
fying more closely with the concept which best characterizes
the course's subject matter. In other words an inverse rela—
tionship would be expected between student interest in the
course and the differentiation of the concept "me" with the
concept representing the domain of instruction. Woelfel and
Danes (1977) assert that this type of relationship "is pre-
dictive of approach behavior" (p. 28). In an educational
context, then, such a relationship may be representative of
student effort to learn course material. It may even further
indicate the probability of students to pursue relevant sub-
ject matter beyond the courses in which they are enrolled.

To determine the extent of teaching effectiveness,
more Specific information is required. It is not enough to
merely examine general trends in student learning patterns.
(A standard for comparison of student progress towards learn-
ing appropriate interrelationships among the selected con-
cepts must also be provided. Two primary sources are avail-
able for the construction of such a standard. Already knowing
the content of the course and its goals, instructors would
be in a good position to provide this information. The in-
structors own meaning for the specific domain of instruction

:may not be the same as that which would be expected of

 

36

students. Instructors could instead provide a multidimen-
sional configuration based on their expectations of the re-
sponses that could reasonably be eXpected of students who
have mastered the course material. Unless, however, a size-
able group of qualified instructors could be used for this
purpose, the procedure could result in substantial errors
due to inaccurate projections.

The second source of information is students. That
is, the meanings develOped by students who have previously
been successful in learning specific course material could
provide a standard for comparing other students. There may,
however, be problems in identifying the appropriate students
and in comparing levels of student achievement across time.
It would seem, then, that the most desirable standard of com-
parison would be a weighted average between instructors ex-
pectations and previous successful students. This would min-
imize the extent of errors in instructor estimates and would
contain the flexibility for future student performances to
exceed or differ from current levels. The weights are de-
pendent upon the number of instructors and students contrib—
uting to the standard, changes in the course, and differences
in students enrolling in the course.

A standard representing desired course outcomes en-
ables student learning to be more thoroughly assessed.
Periodic comparisons of student responses with the standard
reveals not only what has been learned but also what needs

to be learned. The extent to which students progress toward

 

 

37

this standard is, then, a direct measure of teaching effec-
tiveness.

While the procedures described thus far, are more than
adequate to precisely assess teaching effectiveness, a
thorough evaluation of instruction should also contain more
detailed information regarding the specific instructors'
classroom behavior. In addition to knowing the extent of
student learning that took place in a given course, it is
also informative to know what the instructor did or did not
do that resulted in the learning that occurred. Multidimen-
seional scaling has been found to be a useful tool in this
type of situation as well (Cody, 1976; Wakshlag and Edison,
1975).

For this purpose, salient teaching characteristics
need to be identified and scaled into the multidimensional
space along with the concept "instructor" which, of course
represents the specific instructor whose teaching performance
is being examined. A list of appropriate concepts or char-
«acteristics (i.e. clarity, stimulating, effective, rapport)
sshould be obtained from students, instructors, administrators,
(and.from previous research (Deshpande, Webb, and Marks, 1970;
.Sharon, 1970). These concepts must then be submitted to a
Q—sort technique such as the one described by Wotruba and
‘Wright (1975). Essentially what needs to be done here is to
locate those concepts which are perceived to be the most im-

portant and which students are able to accurately make

38

judgments of. Only concepts that meet both of these criteria
as rated by students, instructors, and administrators should
be included in the instrument. The lack of consensus would
likely result in the obtained information not being fully
utilized by all those who could benefit from it.

Along with the obtained concepts and "instructor," a
concept entitled "the ideal instructor" could also be used
for comparative purposes. The ideal instructor would be de-
scribed as an individual from which the students perceive
that they would maximize their learning potential. In other
words, the descriptive teaching characteristics would be used
to define an optimally effective instructor. The actual in-
structor would be compared to the ideal one according to more
or less dissimilarity with these concepts. Since any direct
comparison between the ideal and actual instructor by the
students would be highly susceptible to bias, evaluation in-
struments should contain one or the other. The comparison
can, then, be made more impartially. The information obtain-
‘ed.from this procedure could be used to assist instructors in
identifying particular teaching skills that need improvement
and to assist students in locating instructors with specific

desirable teaching styles.

Advantages

 

The overall advantage of the measurement system pre-

sented here is that it possesses at least the potential for

 

39

overcoming the problems confronting traditional evaluation
procedures. Specifically, this system of evaluating instruc-
tion (1) is based on a theory of the teaching-learning rela-
tionship and as such enables the direct assessment of teach-
ing effectiveness to be made for certain courses. These
procedures would appear to be most useful for evaluating in-
struction in courses that emphasize acquiring knowledge of
course material in contrast to courses which stress improving
skills. It (2) does not require advance knowledge of all
relevant criteria on which student responses are based. This
measurement technique assumes that the overall perception of
difference precedes the perception of attributes. Consequent-
ly, specific attributes used for making distance judgments
need not be known nor directly utilized in the measuring
instrument. They can, however, be subsequently interpreted
from the results. Making complete paired comparisons on all
concepts that are provided (3) allows multiple criteria to
be simultaneously examined. Moreover, the paired comparison
technique (4) is not readily susceptible to the influences of
social desirability factors. Doyle (1975) reports that
several studies, using traditional Likert type measures,

have found that students become more lenient in their evalua-
tions when they are told that the results are to be used for
administrative purposes. This is less likely to occur in
this situation because actual comparisons to the standard of

evaluation are not directly made by students and the scaling

 

4O

procedure provides no inherent indication of how one would

go about making an instructor appear more favorably. This
evaluation system also (5) has the capacity to incorporate
standards for direct evaluation; (6) provides relevant in-
formation for administrators, instructors, and students;

(7) is precisely ratio scaled; and (8) permits powerful time-
series analysis of the learning process. This last advant-
age is particularly important because it enables instructors
to monitor their effectiveness throughout the term. As a
result, appropriate modifications in lesson plans and teach—

ing styles can be made.

Limitations

 

This evaluation system is, however, faced with two
potential problems that should be mentioned. One objection
to the system has to do with the burden that it places on the
students. Multiple measures for several courses could begin
to require students to spend a good deal of their time eval-
uating instruction. Several steps could be taken to minimize
this problem. First, samples of students could be taken at
each time interval so that all students would not always be
asked to participate. Secondly, the instrument could be
broken up such that students would only have to fill out a
portion of the total number of responses. Finally, specific
relational patterns may begin to reappear with a great deal

of regularity after the instrument has been in use for awhile.

 

41

It would only be necessary to periodically check to make sure
that these patterns have not been broken. Consequently the
number of required responses would be reduced. Although these
remedies help alleviate this problem, they do not eliminate
it. Active student participation is necessary to make this
system successful. It is expected, however, that their
efforts will be rewarded with generally improved teaching and
with more detailed information regarding the teaching styles
of perspective instructors.

This evaluation system may be difficult to adapt to
skill and performance oriented courses. However, the com-
parison of the actual instructor to the ideal instructor would
still provide useful information. Moreover, it would also be
of interest to determine certain attitudes such as favor-
ability toward the course's subject matter. After all, train-
ing students to be excellent typists would not reflect teach-
ing excellence if as a result, all of the students hated to
type and would avoid doing so in the future. It should also
.be pointed out that the skill and performance oriented
courses are in much less need of new evaluation procedures.
That is, the extent to which students master the skills
taught has always provided a directly observable means of

measuring teaching effectiveness.

 

CHAPTER III

EXPERIMENTAL METHODS AND PROCEDURES

The assessment of a training program for undergraduate
teaching assistants was selected for an initial application
of the evaluation system developed in the last chapter. It
was previously suggested, in Chapter I, that the resistance
of instructors to be evaluated may be at least partially due
to insecurity resulting from their lack of formal classroom
training. This, of course, would not pose a problem unless
it was assumed that training produces more effective instruc-
tion. In other words, the results of such training are
generally expected to be manifested in the actual performance
of the instructors in the classroom. In the situation exam-
ined in this report, upper division undergraduate communica-
tion majors are used to facilitate small group activities in
the freshman level introductory communication course. Rather
than the traditional teaching methods training, the program
established for these teaching assistants emphasized the im-
provement of leadership skills. Successful training would
not only be expected to lead to more student learning but
perceptions of teaching or leadership competence should also
increase. However, one potentially serious problem should

42

 

 

 

43
be mentioned. That is, the individuals whose teaching
effectiveness is being examined are only teaching assistants.
Consequently, any differences in their effectiveness may very
well be overshadowed by the influences of the instructors

with whom they are working.

Undergraduate Teaching Assistants

Over the past several years, a great deal of time and
money has been put into the development of more advanced
methods of instruction. Modern technology has enabled edu-
cators to increase the quality and quantity of materials
that they are able to provide for the large numbers of stu-
dents in today's schools (Association for Educational Com-
munications and Technology, 1977; Cantwell and Doyle, 1974).
The methods used to present these materials are, however,
merely extensions of the only two basic techniques that are
at the instructor's disposal: directly supplying educational
information for student consumption and providing experiences
from which the student is expected to master the desired
Inaterial. This latter technique has been referred to in the
literature as the "discovery process" (Ausubel, 1968; Morine
and Mbrine, 1973) and also has informally come to be known
as "experiential learning." It has typically been associated
‘with creative and applied areas where such experiences have
long been recognized as an appropriate and desirable means

of instruction.

 

 

 

 

 

44

Recently, the experiential learning technique has
begun to be utilized in the social disciplines (Barbous and
Goldberg, 1974; Johnson, 1972; Pfeiffer and Jones, 1971).
The field of communication is in an especially advantageous
position to adopt this teaching strategy in certain situa-
tions. It seems to be a particularly appropriate means to
introduce students to the field. That is, many students
enter their first communication class with the notion that
since they learned as a child to read, write, speak, and
listen, and as a result are able to adequately function with-
in their environment, communication is not a complex phen-
omenon and it can therefore be taken for granted (Williams,
1974). It would seem, then, that a major consideration of
any introductory communication course would be to eliminate
this misconception. Rather than merely trying to explain
that to the students, it would appear to be helpful to also
demonstrate this point by allowing students to participate
in structured exercises where they will actually be con-
fronted with certain designated communication problems.
frhis is, in fact, the approach taken by many departments
around the country.

Introductory courses in communication have recently
Ibeen faced with a dilemma. That is, exercises in communica-
tion typically require close supervision. However, these
courses have experienced increased pOpularity among students

and curriculum committees. As a result, the enrollment per

45

class has increased such that the instructors are no longer
able to provide the personal attention necessary to success-
fully carry out these exercises. To help resolve this prob-
lem, many instructors have begun to utilize both undergrad-
uate and graduate teaching assistants. The following guide-
lines were established by the Department of Communication at
Michigan State University for the regulation of undergraduate

teaching assistants in the introductory communication course:

GUIDELINES FOR THE FUNCTION AND EVALUATION
OF THE UNDERGRADUATE TEACHING
ASSISTANTS IN COMMUNICATION 100

The undergraduate teaching assistant (UTA)
is a student who has taken at a minimum the Com-
munication 100 course as an undergraduate and who
has expressed interest in participating as a mem-
ber of a team of instructors in the teaching of
Communication 100. While priority is given to
communication-education majors for placement into
the UTA program, since it is a requirement for
them, communication majors and majors from other
areas are welcomed. (Those students whose majors
are outside the communication department should
be certain that a Communication independent study
is acceptable with their department.)

The UTA registers for three credits of inde-
pendent study (Com 299 for freshmen and sophomores,
and Com 499 for juniors and seniors). The grad-
uate student who is the senior instructor of the
Communication 100 section in which the UTA works
becomes the Independent Study Director and assumes
responsibility for the grade given the UTA. To
determine the responsibilities and means for eval-
uation of the UTA, the instructor and UTA will
write a contract, which must be signed by the
instructor, the UTA, the course chairman and the
director of undergraduate study. The exact nature
of the contract is determined through conference
between the instructor and the UTA. However, it
is recommended that specific criteria for success-
ful completion of each grade (A, B, C, etc.) be
determined in the contract in order to allow the

 

 

46

UTA to know exactly what he/she is required to do
to attain the grade desired, and to provide the
instructor with specific guides on which to base
the final grade assigned to UTA. It is recom—
mended that the contract identify a variety of
teaching experiences for the UTA, such as func-
tioning as a discussion leader, directing an ex-
ercise or game, preparing the class to see a
movie or conducting a follow-up discussion, etc.

UTA's are required to attend all class ses-
sions (unless a verified reason is provided for
not attending a session) and it is expected that
the UTA will participate as a member of the instruc-
tional team. THE UTA IS NOT TO BE USED AS A READER
OR GRADER. While the UTA may grade some papers
or examinations (the specific amount to be deter-
mined in the contract), a UTA must NOT BE SOLELY
RESPONSIBLE for a student's grade. It IE advised
that both instructor and UTA grade a set of papers
and compare their evaluations so that grading is a
learning experience for the UTA and that the ulti-
mate responsibility for the grade rests with the
instructors.

 

The UTA is expected to interact fully with the
undergraduates and often has been found to act as
liaison between the instructor and the students.

(In addition, the UTA, like the instructors, will be

evaluated by the students in the section through the

use of SIRS or alternative forms.)

In total, the UTA opportunity should provide
undergraduate students with a teaching-learning-team
functioning experience, and should be an asset to
the instruction of Communication 100.

Teaching assistants are typically selected according to
their: (1) desire, (2) availability, (3) compatibility with
the prospective instructor, and (4) quality as a student.

None of these criteria are necessarily related to the assist-
ants performance in the classroom. Moreover, this is actually
their first experience at formally teaching a group of stu-
dents. Recognizing this as a potential problem, a training

program was developed to provide teaching assistants with the

 

47

leadership skills that they would need in their classroom role.

The Teaching Assistants Training Program

Taking into consideration the teaching assistants
primary responsibility of guiding exercises and discussions,
the focus of the workshop is on small group processes. In
particular, the function of leadership in these classroom
situations is emphasized. The program is essentially struc-
tured in the same manner as the classes that they will be
assisting. That is, exercises are used to help demonstrate
key points as well as to give the participants practice at
applying their leadership skills in a situation where con-
structive feedback is provided. The teaching assistants
also have the opportunity to actually participate in many of
the same exercises that they will ultimately be using in
their classrooms.

The program is offered each term and teaching assist—
ants usually take it in conjunction with the class that they
are working with. In order to maximize the benefits of
their participation, the workshop is condensed into the be-
ginning of the term. Four three-hour sessions are held one

evening a week for the first four weeks of classes.

Session One

 

The initial meeting is primarily used to get the par-
ticipants familiar with one another and with the workshop.

Several warm-up or get acquainted type exercises are used

48
for the purpose of getting the participants to feel more
comfortable and to begin to develop an atmosphere of cohe-
siveness and trust. This is considered to be an important
first step because the teaching assistants can become valu-
able resources to each other as the term progresses. More-
over, they will be asked to discuss one another's leadership
performances in later sessions.

An explanation of what is to transpire at the next
three sessions is then provided. In particular, they are
told that each of them will be given the opportunity to
select and lead a group exercise. This session ends with a
discussion of the participants concerns as teaching assist-
ants and suggestion of topics that they would like to have

covered in the workshop that weren't originally scheduled.

Session Two

 

In the second session, the participants are divided
into two groups for the purpose of working on a problem
solving situation. The designated leader in one group is
instructed to be a socio-emotional type leader while the
other group has a leader that has been told to be task-
oriented. This is done to demonstrate the general differ-
ences regarding efficiency and satisfaction that is typically
found with groups having these two types of leaders. In
addition, confederates acting as deviants are planted into
each group to stimulate a discussion of techniques that may

be used to deal with disruptive students in the classroom.

49
The exercise is followed by a discussion of leadership and

group dynamics.

Session Three

 

The third meeting consists of as many of the partici-
pants as possible leading the group in an exercise and a
follow-up discussion used to draw out the major points
covered by the exercise. The group, then, examines each
exercise in terms of how well it demonstrated the major
points and additional points that could be brought out of
the exercise. Suggestions are also made regarding other
means of covering the same points. The specific leadership
behaviors and strategies of each participant—leader are then
discussed. The success of those strategies is assessed and

possible alternative strategies are brought up.

Session Four

 

The remainder of the participants who have not pre-
viously had the Opportunity to lead a group exercise are
able to do so in this final session. The procedures are the
same as in the third session. In addition, lecturing tips
and other such tOpics of concern to teaching assistants are
attempted to be squeezed in. The participants finish off
the workshop by exchanging the experiences that they have
already had in the classes that they are assisting.

In summary, the training program briefly described

here is designed to prepare undergraduate teaching assistants

 

 

50
for the classroom by increasing confidence in and improving
their leadership skills as well as providing them with a
general understanding of small group processes. In addition,
the assistants become thoroughly familiar with the use of

exercises as a teaching devise.

The Introductory Communication Course

The particular introductory communication course that
the teaching assistants in this study worked with is essen-
tially a survey course. That is, the course attempts to rep—
resent most of the various interest areas which make up the
discipline. The following general areas are covered: basic
concepts and models, the nature of meaning, code systems,
message construction, public speaking, interpersonal rela-
tions, small group interaction, organizational systems, and
mass media. In the course, students are provided with a
brief and rather narrow exposure to each of these areas. In
addition, the course focuses attention on communication
skills. Students are required to give several oral presen—
tations and to write several papers.

It would appear that for such a course to be success-
ful, one of its overriding concerns must be in stimulating
student interest not only in the course itself but also in
what the department has to offer. A helpful first step in
this direction would be to demonstrate that there is, in
fact, something important to be studied. That is, students

must be instilled with the notion that communication is a

51
much more complex process than they had previously thought.
Building on this, the students must also gain an understand-
ing of how knowledge of this complexity can be beneficial to
them. Exercises are typically used as a teaching technique
to help make these points. Students frequently find this
type of experience enjoyable as well as informative. Regard—
less of the method used, if the points are successfully made,
they would be expected to stimulate student interest in
learning about the communication scholars, then,
provides direction for students interested in pursuing

courses in the department beyond the introductory level.

Measuring Instrument

Given the previous accounts of the functions of under-
graduate teaching assistants, the training that they receive,
and the nature of the course in which they work, attention
can now be focused on the development of the specific measur-
ing instrument (see Appendix A) to be used for the purpose of
evaluating their effectiveness in the classroom. Teaching
effectiveness was defined in Chapter II as the progress of
students in learning the appropriate interrelationships
among the concepts which constitute the domain Of meaning for
the course. The task at hand is, then, to select the rele—
vant concepts for this particular course and workshop. As
an introductory course, there are many more concepts than

could be considered in this instrument. The general meaning

 

 

52
must be considered to be important. As a result, the fol-
lowing concepts were included in the instrument: communica-
tion, humanities, physical science, and social science.
Since the academic approach of this department is a social
scientific one, it is expected that if instruction was suc-
cessful, communication and social science would become more
strongly associated with one another as the term progresses.
To examine more specific content, leadership and democratic
were also included. Neither of these concepts are intuitive-
ly thought of as communication concepts. They are, however,
taught as such in the group interaction section of the
course. Effective teaching should move them both closer to
communication in the cognitive structure. Moreover, they
should move closer to each other since democratic is taught
as a specific leadership style.

It was suggested in the previous chapter, in ad-
dition to student learning, the stimulation of student
interest in the general tOpic area covered by the course
should also be considered as a criterion of teaching effec-
tiveness. This is, in fact, one of the overriding goals for
the particular course examined in this study. The stimula-
tion of student interest would be reflected by a reduction
of the perceived differentiation between the concepts come
:munication and me (representing the students' self concept).
In other words, this change is assumed to reflect increased

student interest in communication. To obtain an

 

53

overalealue judgment of communication, the concept good
was added. A more favorable student attitude toward com-
munication would be demonstrated by a convergence of com-
munication and good.

In order to more directly evaluate the training pro-
gram itself, it would be useful to more closely examine the
actual classroom performances of the teaching assistants.
For this purpose, the teaching assistants and the teachers
for whom they work were included in the instrument. Since
the teacher would presumably serve as a role model for the
assistants, the differentiation between the two would re—
flect how well the assistants were carrying out their role
relative to some standard. The teacher probably does not
represent the ideal standard, but the goal of perceived
equivalence with the teacher would in most cases be con-
sidered a substantial step in that direction. While leader-
ship and democratic were previously used in the assessment
of student learning, they can also be used here to provide
more specific information regarding the teaching assistants.
That is, effective assistants would be expected to be more
strongly associated with leadership. Since the democratic
style was presented in the workshop as the most apprOpriate
for facilitating group exercises and discussions, effective
assistants should also be more closely identified with demo-
cratic in the cognitive structure. The final two concepts

that were included to reflect the assistants classroom

 

54

performance are "confident" and "expertise." Assuming

that confidence is behaviorally manifested, teaching assis-
tants participating in the training program would be ex-
pected to display more confidence. The workshOp discus-
sions on group processes and leadership should make the
participants more knowledgeable in the area that they will
be working. Moreover, the importance placed on the follow-
up discussions may stimulate them to better understand the
material covered by the exercises they use. Participants
in the workshop should, then, be more closely associated
with expertise.

To put the questionnaire in more of a general educa-
tion frame of reference, the concepts studying, thinking
and learning were also included in the instrument. While
no specific expectations are made regarding these concepts,
they may be useful in interpreting the results. In all,
the following sixteen concepts were incorporated into the
complete paired-comparisons format on the questionnaire:
communication, humanities, physical science, social science,
good, me, instructor, undergraduate teaching assistant (2):
leadership, democratic, confident, expert, thinking, study-
ing and learning. The ordering of the pages on which these
concepts appeared was alternated to equally distribute the
possible influence of fatigue on the part of the respon-
dents. The questionnaire also asked the respondents to

provide basic demographic information as well as information

55

regarding their educational backgrounds.

Research Design

This evaluation is primarily based on the idea that
the success of the program should be determined by the act-
ual classroom performances of the teaching assistants.

Twenty sections of the introductory course were offered the
term that this study was conducted. The teaching assistants
from ten of these sections were randomly selected to partici-
pate in the training program while the assistants in the
other ten sections served as the control group. The division
was made according to section rather than individual assist-
ants because six sections had two assistants. It was felt
that to have one assistant participating in the workshop and
the other not participating might cause some unnecessary
problems. That is, the untrained assistant may indirectly
benefit from the workshop by picking up on what the trained
assistant had learned. Moreover, repeated exposure to

both assistants could make it difficult for students to make
a clear distinction when assessing the assistants separately.
The repeated exposure to both assistants would also make it
difficult to attribute the extent of student learning to the
teaching effectiveness of one teaching assistant or the other.

The control group consisted Of fourteen teaching as—
sistants, nine male and five female, with a mean age of
twenty-two. Their grade point averages ranged from 2.4 to

3.9 with 3.1 as the mean. Ninety percent of the group were

56
juniors or seniors. Sixty-nine percent were majoring in com-
munication. In contrast, there were twelve assistants in
the experimental group. Four of them were males and eight
were females having an average age of twenty. Their grades
went from a low of 2.5 to a high of 3.8. The mean grade
point average for this group was 3.2. Seventy-three percent
were communication majors. Eighty-nine percent were upper-
division students.

The ten sections whose assistants were in the control
group were found to be quite similar to those sections whose
assistants participated in the training programs (see
Appendix C for specific comparisons). In the sections from
both groups, class time was just about equally distributed
between lectures (32%), class discussions (30%) and exercises
(36%). The number of students ranged from 38 to 71 in the
sections represented in the control group and from 34 to 76
in the experimental sections. The mean for both groups was
63 students. The mean age of the students was 19.1 and
19.0 for the experimental and control group sections respec-
tively. Their respective grade point averages were 3.05 and
3.02. In both groups, there was approximately a 2 to 1
ratio of female to male students. Seven percent of the
students in the control group sections were majoring in com-
munication. The experimental group sections had ten percent
communication majors. The only other sizable group of stu-

dents were those who had not yet made an academic preference;

57
26% in the control sections and 19% in the experimental
sections. This is not particularly surprising since the
course is at the introductory level and high percentages Of
freshman (58% in control sections and 49% in experimental
sections) were found to be enrolled in it. Only 28% of the
students in sections of both the experimental and control
groups were in the upper division.

The descriptive data discussed thus far suggests that
the experimental and control groups are comparable in re-
gard to the teaching assistants, the structure of their
classes and the students in those classes. The differences
that exist are either minor or would not be expected to in-
fluence the results of this study. There are, however, three
areas in which the two groups differ that is a cause for
concern. First, 85% of the teachers that the assistants are
working for in the control group have had previous teaching
experience while only 54% of the teachers in the experi-
mental sections are experienced teachers. Moreover, teach-
ing assistants had been previously utilized by 62% of the
teachers in the control sections. Only 29% of the experi-
mental group teachers had used teaching assistants in the
past. Finally of the teaching assistants themselves, 38%
in the control group and 21% in the experimental group have
had some sort of past experience as a teaching assistant or
small group leader. In each of these areas, the control

group would appear to benefit from more experienced teaching

58
backgrounds. This essentially provides for a more rigorous
test of the training program's effectiveness. That is, for
the training program to appear successful, the trained
assistants have to be more effective in the classroom than
the untrained assistants who have more experience and work
with teachers who also have more experience.

The problem of unequal past teaching experience is
somewhat neutralized by the longitudinal nature of the study.
At approximately three week intervals, data was collected at
four points in time during the term. Thus the assistants
teaching effectiveness can be evaluated in terms of growth
and improvement over time.

In summary, the current study evaluates a training
program for teaching assistants by examining the teaching
effectiveness of a group of trained assistants and a group
of assistants that were not trained. The evaluation concen-
trates on student learning and actual classroom performance
as the criteria for assessing the program's success. Data
wasgathered four times during the term from student

volunteers.

CHAPTER IV

RESULTS

The major findings of this study will be discussed in
two parts. The first and primary analyses will examine the
data in a manner that will attempt to illuminate the proced-
ures used to evaluate instruction. As was stated in Chapter
I, the study reported here served as an initial application
of a new set of procedures developed for instructional eval-
uation. Of specific interest is, then, an exploration into
the success of these procedures in assessing the teaching ef-
fectiveness of this particular group of teaching assistants.
This is the principle focus of the study.

It should be explicitly pointed out, however, that
this study is a demonstration of how instruction might be
evaluated under ideal circumstances. The particular data set
used here is really not appropriate for the analyses that
will be performed. Specifically, every available student was
asked to volunteer at each data collection. Many students
participated more than once but not at all four points in
time. Consequently, there were insufficient sample sizes
for a normal panel design. Moreover, the samples were not

randomly drawn. Thus, the data was correlated at least to
59

60
the extent that some subjects participated in more than one
data collection. The analysis performed here does, for
illustrative purposes, treat the data as if independent
random samples, without replacement, were drawn at each
time interval.

Sample sizes per section for the experimental and
control groups for each data collection are presented in
Table 1. In the primary analyses, individual student re-
sponses were aggregated across all sections for the experi-
mental and control groups. In the secondary or adjunct
analyses, the data will be re-examined to enable more gen-
eral statements to be made regarding the teaching-learning
process. For these analyses, as well as for the examination
Of the manipulations and the precision of measurement, each
of the sections that had students participating in all four
data collections will serve as a unit of analysis. The in-
dividual student responses within a section will provide an
estimate for that particular section. The scores for each
section should as a result be fairly stable. Each of the
appropriate sectional scores will then be averaged for the
experimental and control groups, and comparisons made on
that basis.

In the primary examination of data, attention will

first be given to an exploration of teaching effectiveness.

61

Table 1. Sample Sizes per Section for the Experimental
and Control Groups at Four Time Intervals.

 

T T T T

 

 

 

l 2 3 4
Experimental Group
Section 1 18 14 7 8
Section 3 7 24 16 15
Section 6 24 0 l 4
Section 8 7 4 2 1
Section 9 19 20 10 13
Section 12 20 16 2 9
Section 15 4 ll 0 9
Section 16 10 0 0 0
Section 19 20 8 3 0
Section 21 _20 __4 .__Z _11
149 101 8 70
Control Group

Section 2 15 6 7 5
Section 4 18 10 6 4
Section 5 23 6 6 3
Section 7 13 ' l8 2 4
Section 10 10 8 4 13
Section 11 4 2 l 14
Section 13 ll 42 12 16
Section 14 9 13 6 0
Section 18 21 18 8 6
Section 20 __2. .__E __1 __§
127 125 53 68

 

62
More specifically, this set of analyses will examine differ-
ences in the extent of student learning that occurred in
courses aided by trained teaching assistants in contrast to
courses aided by teaching assistants who received no training.
The integration Of course material as reflected in the follow-
ing six paired comparisons will be analyzed:
. Communication and Social Science
. Communication and Leadership
. Communication and Democratic
. Leadership and Democratic

. Communication and Good
. Communication and Me

O‘Ul-bUJNH

The first four of these pairs focuses specifically on student
consumption of course content while the latter two are pri-
marily concerned with assessing more general student atti-
tudes regarding communication. For the purpose of providing

a standard for determining student progress, a criterion score
for each pair was established. Because the teaching assist-
ants had already successfully completed the course and the
instructors presumably understand the course material, their
judgments of the above items were pooled to create this cri-
terion. The small sample size, however, makes the utility of
the criterion scores for this particular study somewhat ques-
tionable. While specific comparisons with the criterion
scores will be made, these scores will primarily serve in more
of a directional capacity rather than in any absolute sense.
The thrust of these analyses will be on examining differences
in student learning between the experimental and control group

classes.

63

The second set of analyses focuses more directly on
the teaching assistant's actual classroom behavior. The em-
phasis here is on the relationship between the teaching assist-
ants and several key attributes or characteristics which are
fundamental to their role in the classroom. The following
paired comparisons will be investigated for this purpose:
. Teaching Assistant and Expert
. Teaching Assistant and Leadership
. Teaching Assistant and Democratic

. Teaching Assistant and Confident
. Teaching Assistant and Instructor

Lil-book)!“

This set of analyses also contains a standard for comparison.
Student responses regarding dissimilarities between the in-
structors and these same characteristics were averaged across
experimental and control groups at each point in time to
create specific criterion scores for each pair. Since there
would be no dissimilarity between the instructors and them-
selves, the teaching assistant-instructor pair was set, by
definition, at 0.00. The instructors were themselves rela-
tively inexperienced and as such do not provide the ideal
standard suggested in Chapter II. However, it was felt that
perceived equivalence with the teacher, while not the ideal
standard, would in most cases be considered a substantial
step in the appropriate direction. Consequently, the cri-
terion scores for these analyses will serve to provide direc-
tion rather than any definitive comparison. In other words,
the emphasis will again be on differences between the experi-

mental and control groups. The criterion scores will then

64

be an indication of whether those differences reflect more
favorably on one group or the other. Prior to discussing
these two sets of analyses, it is important to first examine
the measurement system and the success of the experimental

manipulations.

Precision of Measure

Theoretically, the concepts not directly manipulated
in the experiment would be expected to remain constant across
all experimental conditions. That is, the means for each of
the paired comparisons among these not intentionally affected
concepts should be the same in both the experimental and
control groups as well as at each point in time. Differences
are attributable to error of measurement. Averaging first
across all groups at each time interval enabled standard
errors of measure to be established by the standard deviation
for each of the unmanipulated pairs. The coefficients of
variability were then calculated, according to the following
equation: V = 100 (Sxi/Y), and then averaged over the four
time intervals. The resultant coefficient represents the
percentage of measurement error for each pair (Woelfel, Cody,
Gillham and Holmes, 1977) . These coefficients for all paired
comparisons (manipulated and unmanipulated) are presented in
Table 2.

The coefficients for all pairs ranged from 27.00% to
85.76% with a mean of 51.31%. The mean for the unmanipulated

concepts was 41.86% with the largest coefficient (V = 61.26%)

mafia pound—435:5 u

.1

 

65

 

 

wm.em o~.~m mm.ms mo.Hm mm.mm He.mo mm.~v mo.mm we.mm va.¢m oe.mm oo.¢m mm.mm mm.mv cos»
IMUflSBENuU
Ha.¢m ea.vv Hq.mv mm.sm sa.om ms.Hm mm.m¢ om.om ma.~s mm.mm oo.am om.~m m¢.mv mocwaom
Hmnoom
No.4m He.~v sm.me oa.qm ne.am Gm.mm mm.Hv os.vv om.pm ma.mq mm.mm mm.mm ucmcamcpu
Hm.mv m~.oo m~.~v om.ao sm.~m H¢.~m mm.am mm.~m mm.ao ~a.mm mm.oo coco
oe.om .ma.oe ~5.Hm .oo.>~ mm.oo 5H.ms .mm.am «mv.ms mo.mv mm.av mmfluacmssm
mm.Hm ma.mm ~m.>m mo.ae Hm.mq va.mm ms.m¢ me.vm me.mv oflumnooemo
mo.mm .mo.me mm.mn mm.mm .GN.HG .mo.v¢ om.~m mm.Hm maacummq
ma.mv om.mm ma.ms em.em mm.om am.mm mm.mm deem
Jumamma
om.mm mm.mm «mo.He .o¢.mm ma.me Ho.vm mocmflom
Haddmmam
mo.mm m~.mm om.om mm.am mv.v¢ as
me.om mm.Hm Gm.~m mn.vm ma
.oo.am om.¢a mm.em mnaxcflsa
mm.om mm.mv maflsusum
mH.~m uouosuumcH
unmdxm
mm. m I W m m m . .1.”
w m. . I m. 1 m. . m. m
Wm“ m u. m. 6 m. m. 6 6 m
1...... a. mu. m. d % 1
me I
um m
a
mans sun Bfiom H26 H95 pom—mumps mung
seepage can Hmpcmeanmaxm mo mmauaumaeaammao cam: “on lfimxmc cos u.>c sueaenmeum> mo mucmfioflmmmoo .N magma

66
being associated with the learning-thinking paired compari-
son. This result along with the relatively large coeffi-
cients found for other similar pairs such as thinking-studying
(V = 51.00) and studying-learning (V = 44.09) are not sur-
prising in light of the sample which primarily consisted of
freshmen entering a new social and educational environment.
This suggests that the coefficients of variability for the
unmanipulated concepts in this study reflect more than merely
error of measure. Previous research (Woelfel gt_al., 1977)
has found coefficients averaging below 10% for unmanipulated
concepts as vague and illusive as the ones used here. The
magnitude of the coefficients in this study may very well be
a result of the effects of the overall educational system.
In other words, this study deals with only a portion of the
educational environment to which the student-respondents were
exposed. Educational influences outside the realm of this
study should, then, be entirely expected to increase the var-
iability of responses. Thus, the error of measurement in
this study is likely to be at least somewhat less than what

is reflected by the coefficients of variability.

Manipulations

The coefficients of variability can also be used to
examine the success of the experimental manipulations. Co-
efficients for the manipulated pairs are expected to reflect
differences across conditions and time resulting from experi-

mental manipulations in addition to error of measure.

67
Assuming that measurement error is reasonably equivalent for
manipulated and unmanipulated pairs, differences in the co-
efficients for these two groups are directly attributable to
manipulation effects. The mean coefficients of variability
for the 11 manipulated pairs (Vfi) and the 10 paired compari-
sons (Vu) which contained no experimentally manipulated con-
cept were computed. All paired comparisons involving the
concepts instructor or me were excluded from this analysis
because these two concepts, while not directly manipulated,
would be expected to change over the three month period of
this study.- To examine differences between the manipulated
and unmanipulated concepts, the Behrens-Fisher statistic for
two means with unequal population variances was calculated.
It should at this point be mentioned that the use of this
statistic in this context is illustrative of what would be
appropriate for the ideal case. Since the section was used
as the unit of analysis, for the purpose of providing more
stable means for the experimental and control groups, the
samples were not randomly drawn. Moreover, as was pointed
out at the beginning of this chapter, the participation of
some of the same subjects who contributed to the calculation
of their sectional scores, in more than one of the four data
collections makes the-data correlated rather than independ-
ent. The Behrens-Fisher statistic yielded no significant dif-
ference between the mean coefficients of variability

for the manipulated and unmanipulated pairs. This, however,

68
does not necessarily indicate that the manipulations were
unsuccessful. As was previously stated, the influence of
the educational environment was beyond the control of this
study and consequently expected to contaminate the data.
Some support for this contention is provided in Table 3
which reports the mean variability coefficient for each
concept paired with all other concepts. The mean variabil-
ity coefficient was obtained by averaging the individual
variability coefficients, found in Table l, for each con-
cept paired with the remaining fourteen concepts. In other
words, the unit of analysis here is the concept rather than
the student respondents. Learning and thinking, two concepts
which could be anticipated to undergo change in meaning as
a result of exposure to a new educational system, had two of
the highest coefficients. Teaching assistant, which was the
most substantially manipulated concept, had by far the largest
average variability coefficient. This, however, is not enough
to warrant any claims about the effectiveness of the teaching
assistant manipulation. Instead, these findings seem to sug-
gest that outside educational influences were sufficient
enough to prevent any conclusive statements regarding the

success of the experimental manipulations.

69

Table 3. Means and Standard Deviations for the Variability
Coefficients of Each Concept

 

 

Concept Mean Standard Deviation
Teaching Assistant 57.40 16.14
Learning 53.94 10.42
Thinking 53.64 11.48
Good 53.54 7.50
Expert 52.38 4.48
Confident 51.92 6.50
Communication 51.73 13.52
Instructor 51.04 6.07
Leadership 50.77 6.66
Me 50.33 6.23
Social Science 50.29 5.38
Democratic 49.65 7.03
Studying 49.44 6.00
Physical Science 48.05 8.42
Humanities 45.55 8.72

 

Student Learning

Although the Behrens-Fisher test revealed no signifi-
cant differences in the mean scores for the course content
related paired comparisons between students in the control
and experimental group classes, several informative trends
can be found in the data. In first focusing attention on

general tendencies over time (Tables 4-7), it appears that

70
what substantial changes did occur took place toward the
latter part of the term. There were only minor differences
between the scores in the time one and time two data sets
for both the experimental and control groups. Beyond this
similarity, however, the two groups tended to follow quite
different patterns. For the experimental group, not only
are the first two data sets similar to one another, but, in
addition, the scores from the fourth point in time resemble
the first two. The only distinction being that in all cases
the final mean score was smaller than the initial one.
Changes in this group appear, then, to have been centralized
around the third data collection. More specifically, the
dissimilarities between the selected pairs of content con—
cepts generally show an increase from time two to time three
which is followed by a decrease at the fourth point in time.
The communication-good pair, for example, jumps from a mean
score of 42.35 at time two to 71.52 at time three and then
comes back down to a final score of 36.97. This pattern is
clearly illustrated in Figures 1 and 2.

While a similar tendency was found in the control
group for the communication-leadership and leadership-
democratic pairs, the remaining paired comparisons do not
seem to follow any single consistent pattern over time. If
anything, there merely seems to be much less variation be-
tween the mean scores over the four time intervals. Any

sizable differences that were found are associated with

71

 

 

 

 

and was mm
Adam.amc loam.5mc 144m.omc
dem 52.8 mmmdm ms pm 838358
«NH was em
lenm.mmc lemo.aoac loam.oqc
mmmé. 9.4.2. «9...? 88 can 83851.80
mmH Add ma
imam.~oac Amvo.ohc Aflmm.mmc
Hm¢.Hm ego.mm Hmn.mm ogumuooamd new dflsmumagma
med mma mm
lemb.mmac 15-.mmc lmvo.flmc
mmm.om ~55.mm hop.om ogumuoosmo can :oaumofiaasspu
«ma has mm
lama.omc lemm.mmc lama.~mac
mmm.m~ ema.om vmfl.vm magnumnmmq new coeumoecuesuo
mma was mm
leflm.eaac leam.~vv Amma.mmc
8me 25.2. 839,. 888m 8E8 65 83855
Hmucmeaumaxm Houugdu coaumueuo “and uamogpu
:oauguggo

 

95 THE. um museum HOHEOO new gaging
How 33080 ucmusou mo mam mag 83m 398m can . 203mg pants—mum .8053me :82 S 033...

72

 

 

 

om moa mm
imma.mmc imma.moac Assm.omc
oom.om mma.ns mam.m~ m: can coaumoflcgaapo
ma Add em
lamm.mmc lomm.mmc loam.oqc
emm.~s mac.~m ~mm.am cooo.o=m coaumoaaaaspu
mm H~H mm
isom.mmc Aopm.ch Aaam.wmc
mmm.mm Nem.mm Hme.mm oflumuoosmd_ncm dwamHmcmmq
\ mm mma «a
Aaem.oec 1vm5.mmc Amvo.amc
80.3 «3.3 Show 63888 new 838%
mm mod mm
Ammo.amc Aamm.msc AHmH.~mac
318 118.3. «2.3 mg 8.... 2038560
mm HHH om
laco.mmc lawm.evc Amma.mmc
hmH.ea hev.me mea.mm mocwaom Hmaoom_ccm cassavaczsunu
.Hmugmsgumdxm Houucoo goaumuauo “Add uamoaao
coauedcpu

 

03H. mafia. um mg Hausa can Hound-Fug
How mumwoaoo aficoo mo muemm 95:4 mmuwm 395m pom .moowumgmo gm .gmflo saga

.m OHQMB

73

 

 

 

me he mm
lhna.mvv Aomm.mmc lqvm.omc
43.8 «5.8 888 ms 8 81.3888
we we em
14mm.mmac Amoo.mvav loam.o¢c
«3.: $83 «8.2 88 68 8883.88
mm em mm
loom.mmc imam.mmac Aamm.mmc
mom.mm nom.H> Hme.mm oaumuooemd can magnumcmuq
he Hm mm
Ammo.mec Amnm.>¢c Amso.amc
www.mm Names 858 03% pan. GOUOOEEEO
me me am
lemm.Hmc 1me~.msac lama.mmac
85.8 Stem «2.3 .5888 68 88.8358
me me mm
.mmm.~vac Amma.>hc Amma.omc
GEE. 23.3 Seam 888m 888 o8 5.3.88
ashamemumaxm Honucoo comumueuu Hand havocuo
coaufiocoo

 

OOHF mafia. um 83on H9380 can. afififig
8m «88:8 83:8 «0 88d 898 83m 398 68 .8388 88m . «8.8me 892 S 638.

74

 

 

 

we as am
love.mmc lemo.omc lqvm.omc
08.8. 18.8 328 ms 88 83888
mm mm hm
Asma.¢oc Ammp.mmc loam.oec
oem.mm mme.~q ~mm.am coco new :oflumoacsssuo
as me we
1me~.moec lomm.Hsc Adam.mmc
Hen.m¢ qvm.em Hm>.mm ogumuoosmo new danmuonamH
on op mm
Imam.moac lom~.mmac Amqo.Hmc
08.8 Ema» Shem 63688 9a. 88888
ch mm mm
Aaee.amc Ampm.mmav lama.~mac
mmm.m~ eao.aq vmfl.vm magnumcmmq_cam coeumoflgzsspu
me no em
Imam.moac inmm.emac Immfl.mmc
~88 838 81mm 888m 88 68 8388580
gem—Eng HOHEOU GOMHOWHHU Hun—mm £888
coaugcgpu

 

upon." mane um mgouw H9380 can. 32.396
How muemoooo £8.58 mo mam mag—m mONHm 396m paw . macaw—Symon 35m . $08930 com: .5 03.3.

 

75

 
    

 

 

120
110
100
90
80
70 (70/161)
60 ,(81 186)‘ 631905 omtml
I ' Group
50 ~ ~ \ \. Experi-
(54.382): ‘ ‘ ‘, (53.892 nental
0
4° (45.948) (46.137) 3m“?
30
20
10
0
1 I I I
Oct 1 Oct 15 Nov 4 Nov 20
Figure 1. Plot of the Cbmmicatim—Social Science Paired
Oarparison for Experimental and Control Groups
Across Fbur Points in Time
Key: —-—- Control Group

——- Experimental Groxp

120
1.10
100
90
80
70
60
50
40
30
20
10

Key

 

76

(71. 522)
/'\

I \
I

\
(52.018 ,’ (52.553)\
4 \

/ /' \
(48.155) , ’ \

 

 

-....—----/ (42 769) Control
0" 0
(40.556) 42'3“ X Grow
36.970 Experi-
rental
Grow
I I I I
Oct 1 Oct 15 Nov 4 Nov 20

Figure 2. Plot of the Oamunication-Good Paired Oarparison
for quaerimental and Control Grows Across Fbur
Points in Tine

Control Grow

---- Experimental Grow

77

pairs involving the concepts leadership and democratic, or,
in other words, the more directly taught course material.
There are several plausible explanations for these
findings. First, two of the concepts, leadership and demo-
cratic, were the only ones being examined which were actually
taught to the students as specific course material. It is
not surprising, then, to see pairs including these more
directly manipulated concepts to experience more variation.
Moreover, these concepts were not taught until just before
or after the time of the third data collection. Consequently,
the three paired comparisons which include at least one of
these concepts would be expected to have remained relative-
ly stable in the early part of the term. A second explana-
tion deals with the all encompassing nature of several of
the paired comparisons being examined. The relationship be-
tween communication and social science, for example, may not
become particularly clear to students until they have had at
least a minimal exposure to the course. In other words, it
may have taken a while for the students to put specific
course content into an overall assessment of where the field
of communication fits into the larger academic picture.
More specific content pairs should, then, show more immediate
results. These findings may also have been a result of the
execessive variability found in the data. The high standard
deviations reported in Tables 4-7 suggests that the mean

scores are probably not stable enough to obtain a clear

78
picture of the differences between the experimental and
control groups. A final contributing factor to these results
may be found in the response burden placed on the student
participants. Although participation was voluntary and the
students presumably became more proficient in filling out the
questionnaires, they may have become apathetic or even a
little put out by the longevity of the project. This is at
least somewhat reflected in the sizable decrease in sample
sizes for the final two, and particularly the third, data
collections. Moreover, the lack of participant awareness
regarding the value of repeated measures may have also con-
tributed to less care being exercised in filling out ques-
tionnaires.

Prior to making more specific comparisons between the
experimental and control groups, one last general finding
should be briefly mentioned. It should, however, first be
more explicitedly pointed out that the criterion scores are
assumed to represent a standard that students appropriately
learning course material should proceed toward. Of the six
paired comparisons reflecting course content, only one,
namely the communication-me pair (Figure 3), experienced
relatively continuous movement toward the criterion score.
While it is tempting to discuss the implications for this

particular pair, it seems more apprOpriate to suggest that

79

 
   
    
  

 

 

120
110
90
80
70
60
50
40 (47.138)
(40.127)
3° 30.800
20 (38376)”‘ - - - N___ _ Control Grow
“”""~-..-~ (30.934 .
10 ~ - . ri-
26.500 mental
0 Grow
J L I J
Oct 1 Oct 15 Nov 4 Nov 20
Figure 3 . Plot of the Camdatim—Me Paired Carparison
for Experimental and Control Grows Across
Fbur Points in Tine
Key: —— Control Grow

--— Experinental Grow

80

these findings demonstrate the complexity of the learning
process.

A more thorough examination of the data relative to
specific criterion scores reveals several noteworthy dis-
tinctions between the experimental and control groups. The
mean scores of three of the six pairs in the final data set
for the experimental group were closer to their correspond-
ing criterion scores than at any other point in time. How-
ever, one of the remaining scores was at that time further
from the criterion score than it had ever previously been.
For the control group, just two of the six pairs had final
mean scores that were closer than at any of the three pre-
ceeding time intervals. Moreover, three of the other four
paired comparisons had final scores that were then furthest
away from their respective criterion scores.

At the initial data collection (Table 4), scores
from only two of the six pairs for the experimental group
were closer to the criterion scores than their counterparts
in the control group. On the other hand, at the fourth and
final point in time (Table 7), the scores from the experi-
mental group were closer to their specific criterion score
in five of the six content pairs.

Although these results seem to slightly suggest that
students in the experimental group classes progressed more
than students in the control group classes, they are, at

best, only a weak indication in that general direction. It

81
should again be pointed out that there were no significant
differences found between the experimental and control
groups and that the criterion scores were at least somewhat
questionable. Thus, no conclusive distinction can be made
regarding the extent of student learning that occurred in

the experimental and control group classes.

Instructional Performance

The time one through time four data sets for the in—
structional performance paired comparisons are reported in
Tables 8 through 11 for the control and experimental groups.
Prior to examining these data, it should first be pointed
out that the sample sizes shown in these tables usually ex-
ceed the number of student participants reported in Table 1.
Although this appears to be an inconsistency, it is actually
a result of several of the class sections having more than
one teaching assistant. More specifically, two sections in
the experimental group and four sections in the control
group had two assistants. Consequently, each of the student
volunteers in those sections responded to the teaching assis-
tant-attribute pair twice, once for each teaching assistant.

The fact that several of the subjects made assessments
of the instructional performance of more than one teaching
assistant poses a problem.with the analysis of these data.

It was previously pointed out that the data across time was

correlated because several subjects participated at more than

82

 

 

 

ops mos

smsm.mssc sshm.mmc

mam.om ooo.nm ooo.o ooooonumos coo sooumsmmagmosnooog
mos mos emN

smoo.-sc Ammo.mmv lmso.smc

oms.pm «mo.mv mom.mm powossooo_oom somumsmma.mosoooma
mos mms mom

smm~.m~sc smmm.soc sswo.~ssc

omo.mo ~o~.vm sms.mo osuosooemo_oom pooomsom<_mosoooma
oos mos mom

Amps.ossc Ammo.soc As-.n>c

oso.mm smmdm 83mm .1288on 68 88682 65868.
mos ems smm

Ammo.mmsv iaeo.mmsc smom.~mc

omm.~m moo.oss omm.om soooxm now pooumsmma.mosnumma

souomsssooxm sosucoo dossmusso usmm ummoooo
oososoooo

 

08 THE. no 6995 39.500 Odo 33% Mom
madwocoo 303% m0 go 02 mmusm 308m o8 .mgflmgma pudendum £85.83 882 .m manna

83

 

 

 

mms mos
loco.mmc s~m~.smsc
mo. mom.mm msm.oo ooo.o oopoooumos ooo sooomsmma.mosooome
mms ems has
semm.moc som~.mmsc soma.~oc
mo. mos.ms ~mo.o> mae.mm homossooo.oo6 oomumsmmagmosooooe
«ms oos pom
loam.mosc laem.ss~c smeo.smc
m: oss.oo mmo.sm mmm.os osumooosoo_oom somomsmmaxmosooooe
mas cos mom
imam.emc lomm.mmsv lama.~osc
mo. 3&8 RES 80.2 .1888 o8 unfimsmma 888.1,.
mms sos mom
soo.ooc isos.eomc lmsm.ooc
so. ohm.mm sne.mos sme.ns sowoxm oom somumsmm¢.oosoomoa
m 38.5% soHEou ooflmfiso Husmm ”Emooou
oesosoooo

 

03s. mesa. no 6995 sonucoo o8 35.5% How
mummocoo Hmsofloshumfi .uso mus—om mag 83m msdEmm can 5:03ng ego. .mmofimumfla cogs .m manna.

84

 

 

an op

ssoo.oov loos.soc

smo.~o oso.oo ooo.o ooooouomos_ooo poooosmmogmosooooa
oo on no

somo.s~sc sons.omc sous.soc

mos.oo ohm.~o asm.oo .oomosoooo new sooumsomoxmosooooa
se me mos

Aeoo.omsc Amos.oosc “moo.~mc

«3.3. osmsm 898 688098 8 88884 65888.
mo me no

Iooo.oec soao.mosc 1oo~.ooc

838 Show omooo 88883 new 6.8884 6888
me me no

loom.eoc lomo.mosv iono.mmsc

Sooo oomam moods. 888 s8 8838a 65.888.

sanguondsm saucoo cosmsmuwnu Mamas udwosoo
oososoooo

 

mummosoo smsofloaumss .uso mam 955

mass. mews. um £5on .3380 cam Egg How
.88 osofio o8 .88688 88o . 80888 :82 .2 osons.

85

 

 

«a um
Amov.mmv Aomm.omv
ma mmm.mm omm.¢¢ ooo.o uopusuumcH cam unnumflmm¢.mcflnumma
mm mm mma
Anam.omv Amhv.mnv Adam.omv
ma vaa.vm «mH.Hm vuv.mm uamcflmcoo_oam u:mumamm4.maflnumma
mm mm H¢H
Ammv.¢mv Avmm.mmav Am¢~.fimav
mo. 83:. 35k. 52.8 038092 98 unfiflmg E89
mm mm mma
Aama.m~v Aamm.qoav Avmm.mmv
mo. $98 :98 mS.mm 3:9."ng cam uqfiflmmm mange
mm mm and
Aomo.mmv Amam.hvav Anom.mmv
m: ovm.nm 5H¢.>m mmH.mm unmmxm new unnumflmm¢_mcflnumma
m afiggfl HOHEOU :oflumufluo “Ham ”Emocou

 

cofluflncoo

 

Hsom meg. um mmsoaw Hoficoo can 32% Mom
mummocoo adsofluoaumfi mo mam 955m mwufim 398m 98 .mcowumgmo 65:5...“ .mmocmumflo :82 .HH manna.

86
one point in time. The issue raised here is a somewhat dif-
ferent problem of correlated data. Rather than data being
correlated across time, the data is instead correlated with-
in each of the time intervals. The problem is essentially
that each response to the teaching assistant-attribute pairs-
is not independent. The responses given by each subject who
rated two assistants are interrelated and as such violate the
statistical assumption of independent observations. Thus,
the use of the Behrens-Fisher statistic in this context
should be taken as an illustration of the analysis that
should be performed when independent observations have been
taken. It should be mentioned that most courses have only
one instructor. This problem would, then, not be expected to
be present in most educational situations.

As was the case in the previous set of analyses, it
seems to also be informative here to first explore general
trends in the data over time. The only distinctive trend
that appears to hold across both groups is the presence of a
consistent decrease in the mean scores from the third to the
fourth point in time. Focusing attention on the experimental
group data first, it seems that while in the previous analy-
ses, the mean scores experienced relatively large variability
over time, the scores for the instructional performance
pairs are comparatively more stable. With the exception of
a substantial decrease for the teaching assistant-expert

pair from time one to time two, the rest of these data seem

87

to follow the same general pattern as the content pairs.
The changes that occur are, however, much less substantial.
There are, then, minimal differences between time one and
two scores, with a slight tendency for the latter to be
smaller. An increase at time three is then followed by a
decrease at the last data collection which makes the final
mean score always smaller than its corresponding time one
score; this can be seen in Figures 4 and 5.

The control group had more variation for the instruc-
tional performance pairs. There was, however, no uniform
pattern for the six pairs over time. Probably the closest
thing to an overall pattern is illustrated in Figure 4 where
the mean score for the teaching assistant—leadership pair
increases from time one to time two, then generally levels
off between times two and three, and finally decreases from
the third to the fourth data sets. Although there are
noticeable deviations, the data from the six pairs, when
taken together, roughly fit this pattern.

Differences between these general patterns for the
control and experimental groups can be at least to some
extent attributable to the training program. When the ini-
tial set of data was collected, only the first week and a
half of the term had been completed and only the first of
the four training sessions had been run. It is not sur-
prising, then, to see that mean scores for the two groups

are initially quite similar. Moreover, the differences

88

    

 

 

120
110
1.00
(89.780)
90
80
70
60 (53 (58.203)
50 . (45.748) ’ ’2“ 60.071 Control Group
(50.3514 ‘ .. I ' \
40 ~..’ I \ s
\
3o ‘,.
(33.848) Experimental Group
20
10
0
l 1 l J
Oct 1 Oct 15 Nov 4 Nov 20
Figure 4. Plot of the Teaching Assistant-Leadership Paired
Omparison for Experinental and Control Groups
Across Four Points in Time
Key: —- Control Group

--—Experinental Group

120
110
100
90
80
70
60
50
40
30
20
10

 

89

 

 

(64.218)
(57.090)
x (471321) (49.320) Control
(50.995L‘ , , ‘2 \ 4‘— Group
~\ ,I (46.618) ‘~ ~
\ z ‘0
‘N’ ’ (39.638) Experi-
(35.303) mental
Grow
I l J l
Oct 1 Oct 15 Nov 4 Nov 20

Figure 5. Plot of the Teaching Assistant-Instructor Paired
Comparison for Experimental and Control Grows
Across Four Points in Tine

-—— Control Grow

--—-- Experimental Grow

90
that had begun to develOp several weeks later are quite ex-
pected. But the tendency for the trained teaching assist-
ants to become more strongly associated with the specific
attribute concepts was only slight at best. Instead, the
difference between the experimental and control groups re-
sulted from increases in the teaching assistant—attribute
pairs for the control group. In other words, it may have
been that rather than helping the trained assistants to be
perceived more positively, the training program may have
merely prevented them from becoming perceived more negative-
ly. Students may have, at the early stages of the term,
overestimated their teaching assistants. The training pro-
gram could have enabled the trained assistants to quickly
attain that level of performance while the lack of such
training showed up in the reassessments made by the students
in the second data set. Increases in mean scores for the
experimental group from time two to time three may be a re-
flection of the training programs completion. That is, the
teaching assistants in the training program may have reach-
ed a peak level of performance while they were learning to
improve their classroom skills. A slight decrease in their
performance may then occur until sufficient time has allowed
them to improve by actually putting their skills to use.
Along these same lines, the consistent decrease in the mean
scores at the final point in time may merely be a function

of experience. That is, all of the teaching assistants,

91

whether they were trained or not, presumably improved their
classroom skills as the term progressed. This would, of
course, be expected to show more as they gained added ex-
perience. Thus, decreases in scores from the third to the
final data sets for both groups may be accounted for by
direct practical experience.

One additional finding should be mentioned before
proceeding with more specific comparisons between the in-
structional performance pairs for the control and experi-
mental groups. There is an exceptionally close resemblance
between the mean scores at each point in time for the teach-
ing assistant-leadership and teaching assistant-confident
pairs. With the exception of the third data set in the con-
trol group, where the difference was 26.81, the scores are
otherwise very similar. The differences range from .35 to
3.82 in the experimental group to a high of only 9.51 in the
control group. The two pairs also follow identical patterns
over time. While this similarity may merely reveal a strong
covariation, it may also be an indication of the importance
of building the teaching assistants) confidence in their
leadership skills. In other words, at least perceptions
of the teaching assistants' leadership ability may be in a
large part dependent upon the confidence that they display
in the classroom.

In discussing more specific differences between the
control and experimental groups in relation to the criterion

scores, it should first be recalled that the criterion score

92

for each pair was allowed to vary over time. The instruc-
tor-attribute mean score for each point in time was used as
the criterion. This enabled the criterion scores to adapt
to the changing demands of the course as modeled by the
instructor. Assuming that improvement in instructional
performance should be reflected by mean scores converging

on their respective criterion scores, the experimental group
would appear to be the most improved over the term. That
is, three of its five pairs were closest to the criterion
score in the final data set. One of the remaining scores
was at that time further than it had previously been in the
preceding three data sets. None of the time four scores for
the control group were either closest to or further from the
criterion scores. These data additionally show that the
mean scores for the experimental group were generally closer
to the criterion scores than were their counterparts in the
control group. Mean scores for the experimental group were
closest for three of the five pairs in the initial data set
(Table 8). Over the next three data collections (Table 9-
11), the experimental group was closer to the criterion in
fourteen of the fifteen comparisons.

The indication in these findings that the trained
teaching assistants were perceived to be more closely ident-
ified with appropriate levels of teaching performance is
further supported by several significant findings. However,

these results must be interpreted with extreme caution since

93
the Behrens-Fisher statistic does require independent random
samples and has been previously mentioned, the data examined
here is to some extent correlated. The discussion of these
findings should thus be treated as a demonstration of what
might result with a more apprOpriate data set. Of the six
significant differences between the instructional perform-
ance pairs for the control and experimental groups, four
occurred in the second time period (Table 9). In other words,
there was a perceived difference between the trained and un-
trained assistants during the time in which the training pro-
gram was being held. At this time, all but the teaching
assistant-democratic pair yielded significant differences
and even that pair approached significance. In all cases,
the mean scores for the experimental group were closest to
the criterion. The remaining two significant differences
between the experimental and control groups occurred in the
final data set (Table 11). It is especially informative to
note that the teaching assistant-leadership pair (Figure 4)
was the only pair that was significant in both the second
and fourth data sets. At both times, the mean score was with-
in a single point of the criterion score. It should be
further pointed out that while there was no significant dif-
ference between the experimental and control group for this
pair at time three (Table 10), there is still a sizable dif-
ference (31.58) between the mean scores. The teaching

assistant-leadership pair is of particular importance

94
because it reflects most directly on the training program
which, of course, was specifically intended to improve the
teaching assistants' leadership skills.

In summary, there were fifteen possible comparisons
between the mean scores for the control and experimental
group after the initial data set. Six were found to be sig-
nificant and four others had substantial differences of over
30.00 which approached significance. In all but one case,
significant or not, the mean for the experimental group was
closest to the criterion. While these results are encour-
aging, the 1ack of additional significant findings suggests
that caution against over interpretation should be exercised.
However, two conclusions do seem to be warranted. First,
the trained teaching assistants were perceived to be more
closely identified with appropriate levels of teaching per-
formance during their participation in the training program.
Secondly, the trained teaching assistants were more strongly
associated with the concept of leadership during and after

their participation in the training program.

Adjunct Analyses

In the primary analyses, individual student responses
were aggregated for all sections of the experimental and
control groups. By using all available volunteers at each
time interval, the samples were not random. Instead, the

data, although treated as random, was correlated due

95
to some subjects participating more than one time. In the
analyses to follow, the data from students in each section
was aggregated and then each section in turn individually
contributed to the experimental or control group mean scores.
While this procedure does not produce random samples, it
should make the mean scores more stable. Only those sections
who had students participating in all four data collections
were used in these analyses (see Table 1.). This procedure
however, considerably reduced the sample sizes (experimental
group, N=6 sections; control group, N=9 sections) and as such
made any statistical comparison of means a relatively sterile
endeavor. A discussion of general over time trends would,

in any case, be informative.

Course Context

 

The effectiveness of the instructor, while expected
to influence the rate and extent of student learning,
would not necessarily be expected to also alter the overall
learning process. In other words, the process by which stu-
dents internalize course content would generally be expected
to remain relatively constant. However, there appears to be
no predominant trend that is entirely consistent for all six
content pairs across the control and experimental groups
(Tables 12 and 13). This is particularly evident in the
control group data where changes over time seem to follow no
trend whatsoever. Each of the pairs had their own unique

pattern over time. On the other hand, each of the pairs in

96

 

 

 

m m m m

Asmo.omc imam.o~c Amm¢.ovv 1H66.6Ho

moo.sv mmo.mm www.mo smm.ov coco new coaumoflczsego
m m m m

imam.wac Aomm.mac Aoem.msv mmm.vmc

$98 Seam £33 «3.8 m: 68 8389280
m m m m

xamm.m~v Aomm.amc Amvm.oav Asmo.o~c

Hmm.mm mam.~m vso.m¢ ~HH.¢6 mflamumnmmq 6:6 owumuooamo
m m m m

loco.~6c xomm.m~c AH~6.6H. Aamn.mac

68.? «2.8 $6.8 mom.mm 686898 6:6 83ng
m m m m

Aamv.mdo Ammm.mav AmHN.HHV roam.sav

oma.mm 6mm.m¢ mmo.~m pom.am annmumnmmq 6:6 coflumoncnaaao
a m m m

imam.mmc Amm¢.mmc Asso.amc Amma.mc

2.6.3 89% Scam $6.8 8:38 888 can 838%

me me «a He uflmm “avenue
gofluflecpo

 

uses an mucsom uses mmonu¢_msono Houucao

m5 MOM mummocou ucmpca mo muflmm mega mmNHm 395mm cam .mcoflumgma Emofim .mmocmuma com: .2 3nt

97

 

 

 

o o o o

Ammo.HHV Amoo.mmv Amam.omc roam.moc

mam.mm mma.s¢ mmm.ao omo.vm 6600 now :oflumoficossoo
o o o o

Aaoo.mo Amw>.mc Aooo.mav roam.~mc

$93 36.8 ommém ST? 62 com 8363558
o o o o

Amao.mc AFNH.mHV xomm.mo roam.omv

ooe.mm mom.om mmm.a¢ mom.vm oflomumommq com oflomnooamo
o o o o

Ammo.oac Ammo.omv Aamo.mav loam.ovc

moods mmoém 25.3 86.3 03669.8 6:6 888%
o o o o

Aamo.mv Ammm.mav Aooo.mac Amom.oov

898 ST? 858 moods oflhmumomfl com 838%
o o o o

Aomm.ov xoo~.-v Aooo.o~o Ammm.mmv

ovm.mv mnm.ev hmo.mo oao.mm ooooflom Hmnoom new ooflumoflcoesoo

so me me He memo oomoooo
£03638

 

9a. 5 350m Moon 383 9..on Egg
on» How mumwocoo paucoo mo muflmm mega mmnwm 398m pom . 9536ng gum . moocmumfla com: .3 manna

98

the experimental group, with the exception of the democratic-
leadership pair, tend to follow the same general tendencies.
While the changes over time are not substantial, they are
nevertheless quite consistent. For these pairs, there is a
uniform decrease in the perceived discrepancy between the
paired concepts from the first to the second time interval.
There appears to be only minimal change from time two to time
three. A decrease in mean scores then occurred from the
third to the final points in time. Generally, this appears
to suggest that for the experimental group, the concepts in
the content pairs became more strongly associated with one
another as the term progressed.

This convergent tendency for the content pairs in
the experimental group is even more apparent when comparing
changes in the two groups. For five of the six pairs in the
time one data, the perceived differentiation between the con-
cepts was greater for the experimental group than for the
control group. In contrast, on the final data collection,
all six of the paired concepts were perceived to be closer
for the experimental group than for the control group.

Before proceeding to an examination of the instruc-
tional performance data, it is of interest to note the simi-
larity in over time patterns between the experimental and
control groups for the communication-me and leadership-
democratic pairs. While the remaining four pairs had quite

different trends in the two groups, the parallel findings

99
for these two pairs may be an indication of the importance
of the particular course material being taught in the learn-
ing process. This may in fact explain why no overall tend—

ency was found for the entire data set.

Instruction

 

Instructional performance pairs for the control and
experimental groups (Tables 14 and 15) both seem to follow
a quite consistent but different general pattern over time.
Very little noteworthy change occurred in the control group
over the four time intervals. In other words, student per-
ceptions of instructional performance for the untrained
teaching assistants remained relatively stable throughout
the term. Conversely, the experimental group did experience
some change. In particular, substantial decreases in the
values for the teaching assistant-instructional characteris-
tic pairs occurred from the first to the second time periods.
A slight increase then took place from time two to time
three which was followed by a relatively consistent decrease
moving from the third to the final data sets. Thus, the
trained teaching assistants appear to have generally become
more strongly associated with the relevant instructional
performance concepts.

In contrasting the two groups, the yet to be trained
teaching assistants in the experimental group were furthest
from the instructional performance concepts for four of the

five pairs at the initial point in time. At the second and

100

 

 

 

m m m m

Ammm.~mc Imam.mac Amom.mac xema.mmc

mmm.ov omm.sv He~.oo osm.nv oopoouomoHroom oompmnmmm.oonnomoa
m m m m

xsoH.oHc Amqo.mac Amom.oac Ammo.mflc

oev.ov mso.~6 osa.eo emo.eq uooonmcoo com oomomnmm<.oofloowma
m a m o

lemo.6~. loo~.~mc Ammm.oac Ammm.mac

HoH.oo onm.om mmm.pm oom.~o ooomuooamo 6:6 oomomnmma_ocnnomma
a m m o

Amoa.o~c Ammm.H~c xmoH.-o loom.mac

23.5 63.8 86.2. 29?. oflfloomfl 68 6.8888 82668.
m m m m

Avo~.mmc 16mm.amo logo.mac 1066.6Nc

moo.me mom.eo 6o~.oo om~.va oumoxm.ocm oomomnmma.oonoomms

NH. me me He memo “58:8
8368

 

gag

How 958:8 Hmsofluoaumfi mo muflmm mg mmwflm mam—5mm tam. . mcoflwfirmo Empgm .mmocmumfln cog

mufiom “Boa mmouom 9380 HOHHEOU on”.
. 3 manna

101

 

 

 

o o o o

lemo.mao xemm.mac lmao.mac Amma.m~c

omm.om omv.mv omo.mm mam.mo HopoouumoH 6:6 oomomnmmm.mongomoa
o o o o

Amen.oav Amma.omc loom.mmv mmm.mmac

mmm.am mHH.om -H.>o omo.mm osmoflmcoo 6:6 oomumfimmm_mowaomoe
o o o o

“who.mao Aomv.mmc Aamo.oao Aooo.omc

833 86.8 03.8 313 6388.8 68 23682 ofifimoe
o o o o

Amoo.~Hv loqe.amc Amme.vac omo.amc

mav.vm oms.vm smm.mv moo.mv oflomuoommq 6:6 osmomflmm¢.oofloomme
o o o o

loos.omc Amom.~mo Ammm.mmo Amoa.amac

mme.mm omo.~o mmo.mm usm.oms ouooxm 6:6 homomflmm¢_ooflnommp

69 me me He “Ham poooooo
COHUEGQU

 

gm. 5 350m .58 mg 9090 Wyoming 9.?

How mumwocoo 1:030ng mo wudmm mag momwm 398m can $9039.98 885m .mmocmumfia coma

. ma 3”nt

102
third time intervals, there appears to be little perceived
difference between the untrained and the trained teaching
assistants. In the last data set, the trained assistants
had become more closely identified with all five of the in-
structional assessment concepts. These findings seem to sug-
gest that there was at least a perception of improvement in
the instructional performance of the trained teaching assis-

tants.

Summary of Results

Preliminary analyses on the precision with which stu-
dent respondents were able to use the measurement system
revealed relatively high coefficients of variability, par-
ticularly for unmanipulated pairs that included educational
concepts of a general nature (i.e., learning, thinking, and
studying). Because the sample consisted of freshmen enter-
ing a new social and educational system, it was suggested
that these variability coefficients reflected not only measure-
ment error but also student exposure to a new environment.
This noise in data may also at least partially account for
the absence of a significant manipulation effect. That is,
concepts associated with the experimental manipulations
would be expected to experience more variability than con-
cepts that were not attempted to be altered. Such a differ-
ence was, however, not found in this study.

In the major analyses, no significant differences

were found in the extent of student learning between the

103
experimental and control group classes. However, while the
training program was in progress, the teaching assistants
who participated in it were perceived to be more closely
associated with effective levels of classroom performance
than teaching assistants who did not participate in the pro-
gram. Moreover, the trained teaching assistants continued
to be more closely associated with effective leadership
through the remainder of the term. It appears, then, that
while the trained teaching assistants may have been per-
ceived to be more effective in their classroom performance,
their effects did not result in more student learning.

An adjunct analysis of the data over time revealed
no specific overall pattern of change. There was, however,
an indication that the pattern of learning seems to be de-
pendent on the course material being taught. The over time
trends for the instructional performance pairs appear to
support the major analyses. That is, the trained teaching
assistants were perceived to have improved on their class-
room performance while no change at all was perceived in the

instructional performance of the untrained assistants.

CHAPTER V

DISCUSSION, RECOMMENDATIONS, AND SUMMARY

Discussion

The results of this study did not reveal any signifi-
cant differences in the extent of student learning that
occurred in classes aided by trained teaching assistants as
compared to classes that were aided by teaching assistants
who did not participate in the training program. While the
lack of more student learning by students taught by the
trained teaching assistants may suggest that the training
program was unsuccessful in substantially improving teaching
effectiveness, there are several alternative interpretations
for these findings. The responsibility for student learning
in this course was placed on the instructors and not their
assistants. The influence of the instructors undoubtedly
contaminated differences in effectiveness between the train-
ed and untrained teaching assistants. It may have also
been the case that the students sampled in this study did
not learn a great deal from the small group exercises in
which they participated. The more important determinant of
student learning may have instead been lectures or other

types of more direct instructional techniques which are
104

105

among the duties assumed by the instructors. The lack of
significant findings may have also resulted from the in-
ability of the measurement procedures to adequately assess
student learning. In other words, differences in student
learning may have actually occurred but were not able to be
detected. Without additional information, it would be dif-
ficult to weight these possibilities. It is, however, im-
portant to acknowledge their presence and, in light of them,
to discuss the implications of these results for the learning
process, the measurement of learning, and the teacher train-
ing program.

The most predominant feature demonstrated throughout
these data would appear to be the lack of any uniform
pattern of student learning. Although the adjunct analyses
revealed a slight tendency, in the experimental group, for
the concepts in the content pairs to converge, the overtime
sequence of mean scores for the content pairs was generally
non-monotonic. Changes were not always in the same direction
and, for this student sample, did not result in a reduction
in the difference between the mean scores and their respec-
tive criterion scores. Moreover, the mean scores did not
oscillate about the criterion scores. While there were
similarities between some specific pairs, there was enough
deviation to prevent any conclusion to be made regarding
student learning.

An examination of the trace from the various spatial

coordinates matrices (Tables 16-23 in Appendix B) may be

106
useful in providing additional insight into the general
nature of the learning process. It should first be recalled,
from Chapter II, that the spatial coordinates matrix is con-
structed to be a square matrix consisting of orthogonal di-
mensions as columns and the projections of the concepts on
each of the dimensions as rows. Variances of these projec-
tions are represented by eigenvalues which when summed across
the matrix represents the total variance in the matrix re-
ferred to as the trace. In terms of learning, it would
appear that effective teaching of specific course content to
a class of students would yield a relatively uniform way of
organizing that material in the cognitive structure. Con-
sequently, variability should be low. It should also be
pointed out, however, that if the goal of the course was to
promote individualistic points of view, success in attaining
this goal would be reflected by a relatively large trace.
The data used here to obtain the spatial coordinates came
from the primary analyses where individual student responses
were aggregated across all sections for the experimental and
control groups. The interpretation of these findings are,
once again, restricted by the inappropriateness of this cor-
related data set. Consequently, the analysis primarily
serves an illustrative function. In this particular study,
the representation of the trace across four points in time
for both the control and experimental groups (Figure 6)

clearly illustrates the differences in the total variance

50000
45000
40000
35000
30000
25000
20000
15000
10000

5000

0000

107

(39696.560)

  
 
     

(33510.806)
.\

(29297.841)

31461.348)

 

 

‘ \ \ , ’ \ Grow
\ Of \
(22858.393) \‘
\
3

16014. 908 Experi-
Rental

Grow

I J I J_
Oct 1 Oct 15 NOV'4 NOV 20

Figure 6. Plot of the Trace (Total variance) fronlthe Co—
ordinates merix for Experimental and.Cbntrol
Groups Across FDur Points in.Tine

-————-Cbntrol Gtoup

--- Experimental Group

108
for each of these matrices. It appears that after the in-
itial data collection, the experimental group maintained
less variability. This is particularly evident at the end
of the term where the total variance for the experimental
group (16,014.91) was approximately half of that for the con-
trol group (31,591.12). An examination of the trace would
appear, then, to be informative and should be further pur-
sued in future research.

In the previous chapter, it was suggested that the
all encompassing nature of some paired comparisons may have
resulted in changes being delayed until students had ab-
sorbed enough course material to draw some general conclu-
sions. Information of this kind was not specifically
taught to students as course content but should eventually
have been affected by it. On the other hand, pairs includ-
ing concepts (leadership and democratic) that were directly
covered by course materials showed more immediate changes.
This would seem to suggest that different methods of teach-
ing (i.e. direct versus indirect) may have resulted in
entirely different patterns of learning.

In addition to bringing up the potential affects
that various teaching methods may have had on student learn-
ing, it may also be informative to consider factors which
influence how concepts change in the cognitive structure.
Researchers have, while using procedures similar to those

used in this study, found that the accumulation of concept

109
relevant information is inversely related to the change in
that concept over time (Danes, 1976; Saltiel and Woelfel,
1975; Woelfel and Saltiel, 1974). Put in the current con-
text, the more information that students had regarding the
content of this particular course, the less change would be
expected to result from exposure to relevant course material.
Student learning may have been dependent upon the amount of
previous information that the students brought with them to
the classroom. The particular method used in presenting con-
cept relevant course material and the susceptibility or re-
sistance of those concepts to change in the cognitive struc-
ture would appear, then, to be important considerations in
assessing learning.

The results of this study have several additional
implications which focus more directly on the training pro-
gram. Five of the seven significant differences between the
control and experimental groups for the instructional per-
formance pairs occurred during the time in which the teach-
ing assistants were actually participating in the training
program. At that time, they were perceived to be more
closely associated with effective levels of instruction.
However, these differences did not hold up in succeeding
time intervals. This would seem to indicate that while it
was important to provide the teaching assistants with early
training that they could put to immediate use, the training

may have been prematurely terminated. Additional, less

110
lengthy, sessions held periodically through the remainder
of the term may have been helpful in improving or at least
maintaining the levels of teaching performance attained
during the early part of the training program.

The need for establishing some sort of standard for
assessing teaching performance is clearly demonstrated by
the teaching assistant-democratic pair in the final data set
(Table 11). For this pair, the criterion score is 59.14 with
the mean score for the control and experimental groups being
77.76 and 43.66 respectively. The criterion score is near
the center of the difference between the control and experi-
mental groups. The teaching assistants from the two groups
are then almost equally distant from what is considered to
be the optimal level of teaching effectiveness for this par-
ticular attribute. However, improvement for the untrained
teaching assistants would require them to become more demo-
cratic. The trained teaching assistants, on the other hand,
appear to have appropriated too much of this characteristic
and as a result have begun to become counter productive.
Improvement for them would call for a less democratic style
and possibly more authoritative leadership. Without such a
standard, as is typically the case in current evaluation
procedures, the changes that are necessary for improvement
are not clearly defined. In the current example, the posi-
tion of the trained teaching assistants could easily be mis-

interpreted as being superior to that of the untrained

lll
teaching assistants. Improvement may then be associated with
becoming more democratic which for the experimental group
would lead to less effective instruction. Thus, the estab—
lishment of a standard representing optimal levels of teach-
ing performance for relevant attributes would appear to be
an essential ingredient in the evaluation of instruction.

The results from the major analyses seem to demon-
strate the most important deficiency in current evaluation
practices. That is, current procedures primarily focus on
the assessment of instructors in regards to certain specific
teaching characteristics (Genova §t_§1., 1976) which is
essentially the same as the determinations being made with
the teaching performance items in the current study. If
this was all that was being used to determine teaching effec-
tiveness, it might generally be concluded, with some limita-
tions, that the training program was successful in producing
teaching assistants that were at least for a short time per-
ceived to be more effective in the classroom. By additional-
ly examining student learning, however, it can be seen that
these perceived differences in effectiveness between the
trained and untrained teaching assistants are not reflected
in corresponding variations in the extent to which students
learned course material. It would appear, then, that evalua—
tions based strictly on teaching performance may be very mis-
leading. That is not to say that examining essential teaching

characteristics is unimportant and should be eliminated from

112
evaluation procedures. To the contrary, such assessments
are informative, particularly for diagnostic purposes. How-
ever, the examination of teaching performance in evaluation

should be used in conjunction with measures of learning.

Future Research

While possibilities for future research were implic-
itedly made in the previous discussion, this section will
provide some more specific suggestions. The most immediate
concern for researchers should be to thoroughly examine the
reliability and validity of the evaluation system. Reli-
ability can be assessed by simply including several paired
comparisons more than once in the instrument and computing
the correlations between the responses to the same pairs.
Concepts not manipulated and whose meaning would not be ex-
pected to change could also be included in the instrument in
order to examine the stability of the system over time.
Additionally, control groups may be employed to examine
manipulation effects. An indication of the participants
ability to adequately use the measurement procedures can be
determined by inserting the criterion pair in the instrument.
Substantial deviations from the score provided for the sub-
jects use would seem to be grounds for assuming that subjects
weren't able or willing to properly fill out the question-
naire. This would appear to be justification for eliminating
such cases from the subject pool. Validity can be addressed

by partialling students according to the grade they received

113
in the course. A strong positive relationship would be ex-
pected between the grade received by the students and their
success in learning the apprOpriate meanings for the domain
of instruction. Moreover, a negative relationship should
exist between student grades and the perceived difference
between the actual and ideal instructors.

Beyond this preliminary work, the research possibil-
ities for this system are enormous. The teaching-learning
relationship is the foundation of formal education. A pre-
cise, reliable and valid measure of teaching effectiveness
which is based on this relationship can be very useful to
educational researchers. One specific application that
should be mentioned is the use of this system to improve the
quality of instruction. The use of specific instructional
performance characteristics in conjunction with some ideal
standard has previously been shown to be able to: (l)
locate Optimal levels at which such characteristics would be
most effective, (2) determine the deviation of instructors
from those Optimal levels, and (3) suggest what might be
done to help the instructor become more effective. In
addition, various teaching styles (made up from combinations
of these characteristics) can be identified. By additional-
ly video taping and analyzing a sample of instructors, spe-
cific behavioral indicators for each characteristic and style
can be found. Used with measures of learning, this informa-

tion can lead to an assessment of the relationship of various

114
teaching characteristics and styles with student learning.
Training programs could then be devised in accordance with
this information.

This system would appear to be most useful in the
multi—sectioned course such as the one examined in this study.
Averaging the data across all sections would enable the di-
rectors of such courses to obtain general information re-
garding the Specific strengths and weaknesses of their staffs.
Training sessions could then be established in light of this
information. The addition of each instructor as a row and
column to the original means distance matrix allows feedback
to be obtained for each instructor in regards to their rela-
tive position to the ideal standard as well as in relation
to their colleagues. If data were collected early enough,
it would be possible to improve teaching effectiveness in
time to have a positive influence on student learning in the
same term.

Two specific studies that are already in progress
should be briefly mentioned. In the first study, the most
essential concepts from an undergraduate research methods
course were incorporated into a measuring instrument similar
to the one used in the present study. Each student in the
class randomly selected, from the student directory, one
student that was not in the course and willing to partici-
pate in the study. These selected students served as the

control group. With the exception of weekends, four students

115
from both the class and the control group filled out a ques-
tionnaire on a daily basis. Specific days for filling out
the questionnaire were randomly chosen. Every student par-
ticipated twice during the term. Taking into account the
specific dates in which concept relevant material was
covered in class, an examination of these data across time
should produce some very meaningful findings in regards to
the learning process.

The second study is an actual comparison between the
evaluation procedures proposed in this report and more tra-
ditional methods of assessing teaching effectiveness. The
same teaching performance characteristics are represented in
both instruments. In addition, different levels of each of
these characteristics were incorporated into what has been
pretested to be effective and ineffective video taped lec-
tures on leadership. The results will be examined in order
to determine the extent to which these instruments are able
to detect these manipulated differences. This study also
examines the susceptibility of the two instruments to the
halo effect. For this purpose, subjects were either told
that the lecturer was extremely cooPerative in projects that
would eventually benefit students or that the lecturer was
very uncooperative and was not particularly interested in
determining means for improving the quality of classroom
instruction. A neutral condition in which nothing was said

regarding the lecturer was also included. The presence of

116

the halo effect would be reflected in dissimilarities of
responses across these three conditions. These two studies
represent attempts to shed additional light on the procedures
presented in this report such that their usefulness can be

further maximized.

Summary of Report

This report begins by pointing out the controversial
history associated with the evaluation of instruction. Some
of the major theoretical and methodological problems with
current practices in evaluating instruction are reviewed.

It is argued that such evaluation should be based on a theory
of the teaching-learning relationship. It is further sug-
gested that a cognitive approach to student learning would
be the most fruitful for the purpose of evaluating instruc-
tion. Such a perspective based on the definition of learn-
ing as the evolution of meaning is then provided. On the
basis of this perspective, a multidimensional scaling tech-
nique for precisely evaluating instruction is presented.

The current study evaluates a training program for
teaching assistants by examining the teaching effectiveness
of a group of trained assistants in contrast to a group of
assistants that were not trained. Student learning and per-
ceptions of actual classroom performance were used as the
criteria for the evaluation. Data was gathered from student

volunteers at four different times during the term. The

117
participation of some subjects in more than one data collec-
tion was in violation of the statistical assumption of inde-
pendent random samples. The analyses were performed, in any
case, for illustrative purposes.

No significant differences were found in the extent
of student learning between the classes aided by trained
teaching assistants as Opposed to those classes aided by un-
trained teaching assistants. However, while the training
program was in progress, the teaching assistants who par-
ticipated in it were perceived to be more closely associated
with effective levels of classroom performance than teaching
assistants who did not participate in the program. Moreover,
the trained teaching assistants continued to be more closely
associated with effective leadership through the remainder
of the term. These findings suggest that while the trained
teaching assistants may have been perceived to be more effec-
tive in their classroom performances, their efforts did not

result in more student learning.

APPENDIX A

Questionnaire

118

MICHIGAN STATE UNIVERSITY

 

Department of Communication
Fall, 1975

Dear Participant:

This term we are engaged in a project involving Com-
munication 100. All twenty sections will be assisting us
in this endeavor. Periodically, we will be asking each of
you for your cooperation in giving us the information neces—
sary to make the project a success. It is our hOpe that
with your help, this information will lead to improvements
in the COM 100 course.

We appreciate your response to our initial request
two weeks ago. We are grateful to those of you who par-
ticipated and ask for your continued support of this project.
We wish that those of you not participating initially will
join our project at this time. Your c00peration will
greatly contribute to the project's success.

Should you have any questions regarding the project,
please feel free to call one of us or stop in at 535 South
Kedzie.

Thank you,

Kim Kanaga 353-3237 or
487-1641

Ilene Benison 355-5557
Marianne Mnich 353-2824
David Palmer 353-0577
Donna Paquette 353-0274
Jean Riker 355-0436

 

 

 

 

 

 

 

 

 

 

119

Instructor =

UTA #1 =

UTA #2 =
1) ID# 2) Name
3) Local Address
4) Date 5) Telephone 6) Age___
7) Sex 8) Major 9) GPA;___ .

10) Class:Fr Soph Jr________ Sr 2

 

ll) COM 100 Section #

 

 

12) COURSE STATUS (Circle one and answer the adjoining
questions)

a) Student
al) In the last two weeks, how many different times
was your classroom activities (i.e. exercises,
group discussions, lectures) lead by:

 

 

UTA #1 times
UTA #2 times
Instructor times

 

a2) How many minutes during the past two weeks were
our classroom activities (i.e. exercises, group
discussions, lectures) lead by:

 

 

UTA #1 minutes
UTA #2 minutes
Instructor minutes

 

a3) In the past two weeks, what percentage of class
time was spent on:

 

 

 

Lectures %
Discussions %
Exercises %
Other %

 

b) Undergraduate Teaching Assistants (UTA)

bl) During the past two weeks, how many different
times were you responsible for leading some
classroom activity (i.e. exercises, group
discussions, lectures)?

times

b2) How many minutes during the past two weeks
were you responsible for leading some class-
room activity (i.e. exercises, group dis-
cussions, lectures)?

 

minutes

 

C)

b3)

b4)

b5)

120

When your class was divided into groups, what
was the average size group (during the past
two weeks) that you were responsible for
leading?
people
Prior to this term, have you ever been a teach-
ing assistant or had any other similar experience?
Yes No
Are you participating in the UTA workshop offered
by the Communication Department?
Yes No
If yes, how many sessions have you attended?

 

 

 

 

 

 

Instructor, Jr. Sr.

cl)

c2)

c3)

c4)

c5)

c6)

c7)

c8)

 

During the past two weeks, HOW many different
times were you responsible for leading some
classroom activity (i.e. exercises, group
discussions, lectures)?

times
How many minutes during the past two weeks were
you responsible for leading some classroom
activity (i.e. exercises, group discussions,
lectures)?

 

minutes
When your class was divided into groups, what
was the average size group (during the past two
weeks) that you were responsible for leading?
peOple
Have you had any teaching experience prior to
this term?
Yes No
In the past two weeks, what percentage of class
time was spent on:
Lectures
Discussions
Exercises
Other

 

 

 

 

 

 

 

anaconda

 

Class size: students

UTA's

instructors
Dates on which small group communication was
taught (to be filled out only after this area
has been completed)

 

 

 

 

Dates on which leadership was taught (to be
filled out only after this area has been completed)

 

121

Just as we can measure the distance between two phy-
sical objects (in terms of inches, yards, miles, etc.), we
can also measure the distance between concepts or ideas.
This questionnaire asks you to make judgments about how dif—
ferent (or in other words "far apart") certain concepts are
from each other. Differences between concepts are measured
in units, such that the more different two concepts are, the
more units apart they are from each other. Two concepts
that are identical in meaning, then, would be zero (0) units
apart.

To help you know how big a unit is, Red and White are
100 units apart; that is, imagine that the difference
(distance) between the colors Red and White is 100 units.
We would like you to use this idea of distance in the com—
parison of the concepts on the next few pages.

You are supposed to tell us how many units apart the
concepts on the next few pages are from each other. Remember,
the more different the two concepts are from each other, the
larger the number of units apart they are. If you think
that any of the pairs of concepts are more different than
red and white, write a number larger than 100. If you think
they are not as different, use a smaller number. Remember,
the more different the concepts are from each other, the
higher the number you should write.

There are no correct or incorrect responses, only
your perceptions of the differences between concepts. Con-
sider each pair carefully and indicate the number of units
that you feel separate the concepts.

 

Your cooperation is most appreciated.

Thank you for your help.

 

REMEMBER:

RED AND WHITE ARE 100 UNITS APART.
FERENT THE CONCEPTS ARE FROM EACH OTHER,
THE NUMBER YOU SHOULD WRITE.

HIGHER
NUMBER

122

YOU WANT.

Instructor

UTA #1
UTA #2

How far apart are

THE MORE DIF-
THE
WRITE ANY

 

 

 

Units

 

Expert
Expert
Expert
Expert
Expert
Expert
Expert
Expert

and
and
and
and
and
and
and
and

Instructor
Studying
Thinking

Me

UTA #2

Physical Science
Leadership
Learning

file-7' ' 3W

 

How far apart are

Units

 

Expert
Expert
Expert
Expert
Expert
Expert
Expert

and
and
and
and
and
and
and

Instructor

Democratic
Humanities
Good
Confident
Social Science
UTA #1
Communication
and Studying

 

How far apart are

Units

 

Instructor
Instructor
Instructor
Instructor
Instructor
Instructor
Instructor
Instructor

and Thinking

and Me

and UTA #2

and Physical Science
and Leadership

and Learning

and Democratic

and Humanities

 

REMEMBER:

 

123

RED AND WHITE ARE 100 UNITS APART.
FERENT THE CONCEPTS ARE FROM EACH OTHER, THE

THE MORE DIF-

 

 

 

 

 

 

 

 

HIGHER THE NUMBER YOU SHOULD WRITE. WRITE ANY
NUMBER YOU WANT.
Instructor =
UTA #1 =
UTA #2 =
How far apart are Units
Instructor and Good
Instructor and Confident ‘
Instructor and Social Science 1
Instructor and UTA #1 :
Instructor and Communication F
Studying and Thinking t;
Studying and Me
Studying and UTA #2
How far apart are Units
Studying and Physical Science
Studying and Leadership
Studying and Learning
Studying and Democratic
Studying and Good
Studying and Confident
Studying and Social Science
How far apart are Units
Studying and UTA #1
Studying and Communication
Thinking and Me
Thinking and UTA #2
Thinking and Physical Science

Thinking
Thinking
Thinking

and Leadership
and Learning
and Democratic

 

REMEMBER:

RED AND WHITE ARE 100 UNITS APART.

124

THE MORE DIF-

FERENT THE CONCEPTS ARE FROM EACH OTHER, THE

HIGHER THE NUMBER YOU SHOULD WRITE.

NUMBER YOU WANT.

Instructor
UTA #1
UTA #2

How far apart are

WRITE ANY

 

 

 

Units

 

Thinking
Thinking
Thinking
Thinking
Thinking
Thinking

and Humanities

and Good

and Confident

and Social Science
and UTA #l

and Communication

Me and UTA #2
Me and Physical Science

.3

:‘Llnn‘inifilfifi.

 

How far apart are

Units

 

Me
Me
Me
Me
Me
Me
Me
Me

and
and

and
and
and
and
and

Leadership
Learning
Democratic
Humanities
Good
Confident
Social Science

UTA

#l

 

How far apart are

Units

 

Me

UTA
UTA
UTA
UTA
UTA
UTA
UTA

and
#2
#2

#2
#2
#2
#2

Communication

and
and
and
and
and
and
and

Physical Science
Leadership
Learning
Democratic
Humanities

Good

Confident

 

REMEMBER:

125

RED AND WHITE ARE 100 UNITS APART. THE MORE DIF-
FERENT THE CONCEPTS ARE FROM EACH OTHER, THE
HIGHER THE NUMBER YOU SHOULD WRITE. WRITE ANY
NUMBER YOU WANT.

Instructor
UTA #1
UTA #2

 

 

 

How far apart are Units

 

UTA #2 and Social Science
UTA #2 and UTA #1
UTA #2 and Communication

Physical
Physical
Physical
Physical
Physical

Science and Leadership
Science and Learning
Science and Democratic
Science and Humanities
Science and Good

 

How far apart are Units

 

Physical
Physical
Physical
Physical

Science and Confident
Science and Social Science
Science and UTA #1

Science and Communication

Leadership and Learning
Leadership and Democratic
Leadership and Humanities
Leadership and Good

 

How far apart are Units

 

Leadership and Confident
Leadership and Social Science

.Leadership and UTA #1

Leadership and Communication
Learning and Democratic
Learning and Humanities
Learning and Good

Learning and Confident

 

126

REMEMBER: RED AND WHITE ARE 100 UNITS APART. THE MORE DIF-
FERENT THE CONCEPTS ARE FROM EACH OTHER, THE
HIGHER THE NUMBER YOU SHOULD WRITE. WRITE ANY
NUMBER YOU WANT.

Instructor
UTA #1
UTA #2

 

 

 

How far apart are Units

 

Learning and Social Science
Learning and UTA #1

Learning and Communication
Democratic and Humanities
Democratic and Good
Democratic and Confident
Democratic and Social Science
Democratic and UTA #l

 

 

How far apart are Units

 

Democratic and Communication
Humanities and Good
Humanities and Confident
Humanities and Social Science
Humanities and UTA #1
Humanities and Communication
Good and Confident

Good and Social Science

 

How far apart are Units

Good and UTA #1

Good and Communication

Confident and Social Science
Confident and UTA #1

Confident and Communication
Social Science and UTA #1

Social Science and Communication
UTA #l and Communication

 

 

 

APPENDIX B

Spatial Coordinates Matrices for the Experimental
and Control Groups at Four Points in Time

127

 

 

89m 89m- $6.? SET 3...; Roz? $3.. Rim- 838g
vflm.¢ mme.m voo.ma «6H.H~- mem.oH mem.mau -m.¢m ugm.a~u as oomomnmmm.oonnommp
mem.on coo.) Hao.vu Hmm.~H- ooa.ov moo. mum.oau «mo.oau oozmflom Honoom
msq.q mmo.a- www.mau om~.o~- ovo.mmu ooa.mau mvm.u 565.6 osmoflmcoo
H4H.HH- ~H¢.o~ avo.a qom.m moa.nau «~5.m- mvv.ou o~o.¢- oooo
mmo.m Hmm.¢ ooa.e~- mma.- mom.ma omm.m www.mau HHH.~N: mononooeom
m4m.¢ emm.m: Hoo.om vam.m~ mam. Hom.mu mma.on use.omu oflomoooemo
omv.m mqa.m omo.v mam.a oao.en mHo.m who.a -¢.m mascumoq
Hem.mn mem.mau moo.~u H¢H.ou NmH.HH- oo¢.¢a- odo.mu mmo.mu oqnmuoommq
Hoo.mu «mo. Neo.m Hmo.m mmn.m «mm.m~ mom.sq Hmv.ov moowflom Hmoflmmnm
mom. mae.¢u mmv.nau moe.ma oo¢.eflu 6mm.n mmm.mm omv.umu me “coumnmma_ooneoooa
Hme.a- nae. mom.> moo.>au HAH.¢H- emm.¢m oma.mmu ~¢H.m~- oz.
evv.mu eflm.u Nee.m oeu.m Hm~.m- mom.oa coo.qu mvo.oH ooflxonna
www.ma mom.) ooo.~- om¢.h- som.¢ omm.m~ emv.ma evp.h~ ocflsooom
Hoe.~ mo>.H mmo.~s nmm.mu omm.HH mmH.Hm- smo.m- «em.a uoooouumoH
H¢V.H mmm.- ome.~ «4m.ma 98.?w meo.~mu Hom.mmu www.mm oumoxm
m e o m w m N H moomoooo
95 359 on 99.6 35.8 “on mooflmomfio 6.88 hour.“ mo 83888 836% .3 6369

128

 

vm¢.oma

vvm.mm

vmm.a

Ha

MHH.Hmn

onw.mmH

omm.wm

omw.m

ov

Hom.mmma

fl 86% Home 5 98886.5 no 8852
va.>mmm~ moans
mHH.mHH mmo.oHH bmm.mm mom.om vow.mo oom.o¢
moo oBSoB¢ 8:368 :55an 68 Home 38. no 688888 936356
Hmm.mm mom.mm mmv.hh mmm.hw noH.Hm Hmm.am
How @353 853mg Hag mo mmmficmoumm 93mg
vwo.> moa.m mmm.m mav.oa omn.ma Hmm.am
“88> 36885 3 you 68582 856me no 8538
m ma ma an ma h
#08 m5 9%me B mcoflmuwfi mo HBEZ
mmm.mmom Ham.hoom mmw.m¢hm mom.mmam vmm.avvh www.momaa
xfiflmz Houompcmmwm mo A309: mgagcmmflm

A 6.9895 ma Canon.

129

p

 

mmm. Hmm.¢I www.mal mmm.m| wmm.o mom. Hmm.H va.bl COHuMUngfigpu
moo.mal oam.m NMB.NH wmm.ml omw.mm moa.hHI vmm.v~l Nbv.vNI Ht u:mumwmm€.mcflnummfi
vow.vl hom.mal Hmo.mH omm.mHI hmH.N 5mm.bv ovb.va mmm.H mocwfiom waoom
Nah.m 5mm.| Nam.m omm.l m¢H.mMI mm~.HMI H¢v.mal vmo.vml ucmcflmcoo
hmm.m1 Hma. hmm.HHI mma.amt mmm. www.mal www.ma mmo.oHI @000
com .m mmH .om Hmm . 3H mam . mm: Now .HI mam .5” Hmm .m mmv .I 8.3%:
vwm.vl oom.ma www.mal ovm.wm mmm.HH vmm.hm hmv.ml MFH.wMI OHHMHOQEHH
moq.ml omv.NI mmm.mt mmo.a mmw.n mmm.MHI mmm.w mom.oa mafichmq
mmm.m th.m mmb.HHI vvm.ml va.mal wvm.m VHN.MHI mmh.hml mflnmumcmma
Hoo.HI mNm.o wam.val mvv.m hHM.NH oom.ml onw.m~ mmv.mh mocwwom Hwowmmflm
mmm.m~ www.ml hum.¢ moH.N www.mm ovm.oau NHH.¢NI HvN.HI N¢ u:mumamm¢.m:HnUmmB
mmo.v mmh.NI moo. mmm.m mow.mal mna.mn mho.mm mvc.~ml m2
NmN.h omm.l 5mm.m vs¢.ma mmm.~l omm. NHN.5 hmm.bH mafixaflna
mmm.mt www.ml th.bN mov.mm mmo.hl mmm.mau mhm.m wa.ma mcflmvsum
N>O.ml HHH.mHI www.ml Hmm. mmH.HI hva. mmh.mal oam.wl HOQQSHumcH
eom.l mam.~ moo.ml ¢Ho.a Nwm.mml mmm.>a wov.vml mmH.o¢ pummxm

m h m m v m N H mummogpu

 

05 win. an 90.5 gaging How magma—50 ”Ema umfim mo mgéoa 333m .5. mafia.

130

 

 

mom.vma

om>.hm

mvm.~

www.mama

hmo.HmH

mmm.vm

th.m

mmm.mmma

fl mommm Hamm 5 98me mo Hmnebz
oom.oammm moans
Hmm.m~H www.maa ~mm.mOH mam.mm mum.Hh www.mm
you @3588 magma Agmfi can. 135 138. «0 mmmfiawoumm 93mg
mam.om mmm.mm nm~.oh mmm.¢o mmm.~m hum.m~
How mumps—58¢ 8:539 Hmmm no mwmmucmouwm 9.3020
mm~.m mmm.m mom.HH mop.~a moa.m~ hum.m~
“88> fiéfififi 3 you @3582 85.0.8 mo mmficmoumm
a HH NH ma m m
~08 mfi «EH8 B mSflfifiH «0 p.352
~mv.nv¢~ mvm.a~m¢ mvo.momm qmm.m~mm mmm.~amoa m~m.q-ma
xauumz Hopompcmmfim mo Amuoomv mmsflmpcmmflm

A.U.H:UOV~WH manna

131

 

35:. 2%.? 93.? 2&5- ~32 ~25- $86 $3. 838%
~25 $9? 936 Eda Sad? $5.3 33$. 83:. S £5632 mfifimfl.
moa.~a mum.m- mmh.m~ mmv.m~- H¢H.¢Hu o-.~- oom.~ nvo.m mocwaom HMfloom
cam.HH- m-.~- 0mm.m~- HH~.5- «vv.m- osm.H~ www.mH- mom.H- uamcflmcao
.63: 834 $5.3- 8mg. Sm. Ema «8.2.. $93.. 88
32:- 3.18 958 3.3m $97 mafimu www.mu mmm.m mmflflfifim
H86 8:- 8o. v3.2 39% «3.3- 5.? 33:- oflmfiofia
8.: 3d? 89? ma; 98.? 03.? £64 89:- 9553
oz; ~23 o3.~ www.mT $3.3 mm; :86? 8.5.? mg
33” «8.? $9? 293 3%.“... 33:- 33% 30.9.. 8:38 amoflmfim
m2; mam. 3;. 2.5.? H86 :55. mm~.R Emma 2 ucfimflmfl manna
mar 39? ~25- «86.. $3... Smam- 2&5? ~32- w:
a: .2- 0mm .7 o; .NH 2m . m NS 6.. mom .mu 3m .7 3; .3. mg
Boa- flm .v 53H- omméu 3.93- 35.3- 333 mum .mu mfigm
Sud- 293.. H86 83w 3; 89: mama 80.3 ~88”qu
pvm.v Ham.m moo.m mmo.oa- mma.m www.mm omH.vH www.mm- uummxm

m h o m w m m H mummofio

 

CE. 959 pm @505 HOHEOU How gage ”Ema pmHE mo mmpmfivuoou 33am .3 @309

132

 

 

mom.mma

mnm.mm

mmv.m

AH

oNH.Hmma

mvm.mHH

mvm.vm

hem.m

vH

vmm.vmma

OH woman Hmmm :H mcofimcmeaa mo.umnE:z
mvm.amvam mamas
mmm.qHH mom.moa mmm.ma mmm.~m www.mo Hus.mv
you @8582 8:3m8 A535 can H85 38. mo mmmficmoumm 93mg
mam.om vom.mm mmm.mn h~5.mm mma.¢m mmn.¢m
How @353 RENEE mem mo mmmmucmgmm 93mg
mmo.h mma.m moa.oa mmm.aa mmq.ma mmn.qm
“88> gwfiflcfi B How @3582 BEEQ mo $3888
am «H mm ma w m
poem may m>fiumo on mcoflumumpH mo_umnsnz
mmm.mmhm oaw.m-m mam.ooov omm.vpmq mma.~a~h omo.H~nma

xfluumz “809563 mo 7.309: ggcgflm

A.U.ucoovMWn manna

133

 

 

mama- $34... mama? 03.? HQ: :3. 33.- $9? 838550
~¢m.oau mam.HH mmm.oau aam.o~u mum.¢ boo.o mov.ma www.mau H¢ u:mumwmm«.mcfinumma
«wo.nu mmn.vu mmv.v osm.man mo¢.mu Hmm.¢m mvm.~mu Hom.m~ mocmflom Hmauom
oom.~ ~oa.>u mm~.H voo.~ mew. mm>.o~u mHH.om mmm.mu unmcamcao
Hmm.mu omm.o~ mm~.H- mmm.na vwm.m mmv.HHn vmo.m pv~.oau comm
mam.v mom.m mom.mau mom.m~ www.mau Hmv.ma mmm.mmu www.mn mmfiuficmssm
mom.vu mnm.a mum.>~ mmm.~ nam.vu mew. nmm.nau ¢h~.vmn ofiumuooemo
~mm.o omm.mu Hmo.HHu mam.mu Hma.n Hmm.~H: wwm.m Nm5.HH mcflcnmmq
moo.mu Hom.mH- hem. vmm.ma www.mu HHN.~- >HH.~H ~m~.mH- mqnmumcmmq
oan.mu mmm.a «hm.m mv>.> hm¢.- mam.mau mm~.~au mmn.om mocmfium Hmoflmmnm
oam.mH Ham.m vmm.¢ mqm.n mwm.am omm.~m www.ma mom.HHn m¢ ucmumflmm¢xmcflgomms
mmv.m mmm.~- mum.vu mmm.m: mna.p nah.mmu mva.n~u aHH.Hmn ma
~m¢.m mah.a mmm.m omm.mu mam.mn mmfl. mah.mu mmu.v mcflxcflsa
hmH.m nmv.m Hov.~u mmm.m- www.mmn mmm.m~- 5mm.nu ~mo.m~ mcfluusum
oam.v woo. omm. mme.mu >~>.HHI hm¢.m wav.oa www.mn nopupuuqu
onv. mmm.au mmm.~ mao.m mv~.¢au ovm.ma ~ma.wv vmm.>m pummxm
m h o m ¢ m N H mammocoo
03. 9E. an 98m gaming you 986853 Ema umuE mo m3§o8 33.2mm .3 manna

134

 

mmm.mNH

oom.bm

ovm.N

mmm.mvh

Nmm.NNH

omm.mm

mmm.m

ma

Hmm.meH

3 «commammmfimcofimqmfiamoumfiaz
mamémmmm momma.
omm.hHH bmm.moa oam.ooa hHH.om oom.on NmH.hm
How kugog 85..me Aamfimg 0cm Hum»: Eon. mo mmmmucmouwm 93mg
mom .3 mmm.mm www.mh m: .2 m3. .vm mmmém
Mom @3588 85..me Hmmm no mmmfifioumm 93mg
mvmfi mmmK mmoé vafia mom.m~ mmm.mm
H800> Eggs" 3 Mom wwflfioog 853mg mo $3583
ma NH ma ma m 5
908 9? 9..."me 3 maoflmnmuH m0 .3052
moméa: www.mmam «$6th www.mmmv maméhmh mmmémvm
xflumz ugggfi mo Afioflv mQEcmmE

A .U .ucoov ma. manna

135

hm5.m

 

 

82:. $37 $37 03.- 892 33%. 512.. 838358
m2. 8:- 89% amp. 93.3 393 was? om~.mm : ucmuflmfl mg
mama swam 83: 3m... 2.3m 25.2.. $93 33%. 8:38 $88
83 30.3. «$4.. $3: H32- §.~ «3.? v8.2 panacea
33:- 33 Sad ~23 892.. $98.. $98.. 5:. 88
83:- 23 mama? Elm- H35- 832- 8:. 35.2- mmflflgm
23: Rad? Ema- 23.2- 33: 2:- $23.- mm~.-- oflmfiofin
BEN :97 out? $97 284.? $3.? and? 89:- mfifimfl
83.... H86 maid- $18 $17 mmm.m~ «.53. 852- mflflflmfl
m3; mad? :9? v24 :3: £93 393 236 883m HBHmEm
89w. Ex? mm~.m- Ev... 89m- $3? 2:. 8de a ufiumflmfl mfifimme
:3. 83:- Ema- 298 Smé «3.3... H83 Rim? 9..
So. ~23 Rm? 3mg“. and- Ems Nani 89mm- aqua
29m $2. $3.3 08.2- 5.3- «8.3.. «.34: «gm mfifium
:92 $3: «8.2.. 39:- moms mafimm $92.. Sham “8259:
23:. ~35- $3: 89:- 25.8.. $33 $3. «2.3. fimmxm
m b m m v m N H 3%
8H5 23. an 98w 3580 you 80%:th ”Ema 55.. mo mflmfiflg 33% .8 mafia

136

 

vNH.mma

Hoo.wm

omm.m

Hmm.mmmH

oom.mma

Hmh.vm

Hmm.v

Ha

NmH.mmmm

Ha mommw Hmmm ca mQOHmcmafin mo HmnEpz
omm.mmomm menus
mm¢.>~a ~m¢.mHH mmo.oaa hmm.bm www.mm www.mm
you 8u508< 8535 Egg Em H85 32. mo mmmfinmoumm 93mg
ovv.om mmh.qm mmo.mh ov~.mm omo.mm ma¢.mm
“on cmucsoouanocmumfio Hmmm no mmmmucwoumm m>fiumasasu
Ho~.m «hm.m mmm.m omm.oa H¢~.ma ma¢.mm
(86g aficfiflcfi B “on c3582 .85me mo $388.8
OH «H m on OH m
88 m5 mama B maoflmuwfi mo uBEz
mmm.mmam mmm.mmhm Num.hmmv vmm.¢amm mmp.qm~oa mmm.¢mo-

xflnumz uoaom>cmmflm mo Amuonmv mmsam>cmmflm

A.v.quanxw mange

137

 

 

8:. mm: 39m «$5 mus- 833. 8a.? mmméu 838%
End v8.3 89m 2%.? $2.? 3:. 8m.» MEAT : ufiumflmmm mfifimma
mmm. $44? Sea :52- Rafi? ~85 v8.5- 2Y8 853m .9308
89.... 8a.? «8.3. mmm.m~ mafia 3.6 8.12 «#4.? aficflcoo
8m.» mmmfl mam. 392- 8.2: Sham $4; $567 88
Sm... m2; mama $25 527 83% £18- mafia 8335:
8%? mg... :55. ~36? 35.3. 33:: 89mm: 513.. oflfioemd
v2.3 E: 897 S3”? 59$ 3%.? mmméa EA: mg
Qua- SQNT modem Rig- 3%. £34? «2.2- 83.: mflmnmomfl
$04.. omm.m omm.m m2}... 893 Sum? momK 8mg 888m H833.”
MEWS: «8.3 2:7 35.: 83:- «3.? m8... mmpé Q. ”Emuflmmm gunman.
mama- «max..- 89?. am. $98 mom.~ $33.. «8.7 m2
«8.? 893 «3:2: 3mg- 8:- $6.3- 33. 3:... mg
$5.? 5:? 03.2- £52 mmmd 8m. 85.: 893 mfigum
39$ 83.? 3...? £33 33:. SQNT 392 «$.31 Banana
~H>.mau mom.hn m-.m- mmm.maa oom.oau Hoo.oH Hem.vm oam.aan uummxm
m b m m w m m H £880
mmufi 8E. pm @596 355896 How 90355 ”Ema umHE mo mflmfifiooo fiflfim Am 3nt

138

 

NoH.mHH

mHN.mm

vvm.v

mmo.mmma

mom.maa

Hum.am

Nmm.m

NH

mmh.ammm

3 8mmm 3mm 5 mcoflmcmfia mo hfi@252
mmm.vmmvm mamas
ov~.hoa vmm.hm www.mm www.mb wam.mm www.mm
you Umucsooum mocmumaa AxumcflmmEH_wcm Hmmmv Hmqu mo mmmmucwouwm m>flumanaso
mmm.wm oam.mh Hmm.mm Nmm.mm wmm.mv mmm.w~
How Umncbooam mocmumfia Hmmm mo mmmmucwoumm m>flumauesu
muo.m mmm.m mmw.oa mmm.~a o~¢.mH mmm.mm
889,. 36385 3 no“... @3582 85..me mo $3688
ma «a ma ma m h
poem may m>flumo on macaumnmuH mo_umnasz
omn.oomm ham.mmmm mma.mnmv omm.hmmm mmm.vavm bam.hmvaa

xfluumz_uouum>cwmflm mo Amuoomv mmsam>cmmwm

A.mu.#fi8v HN mHQMB

139

. 'EIE.

 

 

«2 . o- ««« .««- «8 .. «8 .3 H«« . omv .«.. «m« .2. «E .3 838328
«o«.« «36 «8.? «Ed- «3.3. «8.8.. and «RS : “EBflmmm «:20me
«««.7 «24. «««.« «mo.«« «3.2 «3.2 «$6 5.5- 888m «£00m
«Hm.«T «$4 «3.2 «3.3.. «3.2.. «o«.««. «24 «8.: unmoflSO
8:: «8.7 «3.: «3.2- «BS :54 «««.«n «o«.m 88
«2;- «8.« «8.« «8.7 m««.«« «8.2- «8.5.. «««.2... mmfldfifim
E: «3.3 «8.? 3.92. «84 «2.: «$2.. «a; 03%
«:4: «SAT «8.3.. «9.3 «««.«.. «fl. «3.? «3.3.. «35.8
«4%? «Si «8.2- «8.? «$5.. «8... «8.8.. «um.«« «Eflmnmfl
«2.4.. «$.«. «3.? «3.3- «8.2- «2.... «««.«m «3.3.. 888« 3395
Ema- «««.«- «86 m8. Sm; «3.2 ««o.«« «««.«m «* ucfiflmg «5689
«34.. «««.o«- «mm.«.. «3.3- «8.2- «mm.« 2:67 «3.2- m:
«8.? «mo.m «««.«- «««.T «$57 o««.« «RA «8.? «qflfifi.
83... «$6.. «««.« 5...: «2.7 SW? $9.0. «84$. «55%
«8.2 «84 S«.««. «S.« «.«mf «2.3- o««.« «3.3 «Banyan
m«o.« «oa.m« «H«.«H m«o.«« «mm.m«u ««H.««. «o«.«. «om.u pummxu
« « m m u m « H £889
H58 «.59 um «no.8 «Sacco you meoficmfio 23m ”6.5 «8 8umfi880 33QO .«« magma

140

 

 

mmm.mma

th.hm

omb.m

voa.mwma

vom.vma

5mm.mm

mma.o

bw

www.0omm

«a momma «was :« «mafimcmaan mo.nmnEsz
o««.««m«m momya
H«o.m«« mm«.«oa «mm.m« m««.«« o«o.«m o««.¢«
H8 @3982 magma Agog can «85 «38. mo mmmficmemm 93mg
«««.«« mom.o« vvo.o« oo«.«m «m«.«v m««.m«
How @3583 823mg Hmmm no mmmmucmonwm $3de
moo.« «w«.o« «wa.«« ««H.m« «m«.o« «««.m«
Houom> «msw«>«ncH «n «on nmucsooo«_mo:mum«a «0 mmmucmoumm
«a «v «a « «H «
uoom man m>fluma on mgoaumumuH mo_umnspz
«««.o««« m««.««m« m««.«««m «««.««mo «oo.o«o« ov«.mm«o«

xflumz Hofigcwmfi «0 A385 «Saga

Afihfiv NN manna.

141

 

 

 

ooo.«- «mH.« o«m.«H «OH.«: mH«.«u owo.m «mm.H m«m.mu coHumngannpu
owo.«H «o«.« ««m.«H H««.«- «m«.H «HH.H- «««.«- o«m.mu Ht unnumHmm«.«cngmma
«««.m- «««.m- ««m.« o««.«H «««.«« m««.o«- o««.m- mo«.«H mocmHom HMHuom
«m«.« v««.«- ««o. «mo.« «mo.«u «mo.H- «em.«H «mo.m«- ucmchcao
«mH.« «OH.« H««.«. «mm.«H «««.m «o«.« HHo.mu m««.m«n coco
«Ho.« «m«.«H m««.«H- «««.m- «mH.HH- «mm.«H- www.c«u ««o.«H mmHuHamesm
«m«.« ««o.OH- ««m.«Hu «H«.«Hu m««.o« «««.«H mo«.H«u «««.H- oHumuooemo
«mm.«- o««.«- ««o.«- moo.«u H«o.«Hu «H«.HH ««H.« ooq.« «chummH
«««.H m«o.«u m««.H- m««.m- H««.«. o««.«. om«.m- «H«.«H- «HgmnmnmmH
o«H.- m««.m «mo.u m««.m m««.«H «m«.«m «m«.mH m«v.«« mocwHom Hmon«n«
««o. «m«.« «H«.«H «mo. «mo.« «o«.v w««.«. «««.«u «* ucmumHmmmgmcHnumme
mmm.«- «o«.«- «OH.«- H««.«H ««O.oHu mm«.« om«.m«- «m«.« «2
«mm.. «v«.H- H«m.- «««.« «««.H- «m«.- «««.« m«o.« «:quHsa
«w«. H««.«. o««.« www.mu «o«.HH- www.mu H««.« H««.«« «:Hmusum
«««.«H- «o«.m «mm. «««.«H- «««.H m««.«. «««.m «««.««u «opusuumcH
«H«.« «««.H Hmo.«H- «o«.- «H«.« «mo.«Hn H««.«« «««.H- pummxm
« « m m « m « H mummocao
«:8 «EB. pm. 986 Hflgmenm «om «:ng ”Ema NEE no «33838 Hmflmmm .m« «.38.

142

 

 

HNm.mHH

mom.mm

mwv.m

mom

mvm.omm

hom.vaa

mvh.am

mva.m

mvm.mNoH

«H 885 Hmmm :H «83:93 «0 «8.52
wom.vHowH moans
Hom.hoH mum.mm mho.mm mmm.m> mmm.mm mmh.Hm
you @3582 magma Emfimg Ham H85 H38 «0 mmmficmoumm 93mg
«om.mm «mm.«n omv.Hn mHm.om Hmv.v¢ www.mm
Hon Umpcsooua mocmumfio.ammm mo mmmmucmonmm m>flumagesu
OH«.o Ho¢.m mum.0H mmv.ma mnm.ma www.mm
Houom> Hmnwfl>fl02H an Hon nopcsooum magnumfla mo mmmucmoumm
«H vH mm m m h
poem may m>fiumo on mcofiumnmpH mo.uwnE=z
oom.mmmH va.mhmH www.0HHm mwm.mmmm mum.wmhm abv.mmom

fiflmz “80983..“ «0 E85 mBHmEmmHm

A6382 mm 3nt

 

APPENDIX C

Descriptive Statistics of Students from Experimental
and Control Group Sections

143

Table 24. Descriptive Statistics of Students from Experi-
mental and Control Group Sections.

 

 

Control Experimental
Class Activity
Mean Percentage of Time
on Lectures 29.75 33.04
Mean Percentage of Time
in Discussions 29.75 29.80
Mean Percentage of Time
in Exercises 39.25 31.87
Mean Age 19.02 19.19
Mean Grade Point Average 3.02 3.05
Number of Students in Sections
Mean 62.5 62.5
Minimum 38 34
Maximum 71 76
Percentage of Female Students
(Mean) 62.5 65.4
Percentage of Male Students (Mean) 37.5 34.6
Percentage of Communication Majors 6.9 11.4
Percentage of No Preference 26 19
ercentage of Freshman 57.9 49.2

Percentage of Upper Division
(Jr and Sr) 28.3 27.7

 

REFERENCES

144

REFERENCES

Association for Educational Communications & Technology.
Educational technology: Definition & glossary of
terms. Washington, D.C., 1977.

Astin, A.W., & Lee, C.B.T. Current practices in the evalua-
tion and training of college teachers. Educational

 

Ausubel, D.P. Educational psychology: A cognitive view.
New York: Holt, Rinehart & Winston, 1968.

 

Bourbour, A. & Goldberg, A. Interpersonal communication:
Teaching strategies and resources. Falls Church,
Va., Speech Communication Association, 1974, 85.

 

Barker, L.L. Listening behavior. Englewood Cliffs, N.J.:
Prentice Hall, 1971.

Blackburn, R.T., & Clark, M. An assessment of faculty per-
formance: Some correlates between administrator,
colleague, student and self ratings. Sociology of
Education, 1975, 242-256.

 

 

Bolles, R.C. Learning theory. New York: Holt, Rinehart
and Winston, 1975.

Bruner, J.S. Toward a theory of instruction. Cambridge:
Harvard University Press, 1966.

Bryant, P.T. By their fruits ye shall know them. Journal
of Education, 1957r.§§r 326-330.

Campbell, N.R. An account of theprinciples of measurement
and calcuiation. London: Longmans Green, 1928.

Cantwell, Z.M., & Doyle, H.A. Instructionaltechnology.
Metuchen, N.J.: Scarecrow Press, 1974.

 

Centra, J. Effectiveness of student feedback in modifying
college instruction. Journal of Educational Psye
chology, 1973, 65, 395-401.

 

145

Cody, M. An application of the multiple attribute measure-
ment model: Measurement and manipulation of source
credibility. Unpublished M.A. thesis. Michigan
State University, East Lansing, Michigan, 1976.

 

 

Cohen, A.M., & Brawer, F.B. Measuring faculty performance.
Washington D.C.: ERIC Clearinghouse for Junior College
Information/American Association of Junior Colleges,
1969.

Costin, F., Greenough, W.T., & Menges, R.J. Student ratings
of college teaching: Reliability, validity, and use-
fulness. Review of Educational Research, 1972, 41,
511-535.

Cushman, D.P. The message-attitude-behavior-relationship.
Bureau of Business Research, Michigan State University
Press, forthcoming.

Deshpande, A.S., Webb, S.C., & Marks, E. Student perceptions
of engineering instructor behaviors and their relation-
ships to the evaluation of instructors and courses.
American Educational Research Journal, 1970, 1, 289-305.

Doyle, K.O., Jr. Student evaluation of instruction.
Lexington: D.C. Heath, 1975.

Doyle, K.O., Jr., & Whitely, S.E. Student ratings as criteria
for effective teaching. American Educational Research
Journal, 1974, 11(3), 259-274.

Eble, K.E. The recognition and evaluation of teaching. Salt
Lake City, Utah. Project to improve college teaching,
1970.

Einstein, A. Relativity: The special and general theory.
New York: Crown, 1961.

Fenker, R.M. Teacher evaluation at TCU: An analysis of the
perceived roles of faculty by students, administrators,
and faculty. In the Evaluationgf College Teaching.
R.I. Miller. New York: Jossey-Bass, 1974.

Frey, P.W. Student ratings of teaching: Validity of several
rating factors. Science, 1973, 182, 83-85.

Friedrich, G.W., Galvin, R.M., & Book, C. Growing together:
Classroom communication. Columbus, 0.: Charles E.
Merrill, 1976.

 

146

Genova, W.J., Madoff, M.K., Chin, R., & Thomas, G.B. Mutual
benefit evaluation of faculty and administrators in
higher education. Cambridge, Mass.: Ballinger
Publishing, 1976.

 

 

Gessner, P. Evaluation of instruction. Science, 1973, 180,
566-570.

Gillham, J., & Woelfel, J.P. The galileo system: Prelim-
inary evidence for precision, stability, and equiva—
lence to traditional measures. Human Communication
Research, Fall, 1976.

 

 

Gray, C.F. The teaching model and evaluation of teaching
effectiveness. Improving College and University
Teaching, 1970, 18, 275-278.

 

 

Grush, J.E., & Costin, F. The student as consumer of the
teaching process. American Educational Research
Journal, 1975, 12, 55-66.

 

Gustad, J.W. Politics and practices in faculty evaluation.
Educational Record, 1961, 42, 194-211.

 

Hammond, P.E., Meyer, J., & Miller, D. Teaching versus re-
search; sources of misperceptions. Journal of Higher
Education, 1969, 49, 682-690.

 

 

Hebb, D.C. A textbook of_psychology. Philadelphia: W.B.
Saunders,E1958.

 

Hebb, D.C. The American revolution. American Psychologist,
1960.

 

Helm, C.E., Messick, S., & Tucker, L. Psychophysical law
and scaling methods. Princeton: Educational Testing
Service, 1959.

 

 

Hildebrand, M. How to recommend promotion for a mediocre
teacher without actually lying. Journal of Higher
Education, 1972, 43, 44-62.

 

 

Hilgard, E.R., & Bower, G.H. Theories of learning. New
Jersey: Prentice-Hall, 1975.

 

Howe, H. Less teaching, more conversation. In C.B.T. Lee
(ed.), ImprovingCollege Teaching. Washington, D.C.:
American Council on EducatiOn, 1967.

 

Jacobi, C.G.J. Uber ein leichles Verfahren die in der Theorie
der Sacularstorvngen vorkomnenden Gleichuagen numerisch
aufzulosen. Jouranl fur die Reine und Angewandyte
Mathematik,.3g:51-94.

 

 

147

Johnson, D.W. Reaching out: Interpersonal effectiveness
and self-actualization. Englewood Cliffs, N.J.:
Prentice Hall, 1972.

 

Kauffman, J., Hallahan, D., Payne, J., & Ball, D. Teaching/
learning: Quantitative and functional analysis of
education performance. Journal of Special Education,
1973, 1, 261-268.

Keast, W.R., & Macy, J.W., Jr. Faculty tenure. San Francisco:
Jossey-Bass, 1973.

 

Kent, L. Student evaluation of teaching. Educational Record,
1966, 41, 376-390.

Koffka, K. Principles ofggstalt psychology. New York:
Harcourt-Brace, 1935.

Kohler, W. Gestalt psychology. New York: Liveright, 1947.

 

Lewis, L.S. Scaling_the ivory tower. Baltimore, Maryland:
The John Hopkins University Press, 1975.

 

Liska, A.E. The consistency controversy. New York: John
Wiley, 1975.

Luthans, F. The faculty promotion process and analysis of
the management of large state universities. Iowa City,
Iowa: Bureau of Business and Economic Research,
College of Business Administration, the Unviersity of
Iowa, 1967.

McKeachie, W.J., Lin, Y., & Mann, W. Student ratings of
teacher effectiveness: Validity studies. American
Educational Research Journal, 1971, 8, 435-445.

Miller, R.I. Evaluatin faculty performance. San Francisco:
Jossey-Bass, 197 . '

Morine, H., & Morine G. Discovery: A challenge to teachers.
Englewood Cliffs, N.J.: Prentice Hall, 1973.

Mortensen, C.D. Communication: The study of human inter-
action. New York: McGraw-Hill, 1972.

Nobel, C. An analysis of meaning. Psychological Review,
1952, 52, 421-30.

Osgood, C.E. A behavioristic analysis of perception and
language as cognitive phenomena. In Contemporary
Approaches to Cognition, Cambridge: Harvard University
Press, 1957.

 

148

Pittenger, C.E., & Gooding, C.T. Learning theories in edu-
cational practice. New York: John Wiley and Sons,
1971.

 

 

Pfeiffer, J.W., & Jones, J.E. A handbook of structured
Experiences for human relations training (3 vols.).
Iowa City, Iowa: University Associates Press, 1971.

 

Rippey, R. Student evaluations of professors: Are they of
value? Journal of Medical Education, 1975, 50,
951-954.

 

Serota, K. “Metric multidimensional scaling and communica-
tion: Theory and impiementation. Unpublished M.A.
thesis. Michigan State University, East Lansing,
Michigan, 1974.

 

Sharon, A.T. Eliminating bias from student ratings of
college instructors. Journal of Applied Psychology,
1970, 54, 278—281.

 

Snelbecker, G.B. Learning theory, instructional theory,
and psychoeducational design. New York: McGraw-Hill,
1974.

Stevens, S. Mathematics, measurement, and psychophysics.
In S. Stevens (ed.), Handbook of Experimental Psy-
chology (lst ed.). New York: Wiiey, 1951.

 

Suppes, P., & Zinnes, J.L. Basic measurement theory. In R.
D. Luce, R.R. Bush, & E. Golanter (Eds.), Handbook
of Mathematical Psychology. New York: John Wiley
and Sons, 1963, 1-76.

 

Tolman, E.C. Purposive behavior in animals and men. New
York: Appleton-Century-Crofts, 1932.

 

Torgerson, W. Theory and methods of scaling. New York:
Wiley, 1958.

Tyler, R.W. The evaluation of teaching. In R.M. Cooper
(Ed.), The Two Sides of_the Log. Minneapolis:
University of Minnesota Press, 1958.

 

Wakshlag, J., & Edison, N. A model for thevalidation of
the source credibility construct. Paper presented
at the Speech CommuniCation Association Annual
Convention, December 1975.

 

Wertheimer, M. Productive thinking. New York: Harper, 1959.

 

149

Williams, R.E. Communication: Definitions, concepts and
models. In J. Shubert (Ed.), Human Communication:
Concepts, Principles and Skills. East Lansing,
Michigan: Michigan State University, 1974.

 

Woelfel, J. Sociology and science. Unpublished manuscript.
Michigan State University, East Lansing, Michigan,
1972.

 

Woelfel, J. Metric measurements of cultural processes.
Paper presented to the Speech Communication Associa-
tion, Chicago, 1974.

 

Woelfel, J. Foundations of cognitive theory. In D.P.
Cushman (Ed.), The Message-Attitudes-Behavior Rela-
tionship, Bureau of Business Research, Michigan
State University Press, forthcoming.

 

Woelfel, J., & Danes, J. New techniques for the multidi-
mensional analysis of communication and conception.
In P. Monge and J. Cappella (Eds.), Multivariate
Analysis in Communication Research, forthcoming.

 

 

Woelfel, J. Cody, M., Gillham, J., & Holmes, R. Basic
premises of multidimensional attitude change theory-
an experimental analysis. Unpublished manuscript.
Michigan State University, East Lansing, Michigan,
1977.

 

 

 

Woelfel, J., & Saltiel, J. Cognitive processes as motions
in a multidimensional space: A general linear model.
Unpublished manuscript. Department of Communication,
Michigan State University, East Lansing, Michigan,
1974.

 

 

Wotruba, T.R., & Wright, P. How to develop a teacher-rating
instrument. Journal of Higher Education, 1975, 46,
653-663. -

 

Young, G., & Householder, A. Discussion of a set of points
in terms of their mutual distances. Psychometrika,
1938, 3, 19-22.

 

"11111111111111?