PLACE IN RETURN BOX
to remove this checkout from your record.
TO AVOID FINES return on or before date due.

 

DATE DUE ME DUE DATE DUE

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

1M W14

 

SITUATIONAL JUDGMENT AND TACIT KNOWLEDGE:
AN EXPLORATION 0F CONSTRUCT SIMILARITY
By

Morell E. Mullins, Jr.

A THESIS

Submitted to
Michigan State University
in partial fulﬁllment of the requirements
for the degree of

MASTER OF ARTS

Department of Psychology

1997

ABSTRACT

SITUATIONAL JUDGMENT AND TACIT KNOWLEDGE:
AN EXPLORATION OF CON STRUCT SIMILARITY

By

Morell E. Mullins, Jr.

While situational judgment tests have gained popularity in recent years in both
research and applied settings, it remains an open question as to what they are actually
measuring. To answer this question, a study was conducted to examine the similarity
of constructs underlying a situational judgment test and a measure of tacit knowledge.
A situational judgment test, a measure of tacit knowledge, and a set of variables that
were believed to inhabit the nomological networks of both measures were
administered to a sample of 358 undergraduates at a large midwestern university.
Results of correlational and structural equations analyses indicated that situational
judgment and tacit knowledge are likely not isomorphic with respect to their

underlying constructs.

For Cindy -- without you, kid, I wouldn’t be here today

iii

ACKNOWLEDGMENTS

The idea that I might have ﬁnally done something worthy of writing
acknowledgments for is a little intimidating, but it would be utterly inappropriate for
me to submit this ﬁnal copy of my thesis without doing so. There is no way I could
have made it through the thesis process without the support of any number of people
who have offered advice, guidance, and other intangibles along the way.

I would ﬁrst like to thank Neal Schmitt, my Thesis chair, for his patience with
my sometimes-unpredictable output of product. Whether I gave him a revision a day
or a week (or more!) after his comments, he was always able to do the mental shift
necessary to deal with my revisions, and always had insightful commentary on what I
had done. I would also like to thank Steve Kozlowski and Rick DeShon, my other
committee members. Steve’s rigorous quest for good science has been invaluable to
me as a model to follow now and throughout my career; Rick’s skill in picking out
the questions that most need answering (especially in defenses!) is one that I hope to
develop, and pass on to my own students.

My thanks also go out to my fellow graduate students, especially Ann, Ken,
Lori, and KD, each of whom has had to deal with my various tirades at one time or
another. Most most of all, though, I must thank my family -- Dad, Mom, Jeff, and
Carol -- and especially my wife Cindy, without whom none of this would have ever

been possible.

iv

TABLE OF CONTENTS

LIST OF TABLES ....................................... vi

LIST OF FIGURES ....................................... vii
CHAPTER 1

LITERATURE REVIEW ................................... 1

Situational Judgment Tests .............................. 5

Development .................................. 6

Constructs Underlying ............................ 9

Empirical Research .............................. 13

Practical Intelligence .................................. 18

Conceptual Underpinnings .......................... 18

Implicit Learning ........................... 30

Empirical Research .............................. 32

Intelligence and g ........................... 40

Hypotheses ........................................ 45
CHAPTER 2

METHOD ............................................. 56

Participants ....................................... 56

Procedure ........................................ 58

Measures ......................................... 59

Cognitive Ability ............................... 59

Personality ................................... 6O

Demographic Information ................ A .......... 66

Tacit Knowledge ................................ 66
Situational Judgment ............................. 70
Experience ................................... 72
Self—Regulation ................................. 72
Learning Style ................................. 73
CHAPTER 3
RESULTS ............................................. 74
CHAPTER 4
DISCUSSION .......................................... 89
Contributions ...................................... 90
Limitations ....................................... 101
Directions for Future Research .......................... 107
APPENDICES ......................................... 1 10
APPENDIX A: Consent Form ............................ 110
APPENDIX B: Debrief Form ............................ 111
APPENDIX C: Demographic Information ..................... 112
APPENDIX D: Tolerance of Ambiguity Measure ................ 114
APPENDIX E: Communication Competence Self-Report ............ 115
APPENDIX F: Machiavellianism Measure .................... 117
APPENDIX G: Tacit Knowledge Measure .................... 118
APPENDIX H: Situational Judgment Test ..................... 131
APPENDIX I: Experience Questionnaire ..................... 147

vi

APPENDIX J: Self-Regulation Measure ...................... 154

APPENDIX K: Learning Style Measure ...................... 155
APPENDIX L: LISREL 8 Control Statements .................. 157
REFERENCES ........................................ 159

vii

Table 1:

Table 2:

Table 3:

Table 4:

Table 5:

' Table 6:

Table 7:

LIST OF TABLES

Characteristics that Differentiate Academic and Practical
Kinds of Problems ............................... 24

Relationships Between Relevant Constructs and Conceptual and
Empirical Notions of the Situational Judgment

Test and Practical Intelligence ........................ 39
Descriptive Statistics for the Scales .................... 61
Raw Correlations Among Study Variables .............. 63-65
Reliability Coefﬁcients for Tacit Knowledge Scales ........... 69

Tests for Differences Among Correlations of Indicators with the
Situational Judgment Test and Positively-Keyed
Tacit Knowledge Items ............................ 75

Parameter Estimates from Structural Model ............... 86

viii

Figure 1:

Figure 2:

LIST OF FIGURES

Relations Among SJT and PI Measure Subcomponents and the
Network of Constructs Around Them ................... 47

Parameter Estimates Among SJT and PI Measure Subcomponents and
the Network of Constructs Around Them ................. 87

ix

Chapter 1

LITERATURE REVIEW

In recent years, situational judgment tests have begun to gain popularity among
industrial and organizational psychology practitioners, largely based on the work of
Motowidlo (Motowidlo, Dunnette, & Carter, 1990; Motowidlo & Tippins, 1993).
These instruments represent an advance in selection procedures through their focus on
skills and abilities directly relevant to the job, measured in the form of a low-ﬁdelity,
low-cost simulation.

The use of situational judgment tests is not without precedent; assessment
centers (see Gaugler, Rosenthal, Thornton, & Bentson, 1987, for a partial review of
this topic), which also incorporate aspects of simulations, have been in use since the
mid-1950’s (Bray & Grant, 1966). More generally, Wemimont and Campbell (1968)

made a strong call for the measurement of behavior, rather than simply "signs" of

 

behavior, in making predictions about performance. It was inevitable, then, that a
measure like the situational judgment test would become popular.

The cliche that the best predictor of behavior is behavior certainly holds in
selection situations, and must be kept, at the very least, in the back of the mind when
making selection decisions. However, it is difﬁcult from a purely pragmatic

standpoint to measure behavior via the traditional paper-and-pencil measures that

2

continue to dominate selection situations. The best we can often do is to gauge the
behavioral intentions of applicants, and draw inferences about their likely behavior
from these reports. In examining a test such as a situational judgment test, such
intentions become all the more crucial since they are the criteria by which decisions
are made.

Another consideration is that situational judgment tests possess something that
traditional cognitive ability measures and other paper-and-pencil selection instruments
may lack: face validity, whether a test "looks like" it measures what it purports to
measure (Nunnally, 1959). This is important for several reasons, one of the foremost
being applicant reactions. While they are not the focus of this paper, they bear note
because practitioners must be aware of the effect their work will have on applicants
(Smither, Reilly, Millsap, Pearlman, & Stoffey, 1993). A robust ﬁnding in the
literature is that applicants favor selection procedures that are strongly related to job
content, usually in the form of work samples or high-ﬁdelity simulations, to the more
traditional written measures (Cascio & Phillips, 1979; Rynes, 1992; Schmidt,
Greenthal, Hunter, Berner, & Seaton, 1977). The situational judgment test represents
a method by which these same positive reactions may be obtained without going to the
expense of developing a work sample measure or a full simulation of the job position
to be ﬁlled.

The use of simulations, as well as work samples, has a well-documented
history in the literature, with demonstrated utility. The situational judgment test may
prove to be something of a boon to psychologists, but it remains unclear exactly what

constructs are tapped by the behaviors sampled (Motowidlo et al. , 1990). While a

3

simple "It works," may prove sufﬁcient for the needs of clients, it is our
responsibility as scientists to ask the natural question that follows such an assertion:
"Why?"

One explanation arising from a review of the cognitive literature involves the
notion of practical intelligence advanced by Stemberg and Wagner in their ongoing .
stream of research. Practical intelligence, they argue, is distinct from academic
intelligence in the underlying mechanism by which it operates. Academic
intelligence, that displayed in classroom situations, most generally taps formal
knowledge about the world around us. An example of this is the ability to recall the
date on which the United States entered World War H. The ability to recall such
facts and to apply them in structured learning situations would be one of the
hallmarks of academic intelligence.

Practical intelligence is far more dependent on Lac_it knowledge, which is not
directly taught but is instead gleaned from our daily lives, from interactions with
others and from the world around us (Stemberg & Wagner, unpublished manuscript).
For example, knowing how to deal effectively with an authority ﬁgure is a skill which
is not normally taught, but instead must be gained through experience. It therefore
represents tacit knowledge, and acting to defuse a tense situation with an angry
superior might be one example of practically intelligent behavior.

Because the situational judgment test requires job candidates to indicate the
manner in which they would approach a potentially problematic situation, it seems
reasonable to think that it might be tapping something akin to practical intelligence or

tacit knowledge. Some proponents of situational judgment tests (e.g. , Motowidlo), in

4

fact, maintain that their measures may constitute examples of practical intelligence in
work situations. In this study, I will test the relationship between a situational
judgment test and one of Wagner and Stemberg’s measures of tacit knowledge, in an
attempt to determine whether they measure the same underlying construct. This
comparison of measures can occur only after the basis for the construction of such
measures, their conceptual foundations, have been understood and compared as well.
That will be one of the major undertakings of the introductory sections of this paper.

Information about the similarity of these two types of tests (as well as more
traditional measures of intelligence or ability) is essential if we are to continue
utilizing the situational judgment test, because we do both our science and our clients
a disservice by not understanding the theoretical underpinnings of our measures. In
addition, this would provide a real-world application of the practical intelligence
construct, long a goal of researchers in that domain (see Wagner & Stemberg, 1985,
1986; Stemberg & Wagner, 1992).

A ﬁnal consideration involves the important synthesis such research represents.
Applied and theoretical concerns are often inconsistent, as has long been noted in the
literature (Freedheim, 1969). As such, any research which seeks to explicate the
relationship between theory and practice adds to the ﬁeld. Moreover, these two
literatures would both beneﬁt from being drawn together in such a manner, with each
receiving some measure of validation through its relationship with the other.

With that in mind, I will now turn to a more in-depth discussion of the
relevant literatures. For ﬁrst situational judgment, and then practical intelligence,

both the conceptual and empirical work will be reviewed, so that qualitative and

5

quantitative comparisons may be made. Because I want to demonstrate empirically
that the situational judgment test measures something closely related to tacit
knowledge, one concern in each empirical section will involve determining the
correlates of each measure. I will summarize the areas where such similarities may
exist at the conclusion of my literature review, in the form of hypotheses. Then,
using a methodology similar to that utilized in construct validation research, I will
evaluate the similarity of the constructs underlying the situational judgment test and
tacit knowledge test.

Situational Judgment Tests

Simulations, because of their direct relevance to the knowledge, skills, and
abilities used on the job, have long been recognized as valuable in selection situations
(W ernirnont & Campbell, 1968). However, the process of developing, marketing,
and implementing a large-scale, high-ﬁdelity simulation can be a costly one, and it is
a matter of guess-work on the part of the practitioner as to whether the costs will be
outweighed by the beneﬁts the simulation provides (Motowidlo et al., 1990).

As an alternative to these high-ﬁdelity, ﬁnancially risky simulations, the
situational judgment test was developed. While the situational judgment test
(hereafter, SJT) has its origins nearly four decades ago (see Bruce & Learner, 1958),
only recently has it come to be discussed in the literature. These discussions have
been few, however, and of varying quality, but they seem to be in agreement on two
issues: ﬁrst, that SJTs are valid measures in terms of concurrent validity (Motowidlo
et a1. , 1990) and predictive validity (Dalessio, 1994); and second, that more research

is needed to understand the constructs underlying performance on SJTs.

6
SJ T Development

The procedure utilized by Motowidlo et al. (1990) in the development of their
low-ﬁdelity simulation is in many ways one with the potential to generalize beyond
their content domain. More importantly an examination of their development
procedure should prove helpful in determining what constructs their SJT is built to
measure.

The ﬁrst step of their development process involved a review of previous job
analyses to determine the core skills necessary for the performance of the job in
question. Such prior documentation is useful, but it is not necessary if time and
money are available to conduct a job analysis for the express purpose of developing
an SJT.

Once the core skill dimensions have been identiﬁed, the next step in the
development process involves meeting with job incumbents and supervisors to collect
critical incidents. In their initial work, Motowidlo et a1. were concerned with
managerial effectiveness, and so collected incidents from a sample of managers and
their supervisors that reﬂected both effective and ineffective performance. These
incidents encompassed multiple dimensions of managerial effectiveness, determined by
the job analysis review from step one. Although their initial study included managers
from various functional specialties (ranging from administration to programming),
they saw three sets of skills as central to all areas: communication, problem-solving,
and interpersonal. For the purposes of the simulation, they focused on the problem-
solving and interpersonal areas as their primary targets. These were further broken

down as follows.

7

Problem-solving was seen as made up of organization, thoroughness, drive,
and resourcefulness. Interpersonal areas were deﬁned in terms of leadership,
assertiveness, ﬂexibility, and sensitivity. These sub-components of the primary
variables guided the collection of critical incidents and helped form the basis for later
portions of the study. Hence, these seem to represent the core constructs measured
by their SJT. I will return to these factors once I complete my discussion of the
development procedure to attempt a more in-depth analysis of the personality factors
associated with them. Because these constructs represent the most basic level at
which most analyses of situational judgment tests can take place, a thorough
understanding of their meaning and measurement is crucial.

With critical incidents collected, the next step in the development of a
situational judgment test involves writing descriptions of task situations. These
situations, based on the critical incidents, should reﬂect the behavioral categories
determined by the job analysis. Because almost any job for which such a selection
procedure is desired will involve more than a single task or type of task, it is unlikely
that an SJT will measure only a single construct. More likely, several constructs, all
related to different aspects of performance on the task, will be measured by different
questions on the SJT. As I already mentioned, Motowidlo et a1. (1990), attempted to
measure interpersonal elements of management and problem-solving elements of
management, in addition to more general measures of communication effectiveness
and overall effectiveness. Speciﬁc operationalizations of these dimensions (i.e.,
items), since they are the result of the analysis of this M jg), will not

necessarily be represented in situational judgment tests developed for other jobs within

8

a functional domain (e. g., managerial jobs). There is no single SJT, but the general
constructs measured may be the same across jobs. In this study, I am arguing that an
SJT is actually measuring practical intelligence, regardless of the job for which it is
designed to select applicants.

Returning to the issue of SJT development, the situations developed by
Motowidlo et a1. were given to 150 job incumbents. These incumbents were asked to
write a few sentences describing how they would handle each problem situation in the
most effective manner. For each situation, several general strategies from the
incumbents’ responses were selected as response options.

After the response options have been selected, the next step in such an effort
involves asking highly experienced, senior employees with a great deal of knowledge
about the job in question, to evaluate the relative effectiveness of the alternate
strategies for each task situation. The best and worst alternative strategies must be
identiﬁed, and all options are rated as to their effectiveness. From these data the
most and least effective response options are determined.

The scoring system is dualistic, in the sense that respondents are asked to rate
the response option they would most likely choose as well as that which they would
least likely choose. The option determined to be the "best" via the responses of the
experienced managers is assigned a score of +1 if it is the "most likely" response for
an individual, and a score of -1 if it is the "least likely" response. Similarly, the
"worst" item is assigned a score of +1 if it is the ”least likely" response, and -1 if it
is the "most likely." All other responses are coded as zero, so the score on any given

item ranges from -2 to +2.

9

The procedure and scoring system outlined above, while described in
Motowidlo et a1. (1990) can easily generalize to other situations. It is not content-
driven, and as such has the potential for broad applicability.

Constructs Underlying the SJT

The foregoing discussion of the SJT included a certain amount of speculation
on the nature of the constructs underlying the SJT. An understanding of these
constructs is essential to our progress, since the crux of what I will want to eventually
argue is that the constructs underlying the SJT are the same as those underlying a
measure of tacit knowledge or practical intelligence. This section will continue the
discussion of SJT constructs begun in the previous pages, and will attempt to expand
on what has been said.

The fact that such a stringent development procedure is used to formulate any
SJT virtually ensures that the measure will be highly face valid. It should also be
highly related to the actual knowledge, skills, and abilities involved in performing the
job for which persons are being selected (Cohen, Swerdlik, & Smith, 1992). A major
question, however, and one which I have already discussed to some degree, is the
following: Given this development procedure as an archetype, what constructs are
being measured? The situations with which people are faced are highly job-relevant,
the answers determined by experienced job incumbents. We want to determine how
well people who may have never performed this job in the past can respond to the
types of challenges that will face them, should they be hired. Whether these
measures work or not is a question of predictive validity, and will be taken up in my

review of the empirical ﬁndings on situational judgment tests. But if they do work,

 

10

what does it mean? Does it mean that high-scoring examinees possess practical
intelligence? This is a question we will return to in the conceptual review of the
practical intelligence literature. For now, the best we can do is a more in-depth look
at the general constructs measured by the Motowidlo et al. (1990) version of the SJT.

The KSAs likely measured by the Motowidlo version of the SJT fall into the
categories mentioned above -- problem-solving effectiveness, interpersonal
effectiveness, communication effectiveness, and overall effectiveness. Problem-
solving, interpersonal, and communication effectiveness all represent very broad, very
vague factors. It is necessary, in order to measure them effectively, to break them
down as Motowidlo et al. did, but it is reasonable to expect that as we move from
more macro- to more micro-levels, we will move more toward speciﬁc factors
associated with the job in question and away from anything general that may underlie
the archetypal SJT. To help us understand what may constitute that archetype, we
must look at the general factors, but eventually, since we are ultimately comparing
two measures, we must examine the breakdown of the speciﬁc factors as well.

The problem-solving factor is interesting, and makes some conceptual sense a
priori because we are asking people to indicate their likely responses to problem
situations. However, I mentioned previously the fact that different jobs will require
different problem-solving skills, or at the very least present different problems, due to
the varying challenges involved in each. Although the literature does not support the
importance of general problem solving ability as a determinant of test performance
(Motowidlo et al., 1990; Motowidlo & Tippins, 1993), the fact that it is a conceptual

piece of situational judgment means that we should consider which variables might be

11

most closely related to it. For instance, the level of an individual’s work experience
should have an impact on the skill he or she possesses at solving work-related
problems. From a more personality-based standpoint, several variables suggest
themselves: individuals who are more conscientious will stick with problems to the
end, and therefore develop more problem solving skill; individuals who are open to
experience will gain a broader base from which they can work to solve problems; and
individuals who are tolerant of ambiguous situations will be better able to deal with
the types of real-world problems they will face every day, in which critical pieces of
information are impossible to obtain.

If, as the literature seems to suggest, factors SJTs have in common across
content domains also include interpersonal effectiveness and communication
effectiveness, then we must ask again what personality variables are likely to
demonstrate or explain this relationship. In looking at interpersonal
effectiveness/skills, we might consider such variables as sociability, or the somewhat
broader extraversion construct (Baron & Byme, 1991). The dimension of
extraversion, as it is commonly considered, represents a measure of whether a person
is more talkative, frank, sociable, and so forth, as Opposed to being reclusive,
cautious, and generally silent. It might be expected that extraversion, since it will
partially determine the amount and quality of interaction an individual has with
others, will have an impact on performance on an SJT.

Other personality characteristics might prove important, however, in
understanding the factors underlying SJT performance. At the very least, a more

complete conceptual framing of those factors related to SJTs will allow for a more

12

comprehensive nomological network to be constructed.

Take, for example, tolerance of ambiguity (Furnham, 1994). In many
instances, the situations to which applicants must respond on SJTs will be unlike those
with which they have had experience in the past. The problems may not be clearly
deﬁned, due to their real-world base, and the way to proceed in order to reach a
solution may be ambiguous. As such, tolerance of ambiguity should be related to
performance on SJTs.

In addition, conscientiousness may prove important. If we accept that
communication and interpersonal skills are important variables, then it is reasonable
to expect that they are most likely to be utilized by individuals who feel the greatest
sense of responsibility for their work. That is, people who are more conscientious
will be more willing to communicate with others and utilize their interpersonal skills
than will those who do not exhibit this trait.

Another potentially related personality variable is openness to experience (see
Borkenau, 1988; Costa & McCrae, 1995). It seems clear that some ambiguity and
novelty will be characteristic of the situations described in SJTs, regardless of the job
for which they are designed. As such, we might expect that individuals who are more
open to new experiences will be better able to deal with such challenges, as opposed
to those individuals who are more narrow in their experiential focus.

This is by no means a comprehensive list of all potential personality variables,
but such a listing would make for a cumbersome text and an even more cumbersome
set of measures. The personality variables outlined above represent a sampling of

several potentially relevant domains, each of which is important individually, and all

 

13

of which may be the underlying factors involved in generalized SJT performance.
These factors include experience, age, education, communication, sociability/
extraversion, tolerance of ambiguity, conscientiousness, and openness. In the interest
of pragmatism, factors beyond those must be left for others to study.

In these two sections, I have reviewed the procedure used in developing SJTs
in order to come to a better understanding of the constructs, and I have considered
several potentially relevant personality variables. The nomological network
surrounding SJTs is exceptionally broad. For a greater understanding of this breadth,
I will now turn my attention to the empirical ﬁndings on SJTs.

Empirical Research

The previous section addressed the question of what SJTs measure from a
conceptual standpoint; now, I hope to approach the same question from an empirical
frame of reference. I will ﬁrst look at the validity evidence from the two studies by
Motowidlo, and then examine the intercorrelations of the SJTs with other potentially
relevant variables. In this way, I hope to uncover the empirical "meaning" of
situational judgment tests.

Motowidlo et al. (1990), as well as Motowidlo and Tippins (1993)
demonstrated that the SJT could, in fact, predict performance as measured by
supervisory ratings. The three general dimensions the SJT was designed to measure
(i.e. , interpersonal effectiveness, problem-solving effectiveness, and communication
effectiveness) as well as overall effectiveness, all correlated signiﬁcantly (p < .01)
with supervisory ratings on those dimensions (Motowidlo et al., 1990). These results

held for both job incumbents (technically a question of concurrent validity) as well as

l4
applicants hired using the SJT (Motowidlo & Tippins, 1993). The ﬁrst response one

might have to this could be a complete lack of surprise; after all, the test was
designed such that individuals who did well on it would have the skills needed to
perform the job, and would therefore receive higher ratings on those aspects of
performance tapped by the measure. But such a simplistic interpretation completely
ignores the question of what allowed job applicants, whose experience with the job
could be minimal at best, to do well on either the SJT or the later performance
review.

The correlations of the SJT with other variables that might impact performance
on the test will allow us to better answer this question. Fortunately, Motowidlo et al.
(1990) present precisely this data for two situational judgment tests, a 58-item
measure and a 30-item measure. Their data were collected from two samples of
managers (externally hired and internally promoted) as well as job applicants. Each
externally hired manager, as well as each applicant, was given an aptitude test
consisting of items measuring quantitative, writing, reading, and direction-following
skill, and each was asked for information about academic achievement (major GPA
and class rank). Predictor data on the internally promoted managers consisted of
assessment center ratings across several dimensions. In addition, interviews were
conducted with all participants in which leadership experience, interpersonal skill,
problem-solving skill, and communication skill were measured.

The results of the study are interesting, because the SJT had negligible
correlations with all facets of the aptitude test score (I = .01 to .16, ns). This is not

surprising, considering previous ﬁndings that success on simulations hinges more on

15

the ability to cope with the complexities of the situation than on verbally stated
knowledge (Broadbent & Aston, 1978). In addition, the SJT was uncorrelated with
academic achievement in all but the applicant sample, where major GPA correlated
.30 (p < .05) with scores on both versions of the SJT. It is possible, however, that
such a correlation may be an age or experience effect; the knowledge gained in a
formal educational setting may be all that young applicants have to call upon when
faced with a measure such as the SJT, while more experienced individuals have little
or no need of formal knowledge to respond appropriately. To put the matter more
succinctly, what you use may be a function of what you ppm.

An interesting corollary to this is the possibility that the SJT measures job
knowledge, and to some extent this may be the case. Individuals who have actually
performed the job in question will undoubtedly do better on an SJT than will those
who have not; however, if we conceive of the archetypal SJT as an entry-level
selection instrument, it makes sense to think that the proportion of new applicants
who will have direct experience with the job in question should be low. Thus, job
knowledge should certainly aid performance on an SJT, but very few applicants
should possess more than the most rudimentary version of such knowledge; therefore,
other forms of knowledge and abilities must be utilized if applicants are to perform
well.

In the interviews, several consistent ﬁndings emerged. In all three samples,
communication skill (deﬁned as the effectiveness of communication content and style,
as well as nonverbal skills displayed during the interview) correlated signiﬁcantly with

at least one of the two SJT forms, and in the sample of internally promoted managers

16

several assessment center ratings, all of which lay conceptually close to
communication skill, showed relationships to SJT score. Interpersonal skill was also
related to SJT performance in the sample of externally hired managers, but was not so
in either of the others. There may be a theoretical reason for this, or it may simply
be a function of sampling error in the large number of correlations generated for this
portion of the project.

One variable which should be correlated with performance on an SJT would be
relevant experience. In Motowidlo et al.’s (1990) study of managers, performance on
the SJT thus developed should be strongly related to either managerial experience or
experience with managers, since in either case the individual would gain experience
with examples of the right and wrong ways to perform managerial tasks.

Performance on an SJT designed to assist in the selection of police ofﬁcers should be
strongly correlated with prior experience in the types of situations law enforcement
ofﬁcers face, or with experience with legal matters.

While academic variables have not been shown to consistently impact SJT
performance, it is conceivable that there may be aspects of an individual’s education
that do have an effect. These will once again refer back to experience, however,
particularly to the extent that educational experience accounts for a large proportion of
an individual’s total experience.

The amount or quality of education received by different people may differ
(this is something of a truism, but bears stating nonetheless). When amount of
education is viewed as a predictor variable (since it represents additional real world

experience), and looked at not in terms of its outcome measures (such as GPA or

17

class rank), then it becomes reasonable to view it as potentially correlated with
performance on a test of situational judgment. In-depth theoretical reasons for the
effect of experience, in its various guises, will be discussed in the sections to follow.

A ﬁnal potentially relevant construct is Machiavellianism. Based on the
writings of Niccolo Machiavelli, the construct that bears his name is treated as a
personality trait stressing self-interest, deception, and opportunism (Mudrack &
Mason, 1995). Recall that one of the things supposedly measured by SJTs is
interpersonal skill. Because the subcomponents of Machiavellianism reﬂect (albeit
negative) interpersonal skills, performance on a measure of this construct may be
related to the individual’s ability to respond appropriately to a managerial SJT.

To understand a construct it is important to deﬁne to what it will relate, but it
is also important to specify to what the construct will not relate. It has already been
mentioned that SJTs should be unrelated to measures of cognitive ability (Motowidlo
et al., 1990). There is also no reason to expect that they should be related to such
demographic variables as gender, with the caveat that to the extent that gender (or
other) differences result in experiential differences, there may be a slight impact on
SJT performance.

Situational judgment tests, then, are low-ﬁdelity simulations whose conceptual
underpinnings are uncertain. Each measure is built around a different set of
circumstances, but because of the consistency of the development procedure and the
apparent consistency of the KSAs (particularly interpersonal and communication
skills; see Motowidlo & Tippins, 1993, and Dalessio, 1994) practitioners have tried to

measure using SJTs, we may expect that SJTs developed for different jobs assess

18

similar constructs. For instance, when faced with novel situations and asked to make
judgments about the best and worst options, perhaps individuals tap into their general
knowledge about that up; of situation; we might even call this "common sense."

The few studies that have been reported on SJTs lend some direction in the
search for answers to our conceptual difﬁculties, but to make worthwhile progress we
must turn our attention to a domain suggested by Motowidlo et al. (1990), the
literature on practical intelligence.

Practical Intelligence
Conceptual Underpinnings

The construct of practical intelligence was deﬁned earlier as that which is
gained from our experiences in the world around us, a deﬁnition which is only
complete when it excludes formal, academic knowledge gained in a classroom
environment (Wagner & Stemberg, 1985; Stemberg, Wagner, & Okagaki, 1993).
While it is useful to deﬁne what practical intelligence is M, it is absolutely necessary
to deﬁne what the construct i_s.

Neisser (1976) deﬁned it as "responding appropriately in terms of one’s long-
range and short-range goals, given the actual facts of the situation" (p. 137). An
individual confronted with a given situation (to return to our earlier example, dealing
with an angry superior at work) ﬁrst collects facts about the situation. This collection
process may or may not occur consciously, since many of the cues as to important
facets of the situation may not be obvious. For example, an individual may notice
that whenever his supervisor is angry, the supervisor makes more trips than usual to

the coffee machine. When the employee notices that his supervisor has made four

19

trips to the coffee machine in twenty minutes, and is snapping at other employees, he
will be alerted to the potential anger.

Once the available facts about the situation are collected, the individual will
then assess his goals relative to the situation. If he plans to advance through the
company and will need the support of his supervisor to do so, or if he has a review
that afternoon and needs to ensure the supervisor is in a good mood, he may act to
defuse his supervisor’s anger. If he has no such aspirations and only wishes to
remain in his current position and "not make waves," then he would likely avoid the
entire confrontation (assuming that it did not involve him personally). In either case,
he would be performing a practically intelligent behavior based on his assessment of
the facts of the situation relative to his goals. It is not necessary, however, for the
rules to be so concrete, nor is it necessary for the individual to be able to verbalize
them. Our employee might not even realize what his decision rule is for avoiding the
supervisor, but would be able to follow that rule nonetheless if he possessed practical
intelligence.

In order to truly understand, conceptually, what practical intelligence is, we
must take another step back and ask about the processes that underlie it. When we do
this, we ﬁnd ourselves confronted with the notion of tacit knowledge (Polanyi, 1976).
Here we must take care not to get overly engaged in the jargon; Stemberg and
colleagues often seem to use "practical intelligence" and "tacit knowledge"
interchangeably. However, I will endeavor to adhere to the following distinction,
which seems implicit in the literature: Practical intelligence is behaviorally-linked, in

the sense that for a person to be deemed practically intelligent, she or he must

20

perform a behavior (or behaviors) ﬁtting the "practical intelligence" concept we hold
in our minds. This deﬁnition is obviously a tautology, and requires us to take another
step back. This step takes us to tacit lmowledge, which is not so behaviorally-linked.
Tacit knowledge is that knowledge base from which people draw, which enables them
to perform in ways that are perceived as practically intelligent. I will do my best to
remain true to this distinction, between practical intelligence and tacit knowledge,
throughout the remainder of this paper.

As was discussed earlier, tacit knowledge is that which is not only not taught,
but also is rarely expressed or stated. Instead, tacit knowledge is gleaned through
experience (Stemberg et al. , 1993). For the purposes of measurement, Stemberg and
colleagues break it down into three sub-components, all of which can be tied to the
workplace.

Managing oneself deals with knowledge about self-motivational and self-
organizational aspects of performance at work. The self-regulatory aspects of
performing as a manager, then, are important in Wagner and Stemberg’s model.

Managing others deals with knowledge about work related interactions with
subordinates, superiors, and peers. This is the type of tacit knowledge utilized by our
employee, above, and may be reﬂected in a variety of ways; while a discussion of
speciﬁc measurement options would likely be premature at this point, one might
suspect that such constructs as Machiavellianism might prove relevant.

Finally, managing tasks deals with knowledge about how to perform speciﬁc
work-related tasks well. All three of these dimensions are important for managers if

they are to be deemed "practically intelligent" (Stemberg & Wagner, manuscript

21

under review).

The primary measures of tacit knowledge were developed (unsurprisingly) by
Wagner and Stemberg (e.g., 1991). Their design reﬂects their tri-dimensional view
of the construct of tacit knowledge, with items included to tap each of the three
aspects -- managing self, managing others, and managing tasks.

The development of the tacit knowledge measures is very similar to that of a
situational judgment test. As with the SJT, the tacit knowledge measures are job-
speciﬁc and are based on the collection of detailed information from job experts about
typical work-related situations and possible responses to them. Several examples
from two different tacit knowledge scales are included in their research reports (see
Wagner & Sternberg, 1985; 1986). The ﬁrst scale includes items tapping the tacit
knowledge of academic psychologists, the second tapping that of business managers
(Wagner & Stemberg, 1991). For each item, a brief scenario is described and a
number of response options (varying from six to twenty, with no apparent guidelines
for a target number) are offered. Each response option is then rated by the
respondents on a 7-point Likert-type scale, in terms of its importance "by its priority
as a means of reaching your goals" (Stemberg, Wagner, Williams, and Horvath,
1995, Appendix A).

It should be apparent that in terms of tapping the universe of content
associated with the target job, the SJT and Wagner and Stemberg’s method of
measuring tacit knowledge are very similar. Additionally, they ask for very similar
discrirninations in terms of potential responses to (presumably) novel situations.

These similarities have been noted by Motowidlo (Motowidlo et al. , 1990) as well as

22

other authors (Dalessio, 1994), and are important because they suggest that the same
set or sets of skills are important for performing well on both types of tests.

It must be noted that while Wagner and Sternberg do not set up their measure
as a simulation, almost any measure that seeks to investigate the manner in which
people operate in everyday situations will, if content-valid, take on the characteristics
of a low—ﬁdelity simulation as described by Motowidlo and his colleagues.

This framing -- or, more aptly, re-framing -- is important, because it allows us
to examine the work on practical intelligence and tacit knowledge in a more applied
light. In spite of the work done to move the construct of tacit knowledge to the ﬁeld
(see esp. Stemberg et al., 1993; Stemberg & Wagner, 1992; Wagner & Stemberg,
1985, 1986), one might suggest that the framing of this work has forced it to remain
on the "theory" side of the fence. While this is certainly speculative, it might be that
the jargon used by Stemberg and Wagner tends too much toward the language of
traditional experimental psychology, and speaks less in the language of modern
applied psychology and its constituents. Whatever the reason, we must do at least
this modicum of reframing in order to make a meaningful comparison between
practical intelligence and the situational judgment test.

Another interesting aspect of tacit knowledge, and more generally practical
intelligence, is the fact that it is deﬁned by what it is no_t (Wagner, 1994). Because of
the entrenchment of traditional notions of intelligence as so tightly tied to academics
and classroom performance, such a deﬁnition is necessary. Wagner maintains that the
types of problems, and therefore the mental competencies needed to deal with the

problems, are very different in the work and academic environments. However, since

23

the popular conception of intelligence has to do with IQ, any deﬁnition of practical
intelligence must differentiate itself from that conceptualization and deﬁne what it is
not in order to better explain what it is. Perhaps the chart (see Table 1) from Wagner
(1994), adapted from the work of Neisser (1976) and Wagner and Stemberg (1985)
demonstrates the differences between the two domains of intelligence best. In
addition, it shows several of the ways in which practical problems are deﬁned by
exclusion.

Tacit knowledge would then be that knowledge which helps the individual to
deﬁne and formulate the problem, determine what information is required, and sort
through the multiple correct answers and the multiple potential solutions to a problem
embedded in everyday experience. This form of knowledge is primarily gained
through direct experience or through interactions with an experienced person, such as
a mentor (Wagner, 1994).

A further implication of the information contained in this chart is that practical
problems are more ambiguous. This leads us back to our earlier discussion of
personality characteristics likely associated with SJT performance. Obviously, if this
chart is accurate, individuals with a higher tolerance of ambiguity should perform
better on tests of tacit knowledge because of the ambiguous nature of the problems
presented. Closely related to this, and also discussed earlier, is an individual’s
openness to experience. Since the tacit knowledge measure, like the SJT, often
involves hypothetical and potentially unfamiliar situations, individuals who are open to
new experiences will likely be able to cope more effectively with the challenges

presented therein.

24

 

 

Table 1: Characteristics that differentiate academic and practical kinds of problems

Academic Problems Practical Problems

1. Well deﬁned 1. Ill deﬁned

2. Formulated by others 2. Unformulated

3 Necessary information 3. Additional information
provided required

4. One correct answer 4. Multiple correct

answers

5. One method to obtain 5. Multiple methods to
answer obtain answers

6. Disembedded from everyday 6. Embedded in everyday
experience experience

 

Source: Wagner (1994)

25

Two other personality variables were mentioned in my discussion of SJTs, and
I will reintroduce them at this point. Tacit knowledge, since it is experience-based,
will be dependent on the type and quality of interactions an individual has with others.
Hence, individuals who are more extraverted or sociable will have more opportunities
for potentially relevant experiences, and should do better on the tacit knowledge
measure. They should, in other words, communicate more and have better
interpersonal skills. Similarly, conscientious individuals will likely engage in more
information-seeking strategies in order to increase their knowledge base; this
information seeking, while increasing their formal knowledge, will also (potentially
unavoidably) increase their tacit knowledge as well, by providing them with
information they had not requested and did not attend to when it was received. For
instance, recall the example from the beginning of our discussion of tacit knowledge.
Imagine that our employee, in his early, naive days with the company, approached his
supervisor with a question while the supervisor was on the way back from the coffee
machine. If the question were answered in a more curt manner than that to which the
employee was accustomed, our employee would have received not only the answer to
his question, but also information about when not to approach his supervisor.
However, the relationship between the coffee machine and the supervisor’s curtness
might not be noticed at that time; it would be noted, on some level, and over time
this tacit knowledge could lead to changes in the employee’s behavior to correspond
with the supervisor’s coffee-drinking habits.

A ﬁnal parallel can also be drawn between SJTs and Stemberg and Wagner’s

measure of Tacit Knowledge. We noted earlier that because the SJT is believed to

26

measure interpersonal skills, Machiavellianism may be related to performance on it.
Similarly, one of the dimensions of the tacit knowledge measure is "Managing
Others." Self-interest, deception, and opportunism, three characteristics of the
Machiavellian personality type, seem to be quite relevant when we consider
individuals who manage others, so we would expect some relationship to exist
between Machiavellianism and performance on a measure of tacit knowledge.

The personality correlates of SJTs, then, should reasonably be expected to
correlate with tacit knowledge measures as well. Some aspects of the tacit knowledge
measure, however, should be less parallel to SJTs. These differences center around
the Managing Self dimension speciﬁed by Stemberg and Wagner.

Managing Self, as it was discussed earlier, deals with knowledge about self-
motivational and self-organizational aspects of performance at work (Stemberg &
Wagner, manuscript under review). To some extent, this parallels the SJT’s concern
with problem solving, when problem solving is conceivedof as consisting of the four
elements noted by Motowidlo et al (1990) -- organization, thoroughness, drive, and
resourcefulness. In particular, organization and drive seem to reﬂect Managing Self’ 5
concerns with self-motivation and self-organization. However, our tools to measure
problem solving are less useful when we start talking about something so conceptually
distant (at least in common thought) as the ability to manage oneself. Two distinct
indicators for Managing Self seem clear, which should be related to indicators for
problem-solving ability. These are self-regulatory skill and learning style.

Self-regulation has long been a topic of interest to educational psychologists,

as well as others concerned with learning, but only recently has it gained much

27

attention in the I/O literature (see Kanfer & Ackerrnan, 1989, for the most prominent
example of its application). A review of this literature indicates that, with respect to
management science, only a few recent articles have dealt with this area. One of the
primary theorists on self-regulation, Albert Bandura (1991), deﬁnes it as having three
primary components. Self-observation, or self-monitoring, includes observations on
both the performance dimensions and the quality of the monitoring itself. Self-
judgments involve one’s behavior with respect to personal standards, referential
performances, valuation of activities, and the various performance determinants.
Finally, affective self-reactions can be either evaluative or tangible, but they are also
a part of the self-regulatory process.

The self-organizational and self-motivational aspects of Managing Self should
be related to self-regulatory activities. However, self-regulation is a broad concept,
and we must narrow it somewhat to make it useful for the purposes of this study.

Ashford and Tsui (1991; Tsui & Ashford, 1994) note that one of the self-
regulatory behaviors many successful managers engage in (where "success" is
measured in terms of ratings of effectiveness by superiors, subordinates, and peers) is
negative feedback-seeking. This makes a great deal of sense when you consider that,
in many organizational contexts, speciﬁc information about performance is not made
explicit all the time; hence, if a manager wants to know how she is performing, she
will have to attempt to obtain feedback independent of the annual or semi-annual
review process. Negative feedback should be more diagnostic than positive, if the
ultimate goal is performance improvement, so we can thus understand why Ashford

and Tsui (1991) saw negative feedback—seeking as such a crucial part of managerial

28

self-regulation. Hence, while it has not been tested in the literature, if Stemberg and
Wagner are correct in their deﬁnition of the Managing Self dimension, items that
purport to measure that dimension should be related to the individual’s ability to self-
regulate, and more speciﬁcally to seek negative feedback as part of the self-regulatory
process.

The second indicator to be discussed with respect to the Managing Self
dimension, but not the remainder of the TK dimensions, is learning style. Much of
the important work on learning style, particularly as it is measured, has been done by
Kolb (1985). This work, based on Kolb’s (1976) experiential learning theory in
which learning proceeds from concrete experience, to reﬂective observation, to
abstract conceptualization, and ﬁnally to active experimentation, sought to test and
validate this model of learning (Romero, Tepper, & Tetrault, 1992). However, the
measures developed by Kolb have come under considerable scrutiny, and much work
has been done to improve them or to devise new measures of the construct (Romero
et al., 1992; Veres, Sims, & Shake, 1987; Carraher, 1993; Geiger, Boyle, & Pinto,
1993).

Two primary dimensions on which people differ, under the conception of
learning style held by Kolb and many of his fellow researchers, are
concreteness/abstractness and reﬂection/action. The style of learning people prefer, if
it cannot be said to dictate career choice, has at least been shown to correlate with
one’s chosen ﬁeld; for instance, from a sample of college students those individuals
whose learning styles tended to be more concrete than abstract and more reﬂective

than active have been shown to prefer such majors as sociology or foreign languages,

29

while those who were more abstract than concrete and more active than reﬂective
preferred majors such as engineering and computer science (Romero et al. , 1992).
Hence, learning style does appear to be measuring something real.

Because the two dimensions of learning style exist, and because we must
assume that, if Stemberg and Wagner are correct and tacit knowledge does indeed
correlate with managerial performance, we may make the following statement:
individuals who have learning styles that are consonant with those of managers will
likely do better at a tacit knowledge measure of managerial effectiveness. It is
through the Managing Self dimension that this will occur, since learning styles
originate within the individual.

While the preceding discussion may be slightly longer than that provided for
SJTs, the above still does not represent a comprehensive list of all the potentially
relevant constructs underlying practical intelligence. Rather, it should serve as a basis
for further development in later, potentially broader, research projects.

The conceptual basis of practical intelligence must be traced even farther back,
since by themselves practical intelligence and tacit knowledge offer little if any
mechanism by which such knowledge or intelligence might be gained. One such
explanation, which is largely concordant with the assumptions underlying the work of
Stemberg, Wagner, and their colleagues, is to be found in the literature on implicit
learning.

Implicit gaming
Implicit learning (Reber, Walkenfeld, & Hernstadt, 1991) is "the process

whereby a complex, rule-governed knowledge base is acquired largely independently

30

of awareness of both the process and the product of the acquisition" (p. 888). It is, to
put it in common parlance, primarily unconscious (Lewicki, Czyzewska, & Hoffman,
1987), or at the very least _no_nconscious (Reber & Lewis, 1977). Its foil is explicit
learning, the mechanisms of which are somewhat similar to what we normally think
of as problem solving (Mathews, Buss, Stanley, Blanchard-Fields, Cho, & Druhan,
1989). Thus, the difference between the two is just what their names suggest;

implicit learning happens somewhere beyond (or beneath) our awareness, while
explicit learning is that which we notice and control.

The task that really helped deﬁne the ﬁeld of implicit learning involves
artiﬁcial grammars. In experiments with artiﬁcial grammars, subjects are exposed to
15-25 letter strings, constructed using an artiﬁcial grammar ﬁrst used by Reber (1965;
cited in Reber, 1989). After repeated exposures to these strings (either in the form of
memorization to criterion or simple observation), subjects are then asked whether
novel strings, constructed using the same artiﬁcial grammar, are grammatical (Seger,
1994). Performance levels of 60-80% are common on such tasks (Reber, 1989), but
subjects cannot completely verbalize the rules by which they made these judgments,
lending credence to the notion that their knowledge of the grammars is implicit, or
unconscious (Mathews et al., 1989).

Other procedures have been used to test for implicit learning, and an excellent
review of these is provided by Seger (1994). The work on artiﬁcial grammars is
sufﬁcient, however, for the reader to come to an understanding of the logic and to a
lesser extent the methodology underlying research on this topic.

The relationship between implicit learning and tacit knowledge should be

31
obvious, and in fact it has been noted by Reber (1989), who concluded that implicit

learning allows for the constructiOn of a tacit knowledge base. A summary of the
likely mechanisms behind practically intelligent behavior is useful at this point.

Every minute of the day, we take in information from the world around us.
This occurs independently of conscious efforts to learn, and once such information is
taken into the mind, we may say that implicit learning has occurred. We have
learned something about the world around us, about its patterns and the ways in
which it functions, without necessarily wanting to learn or even knowing we were
engaged in the learning process (Lewicki et al., 1987). Once this learning has
occurred, it is abstracted into a mental representation of our environment. This
structure is the foundation for our tacit knowledge, and allows us to solve problems
and make judgments based on novel stimuli by comparison with the tacit knowledge
framework developed via implicit learning. Such ability, as we saw above, is the
hallmark of practically intelligent behavior.

Implicit learning, then, provides one plausible mechanism by which tacit
knowledge, and therefore practical intelligence, may be understood. While this study
will not test the linkage between tacit knowledge (as measured by Wagner and
Stemberg) and the more deeply cognitive notion of implicit learning, the relationship
between these areas was important to establish in order to fully understand the
conceptual foundations on which this study is built. With this conceptual framework
in mind, we should now turn our attention to the empirical literature on practical
intelligence.

Empirical Research

32

As with my review of the empirical research on SJTs, I have a pair of goals
for my review of the empirical literature on practical intelligence. First, I hope to
demonstrate the broad range of empirical studies that have investigated practical
intelligence, many of which sought to assist in the deﬁnition of the construct as
distinct from academic intelligence. Then I will parallel my discussion of SJTs by
examining the correlates of measures of practical intelligence/tacit knowledge.

Empirical studies of practically intelligent behavior abound in the literature.
For instance, Lave, Murtaugh, and de la Rocha (1984) studied women who did the
grocery shopping for their families. The authors found that these women, who were
incapable of completing a paper-and-pencil arithmetic test, were able to perform the
em _sa__me computations in their heads when computing prices in the supermarket.

Ceci and Liker (1986) studied men who handicapped horse races as a hobby,
and found that men with low Iqs could show considerable expertise and cognitive
complexity in their handicapping efforts. They concluded that something other than
the traditional, IQ-based notion of intelligence was at work in these men, and this has
been interpreted by later authors as evidence of practical intelligence (Stemberg et al.,
1993).

Scribner (1986) studied men working in a milk-processing plant, and showed
that in Spite of what are often low IQs, they can ﬁnd new and creative ways to speed
up their work without sacriﬁcing quality. Parallel to this is the example, given by
Sternberg et al. (1995) of garbage collectors in Tallahassee. In spite of having a job
low in intellectual challenges, these individuals still show innovative behavior that

assists them in accomplishing their goals in the most expedient manner.

33

Stemberg claims that all of these examples have an underlying mechanism in
common. The employee with the angry superior, the women in the supermarket, the
handicappers at the race track, the men in the milk-processing plant, and the garbage
men, are all accessing tacit knowledge (Wagner & Stemberg, 1986; Polanyi, 1976).
This tacit knowledge, in all likelihood, arose from implicit learning that occurred
while they were engaged in their respective tasks.

As with the SJT, in looking at practical intelligence and tacit knowledge the
conceptual underpinnings of the construct are only the beginning. Empirically, what
is it? What does it do? With what variables does it correlate? In order to answer
these questions, we must make the same examination of practical intelligence/tacit
knowledge that we made of the SJT.

It should be obvious from the examples cited above (e. g., Ceci & Liker, 1986)
that the literature on practical intelligence has done all that it can to distinguish
practical intelligence from what we normally think of as "intelligence," up to and
including deﬁning one as what the other is not. The reports very carefully state such
things as, "women who can compute effectively in a supermarket price-comparison
situation may not be able to compute effectively in a paper-and-pencil arithmetic test
of isomorphic operations" (Stemberg et al., 1993, p. 205), or, "tacit knowledge was
not highly related to verbal intelligence as measured by a standard psychometric test"
(Wagner &. Stemberg, 1985, p. 450). Measures of cognitive ability sometimes seem
meticulously chosen by Wagner and Stemberg, and may measure only a single aspect
of ability, such as verbal intelligence from the example above.

Some measure of methodological skepticism is understandable; however,

34
researchers other than Stemberg and Wagner have replicated the ﬁnding that tacit

knowledge is unrelated to measures of ability such as the ASVAB (Armed Services
Vocational Aptitude Battery; Eddy, 1988, cited in Stemberg et al. , 1995). Our
criticisms, then, must be tempered with the knowledge that a fairly robust ﬁnding
with regard to practical intelligence is that it is relatively unrelated to other measures
of intellectual ability.

A more detailed examination of the distinctiveness of practical intelligence
from IQ and more general measures of cognitive ability will be taken up in the next
section, but for now we can only say that measures of practical intelligence are
unrelated to some measures of cognitive ability. Possibly the most critical fact about
the relationship between these two variables is to be found in Stemberg et al.’s (1995)
article.

Using the technique of hierarchical regression, Stemberg et al. ﬁrst entered IQ
(as measured by the Shipley, an intelligence test), followed by tacit knowledge in
order to predict behavioral-assessment-data ratings in small-group managerial
simulations. The increment in R2 resulting from the addition of tacit knowledge was
a highly signiﬁcant .32, bringing the R2 for the total model to .46. In other words,
with the addition of tacit knowledge as a predictor, 32% more variance in the
criterion variable was explained. Stemberg et al. (1996) do not, unfortunately,
provide some pieces of information that might allow us to approach this study with a
more critical eye; no information is provided with respect to the size of the sample
employed, nor do they provide information about the incremental contribution of IQ

when tacit knowledge is entered ﬁrst. Taken at face value, though, the literature

35

seems to suggest that tacit knowledge and cognitive ability are unrelated constructs.
If Stemberg et al. are correct, their example demonstrates empirically that tacit
knowledge explains a signiﬁcant portion of variance not accounted for by this
traditional measure of cognitive ability. However, it is clear that further research
must be undertaken to bear this claim out.

In addition to cognitive ability, the literature indicates that tacit knowledge,
like SJTs, should be unrelated to demographic variables such as gender or race
(Eddy, 1988, in Stemberg et al., 1995). However relevant experience has been
shown repeatedly to be related to performance on the tacit knowledge measure (e.g. ,
Stemberg et al., 1995).

An area that has received a fair amount of attention in the literature not only
on practical intelligence but also in the conceptually-related domain of practical
problem-solving involves age and education level. This is more likely due to the tight
linkage of age and education to the opportunities an individual has to develop a broad
experiential base.

Because a portion of their research has included in its sample psychology
graduate students, Wagner and Stemberg (1985, 1986) have had an opportunity to
examine tacit knowledge at different levels of graduate education. Their results
indicate a signiﬁcant relationship between total tacit knowledge test score and the
number of years of graduate school completed. The more years of graduate school
psychology students in their sample completed, on the average, the better they did on
a tacit knowledge measure designed for academic psychologists. This brings up an

apparent contradiction, since the deﬁnition of practical intelligence that has been

36

stressed from the outset of this paper is that it is n_ot academic intelligence -- yet here
we ﬁnd a link between educational level and tacit knowledge, our indicator of
practical intelligence. The resolution, fortunately, is fairly simple; the tacit
knowledge measure is not tapping academic knowledge, it is tapping the experiential
base accumulated by the graduate students, which allows them to better perform as
academic psychologists. Hence, it is not the formal knowledge gained from courses
that is tapped, it is the experiences that provide insights into the world of academia
and its characteristic problems.

The results from Wagner and Sternberg (1985, 1986) are mirrored by the
ﬁndings of Klaczynski, Laipple, and Jurden (1992), who found that in a sample of
high school students, sophomores and seniors had different interpretations and
solutions to practical problems. The difference, according to the authors, lay in the
fact that sophomores sought social solutions to practical problems. Seniors, being
more concerned with issues of planning, put forth solutions that tended to be more
goal—directed. If we recall Neisser’s (1976, p. 137) discussion of practical
intelligence as "responding appropriately in terms of one’s long-range and short-range
goals, " we can then see that seniors, when compared to sophomores, showed more
practically intelligent behavior.

The problem when interpreting such data lies in teasing out the effects of
educational experience from more general age effects. Closely related to (and
potentially confounded with) educational issues is a set of ﬁndings regarding age and
practical intelligence/problem solving ability. Cornelius and Caspi (1987) found that

everyday problem-solving performance (their measure of practical intelligence),

37

increased linearly with age. Other measures of practical problem-solving ability have
also been shown to be related to age, though the form of the relationship sometimes
differs; Denney and Palmer (1981), for instance, found a quadratic relationship in
which middle-aged individuals demonstrated maximal practical intelligence. It is
uncertain at this point whether such differences across studies are a function of the
speciﬁc measures used. The fact that two scales purport to measure the same
construct is no guarantee that they in fact do so.

The importance of whether or not a drop—off occurs in practical intelligence
following middle-age is minimal for the purposes of this study, however, due to the
relatively young nature of the proposed sample. The critical fact is that, at the very
least, practical intelligence continues to rise until the 408 or 505 (Denney & Palmer,
1982). Thus, we would expect age differences to be visible in measures of practical
intelligence such as Wagner and Stemberg’s tacit knowledge. Again, though, as with
SJTs, it is unclear whether age, education, and experience are truly distinct in their
effects on SJT and tacit knowledge performance. This question, while beyond the
scope of this research, must certainly be addressed in future work.

One area of study which has been under-researched, if not ignored completely,
involves the relationship between practical intelligence and interpersonal factors. If
our conceptual understanding of practical intelligence as deriving from implicit
learning mechanisms is accurate, then it would make sense to think that individuals
who interact more with others will have a greater potential for such learning. More
directly, we would expect that those individuals who are able to communicate most

effectively with others would gain the most information. Both of these predictions, it

38
should be noted, parallel ﬁndings from Motowidlo et al.’s (1990) study of SJTs.

Unfortunately, we have no empirical data relating practical intelligence to the various
personality measures; that will be one of the potentially interesting pieces of
information provided by this study.

The literature on practical intelligence, while broad, has yet to delve deeply.
This problem is not unique to the practical intelligence literature, however, and is to
be found just as markedly in the more general literature on intelligence (Neisser,
Boodoo, Bouchard, Boykin, Brody, Ceci, Halpern, Loehlin, Perloff, Stemberg, &
Urbina, 1996). A great deal can be gained from this literature that can inform our
discussion. One major issue remains to be dealt with, however, before we can move
on to the hypotheses for this study -- the distinctiveness of practical intelligence from
traditional intelligence and general cognitive ability. In the next section, the literature
on these two topics is reviewed, and conceptual arguments are presented.
Intelligence and g

The research trail on intelligence leads back to Francis Galton’s studies of
eugenics (Hothersall, 1995), during the time in which psychology was developing into
a legitimate science. Because of the lengthy time frame and the breadth of research
done over the last century, any attempt at a comprehensive review of this literature
would fall far beyond the scope of this thesis. What I will do in the next few pages is
review recent research regarding the dimensionality of human intelligence, as well as
the theory of g, Spearrnan’s single, general factor in cognitive ability. It has been
established that general cognitive ability is predictive of job performance (e. g.,

Hunter, 1986). If the situational judgment test is to possess theoretical or practical

39

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Table 2: Relationships between relevant constructs and conceptual and empirical notions of the
situational judgment test and practical intelligence
ll Construct SJT Concept SJT Empirical PI Concept PI Empirical I
Experience Yes ; = .08‘" Yes nsh
Age Yes NRc Yes ; > 0.0‘
Education Yes‘ --‘ Yes ; = .40
Cognitive Ability No No
Communication Yes r = .16 Yese --
Sociability Yes 5 = .15 Yese --
T01. of Ambiguity Yes Yes
Conscientiousness Yes Yes
Openness Yes Yes II
Gender No No II
a: where correlations are reported, they are the weighted average of correlations available from
the literature
b: the studies described by Motowidlo et al. (1990) and Stemberg et al. (1995) used only a
narrow deﬁnition of experience, which likely accounts for the low effect size
c: NR = Not Reported

in the Motowidlo et al. (1990) study, the only measures of education dealt with achievement
and as such are inappropriate to include as summary information

e: these responses were extrapolated from the literature, and are reﬂected in the hypotheses for
this study
r: Denney & Palmer (1981) showed that practical problem solving ability increased until age 40-

50, then dropped off

40

signiﬁcance beyond the standard measures of cognitive ability used in the workplace
today, it must ﬁrst be demonstrated that it measures something beyond other cognitive
ability measures. Portions of this argument have been made earlier in this paper; now
I hope to draw them all together, to present a cogent argument for the importance of
SJTs and, more generally, the concept of practical intelligence.

Neisser et al. (1996) provide an excellent review of what is known about
intelligence. One of the things that they note from the outset is the lack of consensus
as to what is meant by " intelligence, " an historical problem with the construct.
Spearman (1927) argued for g, a single, general factor of cognitive ability that
underlies all measures of intelligence. His position was contested by Thurstone
(1938), as well as others who held that intelligence was made up of several different
abilities that could not accurately be lumped into a single category without losing a
great deal of information about each individually.

Because this debate still rages, and because it bears so directly on the concerns
of this research topic, it is worthy of discussion. I’ll begin with g, examine its
characteristics and the support for its existence, and then move to opposing views.

To say that g is ubiquitous in the literature on cognitive ability might be an
understatement. The construct, along with its explanations and its defenses, has been
the topic of hundreds of articles and has been the subject of entire issues of various
journals (Gottfredson, 1986). But what do we know about it? Or, a more
appropriate question may be, what do we m we know?

We know, mathematically, that it is the primary factor when ability tests are

factor analyzed. Utilizing oblique rotations, which move common variance in

41

correlated factors to successively higher-order factors, g is the ﬁnal factor derived and
sits at the pinnacle of the factor analytic pyramid (Jensen, 1986). It represents that
quantity which accounts for the variance common to all the correlated ability factors.
Moreover, this factor falls out of virtually all cognitive ability tests, and these
measures show moderate to high correlations with one another. This allows for the
formation of a "positive manifold, " meaning that all these tests measure one factor --
the g factor (Neisser et al., 1996). Moreover, because they are all assumed to
measure the same underlying factor, it may be that the tests all provide equally valid
estimates of g (Larson & Wolfe, 1995).

Conceptually, the most important thing that we seem to "know" about g is that
it is the single factor underlying what is normally perceived as intelligence, with some
authors going so far as to equate g with intelligence (Hunter, 1986). It is treated as
the "key to a comprehensive account of the nature of mental abilities" (Undheim,
1981, p. 263), yet even the most astute researchers have problems deﬁning precisely
what g, as a construct, means. The following passage from Jensen (1986) perhaps
best exempliﬁes the fullness to which deﬁnitions of g can be taken. He says that g

reﬂects some property or processes of the human brain that are manifested in

many forms of adaptive behavior, and in which people differ, and that increase
from birth to maturity, and decline in old age, and show physiological as well
as behavior correlates, and have a hereditary component, and have been
subject to natural selection as a ﬁtness character in the course of human
evolution, and have important educational, occupational, economic, and social

correlates in all industrialized societies, and have behavior correlates that

42

accord with popular and commonsense notions of "intelligence" (p. 329).
The breadth of this deﬁnition is amazing, as it takes in virtually all aspects of
cognition and attempts to explain them all with a single concept. Such a mechanism
is a common goal in all the sciences, be it a uniﬁed ﬁeld theory in physics or a theory
of general cognitive ability, and there can be no argument that such a search is an
admirable one. The development of a uniﬁed theory of cognitive ability would be
something of a blessing to psychology, but we have to ask ourselves if g is that
theory.

The answer, unfortunately, is not a simple yes or no. For instance, I have
mentioned several times that cognitive ability (g) is the best single predictor of job
performance (Hunter, 1986; Hunter & Hunter, 1984). However, the work of
Stemberg et al. (1995), particularly the Center for Creative Leadership study
discussed earlier, demonstrated that the addition of tacit knowledge to a traditional
measure of cognitive ability resulted in a substantial increase in predictive capacity.
The two in combination did not completely explain performance, though they did
account for almost half of its variance; the study is most important in that it points out
the existence of something that measures of g do not approach.

One of the major problems seems to be that the deﬁnition and
operationalization of g may or may not match. If we accept Jensen’s
conceptualization of g as one of the most comprehensive then we must ask ourselves
if the measures of cognitive ability that are taken to be indicative of g measure the
construct accurately. To be sure, they all correlate highly with one another (Neisser

et al., 1996; Jensen, 1986). This kind of consistency would allow us to say that, to

43

the extent that a general factor might exist, we have some measure of evidence for it.
However, if predictions based on g-centered measures can be improved, then we may
be led to doubt the accuracy of Jensen’s seemingly all-inclusive deﬁnition of the
components of g.

To put it more bluntly, we may deﬁne g in whatever terms we want, but so
long as it can only be Operationalized as the general factor (often of the third level)
that can be extracted from a test speciﬁcally designed to measure cognitive ability,
problems will arise (Neisser, 1979). The fact that a test such as Wagner and
Stemberg’s tacit knowledge measure, which would seem to fall under the domain of
Jensen’s deﬁnition, adds explanatory power to a more traditional measure of cognitive
ability, indicates one of two things.

The ﬁrst possibility is that g is distinct from the ability measured by the tacit
knowledge test, since it accounts for outcome variance independent of that accounted
for by g. If this is the case, it may be that general cognitive ability is not so general.
A second possibility is that measures of g are incomplete, and need to take tacit
knowledge and related constructs into account. This, as was alluded to earlier, would
indicate a disconnect between the conceptualization and operationalization of g, and
would suggest that the nomological network of theorists in that area needs reworking
(Neisser et al., 1996). Whatever the case, it seems clear that something about g is
amiss -- possibly that our conception of cognitive ability is overly narrow
(Frederiksen, 1986b).

In order to determine which of the two Options above is more likely correct,

other literatures must be consulted. The physiological literature, particularly as it

44

relates to implicit learning, is one such area.

A fairly robust ﬁnding in this literature is that, even in situations under which
the normal cognitive functions of the mind are impaired (or "insulted," as Reber et al.
(1991) put it), such as Korsakoff’s syndrome, Alzheimer’s disease, various psychiatric
disorders or even chronic alcoholism, individuals still display implicit learning.
Traditional measures of g are useless in such situations. Because of the functions that
remain along with the capacity for implicit learning, researchers theorize that different
portions of the brain may be responsible for implicit learning than for the more
explicit learning normally tapped by measures of g (see Seger, 1994, for a thorough
review of this topic). If we are correct in making the connection from implicit
learning to tacit knowledge (a connection supported by the cognitive literature), and
Stemberg and his colleagues are correct in moving from tacit knowledge to practical
intelligence, then we can make the following statement with some degree of
conﬁdence. Because implicit learning and other cognitive functions are localized in
different areas of the brain, and because practical intelligence is the external
manifestation of implicit learning, practical intelligence may be separate from general
cognitive ability.

The debate between g-theorists and their opponents is a difﬁcult one to fully
grasp, and more difﬁcult still to grasp and summarize. However, even from this
brief overview it should be clear that, at the very least, practical intelligence has
something to add to notions of g in terms of prediction. It may represent an
extension of the construct of g, or it may be something completely separate. Given

this review of the literature, I will now turn my attention to the speciﬁc hypotheses of

45
my study.
Hypotheses

In formulating hypotheses that will facilitate a comparison of an SJT with a
measure of tacit knowledge, an approach similar to that used in construct validation
research is appropriate. One such method is suggested by Frederiksen (1986a), and is
of particular interest as it deals with the issues of construct similarity that must
necessarily lie at the heart of this study, since I seek to compare measures of tacit
knowledge and situational judgment in order to determine whether they represent
distinct constructs.

The primary concern of this paper is a determination of the degree of
similarity of the constructs underlying tacit knowledge and situational judgment
measures. As such, the ﬁrst hypothesis I will state is central to all further analyses.
If the situational judgment test and the measure of tacit knowledge are in fact
measuring the same things, then they will correlate strongly with one another.

Hypothesis 1: A strong positive correlation will exist between

the tacit knowledge measure and the situational
judgment test.

My literature review focused on areas in which similarities might arise;
following Frederiksen’s (1986a) lead, I propose to compare the SJT and the measure
of tacit knowledge to the same set of constructs, and observe whether they correlate
similarly with such external variables. To the extent that they do so, we may infer
that they measure the same underlying construct. Such external correlates are not

chosen randomly, but rather with consideration of the conceptual and empirical

literature as summarized above. A complete breakdown of the two measures, in

46

terms of their subdimensions and those constructs that should be related to each
subdirnension, can also be drawn from the earlier review of the literature. This
information is summarized in Figure 1 and is discussed in greater detail below.

In examining the breakdown of constructs underlying the practical intelligence
and situational judgment test headings, two clusters become evident. The first cluster
involves the Managing Others dimension from practical intelligence and the
Communication Skill and Interpersonal Skill dimensions from the SJT. We will refer
to this as the Interactive Cluster, because of its focus on other people and our
interactions with them. The second grouping of constructs includes the Managing
Self and Managing Tasks dimensions from practical intelligence and the Problem
Solving Skill dimension from the SJT; this one, we will refer to as the Active Cluster,
because of its focus on accomplishing various task-related and personal goals.

The differences between the Interactive and Active clusters become apparent as
we examine the indicators that will likely inform us about the elements of each. For
instance, the Interactive Cluster will likely include such indicators as extraversion,
communication competence, and Machiavellianism, which are found nowhere in the
Active Cluster. Similarly, the Active Cluster should include indicators such as
conscientiousness, openness, tolerance of ambiguity, learning style, and self-
regulation, none of which are treated as indicators of the Interactive Cluster.

A problem variable exists, though, in the form of experience. Of the six
constructs underlying the two tests, only Managing Self should be relatively unrelated

to experience. There exists, then, some overlap between the Active and Interactive

47

wEZom 82095 9—39 magi _ Allie

:55 9598 32538 no Oregon 05 use

€25 EconomEoonzm 8338 E was Ham macaw macaw—om 3 «Earn
.3 852%." Egg

hO ago—Oh.

      

  

 

 

 

 

 

 

 

.\ .g

e

 
    

coexconxm

E§_=o>£:32

 

 

Exm
3.8.088:—

 

    

 

 

 

 

.@ _zxm =o=3_§EEoU
.1

 

 

 

 

:3 means:

 

 

 

 

 

 

    

coigubxm

Smﬁaéag

48

clusters, in the form of experience. The solution to this problem is to break
experience down.

My original discussion of experience indicated that several facets of it will be
relevant to performance on these measures, and the test of experience I designed
reﬂects these dimensions. Two of the most important are educational experience and
work experience.

One of the competencies stressed in formal education, from the time we enter
pre—school until the time we ﬁnish graduate school, is the ability to interact well with
others. Early on we are taught the value of sharing and working together, and toward
the end of our student careers it is increasingly stressed that we must be competent to
discuss our chosen ﬁeld in front of large groups of people, and to defend our beliefs
and assertions in that regard. As such, it seems reasonable to expect that educational
experience should be related to the Interactive Cluster, while work experience should
be less so.

However, work experience should be much more related to the Active Cluster,
since no amount of schooling can fully prepare an individual for the challenges of
daily work. The ability to motivate and organize yourself (Managing Self), to
perform work-related tasks effectively (Managing Tasks), and to be thorough, driven,
and resourceful in your problem-solving efforts are all aspects of the Active Cluster,
and are all skills that are more likely to be gained from actually wppkmg than they are
to be gained from simply talking about work (as in early education). Applications of
the model in Figure 1 will be made throughout the remainder of the hypotheses.

First, consider a managerially-targeted SJT (as in Motowidlo et al., 1990) and

49
a tacit knowledge measure targeting the same group (as discussed in Stemberg et al.,
1995). Since both are so closely related to the content of the job for which they are
to be used relevant work experience should be related positively to both.

Hypothesis 2: The correlation of tacit knowledge with work
experience will be nonsigniﬁcantly different from
the correlation of situational judgment with work
experience.

However, as is reﬂected in Figure 1, such experience should be primarily
related to the elements of the Active Cluster, particularly Managing Tasks and
Problem Solving Skill, rather than to the elements of the Interactive Cluster. I would
then put forth hypothesis 2a.

Hypothesis 2a: Relevant work experience should correlate
directly only with the Managing Tasks dimension
of the tacit knowledge measure, and the Problem
Solving dimension of the SJT. These correlations
should be nonsigniﬁcantly different from one
another.

Similar to Hypothesis 2, positive relationships have been demonstrated
between educational experience and SJT performance (Motowidlo et al., 1990) and
tacit knowledge (Stemberg et al., 1995; Klaczynski, Laipple, and Jurden 1992),
which should be consistent across measures if they in fact tap the same underlying
construct. As with Hypothesis 2, a caveat must be added to this hypothesis. There is
little reason to expect that Educational Experience, with its emphasis on self-

motivation and self-organization, should be related to Active Cluster elements, but it

should be much more related to Interactive Cluster elements. Hence,

IivnethesLs;

Hypothesis 3a:

50

The correlation of the tacit knowledge measure as
a whole with Educational Experience will be
nonsigniﬁcantly different from the correlation of
situational judgment with Educational Experience.

Relevant educational experiences should correlate
directly only with the Managing Others dimension
of the tacit knowledge measure, and the
Communication and Interpersonal Skill
dimensions of the SJT. These correlations should
be nonsigniﬁcantly different from one another.

A great deal of attention has been paid to the difference between cognitive

ability and the skills tapped by both the SJT and the tacit knowledge measure. While

Motowidlo et al. (1990) did ﬁnd some relationship between self-report measures of

academic achievement and SJT performance, they also concluded that cognitive ability

did not covary signiﬁcantly with performance on the SJT. Similarly, Wagner and

Stemberg (1985) and Stemberg et al. (1993) have shown that individuals can perform

complex cognitive tasks even when traditional measures of cognitive ability would

predict that they would be incapable of doing so. Additionally, tacit knowledge adds

signiﬁcant variance beyond that of cognitive ability when predicting performance in a

simulation (Stemberg et al., 1995). Based on this, I present Hypothesis Four.

Hypothesis 4:

The correlation of tacit knowledge with cognitive
ability will be nonsigniﬁcantly different from the

correlation of situational judgment with cognitive
ability. Both correlations will be nonsigniﬁcantly
different from zero.

While the literature on SJTs is silent on the effects of age, the literature on

practical problem solving shows marked effects, with ability increasing until age 40-

50 and declining thereafter. If the two measures tap the same construct, then both

practical intelligence and situational judgment should correlate positively with age.

51

Hypothesis 5: The correlation of tacit knowledge with age will
be nonsigniﬁcantly different from the correlation
of situational judgment with age.

There is no reason to expect that demographic variables such as gender will
have any differential effect on either an SJT or a measure of tacit knowledge. While
subgroup differences may exist, they should be constant across measures if they tap
the same construct. The critical hypothesis, for our purposes, is this:

Hypothesis 6: The correlation of tacit knowledge with
demographic variables (e.g. , gender) will be
nonsigniﬁcantly different from the correlation of
situational judgment with demographic variables.

Motowidlo et al. (1990) demonstrated that both communication skill and
interpersonal skill were related to SJT performance in one or more of their samples.
Because of the manner in which tacit knowledge is likely gained, via implicit
learning, it is possible that both communication and interpersonal factors inﬂuence the
acquisition of tacit knowledge. The best way to compare these may be by examining
personality constructs such as extraversion and their relationship to SJT and tacit
knowledge measures. In addition, personality constructs such as tolerance for
ambiguity, conscientiousness, and openness to experience have been discussed with
regard to both situational judgment and tacit knowledge measures. More speciﬁc
information about the subdirnensions where each external correlate is relevant are
contained in Figure 1, and are reﬂected as follow. Hypotheses seven through twelve
ﬁrst present equality hypotheses with respect to the correlations tested, then specify

subhypotheses based on the breakdown of Active and Interactive Clusters, which are

reﬂected in Figure 1.

Hypothesis 7:

Hypothesis 7a:

Hypothesis 8:

HERMES—3a:

Hypothesis 9:

Hypothesis 9a:

Hypothesis 10:

52

The correlation of tacit knowledge with
sociability, as measured by extraversion, will be
nonsigniﬁcantly different from the correlation of
situational judgment with extraversion.

Extraversion should correlate directly only with
the dimension of Managing Others from the tacit
knowledge measure, and with the Interpersonal
Skill and Communication Skill dimensions of
situational judgment. These correlations should
be nonsigniﬁcantly different from one another.

The correlation of tacit knowledge with
communication competence will be
nonsigniﬁcantly different from the correlation of
situational judgment with communication
competence.

Communication competence should correlate
directly with only the Managing Others dimension
of the tacit knowledge measure, and with the
dimension of Communication Skill from the
situational judgment test. These correlations
should be nonsigniﬁcantly different from one
another.

The correlation of tacit knowledge with tolerance
of ambiguity will be nonsigniﬁcantly different
from the correlation of situational judgment with
tolerance of ambiguity.

Tolerance of ambiguity should correlate directly
with only the Managing Tasks dimension of the
tacit knowledge measure and with the Problem
Solving Skill dimension of the situational
judgment test. These correlations should be
nonsigniﬁcantly different from one another.

The correlation of tacit knowledge with openness
to experience will be nonsigniﬁcantly different
from the correlation of situational judgment with
openness to experience.

Hypothesis 10a:

Hypothesis 11:

Hypothesis 11a:

Hypothesis 12:

Hypothesis 12a:

53

Openness to experience should correlate directly
only with the Managing Tasks dimension of the
tacit knowledge measure and with the Problem
Solving Skill dimension of the situational
judgment test. These correlations should be
nonsigniﬁcantly different from one another.

The correlation of tacit knowledge with
conscientiousness will be nonsigniﬁcantly
different from the correlation of situational
judgment with conscientiousness.

Conscientiousness should correlate directly only
with the tacit knowledge dimension of Managing
Self and the situational judgment dimension of
Problem Solving Skill. These correlations should
be nonsigniﬁcantly different from one another.

The correlation of tacit knowledge with
Machiavellianism will be nonsigniﬁcantly
different from the correlation of situational
judgment with Machiavellianism.

Machiavellianism should correlate directly only
with the tacit knowledge dimension of Managing
Others and the situational judgment dimension of
Interpersonal Skill. These correlations should be
nonsigniﬁcantly different from one another.

The Managing Self dimension, as was noted earlier, reﬂects something which

seems to be absent in SJTs with its focus on self-motivation and self-organization. As

such, the next two hypotheses regarding this dimension will take a somewhat different

form from the preceding hypotheses.

Hypothesis 13:

The correlation of the tacit knowledge dimension of
Managing Self with self-regulation will be signiﬁcantly
higher than the correlation of the situational judgment
test with self-regulation.

Learning style is not a continuous variable as it will be measured for this

study. It exists along two dimensions, concrete versus abstract and reﬂection versus

54
action, and the results for an individual are reported in terms of which aspects of the
two dimensions describe them best. Repeated research (Romero et a1, 1992; Wolfe &
Kolb, 1984) has demonstrated that business majors are more concrete and more
action-oriented. Since business majors and those with similar personalities might
reasonably be expected to perform best on a test in the managerial domain (as both
the SJT and TK measure for this study are), we can move to the following
hypothesis. Again, though, note that there is little reason to expect a speciﬁc
relationship of learning style to any of the subdimensions of either measure other than
Managing Self.

Hypothesis 14: Individuals who score highly on the tacit knowledge
dimension of Managing Self are signiﬁcantly more likely
than those who score low on it to demonstrate an
"accommodator" learning style, high in concreteness and
action orientation.

A ﬁnal comparison is also possible, and is enabled through the application of
the principles of structural equations modelling. Hypotheses 2-14 deal with the
relationships among situational judgment, tacit knowledge, and various external
measures. Each of these individual comparisons is a relatively low-power test, but a
comparison of the general nomological networks surrounding situational judgment and
tacit knowledge, taken as a whole, is also possible. To do this, we can compare the
structural model for tacit knowledge and the external variables with the structural
model for situational judgment and the external variables using LISREL 8 (JOreskog
& SOrbom, 1993). As such, my ﬁnal hypothesis is as follows.

Hypothesis 15: The structural models of tacit knowledge and situational

judgment will exhibit nomological invariance; that is,
they will be nonsigniﬁcantly different.

55

With my hypotheses stated, it is now time to turn our attention to the methods

used to carry out this study.

Chapter 2

METHOD

Participants

Because of the nature of the hypotheses tested in this study, it was important
that a sufﬁciently large sample be obtained to accurately reﬂect differences among
observed correlations. To this end, 358 undergraduates from a large midwestern
university participated in this study. The logic underlying this choice of sample size
was based on a preliminary power analysis, the logic of which follows.

Cohen (1992) provides much of the information for the computation of this
ﬁgure. Power was assessed against the alternative hypothesis that differences between
Fisher 2 transformations of correlations was .30. (In Fisher Z terms, this is a
medium effect size as deﬁned by Cohen (1992)). I wanted to be able to detect if
Fisher z-values that differed as little as .30 were signiﬁcantly different. Assuming
that the correlations would cluster around Cohen’s (1992) medium effect size, I used
:12 = .435 and [,3 = .165 as estimates of correlations for the purposes of this power
analysis. Cohen (1992) provides power estimates for the test of differences in z
computed on independent samples, but not for correlations based on the same sample
as is true in the proposed study. Hence, two approaches to power could have been

utilized.

56

57

First, the power tables Cohen provides offered estimates for the difference
between independent gs. The test of difference between independent gs is less
powerful than the test of difference between dependent gs, as was done in this study,
so Cohen’s estimate of power for a medium effect size (a difference in Fisher’s z-
scores of .30) should serve as a lower bound for our estimate of power for this study.
Cohen’s (1992) recommendation for an g1 sufﬁcient to detect such a medium effect
size at Power = .80 is 177 subjects.

A second method of approaching power, which was technically an extension
and reapplication of Cohen’s methodology, involved an examination of Steiger’s
(1980) paper in which he directly addressed the issue of dependent correlations and
the signiﬁcance of their difference. The ﬁrst step in applying Steiger’s work was to
determine the standard error for our formula, which would ultimately be a
comparison of Fisher z-scores. Based on the use of Steiger’s formulas 3, 10, and 14,
I found that the standard error was

[lfx/(N—3)](1.039)
For an N of 177, our standard error would be equal to .0788. Based on this standard
error value, we can then apply Cohen’s methodology to determine the power for this
test, the probability that we would correctly reject the null hypothesis. I found that
the dependent test of differences was, as expected, much more powerful than the
independent test of differences. The power estimate based on this standard error is
actually .9645, a substantial increase in power over the other methodology.

In the interest of making this test as powerful as possible, however, I also

tested a second, less conventional effect size. Utilizing the same method I applied in

58

the previous paragraph, I tested the level of power the study could obtain if I
increased the sample size to n = 350 and decreased the effect size to be tested to a 2-
difference of .20. While this was not one of Cohen’s (1992) tabulated effect sizes, it
provided a more stringent test of whether a true difference between the correlations to
be compared actually existed. The power computations indicated that this test
(assuming gs of approximately .39 and .21) would have a power of .9474 to detect the
difference between the z-transformed correlations. Because my tests of signiﬁcance
all involve acceptance of the null, a high degree of power was deemed desirable. In
this way, the decision to recruit a minimum of 350 participants for this study was
made.

Ten participants were excluded from all analyses due to either incomplete data
or the presence of a distinct pattern (e. g., 180 straight 4’s on the response sheet) in
their responses. Of the 348 remaining, 134 were freshmen, 75 were sophomores, 67
juniors and 72 seniors. 304 of the participants were Caucasian, 20 were African-
American, and the remainder were divided among several other groups, none of
which made up a signiﬁcant portion of the sample. The participants were primarily
between the ages of 17 and 22, and 59% (204 of 348) were female.

Procedure

Participants were recruited from the introductory psychology subject pool and
by visiting classes. Groups of 10-60 students took part at the same time. Upon
arrival at the room in which the experiment took place, participants were given a
packet containing all the tests and forms for the study. They were then asked to have

a seat, put the packet beneath their chairs, and wait until everyone had arrived. No

59

later than ﬁve minutes past the scheduled start time, they were instructed to remove
the UCRIHS-approved consent form (see Appendix A) from their packet, read it, sign
and date it, and return it to the rear of their folders.

Once all students read and signed the consent form, the Wonderlic Personnel
Test was administered. This test was administered ﬁrst due to its timed nature, which
would have made it logistically difﬁcult to administer at any other time. After the
twelve-minute Wonderlic session, they were then instructed to complete the N EO-
FFI. Pilot testing indicated that eight minutes was sufﬁcient for most participants to
complete the NEO-FFI; hence, after eight minutes of working on the test, participants
were given the ﬁnal set of instructions for the major packet, which contained all of
the remaining test materials for the experiment. It was emphasized at all points
during the presentation that the tests the participants were taking were the same types
of tests that were used to hire individuals in the "real world, " and that they should
treat the tests as they would actual selection instruments.
Measures

Cognitive Abilig. The Wonderlic Personnel Test, a measure of general
cognitive ability, was chosen because of its wide application in business and industry
as well as its demonstrated psychometric adequacy. The user’s manual for the
Wonderlic (1992) offers predictive validities ranging as high as .63, with reliability
estimates ranging from .73 to .95, depending on the speciﬁc type of reliability being
estimated. The test reﬂects multiple dimensions of cognitive functioning, including
verbal, quantitative, and other skills, and as such represents a thorough test of the

domain of cognitive ability. The test was distributed face-down. The instructions for

60

the completion of the Wonderlic were read, and students were given twelve minutes
to complete as much of the test as they could, per the test publisher’s speciﬁcations.

At the conclusion of the twelve minutes allotted for the Wonderlic, participants
were instructed to return it to the rear of their packet. The mean score on the
Wonderlic was 23.48 (out of a possible 50), with a standard deviation of ﬁve points.
Means and standard deviations for all scales are provided in Table 3.

Personalig. The NEO-PI (Costa & McCrae, 1995; Mche & Costa, 1987),
a measure of the Big 5 personality factors, was used to measure several of the
personality correlates (extraversion, conscientiousness, and openness to experience)
hypothesized to be related to tacit knowledge and situational judgment test
performance. The factors measured by this scale (neuroticism and agreeableness
round out the "Big 5") had been noted in the literature even prior to Norman’s (1963)
attempt to taxonomize personality attributes. Their ubiquity over time, in addition to
the evidence collected by Costa, McCrae, and others for their validity, allowed
conﬁdence in the NEO-PI as a measure.

Tolerance of ambiguity also has a long research history underlying its use
(Frenkel-Brunswick, 1948). However, no single best scale presented itself here, since
many such measures produce validity data that may be classiﬁed as passable at their
best, and as poor with an unfortunate frequency (Furnham, 1994). Because of this, I
reviewed some of the recent literature in I/O and discovered a tolerance of ambiguity
measure that dealt with the kinds of work-related issues that dominated the selection
of other measures, such as the Tacit Knowledge and Situational Judgment tests that

will be discussed shortly. This seven-item scale, from Major (1990), has

liable}: Descriptive Statistics for the Scales

61

 

 

Scale Mean SD
Wonderlic 23.48 5.00
Extraversion 32.05 6.07
Conscientiousness 32.29 7.04
Openness to Experience 29.74 6.46
Tacit Knowledge (positive) 62.43 8.31
Managing Self (positive) 38.19 5.57
Managing Career (positive) 13.88 2.70
Managing Others (positive) 10.36 1.94
Tolerance of Ambiguity 23.47 7.53
Communication Skill 2.42 2.04
Machiavellianism 17.45 2.27
Situational Judgment 24.36 10.58
Work Experience 46.82 10.09
Educational Experience 32.47 10.20
Community Experience 24.19 13.76
Self-regulation 22.68 3.46
Concreteness 22.59 6.01
Action Orientation 25.83 5.56

 

CC
C0

re].

Prob
Often

(Mudr

62

demonstrated a reliability of .77 in previous research. In this study, a = .74. The
intercorrelations of all the scales, as well as the a values for each (where applicable)
can be found in Table 4. All unpublished and/or non-proprietary scales are included
in the appendices to this report; the tolerance of ambiguity measure is included in
Appendix D.

The assessment of communication skill within the context of a battery of tests
is difﬁcult, because the measure will necessarily be a self-report and therefore be
subject to potential response bias in the form of positive self-presentation. Few such
measures seem to exist, and most focus on public speaking (e. g., McCroskey, 1970).
One of the better measures available, which focuses on general communication skills
in addition to public speaking, seems to be Rubin’s (1986) Communication
Competence Self-Report (CCSR). The CCSR is a 19—item scale in which subjects are
asked to respond to each item on a ﬁve-point scale ranging from "Always" to
"Never." Evidence for the validity of this scale has been provided by its high
positive correlation with instructors’ ratings of the student respondents’
communication skills (Rubin, 1986). Previous research indicated that the scale had a
coefﬁcient a reliability of .87. In this sample or was .81, still an acceptable level of
reliability. This scale is included in Appendix E.

The primary measure of Machiavellianism available currently, and the standard
by which many others are gauged, is the Mach IV (Zook, 1985). However, one
problem that has been noted in the literature is that results based on this scale are
often confounded by its relationship to locus of control (speciﬁcally external)

(Mudrack & Mason, 1995). Recent work has produced a brief, ten-item research

v..u2_...:...> A325. m:.:=< 25.:Eetcb 3cm ..quzN.

 

63

.888 05 8.. 8332.9. 933 Egg-coco
8a .58ch 05 E «on?» :26— S. 05 8 835:»? 2m 9588 3.8 33 E 8252.8 :30. no. 05 8 88555 8a 090 22. E mecca—otou ”8oz
Axe-0:588 035V

 

:8.- 88- 88.- 88. 88. 88.- 28.- 88.- :8. 82. 82. 88.- 88. :2. 28::
Em_:a_=o>anm_2

5:3 88. 88. 88.- 88. 22. a8: 88. 82. 82. 82. 82. 2.8.- 88. 3.5 88.8
528238.50

2.85 88. 88.. 82. 88. 82.- 8:.- 82. 8:. :8.- 88: 88. $8. 883:2

..o 3:823.

Ti 8:.- 8....- 88.- 82..- 82: 88. 88. 88.- 82.- 88. 88.- 82

Ti :8.- 88; 88.. $8. 88.- 88.- R2. 88. :8.- 2:. 58

Ti 82. 88.- 88.- 28. 88. 88. 28.- 8:. 22. 8<

:85 88. 88‘. 88. 82. 88. S8. 88. 82. 25:88

82:0 maﬁa—82

$83 88‘. 88.- 888. 88. 88k. 5:.- 88. 98283

8080 @3982

£25 88.- 88. 88.- 88. 88.- 88. 98:88

how 8:3an

8:3 88. 82. 88.- 88. 88. :8 8:85

. w=_>_om E292;

88: 8:. 2.8. 98.. «in. 28.085

cocmoEaﬁc-SU

:88 88.- 88. $8. :88:

:65 328.582:—

225 8.2.- 88. $8888
032265. :3...

288 38. GEE-F
Owen—:65. gun-r

£85 $838555
-35.. 3:28.55

 

9:00 ESE. coax xom ow< 502 .80—2 tom—2 .35 mm .350 ..xmﬁ: +vC- v:- Ham

83$8> 88w wcoE< meets—8.80 33— Mining

8002800 0380

 

«V3.- 28... mme. ”NS.- 33. n0:- ameﬁ :2. 3.:- 22. 3.2. “RN. 2.8. 3.3. 3.8000
80:82:23.:00

3:. 38.- 02.0. 3:. 030.- 83.- N20. N87 260. :2. 2::- 020: 3:. VMNN. A0083
80:03:88.0.

R2. 3.00.. 3:. 003. 2.3. vmoo. $3..- Ebc... 0000. 0.80. 08—. 003... .08.- 00:. A.:0:OV.._00xm
0: 808000

Mn:- 33. 02:. 09:. mag.- NmE. 3:. ~08. _NS. 030. .22. v":- 037 000m. 25me
8803be

0.0.3.. 00$... 38. ‘MVN. 020.- N80. :8. 300. M02... ”30.. Nm:.. SS. 38.. 305. A8020
8228802

33. NS:- RMN. NNR: 38.- $3. 035.- «0:... 2:0.- wmmov ammo. ~08.- 008: $5.- 33000
8:2 35:80

3:.- 0mm0. 3.8. $2.. 23. cm:- whnc. Q:- 008. «20. $5.. 3.3. 9.8. $00. 828$
8:00:25 :28<

:20. 0:3.- omocv RR: 33. ~30. N2:- 28. 300. 2.00.- vows. 0000. Rae. 08.0. 600000
805.0880

N0:- mvmcr 38. ”.32.. 2.3. .000. .02. 50.. PS. 300. SS. mnnﬁ «2.0. 300. 33%;:
25 05.53

M28.- 300. mnmor 020. :8.- NR-N. 3.:- C:. 003. 3.:- N‘aﬁ 0.0:. chr :3. $02.9
:0_8_=w0~_.-=0m

5:. 080.- 5:.- RS. 28.. Nﬂo. 3.3.- $00. .08.- 5:.- mmmo. 38. $00. 33.. €06.00 00:28
.08: 2:08:80

22.n- emuﬁ memo. _NS. 2.8.- $00. mmmo... :2. ~— wc. S _ —. «.38. 2.8. 9.2.. mmS. Exm 09 00:28
.96 3:280:3—

MgN. 38. name. :3. 3.3. memo. nova. ammo. 0:5. 0000. ”N00. 9.00. _N:.- 030. 306.30
3:28qu 283

 

9:00 830:. 0001 x0m 0w< 502 .80—2 .:0m$_ ..xm mm ..xm.0 ._V_m.::_ +VE. 0:. ...nm

80:50:03 maﬁa-H

 

‘

~P..=:.::c3 V2.2;

 

 

 

65

Tat-v 2K5 @81- ‘3»: 3.3-... 83.- ‘ée‘. .22.. naa‘. an“. 38. eng- 52. ammo. 8053353800

A ..... v :8. 2%. 88.. 8:: 88.- 88.. 82:.- 82. 88. 88. 88. 2:. 3293823
A-i-v 88. 88. 88. $2.. 2:..- 88.- :8. 88. 28. 8:. £8. 85.26

2 82.:qu

Ti 8%.. 88.- 88. 8:- 88. 88. 8:. 8:: 8:: E... 52388

Ti 88.- 82.. 88.- 88.- 88: 88.- 82. 88.- 88.- 53232

A ..... v 88. 88. 28. 88.- 88.- 88.- 28. 3:.- b=s< 963ch

A888 2:. 88.. $8. 88. 5:. 82. 32.- 83585 8:2

885 88. :8. 88.- 88. $8.- 82.. 3222050
885 22. 88.- 82. .88. 2%.. 23 3.53
385 82. 82. EN. 88. 58.82.83

38$ 28. 22. 88. acacaxm

DEE—goo

:85 88. 88.- 85:38

_acocaoaum

8.3 m8..- acotaxm v.53

Ammo: EﬂﬁEoﬁES—z

 

.omcou oohw< .550 >955 .502 $.on :o:u< .250 038mg. may; axmd nxmﬁm 9a.? :82

63883 338

66

measure of Machiavellianism that lacks this potential confound. Coefﬁcient a
reliability estimates for this scale range from .68 to .84 (depending on the sample),
but generally do fall within the acceptable range. For this study, a was .76, the
midpoint of the range noted in the literature. This scale is included in Appendix F.
Demographic information. This includes information about years of
educational experience, as well as more traditional demographic measures. This
information was collected in order to determine whether the two target measures, tacit
knowledge and situational judgment, correlated similarly with demographic data.
Please see Appendix C for a copy of the items used to collect demographic data.
Tacit knowledge. Tacit knowledge was assessed using Wagner and
Stemberg’s (1985) measure of tacit knowledge for managers. This self-scoring
measure required respondents to rate each of several response options to a situation in
terms of its quality. A measure speciﬁc to managerial issues was chosen for several
reasons, the primary one being that tacit knowledge, by its nature, is domain-speciﬁc.
In reviewing the literature on this topic, it became clear that very few tacit knowledge
measures were in existence, and after careful consideration of the content of those
available, I decided that most people would have at least a modicum of experience
with management in one form or another. Moreover, Wagner and Stemberg use their

tacit knowledge measure to select managers from a pool of participants similar to the

 

one recruited for this study.
The tacit knowledge measure of Wagner and Stemberg breaks tacit knowledge
down into three subdomains, which were reviewed earlier. The full measure presents

twelve scenarios, each followed by six to twenty options which respondents must rate

67

on a seven-point scale, with one representing an extremely poor response to the
situation, and seven representing an extremely good response. Of the 166 ratings
participants made, 35 of the items were scored as reflecting either Managing Self,
Managing Tasks/Career, or Managing Others. Although the vast majority of the
items (131) were not used in any analyses, all items were presented based on the
developer’s belief that only using the keyed items might result in the measurement of
something other than tacit knowledge (Wagner, personal communication). Four items
were noted in the test manual as distinguishing between good and poor performers on
the test, but no theoretical reason existed for their inclusion and they were therefore
not considered in any analyses. Of the 35 items used, 16 were keyed to the
Managing Self scale, 15 to the Managing Tasks/Career scale, and 4 to the Managing
Others scale.

Initial reliability estimates for the three subscales and for the overall tacit
knowledge test were not encouraging. Coefﬁcient a for the overall measure was
.3931, for Managing Self .0731, for Managing Tasks/Career .5730, and for
Managing Others -. 1073. A factor analysis indicated that a substantial portion of the
problem lay in the fact that some of the items within each scale were reverse-coded,
and that some of the factors actually reﬂected coding differences. This phenomenon
has been noted previously in the literature (Schmitt & Stults, 1985). In this instance,
it appears that a lack of motivation on the part of participants may have led them to
be lenient in their answers, giving every option they were presented a general positive
response (something of a "Yeah, I guess that sounds good" phenomenon). In this

way, when part of the items were reverse-coded, all of those items would cluster

68

together (as they would all be low scores, since everything was previously at the high
end), and the items that were not reverse-scored would remain clustered together at
the top of the scale.

Not surprisingly, when the positively-coded and negatively—coded items were
split out from one another, more reasonable results were obtained. These results are
summarized in Table 5. The reader will note that the alphas obtained for the six
subscales were still widely disparate, but this variation correlates almost perfectly
with the number of items in each scale. The last column of the table, labelled "New
(1," reﬂects the reliability coefﬁcient that would have likely been obtained, had each
of the six subscales contained the same number of items (12) as the largest subscale.
These values were computed using the Spearman-Brown Prophecy Formula (Nunnally
& Bernstein, 1994). Based on this analysis, it appears that the scales are roughly
equivalent to one another, in terms of their internal consistency reliability.
Unfortunately, in an operational sense all analyses will necessarily be based upon the
number of items actually included in the scale, and this must be kept in mind when
interpreting the number and quality of relationships found among variables.

Given the values for a that are reported above, it then became important to
decide how to analyze the correlational hypotheses put forth in this study. Because of
the problems that would be inherent in any attempt to use all of the tacit knowledge
items (low or negative internal consistency reliabilities), I opted to use only the
positively-scored items in later analyses, since these items most likely reﬂect the true
intentions of study participants, where the negatively-scored items might not. The

correlations between the overall TK scale and both subscales were signiﬁcant, with

69
Iahlej: Reliability coefﬁcients for tacit knowledge scales

 

 

Scale Original or # of items New a'
Tacit Knowledge (overall) .3931 35 «-
Tacit Knowledge (positive) .7191 13 «-
Tacit Knowledge (negative) .8115 22 ---
Managing Self (overall) .0731 16 --
Managing Self (positive) .6260 8 .7152
Managing Self (negative) .6429 8 .7298
Managing Career (overall) .5730 15 -..
Managing Career (positive) .2673 3 .5934
Managing Career (negative) .7538 12 ~-
Managing Others (overall) -. 1073 4 --
Managing Others (positive) .2822 2 .7023
Managing Others (negative) .2303 2 .6422

 

' Values in the column labelled "New or" reﬂect the coefficient alpha reliability that subscale would have it
were composed of twelve items.

70

the correlation between overall tacit knowledge and negative items being .8227, and
that between overall and positive items being -. 1359. The difference in these
correlations, in terms of magnitude and direction, is explained by the fact that the
bulk of the tacit knowledge items (22 of 35) were negatively-scored; hence, we would
expect that the items scored in the predominant direction would correlate more highly
with the overall measure than would the items scored in the less dominant direction.
The fact that the negative items correlated -.6889 with the positive items would again
seem to indicate that the sets of variables are relatively redundant, thereby justifying
our use of only one of the two sets of items. In addition, alphas for the positive
subscales were comparable to those for the negatively-scored scales. The full tacit
knowledge measure is included in Appendix G.

Situational judgment. The situational judgment test utilized was developed as
part of an effort to restructure the selection system for a large federal organization. It
was developed utilizing the same stringent procedure outlined by Motowidlo et al.
(1990), and a full reiteration of that procedure would be redundant at this point.
Some information, though, bears repeating.

As in the earlier research, each question was of two parts: Following the
presentation of a given situation, the respondent was asked to indicate ﬁrst which
response he or she would most likely make, and second which response he or she
would least likely make. The maximum score an individual can get on a situational
item is +2 (if they identify one of the "best" options as their "most likely” response,
and one of the "worst" options as "least likely"), and the minimum score would be -2

(if they do the reverse).

71

The available pool of items constituted more than simply managerial
effectiveness, so those items reﬂecting skills with apparent utility for success in
management were selected based on a careful consideration of their content and
grouped into a 30-item scale. Coefﬁcient a reliability for this measure was .7003.

In addition, some of the hypotheses for this study made use of Motowidlo et
a1. ’5 (1990) trichotomization of situational judgment, in terms of Interpersonal Skills,
Communication Skills, and Problem-Solving Skills. Because no a m structure
existed for the items included in this situational judgment test by which the items
could be broken down into these categories, a group of local subject matter experts
was recruited to rate the items. Copies of the situational judgment test were provided
to twenty graduate students in Industrial/Organizational psychology, based on their
presumed familiarity with the literature on jobs and the dimensionality of work skills.
Sixteen SJTs were returned, and the data were analyzed. Any question for which
two-thirds of the respondents agreed on the skill dimension tapped was retained for
use in the hypothesis tests regarding the subscales of the SJT. A total of 16 questions
were retained in this manner (6 Interpersonal, 3 Communication, and 7 Problem-
Solving). Coefﬁcient a reliabilities for these subscales are low (.2601, .1988, and
.2782 for Interpersonal, Communication, and Problem-Solving, respectively), but as
has been pointed out by previous authors (Clause, Mullins, Nee, Pulakos, & Schmitt,
1996), this is likely due to the fact that the items that make up a situational judgment
test are multidimensional, and not likely to produce high estimates of internal
consistency reliability. As such, coefﬁcient a may be an inappropriate estimate of

reliability for this measure (Cronbach, 1947, 1951). See Appendix H for the form of

72

the situational judgment test used in this study.

Experience. A measure was constructed to tap into those aspects of
experience likely to inﬂuence performance on the situational judgment and tacit
knowledge tests. Since the theoretical outline provided in the introduction to this
paper postulated that implicit learning underlay tacit knowledge, and that tacit
knowledge and situational judgment measured the same skill/knowledge base, careful
consideration was given to those situations in which individuals might experience
implicit learning that would better enable them to perform on management-related
tacit knowledge and situational judgment tests. These content domains included work,
school, and community experience with respect to leadership, as well as involvement
in various activities and organizations. The full experience measure had a coefﬁcient
a of .8983, while the subscales of job experience, school experience, and community
experience had a values of .8193, .7997, and .9052, respectively. Equally important,
the intercorrelations among the experience scales were low (see Table 4), indicating
that the three scales were measuring conceptually distinct facets of experience. Please
see Appendix I for a copy of this measure.

Self-regulation. A self-regulation measure was adapted from the work of
Ashford and Tsui (1991), to reﬂect both a somewhat traditional approach to self-
regulation (attention to monitoring those cues the authors deﬁned as crucial to
performing managerial jobs well) as well as a recognition of the importance of
negative feedback-seeking behaviors. Because Ashford and Tsui’s measure reﬂected a
more social approach to self-regulation than is common in much of the self-regulation

literature, items were re-written to remove many of the social aspects while retaining

73

the same basic content that Ashford and Tsui (1991) emphasized. The resultant ﬁve-
item measure had an internal consistency reliability of .7432. Please see Appendix J
for a copy of this measure.

Learning Sgle. The reader may recall that learning style, as it is most
commonly measured, exists along two dimensions. The ﬁrst is concrete versus
abstract; the second is reﬂection versus action. One version of this measure was
published by Romero et al. (1992), and demonstrated coefﬁcient a reliabilities around
.80 for each dimension. Within the conﬁnes of this experiment, this learning style
measure possessed an overall a of .7832, while a for the concreteness scale was
.7669 and a for the action orientation scale was .6608. The reason for the drop in
the internal consistency reliability for the action orientation subscale is unclear. The
correlation between these two scales is .4113 (see Table 4), indicating that they
represent fairly distinct dimensions of learning style. This measure is reproduced in

Appendix K.

Chapter 3

RESULTS

Table 4 contains the intercorrelations of the primary variables examined in this
study. This table formed the basis for many of the early hypothesis tests, particularly
those that hypothesized differences (or lacks thereof) among correlations. My
discussion of the hypotheses for this study will proceed in the order in which the
hypotheses were presented; for the ﬁrst hypothesis test dealing with the differences
among correlations, I will specify all steps of the process used in comparing the
correlations and testing for their difference. For all further hypotheses of this form, I
will refer the reader to Table 6, where the speciﬁcs of the hypothesis tests are
contained, while providing only the relevant details for each.

The ﬁrst hypothesis suggested that a strong positive correlation would exist
between scores on the tacit knowledge test and scores on the situational judgment test.
Using only the positively-keyed tacit knowledge items, as was discussed above, the
correlation between tacit knowledge and SJT was actually .0844, a nonsigniﬁcant
result. Correcting for attenuation due to unreliability in both measures (which may
not be wholly appropriate, given that both are multidimensional (Schmitt, Rogers,
Mullins, & Clause, in progress), but is potentially informative here) gives us a

correlation of .0974, which is only a slight improvement. The absolute size of both

74

75

 

 

Iahleﬁ: Test for differences among correlations of indicators with the Situational Judgment
Test and positively-keyed Tacit Knowledge items
Indicator rs, rm raw diff zs, ch Hyp. t
(rs,-rTK) test

Age .1412 (.1687) -.0110 (-.0130) .1522 .1421 -.0110 H5 1.53
Sex .1117 (.1335) .0907 (.1070) .0210 .1249 .0909 H6 .34
Race -.0321 (-.0384) -.1384 (-.1632) .1063 -.0321 -.1393 1.07
SES .0768 (.0918) -.0731 (-.0862) .1499 .0770 -.0732 1.49
Region -.0201 (-.0240) -.0245 (-.0289) .0044 -.0201 -.0245 .04
Tol.Amb. .0313 (.0434) -.1204 (-.l649) .1517 .0313 -.1210 H9 1.52
Comm. Comp. .2574 (.3415) .1635 (.2141) .0939 .2633 .1650 H8 1.02
Mach .1911 (.2616) -.0762 (-.1029) .2673 .1935 -.0763 H12 2.77
Work Exp. .0956 (.1262) .0943 (.1229) .0013 .0959 .0946 H2 .01
Ed. Exp. .1023 (.1367) .0875 (.1154) .0148 .1027 .0877 H3 .15
Comm. Exp. -.0795 (-.0999) .0329 (.0408) .1124 -.0797 .0329 1.11
Self-Reg. .2613 (.3622) .1578 (.2159) .1048 .2675 .1578 1.14
Learn. Style .0778 (.1051) .1235 (.1646) .0457 .0780 .1241 .46
Concrete .0439 (.0599) .0669 (.0901) .0230 .0439 .0670 .23
Action .0669 (.0983) .1449 (.2102) .0780 .0670 .1459 .79
Cog. Ability -.0167 (-.0200) -.0908 (-.1071) .0741 -.0167 -.0911 H4 .73
Neuroticism -.1667 (-.1992) .0712 (.0840) .2379 -.l683 .0713 2.43
Extraversion .2009 (.2401) .1124 (.1325) .0885 .2037 .1129 H7 .92
Openness .1106 (.1322) -.0506 (-.0597) .1612 .1111 -.0506 H10 1.61
Agreeableness .2234 (.2670) -.0830 (-.0979) .3064 .2272 -.0832 3.23
Conscientious .2639 (.3152) .2215 (.2612) .0424 .2703 .2252 H11 .47

 

Note: Results in BOLD type indicate p < .05; ﬁgures in parentheses are correlations corrected for
unreliability; Comm. Comp. = Communication Competence; Tol. Amb. = Tolerance of Ambiguity; Mach =

Machiavellianism; Work Exp. = Work Experience; Ed. Exp. = Educational Experience; Comm. Exp. =
Community Experience; Self-Reg. = Self-regulation; Learn. Style = Learning Style; Concrete =
Concreteness; Action = Action Orientation; Cog. Ability = Cognitive Ability

76

the observed and corrected correlations provides no support for the notion that these
are measures of similar constructs. Hypothesis 1 was therefore not supported.

Hypotheses 2 and 2a dealt with the relation of the TK and SJT measures with
the measure of work experience used in this study. Speciﬁcally, the correlation of
tacit knowledge with work experience was hypothesized to be nonsigniﬁcantly
different from the correlation of situational judgment with work experience
(Hypothesis 2). To test this hypothesis, as well as the other hypotheses of this form,
a two-step process was followed. First, the correlations to be compared were
transformed to Fisher Z-scores. The Z—transformed correlations were then used in the
following formula, from the work of Steiger (1980):

t = __([12_-_l'13)[(N " l)(l + 133)}5
[2(n - l/n - 3)|R| + rm2(l - r23)3]'5

where |R| = (1 - r122 - r132 - r232 + 2rurl3rz3) and is the determinant of the
correlation matrix of the coefﬁcients being tested and rm = .5(r12 + r13). r12 and r13
are the Z-transformed correlations between the external variable (here, work
experience) and the two focal tests, and r23 is the correlation between tacit knowledge
and situational judgment. Plugging in the Z-transformed correlations (rs,_,,,o,k = .0959
(.1262), rm,” = .0946 (.1229), l’smc = .0691 (0974)), we obtain a t value of .01,
which is obviously non-signiﬁcant. The correlations of the two focal measures with
work experience, then, are not signiﬁcantly different, so that the data do not support
the rejection of Hypothesis 2. This phrasing, while awkward, is most correct and in
line with the logic of null hypothesis testing. Results in accord with the hypotheses

can only be "not rejected," they can never be "supported" in any true sense. An

77

additional note that bears mentioning is that statistical tests were conducted using raw
correlations. But because this is essentially a construct validity study, correlations
corrected for unreliability are also reported in parentheses following each raw
correlation.

Raw correlations, the correlations corrected for attenuation due to unreliability,
the raw difference between the correlations, the values of the Z-transformed
correlations, and the ﬁnal t-value for each test of signiﬁcance are presented in Table
6. The reader will note that of the comparisons made between correlations, the
signiﬁcance test only detected a difference across measures for three of the twenty-
one pairs of correlations tested: Machiavellianism, Neuroticism, and Agreeableness.
Two of these were not the object of any speciﬁc hypotheses, so their difference is
only remarked upon in the event that future researchers see the need to understand
how personality characteristics such as neuroticism and agreeableness relate to
performance on these types of measures. It should also be noted that three of the
hypotheses postulated differences among correlations (either directly or indirectly) that
failed to materialize for the self-regulation, concreteness, and action orientation
variables.

Hypothesis 2a went a step further, by specifying which subscales of the two
focal measures should be most strongly related to work experience. It was
hypothesized that work experience should be relevant to the Managing Others and
Managing Tasks/Career subscales of the TK test, and to the Interpersonal Skill and
Problem Solving Skill dimensions of the SJT. A review of the correlation matrix in

Table 4 demonstrates little support for this hypothesis. Work experience was not

78

signiﬁcantly correlated with any of the subscales from either focal measure. Of the
SJT subscales, in fact, it correlated most highly with the Managing Self scale (r =
.0929 (. 1214)), which it was not hypothesized to be related to, while correlating
.0845 (.1114) and .0405 (.0581) with Managing Others and Managing Tasks/Career,
respectively. Similarly, none of the SJT subscales correlated signiﬁcantly with the
Work Experience measure. However, the dimensions that were hypothesized to be
related to the measure correlated with it more strongly than the one that was not
hypothesized to do so (Interpersonal Skill and Problem-Solving Skill correlated .0628
(.1360) and .0816 (.1709), respectively, with Work Experience, while
Communication Skill correlated .0064 (.0159) with the measure). Again, then, weak
support was found for Hypothesis 2a.

Hypotheses 3 and 3a took the same form as did 2 and 2a, but here the focal
variable switched from Work Experience to those Educational Experiences that might
impact managerial ability (clubs, leadership positions, and so forth). The difference
between the correlations again was non-signiﬁcant (.1023 (.1367) compared to .0875
(.1154)), thus Hypothesis 3 could not be rejected. As with Hypothesis 2a, Hypothesis
3a had only weak support; it was hypothesized that Educational Experience should be
primarily related to Managing Others, from the TK test (note that, from this point
forward, references to the tacit knowledge test simply as "TK" refer to the positively-
coded items chosen for inclusion in the analyses rather than the entire test), but
instead it was most strongly related to Managing Self (_l_' = .1312 (.1735), p < .05);
the correlation of Educational Experience with Managing Others was .0453 (.0604; p

> .05). This ﬁnding is not terribly surprising, given the college-student sample used

79

in this research. For such a sample, the impact of education would still be in
proximal domains -- how to allocate time and get along with people -- rather than
those distal outcomes, such as actually applying their educational experiences to
manage others.

On the other side of the model, Educational Experience was hypothesized to
be most strongly related to the Communication and Interpersonal Skill aspects of SJT
performance. The strongest correlate of Educational Experience of the three was, in
fact, Communication Skill (1, = .1117 (.2801), p < .05), but its correlation with
Interpersonal Skill was only .0843 (.1848), while the correlation with Problem-
Solving Skill was similar (.0812 (.1721)). None of these correlations are signiﬁcantly
different from one another. Thus Hypothesis 3a, like Hypothesis 2a, received only
weak support.

One of the important concerns with any study offering a new predictor
measure is the distinctiveness of the predictor from traditional cognitive ability tests.
In this study, the literature from both the tacit knowledge and situational judgment
domains suggested that something other than cognitive ability, as it is traditionally
measured, was being tapped. It was also hypothesized (Hypothesis 4) that due to
underlying construct similarity, the correlation of the tacit knowledge measure with a
test of cognitive ability would be nonsigniﬁcantly different from the correlation of the
SJT with cognitive ability, and that both would be nonsigniﬁcantly different from
zero. As Table 6 shows, this hypothesis was not rejected. The correlation of
situational judgment with cognitive ability (as measured by the Wonderlic Personnel

Test) was nonsigniﬁcant (r = -.0167 (-.0200)), as was the correlation of tacit

80
knowledge with Wonderlic score (_r = -.0908 (-.1071)). The difference between these

correlations was nonsigniﬁcant.

Part of the nomological network surrounding any test is the impact of
demographic variables on test performance. Hypotheses 5 and 6 dealt with
demographic variables, speciﬁcally age (Hypothesis 5) and gender (Hypothesis 6).
Hypothesis 5 was not rejected, as a nonsigniﬁcant difference existed between the
correlation of age with the SJT (r = .1412 (.1687)) and that of age with TK (r =
-.0110 (-.0130)). Hypothesis 6 was also not rejected, as the correlation of gender
with the SJT (r = .1117 (.1335)) was nonsigniﬁcantly different from that of gender
with TK (r = .0907 (. 1020)). Decent support, then, was found for that portion of the
nomological network dealing with demographic variables.

The remainder of the correlational hypotheses took on the same form as
Hypotheses 2 and 2a. For each, the similarity of the correlations between each of the
focal measures and the external measures were examined, and then the model
specifying the points within each measure where the external measures should be
relevant was examined.

Hypotheses 7 through 12a dealt with personality variables. Hypothesis 7 put
forth that no difference should exist between the correlation of Extraversion (as a
proxy for sociability) with TK and its correlation with SJT performance. The data
failed to support the rejection of this hypothesis. Hypothesis 7a speciﬁed that
Extraversion should be most relevant to the Managing Others domain of the Tacit
Knowledge test, and the Interpersonal and Communication Skill domains of

Situational Judgment. Partial support was obtained for this hypothesis, as

81

Extraversion correlated most highly with the Interpersonal Skill and Communication
Skill dimensions of the SJT (.1915 (.3574‘) and .0959 (.2151), respectively, while its
correlation with Problem Solving Skill was .0721 (.1367); again, though, none of the
differences among these correlations were signiﬁcantly different from one another at
the .05 level). However, the differences among the correlations of Extraversion with
the three Tacit Knowledge subscales were minimal (13,”qu = .1101 (.1429), roam
= .1032 (.1231), L9,“ = .0842 (.0996)), indicating roughly equivalent importance of
Extraversion to performance on all of the subtests. This may call into question the
distinctiveness of the scales, as deﬁned by their authors.

Hypothesis 8 dealt with the relationship of Communication Competence to both
focal measures. Failure to reject this hypothesis resulted from the non-signiﬁcant
difference between these two correlations. While the correlation of the SJT with
Communication Competence was slightly higher than that for TK (.2574 (.3415) as
opposed to .1578 (.2141)), both are relatively strong positive correlations.

Hypothesis 8a again found mixed support; it was hypothesized that Communication
Competence would be most relevant to the Managing Others dimension of the Tacit
Knowledge test and the Communication Skill dimension of the SJT. While the
strongest correlations for each of the sets of subscales were for those subscales named
in the hypothesis, there was not much variance across subscales within each test.

That is, none of the correlations differed signiﬁcantly from one another. For the SJT,
correlations of the three subscales with the Communication Competence measure

‘ Corrected correlations involving the ﬁve NEO-FFI dimensions, age, sex,

race, and cognitive ability are corrected using only one reliability estimate as only
scale scores or single items were used for those variables.

82

ranged only from .1552 (.3267) up to .1700 (.4233), suggesting relatively equal
importance of Communication Competence for all items within the SJT. While the
range of correlations was a little broader for TK subscales (.1039 (.1497) to .1519
(2012)), it again appears that Communication Competence may be one of the major
factors underlying performance on the test.

Hypotheses 9 and 9a dealt with Tolerance of Ambiguity in the same manner as
the hypotheses above. Hypothesis 9 (no difference between the correlations) was not
rejected, though a moderate negative relationship (r = -.1204 (-.l649), p < .05) was
found to exist between the Tolerance of Ambiguity measure and the TK test, while
virtually no relationship (r = .0313 (0434)) was found between Tolerance of
Ambiguity and the SJT. Hypothesis 9a, while rejected, provided some potentially
interesting insights and questions. It was believed that Tolerance of Ambiguity would
be most strongly linked to the Managing Tasks/Career dimension of the TK test, and
to the Problem Solving Skill dimension of the SJT. While this was the case, for the
SJT, Tolerance of Ambiguity was almost as strongly related to Communication Skill
as it was to Problem Solving Skill (.1182 (.3078) as opposed to .1224 (2695)), while
for the TK test Tolerance of Ambiguity correlated most strongly with the Managing
Tasks/Career dimension, but the relationship was negative (r =
-.1981 (-.2986), p < .01). It is curious that two tests, constructed in such similar
manners (according to the literature) could have such disparate relationships to the
extent to which individuals are tolerant of ambiguity in their lives. We will return to
this issue later, in the discussion section.

Hypotheses 10 was not rejected, while 10a was. A nonsigniﬁcant difference

83

existed between the correlations of TK and SJT with Openness to Experience. The
Openness construct did not allow for meaningful distinctions to be made among the
three SJT dimensions (and certainly not in the hypothesized direction, since Problem
Solving Skill, which was hypothesized to be most strongly related to Openness, was
actually the weakest of the three), nor did it provide interpretable results with respect
to the TK dimensions.

Hypothesis 11 was not rejected, while hypothesis 11a was. Hypothesis 11
dealt with the correlations of TK and SJT with Conscientiousness; these correlations
were both strong and positive, and not signiﬁcantly different from one another. The
attempt to specify the subscales of each test to which Conscientiousness was relevant
failed, however, as Conscientiousness correlated between .15 and .19 with all six of
the subscales, with none of the correlations being signiﬁcantly different from any
other. With respect to this variable as well, then, we have potential evidence for the
indistinguishability of the subscales.

Hypotheses 12 and 12a, dealing with the relationship of Machiavellianism to
the SJT and TK measure, were rejected. A signiﬁcant difference existed between the
correlations of Machiavellianism and the focal measures (rs, = .1911 (.2616), p <
.01), rm = -.0762 (-.1029), p > .05). Moreover, while Machiavellianism did
correlate most highly (.1383 (.3106)) with the Interpersonal Skill dimension of the
SJT (per Hypothesis 12a), that correlation was nonsigniﬁcantly higher than the
correlation between Machiavellianism and Communication Skill (.1089 (2797)). The
negative correlations that existed between Machiavellianism and all the dimensions of

the TK test may reﬂect an important difference between the two focal measures with

84

respect to the emphasis they place on interpersonal factors.
The focus of Hypothesis 13 was somewhat different from the others, in that it

put forth that a difference should exist between correlations. Speciﬁcally, because the

 

Managing Self dimension focused on self-regulatory activities that were not readily
apparent in descriptions of the SJT, it was believed that the correlations should differ.
This was the case -- unfortunately, the difference in correlations was in the wrong
direction. The correlation of Self-Regulation with SJT was .2613 (.3622), while the
correlation of Self-Regulation with Managing Self was .1141 (.2509). Moreover, all
of the SJT dimensions correlated .16 to .19 with Self-Regulation, while all of the TK
dimensions correlated .11 to .15 with it, providing another instance where the
breakdown of the subscales might be called into question. Hypothesis 13 was thus
rejected.

Hypothesis l4 put forth that individuals who scored highly on the Managing
Self dimension of the TK test would be more likely to demonstrate an
"accommodator" learning style (high in concreteness and action orientation) than
would those who scored low on it. Or, in other words, a positive correlation should
exist between Managing Self and Concreteness, and Managing Self and Action
Orientation. Support was not obtained for this hypothesis, as the correlation between
Managing Self and Concreteness was .0316 (.0427, p > .05), while the correlation
between Managing Self and Action Orientation was .1412 (.2054, p_ < .01). As this
meets only one of the two criteria for an "accommodator" learning style, no choice
remained but to reject the hypothesis. Doing well at Managing Self meant that

individuals were more oriented toward action, but had virtually nothing to do with

85

whether they preferred concrete or abstract problems and situations.

Thus far I have tested on an individual basis a set of hypotheses regarding the
similarities between tacit knowledge and situational judgment. In assessing construct
validity, however, the overall consistency of a pattern of correlations, termed a
nomological network (Cronbach & Meehl, 1955), is probably more important than
individual comparisons. Hence, the SEM represented in Figure 1 was tested. In
those cases speciﬁed by the hypotheses, equality constraints were imposed. This
model was compared with a model in which these constraints were not imposed to
provide an overall test of the degree of SJT and TK equivalence. The parameters
estimated in both constrained and unconstrained models are presented in Table 7, the
control statements specifying the model evaluated and the constraints imposed are
presented in Appendix L, and the model with unconstrained, standardized parameter
estimates is presented in Figure 2.

The model without constraints provided a moderate ﬁt to the data (x2 = 96.30,
df = 50; Root Mean Square Error of Approximation (RMSEA) = .052; Normed Fit
Index (NFI) = .92). As expected, when the equality constraints were applied, the
model ﬁt less well (X2 = 123.87, df = 60; RMSEA = .056; NFI = .89). Using the
chi-square difference test, we ﬁnd that the difference between the two nested models
is 27.57, with ten degrees of freedom, is signiﬁcant. This signiﬁcant difference
means that the "nomological net" related to tacit knowledge and situational judgment
is different. Because the nomological networks of the tests are not identical, we

should conclude that the tests likely do not measure the same constructs. However,

Iahlsl: Parameter Estimates from Structural Model

 

 

Indicator SJT Construct Unconstrained TK Construct Unconstrained Constrained
SJT Parameter TK Parameter Solution
Self- Managing
Regulation n/a n/a Self -.03 .00
Conscient- Problem Managing
iousness Solving Skill .07 Self .08 .07
Action Managing
Orientation n/a n/a Self .08 .09
Managing
Concreteness n/a n/a Self -.07 -.09
Machiavell- Interpersonal Managing
ianism Skill .14 Others -.04 .01
Educational Communication Managing
Experience Skill .01 Others -.01 .00
Educational Interpersonal Managing
Experience Skill .01 Others -.01 .00
Communication Managing
Extraversion Skill .01 Others .02 .03
Interpersonal Managing
Extraversion Skill .09 Others .02 .03
Comm. Communication Managing
Competence Skill .03 Others .02 .02
Work Problem Managing
Experience Solving Skill .02 Career .00 .01
Openness to Problem Managing
Experience Solving Skill .01 Career -.02 -.01
Tolerance of Problem Managing
Ambiguity Solving Skill .04 Career -.05 -.02

 

Note: SJT = Situational Judgment Test; TK = Tacit Knowledge; Comm. Competence = Communication

Competence

87

8. Va u . H82
686 uoﬁmbmuoosb ”mo—8256: _momwoBEo:

E 28 :m Sm $8838 $6883 woﬂEmucﬁm "N 95w:—

   
  

£835
.8 85.20%

   
      

       

  

.8855.
..O vocab—Oh-

4m _ .-
58 @255 \
... . w:_>_om E0305 3.- 8.8.“. maﬁa—52 (”1%

00:05me x83
my
oocotoaxm .3 c. Q.

:35
3:082:85

 

 
      

          
     

 

\omm 33688::

Exm cosmoEzEEoU

    

cons—swoxéow

88

we should also note that the ﬁt indices associated with the two models are not very
different. Thus, while a statistically signiﬁcant difference exists between the
constrained and unconstrained models, it is not an overwhelming effect.

Tests of less constrained models were made as well, in order to determine if
any of the speciﬁc constraints were responsible for the difference in model ﬁt found
above. None of the individual constraints were responsible for a large enough portion
of the X2 difference that the removal of a single constraint would create a constrained
model that was nonsigniﬁcantly different from the original model.

Finally, it should be noted that the unconstrained model provided only a
moderate ﬁt to the data. Some authors (see Bollen & Long, 1993) claim that the X2
difference test should not be applied in those instances in which the original/full

model does not ﬁt the data satisfactorily.

Chapter 4

DISCUSSION

Given the attention situational judgment tests have begun to receive in the
literature (Chan & Schmitt, 1997; Weekley & Jones, 1997), understanding precisely
what they are measuring becomes increasingly critical. This study, with its focus on
the similarity of situational judgment to Wagner and Stemberg’s notion of tacit
knowledge, represents one step in that direction. One step is not enough, however;
the full speciﬁcation of the nomological network surrounding situational judgment
tests must be undertaken if the ambiguity surrounding these measures is to be
removed.

The preponderance of evidence from this study suggests that situational
judgment and tacit knowledge are not isomorphic. First, the correlation between the
two measures is only .08 prior to correcting for attenuation due to unreliability, and
only slightly higher after the correction has been applied; if they measured the same
construct, the tests should reasonably be expected to correlate at least .40. Second,
the test of the SEM constraining the relationships of the two tests with respect to their
nomological networks demonstrated poorer ﬁt than the SEM in which this constraint
was absent, indicating different nomological networks for the two constructs. In

evaluating these models and the difference tests, it should be emphasized that all

89

90

correlations were relatively low, and very often nonsigniﬁcant. While these results
are obviously not what was hoped for, they are still informative and allow potentially
useful statements to be made about situational judgment and tacit knowledge on
empirical and conceptual planes.

In the next few pages I will discuss the contributions of this study, its
limitations, and the directions it suggests for future research. Several of the results
will be examined in greater detail, in an effort to understand precisely what lessons
we can draw from these data.

Contributions

Finding that situational judgment and tacit knowledge were actually isomorphic
would have represented an important contribution by helping to clarify what,
precisely, these situational judgment tests are measuring. But as has been mentioned
in various contexts throughout this paper, while what things a5 is an important
question, we must also keep in mind the question of what they m not. As things
worked out, the latter is the question better answered by this study.

The lack of correlation between SJT and TK, as. well as the relatively weak
evidence for distinctiveness of their nomological networks, represents an important
contribution because it allows us to say that situational judgment is likely not tacit
knowledge in another guise. While this ﬁnding is not as earth-shattering as
determining once and for all what situational judgment _i_s, it is nonetheless important
because it helps us narrow the ﬁeld of likely prospects by one. Incremental progress
such as this is, after all, the way much of the knowledge in our ﬁeld cumulates.

Moreover, this study helped to further our knowledge of the nomological

91

networks of both situational judgment and tacit knowledge. The position of
personality variables such as those measured by the NEO-FFI, of self-regulation, and
of various measures of experience are all pieces of information that assist us in
understanding what characteristics of the individual and his or her life situation
inﬂuence the capability to perform on these types of measures. More speciﬁcs
regarding this new information will be taken up in pages to come.

Perhaps one of the most interesting and important conﬁrmations offered by this
study is the fact that neither measure evinced a correlation with a measure of
cognitive ability (the Wonderlic Personnel Test) that was signiﬁcantly different from
zero. The ultimate value of situational judgment tests (and, to the extent they end up
being used, measures of tacit knowledge) will lie in their ability to add predictive
power to extant test batteries. The fact that neither measure correlates signiﬁcantly
with the measure of cognitive ability used in this study speaks well of their non-
redundancy with traditional predictors of job performance. What remains to be seen
is whether this non-redundancy translates into incremental validity for test batteries
that include situational judgment or tacit knowledge tests; some evidence for this
possibility on the tacit knowledge side has been provided by Stemberg and Wagner’s
work (Stemberg et al., 1995), but more must be collected.

Another potential contribution from this study is methodological; by extending
Frederiksen’s (1986a) work on construct validation, the approach taken in this study
represents a fairly comprehensive method by which the similarity of constructs can be
assessed. Admittedly, aspects of this methodology need reﬁnement; the tests of

differences between correlations are relatively low-power, and taken by themselves

92

might have led us to believe these two tests are very similar when in fact they are all
but uncorrelated! It might be argued that their inclusion does not "buy us" anything,
as researchers, and that the information they provide actually clouds the picture
somewhat. A caveat that must be added, however, is that this may be due to the
relatively low zero-order correlations observed in this study; with higher _rj’s, smaller
differences between correlations would have reached signiﬁcance.

Methodological recommendations that might be made based on this study must
be stated with caution, however, as this research may not typify construct validation
research. This study was highly exploratory, dealing with constructs that had never
(to the author’s knowledge) been examined together prior to this effort. Most
researchers, when approaching a question of construct similarity, will already have
some idea of the extent to which the measures correlate with one another, and may in
fact have based their assertion of construct similarity on such a correlation. It is
reasonable to say that the presence of a correlation between your two focal variables
is a necessary (though not sufﬁcient) condition for any positive conclusions about
construct similarity to be reached. Given this, the ﬁrst (relatively obvious)
recommendation to be made from this study is that before any other examinations of
construct similarity are made, the researcher should ﬁrst have evidence that the
measures in question actually correlate and that they are measured adequately.

Once a relationship is established between the measures, attention should then
be turned to the nomological networks surrounding them. The best analysis for this
purpose seems to be structural equation modeling, as it allows the researcher the

capability to not only model the data as it actually is, but to compare this model to

93

one in which the nomological networks of the two measures are constrained to be
equal. Moreover, the methodology allows the researcher ﬂexibility in determining
which aspects of the networks are dissimilar, if the equality constraints result in a
poorer-ﬁtting model.

Perhaps one of the most important contributions of this study, or any study
like it, is in the information it provides us about the relationships of the focal
measures to the various external variables. That is, what did we learn about the
nomological network, independent of tests of the similarity/difference of our models?

My approach to this will be straight-forward; I will examine the situational
judgment test ﬁrst, looking at the external variables it correlated with. I will also
examine the correlates of its subscales for hints of the extent to which they are
distinct from one another, and to help explain the speciﬁc correlations observed for
the overall scale. I will follow the same basic process for the tacit knowledge test.

The overall situational judgment test correlated signiﬁcantly with the tacit
knowledge dimension of Managing Others, such that as SJT performance increased,
respondents also performed better on those items labeled as Managing Others. The
only subdimension of the SJT that correlated with Managing Others was
Communication Skill, which is consistent with the belief that an important aspect of
Managing Others is the ability to communicate. SJT also evidenced signiﬁcant
positive correlations with age and sex; the age ﬁnding is consistent with existing
literature that suggests older respondents perform better on tests of situational
judgment, and is further consistent in that the only subscale with which age correlates

is Problem-Solving Skill (Denney & Palmer, 1981). The sex ﬁnding is interesting,

94

since it indicates that females perform better on the SJT than males. The SJT items
that seem to be most highly correlated with sex are those involving Interpersonal
Skill.

Three variables correlated signiﬁcantly with the overall SJT as well as all three
of its subscales, which might lead us to consider these as indicative of what the SJT
as a whole measured. The ﬁrst of these is Communication Competence, a ﬁnding
that supports one of the hypothesized dimensions of situational judgment (Motowidlo
et a1. , 1990) and should therefore not surprise us. The other two, however, are more
surprising.

Self-regulation, in terms of paying attention to the learning process and the
elements of that process (see Appendix J) showed signiﬁcant positive relationships
with all aspects of situational judgment. While this relationship was not hypothesized,
it is very interesting; most of the thinking about managerial situational judgment tests
has focused on the way managers interact with others and the skills involved in such
interactions. This ﬁnding suggests that in order to really tap the way managers go
about solving problems and interacting with others, more attention must be paid to
their cognitive, and indeed their metacognitive (Flavell, 1979), processes.

The fact that the SJT and its subdirnensions correlate signiﬁcantly with
conscientiousness may be interpreted in one of two ways -- one conceptual, one
methodological. The ﬁrst interpretation would be that individuals who are more
conscientious make better managers, potentially because they are more concerned
about the welfare of their employees and of the organization as a whole. Given the

use personality variables have seen in recent years as predictors of job performance

95

(see Barrick & Mount, 1991), such an interpretation might not be far aﬁeld.
However, an assertion that this is the case may be a greater logical leap than can be
made on the basis of this study. The methodological interpretation might argue that
conscientiousness correlated with the SJT and its dimensions for the same reason that
it correlated with the positively-keyed tacit knowledge items, and the tacit knowledge
subdirnensions: because individuals who were more conscientious were the only ones
who took the tests seriously, and are the only ones whose scores reﬂect effort and
motivation to perform on the test. While the second interpretation is admittedly
somewhat pessimistic, it cannot, within the context of this study, be discounted.

The overall situational judgment test also correlated signiﬁcantly with
Machiavellianism, as did the two "soft" subdirnensions (Interpersonal Skill and
Communication Skill). This relationship was hypothesized, and is consistent with the
theoretical framework from which this study approached the notion of effective
management.

Perhaps the most interesting ﬁnding was that responses to the overall SJT were
signiﬁcantly related to all of the Big Five personality dimensions, as measured by the
NEO-FFI, and all of these were in the hypothesized direction. That is, individuals
who were lower on Neuroticism and higher on Extraversion, Openness to Experience,
Agreeableness, and Conscientiousness (as mentioned) tended to do better on the SJT
than did individuals who were not. Previous studies (Pulakos, Schmitt, & Keenan,
1994) have found little or no relationship between personality variables and SJT
performance, so the fact that the entire Big Five related to SJT in this study suggests

that any notions of abandoning research relating personality to situational judgment

96

are premature. The nomological network of situational judgment, particularly with
respect to personality, must be examined in greater detail in a variety of contexts to
determine whether these results are replicable.

Communication Skill was the only SJT subscale which correlated with external
variables not correlated with the overall scale. Educational Experience demonstrated
a signiﬁcant positive correlation with Communication Skill, while Community
Experience demonstrated a signiﬁcant negative correlation with the dimension. The
positive relationship to Educational Experience is easy enough to explain; individuals
who have had a greater variety of educational experiences, in terms of not only years
of schooling but also number and variety of extracurricular activities they have been
involved in, will likely have developed greater communication skills through their
interactions than those who have not had the broad variety of educational experiences.
More difﬁcult to explain is the negative relationship with Community Experience.
One would expect individuals who are more involved in religious groups, volunteer
organizations, other community groups, or camp counseling to have developed the
same kinds of communication skills that individuals in other organizations have. It
may be that the structure of such community organizations is different than either
educational or work environments, in terms of being more cooperative and less
hierarchical. If that were the case, then even though a signiﬁcant positive relationship
exists between all three types of experience, different people may, based on various
personality characteristics, take part in different aspects of the different experiences;
individuals who possess managerial characteristics may be less apt, that is, to take

control of a community group in the same manner they would an educational or

97

business group, or to communicate with others in those groups in the same manner.
Clearly, personality and situational factors are important here, and there is room for
potentially interesting research to be done on the factors that inﬂuence leadership and
communication behaviors in the different experiential domains outlined in this study.

With respect to tacit knowledge, although the measure is of questionable
validity some potentially interesting correlates emerged. As has been the case
throughout the latter portions of this paper, when referring to the tacit knowledge
measure I will discuss only the positively-keyed items for both the overall measure
and the subscales.

First, Tacit Knowledge showed a signiﬁcant positive correlation with race, a
correlation which was also present for the Managing Career items. Given the
relatively small proportion of non-white individuals in the sample, interpretation of
this ﬁnding is probably not appropriate. More interpretable is the ﬁnding that
Communication Competence was signiﬁcantly positively related to performance on the
Tacit Knowledge test, indicating that the skill with which individuals communicate
with others is critical to their ability to demonstrate tacit knowledge. Given that tacit
knowledge is believed to be gathered through interactions with others and with the
world around us, this ﬁnding is consistent with previous theory.

As with situational judgment, tacit knowledge and all of its subdimensions
correlated positively with self-regulation. The interpretation made above, regarding
the importance of self-regulation and metacognitive skill to management, is one
approach to explaining this ﬁnding. However, an examination of the correlation

matrix in Table 4 demonstrates that self-regulation correlated with virtually all other

98

measures. This raises the alternative explanation that individuals who self-regulate
more or better (as measured by the items utilized here) may simply perform better on
written tests of the type included in this study. While this is not an unreasonable
assertion, it does raise questions about the interpretability of ﬁndings regarding self-
regulation in this study.

Action Orientation was hypothesized to relate to the Managing Self dimension
of Tacit Knowledge; as it turned out, it also correlated with the Managing Others
dimension and overall TK. The fact that two aspects of Tacit Knowledge correlate
with Action Orientation argues somewhat for the distinctiveness of the Managing
Career scale from the other scales; identifying areas in which the scales differ is one
way we can go about identifying what the meaning of the dimensions, as measured in
this study, actually was. The Managing Career dimension was signiﬁcantly related to
the other Learning Style variable, Concreteness, such that individuals who are more
concrete learners tend to score higher on items from the Managing Career scale.
Hence, active learning is important when dealing with yourself or with others, while
concrete learning may be most important when dealing with tasks or the more macro-
level career variable. Given that active, concrete learners are supposedly the
individuals who are best suited to managerial jobs (Romero et al. , 1992), these results
may help explain which aspects of managerial jobs call for the application of which
type of learning.

With respect to personality variables, we already noted that Conscientiousness
related to all the subdirnensions of Tacit Knowledge, just as it did with the SJT. As

such, this variable will not be discussed again. Extraversion was also related to

99

overall tacit knowledge test performance, correlating .1124 with it. Thus, as the
literature review for this study suggested, individuals who tend toward extraversion
should have more opportunities to interact with people, and should therefore be more
likely to develop tacit knowledge. Similarly, while Openness to Experience did not
correlate with the overall TK measure, it did correlate with Managing Career; again,
this makes sense in light of the literature review, given that individuals who are more
open to experiences will develop a broader experiential base from which they can
draw when issues of career management become salient.

Several other variables correlated with subscales but not with the overall scale.
Educational Experience correlated positively with Managing Self. It is possible that
this correlation is due to the undergraduate nature of the sample, for whom
educational experiences are the most salient in determining how they manage their
lives, while this would not be the case for a sample from an organization. In
addition, the Communication Skill dimension of the SJT correlated signiﬁcantly with
the TK dimension of Managing Others (as was noted above), and age was positively
correlated with Managing Others as well; this last ﬁnding is explicable in terms of the
greater experience individuals have with others as they age, and the knowledge they
gain as a result of this experience.

The last correlation I will discuss with respect to the tacit knowledge measure
is the one which makes the least sense; both the overall tacit knowledge scale and its
Managing Career subscale evinced signiﬁcant negative correlations with Tolerance of
Ambiguity. In considering the literature review for this study, it was my belief that

individuals who were more tolerant of ambiguity would be able to handle the types of

100

ambiguous situations presented in both situational judgment and tacit knowledge
measures more adeptly than would those who were less tolerant of ambiguous
situations. To ﬁnd that individuals who are more tolerant of ambiguity actually do
worse on a test of tacit knowledge leads me to question whether it is possible for
individuals to be too tolerant of ambiguous situations; if they treat ambiguity as
commonplace and don’t get overly upset about having to deal with the ambiguity they
face on the job, are they likely to miss important aspects of the situations? Or might
it be that individuals who are tolerant of ambiguity do not actively question
ambiguous situations, instead allowing them to simply remain ambiguous, and
therefore do not gain the tacit knowledge they might have obtained if they had been
more active in questioning all aspects of the situation? Given the prevalence of
ambiguous situations and the importance (to managers) of being able to deal with
them efﬁciently, it will be critical to further investigate this relationship in future
work.

Examining the correlations among all the variables, rather than only those for
which equality constraints were hypothesized, allows us the opportunity to look at
something broader than simply the conceptual model. By considering where sizable
and statistically signiﬁcant correlations did and did not appear with respect to tacit
knowledge and situational judgment, we are better able to specify future iterations of
the nomological networks surrounding both types of measures. With this said, I will
now turn to the limitations of my study.

Limitations

The most obvious limitation of this study is the apparent motivational problems

101

of the participants. While no motivation scale was included in the battery of tests,
there can be little doubt what one would have demonstrated, had it been included (and
had the participants responded accurately!)

The battery administered to study participants included over 400 items, when
the Wonderlic and the NEO-FFI are included. Of the items, over US were tacit
knowledge items, 131 of which were not scored. While there is no way participants
could know that many of the items to which they responded would not be used, the

sheer volume of items was certainly sufﬁcient to overwhelm them, and this might

 

have been avoided by including only those items which would be scored. '

That motivation among participants was problematic can be inferred from
patterns within the data. Speciﬁcally, I noted earlier that the reverse-scored items
ended up clustering together at the low end of the 1-7 scale, while the items that were
not reverse-scored clustered together at the high end. This seems to indicate a
response set on the part of participants, such that every option presented was met with
a general positive response. The apparent demotivation might be due to the fatigue
resulting from the length of the test battery, or it might be due to the lack of any sort
of reward for performance on the battery (e.g. , some form of monetary inducement
might have encouraged higher levels of motivation).

The use of only the positively-scored tacit knowledge items represented an
attempt to circumvent these motivational issues, as we may expect that at the very
least, responses to the positively-keyed items represent the true feelings of the
respondents with respect to the items. While this solution was by no means ideal, it

did allow the analyses around which this study was designed to be conducted.

102

However, the results must be classiﬁed as preliminary.

Another study limitation also involves the tacit knowledge test. To all
appearances, the key provided by the test developers was empirical, and likely based
on a sample collected for research published in 1985. Empirical keys often
demonstrate questionable cross-validities (Murphy & Davidshofer, 1994), casting
doubt on the ultimate usefulness of such measures for research purposes. The
creation of a new empirical key for the sample gathered in this research was
considered; however, the only external criteria available for the creation of such a key
were measures of cognitive ability. Not only would this create an artifactual
correlation between variables that are hypothetically uncorrelated (cognitive ability
and tacit knowledge), it is incorrect from a theoretical standpoint, so the creation of a
new key was dismissed as a possibility.

Unfortunately, no other research measures of tacit knowledge are widely
available. A more advanced, updated version of the tacit knowledge scale used here
is in publication; budgetary and other logistical constraints made this option
infeasible, though, leaving only the scale used in this study as an option. While it
may prove impossible to construct, a rationally-keyed scale measuring tacit knowledge
for managers would likely prove useful to researchers and practitioners and would
eliminate the kinds of difﬁculties encountered here.

Some apparent measurement deﬁciencies of the focal measures must be noted
as well. First, with respect to the situational judgment test, expert raters reported
difﬁculty in making distinctions between items tapping communication skill and those

tapping interpersonal skill. This difﬁculty is one symptom of a general

103
indistinguishability of the subscales that made up the SJT . For both the situational

judgment and tacit knowledge tests, the dimensions of each had highly similar
correlations with the Communication Competence, Conscientiousness, and Self-
Regulation scales. As an example, consider Communication Competence; its
correlations with the three tacit knowledge subtests were .1039, .1390, and .1519,
while its correlations with the situational judgment subscales were .1532, .1680, and
.1700. In addition, all of the tacit knowledge subscales had highly similar
relationships to Extraversion, while all of the situational judgment subscales had
similar relationships to Openness to Experience.

The fact that for each of the two scales, its subscales were not distinguishable
with respect to at least four external variables where distinguishability would be
expected is mildly troubling. One of the issues from which this arose has already
been noted (the empirical key for the tacit knowledge test). Another potential causal
factor is the method by which the situational judgment subscales were derived.
Because no a mg classiﬁcation existed of the SJT items used into Interpersonal,
Communication, or Problem-Solving Skills, expert raters were asked to classify each
item into the category they believed it ﬁt best. The raters achieved 100% agreement
on almost none of the items, indicating that some amount of overlap should be
expected. However, the fact that the correlations of the three dimensions for each of
the major tests differed signiﬁcantly for many of the external variables allows us some
level of conﬁdence in the distinguishability of the scales. It was hoped that the use of
expert raters would allow for meaningful subscales to be identiﬁed from the available

items, where simple factor analysis did not. That low estimates of coefﬁcient a were

104

obtained for the scales constructed based on the raters’ work indicates that even when
experts viewed the items as measuring the same construct, the items themselves were
not internally consistent. This might be due to one of two reasons.

The ﬁrst possibility is one already noted in an earlier section of this paper,
that being the likelihood that situational judgment tests are multidimensional at the
item level (Clause et a1. , 1996). Items which are themselves multidimensional contain
multiple constructs which may prove more or less salient to different respondents.

Under such a situation, it may well be that or represents an inappropriate estimate of

. r'f

reliability and that we should attend to estimates such as test-retest (since within
individuals, there is little reason to believe the constructs measured by the test items
will change over time) or interrater (which is essentially the method used to select
items for inclusion in the subscales for this study).

The second possibility is that the theoretical dimensionality of situational
judgment, put forward by Motowidlo and colleagues, may not generalize. The
dimensionality of managerial behavior, while heavily discussed in various literatures
(Baldwin & Padgett, 1993; Thornton & Cleveland, 1990), may not be to a stage
where we can say with conﬁdence that interpersonal, communication, and problem-
solving skills represent the ultimate triumvirate of managerial behaviors. The fact
that the scales were signiﬁcantly intercorrelated in this study, while simultaneously
having such low alphas, suggests that a general deﬁnition of the dimensionality of
management remains a research need in this area.

Similar things can be said about the tacit knowledge measure. The scales do

not look at all distinctive, initial problems existed with the coefﬁcient a values, and

105

the scales are highly intercorrelated. Rather than reiterate speciﬁc criticisms that have
been levied at both measures in the previous pages, I would like to brieﬂy discuss the
scoring of the TK test.

The response scale for the tacit'knowledge items is such that it makes it easy
to get positive response bias of the sort seen here. For each situation, trainees rate
potential responses on a 1-7 scale, ranging from "Not Important" to "Extremely
Important" with respect to the decision being made. It should not be overly
surprising that respondents tended to treat every response as potentially important to
the decision, since even though they have had a broad variety of experiences it is

unclear based on data collected what they learned from the jobs they held in the past.

 

Adopting a more ipsative format, such as the one utilized in the SJT, might improve
the tacit knowledge test; however, it becomes unclear at that point whether a
measurement effect might become dominant. If such a tacit knowledge measure were
constructed and a stronger relationship between TK and SJT were found, the question
of whether this represents an improvement in measurement quality or simply a
convergence of measurement methods might prove difﬁcult to answer.

The use of an undergraduate sample, while an apparent limitation, is not any
more damning in this study than in any other study of this type. The general
concerns of restriction of range on age, education, and experience are all issues that
must be considered. While it is certainly true that all participants were
undergraduates and the bulk of them were ages 18-24, they showed a range of
experience with respect to the number of jobs held that was surprisingly close to

normal. Most of them (193) held a moderate number of jobs (3-5) while fewer held

106
only 1-2 jobs (61) or 6-10 jobs (73), and fewer still held no jobs (5) or more than 10

jobs (16). The fact that the bulk of the respondents had held at least 3-5 jobs, with
89 having held 6 or more, speaks relatively well to the level of experience
participants in this study had with the types of situations described in the SJT and TK
test. If the fact that a decent range of experiences were brought to the questionnaire
by respondents is considered, the inferences I sought to make are relatively safe; all
revolved around construct issues, and given a broad enough experiential base, such
relationships should not differ greatly when we move from one sample to another. As
such, inferences made about them in this context should be relatively applicable to
any other. One variable for which this was likely not the case was the Educational
Experience variable, for reasons mentioned brieﬂy in the Results section of this
paper.

The reader will recall that among the tacit knowledge dimensions, Educational
Experience was hypothesized to be most strongly related. to Managing Others; instead,
it was most strongly related to Managing Self, though the difference between the
correlations was nonsigniﬁcant. The explanation offered for this ﬁnding dealt with
the impact of education, and the fact that for individuals whose formal education is
ongoing it should be more strongly related to proximal domains (such as time
management and getting along with people) rather than distal domains (actually
applying educational experiences to managing others).

Directions for Future Research

 

Although we now know one more thing situational judgment tests likely do not

measure, we still do not know precisely what they d_o measure. This remains an

107

important question. Motowidlo et al. (1990) mention two options other than tacit
knowledge/practical intelligence as alternative explanations of what situational
judgment tests are measuring. The ﬁrst is behavioral consistency; given that work
experience correlated .0956 with situational judgment test performance, however, this
study does not seem to support this notion. However, it must be noted that the work
experience measure utilized here was constructed independent of the situational
judgment test items, and was designed to assess those general on-the-job experiences
that might have an impact on an individual’s ability to perform well, given a
managerial role. Future studies interested in testing whether situational judgment is
merely tapping behavioral consistency should utilize experience measures constructed
directly from the situational judgment test, in which respondents are asked whether
they have ever been in a given situation. Once such an experiential checklist was
completed, the situational judgment test could be administered and a more meaningful
correlation coefﬁcient computed.

The second explanation offered by Motowidlo et al. (1990) dealt with goal-
setting theory and intentions. The type of research that might best address this
explanation is presented in that paper, and I will not reiterate it here other than to
note that it should, if we are to understand situational judgment tests, be done.

Other directions for research have been mentioned at various points throughout
this paper. One such item deals with the tacit knowledge measure used in this study.
The question of whether the 35 scored tacit knowledge items would measure the same
construct without the administration of the other 131 unscored items is, practically

speaking, very important. The demotivating effects of excessively long tests on

108

research participants are a relatively minor problem in comparison to the possibility of
negative applicant reactions resulting from a test of that length. If this measure is to
ever be used again, research should be conducted to ensure that the same construct is
measured when only the scored items are presented as when the full set of items are
presented.

In addition, the development of a rationally-keyed tacit knowledge test should
be considered. The difﬁculty, of course, lies in deﬁning which of the myriad
potential responses to a given situation are "effective" and which are "ineffective" in a
way that cuts across organizational boundaries. Climate and culture variables will
almost certainly assist in determining what types of responses are appropriate to a
given situation, so the search for a rational key may ultimately be doomed, or at the
very least, be forced to move to such a macro-level (regional or national, rather than
corporate cultures, for example) as to be impractical for researchers interested in
Industrial/Organizational psychology. Backing away from that direction somewhat,
then, I would certainly recommend research be undertaken to deﬁne a new empirical
key based on a broader sample than whatever the basis for the current tacit knowledge
scale might have been. Similar comments can be made about situational judgment
tests.

It should be clear, then, that much remains to be done with respect to
understanding situational judgment tests. Tacit knowledge was one direction that had
potential, but which ultimately fell short. Further research must be conducted in
order to provide a solid research base for these measures. Two overriding

responsibilities demand this. First, we have a responsibility to those organizations we

109

service to offer the best possible product, which not only selects the best individuals
but will also stand up to legal scrutiny. Second, and more importantly, we have a
responsibility to our ﬁeld to cumulate knowledge about the process of personnel
selection that will help us to move forward and better understand the what, when,

why, and how of predicting job performance.

APPENDIX A

Consent Form
Subject #:

Informed Consent
Test Relations Study

The study in which you are about to participate investigates the relationships
among various types of tests used by businesses to select whom they will hire.

Your participation in this study requires that you remain for the entire two and
one-half hour session. While your participation is completely voluntary, the success
of this study depends upon you making a commitment to do your best on the tests so
that we get an accurate appraisal of your knowledge and abilities. No possible risks
or discomforts are anticipated as a result of this study.

At the end of your involvement, you will be provided with feedback explaining
the purpose of this study in more detail. Participation is strictly voluntary. You are
free to discontinue the study at any time for any reason without penalty. Your
responses will be completely conﬁdential. During the study you will not be asked to
put your name on any of the forms you will be ﬁlling out, in order to maintain your
confidentiality. Once all data is collected all records of your name and its relation to
your data will be discarded. You are free to ask any questions you might have about
this study at any time. You may ask questions about the outcome of this study at any
time by contacting Morrie Mullins at 3-9166 or throughthe Department of
Psychology.

Consent: I have been fully informed of the above-described study and its possible
risks. I give permission for my participation in this study. I know that the
investigator and his associates will be available to answer any questions I may have.
I understand that I am free to withdraw this consent and discontinue participation in
this study at any time without penalty.

Name:

 

Signature: Date:

110

APPENDIX B

Debrief Form
Test Relations Study Debrief'mg Form

The study in which you just participated was designed to examine the relationship
between a situational judgment test (the items which asked, "Which of these would
you most/least likely choose? ") and a test of tacit knowledge (the items which
constituted a long list after a statement of a problem). Some scientists have theorized
that situational judgment tests may measure tacit knowledge, but that remains unclear.
By ﬁlling out these measures, plus several others that measure things that should be
related to both, we can compare these two tests to see if they are, in fact, measuring
the same thing.

If you have any questions about this study or would like to receive a c0py of the
results when they are complete, please notify the investigator now. If, in the future,
you have any questions about the study or would like to receive the results when they
are complete, please call the investigator listed below. Finally, thank you for
participating in this study. We tried to make it as interesting as possible and we are
open to suggestions.

Investigator:
Morrie Mullins, 3-9166

111

APPENDIX C

Demographic Information

How old are you?

1 . 17-19 4. 26-29
2. 20-22 5. 30 or older
3. 23-26

What is your classiﬁcation?

1 . Freshman 4. Senior
2. Sophomore 5. Graduate student
3. Junior

Approximately, what is your grade point average?

1. less than 2.0 4. 3.0 - 3.49
2. 2.0 - 2.49 5. 3.5 - 3.99
3. 2.5 - 2.99 6. 4.0

What is your gender?
1. Male 2. Female

What is your race?

1. African-American 4. American Indian
2. Asian 5. Caucasian
3. Hispanic 6. Other

 

(please describe)
Approximately what is the yearly household income for your family?
1. less than $15,000 4. $30,000 - 39,999

2. $15,000 - 19,999 5. $40,000 - 60,000
3. $20,000 - 29,999 6. more than $60,000

112

113

Which of the following best describes your major ﬁeld of study?

1 . Business 4. Arts
2. Social Science 5. Humanities
3. Natural Science 6. Other

 

(please describe)
Which of the following best describes the area in which you were raised?
1 . Rural 2. Urban 3. Suburban

How many books have you read in the last six months that were not required
for school?

1. None 4. 5 - 6
2. 1 - 2 5. more than 6
3. 3 - 4

6.

7.

APPENDIX D

Tolerance of Ambiguity Measure (Major, 1990)

I prefer work assignments with speciﬁc directions to those with vague
directions that require my own interpretation. (R)

I dislike supervisors who expect me to ﬁgure out my work assignments on my

own. (R)

A good job is one where what is to be done and how it is to be done are
always clear. (R)

It’s important for me to know exactly what my work assignments are and
when they are due. (R)

Jobs that have a lot of change and uncertainty are more desirable than jobs
with little change and uncertainty.

I am uncomfortable when I’m not sure what is expected of me. (R)

It is impossible to do a good job when the requirements keep changing. (R)

note: (R) = reverse scored, if the same rating scale (1) = "strongly disagree" to (7)
= "strongly agree" is to be used

114

APPENDIX E

Communication Competence Self-Report Questionnaire
(Rubin, 1986)

This questionnaire is composed of statements concerning your communication
with other people. Please indicate how each statement reﬂects your own
communication behavior by marking if it applies to you:

(I) ALWAYS (2) USUALLY (3) SOMETIMES (4) SELDOM (5) NEVER

1. I mispronounce a lot of words.

2. When speaking with someone, the words I use say one thing while my face
and tone of voice say something different.

3. When giving a speech, I speak clearly and distinctly. (R)
4. When giving a speech, I can be persuasive when I want to be. (R)
5. When I speak with others, my ideas are clearly and concisely presented. (R)

6. When giving a speech, I thoroughly express and fully defend my positions on

issues. (R)
7. I am unable to tell whether or not someone has understood what I have said.
8. I know when I’m hearing a fact and when I’m hearing someone’s personal
opinion. (R)
9. When professors make suggestions in class on how I can improve, I

understand the suggestions. (R)
10. I understand the assignments that are given orally in class. (R)

11. When I tell others about a class lecture I’ve heard, my version leaves out some
important items.

12. When I have to introduce myself in a class, I am able to fully and concisely
describe my interests and let others know who I am. (R)

13. When speaking with others, I have to ask a question several times, in several
ways, to get the inforrnationl want.

115

14.

15.

l6.

17.

18.

19.

116

I have to answer a question several times before others seem satisﬁed with my
answer.

I ﬁnd it difﬁcult to express my satisfaction or dissatisfaction about a course to
the professor.

When I explain something to someone, it tends to be disorganized.
When I give directions to another person, the directions are accurate. (R)

When I try to describe someone else’s point of view, I have trouble getting it
right.

I am able to give a balanced explanation of differing opinions. (R)

note: (R) = reverse scored

10.

APPENDIX F
Machiavellianism Measure

(Allsopp, Eysenck, & Eysenck, 1991
reported in Mudrack & Mason, 1995)

Would you be prepared to deceive someone completely if it were to your
advantage to do so?

Would you be prepared to do a bad turn to someone else in order to get
something you particularly wanted for yourself?

Do you often act in a cunning way in order to get what you want?

Would you be prepared to "walk all over people" to get what you want?

Do you enjoy manipulating people?

Do you tend to do most things with an eye to your own advantage?

Do you agree that the most important thing in life is winning?

Would you be prepared to be quite ruthless in order to get ahead in your job?

Would you be prepared to be humble and honest rather than important and
dishonest? (R)

Would you like to be very powerful?

note: R = reverse scored; items are answered either "yes" or "no" by respondents

117

APPENDIX G

Tacit Knowledge Measure (Wagner & Stemberg, 1985)

All questions ask you to rate the importance you would assign to various items in
making work-related decisions and judgments.

Use a l to 7 rating scale. A rating of 1 should signify "not important" while a 1
should signify "extremely important." A rating of 4 should signify "moderately
important."

 

1 2 3 4 5 6 7
Not Moderately Extremely
Important Important Important

Try to use the entire scale when responding, although not necessarily for each
question. For example, you may decide that none of the items listed for a
particular question are important, or that they all are. There are, of course, no
"correct" answers. You are encouraged to brieﬂy scan the items of a given
question before responding to get some idea of the range of importance: for the
items. Remember, you are being asked to rate the importance you would
personally assign each item in making the judgment or decision mentioned in the
question item.

Note: Items included in the Managing Career Scale are noted with a (C), those
included in the Managing Self Scale are noted with an (S), and those included in the
Managing Others Scale are noted with an (0). Items that were reverse-coded are
noted with a (-).

118

119

I. It is your second year as a mid-level manager in a company in the
communications industry. You head a department of about 30 people. The
evaluation of your ﬁrst year on the job has been generally favorable. Performance
ratings for your department are at least as good as they were before you took over,
and perhaps even a little better. You have two assistants. One is quite capable. The
other just seems to go through the motions, but to be of little real help.

You believe that although you are well-liked, there is little that would
distinguish you in the eyes of your superiors from the nine other managers at a
comparable level in the company.

Your goal is rapid promotion to the top of the company. The following is a
list of things you are considering doing in the next two months. You obviously
cannot do them all. Rate the importance of each by its priority as a means of
reaching your goal.

1. Find a way to get rid of the "dead wood", e. g., the less helpful assistant and
three or four others.

2. Participate in a series of panel discussions to be shown on the local public
television station. (0)

3. Find ways to make sure your superiors are aware of your important
accomplishments. (C-)

4. Make an effort to better match the work that needs to be done with the
strengths and weaknesses of individual employees.

5. Become more involved in local public-service organizations.

6. As a means of being noticed, pr0pose a solution to a problem outside the
scope of your immediate department that you would be willing to take charge
of. (C-)

7. When making decisions, give a great deal of weight to the way your superior

likes things to be done. (C)

8. Accept a friend’s invitation to join the exclusive country club that many higher
level executives belong to. (C-)

9. Accept a position of responsibility in the upcoming United Fund drive.
10. Ask for comments from superiors about important decisions you need to make.

11. Transfer many employees to get some "new blood" in the department.

12.

13.

14.

H.

120

Become involved in the local chamber of commerce and junior achievement
organizations.

Adjust your work habits to increase your productivity.

Write an article on productivity for the company newsletter.

A subordinate has come to you for advice on how to be more successful in the

company. You don’t know the person well enough to speak in anything but
generalities. Rate the following pieces of advice by their importance to succeeding in

the company.

15. Regularly set priorities that reﬂect the importance of your tasks.

16. Try always to work at only what you are in the mood to do.

17. Keep in mind who will ultimately see or use the results of your labor when
setting priorities.

18. Identify those tasks you face which are in fact trivial.

19. Do your absolute best on everything you do.

20. Frequently ask yourself whether anyone else should or could do the task.

21. Keep your long-term career goals in mind when setting priorities.

22. Set your priorities by what your boss thinks is important, unless you can sell
him or her on a contrary priority.

23. Do routine tasks early in the day so as to be sure to get them done.

121
III. Your company has sent you to a university to recruit and interview potential
trainees for management positions. You have been considering characteristics of
students that are important to later success in business. Rate the importance of the
following student characteristics by the extent to which they lead to later success in
business.
24. Aptitude or intelligence.
25. Ability to set priorities according to the importance of your tasks. (S)
26. Being meticulous about all details.
27. Ability to evaluate critically your work and the work of others.
28. Effectiveness with which you organize your time.
29. Motivation. (S-)
30. Interpersonal skills.
31. Ability to follow through and bring completion to tasks. (S)
32. Creativity.
33. Tendency to try to do everything perfectly.

34. Broad-ranging interests and hobbies.

35. Ability to promote your ideas; to convince others of the worth of your words.

(0)
36. Ability to write well.
37. Speaking ability.
38. Ability to read material in your ﬁeld with high comprehension.
39. Perseverance in pursuit of a single pet project or goal.
40. The need to win at everything no matter what the cost. (S)

41. Not one to take orders or direction; better at giving advice than taking it.

122

42. Ability to communicate what you do and think in an interesting manner and to
convey your enthusiasm for what you are doing.

43. Knowledge of relevant areas of business (ﬁnance, production, etc.).

IV. During one of your recruiting interviews at the university, a student asks you
about things one can do to increase one’s chances for success in business. Rate each
of the following things one might do by its importance to a successful career in
business.

44. Avoid criticism of others unless you have a better alternative to propose.

45. Defend your views in the face of all criticism.

46. Make a point to meet the people that may in some way inﬂuence the course of
your career. (C-)

47. Always say exactly what you think.

48. Take on assignments that potentially may bring your work to the attention of
higher-level executives in the company.

49. Be the ﬁrst to volunteer when volunteers are needed regardless of the
importance of the task.

50. Take advantage of opportunities to take on responsibilities beyond the scope of
your immediate assignment.

51. Don’t worry if your reputation in the company is sagging so long as you know
you are doing good work.

52. Dress and behave in a manner that ﬁts the corporate image.

53. Give as much attention to when and how you present something as you do to
what you are presenting.

54. Avoid drawing attention to yourself at all costs.
55. Model yourself after a successful superior. (S)

56. Take advantage of opportunities to get favorable attention from the local
media. (C-)

V.

57.

58.

59.

60.

61.

62.

63.

64.

65.

66.

67.

68.

69.

70.

71.

72.

73.

123

A number of factors enter into the establishment of a good reputation in a
company as a manager. Consider the following factors and rate their importance.

Critical thinking ability.

Visibility (i.e., well-known throughout the company).

Speaking ability. (C-)

Ability to set rather than just follow trends.

Involvement in public service and charitable organizations.
Being regarded as a "team player."

Extent of college education and the prestige of school attended. (C-)
Proceeding in a manner that avoids mistakes at all costs.
Ability to solve "people problems ".

Ability to get results.

No hesitancy to take extraordinarily risky courses of action. (8)
Never being one to rock the boat.

Belonging to the appropriate clubs and being active in the appropriate
organizations.

Ability to listen well.
Ability to get the job done bottom line.
Ability to come across well in meetings.

A keen sense of what superiors can be sold on. (C-)

VI.

124

Rate the following characteristics of a job by their importance in leading to a

successful career in a given company.

74.

75.

76.

77.

78.

79.

80.

81.

82.

83.

84.

The job will bring your work to the attention of higher-level management
personnel.

The job is considered to be an important one by company personnel.

The job should enable you to demonstrate your strengths and to hide to some
extent your weaknesses. (S-)

Your boss seems to be advancing rapidly.

The job is the recent creation of a company reorganization.
You can master the job with almost no effort.

The job will result in frequent exposure to the local media.
The last person who held the job was well-liked.

The job is essential to the day-to-day operation of the company even though it
is not perceived to be important. (C)

The reputation of the department is a poor one, so there is room for
irnprovement.

You have your family’s support on issues such as travel, working late or on
weekends, etc.

VII.

125

Rate the following strategies of working according to how important you

believe them to be to doing well at the day-to-day work of a business manager.

85.

86.

87.

88.

89.

90.

91.

92.

93.

94.

95.

96.

97.

98.

99.

100.

Always have a variety of projects in progress; many "irons in the ﬁre".
Think in terms of tasks accomplished rather than hours spent working. (S)
Be in charge of all phases of every task or project you are involved with. (S)
Do not force yourself to do tasks you don’t feel like doing.

Use a daily list of goals arranged according to your priorities. (8)

Set personal deadlines to supplement externally imposed ones.

Don’t try to do everything well -- many tasks are trivial.

Avoid schedules and "to do " lists because they are too inﬂexible.

Set aside a regular time for accomplishing important tasks.

Delegate tasks to competent others whenever possible.

Make good use of waiting time, commuting time, etc.

Pay careful attention to scheduling.

Carefully consider the optimal strategy before beginning a task. (S-)
Reward yourself upon completion of important tasks. (S-)

Pay particular attention to the details of a task.

Find a way to make a game of quickly completing mundane tasks.

 

126

VIII. You have just been promoted to head of an important department in the
company. The previous head had been transferred to an equivalent position in a less
important department. Your understanding of the reason for the move is that the
performance of the department as a whole was mediocre. There were not any glaring
deﬁciencies, just a perception of the department as so-so rather than very good. Your
charge was to shape up the department. Results are expected quickly. Rate the
following pieces of advice colleagues have given you by their importance in
succeeding in your new position.

101. Always delegate to the most junior person who can be trusted with the task.
102. Give your superiors frequent progress reports. (C)

103. Announce a major reorganization of the department that includes getting rid of
whomever you believe to be "dead wood." (0-)

104. Concentrate more on your people than on the tasks to be clone.

105. Make people feel completely responsible for their work. (0)

106. Never settle for less than a perfect product.

107. Show a concern for the advancement of your people.

108. Fit the job to the person, not the person to the job.

109. Be intolerant of your own mistakes and the mistakes of others.

110. Let your very best people do things pretty much their own way.

111. Be honest in your evaluations of those who are doing poorly. (O)

112. If employees don’t fear you then they won’t give one-hundred percent.
113. Be careful to avoid the company’s "sacred cows".

114. Give some, but very little, positive feedback to keep people trying harder.
115. Make the rules and the process by which you operate explicit.

116. Do not try to do too much too soon.

117. Let employees know their jobs are on the line every day of the week.

118.

119.

120.

IX.

127

Understand the limits of your authority and how "the system" operates.

Know your people well in terms of their strengths, weaknesses, and
aspirations.

Promote open communication.

You are looking for several new projects to tackle. You have a list of possible

projects and desire to pick the best two or three among them. Rate the importance of
the following considerations when selecting projects.

121.

122.

123.

124.

125.

126.

127.

128.

129.

130.

Doing the project should prove to be fun. (S-)

The project will enable me to demonstrate my talents.

The project should attract the attention of the local media. (C-)

The project is of special importance to me personally. (S-)

The risk of making a mistake is virtually nonexistent. (S-)

The project addresses a problem that is recognized to be a serious one.
The scope of the project goes beyond my present responsibilities.

The project will bring favorable attention to my superiors.

The project may enhance cooperation among several departments that have
been too autonomous.

The project will require working directly with several senior executives. (C-)

128

X. Rate the following kinds of experience by their importance in becoming a good
manager.

131. Having completed an M.B.A. degree from a strong business department.
132. Working for several years as a salesperson.

133. Working your way through the ranks of a company from the bottom.

134. Working for a competitor in a similar position.

135. A strong background in ﬁnance.

136. Having worked at a diverse selection of jobs.

137. A strong background in psychology.

138. Having responsibility for things such as paper routes when young.

139. A strong background in the technology used by your company’s production

facilities.

129

XI. In business as in other ﬁelds, there are often several people who are
acknowledged to do extraordinary work. Rate the following characteristics by how
important you believe them to be for the success of these individuals.

140. Unusually creative.

141. Highly self-motivated.

142. Very good at selling their ideas to others.

143. Unusually plentiful resources at their disposal (assistants, facilities, etc.).
144. Tremendous need to achieve.

145. Work unusually long hours.

146. Highly efﬁcient work habits.

147. Socially adept. (O-)

148. Lucky.

149. Willing to take substantial risks.

150. Power hungry.

151. Never one to make a mistake.

152. Always the leader in group situations.

153. Having an extraordinary amount of common sense.

154. Better able than most to grasp and operate in terms of the "big picture," i.e.,
the mission and charter of the company.

155. Having virtually no outside interests -- their job is their life.

XII.

130

Rate the following motivations in terms of their importance as incentives for

pursuing a career in management.

156.

157.

158.

159.

160.

161.

162.

163.

' 164.

165.

166.

I think my abilities are a good match to this career choice.

I enjoy working with people.

I like being a "team player”.

I would like to make a lot of money.

I enjoy responsibility and the power that goes with it.

I enjoy leading others.

I like to be my own boss.

I like a job that I can "leave at the ofﬁce" at the end of the day. (S-)
I am easily bored by routines.

I need to do things my own way.

I want to lead others, but not be lead by them.

APPENDIX H

Situational Judgment Test

SITUATIONAL JUDGMENT TEST
MANAGERIAL ITEMS

For each of the following scenarios, please indicate your responses to the
questions that follow each in the blanks on the scantron sheet you have been
provided. When responding, use the following options:

a = l b = 2 c = 3 d = 4 e = 5
At the completion of an interview of an individual whom you believe could never

qualify to be an acceptable worker for your company, the interviewee asks you about
entry requirements to your company. What response do you give?

a. Give the person your truthful and frank opinion about his/her
qualiﬁcations.
b. Relay the company’s stated employment requirements and discuss the

competitiveness of the process but do not offer your opinion about the
person’s chances of employment.

c. Offer speciﬁc suggestions that you believe will increase his/her
competitiveness.
(1. Refer interviewee to the recruitment ofﬁce of your company where

people can assess the individual’s qualiﬁcations thoroughly.
1. Which of these options would you most liker choose?

2. Which of these options would you least likely choose?

131

132

You and two co-workers are assigned to a project. Co-worker A makes an unwanted
advance to co-worker B. Co-worker B is upset and tells co-worker A that such
conduct is improper. You observe the scene. What would you do?

a.

Since co-worker B handled the situation appropriately, do nothing but
be willing to support the co-worker if needed.

Explain to co-worker A that such behavior is considered harassment
and that you will report it if it does not cease.

Privately pull co-worker A aside and express your concerns that such
behavior may hurt the working relationship and successful completion
of the project.

Advise both co-workers about the importance of not letting this
encounter interfere with conduct of the project.

Tell co-worker A that you consider such behavior unacceptable.
Inform the person to stop it and apologize, or you will urge co-worker
B to ﬁle a complaint.

3. Which of these options would you mos: likely choose?

4. Which of these options would you 153% likely choose?

You are assigned to routine projects with differing deadlines. Several of these
projects develop unanticipated complications and you will not be able to meet all of
the preset deadlines. How would you proceed?

a.

Identify the projects with priorities so that these can be done ﬁrst and
notify your supervisor of the speciﬁc delays and new anticipated
completion dates.

Finish the projects you can on time and request additional help or an
extension for the remainder.

Seek guidance from your supervisor about what to do.

Work overtime as needed to complete as much of the work as possible
on time.

5. Which of these options would you guest likely choose?

6. Which of these options would you m likely choose?

133

You are working on a very important project, and realize that the project requires
more resources than have been budgeted for. You have requested and your supervisor
has refused to provide the additional resources needed. What would you do?

a. Determine alternative methods of doing the task with less resources.

b. Seek input from others who have experienced similar problems on how
to best handle the matter.

c. Discuss with your supervisor the importance of and progress made on
the project, exactly why you need the additional resources, and how the
lack of resources will impact the work. Try to persuade the supervisor
to give you what you need.

d. Put in additional hours as needed so that the project is completed.

e. Request to speak to the next person up the chain of command to request
the resources needed.

7. Which of these options would you meet, likely choose?

8. Which of these options would you 1e_a§t likely choose?

As a project supervisor, others are responsible for securing and keeping inventory of
all project expenses and ﬁndings. You become aware of the fact that the log in which
all expenditures and important activities are recorded is not being kept. What would

you do?

a. Stress to the individual(s) in charge of the log the criticality of keeping
good records.

b. Stop the project and have everyone work on getting the log up to date.
Do not proceed with the work until the log problem is corrected.

c. Determine why the log is not maintained. Put in place procedures to
ensure that the log is kept.

(1. Report your ﬁndings to your supervisor and document your ﬁndings.
e. Reassign the log-keeping responsibility to another worker.
9. Which of these options would you mest likely choose?

10. Which of these options would you least likely choose?

134

You are responsible for setting up electrical equipment for various projects. One of
your team members has performed inconsistently in the past. Your six-member team
was given 48 hours to complete a project. No other replacement personnel are
available and all six members are need. What would you do?

a.

Meet with the inconsistent team member, speciﬁcally discuss what you
want the person to do, and make sure the person understands his
responsibilities.

Closely monitor the individual and carefully check over his work after
the job is done.

Assign the inconsistent worker the easiest tasks and least amount of
responsibility.

Make sure that the worker understands how important his role is in
completing the work and that you need 100% of his effort on this job.

Divide the group into teams of ,workers, assigning the weakest member
with the strongest member.

11. Which of these options would you mag likely choose?

12. Which of these options would you least likely choose?

You are assigned a complicated project, and collect hundreds of boxes of documents
that need to be inventoried, reviewed, and evaluated. You ask your supervisor for
the help of a computer person, but there are none available for a month. You can’t
operate a computer, but you know this is a priority matter. What would you do?

a.

1).

Ask the supervisor to ﬁnd the available help from another source.
Seek funds to hire a temp.

Request training to learn the computer skills that you need.

Solicit help from co-workers more computer literate than yourself.

Have a clerk start the inventory manually until a person with computer
skills becomes available.

13. Which of these options would you most likely choose?

14. Which of these options would you leaﬁ likely choose?

135

You are in the ofﬁce and get a telephone call that requires immediate attention. To
perform effectively, you need to change the conditions of a contract, which
technically needs approval. Your supervisor is out and cannot be reached. If you
change the conditions of the contract, you may be reprimanded later for not having
proper approval. If you do nothing, the contract may be lost. What would you do?

a. Seek approval from the next person in the chain of command.
b. Make a decision about which is more important to the company, the
technical requirement or ensuring that the contract is maintained, and

act accordingly.

c. Do what it takes to close the contract and follow up with documentation
to justify your actions.

d. Follow the procedures and risk the contract if necessary.
15. Which of these options would you most likely choose?
16. Which of these options would you least likely choose?
You are an experienced employee. A new employee comes to you for assistance.
You spend time showing the employee how to do a task. Next month the same thing
happens and you again help the new employee do the same task. This situation
continues and you ﬁnally get upset since the new employee should be able to do the

task alone. How do you proceed?

a. Explain to the person that you do not understand what the problem is
with the task but that you have helped as much as you can.

b. As long as the person was trying, continue to show the person how to
do the task.
c. Ask the employee to take notes or make a copy of the product to use as

a guide in the future for how to perform the task.

d. Inform the employee to pay careful attention because this is the last
time you will demonstrate how to do the task.

e. Sit down with the employee to try to determine what the problem is so
that you can ﬁgure out the best way to deal with the situation from here
on.

17. Which of these options would you moet likely choose?
18. Which of these options would you leg likely choose?

136

You observe another employee performing her job in a questionable manner. The
other employee is assigned to a different group, but her performance could affect the
outcome of your project. On the other hand, if you report the employee, it could
cause friction with the other group and that could affect the long term working
relationship between the two groups. What would you do?

a.

b.

Accept the employee’s deﬁciencies and work around them.

Discuss your problems with the employee, and inform her that if her
performance does not improve you will be forced to report her.

Without making the problem employee suspicious, get another worker
to aid her in getting her work done.

Advise a supervisor of the problem.
Set up a group meeting in which all employees discuss the problems

they have encountered in achieving the group’s goals. Bring up this
employee if necessary.

19. Which of these options would you mogt, likely choose?

20. Which of these options would you leag likely choose?

The supervisor is out of the ofﬁce and an emergency arises that requires the
immediate assignment of personnel. The secretary notiﬁes you of the situation and
explains that the supervisor is out of contact for at least three hours. The supervisor’s
immediate supervisor is also unavailable, but scheduled to return within the hour.
How would you proceed?

3.

Immediately make the assignment of needed personnel, and explain to
the supervisor on his/her return.

Find another supervisor of appropriate status and advise them of the
situation.

Simultaneously continue searching for one of the supervisors, and
notify the personnel likely to be assigned to the emergency.

Call a meeting of the employees to discuss the situation. Collectively
make a decision on what to do.

21. Which of these options would you most likely choose?
22. Which of these options would you least likely choose?

137

You’re working on a deal that involves a lot of paperwork and record reviewing with
another employee. You take leave and come back to ﬁnd that your desk is ﬁlled with
more records. The employee you were working with re-arranged your papers and
desk, and you can’t tell what was already reviewed and what you’ve done. You’re
angry. What would you do?

3. Take a few minutes to cool down and then request that the employee
report what progress has been made in written and oral form.

b. Ask the co-worker if he/she knew where you had left off since the
records are now shifted.

c. Inform the employee that you are angry for what he/she has done and
tell the employee that in the future you do not want him/her to touch

your desk.

d. Assume that the employee has no bad intentions, and try to re-sort the
piles.

e. Realize it is your fault for leaving the ﬁles unattended on your desk.

23. Which of these options would you mes_t likely choose?
24. Which of these options would you m likely choose?

A senior, highly commended co-worker has recently experienced some personal
difﬁculties. He has conﬁded these problems only to you. You have experienced an
increased workload because of his problems. You have talked to him about your
concerns, and empathetically requested that he resume full duties as soon as possible.
A month passes and you are still doing too much of his work. What would you do?

a. Inform your co-worker that you understand his problems, but are no
longer able to perform his work for him.

b. Ask your co-worker to take vacation, sick days, or a leave of absence
until he has resolved his problems.

c. Continue to inform your co-worker of your concerns until he resumes
his full job duties.

d. Talk to your supervisor about the situation.
e. Report your co- -worker’ s actions to his supervisor.

25. Which of these options would you ms], likely choose?
26. Which of these options would you Ml Likely choose?

138

After delegating work to several teams of employees, you subsequently obtain all of
the necessary paperwork/reports from all of the teams, except for one. You
repeatedly approach the person responsible for the work and on each occasion, he
gives you a general excuse and promises to produce them. As you come closer to
concluding the project, you are requested to provide the paperwork/reports. What
would you do?

a. I would advise the person that all other teams have turned in their
paperwork in a timely and professional manner, but his excuses and
delays are undermining the total effectiveness of the project.

b. Confront this person, and threaten to report it to the supervisor.

c. Tell the person that the paperwork is due now, but that you will help
him if needed.

d. Simply turn in the reports that you have.

e. Try to ﬁnish the report yourself, making a notation that this employee
was unreliable.

27. Which of these options would you mest likely choose?
28. Which of these options would you le_a_st likely choose?

You have been told to work with a vice-president of the company to assist you in
negotiating a contract. You have called the VP to try to set up a meeting. The
morning of the meeting she calls to cancel, but tells you that she will call to
reschedule your meeting. A week goes by and you do not hear from the VP. You
call again and leave messages. The VP does not call. What would you do?

a. You would advise your supervisor of the problem, and ask for his or
her input.

b. Call the VP’s secretary to have her re-schedule the meeting.

c. Ask that someone else be assigned to this case.

(1. Request the VP to re-schedule a date in writing.

e. Physically confront the VP.

29. Which of these options would you meg likely choose?
30. Which of these options would you least likely choose?

139

You are the team leader for a large project with ﬁve others assigned to assist you.
Four of the ﬁve have completed their assignments and turned them in on time. The
other individual is continually advising you the work is forthcoming and that there
must be a problem in the steno pool. The deadline for the project is 9:00 am.
tomorrow. At 2 pm, you check with the steno pool and are advised they have yet to
receive any work from this individual. What would you do?

a.

C.

You would call the co-worker and advise him/her that you know that
he/ she has not turned in any work to the steno pool, and to get it there
immediately.

Go to a superior and advise him/her of the situation.

Take any necessary action to get the work done, and then report it to a
superior.

Help the individual as necessary, and remove this person from any
future team projects.

Turn in the incomplete project as is.

31. Which of these options would you mes; likely choose?

32. Which of these options would you least likely choose?

You have been involved in a long project that is rather unique in scope. The project
is far from being ﬁnished. When your new supervisor begins to interject his ideas
and then begins to force you to adopt these suggestions, what do you do?

a.

e.

You would do as he suggests because he is your superior, but document
the times you disagreed.

You would explain your vision of the project, and stand ﬁrm with the
direction you think it should be headed in.

Explain your point of view, and try to reach a compromise.

Explain that you have been working on this longer than him, and you
are best qualiﬁed to determine the direction of the project.

You would talk to his supervisor about your concerns.

33. Which of these options would you most likely choose?
34. Which of these options would you least likely choose?

140

One of your friends calls you to ask for information about prices paid for your
company’s products and services. You know that this is private information. What
would you do?

a. Refuse, explaining that this is not ethical.

b. Give them the name of someone else who might help them.

c. Since you are friends, you feel you can trust him/her, so you supply
the information.

d. Refuse, and then report your friend to the authorities if this solicitation
was illegal.

e. Tell him/her what you can without going against the rules/law.
35. Which of these options would you w likely choose?
36. Which of these options would you 1&5! likely choose?
For the past three weeks you have been working on a presentation to a potential client
interested in a product that will support their needs. The presentation is scheduled for
the next day. Then another person in your group suggests to you that your

presentation might be too heavily technical to be understood by the client. What
would you do?

a. Work all night to adjust the presentation to the expertise level of the
client.
b. Decide on your own if the client will understand the presentation, then

proceed from there.

c. Ask for help from the person who made the suggestion in revising the
presentation.
(1. Conduct a question and answer period after the presentation to clarify

any technical terms or concepts.
e. leave the presentation as it is, and proceed as planned.
37. Which of these options would you mog likely choose?

38. Which of these Options would you least likely choose?

141

Two of your peers (you are not their manager but you all work for the same manager)
dislike each other and are constantly arguing. Their attitude toward each other
creates an uncomfortable feeling of hostility and tension. What would you do?

a.

b.

d.

C.

You advise the supervisor after talking with your peers.

You would do nothing because it is not your responsibility. Only the
supervisor should handle this problem.

You suggest to your peers that they talk with the manager about their
conﬂicts.

You try to mediate their differences on your own.

You let them handle it themselves because they are adults.

39. Which of these options would you meg likely choose?

40. Which of these options would you Leget likely choose?

One of your projects is experiencing a variety of problems that threaten to make it
slip behind schedule. Others working on this project with you want to delay it
because of the problems. Another option would be to work overtime and keep the
project on schedule. What would you do?

a.

You base your decision on the importance of this deadline in
comparison to your other projects and responsibilities.

You work overtime to complete the project by the deadline.
You have everyone vote on what should be done.

You deﬁne the problems and ﬁnd solutions, working overtime only if
necessary.

41. Which of these options would you meet likely choose?

42. Which of these options would you le_a.ﬁ likely choose?

142

You have volunteered to serve on the United Way Committee. The person in charge
of the committee is disinterested in it. He frequently arrives late for meetings
discussing personal matters, and complains that his talents are wasted there. What
would you do?

a. You convince him to resign and take his position or get someone more
interested assigned.

b. You approach the leader and offer assistance with some aspects of the
project in an attempt to motivate him.

c. You ask him if you could head the drive while reporting to him.
(1. You threaten to stop helping unless he shows more interest.
43. Which of these options would you ms; ﬁleely choose?
44. Which of these options would you his; l_ik_ely choose?
Your department has been following a particular procedure for the past two years.
Lately though, some people have been complaining about it. Your manager asks you

to revise the procedure. What would you do ﬁrst?

a. You talk to members and get their suggestions to come up with the best
possible procedure.

b. You ask everyone to list their complaints and revise the current
procedure accordingly.

c. You evaluate the existing procedure and implement changes you believe
will resolve the problem.

d. You review the complaints with the people involved to see if their
complaints are valid before proceeding to suggest any revisions.

45. Which of these options would you mag likely choose?

46. Which of these options would you lees: likely choose?

 

143

You work with people who are not as well educated as you and who seem to resent
your presence there. As a result, you are ﬁnding it hard to get their cooperation
when you need it. What would you do?

a.

b.

You play dumb and try not to appear educated.

You try to talk with them to resolve the problem and stress the
importance of teamwork.

You make sure that you do not talk above their level or talk at them to
facilitate communication.

You keep working hard so that you can gain their respect.

You try to be their friend and create the impression that you are all
alike.

47. Which of these options would you mag likely choose?

48. Which of these options would you PM likely choose?

Your manager gives you a major assignment and spells out in a general way how you
should approach it. As you begin, you become convinced that your manager’s
method is inefﬁcient and you want to do it another way. What would you do?

a.

You do it your way convinced that the end result is the only important
consideration.

You develop your way and approach the manager with a detailed
reason why you are heading in a direction opposed to his.

You use some of the manager’s ideas and your own.

You use your best judgment to get the project done the best way you
can.

You make suggestions and act on your manager’s opinions after
considering the suggestions.

49. Which of these options would you most likely choose?

50. Which of these options would you le_as1 likely choose?

144

You just spent two days working on a very sensitive letter to a customer. You gave it
to your manager for review. A couple of hours later she gives it back to you with
extensive suggestions for revision, primarily involving editorial changes that reﬂect
her preferred style of letter writing. What would you do?

a.

d.

C.

If you feel the letter is accurate, you let the letter go out as is; that is,
you do the letter your way.

You tell her the sensitivity of the letter and that it needs to be said in
your style of writing and not hers.

You change the letter, but explain why you wrote it as you did.
You try to compromise and do some of her suggested revisions.

You make the changes, but have the manager sign her name to it.

51. Which of these options would you meet likely choose?

52. Which of these options would you least likely choose?

You are working on several important assignments simultaneously. People are
counting on you to complete them when you promised. However, events beyond
your control make it impossible for you to ﬁnish them all on time. What would you

do?

You prioritize the assignments, knowing that some will not get done.

You discuss the problem with each person involved and explain why
the projects won’t be done on time.

If you cannot get extensions to the deadlines on these projects, ask for
help.

You ﬁnish the assignments when you can.

You meet with those involved, explain the situation and request their
input and assistance.

53. Which of these options would you mogt likely choose?

54. Which of these options would you least likely choose?

...1“

145

You are part of a three-person team working on a project with a short deadline. One
member of the team is not pulling her weight. She avoids assignments, complains
about the amount of work that has to be done, and says the project doesn’t really
matter anyway. All three of you are at the same level, but you are the team leader.
What would you do?

a.

b.

C.

d.

C.

You discharge her and get a replacement.

You try to talk to that person and inform the supervisor of the problem.
You ask her to do her share of the work or leave.

You let her know the importance of her participation with the project.

As the team leader, you tell her to stop complaining and do her job.

55. Which of these options would you mag likely choose?

56. Which of these options would you m likely choose?

You are a member of a team that just completed a project. The person for whom the
project was done calls you to complain about an error. He is very upset, but the
error is not related to the part of the project for which you were personally
responsible. What would you do?

a.

(1.

You bring it to the attention of the person that was responsible and
have that person contact the complainant if you cannot correct the
problem yourself.

You tell the person responsible for the error that he should contact the
team leader.

You tell the person for whom the project was done that you will handle
the problem and then work with the person responsible for the error to
remedy the situation.

You take the responsibility, try to correct the error and apologize.

57. Which of these options would you most likely choose?

58. Which of these options would you le_as;t likely choose?

 

146

You have just made a formal presentation to a group of people at your own
supervisor’s level. One of them tells you that if you worked for her, she would never
allow you to speak before a group because your presentation was totally inept. What
would you do?

a.

You consider comments from other as well which may discount her
criticisms.

You smile and thank her for her remarks.
You ask her for speciﬁcs as to why your presentation was poor.
You take the criticism constructively and use it to improve next time.

You talk to your supervisor about the comment.

59. Which of these options would you most likely choose?

60. Which of these Options would you l_ees_t likely choose?

u 1“.”

APPENDIX I

Experience Measure

Experience Questionnaire

For each of the following questions, please place your answer on the scantron sheet
provided. Be sure to match the number of the items to the numbers on the scantron
sheet, and begin with number [1] on your scantron.

[Work Exmriencel

1.

How many jobs (both part-time and full-time) have you held since you began
high school?

1. None 4. 6-10
2. 1-2 5. More than 10
3. 3-5

How many of your jobs have been in service-related ﬁelds (for example, food
service or retail organizations)?

1. None 4. 6-10
2. 1-2 5. More than 10
3. 3-5

How many of your jobs have been clerical jobs (for example, secretarial
work)?

1. None 4. 6-10
2. 1-2 5. More than 10
3. 3-5

How many of your jobs have been in construction or manufacturing ﬁelds (for
example, working on the line at a plant)?

1. None 4. 6-10
2. 1-2 5. More than 10
3. 3-5

147

148

5. How many of your jobs have been in ﬁelds other than those listed in questions
[2, 3, and 4] above?

1. None 4. 6-10
2. 1-2 5. More than 10
3. 3-5
6. In how many of your jobs have you held a position designated by your

employer as a "management" position?

1. None 4. 6-10
2. 1-2 5. More than 10
3. 3-5

Please use the following options to respond to the next set of questions, using the
scantron sheet to record your answers. If you are currently employed, please respond
to questions [7-11] with respect to your current job; if not, please respond with
respect to the last job you held.

If the question does not apply to you, please enter a 7.

 

 

 

 

 

6 5 4 3 2 1
Very Much A Large A Moderate A Small A Very Small Not Much
Amount Amount Amount Amount At All
7. How much responsibility were you given in your most recent or current job?
8. To what extent were you responsible for training new employees in your most

recent or current job?

9. In your most recent or current job, how responsible were you for deciding on
how to proceed in performing the tasks required of you?

10. In your most recent or current job, how much emphasis was put on your duties
with respect to customer service?

149

11. In your most recent or current job, how much interaction did you have with
members of management?

12. Over the course of your last three jobs, how much responsibility have you
been given?

 

 

 

 

6 5 4 3 2 1
Very Much A Large A Moderate A Small A Very Small Not Much
Amount Amount Amount Amount At All

If the question does not apply to you, please enter a 7.

13. Over the course of your last three jobs, how much responsibility have you
been given for training new employees?

14. Over the course of your last three jobs, how much responsibility have you
been given for deciding how to proceed in performing the tasks required of
you?

15. Over the course of your last three jobs, how much emphasis has been put on
your duties with respect to customer service?

16. Over the course of your last three jobs, how much interaction have you had
with members of management?

| School Experience |

The next set of questions will ask you about your experiences in high school and
college. When answering these questions, consider academic clubs to be any club
focused on issues of academic content (for example, Math or Science Clubs) as well
as any club whose requirements for entry were certain academic standards (like
National Honor Society). Consider extracurricular activities to be those for which
grades may or may not have been given (such as band or athletics) but whose primary
purpose was the development of something other than a strict "academic" ability.
Examples are given to help you determine where various types of activities ﬁt best, so
be certain to read the questions carefully.

1. How many academic clubs (for example, National Honor Society) were you a
member of in high school?

1. None 4. 6-10
2. 1-2 5. More than 10
3. 3-5

150

How many extracurricular activities (for example, band, athletics, or drama)
were you involved with in high school?

1. None 4. 6-10
2. 1-2 5. More than 10
3. 3-5

For how many years in high school were you involved in at least one
extracurricular activity from question [2] above?

1. None 4. 3 years
2. 1 year 5. 4 years
3. 2 years

To how many leadership positions were you elected in the clubs and other
organizations you were involved with in high school?

1. None 4. 6-10
2. 1-2 5. More than 10
3. 3-5

How many academic clubs (for example, Physics Club, Phi Beta Kappa, etc.)
have you been a member of since beginning college?

1. None 4. 6-10
2. 1-2 5. More than 10
3. 3-5

How many extracurricular activities (for example, fraternities or sororities)
have you been involved in since entering college?

1. None 4. 6—10
2. 1-2 5. More than 10
3. 3-5

For how many years in college have you been involved in at least one
extracurricular activity from question [6] above?

1. None 4. 3 years
2. 1 year 5. 4 or more years
3. 2 years

151

8. To how many leadership positions have you been elected in the clubs and other
organizations you’ve been involved in since entering college?

1. None 4. 6-10
2. 1-2 5. More than 10
3. 3-5

Please use the following options to respond to the next set of questions, using the
scantron sheet to record your answers.

 

 

 

6 5 4 3 2 1
Very Much A Large A Moderate A Small A Very Small Not Much
Amount Amount Amount Amount At All

If the question does not apply to you, please enter a 7.

9. How much responsibility did you have for organizing the activities of the clubs
and organizations you were involved in during high school?

10. How much responsibility have you had for organizing the activities of the
clubs and organizations you have been involved in since entering college?

11. How much interaction did you have with the faculty sponsor(s) of the clubs
and organizations you were involved in during high school?

12. How much interaction have you had with the faculty sponsor(s) of the clubs
and organizations you have been involved in since entering college?

13. How much responsibility did you have, in general, in the clubs and
organizations in which you were involved in high school?

14. How much responsibility have you had, in general, in the clubs and
organizations you have been involved in since entering college?

[Church/Commqu Experience]

1. How many social groups have you been involved in at your church/place of
worship?

1. Does not apply 4. 3-5
2. None 5. 6-10
3. 1-2 6. More than 10

 

152

2. How many non-religious community groups (for example, Neighborhood
Watch, Big Brothers/Sisters) have you been involved in?

1. None 4. 6-10
2. 1-2 5. More than 10
3. 3—5
3. How many leadership positions have you held on committees organized by

members of your church/place of worship?

1. Does not apply 4. 3-5

2. None 5. 6-10
3. 1-2 6. More than 10
4. How many leadership positions have you held in non-religious community
groups?
1. None 4. 6-10
2. 1-2 5. More than 10
3. 3-5
5. How many times have you served as a camp counselor?
1. None 4. 6-10
2. 1-2 5. More than 10
3. 3-5

Please use the following options to respond to the next set of questions, using the
scantron sheet to record your answers.

 

 

 

 

 

6 5 4 3 2 1
Very Much A Large A Moderate A Small A Very Small Not Much
Amount Amount Amount Amount At All

If the question does not apply to you, please enter a 7.

6. How much responsibility have you had in the social groups in which you have
been involved at your church/place of worship?

7. How much responsibility have you had in the community groups in which you
have been involved?

8. How much responsibility have you had in the volunteer groups in which you
have been involved?

153

9. How much responsibility have you had when serving on committees organized
by individuals in your church/place of worship?

10. How much responsibility have you had when serving as a camp counselor?

11. How much interaction have you had with the leaders of social groups (other
than yourself) in your church/place of worship?

 

 

 

 

 

6 5 4 3 2 1
Very Much A Large A Moderate A Small A Very Small Not Much
Amount Amount Amount Amount At All

If the question does not apply to you, please enter a 7.

12. How much interaction have you had with the leaders of community groups
(other than yourself) in which you have been involved?

 

13. How much interaction have you had with committee chairpersons (other than
yourself) in your church/place of worship?

14. How much interaction did you have with the camp director and his/her upper-
level assistants while serving as a camp counselor?

APPENDIX J

Self-Regulation Measure
(adapted from Ashford & Tsui, 1991)

Please use the following scale to respond to the following questions. For each
question, consider how much the statement applies to your behavior in school and
work environments over the past year.

 

 

 

6 5 4 3 2 1

Always True Usually Sometimes Infrequently Rarely Never
True True True True True

1. Negative feedback stimulates me to consider how I can perform better.

2. I often think about how I can improve my performance even when no one has

offered me feedback.
3. I pay attention to the work I’m doing to see if it can be improved.
4. I often think about ﬁnding feedback from informal sources.

5. When the task I’m working on does not seem to be going well, I like to ﬁgure
out what’s going on.

154

10.

11.

12.

13.

14.

APPENDIX K

Learning Style Measure
(Romero, Tepper, & Tetrault, 1992)

Iwould describe myself 1 2 3 4 5 6 Iwould describe myself as

as impartial (open-
minded)

I would describe myself 1
as reﬂective

I like to be speciﬁc 1
I value patience I

I like things to be varied 1
and colorful

I would describe myself 1
as a doer

I take a creative and 1
imaginative approach to
solving problems

I feel good when I i
understand

I like to stay ﬂexible I
(not get too focused)

I am good at getting 1
things accomplished

I would describe myself 1
as evaluative and logical

I like to watch what is 1
going on

I strive for versatility 1

I am reserved 1

155

explicit (deﬁnite)

I would describe myself as
action-oriented

I like to remain ﬂexible

I value getting things done

I like things to be exact and
precise

I would describe myself as an
observer

I take a precise and calculated
approach to solving problems

I feel good when I have an

impact on things

I like to get as focused as
possible

I am good at seeing things from
many perspectives

I would describe myself as
receptive and accepting

I like to see the results of my
actions

I strive for accuracy

I am prepared

 

 

156

note: Odd-numbered items reﬂect the concreteness (L) vs. abstractness (R)
dimension, while even-numbered items reﬂect reﬂection (L) vs. action (R); 3, 6, 10,
& 11 are reverse-scored

APPENDIX L

LISREL 8 Control Statements

1. Model Without Equality Constraints

Initial Thesis Analysis -- Micro Models

observed variables=sjt tkpos intrprs commskil probsolv selfpos carpos
othpos tolamb commcomp machscor workexp edexp selfreg
concrete action extrav open consc

correlation matrix from ﬁle thesisS.cor

standard deviations from ﬁle thesis2.sd

sample size=348

selfpos=selfreg consc concrete action

othpos=machscor edexp extrav commcomp

carpos=workexp open tolamb

commskil=extrav commcomp edexp

intrprs =machscor extrav edexp

probsolv=workexp consc open tolamb

let the errors between selfpos and probsolv correlate

let the errors between othpos and intrprs correlate

let the errors between othpos and commskil correlate

let the errors between carpos and probsolv correlate

let the errors between selfpos and othpos correlate

let the errors between selfpos and carpos correlate

let the errors between carpos and othpos correlate

let the errors between commskil and intrprs correlate

let the errors between commskil and probsolv correlate

let the errors between probsolv and intrprs correlate

path diagram

lisrel output rs ef mi va so

157

 

158
11. Model With Equality Constraints

Final Thesis Analysis -- Micro Models

observed variables=sjt tkpos intrprs commskil probsolv selfpos carpos
othpos tolamb commcomp machscor workexp edexp selfreg
concrete action extrav open consc

correlation matrix from ﬁle thesisS.cor

standard deviations from ﬁle thesi52.sd

sample size=348

relationships:

intrprs =machscor extrav edexp

commskil=extrav commcomp edexp

probsolv=workexp consc Open tolamb

selfpos=selfreg consc concrete action

carpos=workexp open tolamb

othpos=machscor edexp extrav commcomp

set tolamb -> carpos = tolamb -> probsolv

set commcomp - > othpos = commcomp - > commskil

set machscor - > othpos = machscor - > intrprs

set workexp -> carpos = workexp -> probsolv

set edexp - > intrprs = edexp - > othpos = edexp - > commskil

set extrav - > othpos = extrav - > intrprs = extrav - > commskil

set Open -> carpos = open -> probsolv

set consc -> selfpos = consc -> probsolv

let the errors between selfpos and probsolv correlate

let the errors between othpos and intrprs correlate

let the errors between othpos and commskil correlate

let the errors between carpos and probsolv correlate

let the errors between selfpos and othpos correlate

let the errors between selfpos and carpos correlate

let the errors between carpos and othpos correlate

let the errors between commskil and intrprs correlate

let the errors between commskil and probsolv correlate

let the errors between probsolv and intrprs correlate

path diagram

lisrel output: rs ef mi va sc

References
Ashford, S. J., & Tsui, A. S. (1991). Self-regulation for managerial
effectiveness: The role of active feedback seeking. Academy of Management
Journal, 34, 251-280.
Baldwin, T. T., & Padgett, M. Y. (1993). Management development: A

review and commentary. International Review of 1/9 Psychelogy, 8, 35-85.

Bandura, A. (1991). Social cognitive theory of self-regulation.
Organizational Behavior and Human Decision Processes, 50, 248-287.

Baron, R. A., & Byme, D. (1991). Social psychology: Understanding
human interaction (6th edition). Boston: Allyn & Bacon.

Barrick, M. R., & Mount, M. K. (1991). The Big Five personality
dimensions and job performance: A meta-analysis. Personnel Psychelogy, 44, 1-26.

Bollen, K. A., & Long, J. S. (Eds). (1993). Testing structural eguatipn
meg]; Newbury Park, CA: Sage.

Borkenau, P. (1988). The multiple classiﬁcation of acts and the big ﬁve
factors of personality. Journal of Research in Personality, 22, 337-352.

Bray, D. W., & Grant, D. L. (1966). The assessment center in the
measurement of potential for business management. Psycholpgical Monographs, 80,
(17, Whole No. 625).

Broadbent, D. E. , & Aston, B. Human control of a simulated economic

system. Ergonomics, 21, 1035-1043.

159

160
Bruce, M. M., & Learner, D. B. (1958). A supervisory practices test.

Personnel Psychology, 11, 207-216.
Carraher, S. M. (1993). Another look at the dimensionality of a learning

style questionnaire. Educational and Psychological Measurement, 53, 411-415.
Cascio, W. F., & Phillips, N. (1979). Performance testing: A rose among

thorns? Personnel Psychology, 32, 751-766.
Ceci, S. J., & Liker, J. K. (1986). A day at the races: A study of IQ,

expertise, and cognitive complexity. Journal of Exmrimental Psychology: General,
115, 255-266.

Chan, D., & Schmitt, N. (1997). Video-based versus paper-and-pencil
method of assessment in situational judgment tests: Subgroup differences in test
performance and face validity perceptions. Journal of Applied Psycholpgy, 82, 143-
159.

Cohen, J. (1992). A power primer. Psychological Bulletin, 112, 155-159.

Cohen, R. J ., Swerdlik, M. E., & Smith, D. K. (1992). Psycholpgical
testing and assessment: An introduction to tests and measurement (2nd edition).
Mountain View, CA: Mayﬁeld.

Cornelius, S. W., & Caspi, A. (1987). Everyday problem solving in
adulthood and old age. Psychology and Aging, 2, 144-153.

Costa, P. T. Jr., & McCrae, R. R. (1995). Domains and facets:
Hierarchical personality assessment using the revised NEO personality inventory.
m1 of Personelitv Assessment. 64, 21-50.

Cronbach, L. J. (1947). Test "reliability": Its meaning and determination.

161
Psychometrika, 12, 1-16.

Cronbach, L. J. (1951). Coefﬁcient alpha and the internal structure of tests.
Psychometrika, 1e, 297-334.

Cronbach, L. J ., & Meehl, P. E. (1955). Construct validity in psychological
tests. Psychological Bulletin, 52, 281—302.

Dalessio, A. T. (1994). Predicting insurance agent turnover using a video-
based situational judgment test. Journal of Business and Psychelogy, 2, 23-32.

Denney, N. W., & Palmer, A. M. (1981). Adult age differences on
traditional and practical problem-solving measures. Journal of Gerontology, 36, 323-
328.

Flavell, J. H. (1979). Metacognition and cognitive monitoring: A new area
of cognitive-developmental inquiry. American Psychologist, 34, 906-911.

Frederiksen, N. (1986a). Construct validity and construct similarity:
Methods for use in test deveIOpment and test validation. Mpltivariate Behavior
Research, 21, 3-28.

Frederiksen, N. (1986b). Toward a broader conception of human
intelligence. American Psychologist, 41, 445-452.

Freedheim, D. K. (1969). Editorial: Not another journal! Brofessioml

Psycholegy: Research and Practice, 1, 1-3.

Frenkel-Brunswick, E. (1948). Intolerance of ambiguity as an emotional and

perceptual personality variable. Journal of Persenality, 18, 108-123.

Fumham, A. (1994). A content, correlational and factor analytic study of

four tolerance of ambiguity questionnaires. Personality and Individual Differences,

162
1_6_, 403—410.

Gaugler, B. B., Rosenthal, D. B., Thornton, G. C. III, & Bentson, C.
(1987). Meta-analysis of assessment center validity. Jeurnal of Applied Psychology
(monograph), 12, 493-511.

Geiger, M. A., Boyle, E. J., & Pinto, J. K. (1993). An examination of
ipsative and normative versions of Kolb’s revised learning style inventory.

Educational and Psychological Measurement, 53, 717-726.
Gottfredson, L. S. (1986). Foreword: The g factor in employment. Journal

Of Vocational Behavior, 29, 293—296.
Hothersall, D. (1995). Histog of psychology (3rd edition). New York:

McGraw-Hill.
Hunter, J. E. (1986). Cognitive ability, cognitive aptitudes, job knowledge,

and job performance. Journal of Vocational Behavior 29, 340-362.

 

Hunter, J. E., & Hunter, R. F. (1984). Validity and utility of alternate
predictors of job performance. Psychological Bulletin, 96, 72-98.

Jensen, A. R. (1986). g: Artifact or reality? Journal of Vocational
Behavior, 22, 301-331.

Kanfer, R., & Ackerman, P. L. (1989). Motivation and cognitive abilities:
An integrative/aptitude-treatrnent interaction approach to skill acquisition. Journal of
Applied Esyehplogy Monograph, 74, 657-690.

Klaczynski, P. A., Laipple, J. S., & Jurden, F. H. (1992). Educational

context differences in practical problem solving during adolescence. Merrill-Palmer

Quarterly, 38, 417-438.

163
Kolb, D. A. (1976). The learning sgle inventeg: Technical manual.

Boston: McBer.

Kolb, D. A. (1985). Ihe learning style inventog: Technical manual.
Boston: McBer.

Larson, G. E., & Wolfe, J. H. (1995). Validity results for g from an
expanded test base. Intelligence, 20, 15-25.

Lave, J ., Murtaugh, M., & de la Rocha, O. (1984). The dialectic of
arithmetic in grocery shopping. In B. Rogoff & J. Lave (Eds), ve da co niti 11
(pp. 67-94). Cambridge, MA: Harvard University Press.

Lewicki, P., Czyzewska, M., & Hoffman, H. (1987). Unconscious

acquisition of complex procedural knowledge. Journal pf Exmrimental Psychology:
Learning, Memog, and Cognition, 13, 523-530.

Major, D. A. (1990). Differential effects of interaction content and proaction

on organizational socialization outcomes. Unpublished Master’s Thesis: Michigan
State University.
Mathews, R. C., Buss, R. R., Stanley, W. B., Blanchard-Fields, F., Cho, J.

R., & Druhan, B. (1989). Role of implicit and explicit processes in learning from

examples: A synergistic effect. Jouml of Experimental Psychology: Learning,
Memog, and Cognition, 15, 1083-1100.

McCroskey, J. C. (1970). Measures of communication bound anxiety.
Speech Monographs, 37, 269-277.

McCrae, R. R., & Costa, P. T. Jr. (1987). Validation of the ﬁve-factor

model of personality across instruments and Observers. Journal of Personalig and

 

164
Social Psychology, 52, 81-90.

Motowidlo, S. J., Dunnette, M. D., & Carter, G. W. (1990). An alternative
selection procedure: The low-ﬁdelity simulation. Jour_rg_1i of Applied Psychology, 6,
640-647.

Motowidlo, S. J ., & Tippins, N. (1993). Further studies Of the low-ﬁdelity
simulation in the form of a situational inventory. Joumal of Occupational and
Organizetional Psychology, 66, 337-344.

Mudrack, P. E., & Mason, E. S. (1995). Extending the Machiavellianism
construct: A brief measure and some unexplored relationships. Journal ef §ociel
Beﬂwior apd Personality, 10, 187-200.

Murphy, K. R., & Davidshofer, C. O. (1994). Psychological testing:
Principles and applicatipns. Englewood Cliffs, NJ: Prentice Hall.

Neisser, U. (1976). General, academic, and artiﬁcial intelligence. In L.
Resnick (Ed.), The nature of intelligence (pp. 135-144). Hillsdale, NJ: Erlbaum.

Neisser, U. (1979). The concept of intelligence. Intelligence, 5, 217-227.

Neisser, U., Boodoo, G., Bouchard, T. J. Jr., Boykin, A. W., Brody, N.,
Ceci, S. J., Halpern, D. F., Loehlin, J. C., Perloff, R., Stemberg, R. J., & Urbina,
S. (1996). Intelligence: Knowns and unknowns. American Psychologist, 51, 77-
101.

Norman, W. T. (1963). To'ward an adequate taxonomy of personality
attributes: Replicated factor structure in peer nomination personality ratings. Jeer_n_al

of Abnormal end Social Psychology, 66, 574-583.

Nunnally, J. C. Jr. (1959). Tests and measurements: Assessment and

165
prediction. New York: McGraw-Hill.

Polanyi, M. (1976). Tacit knowledge. In M. Marx and F. Goodson (Eds.),
Theories in contemporag psychology (pp. 330-344). New York: Macmillan.

Pulakos, E. D., Schmitt, N ., & Keenan, P. (1994). Validation and
implementation Of the FBI Smial Agent Ema-Level Selection System. Final Report
(HUMRRO FR-PRD-94-20). Alexandria, VA: HumRRO.

Reber, A. S. (1989). Implicit learning and tacit knowledge. Journal pf
Experimental Psychology: General, 118, 219-235.

Reber, A. 8., & Lewis, S. (1977). Implicit learning: An analysis of the
form and structure of a body of tacit knowledge. Cpgnition, 5, 333-361.

Reber, A. S., Walkenfeld, F. F., & Hernstadt, R. (1991). Implicit and
explicit learning: Individual differences and IQ. Journal of Experimental
Psychology: Learning, Memog, and Cognition, 17, 888-896.

Romero, J. E., Tepper, B. J ., & Tetrault, L. A. (1992). Development and
validation of new scales to measure Kolb’s (1985) learning style dimensions.
Educational and Psychological Measurement, 52, 171-180.

Rubin, R. B. (1986). The validity of the communication competency
assessment instrument. Communication Monographs, 52, 173-185.

Rynes, S. L. (1992). Who’s selecting whom? Effects Of selection practices
on applicant attitudes and behavior. In N. Schmitt & W. C. Borman (Eds.),
Personnel selection in organizations (pp. 240-274). San Francisco: Jossey-Bass.

Schmidt, F. L., Greenthal, A., Hunter, J. E., Bemer, J., & Seaton, F.

(1977). Job sample vs. paper-and-pencil trades and technical tests: Adverse impact

166
and examinee attitudes. Personnel Psychology, 30, 187-197.

Schmitt, N., Rogers, W., Mullins, M. E., & Clause, C. S. (In Progress).

Scribner, S. (1986). Thinking in action: Some characteristics of practical
thought. In R. J. Stemberg and R. K. Wagner (Eds.), Practical Intelligence: Nature
and origins of competence in the evetyday world (pp. 13-30). New York:
Cambridge University Press.

Seger, C. A. (1994). Implicit learning. Psychological Bulletin, 115, 163-
196.

Smither, J. W., Reilly, R. R., Millsap, R. E., Pearlman, K., & Stoffey, R.
W. (1993). Applicant reactions to selection procedures. Personnel Psychology, 46,
49-75.

Spearman, C. (1927). The abilities of man. New York: Macmillan.

Steiger, J. H. (1980). Tests for comparing elements of a correlation matrix.
Psychological Bulletin, 87, 245-251.

Stemberg, R. J ., & Wagner, R. K. (1992). Tacit knowledge: An unspoken
key to managerial success. journal name here, i, ??-??.

Stemberg, R. J., & Wagner, R. K. Practical intelligence and tacit knowledge.
Manuscript submitted for publication.

Stemberg, R. J., Wagner, R. K., & Okagaki, L. (1993). Practical
intelligence: The nature and role of tacit knowledge in work and at school. In J. M.
Puckett and H. W. Resse (Eds.), Mechanisms of everyday cogpitipn (pp. 205-227).
Hillsdale, NJ: Lawrence Erlbaum.

Stemberg, R. J., Wagner, R. K., Williams, W. M., & Horvath, J. A.

 

167
(1995). Testing common sense. American Psychologist, 50, 912-927.

Thornton, G. C. III, & Cleveland, J. N. (1990). Developing managerial
talent through simulation. American Psychologist, 45, 190-199.

Thurstone, L. L. (1938). Pm eiy mentel abilities, Chicago: University of
Chicago Press.

Tsui, A. S., & Ashford, S. J. (1994). Adaptive self-regulation: A process

view of managerial effectiveness. Journal pf Management, 20, 93-121.

Undheirn, J. O. (1981). In intelmméﬁowndrﬁaheﬂg havef Exam.

(1992). Milwaukee, WI: Wonderlic Personnel Test, Inc.
Veres, J. G. 111, Sims, R. R., & Shake, L. G. (1987). The reliability and
classiﬁcation stability of the learning style inventory in corporate settings.

Educational and Psychological Measurement, 47, 1127-1133.

Wagner, R. K. (1994). Practical intelligence. In R. J. Stemberg (Ed.),

Encyclopedia Of human intelligence, Volume 2 (pp. 821-828). New York:
Macmillan.

Wagner, R. K., & Stemberg, R. J. (1985). Practical intelligence in real-
world pursuits: The role of tacit knowledge. Journal of Personalig and Social
Psychology, 49, 436-458.

Wagner, R. K., & Stemberg, R. J. (1986). Tacit knowledge and intelligence

in the everyday world. In R. J. Stemberg and R. K. Wagner (Eds.), Practical

 

Intelligence: Nature and origins of competence in the evetyday world (pp. 51-83).

 

168
New York: Cambridge University Press.

Wagner, R. K., & Stemberg, R. J. (1991). Tacit knowledge inventory for
managers. San Antonio, TX: Psychological Corporation.
Weekley, J. A., & Jones, C. (1997). Video-based situational testing.

Personnel Psychology, 50, 25-50.

Wernimont, P. F., & Campbell, J. P. (1968). Signs, samples, and criteria.
Journal of Applied Psychology, 52, 372-376.
Wolfe, D. M., & Kolb, D. A. (1984). Career development, personal growth,

and experiential learning. In D. A. Kolb, I. R. Rubin, & J. M. McIntyre (Eds.),

Organizational psychology: Fourth edition. New Jersey: Prentice-Hall.

look, A. II. (1985). On measurement of Machiavellianism. Psyehological

Reports, 57, 982.

 

MICHIGAN smTE UNIV. LIBRARIES
1N7|WIIIHIIWIWWI“llINWWNWIIHIWI
31293014268969