WEMS OF RELIABILITY IN EVALUATENG
STORY COMPLETEOHS AEQLST SOCEAL
CONFUCTS SY GEQMAN ADGLESCENT

CMLDREM

MM the W of M. A.
MICHiGAN S'f'A‘S'E COLLEGE

Franz Georg Geiarhaas

$955

a

12.1%“!

THLSM

This is to certtfg that the

thesis entitled

Problems of Reliability in Evaluating Story Completions
about Social Conflicts by German Adolescent Children.

presented by

Franz Georg Geierheaa

has been accepted towards fulfillment
of the requirements for

_MA_ degree in W

NW Mm.

Major professor

 

Date M8 roll 99 1955

O~169

PROBLLuS OF RELIABILITY IN EVALUATING STQRY COJPLETIONS

ABOUT SOCIAL CONFLICTS BY GLRAAN ADOLLSCENT CHILDREI

By

Franz Georg Geierhaas

A THESIS

Submitted to the School of Graduate Studies of Michigan
State College of Agriculture and Applied Science
in partial fulfillment of the requirements
for the degree of

AASTER OF ARTS

Department of Psychology

Year 1955

THESIS

I
Ll

ABSTRACT

This study investigates the problems of reliability in evaluating
story completions about social conflicts by German adolescent children.

Story # 2 of Series A of the Anderson Incomplete Stories was used. The

 

incomplete stories were developed in order to get an expression by child-
ren as to their judgment of the processes and outcomes of social conflicts.
Data were collected from 1,1h6 children in Karlsruhe, Germany. This study
also makes use of small samplings from data collected.in.Mexico. All of the
Karlsruhe stories were translated from German into English and problems of
translations discussed by the'writer.

The coding manual for Story # 2 which was refined throughout the course
of this study comprises 80 process categories and six evaluative outcome ca-
tegories. It was found that the major source for disagreements in coding
lay with categories which required.making inferences.

Reliabilities of coders in coding for outcome of 89.2%, 100%, and 87.1%
were obtained. When coding process categories, reliability of 88.7% and
after a four weeks' lapse, reliability of 88.0% were obtained. The time re-
quired for one person to become a reliable coder (86.7% for categories and
8h.0% for coding outcomes) was found to be 22 hours of preparatory work.

The validity of the translation was investigated by coding independently
German originals and.English translations. Reliabilities of 92.6%, 95.0%, and
92.9% for coding categories, and reliabilities of 89.8%, 100%, and.87.2% for
coding outcomes were obtained. The reliability of the translator was tested
by coding two sets of translationS'which the writer made'with one year inter-
vening. The reliability obtained was 97.2%. Consistency of coder tested by

coding - re-coding method was 97.2% for coding categories and 97.0% for outcome.

3506223

The high reliabilities obtained demonstrate that after a short, though
intensive training period, reliable coding of Story # 2 can be done.

For purposes of obtaining the degree of reliability with which child-
ren used similar themes in writing their stories again after an interval of
two weeks, two groups of 3h boys and 3h girls were given the form twice.
7252 h and.£ig_themas in both stories were used by 6h.7% of the boys and by

67. 6% of the girls. Categories of Motherig domination and of‘Micheel'g.

 

submission'were used in both stories by 6h.7% of the boys and 76.h% of the

 

girls. Identical outcome categories were found in 70.5% of the boys and
in 88.0% of the girls. Girls were more reliable than boys in writing sto-
ries yielding the same coded outcomes.

The writer coded Story # 2 for all children from Karlsruhe. He made sex
comparisons as to several category groupings. More girls used themes of 22232,

themes of Emotion and stories which yielded.Dominationesubmission outcomes,

 

than did boys. The boys wrote significantly more stories which were coded

for Ambiguous, indecisive outcome.

 

 

Two groups of boys and girls were selected from schools which serve child-
ren from low and from high socio-economic homes. Comparisons were made between
the low and the high groups. The high group of boys used significantly more

themes of Truth, and fewer themes of Domination after truth, as well as fewer

 

stories wdth.Ambiguous, indecisive outcome than the boys in the lOW'SOciO-

 

economic group. The high group of boys had significantly more stories‘with

outcome of’Domination-submission, than did the boys in the low group. Mere

 

girls of the high group used themes of Truth after lie and themes of Domina-

 

 

tion after truth, then did the low group of girls. Girls of the low group

 

wrote significantly more stories with Ambiguous, indecisive out come, than

 

did the high group of girls.

Acknowledgements

 

The Author wishes to extend his sincere appreciation for the
continuous guidance and help he received from Dr. Harold H.
Anderson and Dr. Gladys L. Anderson in preparing this thesis.

The author also wishes to express his gratitude to Mrs. Barbara
Robinson and.N . Richard Heber for their cooperation in'working
out some of the problems of reliability, to Dr. Charles Henley
for statistical counsel, to Miss Leona DeRosia and.Miss Nancy
Lobdell for typing and other assistance.

TABLE OF CONTENTS

LIST OF TABLES. . . . . . . . . .

II

III

Page

00000001

I INTRODUCTION . . . . . . . . . . . . . . . . . . . . . . . . 1
.Background and Hypotheses . . . . . . . . . . . . . . . . l
2.TheKarlsruheStudy.. ................ 2
3. Cross-cultural and Cross-national Research Program . . . 3
h. Aims of the Present Study . . . . . . . . . . . . . . . 3
5. Subjects Used in this Study . . . . . . . . . . . . . . 3
6. Story'# 2 . . . . . . . . . . . . . . . . . . . . . . . 3
TRANSLATING AND CODING................ . h
l. Bi-lingual Problems of the Present Study . . . . . . . h
2. Coding Categories . . . . . . . . . . . . . . . . . . . 6
(a) Categories for the Outcomes of Stories . . . . . . 7
(b) Categories of Process for Michael and the Mother . . 10
3. Problem of the Unit of Coding . . . . . . . . . . . . . 17
h. Problem of multiple Tallies . . . . . . . . . . . . . . l9
5. Evaluative and Emotional Expressions other than Finite
verbs: Adjectives, Participles, Adverbs, Interjections . 20
6.TheLawofParsimony..................22
Rmmsmn‘r 0F CODERS mm TRANSLATOR . . . . . . . . . 25
1. Reliability in Ceding Outcomes of Story'# 2 , . . . . . 25
2. Reliability in Coding Categories . . . . . . . . . . . . 3h
3. Training a Coder to Become Reliable . . . . . . . . . . 39
h. validity of Translation . . . . . . . . . . . . . . . . bl
S. Reliability of Translator . . . . . . . . . . . . . . . hS
6. Consistency of One Coder in Coding - Re-Coding . . . . h?

Page
Iv RmIABIIJITY OP‘ CHEDREN O O O O O O O O O O O 0 O O O 0 0 0 2.1.8

1. Method of Obtaining Date . . . . . . . . . . . . . . . . h8

2. Some Numerical Comparisons as to Number of Tallies and
Themes Coded in Both Stories . . . . . . . . . . . . . . h9

(a) Means of Tallies and Themes Coded in Both Stories. . h9

(b) Agreement in Children's Two Sets of Stories in Using
Themes Of Truth and.Lie o e e e e e e e e o e e e o 50

(c) Agreement in Using Themes of Domination and Submis-
Sion in BOth Stories 0 e e e e e e e e e e e e e e 51

3. Reliability of Children in'Writing Stories with Same Out-

conle O O O O O O O O O O O O O O O O O O O O O O O O O O 52

V CONTENT ANALYSIS FOR STORY # 2 OF ENTIRE KARLSRUHE
SAMMIE O O O O O O O O O O O O O O O O O O O O O O O O O O 0 5h

10 methOd Of Obtaining Data 0 o o e e c Q o e o e e e o o e Sh

2. SexDiffe'rence's . - . . . - - - - . - - - - - Sh

Sh

(b) Themas Of Lies 0 o e o o o e o e o o o e o o o o o o 56

(a) Themes 0f TTUth Telling o o o o e o e e e e e o e

(0) Themes of Emotion . . . . . . . . . . . . . . . . . 58
(d) Themes of Mother's Domination . . . . . . . . . . . 59
(e) Outcome of Story # 2 . . . . . . . . . . . . . . . . 61
3. Summary of Sex Differences . . . . . . . . . . . . . . . 63

VI CONTENT ANALYSIS FOR SELECTED GROUPS OF BOYS AND GIRLS FROM
HIGH.AND LOW SOCIO-ECONQMIC HOMES e e e e o e e e e e e e o e 65

1. Selection of Groups . . . . . . . . . . . . . . . . . . . 6S

2. Results . . . . . . . . . . . . . . . . . . . . . . . . . 65

3. Summary of Findings for Boys and.for Girls . . . . . . . 69

(a) Boys . . . . . . . . . . . . . . . . . . . . . . . . 69

(b) Girls . . . . . . . . . . . . . . . . . . . . . . . 69
VIISUMMARY...........................7O
REFERENCES . . . . . . . . . . . . . . . . . . . . . . . . . . . 73

APPENIDIXA O O O O O O O O O O O O O O O O O O O O O O O O 0‘. 0 7h

Table

Table

Table

1.

Table h.

Table 5.

Table

Table

Table

Table

Table

Table

Table

Table

9.

SUI

ll.

12.

13.

LIST OF TABLES

Agreements and Disagreements of Two Independent
Coders in Coding Outcomes of Story #2 . . . . . . . . . .

Distribution of Outcomes, Agreements and Disagreements
of Independent Coders on Outcomes of Stories for Group
of 7 Boys and 10 Girls from.Karlsruhe . . . . . . . . . .

Distribution of Outcomes of Stories and Percentages of
Agreement of Independent Coders for Three Groups of
Karlsme Childr‘en I O 0 e O O O O O O O O O C O O O O 0

Distribution of Agreements and Disagreements in Repeated
Coding of Outcomes of Stories by the Same Coder after a
TWO WEekS' Interval e o o e e e o o e e o e e e o o o o 0

Example of Coding Procedure and Computation of Percentages
Of Agreement 0 O O O C C O O C O O O O O O O O O O O O .

Disagreement in Categories in Reestablishing Reliability
after Lapse of Four‘weeks . . . . . . . . . . . . . . . .

Distribution of Disagreement in Establishing Reliability
between Reliable Coder and Newly Trained Coder . ... . .

Reliability of Independent Coders Coding German Originals
and EngliSh Translations o e e o e e o o o o e o o e e o

Disagreements in Coding Independently German Original
Stories and English Translations, Dichotomized as to
Difficulties with Translation and Difficulties in Defining
Applicable Categories 0 e o e o o o e e o o o e o e o e 0

Distribution of Points of Disagreement as to Source of
Disagreement, Reliabilities, Computed for Three Groups
Excluding Disagreements Due to Translation Difficulties .

Disagreements in Coding German Originals and Trans-
lations Due to Translation Difficulties. Distribution
as to Differences in Coding for Inference Categories . .

Nomber of Stories Completed by Boys and Girls out of the
Six Stories in Series A.During First and Second Admin-
iStratiOn o o e e e e o e e o e o e o e o o o o o e o o 0

mean Frequendy of Tallies and Themes Coded for Boys'
and Girls' Stories in Two Administrations . . . . . . . .

26

28

3O

33

35

3?

ho

A2

112

’43

149

So

Table

Table

Table

Table

Table

Table

Table

Table

Table

Table

in.

15.

16.

17.

19.

20.

21.

22.

23.

Consistencies and Inconsistencies of Children in
Using Themes of Truth and Lie in First and Second
Administrations....................

Consistency of Outcomes for Reliability Group Computed
for Six and Four Outcome Categories . . . . . . . . . .

Frequencies of Stories by Boys and Girls with Tells
Truth Theme as Generalized, Whole Statement W and as
Detailed Statement 2, Percentages of Boys' aha Girls'
Stories in which Truth and Truth After Lie were Told .

 

Frequencies and Percentages for Boys and Girls in
Stories in which Lie Theme Occurred . . . . . . . . . .

Frequencies for Boys and Girls in which Theme Reveals
Anxiety Before and Anxiety After Michael meets Mother .

 

 

Frequency of Types of Punishment and Domination.in
Stories by Boys and Girls. Percentages of Boys and
Girls Having Michael Dominated After Telling Truth and
AfterTelliggLie...................

 

 

Percentages for Boys and Girls in Using Themes of
Mother's Domination. Significant Differences between
Boys and Gir‘ls O O C O O O O O O O O O I O O O O O O 0

Outcome of Story #2. Frequencies and Percentages for
Boys and Girls in the Outcome of Their Stories Classi-
fied According to the Six Outcomes and to the Sum.of

D,S,D-Soeeoeoeeeoeeeoeoeooooee

Frequencies and Percentages of Selected Themes and
Outcomes Used by Boys and Girls of Two Groups of
Children from.Seven Schools. Three Schools were
Attended by Children from Low Socio-Economic Homes and
Four Schools by Children from High Socio-Economic Homes

Percentages and Significant Differences for Boys and
Girls from Two Different Socio-Economic Groups for
Various Category Groupings and Outcomes . . . . . . . .

SI

52

SS

57

58

61

62

67

68

I. INTRODUCTION

1. Background and Hypotheses

 

This study is part of a larger research prOgrem which was designed

by Harold H. Anderson to investigate children's judgments of processes

and outcomes of conflict situations. H. H. Anderson and his students and

colleagues (1, S, 6), had previously developed several hypotheses on the

circularity of certain kinds of social behavior and had tested these hypo-

theses in observational studies of classroom teachers and children. Some

of the hypotheses were:

(1)

(2)

(3)

(h)

"Socially integrative behavior in one person tends to induce
socially integrative behavior in others. Because like tends to
induce like, this is a kind of circular behavior. A relationship
of integration or working with another has been designated a growth
circle.

"Dominative behavior in one person tends to incite dominative be-
havior in others. Because domination tends to prolong or intensify
conflict, reduce communication, and lead to disintegration in human
relations, this relationship of domination-resistance has been de-
signated a vicious circle.

"A third hypothesis is that as domination increases in severity

to a place where it is not clear whether it is safe to resist, the
individual shows confused behavior, indecision, and vacillation
between approachawithdrawal.

" A fourth hypothesis is that if environmental domination becomes
too authoritarian, or too severe, it tends to induce submission or
psychological atrophy. In this relationship there is little re-

sponse, little interaction." (3)

There was evidence that certain teachers were unable to perceive that

integrative behavior can be a problem-solving outcome to conflict. There

'was also evidence that some teachers were meeting resistance with resistance,
thereby intensifying the conflict situation in their schoolrooms. Some
teachers used the circularity of integrative behavior to cut the vicious
circle of domination-resistance. Other adults did not.

The question arose how early one could perceive these two kinds of
circular behavior and what would be perceived by children as the outcome
of conflict.

The immediate occasion for research into children's judgment of
social conflict and social problem—solving was an International'Workshop
on "Modern Psychologies and Education," held in Frankfurt am Main, Germany,
during July and August 1952. The German psychologists at the workshop dis-
cussed the authoritarian relationships between parents and children and
teachers and children (7).

The writer was both an official interpreter and a delegate at this
horkShOp. Following the'Workshop, H. H. Anderson and G. L. Anderson
carried out a study which will be referred to as the "Karlsruhe Study"

in which the writer participated as interpreter and research assistant.

2. The Karlsruhe Study

 

H. H. Anderson and G. L. Anderson devised six incomplete stories,
(The Anderson Incomplete Stories reproduced in Appendix.A). Each of
these stories presented a conflict situation or the possibility of a con-
flict situation between one child and another child, between a child and
a parent, and between a child and a teacher. In each of these stories a
child in the story makes a "mistake"—-or the story may be so construed.

The administration of the stories has been described in a preliminary
report by H. H. Anderson and G. L..Anderson (3).

In Karlsruhe, a city of 210,000 inhabitants in southwestern Germany,
data'were collected from 1,258 children in the seventh school year of 15

out of 21 city schools. Although the sample is not statistically random,

an attempt was made to make it representative for Karlsruhe. Samplings
were obtained from 36 entire classrooms. Nest city districts, including
outlying suburban areas, were sampled so that children from most socio-
economic groups were represented. For purposes of coding, the children's

stories have been translated from German into English.

3. Cross-cultural and Cross-national Research Program

The Karlsruhe Study has become the first unit in a series of cross-
cultural and cross-national projects. It has since been repeated with
children in Mexico and in Knoxville, Tennessee. A second set of stories
has been prepared, Series B of the Anderson Incomplete Stories, and the
research program has been expanded and the age range been extended. Data
on both Series A and Series B have been obtained from children in other

German cities, and in Norway, Sweden, Finland, and England.

h..Aims of the Present Study

 

The purpose of this study is to investigate:
(a) Problems of translation
(b) Problems of reliability of coders and translator
(c) Reliability of children
(d) Contents of children's story completions for Story #2 which

has been selected as the basis for investigating a, b, and c
above.

5. Subjects Used in this Study
Stories used in the present study have been written by Karlsruhe

children except for a few small samplings from Mexican data.

6. Story #2

Michael's mother sends him to the store to get one pound of wieners.
0n the way home he puts the package of meat on the edge of the sidewalk

and plays for a short time with his friends. A shepherd dog darts forward

quickly, claws half of the wieners out of the package and rushes away with

them. Michael wraps up the remaining wieners and takes them home.
'What does Michael say to his mother?
What does his mother do?
How does Michael then feel about it?

Think about these questions, then finish this story quickly with a few

sentences."

II. TRANSLATING~AND CODING

l. Bi—lingual Problems of the Present Study

 

The psychological instrument which is being examined in the present
study has been used with children in seven different countries where six
different languages are spoken. The stories, as well as the instructions
for administration, have been translated into these several languages.

The forms have been administered under standardized procedures by university
students and staff of the respective countries. The author was the adminis-
trator in Karlsruhe, Germany. Each day following the administration,

H. H. Anderson and G. L. Anderson, together with the writer, translated
samples from different classrooms in Karlsruhe. The writer later continued
alone and translated into English all stories written by the Karlsruhe
children. An AudOgraph was used and the records were later transcribed

by trained secretaries.

In spite of great familiarity with the languages in question, there
always remain residuals of subtle shades of meaning, unique and culture-
bound expressions, idiomatic usages which resist even the most scrupulous
attempts at translation. This is not only true of translation of poetry
or philOSOphy, but it is equally applicable in the case of children's oral
or written expressions. In the course of translating the Karlsruhe stories,
the writer encountered several problems.

Uncommon words are frequently heard by the children and are integrated

into their continuously growing vocabularies. These terms may have different

meanings to them from those their adult teachers attach to them. This
semantic difficulty has been found by the writer especially with more
abstract terms. For example, at the end of the stories a few leading
questions were added. In Story #2 one of these questions was, "What is
going on in Michael afterwards?" The original English question read,
"How does Hichael then feel about it?" The meaning of the word "feel" in
this sense has no German equivalent and the sentence has to be translated
as "What is going on in Michael afterwards?" ("was geht in Michael danach
vcr?"). very frequently children had some idea of what this question
meant, but they were not totally familiar with this expression and used it
rather awkwardly in their stories.

Germany, where many local dialects make it almost impossible for a
North German to understand a Bavarian, has developed a version of "Hoch-
deutsch" (High German), which is not only widely accepted as "correct"
grammatically, but, also, is greatly different in enunciation, inflection,
and usage of vocabulary. For most Germans it is hard to master completely.
It is, however, the official German employed and taught in the schools at
all levels. German children who grow up in families where dialect is
spoken throughout, even among the educated people, are faced with the task
of learning practically a new language upon entering first grade. On top
of this unique learning experience, there are literally thousands of local
usages which will creep into this newly learned High German, but'which will
be used and understood with all their implications only by natives of that
area. These areas are often as small as a county. The writer found many
of these purely local idioms and colloquialisms when translating the
Karlsruhe stories. Being a native of that town, he had little difficulty
in understanding them, and he could usually translate them into English.
'When, however, he attempted to translate stories written by children in
Munich, he encountered some difficulties which, in several cases, could

only be eliminated by consulting an available native of that area. The

I: _

above outlined difficulties in translating indicate the great desirability
of employing-awherever possible-—local persons as translators.

Another major difficulty lay with the children's language usage in
general. It was considerably difficult to translate the child-like German
idiom into equivalent English. Faithfulness to the original mode of ex-
pression was important for reliable coding later on. It was often especially
difficult to capture the projected.moods, actions and feelings and transpose
them into equally clear English. As is true of any translation, the original
and.the translation never completely agree as to length of phrases, number
of words, and, occasionally, as to number of verbs. This raised a special
problem for coding reliability with regard to units and multiple tallies,

since the coding method generally required one tally for each verb.

2. Coding Categories

 

After the first data had been collected in Karlsruhe, H. H. Anderson
and G. L. Anderson empirically developed categories for Story'#2 classi-
fying units or themas in the translations made by the writer. Two kinds of
categories were devised to assess outcome of story and detailed processes
of relating in the story. There emerged h categories of outcome or levels
of relating, and 80 process categories of which 33 were for Michael and
h? for the mother. An attempt was made to define categories which would
offer evidence for or against the hypotheses.

In Story #2, three primary "actors" are found:

(a) Michael
(b) The mother
(0) The child author who finished the story

Underlying all projective techniques is an assumption that the subject
projects himself into the experimental material. There was considerable
evidence that the children tended to identify themselves with Michael and

less frequently with the mother. nevertheless, no categories for the

child-author were retained. Identifications with Michael'were coded in

the Michael categories. Examples are:
"I handed the sausages to my mother."
"I, in Michael's place,ywould not have done this, but taken the
sausages home first."
Similarly, identifications with the mother were coded in the mother

categories.

(a) Categories for the Outcomes of Stories

 

Each story is given an over-all classification for the outcome of
the entire story. Each outcome is classified according to the level of
human relating or the degree of domination or authoritarianism on the part
of the environment or, conversely, the degree of spontanéity in Michael's
behavior. The levels of relating have been previously discussed by H. H.
Anderson and G. L. Anderson (2), and have been used by them in a preliminary
report on the Karlsruhe Study (3), from'which the following definitions
are abStracted.

I, Integrative; Problem-solving Outcome

 

The solving of the problem. There must be evidence of high
spontaneity in both the mother and her child; also of harmony and
working together, active communication and evidence of mutual con—
fidence. For harmonious cooperation there must be active communi-
cation in the place of mere thinking and fantasy.

D-R, Domination-Resistance Outcome

 

A process of hostility and aggressiveness. There must'be evidence
of mutual hostility, active tension, signs of working against each
other; the mother's hostility can be expressed in the form of domi—
nation (punishment, scolding, exacting promises), that she uses,
and the child's resistance need not be expressed orally or by

action but simply on the thinking and feeling level.‘

\ I All examples from children's stories cited in this study were selected from
‘ the Karlsruhe data unless indicated otherwise.

A, Ambiguous, Inoecisive Outcome

 

"There is lack of closure, low level of communication, story is

essentially unstructured. There is little or no interaction be-

 

tween the mother and the child. Problem has not been faced, and
no attempt has been made at its positive or negative solution.
Relation between parent and child little changed at end from that
at beginning. The experience did not serve as a basis for social
learning.”

D-S, Domination-Submission Outcome

 

Authoritarian person makes the decisions, has the answers. Child
merely submits, conforms, obeys. These stories show mother's
domination and Michael's subsequent submission. Child accepts
mother's verdict, submits to mother's ideas, punishment, and often
expresses his submissive acceptance in the form of a promise, vow,
or submissive action.“

When coding for outcome with the four categories I, D—R, A, and D-S,
unreliable results were obtained with stories that showed expressed domina-
tion but no expressed submission, or in stories where the child clearly
indicated submissive tendencies in Hichael with no overt domination by the
mother.

In general, these stories which were originally coded as QZS,

Dominative—Submission, but lacked evidence of either domination or of

 

submission, respectively, were coded as A, Ambiguous-indecisive. It
was realized, however, that these stories were rather well structured
and did not show ambiguity, as defined. Upon the suggestion of the writer,

it was therefore decided to divide category D—S, Domination—Submission,

 

8.

W;

into three subcategories which would be differentiated on the basis
of evidence in the story provided by the child:

D, Domination, with no statement of resistance or submission

 

§, Submission, with no statement of domination

 

D-S, Domination-Submission (as originally defined)

 

 

Example of Domination, with no statement of resistance or

 

 

submission:

 

Michael told his mother the truth (14W, Tells whole truth spontaneously).
She scolded him.(72, Scolding after telling truth)

and sent him to bed (68, Sent to bed aftga telling truth)

without dinner (56, Deprivation of food.a£tg§ telling truth).

Example of Submission, with 22 statement of domination by the

 

 

mother:

Michael lied to his mother (10, Lies spontaneously)

by telling her that a dog had approached him.from behind (10, Lies
spontaneously)

and had torn the sausages out of his hand (10, Lies spontaneously).

The mother sent Michael to the butcher once more (73a, Michael sent
to butcher).

Michael regretted to have lied to his mother (2ha, Regrets telling
lie),

promised himself (26, Promises himself, vows — non-communication)

never to stop on the street (26, reinforcement)

and to play (26, reinforcement)

and he never did it again (23, Submission: action, obeys).

(b) Categories g£_Process for Michael and the Mother

 

 

In developing categories for Michael and the mother, the follow—

ing general principles were defined for the coding:

l.

2.

3.

General repetition of the given facts (in the story) is not
coded except where the statement bears on the problem of
truth or falsehood.

Themes of fantasy are recorded separately from themas of
action.

Themas of anxiety and emotional feelings before Michael met
the mother are recorded separately from emotional feelings
after he met her.

When.Michael tells the truth, the record distinguishes between
simple, summary statements that Michael told the truth and
detailed statements of the separate items of truth.

Punishment which follows the telling of the truth is recorded
separately from punishment which follows telling a lie.

Separate recording is made of restitution of sausages by
Michael when made spontaneously upon.Michael's initiative
and when made by Michael under duress. (3)

The list of categories for Michael and for the mother is given

on pages ll-lh.

lO.

Series A, Story #2

"The Lost Meat"

Abbreviated Table of Categories

Category
Number
IN

1D

EM

2D

3W
3D
MW
hD
5*:

SD

22

23
2h

2ha
25

Categorijitle Michael Categories

 

 

Michael tells truth spontaneously: whole truth
Michael tells truth spontaneously: in detail
Michael tells truth after conflict, anxiety, fantasy lie:
‘whOle truth
Michael tells truth after conflict, anxiety, fantasy lie:
in detail
Michael tells truth delayed, after question: whole truth
Michael tells truth delayed, after question: in detail
Michael tells whole truth spontaneously after lie
Michael tells truth in detail spontaneously, after lie
Michael tells whole truth after lie: extorted or after
pressure, punishment, scolding, questioning
Michael tells truth in detail after lie: extorted or
after pressure, punishment, scolding, questioning

Michael delays explanation, delays facing reality

Michael tells non—essential part-truth only

Michael plans, fantasies lie or deception

Michael tells lie after weighing circumstances or question
Michael tells lie spontaneously

 

 

Constructive action by Michael
Michael makes spontaneous restitution
Constructive feeling or thinking by Michael

Hostile, aggressive action or speaking by Michael
Hostile, aggressive thinking or feeling by Michael
Displaced emotion

Michael regrets telling truth

Anxiety, emotion before meeting the mother
Anxiety, emotion after meeting the mother
Tension reduction: escapes punishment

Michael begs, requests leniency, begs pardon, forgiveness
or sympathy

Submission by Michael (Action)

Submission by Michael (Thinking and feeling) (Non-commu-
nication)

Michael regrets lie

Michael voluntarily promises mother (Communication)

ll.

 

Michael Categories (Continued)

 

26 Michael promises himself, vows (Non-communication)
27 Michael promises after punishment, scolding, admonishing

or forgiving, (Communication)
28 Negative statement pertaining to Michael, concerning some

socially unacceptable act or characteristic

12.

Category
Number

 

hla
hlb
he

h3a
th

Category Title Mother Categories

 

 

Mother perceives from Michael: Something is wrong;
she suspects

Mother perceives from.NonéMichael sources: Something
is wrong; she suspects

Mother questions: Fact finder (Open mind), calls
Michael for questioning

Mother scolds, uses verbal domination or abuse
before truth or lie

Mother uses physical dEEination before truth or lie

Mother communicates to Michael her distrust or
disbelief; accuses him

Seeks, or gets outside verification of the facts:

Communicating her intentions to Midhael

Not communicating her intentions and actions to Michael

'Without seeking, by outsider or outside event
(Deus ex Machine)

Mother behaves vaguely

Feeling of disbelief (Non-communication)

Mother becomes angry, excited

Mother's miscellaneous, unhappy emotion

Mother cries

Mother's negative feeling, feels sad

Mother accepts Michael (Action, communication)
Mother's positive feeling, mother "understands"
Mother believes truth

Mother believes lie

Deprivation of food (after Michael told truth)
Deprivation of food (after Michael told lie)
Deprivation of playtime (after Michael told truth)
Deprivation of playtime (after Michael told lie)
Restitution under duress (after Michael told truth)
Restitution under duress (after Michael told lie)
Physical punishment (after Michael told truth)
Physical punishment (after Michael told lie)
Mother tells father (after Michael told truth)
Mother tells father (after Michael told lie)
Punishment by father (after Michael told truth)
Punishment by father (after Michael told lie)
Michael send to bed (after Michael told truth)
Michael sent to bed (after Michael told lie)
Miscellaneous punishment (after Michael told truth)
Miscellaneous punishment (after Michael told lie)
Scolding (after Michael told truth)

Scolding (after Michael told lie)

13.

Mother Categories (Continued)

73a Michael sent to butcher

7h Mother makes restitution

75 Mother admonishes, regarding past behavior

76 Mether admonishes, threatens, regarding future behavior
77 Mother shows tension-reduction

78 Mother forgives Michael (Power relationship)

80 Proverbs by anyone

81 Negative statement of mother's unhappy feeling or act

1h.

The 80 categories which were developed are purposely refined.
This detailed differentiation was adopted with the idea in mind that
each new thought should be coded separately so as to allow the coder
to obtain as many different psychological themas as could be distin-
guished. Nevertheless, upon inspection of the list of categories,
it can be seen that a number of categories form clusters of closely
related themas. For example, the three categories 25, Michael volun-
tarily promises mother; 26, Michael promises himself, vows (non-commun-
ication); 27, Michael promises after punishment, scolding, admonishing
or forgiving (communication), comprise different types of promises.
They are considered submissive behavior and are conceptually closely
related to categories 23, Submission by Michael (Action) and 2h,
Submission by Michael (Thinking and feeling) (Non-communication).
Other examples are the punishment categories 56 through 73 which repre-
sent dominative behavior by the mother or the father, dichotomized as
to whether this domination follows upon truth or lie. Similarly,
Michael's statements of truth are arranged from 1W through SD in
descending order of their spontaneity (IN is "telling truth spontan-
eously, the whole truth"; SD is "telling truth in detail extorted
after lie").

Using categories refined in this way, high reliability when
obtained seems to indicate a high degree of understanding and differ-
entiation ability within the framework of the given hypotheses on the

part of the coders.

15.

In general, there can be distinguished two types of process-
categories:
(1) Process—categories which assess the facts as given by the
author.
examples:
"The mother scolds Michael (72, Mother scolds after Michaeltold
the truth)
and admonishes him not to lay down the sausages in the future"

(76, Mother admonishes Michael regarding future behavior).

"Michael says the truth to his mother" (1W, Michael tells truth
spontaneously, whole truth)

(2) Categories which require inferences on the part of the coder.

 

examples:

"His mother is right (2h, submission, thinking and feeling)
(non-communication)

in giving him a thrashing" (63, physical punishment after lie).

"Michael started to cry (19, anxiety, emotion after meeting the

mother) for he wanted to buy a nice plant for his mother's

birthday (13, constructive thought)". (The above sentence was

preceded by mother's demand for Michael's restitution for the

sausages.)

It was found that there is little difficulty in coding reliably
straightforward statements of facts. Major problems arise, however,
with the second type of category which the writer has called "inference
category." In extreme cases quite different inferences can be made by
different coders.

Example:

"The mother was very angry (h8, mother becomes angry) and
hit Michael very hard (62, Physical punishment after truth).

{ichael left the room and went out into the street to get
rid of some of his feelings."

16.

Coder I could conceivably code last sentence as (21, tension
reduction) whereas Coder II could interpret this as (15, hostile,
aggressive feeling and thinking). The outcomes would be Domination
and Domination-Resistance, respectively.

In the subsequent discussion of reliability of coders, examples
will be cited which further illuminate the difficulties encountered

with "inference categories."

3. Problemofthemgm

One major difficulty for reliable coding lay with the definition
of the unit for which a tally was to be given.

Several general rules were adopted:

(a) Generally, each verb was to be given one tally.

Example:

"Michael told the truth to his mother (1W). The mother
scolded.Michael (72)."

(b) When the child repeated the given facts of the story, no
tallies were to be given to these repetitions:

"Michael's mother sent him to the butcher (no tally). On the
'way home he lay the sausages on the edge of the sidewalk (no
tally) in order to play with his friends (no tally). A
shepherd dog stole the sausages (no tally). Michael took the
remaining sausages home to his mother (no tally). 'Mother,

a shepherd dog has stolen two of my sausages (1D) while I
played with my friends.‘ (ID, Michael tells detailed truth
spontaneously)."

(c) Verbs like "say", "tell", "ask", "answer" were not coded if

they served merely to introduce direct or indirect discourse.

17.

(d)

(6)

"Michael says to his mother (no tally) a shepherd dog has
torn the sausages out of my hands (10, Michael lies spon—
taneousLy). She asks him (no tally) whether he had not
paid any attention to them.(h2, Question by mother, fact
finder)."

In sentences, however, where these verbs appear alone, they
are coded:

"Michael says the truth (IW)."

Since special interest in this study is, among others,
centered on themas of lies, punishment and the child's
subsequent reactions, the appearance of the words "punish",
"lie", were coded each time separately as well as the'word
"promise".

"The mother punished Michael (58) by giving him no sausages
for lunch (58)."

Category (58) is defined as "Mother deprives Michael of

food." The author's elaborate telling of the mother's punish-
ment for Michael is the reason for the two tallies of (58).
This type of multiple tally is called reinforcement, in the
above example, reinforcement of the punishment idea. The

same rationale underlies the coding of the following examples:
"Michael said a lie to his mother (10, Michael lies spontan-
eously) 'Mother, a shepherd dog has jumped up on me (10) and

torn the sausages away‘ (10)."

"Hichael promises himself (26, Michael promises himself, vows)
not to play anymore (26) when he is sent on an errand (26)."

In general, subordinate clauses are coded as reinforcements
as long as they give more details of the same thema or immed-

iate cause and effect relationships.

”The mother tells Michael (no tally) that it is better
to tell the truth (76, Mother admonishes, threatens Michael
regarding future behavior) because sooner or later it is

discovered (76)."

If the subordinate clause, however, introduces a new psycho—
10gical thema which had previously not been mentioned, the
subordinate clause is then coded for the new thema.
"Michael'was very sad (19, Michael shows anxiety, emotion,
after meeting the mother) that the mother had whipped him
(52, Physical punishment after truth)." The thema of

mother's punishment for Michael was not previously mentioned.

"He is happy (21, Tension reduction) that his mother had
believed him (a lie) (55, Mother believes lie)."

"His mother is very angry with him (h8, Mother becomes
angry) for laying them (the sausages) down (ID)." No
direct statement by Michael telling the truth. This is

an inference that he told the truth and is therefore coded
as 1D, Michael tells truth in detail spontaneously.

Compare however the above examples with the following:
"Hother, I did not think much (1D) and put down the sausages
(1D) and played'with.my friends (1D). His mother whips
Michael (62, Physical punishment after truth) because he is
careless (62) and because he stayed with his friends (62)."
"Because he was careless" and "because he played with his
friendsrwere coded as reinforcements because Michael in
telling the truth had already mentioned his carelessness

("I did not think much") and the fact that he had stayed with

his friends.

h. Problem of Multiple Tallies

 

In the first analyses of data from Story #2 (3), as well as in

the present study, multiple tallies, though recorded, were disregarded.

l9.

In developing the coding manual, it was thought to be important

for later closer analysis not to assess simply the occurrence of

a psychological thema but also to make records of the quantity and
frequency with which it occurs. It will be the task for later
investigations to establish the validity of this assumption. But

it is already understood that there could conceivably be a difference
between the first example below, from the Karlsruhe data, and the
second example, which represents an abstracted composite and not the
story from one child.

"Michael lied to his mother (10; the word "lie" is coded) that
a dog had torn them out of his hand (10)."

"When Eichael came home (no tally: connecting sentence between
"givens" in story and author's part of story) he said to his
mother (no tally for "say" if followed by direct or indirect
discourse), 'Imagine (lO), mother, when I came out of the
butcher shop (10) a gang of 5 boys approached me (10) challenged
me to a fight (10) and without'waiting (lO) attackmd.me (10).

I tried to protect the sausages (10) but they pulled them out
from under my arm (10) and ran away (lO).'"

The difference in emphasis and concession of guilt is equally
obvious in the following two examples:

"Michael felt repentant (2h, submissive feeling, thinking)."

"... and to receive (2h) without replying (2h) the well merited

(2h) punishment (no tally for "punishment" since it is the object

of "to receive" which already has been coded) for such bad con-
duCt (2’4) 0"

5. Evaluative and Emotional Expressions other than Finite verbs:

 

Adjectives, Participles, Adverbs, Interjections

 

 

Adjectives, participles, adverbs and interjections which are

strongly evaluative and emotionally charged are coded when they

20.

convey psychological themas, primarily those connected with
expressions of emotions and anxiety. (Categories 6, 1h, 15, 16,
18, 19, 22, 2h)

Examples:

Category 6, Delays explanation, delays facing reality:

"Slowly (6) he went home" (no tally; connecting link
between givens and author's story).

Category 15, Hostile, aggressive thinking, feeling:

"fiichael ran off (21, Tension reduction) crying (19,
Anxiety, emotion after meeting the mother) feeling
very angry (15 ) -"

Category 16, Displaced emotion:

"Eben he got home running (18, Anxiety, emotion before
meeting the mother) very angry (16, reinforced by follow-
ing sentence) because the dog had eaten almost a quarter
of a kilo of meat (16)." (Mexican child).

Category 18, Anxiety, emotion, before meeting the mother:

"Michael'went home (no tally, connecting sentence)
very sad (18)."

Category 1?, Anxiety, emotion after meeting the mother:

"Fearful (l9)and repentant (2h, Submission feeling,
thinking) embraces his mother (22, Begs, requests
leniency) and says to her (no tally), 'Please (22),
mother, forgive me (22).‘"

Category 22, Begs, requests leniency, begs pardon, forgiveness,
sympathy:

"Dear (22) mummy, please (22), please (22), be not angry
with me (22)."

"Dear mummy" is not the usual address but rather inter—

preted as repetition for emphasis. This is equally true
for the repeated "please".

21.

Category 2h, Submission, feeling, thinking, non-communication:
"After what had happened (no tally, connecting phrase)
Michael tells his mother (no tally) very repentantly (2h),
'1 am sorry (23, Submission, action, speaking), I put
down the sausages (2D, Tells truth after conflict,
anxiety) ...'"

The same principle for coding evaluative words such as adjec—
tives, adverbs, participles, and interjections, holds true for
categories of the mother.

Examples:

"The mother asks him (no tally) very angrily (h8, Mother

becomes angry), 'Wnere have you been so long?‘ (h3a, Mother

scolds, uses verbal domination before Michael tells truth

or lie)."

"The mother says (no tally) relieved (77, Tension reduction

of mother), 'Oh my dear boy (77), I am glad (77) that the

shepherd did not bite off your finger'(55, Mother believes

lie after Michael told mother that the shepherd dog snapped

for his hand but only succeeded in getting the sausages)."
6. The Law of Parsimony:

In coding the:3tories, very often inferences have to be made.
In order not to adopt a habit of careless inference—making as the
prevalent mode of coding, the coder has to keep in mind that he
should choose the categories on the basis of the law of parsimony.
There is a danger that the coder might indulge, otherwise, in almost
exclusively making inferences rather than to take the sentence as
written by the child and code it according to one of the "factual
categories." An already cited example shall serve to demonstrate

this.

"Michael slowly walked towards his home".

22.

This sentence would not be coded without the word "slowly". Two
categories immediately come to mind:

Category 6, Delays explanation, delays facing reality,
Category 18, Anxiety before meeting the mother.

It is coded as "Delays facing reality" in preference to "Anxiety
before meeting the mother". The child explicitly stated that he
walked slowly,'whereas coding as anxiety could only be done by
inference. Where there is a descriptive statement and a plausible
inference, the descriptive statement is preferred.

A more subtle example of parsimony is the following excerpt
of a story:

"hichael lies to his mother (lO),his mother gave him more
money (55, Mother believes lie)."

According to the law of parsimony, the latter sentence should be
coded as 7h, Hotner makes restitution. In the framework of this
study, the recording of themas of truth and lie and their conse—
quences are Within the specific objectives of the research. Con-
sistently, 55, Mother believes lie is given preference to 7b, Mother
makes restitution.

Categories 2h, Submission, feeling, thinking, which includes
regrets, repentance and bad conscience and 2ha, Michael regrets
telling lie, must be treated similarly. Michael's regret to have
lied and.Michael's repentant feelings about "misbehavior" or "dis-
obedience" are both interpreted as submissive thinking and feeling

as long as theyzzre not communicated diretly to the mother. Since,

23.

however, in this study, themas of truth and lie, respectively, are
kept separate, category 2ha, Michael regrets telling lie, was set
up to be used for specific regrets about lies.

The subsequent report on coder reliability will show some of
the improvements in the degree of reliability mainly due to con-
tinuous refinement of the categories as well as clarification of

category definitions.

2h.

III. RELIABILITY OF CODBRS AND TRAESLATOR

L. Reliability in Coding Outcomes of Story'# 2.

 

 

In the COirse of revising the coding manual, H.H. Anderson, G.L. An-
derson and the writer coded various classrooms for outcomes to Story'# 2.
This was done with two purposes in mind:

(a) Clarification of outcome categories,
(b) Clarification of interdependency of individual
categories and outcomes.

Reliability of independent coders for outcome of stories is report-
ed in terms of percentages of agreement computed according to the follow-
ing formula:

Two times the number of agreements divided by the total
number of tallies given by both coders.

TABLE 1

Agreements and Disagreements of Two Independent Coders
in Coding Outcomes of Story #2

N : 39 Mexican Children

 

 

 

Outc ome

of Story A gre ements Disagreements
Coder I Coder

I, Integration - 2 -
D-R, Domination-Res is tance - - -
A, Ambiguous , indec is ive h 1 l
D, Domination 14 3 2
S , Subnission l 3 -
D-S , Domination-submis 5 ion 2 O 1 7

Total: 2 9 10

 

Percentage of Agreement: 714.3%

 

Coder I was F. G. Geierhaas
Coder II was G. L. Anderson

26.

Table 1 reports the first computation of percentage of agreement
between two independent coders. This table is presented to show the
initial difficulty in coding for outcome and to show a distribution of
disagreements over five of the six categories. The outcomes were clas-
sified without spedifically coding for detailed categories. It was rather
an assessment of total Gestalten. Later, coding for outcome was always

done at the end of coding the entire story for individual categories.

Seventeen days later after further practice and considerable discus-
sion, these same coders achieved a percentage of agreement of 85.7 in
connection with an analysis of errors in coding process categories re-

ported below in Table 5.

27.

TABLE 2

Distribution of Outcomes, Agreements and.Disagreements
of Independent Coders on Outcomes of Stories
for Group of 7 Boys and 10 Girls from Karlsruhe

 

Outcome Agreements Disagreements
of Story Boys Girls Total Coder I Coder 111*
Boys Girls Boys Girls

 

 

Integr.

Dom.-Res. l l l l
Ambig. 3 3 6 l l

Dom.

Subm. l l 2 l
Dom.-Subm. 2 3 S 1
Totals 6 8 1h 1 2 1 2

 

Percentage of Agreement: 82.3%

 

* Coder III was Richard Heber

28.

Table 2 reports on outcome for coder reliability with a group of
17 Karlsruhe children. Coder I and Coder III achieved a percentage of
agreement of 82.3. The analysis in table 2 was made in connection with
the training of Coder III who at the beginning of his training was com-

pletely unfamiliar with the study. Further details on training will be

given below.

One of the objectives of this investigation of reliability of coders
is to compare the coding of German originals with the coding of English
translations of the same stories. A report of the coding of process ca-
tegories for Michael and the mother is made below. Table 3 shows the dis-
tribution of agreements and disagreements and percentages of agreement for

coding of outcomes.

29.

- TABLE 3

Distribution of Outcomes of’Stories and Percentages of.Agreement
of Independent Coders for Three Groups of Karlsruhe Children

Coder I Coded Translations; Coder III Coded Original German Stories

 

 

 

 

 

Classroom I Classroom II Classroom III Totals
N : 31 Girls : 27 Girls N : 37 Boys N : 95
Outcome .
of Agree- Dis- Agree- Dis- Agree- Agree- Dis—
Story ments agree- ments agree- ments ments agree
ments ments ments
Coder Coder
I III I III
Integr.
Dom.-ReS. l 3 3 3
Ambig. l 2 8 11 20 h
Dom. 2 l l 6 8 h
Subm. 5 1 1 7 1
Dom-Subm. 19 2 l 12 18 39 h
Totals 27 h 27 33 87 8
Percentages
Of 87 01% 100% 89 o 2% 91.6%
Agreement

 

30.

When two coders independently coded German originals and English
translations, respectively, the percentages of agreement for coding of
outcomes for three classrooms of children were:

87.1 100 89.2

In one classroom of 27 girls there was no disagreement whatsoever

in coding the outcome. In each of the pther rooms there were only four

cases of disagreement.

The agreement for the three classrooms combined was 91.6%. These
percentages are regarded as very high and indicate that there is no fun-
damental loss in translating children's stories from German into English

5 o

as far as the outcomes are concerned.

An inspection of the stories with disagreements between the coders
in coding outcomes showed that in the great majority the different out-
comes of the two coders were dependent on their different inferences.

A few examples will illustrate this source of error:
"... the mother scolds with Michael. He should not have played
and sat it (the meat) down on the street but come home imme-
. diately."
Coder I interpreted the last sentence as 2h, Michael‘s submis-
sive feeling and thinking, whereas coder II construed this to

be the Mother scolding Michael (72).

The outcomes were Domination-submission and Domination, respective-
ly

 

 

"Hother, a dog has taken away the sausages (7, Michael tells non—
essential part-truth). I have only these left (7). Franz'who had
played with him on the street (hSc, Mother gets verification of
facts without seeking) betrayed to his mother (héc) that he (Michael)
had not gone home immediately (hSc) after having bought the sau-
sagss(h5c). Then he (Michael) gets a whipping from his father (67,
Punishment by father after lie). The other comrades of Michael's
thrash the traitor Franz ( )."

Coder I interpreted the last sentence as 1h, Hostile, aggressive
action whereas Coder II understood this to be 16, Displaced emotion.

The outcomes were Domination-resistance and Domination, respectively.

31.

 

"... The butcher did not have two pairs yet (10, Michael tells
lie spontaneously). The mother notices that he has lied (h7,
Mother' 8 feeling of disbelief, knowledge of truth). She scolds
him (73, Scolding after lie). Michael becomes nervous (19, An-
xiety, emotion after meeting the mother) and sad in his face (15)

and runs away ( 5.‘ "

Coder I construed last part of last sentence to be an expression
of 15, Michael's hostile, aggressive feeling, whereas Coder II
took this to mean 21, Tension reduction and escape from punishment.

The outcomes were accordingly Domination-resistance for Coder I
and Ambiguous, indecisive for Coder II

The last two exam ples seem to indicate that coders tend to be con-
sistent in their inferenceemaking. It would be very hard to assess what

influences and past experiences induce them to make one or the other

inference.

Another indication for this consistency of one coder in his approach
towards the stories can be found in the case of the coding - re—coding re-

liability of one coder as reported in table h.

32.

TABLE h

Distribution of Agreements and Disagreements,
in Repeated Coding of Outcomes of Stories
by the Same Coder after a Two Weeks' Interval

 

 

 

N = 3h
Agreement Disagreement

Outcome

of Story First and Second Coding First Coding Second Coding
Integr.
DOIno"ReSo
Ambig. 6 l,
Dom. 6

Subm. 2
Dom.-Subm. 19 1

Total 33 1

 

Percentage of Agreement: 97.0%

 

The writer coded the same stories twice with an interval of two weeks
between the first coding and the second coding. In between the two cod-
ing sessions, the writer worked intensively on other similar material.
Table h shows only one disagreement in the outcomes of 3h stories which

yields 97.0% of agreement.

2. Reliability'in Coding Categories.

 

 

Two coders scored the stories independently. Percentages of agree-
ment'were computed by the same formula used above for the coding of out-
comes. Table 5 is given as an example to demonstrate for seven children

the details of coding categories unit by unit and also outcomes.

31;.

TABLE 5

Example of Coding Procedure
and Computation of Percentages of Agreement

1 = 7 Mexican Children

 

Child (Mexican Numbers)

 

 

 

 

 

 

923 92h ' 925 926 927 928 929
Coder
I II I II I II I II I II I II I II
Categories Totals
7 10 18 18 18 18 1D 1D 10 10 10 10 h3a h3a
h5b h5b 18* ZW'ZW 1D 1D 10 10 10 10 1h 9
h5b h5b 9 9 25 25 1D 1D 10 10 10 10 DA hh
h3b h3b 9 9 23 23 62 62 63 63 73 73 ”9 9
2h 19 9 9 78 78 62 62 63 63 63 63 9 9
2h 19 9 9 27 27 62 62 2h 2h 2h 2h 9 9
2h 19 h? h? 2h 2h 2h 2h 2h 2h 9 9
63 63 2h 2h 21 13 2h 2h
63 63 27 27 2h 23
23 23 27 27 2h 2h
23 19 27 27 2b 2b
27 27 2h*
Two times
Agreement 6 20 12 22 18 16 12 106
Total
Tallies 1h 23 12 22 23 16 1h 12h
Percentage of
Agreement h2.9% 87.0% 100% 100% 78.3% 100% 85.7% 85.5%
Outcomes** DS DS DS DS ;DS DS DS DS DS DS DS DS DR D 85.7%

 

**Percentage of agreement for outcomes of stories, by the formula,

equals lZ/lh = 85.7%

Total number of disagreements:

ll

*Disagreement resulted from difficulty in defining unit.

There were:

Three cases of disagreement between categories (2b) and (19)

One case

One
One
One
One
One

1'
II
II
H
fl

1!

N
H
N
I!
H
H

35.

If

H

(23) and (19)
(2b) and (23)
(21) and (13)
(1h) and (9)

(18) and £0 .
(2h) and 0)

It can be seen in Table 5 that there existed difficulties in cod-
ing reliably especially for categories 19, Anxiety, emotion afte£_meeting
the mother, 23, Submission, action, speaking, and 2h, Submission, think-
ing, feeling. These categories were subsequently re-examined and the de—

finitions clarified.

Percentages of agreement of 82.3 for one group of 21 children, and
8h.9 for another group of 20 children, both from Mexico, were obtained
by Coders I and II. In the group of 21 children there were 29 disagree-
ments, out of'Ihich 15 were due to difficulties in defining the coding
units without losing a psycholOgical thema, and 1h'were due to disagree-

ment on categories.

Percentages of agreement of outcome coding for the group of 21 child-

ren was 95.0 and for the group of 20 children 95.0.

Four weeks after having established reliability of 82.35 and 8h.9%
on Mexican data, Coders I and II re—established their coding reliability
“with an agreement of 88.0%. Agreement on Outcomes of stories was 100%.

The distribution of disagreements included the categories shown in Table 6.

TABLE 6

Disagreement in Categories in Reestablishing Reliability
after Lapse of Four'weeks

Number of Children : 12

 

 

 

Categories
Coder I Coder II FrequenCy

18 6 1

19 2h 3

2h 19 1

62 75 2

72 7S 2

76 - 1

- 76 1
Total number of disagreements: 10

 

37.

Although the reliability attained after the four weeks' interval
was higher than before, a distribution of disagreements still showed
that there seemed to be some difficulty in the making of inferences
especially for categories 19 and 2h. The problem of the definition of
the scorable unit resulted in disagreements in only two cases. In both
of these cases, the thema was coded, but there was disagreement as to

number of reinforcements.

This renewed appearance of disagreements with reference to cate-

gories l9 and 2h led to further discussion and the following clarification:

Category 19 now includes only expressions of anxiety feelings
and fears. It comprises the more physiologically determined expressions
of emotion such as speech difficulty, stutter, stammer, cannot speak, red
face, cries, sobs, restless, uneasy, feeling of fear of specified punish—

ment 0

Category 2h, on the other hand, includes the more culturally deter-
mined expressions which more clearly indicate submissive feelings such
as regrets, reflections, feels sorry, repentant, guilty, ashamed, con-
trition, bad conscience, also many expressions using "should”,"would"
and other statements in the subjunctive mood; also statements about
specific and detailed errors such as "I should not have played"( as a
thought, not communicated, otherwise category 23). Expressions of sub-
mission in the form of "mother knows best" and justifiable inferences
as to the fact that the mother does the thinking and deciding; also

expressions of dependence on the mother. (h)

38.

3. Training a Coder to Become Reliable.

 

Since there is a great deal of data in the cross-cultural and
cross-national research program to be coded, an attempt was made to
find out how long a training period it would take until a person com-
pletely unacquainted'with the study would become highly reliable in

coding the stories.

The different steps in this training were:

(a) Reading of Literature pertaining to type of study presented
here.

(b) Discussion with authors H.H. Anderson and G.L. Anderson and
'with the writer concerning the coding manual, specific ca-
tegories and difficulties encountered.

(0) Practice coding of stories for outcome and categories pre-
viously coded by reliable coder; discussion and clarifi-
cation of disagreements between the two coders.

(d) Establishment of reliability with already reliable coder.

The total time required to attain satisfactory reliability of coding

by this person, Coder III, was about 22 hours spent as follows:

h hours of preparatory reading,

6 hours of studying coding manual,

6 hours of discussion and clarification of difficulties, and
6 hours of practice coding.

The first attempt at establishing reliability in coding independently
Story # 2 using data from 13 children in Karlsruhe, resulted in agreements
of 86.7% for categories and 8h.0% for categories and outcomes combined.

1) In general, outcome reliability and category reliability were computed

separately. If not indicated otherwise, reliability figures for cate-
gories do not include outcomes.

39.

TABLE?

Distribution of Disagreement in.Establishing Reliability
between Reliable Coder and Newly Trained Coder

 

 

 

 

N : 16
Categories
Coder I Coder III Frequency of
(reliable coder) (new coder) Disagreement
16 20 2
6 h2 1
10 - 1
2h 70 2
2ha - l
h2 - 1
outcomes outcomes
A S
DR EB

 

hO.

In another trial Coder III established a reliability of 93.2% with
Coder I by coding Story # 2 of 7 boys and 9 girls of Karlsr*he children.

The categories in disagreement are shown in table 7.

From table 7 it can be seen that there were three disagreements due
to difficulty in defining units. These three cases represented disagree-
ments in coding reinforcements. Both coders in each of the three cases
in question had coded the thema. The disagreements between categories 16
and 20 have led to the elimination of category 20 altogether as too vague
and thematically not important. Again, two cases of disagreement involved

category 2h.

h. Validity of Translation.

In this bi-lingual study, it was necessary to investigate how valid
the translations were. The writer previously pointed out some of the dif-
ficulties encountered in translating the German stories into English.

Two approaches to determine translator reliability and validity were used:
(a) coding independently by highly reliable coders German
originals and English translations, respectively.

(Validity of translation)

(b) retranslation of group of stories and coding these two
translations independently with intervening lapse of
we weeks so as to exclude influence of memory.
(Reliability of translator)

Coding the English translations and German originals of Story # 2
of 36 boys and 57 girls resulted in the percentages of agreement shown

in table 8.

bl.

TABLE 8

Reliability 01 Independent Coders
Coding German Originals and English Translations

 

 

N = 93
Number and Reliability for _Reliability for
Groups Sex Categories Outcomes of Stories
I 36 boys 92.6% 89.6%
II 20 girls 95.0% 100%
III 31 girls 92.%% 81.’$

 

An investigation of the disagreements in coding brought to
light that the great majority of differences was due to discrepancies
between the translation and the original. Table 9 shows points of
disagreement due to translation difficulties as compared with dis-
agreements due to differences in interpreting for category placement.
In Table 10 reliabilities are given which were computed by eliminating

disagreements found in the translation.

TABLE 9

Disagreements in Coding Independently German Original Stories
and English Translations, Dichotomized as to Difficulties
with Translation and Difficulties in Defining Applicable

 

 

 

 

Categories
Number and Difficulty in Definition of
Groups Sex Translation Category
Frequency Frequency
I 36 boys 8 12
II 26 girls h 8
III 31 girls h 10
Totals 93 children 16 30

 

h2.

TABLE 10

Distribution of Points of Disagreement
as to Source of Disagreement, Reliabilities,
Computed for Three Groups Excluding Disagreements
Due to Translation Difficulties

 

 

Number Points of dis- Points of dis— Reliabilities

Groups and Sex agreement due agreement due disregarding

to category to translation translation

placement difficulties difficulties
I 36 boys 21 22 96,2%
II 26 girls 7 lb 98.1%
III 31 girls 5 29 98.2%

 

Upon closer inspection, it was found that over 505 of the points of
disagreement due to differences in the German originals and the English
translations, respectively, were due to different numbers of units in the
original and translation. This lends support to the above statement that
it is impossible always to translate one verb in German into one verb in
English. Necessarily when the number of coding tallies is primarily based

on number of verb units, there will then be some disagreement between the

coding of the originals and that of the translations as found in table 10.

It was furthermore found that all other cases of disagreement due to

translation difficulties involved categories which the writer has previous-

ly labeled "inference categories." The distribution of disagreements in

LL30

 

a
"w

coding German originals and English translations due to inference-
making is reported in table ll. The heavy concentration of dis-

agreements is centered in or near categories of submission.

TABLE 11

Disagreements in Coding German Originals and Translations
Due to Translation Difficulties. Distribution as to Differences
in Coding for Inference Categories

 

 

 

 

 

Group Coder of Coder of Frequency of
Number, Sex Translations Originals disagreements
Categories Categories
l6 l9 1
I 25 26 l
(36 boys) 2h 26 l
70 h8 2
72 19 1
II 75 2h 3
(26 girls)
15 2h 1
l9 2ha 1
III 2ha 19 h
(31 girls) 2ha 2h 1
h6 S2 2
19 2h 1

 

In several of the cases, the different position of a word in the
translation and the original, respectively, the insertion of a "but",
or the more submissive meaning of a translated sentence when compared
with the original German, were the source for the resulting disagree-

ments.

 

S. Reliability'gf Translator.

 

The writer was interested in finding out how reliable a translator
he has been. Taking a sample of Karlsruhe children whose stories had been
translated by him in October 1952 in Germany, he retranslated them in
October 1953 while in residence at Michigan State College, He coded both
sets of translations with a two weeks' interval and obtained reliability

of 97.23 for categories coded in 3b stories.

Among the 3b stories four cases of disagreenent were found of which
three were due to differences in the two translations, and one due to

different category placement.

The three cases of iifforences in the translations are:

CaseI
Translation 1
"The mother gives him some money in his hand (7h, Mother makes
restitution) and says ( no tally) 'So now you get two more
sausages'( 73a, Mother sends Nichael to butcher)."

Translation 2
"The mother gave him a coin in his hand (7h) and said, 'now
go (73a) and get me another two sausages'(73a)."

Original German:
"Die Mutter gab ihm ein Geldstueck in die Hand und sagte:'So
jetzt holst du mir noch mal zwei Wuerste.'"

Case II
Translation 1
"The mother put on her coat (th, Vother seeks outside verifi-
cation of facts without telling Michael) and went to the but—
cher (th). The butcher told her (no tally) that he had wrapped
up two pairs of sausages for her son half an hour ago (h5b)."

Translation 2
"The mother put on her coat (LSb) and went to the butcher (th).
.The butcher said to her that he had wrapped up two pairs of
sausages for her son (th) and that happened about half an
hour ago (h3b)."

115 .

 

$'_"'
3

i .

nl

Original German:
"Die Mutter 20g den Mantel an und ging zum Metzger. Der Hetz-
ger sagte ihr, dass er ihrem Sohn zwei Paar fluerste einge-
packt haette und das vor einer halben Stunde."

Case III
Translation 1
"Here you have another to pfennig (7b, Vother makes restitution)
get another pair of sausages (73a, Mother sends Michael to but-
cher)."
Translation 2
"Here you have hO pfennig (7h), go (73a) and get another pair
N f
of sausages \73a)."
Original German:
"Hier hast du hO Pfennig, hole noch ein paar‘fiuerste fuer mich."
In all three cases it is evident that the use of two verbs in the
second translation provided for the disagreement in the number of re—
inforcements. In neither of the two translations as coded, did the trans-
lator lose a psychological thema. Realizing this decided tendency to be
unreliable in translating with regard to codable units ( primarily verbs ),
the writer in recent translations of stories from other German samplings
paid particular attention to adding or cutting off no codable units in
translating the stories. He often used parenthetical explanations to
clarify stilted sounding or awkward terms while keeping the codable units
exactly alike in the German and English. This seems to indicate that when—
ever a translator is used, familiarity with the objectives and.metbods of

this cross-cultural and cross-national research program provides for greater

reliability of translation.

h6.

6. Consistency of One Coder in_Coding - gngoding.

 

During the refinement of the coding manual for Story'# 2, it was
evident that thinking about different hypotheses underlying the cate-
gories and outcomes, and about the categories themselves, brought about
clarification of several hentofore unsolved problems. This process can
be considered one of learning, and the development in thinking might
conceivably be called a change in interpretation and inference-making.
In order to test this possibility of fluctuation and to find the sources

for potential variability in coding connected with it, the writer re-

 

coded one group of stories (3h girls grom Karlsruhe) two weeks after the
first coding. In between the first and the second coding sessions, th
writer worked intensively on other stories, on translations, tabulations,
and above all, on clarification of coding problems. By the true of the
second coding, he did not have any specific recollections about the

first coding. The possibility of influence by memory was thus minimized.

Reliability was 97.2% for categories, and 97.01 for outcomes. There
'were four cases of disagreement. This high reliability for one coder over
a period of time, inspite of learning processes presumably going on in the
lapse between the two codings, is considered by he writer as an indica-

tion of attainable consistency of interpretation in coding.

1x7.

 

IV. RELIABILITY OF CTILDREN

1. Method of Obtaining Data

 

Two classrooms (36 boys; 3h girls)in one school were given
the six stories of Series A twice with two weeks between the two
administrations. At the first administration of the stories,
neither the children nor their teachers were told that they had
been selected as the reliability groups.

The second administration of the stories was preceded by
instructions which read about like this:

"Mr. and firs. Anderson'would like to ask you to be so kind

as to help them write the stories once more. You need not

try to remember what you have'written the last time, but

Simply finish the stories as you see fit."

The writer was again the administrator. The children were
very cooperative also the second time. They did not take as
much time to write the second six stories, and more children
finished all stories during the second administration than
during the first administration as shown in Table 12. For
Story'fi2 there were only 3h pairs of codable stories from the

36 boys.

 

l) The exact text of the instructions was lost and therefore had to
be reconstructed from memory.

h8.

 

 

 

TABLE 12
Number of Stories Completed by Boys and Girls
out of the Six Stories in Series A

During First and Second Administration

Numbers: Boys 36, Girls 3h

 

 

 

 

Number of Number of Boys Number of Girls
Stories First Second First Second
Finished Admin. Admin. Admin. Admin. ”‘
3 2 7
h it 1 9 1
5 13 2 2 3
6 7 33 16 30

 

Several inferences can be drawn from the findings in Table 12:

(a)

(b)

(C)

During the second administration, the children did not need
so much time to read the story beginnings. They remembered
at least the gist of them from the first administration.

They did not spend as much time on writing the first few
stories as they did during the first administration.

They remembered part or all of the conclusions which they
had written the first time. This hypothesis is based on
the finding of several children's stories which were almost
identical, word for word.

Some Numerical Comparisons as to Number of Tallies and Themes
Coded.in.Both.Stories

 

 

(a) Means of tallies and themas coded in both stories

 

Upon first inspection of the tabulation of tallies it seemed that

there were true differences between the length of the first story (as

determined by number of tallies given in the coding process) and the

149.

second story of most children. Means for number of tallies and
number of themas were computed for each of the two administrations

for boys and girls and are reproduced in Table 13.

TABLE 13

Mean Frequency of Tallies and Themes Coded
for Boys' and Girls' Stories in Two Administrations

 

 

 

 

 

Mean N
Administration 7 Boys Girls

Tallies 8.3 9.2
First

Themes 5.h 5.8

Tallies 6.7 8.2
Second

Themes h.h 5.2

 

Standard deviations were computed and t-tests applied which did
not yield significance of the differences found. Although there seems
to be a slight tendency for the second stories to be shorter, as to
variety of themas and as to number of tallies, the differences did
not prove statistically significant.

(b) Agreement in children's two sets of stories in using
themas of truth and lie

 

 

Truth and lie themas in this study are considered important as
indicators of children's levels of spontaneity and submissiveness or
aggressiveness, respectively; The consistency with which the children
use these themas has therefore been determined and are shown in

Table 1h.

SO.

 

TABLE 1h

Consistencies and Inconsistencies of Children
in Using Themas of Truth and Lie
in First and Second.Administrations

 

 

 

Truth in Neither
both truth
adminis- Lie in Truth Lie in nor lie
trations both Total in first first in either Total
Boys
N = 3h 8 1h 22 6 h 2 12
Girls
N : 3h 10 13 23 6 2 3 ll

 

In 6h.7% of all cases the boys used truth or lie themas in both
stories; 67.63 of the girls used truth or lie themas in both stories.
There seems to be no sex differences in these comparisons. Table 13
shows a virtual identity between boys and girls which runs consistently
through the table.

(c) Agreement in using themas of domination and submission

in both stsries

 

 

Categories 22 through 27 indicate submission on the part of Kichael.
Categories h3a, h3b, uh, hS, 56 through 73, 75, 76, 78 indicate domin-
ation on the part of the mother.

Categories of domination and submission in both stories were used
by 6h.7% of the boys and 76.h€ of the girls. For both boys and girls,
the majority showed a high consistency in presenting these themas in

both first and second stories.

51.

'LL. F—

 

V..

r
ol

 

3. Reliability of children in writing stories with same outcome

 

 

It was shown above that there existed considerable consistency in
children‘s usage of themas of truth, lie, domination and submission.
Probably the best indicator of an all-inclusive reliability of children
is the consistency With which both stories show the same overall approach
as proven by the same codes for outcomes. The outcomes are primarily
classifications of the level of children's expressed spontaneity or lack
of spontaneity. Since both truth and lie may be spontaneous expressions,
and since both truth and lie may be expressions of submissiveness or
aggressiveness, respectively, the coding for outcomes must be considered
the best indicator of the level of integration and spontaneity that a
given child shows.

Outcome reliability was computed for all six outcome categories
(I, D-R, A, D, S, D-S) and for the four major categories (I, D—R, A,
9—8). Since domination and submission are very closely related in a
cause and effect relationship, the ,rocedure of subsuming categories

D, S, and D—S was considered justifiable.

TABLE 15

Consistency of Outcomes for Reliability Group
Computed for Six and Four Outcome Categories

 

 

Reliability, 6 categories: Reliability, h Categories:
I, D‘R, A, D, 8L D_S I, D-R) A) D’s (DLS’D—S)
Boys
N 2 3h 50.0% 70-55
Girls
N =- 3h 67.6,“; 58.0%

 

52-

 

 

 

It can be seen from Table 15 that girls were more reliable than
boys in.writing stories which yielded the same outcome codes. he main

disagreements lay with Domination—submission categories vs. the Ambigu-

 

ous, indecisive outcome.

 

 

There were no disagreements betwe;n the Domination-submission categories

 

and the Integrative outcome. Onlv one girl wrote two stories which yielded

{a

 

O .

outcomes of Domination and Domination—resistance, respectively. This {may

 

seems to indicate that the children were very reliable as to writing sto-
ries on the same level of spontaneity and integrative behavior or lack of
it.

Since the reliability groupS'were very small, more conclusive and

 

statistically alid interpretation 'were not justified. Another, possi-
bly m re conclusive test of children's reliability and consistency, will
have to be carried out when all six stories have been coded. The degree
of internal consistency, from story to story, will give more conclusive
evidence as to the reliability of children in their responses to social

conflict situations.

7'm 17. T?"(‘ Ta *1 0‘7 'v-I *rme'r—yfi .«_-)-7

CO

 

V “TENT Anduiom) r93 STOhi # 2 Or mulling seal
1. Tethod'_£ obtaining data

The author, after establishing reliability of

data and on German originals, coded all Stories # 2

ruhe children. Data from a total of 59h boys and 66

collected; 560 boys

ber, 3h boys and 7h girls either had not written or

f.

.. ,1
L')

I

'T‘ ' vr. ,

J'Ji 'u‘
u

1‘ 3‘)
\.’.~

Ladsflmply
and sex comparisons were therefore based on a total
(560 boys and 536 girls). host of the stories were
lations. The stories of 127 boys and EC girls were
German since the transcriptions of the translations

stories had not at that time been completed.

and 556 girls had written Story

‘fiT'V rffi
.5 L\« ;L._J

SKTPLE

coders on Hexican
written by Karls—
h girls had been

'I

ll

20 Of that num-

‘
1".”
Al“‘..

< I
. 1" ‘-\-‘
' U

4
.4 e.

finished

epeated the givens. All subsequent computations

of 1,1h6 children
coded from the trans—
coded freq the original

for these children's

This method.was considered

 

justifiable since previously, reliabilities of 98.2Z had been obtained by

independently coding English translations and German originals.

The data'were tabulated for boys and girls separately and percent—
ages computed for several major category groups and utcomes.

2. Sex differences

 

(a) Thomas 9,: Truth telling

 

The frequencies and percentages of themas of Truth telling (catego-

 

ries IN through 3D) and of themas of Truth after lie (MW through SD) are

 

compared for boys and girls and presented in Table 16.

Sh.

 

 

A0>Hmoaocfl

 

 

 

 

 

 

55.

 

H.om Ham Ham QHm m.mm awn NJH mmH m.s:. 4mm mmH mNH am op eHV spate
mHHwB «HmpOB UGMHU
m.NH msH em NNH N.eH mm 4H Hm m.m mm «H H: wHH “meow spate
”mHmpopnsm
e.m OOH 0w om m.oH Ne NH om ®.e mm w on oHH noses emptopxm .m
m.: we 0 N: e.m mm m Hm ~.N mH : HH mHH “meme
hamoomcmpoomm .4
m.em mm: mam mmH m.mm mmu mNH JOH m.mm How NHH :m cho apnea
"mHmpoppsm
m.m am om a: N.@ Hm mm ow N.@ we mm Hm pemeemHesm
no soapmmSU
Hmpmm «Umhoamm .m
m.m so mm mm m.e 0: am mH m.: am mH 4H poHHmeoo taste .N
m.mm mew oeH aoH H.4m HsH Hm ow a.mm mmH a» as sHmsomempeoam .H
z a + e a a meHe 9 + e a e whom a + e m e
Hmpoe Hence Hmpoe
no a do a no a £39 mHHma.
mHth use mwmm
Irmsw .H n z Hmpoa emm n z mHnHo 0mm u z whom

 

oaoa one; oHA nmpw< spans one
spoke noHns on mofluopm .mamflu_oem .mhom mo mewmpooonmm

 

m_pemEmpmpm omafimpoa mm one mqpemEepwpm macs; «emuflamhmemu mm
samba Spams mHHeH apes.mahflo use whom mp mwfinopm mo meaoeesvmah

0H mqmda

able 16 shows that for categories of Tells truth only 35.9fi of the

 

‘

total number of ooys and 39.5? of the girls were coded. This higher fre-
quency for the girls does not represent a statistically significant dif-
ference.

9.:f of the stories written by boys and in 16.2% of the girls the-

r

54

mas of Truth after 1ie were coded. This difference was not statistically

 

significant.

Combining all the tallies for Truth only and Truth after lie yield-

 

ed totals representing hh.3$ for the boys and 55.82 for the girls. This
difference was tested and a t of h.lO showed the difference to be statisti-

3
f

cally significant eeyond the .033 level.

 

 

Frequencies and percentages for boys and girls using themas of Lies
(categories 8, 9, an“3 10) are compared in Table 17. Although the percent—
ages of children showing Action lie themas appear fairly high: 51.23 of the
boys, h8.lj of the girls, the difference between boys and girls was not

great; the t—test showed the difference not to be statistically signifi—

 

TABLE 17

Frequencies and Percentages for Boys and Girls
in Stories in which Lie Thema Occured

 

 

 

 

 

 

 

 

Boys Girls Total
Tells Lie
N : 560 N : 586 N = 1,1h6
Action
Lies spontaneously
(Category 10) 226 185 hll
Lies after weighing
circumstances or
question
(Category 9) 93 116 209
Total of children using
action lie themafi 287 282 569
Percentages of children
using action lie thema 51.2% h8.l% h9.7%
Fantasy
Plans lie, fantasies
lie
(Category 8) 36 57 93

 

* A child can use both categories 9 and 10 in one story. For
computing the total of children using Action lie themas no
child was counted twice in the sum of categories 9 and 10.

57.

(c) Themas of Emotion
Frequencies and percentages for boys and girls using themas

of Emotion (categories 18 and 19) were compared.

TABLE 18

Frequencies for Boys and.Girls in which Thema
Reveals Anxiety Before and Anxiety After Michael meets mother

 

 

 

 

 

Boys Girls Total
N : 560 N : 586 N : 1,1h6
Emotion before hh 63 107
Emotion after 10h 159 263
Totals 1&8 222 370
Percentages 26.h% 38.3% 32.5%

 

The difference of frequencies with which boys and girls used
themas of'Emotion was found to be significant statistically beyond the
.95% level.

Emotions (18, 19) Boys: 26.h% Girls: 38.3% t : h;3h

Girls used significantly more themas of’Emotion than did boys.
This does not necessarily mean that $932 girls used themas of Emotion

since in these percentages two categories, 18, Emotion befbre meeting

 

the mother, and 19, Emotion after meeting the mother were considered.

 

These categories are not mutually exclusive and therefore do not

provide the number of ooys and girls using them.

58.

 

I”)

(d) Themas 0. Mother's domination

 

Frequencies for boys and girls using the principal types of hother's

domination were compared and percentages c mputed for boys and girls using

 

Domination after truth-themas, after lie themes and the total frequency

of Dom'iation themas. They are reproduced in Table 19.

 

 

\JI
\O

TABLE 19

Frequency of Types of Punishment and Domination
in Stories by Boys and Girls

Percentages of Boys and Girls Having Michael Dominated
After Telling Truth and.After Telling Lie

 

 

 

 

 

 

 

Punishment Following Truth Following Lie Total
and Boys Girls Total Boys Girls Total Boys Girls Total

Domination
Deprivation

of food 12 25 37 12 10 22 2h 35 59
Deprivation

of playtime 13 17 3O 7 13 20 2O 3O 50
Restitution

under duress 16 22 38 8 5 13 2h 27 51
Physical:

slaps, whips 142 35 77 h8 hh 92 9O 79 169
Mbther tells

father 2 h 6 3 2 5 5 6 11
Father punishes 3 5 8 7 6 13 10 11 21
Sent to bed 1 6 7 1 2 3 2 8 10
Miscellaneous

punishment 9 9 18 6 h 10 15 13 28
mother scolds 58 98 156 h9 h5 9b 107 lh3 250

Totals 152 218 370 139 133 272 291 351 6&2
and

Percentages 27.15% 37.25:, 32.3% 214.8% 22.75:; 23.7% 51.9,"; 59.9% 56.073

60.

From the data in Table 19, tests were computed for six differ-
ences in the totals. The statistically significant differences are

given in Table 20.

TABLE 20
Percentages for Boys and.Girls in Using Thomas of Mother's Domination

Significant Differences between Boys and Girls

 

 

Percentages Level of
Boys Girls t Significance
Themas of Domination
after Truth 27.1% 37.2% 3.69 beyond .05% level
Themas of’Domination
after Truth
and after Lie 51.9% 59.9% 2.72 beyond 1% level

 

Girls used significantly more themas of.MOther's domination after

 

Michael told the truth and showed a significantly greater total number of

 

themas of Mother's domination after Michael told truth and after Michael

 

 

.3213 133, than did the boys. Since one child may have used more than one
thema of Punishment or.Domination, this significant difference between
boys and girls does not indicate whether significantly'more girls used
themas of Domination than did boys. It only shows that girls used more
themas of’Domination in writing their stories.
(e) Outcomes 93 M £2
Frequencies and percentages for the six outcomes and a combination of

the outcomes D, S, D-S for boys and girls were compared and are reported

in Table 21.

6]..

 

TABLE 21

Outcome of Story #2. Frequencies and Percentages
for Boys and Girls in the Outcome of Their Stories
Classified According to the Six Outcomes
and to the Sum of’D, S, D-S

 

 

 

Boys: N Z 560 Girls: N : 586 Total N : 1,1h6
Boys 4 Girls . Total
Freq. w Freq. % Freq. %
I. Integrative;
problem solving 10 1.8% 15 2.6% 25 2.2%
D-R.Domination—Resistance 26 h.6§ 18 3.1% hh 3.8%
A. Ambiguous, Indecisive 133 23.7%* 111 18.9% 2hh 21.3%
D. Domination 80 13.8% 86 1h.8% 166 1h.5%
S. Submission h5 8.0% 38 6.5% 83 7.2%

D-S.DominationaSubmission 266 h7.5% 318 5h.2%* 58h 50.9%

D, S, and D-S combined 391 69.8% th 75-h%* 833 72.7%

 

Total 560 586 1,1h6 lOQt

 

* By t—test, sex differences were significant at the 5% level.

62.

Significantly more boys wrote stories having Ambiguous, indecisive

 

outcomes than did girls.

Significantly more girls wrote stories which.were coded for Dominative-
submissive outcomes. This was already indicated by their significantly
more frequent use of themas of Domination as reported in Ta.le 19.

Girls were also significantly higher in frequencies of stories having out-

comes, classified in the combination D,S, and D-S.

3. Summary of sex differences.

 

More girls used themas of Tru h (Truth only, and Truth after lie)

 

 

than did boys. The? also used sienificantl more themas of Domination
.1 t: 3

accompanied by significantly greater use of themas of Emotion, and Domi—

 

nation-submission outcomes. There was no significant difference between

 

frequencies of use of Lie themas for boys and girls. Table 17 seemed to

indicate a tendency for fewer girls to use Lie themas of spontaneous action

 

than for boys. They used themas of Lie after'weighing the circumstances or

 

after questioning more frequently than boys, as well as themas of Planning

 

lie or fantasy lie (category 8). The latter seems to agree with their

 

more frequent use of themas of Emotion, both before and after meeting the

 

mother, as reported in Table 18. The explanation for this seems to lie
in the general assumption that children'who are anxious or fearful show
tendencies of attempting to forestall an increase in this anxiety. In their
anxiety, the girls apparently used more fantasy of lie which did not, how—
ever, break through to the spontaneous action level in many cases.

The only comparison of percentages of boys and girls which yielded

significantly higher frequencies for boys is that of the Ambiguous, inde-

 

63.

 

cisive outcome. This seems to indicate - and is supported by the compa-
rison of length of stories for boys and girls (Table 13) - that girls were

more precise and detailed in their expressions in writing these stories.

 

6h.

VI. COI‘ITEMT ANALYSIS FOR SELECTED GROUPS OF BOYS AND GIRLS FROM HIGH
AND F ROI-I LON SOCIO-ECONQKEC HOIVES

1. Selection of Groups
m—u—u—ufi—n

The author selected three schools located in districts of the city
with low socio-economic structure and four high schools which generally
serve boys and girls from high socio-economic homes.

Since high school training in Germany starts at the age of 10 to 11.
(after four years of elementary schooling), a rigid selection as to intel-
ligence is made for these purely university preparatory schools. Parents
have to pay tuition for the high school training of their children unless
poor parents have an unusually brilliant child who then receives a
partial or total scholarship. The low socio-economic schools which
the author selected send the fewest children to high school, and these
schools are attended by children who are in general about average in
intelligence.

A total of 95 girls and 127 boys from the low socio-economic schools
was used. A total of 122 boys and 137 girls was selected from the four

high schools.

2 0 Re 51111? S

For these groups, frequencies and percentages for the same cate-
gory groupings and outcomes were computed as for'the total sample
from.Karlsruhe. Comparisons as to socio-economic differences were
made between the high group of boys and the low group of boys, and

between the high group of girls and the low group of girls and t-tests

6S.

 

 

were computed.for all such comparisons. Table 22 gives the fre-
quencies and percentages which served as the basis for the socio-
economic comparisons. Table 23 reproduces those percentages in
which comparisons showed statistically significant differences. It
should be pointed out that in Tables 22 and 23 sex differences at

the reSpective socio-economic levels have not been examined.

66.

 

 

 

 

am.ma ooa am.0m mm mm.:e a0 mm.00 me mum .m .a
Rm.ma ma mm.ma ma wm.om mm mm.mm mm ¢ mesoopso
as.e a am.m s a .a a ua.s e mTQ
mm.m NH ma.4 m mH.H H mm. H m>flpmhwopsH
ma.mm mm Rm.ma ma am.em mm mp.om am ma .Ha .a0
.a0 .m0 .m0 maa
«H0 «mm «um poems soapmcHSOQ
mm.~m mm mm.am mm u:.nm pm a0.~m mm ms «Ow «m0
.00 «:0 .m0 Spams
.00 «mm .0m seems coapmsfiaom
no.0: m0 m0.am m0 m0.m: om u0.0m we oa .m mag
ao.em am am.ma ma a0.aa Ha zma.w Ha am . a: maa asses essay
no.0: om «0.Ne mm m®.~m on mm.mm mm an n 5H spans
& .dohm a .Uopm & .vmhm m .vmgm mmfihommpmo mmaoopfio
Nma n z mma u 2 mm H z wma n 2 use mmSoge
mass: when mHaflc whom
macho OHEocoomrowoom swam among oflsosooflrOHoom 30A

 

moses OHEosoomloaoom swam Song cmhcaflso he maoonom Adom use
meson oefiosoomTOfloom seq seam coneaflno mp popcmpp¢ who: mHOnsom mossy

wHoonom cm>em Seam cohpaflgo mo endows 029 Mo mauflo one when as
can: mosoopdo pom mesmee popomaom mo mommpceOHom pew meflocmsvmhm

mm mqmda

67.

TABLL 23

Percentages and Significant Differences for Boys and Girls
from Two Different Socio—Economic Groups
for Various Category Groupings and Outcomes*

 

 

 

 

Category Group Percentages Level of
and Outcome High Group Low Group t Significance
B0 3
Truth h2.6% 29.9% 2.09 5% level
Domination after .
truth 15.5% 30.7% 2.90 beyond .05%
level
Outcome: Ambiguous 12.3% 29.9% 3.h9 beyond .05%
level
D, S, D-S 80.3% 66.9% 2.h3 % level
Girls
Truth after lie 27.0% 11.6% 3.06 beyond .05%
level
Domination after . .
truth 38.7% 26.3% 2.13 5% level
Outcome: ambiguous 13.9% 30.5% 2.98 beyond .05%

level

 

* The other differences were not found to be significant statistically.

68.

3. Summary 33 findings for boys and for ggrls

 

(a) Boys

The high group of boys used significantly more themas of Truth than

 

did the low group. he low group, on the other hand, used significantly more

hemas of Domination after Truth and more boys of the low group wrote

 

stories that were Ambiguous,indecisive in outcome.

 

The high group of boys had significantly more stories of Dominative-

 

submissive character than did the low group. This is in part eXplained by

 

the small percentage of boys in the high group who wrote stories with Am-

biguous, indecisive outcomes.

 

(b) Girls

More girls of the high group used themas of Truth after lie than did

 

the low grou . Also in the high group, there was a significantly greater

percentage of usage of'Domination after truth themas.

 

Girls of the low group wrote significantly more stories with.Ambiguous,
indecisive outcome than did the high group. A hypothesis which is forth-

coming is that the significantly lower frequencies of Ambiguous, indecisive

 

outcome in the boys as well as in the girls of the high group is partly due
to their greater facility in writing, their higher level of intelligence,

and thus of understanding and acuity of expression.

69.

VII STXIARY

This study on the problems of reliability in evaluating story com-
pletions about social conflicts by German adolescent children is part of
a larger cross-cultural and cross-national research program. The present

study is concerned'with Story # 2 of Series A of the Anderson Incomplete

 

Stories. These incomplete stories were developed in order to get an ex-
pression by children as to their judgment of the processes and outcomes
of social conflicts.

The writer used for his study children's stories collected from 560
boys and 586 girls in Karlsruhe, Germany, as well as small samples from
data collected in Mexico.

The writer translated all stories of the Karlsruhe group. Translation
problems are discussed.

The coding manual for Story f 2 comprises 80 process—categories for
Michael and the Eother, as well as six interpretive—evaluative outcomes.
The storieS'were coded as to these 80 process-categories and six outcomes
by three independent coders. In the courde of establishing coder reliabi—
lity, the categories were refined so as to eliminate various sources of
errors in coding. It was found that disagreements between coders arose
mainly with regard to categories where interpretive inferences had to be
made.

Reliabilities of coders in coding for outcomes of 89.23, 100$, and
87.1% for three groups of children were obtained. ‘When coding process-

categories, two independent coders obtained reliability of 86.73. Re-

liability after a four weeks' interval was 88.03.

70

Since in the larger cross-cultural and cross—national study great
mountsof data have to be coded, an investigation was made as to time
required for a person to become a reliable coder. It took one person
22 hours of preparatory work to achieve reliability of 86.73 for cate-
gories and 8h.0% for coding outcomes.

The validity of the translations was investigated since most coding
was based on translations of the stories from German into English. For
this purpose, German originals and English translations were coded by
independent coders and reliabilities of 92.63, 95.03, and 92.9f for cod-
ing categories, and percentages of agreement of 89.8, 100, and 87.2 for
coding outcomes were obtained.

The reliability of the translator was infestigated by coding the
same stories from two translations which had been made with one year in—
tervening. Reliability of 97.23 was obtained.

Since the coding was done during the process of continuous refine-
ment of he coding manual, it was thought important to know how consistent
a coder was over a period of tine during'which there was discussion and
work on the coding manual. By coding - re-coding method reliabilities of
97.23 for coding categories and 97.0f for coding outcomes were obtained.

The high reliabilities obtained demonstrate that after a short, though
intensive training period, reliable coding of Story # 2 can be done.

For purposes of obtaining the degree of reliability with which child-
ren used similar themas in writing their stories again after an interval
of two weeks, two groups of 3h boys and 3h girls were given the form twice.
Truth and Eli themas in both stories were used by 6h.7% of the boys and

by 67.6% of the girls. Categories of Mother's domination and of‘Michael's

— m-n.-.‘ -§... -. .- - --

71.

submission were used in both stories by 6h.7% of the boys and 76.h% of the

 

girls. Identical outcome categories were found in 70.5% of the boys, and
in 88.0% of the girls. Girls were more reliable than boys in'writing sto-
ries yielding the same coded outcomes.

The writer coded Story # 2 for all children from the Karlsruhe sample.
He made sex comparisons as to several category groupings. It was found that

more girls used themas of Truth, themas of Emotion and stories which yielded

 

99mination-submission outcomes, than did boys. The boys wrote significant-

ly more stories which were coded for Ambiguous, indecisive outcome.

 

Two groups of boys and girls were selected from schools which serve
children from low and from high socio—economic homes. Comparisons were
made between the low and the high groups. The high group of boys used signi-

ficantly'more themas of Truth, and.fewer themas of Domination after truth,

 

as well as fewer storieS'with Ambiguous, indecisive outcome than the boys

 

 

in the low socio-economic group. The high group of boys had significantly

more stories with outcome of Domination-submission, than did the boys in

 

the low group. More girls of the high group used.themas of Truth after lie

and themas of Domination after truth, than did the low group of girls.

 

Girls of the low group wrote significantly more stories with.Ambiguous, i2?

decisive outcome, than did the high group of girls.

72.

1.

3.

Anderson,

Anderson,

Anderson,

Anderson,

Anderson,

Anderson,

*4
.14

. H.

H. H.

H. H.

REFEAEHCES

Domination and Integration in the Social
Behavior of Young Children in an Experi-
mental Play Situation. Genet. Psychol.

Monogro, 19: 3h1 ‘ bOB, i932

and.Anderson, G. L. "Social Development",

in Manual of Child Psychology, 2nd Edition,
L. Carmichael, Ed., New YOrE? John Wiley &
Sons, Inc., l95h

 

and Anderson, G. L. Children's Perceptions

of Social Conflict Situations: A Study of
Adolescent Children in Germany. American
Journal of Orthopsychiatry, Vol. XXIV,
17—10. "2:152:11 195E

 

and Anderson, C. L. Coding.Manual for

Anderson Incomplete Stories. Unpublished
Manuscript

H., and H. M. Brewer. Studies of Teachers'

Ho Ho ,

Classroom Personalities, I: Dominative and
Socially Integrative Behavior of Kinder-
garten Children. Appl. Psychol. Monogr.,
NO. 6, 19,45

 

 

and J. E. Brewer and.M. F. Reed, Studies of
Teachers' Classroom Personalities, III:
Follow-up Studies of the Effects of Dominative
and Integrative Contacts on Children's
Behavior. Appl. Psychol. Monogr. Stanford
University Press, 1916_

 

Hylla, Erich, Editor, Bericht ueber die Internationale

Arbeitsgemeinschaft fuer Paedagogische
Psychologie. ‘Mimeographed Report, Hochschule
fuer Internationale Paedagogische Forschung,
Frankfurt, Main, Germany 1952

l.

2.

3.

APPFNDIX A
Text of Anderson Incomplete Stories, Series A

Frank and Peter were walking to school. Suddenly Frank grabs
Peter's cap and throws it high up into the nearest tree so that
Peter could not reach it with his hand. Frank had never done
anything like this before. Neither had Frank and Peter
quarreled with each other the day before.

'Why did Frank do this? What does Peter do?
What does Frank think? How does Peter feel about it?

Think about these questions and then finish the story quickly
with a few sentences.

Michael's mother sends him to thesstore to get one pound of
wieners. On the way home he puts the package of meat on the edge
of the sidewalk and plays for a short time with his friends. A
shepherd dog darts forward quickly, claws half of the wieners out
of the package and rushes away with them. Michael wraps up the
remaining wieners and takes them home.

What does Michael say to his mother?
What does his mother do?
How does Michael then feel about it?

Think about these questions, then finish this story quickly with
a few sentences.

The teacher suddenly discovers that fifty cents has disappeared
from her desk. She looks up and sees that all the class are
working on their arithmetic assignment. She wonders what
happened to the money and what she should do.

What does the teacher do?
Finish this story also with a few sentences. Tell what happened

to the money and also exactly how the teacher feels and what she
does.

Elizabeth is sitting in the living room working on her
homework. Suddenly she remembers her mother's new coat.
She wonders how she would look in it. 'When taking the
coat off she notices that she had got some ink spots on
her mother's new coat. Just as Elizabeth is trying to
wipe out the ink spots her mother enters the room.

What does her mother say? How do they both feel about it?
What does Elizabeth say? ‘What does each one do?

Think about these questions and finish this story quickly
with a few sentences.

George and Karl are playing with a football. They know
that they should not play football on the narrow space

in front of the house. George gives a strong kick and the
ball hits a window and makes a big crack in it. Karl
thought that someone had appeared behind the window. No
one could have seen who had kicked the ball against the
“window.

Finish this story with a few sentences. Tell hOW’the two
Boys felt about it and what they did.

Elsa often handed in her home composition late to the
teacher. This time it was an especially important
composition and she had, moreover, written it on time.
On the way to school she lost her composition notebook
and could not find it anywhere.

What does Elsa say to her teacher?
What does the teacher say?

Think about these questions and finish this story with a
few sentences.

ROOEEA USE ONLY.

If.” 27

I”! 30 957
1 31,1. 3 P.

W...

-Q IDQL—rfi

7W

-I.—q
‘sr - _

Ih'IA -*7 P I P
h ; ‘e - ,
E :\}i 5‘ , ‘31
J