mmmmuw as: HEM. wmwms AS A vmnmm EN A mwmmv SCALE “was, {few five Dogma a? M. A. MECHEGAN STi'E‘E UNIVERSE??? W'aallace George Berger E968 ”mi [J LIBRARY " “85‘; Michigan State ‘ i ii“? University Q" ABSTRACT EMOTIONALITY OF ITEM WORDING AS A VARIABLE IN A PERSONALITY SCALE by Wallace G. Berger It has long been recognized that the form in which an item's message or content is stated, the wording that happens to be used, is one source of variance in personality inventory scores. Probably the main difficulty in dealing with this source of variance in any practical way is that the message contained in a particular item can be conveyed through the use of such a multitude of word combinations. Under the circumstances, not surprisingly, few wording variables have been systematically explored. A reason for this neglect has no doubt been the recognition that wording variables were too specific to particular items or scales. Research was not seen as leading to general conclusions that could serve as guides to inven- tory constructors. One class of wording variable that does, however, occur across many commonly used personality scales and may therefore serve as a general guide is the judged emotionality or affect associated with various ways of stating items. In the present study E posed the following questions: 1. To what extent does varying the emotionality of the wording of a personality item affect its strength of endorsement? 2. How can the endorsement-by-emotionality relation- ship be characterized, i.e., linear or otherwise? 3. Wallace G. Berger To what extent are individual differences in susceptibility to the emotionality of item wording related to the "trait“ purportedly measured by the particular personality scale? The method selected to explore these questions was to take a particular scale, the Short Form Dogmatism Scale (SFDS) and construct two alternate items for each of the original items, with each of these alternate items differ- ing from the original in emotionality of wording. The analysis of the data indicated that: l. a. For 12 of the 20 triplets (one triplet = an original SFDS item and 2 alternate items with a message similar to that of the origi- nal but differing in emotionality of wording) an inverse relationship existed between en- dorsement to the item and emotionality of wording. Three "parallel" forms of the SFDS were constructed from the triplets such that they varied in their general level of emo- tionality of wording (low, medium, and high). Of the three possible paired com- parisons among these forms, two were found to be significantly different in mean en- dorsement (low-high, low-medium), with the higher emotional form being endorsed to a lesser extent. 2. The shape of the endorsement by emotionality relationship appeared to be best characterized as linear. No significant relationship was found between the "trait" designed to be measured by the in- ventory and individual differences in responding to emotionality of wording. It was also noted that neither the variance nor the internal consistency nor the "validity" (the correlation of the low, medium, and high forms with the actual SFDS) of the "parallel" forms were significantly modified by their general level of emotionality of wording. Wallace G. Berger The implication of these findings for personality "test" constructors is that in general, emotionality of wording is not an important factor to take into account under the rather narrow range of conditions which the phenomena were explored in this study. Approved: W /Z. («J—“M Date: 1’77—‘6’8 Thesis Committee: F. R. Wickert, Chairman C. Hanley M. Rokeach EMOTIONALITY OF ITEM WORDING AS A VARIABLE IN A PERSONALITY SCALE BY Wallace George Berger A THESIS Submitted to Michigan State University in partial fulfillment of the requirements for the degree of MASTER OF ARTS Department of Psychology 1968 3...... To my wife, Maureen ii AC KN OWLE DGMENT S I would like to thank Dr. F. R. Wickert, Chairman of the thesis committee, for his extensive guidance in the preparation of this manuscript. His assistance has been invaluable in locating and clarifying both logical and linguistic difficulties. To Drs. C. Hanley and M. Rokeach for their valuable suggestions and comments, I would like to express my gratitude. I would like to extend special thanks to Mr. G. M. Gillmore, who helped construct the alternate items used in the pilot study, and Dr. J. E. Hunter, whose ideas proved instrumental in formulating the problem. iii DEDICATION TABLE OF CONTENTS ACKNOWLEDGMENTS O O O O O O O C O O O O O O O O O O 0 LIST OF TABLES I O 0 O 0 O O O O O O O O O O O O O 0 LIST OF APPENDICES . . . . . . . . . . . . . . . . . CHAPTER I INTRODUCTION 0 O O O O O O O O O O O O O O 0 Review of the Literature. . . . . . . . . Questions Posed . . . . . . . . . . . . . II METHOD. . . . . . . . . . . . . . . . . . . Instrument Deve10pment Study. . . . . . . Phase 1: Item Selection . . . . . . . . Subjects. . . . . . . . . . . . . . . Procedure . . . . . . . . . . . . Phase 2: Instrument Tryout an Item Selection . . . . . . . . . . . . . . . Subjects. . . . . . . . . . . . . . . Procedure . . . . . . . . . . . . . . Phase 3: Employment of th Develope Instrument. . . . . . . . . . . . . . . Subjects. . . . . . . . . . . . . . . Procedure . . . . . . . . . . . . . . III RESULTS 0 O O O O O O O O O O O O O O O O 0 Some Characteristics of the Instrument. . The Relationship of Endorsement and Emotionality Within Similar Messages (Question 1). . . . . . . . . . . . . . "Parallel" Forms. . . . . . . . . . . . . The Endorsement by Emotionality Relation- ship Across Messages. . . . . . . . . . iv Page ii iii Vl vii \JN 10 10 10 10 12 12 12 l4 14 15 l6 16 20 22 24 CHAPTER TABLE OF CONTENTS (Continued) III (Continued) The Shape of the Endorsement by Emotionality Relationship (Question 2). . . . . . . . . . . . . The Relationship Between gs} Response to Emotionality and Total Scale Score (Question 3). . . . . . . . . . IV DISCUSSION. 0 O O O O O O O C C O O O O 0 Some Comments on Message Similarity . . The Relationship of Endorsement and Emotionality Within Similar Messages (Question 1). . . . . . . . . . . . . "Parallel" Forms. . . . . . . . . . . . The Endorsement by Emotionality Relationship Across Message . . . . . The Shape of the Endorsement by Emo- tionality Relationship (Question 2) . The Relationship Between gg' Response to Emotionality and Total Scale Score (Question 3). . . . . . . . . . Some Practical Considerations . . . . . Suggested Research. . . . . . . . . . . BIBLIOGRAPHY O O O O O O O O O O O O O O O O O O O APPENDICES Page 25 27 28 28 29 31 32 32 33 33 34 35 38 Table LIST OF TABLES Likert and successive category values for the emotionality ratings . . . . . . Intercorrelations of the forms . . . . . . Paired comparisons within triplets . . . . Frequency distributions of the mean emotionality ratings of the items in the three "parallel" forms . . . . . . . Means, standard deviations and significant differences of the three "parallel" forms 0 0 O O O O O O O O O O O O O O O 0 Representation of endorsement by emotionality relationship within triplets O O O O O O O 0 O O O O O O O 0 vi Page l7 19 21 23 24 26 LI ST OF APPENDICES Appendix Page A Oral instructions given prior to the administration of the instrument . . . . . 38 B PilOt inStrmnent O O O O O 0 O O O O O O O O 40 C Final instrument . . . . . . . . . . . . . . 52 D Table A -- Mean endorsement and mean emotionality ratings and endorsement intercorrelations within triplets arranged by emotionality level . . . . . . 65 Table B -- The distribution of SFDS total scores and corresponding correlations between degree of endorsement and emo- tionality rating for each subject. . . . . 67 E Message triplets . . . . . . . . . . . . . . 68 vii CHAPTER I INTRODUCTION One of the possible sources of variance in personality measures is the manner in which an item's content (or message)* is worded. A reason for the concern with item wording stems from the fact that the message contained in a particular item can be conveyed through the use of a multitude of word combinations. The wealth of different phrasings that are possible in English allows for the in- troduction of subtle "contaminants." These contaminants could make for selective responding to irrelevant, non- message variables on the part of subjects. Most of these wording variables have been relatively unexplored. Perhaps a reason for this neglect has been the recognition that many of these wording variables were too specific to particular items or scales. Research on such specifics would not lead to useful general principles. *The term “message" rather than "content" will be generally used in this report. "Message"as a term is more precise and less ambiguous in the context of this research. A class of wording variables that does, however, occur across many commonly used personality scales is the emotionality or affect associated with various ways of stating items. The emotionality of the item wording could, as well as the item.message, play a significant role in determining the response given to the item. Emotionality of item.wording as a variable in per- sonality scores should be viewed in the context of the current research on response bias. Rundquist (1966) lists five classes of variables which can be influential in de- termining the responses given to a personality item or scale. "Among the many sources of variance in personality scores, there appear to be five which have par- ticular relevance to response bias: content, item scale-values on such continuums as social desirability, form in which the content is stated, sets to create a definite impression, and the nature of the response scale." (p. 166) In the above classification, emotionality of item wording falls within the category of "item scale-values on such continuums as social desirability."* Of particular relevance to the present study are experiments demonstrating the feasibility of separating *In Rundquist's schema "The term form of statement (the category in which emotionality of wording would logically seem to fall) refers to the fact that an item is so stated that its. . .scale value (on some variable) is on one or the other side of the neutral point." (p. 168) The cate- gory of "form in which the content is stated" is therefore restricted to item reversals on such variables as social desirability, etc. The only possible other category for emotionality of wording to fall into is the one indicated. affect and message in verbal communications. Starkweather (1956) through mechanical means reduced voices to a "low mumble“ so that the message was lost. The subjects were then asked to identify the affect associated with each of the ninety tapes. The results were congruent with the hypothesis that some affect information remains in speech filtered content free. Davitz and Davitz (1959) came to the same conclu- sion when they asked speakers to recite the alphabet and at the same time to vary their emotional expression. Ob- viously, the alphabet represented a constant (message). Subjects exposed to the recordings of these alphabetic recitations were able to a significant degree to identify the various emotions expressed. These two studies, then, showed that in the case of verbal communications subjects were able to identify emo- tional expression independently of message. Propaganda research reports some effects on subjects of using messages with varying degrees of planned emotional content. Two widely referenced articles are relevant. Hartmann (1936), preceding an election, circulated two leaflets (one a so-called "emotional" appeal, and the other a so-called "rational" appeal) in non-overlapping sections of the same city. The results indicated that the "emo- tional" appeal was more effective, i.e., elicited a higher proportion of favorable votes. Janis and Feshbach (1953) administered three levels of emotional appeals (via recorded lectures) to a group of high school students. The results indicated that the mini- mal emotional (the emotion in this case was fear) appeal was the most effective form of communication in producing reported conformity to dental hygiene practices advocated by the message. Within both the Hartmann and Janis and Feshbach studies not only did the emotionality of the presentation vary, but the message itself was modified in the various appeals. Hartmann presented messages that varied along an "emotional-rational“ dimension and at the same time along some potentially salient social issues (economic policy, war involvement, etc.). Similarly, Janis and Feshbach did not keep message content as constant as they might have. They not only varied the degree of fear reflected in the appeals, but also the vividness of the slides shown, the personalization of the threat reference, and the stress placed on bodily harm. Many propaganda studies, then, like those of Hartmann and Janis and Feshbach, did not do a sufficiently precise job of keeping separate the affects of the message and the effects of emotionality of wording. Allman and Rokeach (1967) devised a very simple, direct, systematic way of trying to assess the relative effects of emotional (affective) as compared with cognitive orientations of item wording on responses to several dif- ferent personality measuring instruments. In effect, they compared responses to effectively oriented phrases such as "I feel. . .“ with cognitively oriented phrases such as "I believe. . ." or "I think. . .“ Using a test- retest paradigm these investigators found that the re- sponses to Likert-type items did not vary as a result of manipulating the cognitive and affective orientation in the above mentioned manner. Evidently stronger, more varied ways of getting emotionality into the wording of items was called for. As part of their study, Weiss and Lieberman (1959) attempted to vary the emotionality of a set of written personal descriptions while holding the message of these descriptions constant. The §§_were presented with both a favorable and unfavorable set of personal descriptions which had "equivalent" messages, but varied in emotionality of statement ("emotional“ or “non-emotional"). The §§ were then asked to indicate, through a multiple-choice format, their attitude toward the person described. An example of two positive favorable personal descriptions given in the article is: “he was loved and cherished by all for the fairness and decency of his actions" ("emotional“) versus "he was held in high esteem and with devoted attachment by all for the upright impartiality and correctness of his actions" (“non-emotional“). Weiss and Lieberman found that when an unfavorable description (regardless of its emotionality level) was followed by a favorable non-emotional description, a greater change in attitude toward the person described (p = .02) occurred than when an emotional set of statements followed the initial unfavorable description. This study as presented appears to have several methodological deficiencies, the most important of which (in terms of the present study) is that the personal des- criptions were not equated for similarity of message or rated for degree of emotionality of wording. The present study attempted experimentally to isolate from each other the effects of two variables, emotionality of item wording and item message, on responses E2 make to the items on one commonly used personality scale. The four other possible sources of variance mentioned by Rundquist as playing a role in influencing responses to personality—scale items were, as far as possible, held constant. The items employed in this study were assumed to vary along two dimensions. One of these dimensions was the message carried by the item. These messages were givens, i.e., they were the messages found in the original items on the personality scale which was used in this study. The second dimension, the emotionality of item wording, was the one manipulated by the experimenter. The following statement from Weiss and Lieberman applies to the procedure employed in the present study: "The basic assumption underlying the procedure is that affect may be provoked by the kinds of words used to describe an object (or situation)*. That is, besides their cognitive significations (message)*, words carry an affective loading ygiven by their association with value judgments, ideals, morals, etc., as a consequence of the social and personal experiences of the communi- catees. . . . A further assumption is that the affect is associated with certain words or phrases rather than with cognitively equivalent ones (same message)* . . .“ (p. 129) For each of the original item messages, in the SFDS, then, several alternate items were constructed. Each of these alternate items contained a message similar to the original item from which it was constructed, but differed in its emotionality of wording (independent variable). The dependent measure in this study was the extent to which sub- jects agreed with or endorsed the items. The subjects' de- gree of endorsement was measured by their responses (from "strongly disagree" to "strongly agree") on a Likert scale. The degree of endorsement to particular items on a personal- ity scale could, within the framework of this study, be attributed to the item's message and/or the emotionality of item wording. This study posed three specific questions, the an- swers to which should clarify the extent to which emotion- ality of wording was a special factor in influencing lresponses to items that make up a particular personality scale. The questions were: 1. To what extent does varying the emotionality of the wording of an attitude item, while holding *The present author's own insertion. 3. message constant, affect its endorsement? How can the endorsement by emotionality rela- tionship be characterized, i.e., what shape does this relationship take, linear or other- wise? To what extent are individual differences in susceptibility to the emotionality of item wording related to the personality characteris- tic purportedly measured by the particular personality scale? CHAPTER II METHOD The method here employed to separate item message from item wording entailed constructing items containing similar messages but varying in emotionality of wording. Three "parallel forms" of a scale were so constructed that message was held relatively constant, but emotionality of item wording was expressed in one of three degrees - low, medium, high - for each of the three forms. The personality scale used in this study was the Rokeach Short Form Dogmatism Scale (SFDS). The advantages of this scale were: (1) its constructor, Dr. Milton Rokeach, was available for consultation, (2) it was found to be a reliable measure (Troldahl and Powell, 1956), (3) it was of manageable length, (4) most importantly, the scale's original items happened to span a wide range of emotionality of wording. The procedure used in this study can be divided into three phases. The first two phases (Phase 1 and Phase 2) were concerned with developing the instrument and at the same time served as pilot studies. The pilot studies were necessary in order to present evidence that 10 emotionality of wording could be manipulated independently of message. In the third phase the instrument developed in Phases 1 and 2 was administered for definitively test- ing the basic hypothesis of the study. INSTRUMENT DEVELOPMENT STUDY Phase 1 Subjects Five (5) graduate students (males) and 2 faculty members (males) in the Department of Psychology at Michigan State University served as judges for the first phase of the instrument develOpment study. Procedure The §_had previously constructed a minimum of four (4) new items for each of the original twenty (20) items on the SFDS. The original and newly constructed items were typed on 3 x 5 cards and submitted to the judges. They independently were asked to assign to each item a value ranging from 1 to 5 to indicate the item's strength of emotional wording. The assignment of numbers was ac- complished through the use of a card-sorting procedure. The values (1-5) assigned by the judges to the items were recorded. The E then instructed the judges to compare each of the four or more new items with the original SFDS item (or standard). The judges also indicated whether a constructed item differed in message as well as degree of ll emotionality, from the standard, through verbal report to the E. Three "parallel forms" differing in emotionality of wording were constructed. The choice of three rather than some other number of forms was based on the following con- siderations: (1) with fewer than three forms of a particu- lar message, it is not possible to ascertain whether the relationship between emotionality of wording and degree of endorsement to that message is linear or non-linear, and (2) if more than three forms were used, the task for the subjects would in all likelihood be too lengthy and tedious. The item selections began by discarding all those items judged by two or more of the judges to differ in message from the standard SFDS item. Next, the mean emotionality values of the original SFDS item and the means of the newly constructed revisions of the SFDS items were computed from the data generated by the card-sort procedure. Two new items for each original were selected and retained for inclusion in the first questionnaire if their mean emotionality ratings differed from both the original item and each other by at least 1 point (except in Triplet #10 and 20, where it was found practically impossible to vary the item's emotionality of wording without changing its message). (See Appendix E). An instrument consisting of 20 original items plus 2 constructed items for each of these 20 original items (making a total of 60 items) was then constructed. The 12 questionnaire was constructed such that the 60 items were randomized (through the use of a table of random numbers) with the constraint that each member of a triplet was a distance of 20 items apart. (See Appendix B). Phase 2 Subjects The subjects were a volunteer-group of 57 Michigan State University students (22 males, 35 females) enrolled in a Junior-level Psychology course, during the Spring Quarter of 1966. Procedure The intent of Phase 2 was: (1) to more precisely arrive at the emotionality ratings of the items (as judged by the 57 subjects), (2) to use these ratings as a basis for the construction of the final instrument, and (3) to try out the instructions and the format of the instrument on a student sample. Oral instructions were given by the E immediately prior to the administration of the instrument in order to emphasize the importance of responding independently to each item. (See Appendix A). Before a subject could be- gin work on the instrument, he read to himself approximately the same instructions presented to him in verbal form by the E (see Appendix B). Since a variation of each original item appeared twice more on the instrument, the subjects 13 were instructed not to strive for consistency on the three items but simply to evaluate each item on its own merits, regardless of his responses to similar items. The instrument consisted of three sections. In Section 1 the subjects were instructed to rate each of the 60 items on a five-point Likert scale, 1 to 5, indicating degree of endorsement with the item, (from "strongly dis- agree" to "strongly agree"). The sole use made of these ratings was to test the format of the final instrument, i.e., to see the extent to which the instructions were accurately followed and to gauge the time required for the task. In the final instrument these ratings would be used to determine the degree of endorsement with the items. In Section 2 the subjects were again presented with the same 60 items. This time, however, they were instructed to rate each item in terms of how emotionally worded they thought the item to be. These data were then used to assign scale values to the items relative to their emo- tionality of wording. Section 3 consisted of twelve (12) biographical information items to help describe the subjects. After the above data were gathered, the means characterizing the items were calculated. Items were then grouped into message triplets, i.e., each triplet consisted of the original (SFDS) item and the two items constructed from the original; thus there were 20 such triplets. Each 14 of the three items in the triplet was then designated as either a high, medium, or low emotionality rated item by comparing their mean ratings. Those items in each triplet which did not significantly (p‘< .05) differ from at least one other item in that triplet were excluded from the final instrument. Also an attempt was made to arrive at three emotionality levels (low, medium, high) in which the ranges were non-overlapping with respect to their emotion- ality ratings. For example, if an item were designated as high (emotionality rating) in a particular triplet and at the same time its emotionality rating fell within the range of emotionality ratings of the medium or low 20 items, it was excluded from the final instrument. The above criteria necessitated the elimination of 17 of the 40 items. The E constructed a minimum of two new items for each of the items eliminated and submitted these new items to the same panel of judges employed in Phase 1. The procedure used for selection of the 17 new items was identical to that used in Phase 1. EMPLOYMENT 9E THE DEVELOPED INSTRUMENT Phase 3 Subjects The revised instrument (see Appendix C) was ad— ministered to a volunteer group of 77 Michigan State Uni- versity students (28 males, 49 females) enrolled in several 15 introductory psychology classes, during the Fall Quarter Of 1967. Procedure The format of the instrument and the instructions given were the same as those used in the phases of de- veloping the instrument (Phases 1 and 2). CHAPTER III RESULTS Some Characteristics of the Instrument Before the data were analyzed, it was necessary to test an important assumption that underlay the use of the parametric analyses (or "tests" in the statistical sense) that were to be used in the analysis. An equal-interval scale is assumed when using parametric statistics. A problem that can arise when using ratings on Likert-type scales is that when subjects are asked to scale items on certain dimensions, the resulting intervals between the categories may not be equal. Therefore, before the sta- tistical analysis of results was undertaken, the emotional- ity ratings of the items were scaled by the method of successive categories to test for linearity between the assumed equal-interval Likert Scale (as used in the questionnaire) and the empirically derived scale (as calculated by the successive category method). Guilford (1954) pointed out an inherent problem in attitude scales: “The scaling problem is to estimate the values of the categories or of their limits, along the psychological 16 l7 continuum, and from these reference values to derive interval-scale measurements of stimuli." (p. 223) The values computed by means of the successive category method were compared with those assumed when using a Likert Scale. The relation between these two scales is displayed in Table 1. TABLE 1.--Likert and successive category values for the emotionality ratings Likert Values 1 2 3 4 5 Scaled Values 1.09 2.12 3.00 3.85 4.99 The correlation between the Likert and scaled values was .96. This high a correlation indicated an essentially linear relation between these two sets of values. The strength of this correlation indicates that the use of the raw Likert values or the scaled values would lead to essentially the same results. Therefore, in all of the further analyses in which emotionality ratings were used, these ratings were based on the raw Likert Scale values assigned by the subjects. By the way it was constructed, the instrument could be divided into various parts or components. Each of the components (listed in Table 2) represented a segment of the instrument which was of relevance to this study. These intercorrelations could be interpreted in two ways: (1) as measures of the components' reliabilities ("paralle1" 18 forms) since the intercorrelations are those among scales which have similar messages, and (2) as predictor-criterion correlations, if we view the SFDS score as the criterion and the scores on the other constructed scales as the predictors. Using the first interpretation it is not surprising to find that many of the intercorrelations are larger than the scales' internal consistencies. Internal consistency can serve as an estimate of the scale's reliability only so far as the scale is homogeneous (one factor). Rokeach and Fruchter (1956) have in fact found, upon factor analy- sis, that the Dogmatism Scale was indeed multidimensional. Because of the manner in which the SFDS was constructed (see Troldahl and Powell) it is most likely that it (SFDS) is also multidimensional. In the case of a multidimensional scale, measures of the scale's internal consistency will considerably underestimate the scale's reliability. If the correlations between the new scales and the SFDS are viewed as validities, then their size makes it plausible to assume that the components are to a large extent tapping the same dimension(s) as the original SFDS. This finding lends some indirect support to the proposition that item wording, in the form of emotionality of item construction, can be manipulated without seriously chang- ing the item message or content. 19 It should be noted that there is no statistically significant difference in the scales' internal consisten— cies: .63 (LOW), .65 (MEDIUM), .70 (HIGH), and .69 (SFDS). In addition, the "validities" (correlations of LOW, MEDIUM, and HIGH with the SFDS) were computed. Those items that the LOW, MEDIUM, and HIGH scales had in common with the SFDS were omitted in the computations of these validity coefficients. Then the validities were recomputed correcting the LOW, MEDIUM, and HIGH scales to the same number of items (20). It is interesting to note that the validities were as follows: .80 (LOW-SFDS), .80 (MEDIUM- SFDS), .82 (HIGH-SFDS). None of these validity coefficients was significantly different from each other. TABLE 2.--Intercorre1ations of the forms. F0 m3 LOW MEDIUM HIGH SFDSl LOW (.63)2 .683 .70 .66 MEDIUM (.65) .82 .70 U) g HIGH (.70) .79 13-4 SFDS (.69) 1. This column represents the correlations between the SFDS and the other forms. Any items that the SFDS had in common with another form was removed from the other form before the correlation was computed. 2. The parenthesized values down the diagonal are the internal consistencies of the scales (Cronbach's alpha). 3. All correlation coefficients are significant beyond .01 level. 20 The Relationship of Endorsement and Emotionality WIEHTK_§imIIar Messages (Question 1) In order to determine the degree to which varying the emotionality of wording of an item would affect en- dorsement to the item, Scheffé's method of pairwise com- parisons was performed within item triplets. The Scheffé approach has the optimum pr0perty that Type 1 error is at most for all of the possible comparisons. Thus if three or fewer within-triplet comparisons were significant at some level CK, then the over-all probability of falsely rejecting the null hypothesis for any and all of these comparisons will at most be the pre-determined level CK. Table 3 presents the results of the paired comparisons. The results of the analysis of the emotionality ratings in- dicated all 20 triplets but two (Triplet #8 and #10) had at least one significant difference in the item's emotionality ratings. Out of the sixty possible comparisons, 41 were significant, 34 at the .01 level, and 7 at the .05 level. The results of the analysis of the endorsements indi- cated that of the 20 triplets, 12 were found to have at least one significant difference in the endorsement; the remaining eight (Triplets #2, 4, 5, 8, 10, ll, 13, 14) showed no significant differences. Eighteen (18) of the possible 60 comparisons were significant, 9 at the .01 level, and 9 at the .05 level. 21 TABLE 3.--Paired comparisons within triplets. Significance tests Significance tests LOW- ME D- LOW- LOW- ME D- LOW- TRIP.# MED HIGH HIGH TRIP.# MED HIGH HIGH 1 I * l l O ** ** ** 2. 12. ** ** * ** ** ** 3 O * * l 3 O ** ** ** ** 4. l4. * ** ** ** ** 5. 15. ** * ** ** * ** 6. * * l6. ** ** ** ** ** 7- * l7. * * ** ** ** ** ** 8. 18. ** * * * ** 9. * 19. * ** ** ** ** ** ** 10. 20. ** ** ** * ** *Significant at .05 level. Obtained by the Scheffé Method. **Significant at .01 level. (see Winer (1962) p.88) Note: In each triplet the first row is endorsement and the second row is emotionality ratings. Of these 18 significant differences all suggested the existence of a negative relation between endorsement to the item and the item's rated emotionality (i.e. the higher the item's emotionality, the less the endorsement of the item). 22 In 11 of the 20 triplets (#‘s 1, 3, 6, 7, 9, 12, 16, 17, 18, 19, 20) a significant difference in endorsement was found between one of the triplet items and the original (SFDS) item in that triplet. "Parallel" Forms Another way in which we can evaluate the endorse- ment-emotionality data is by using the 60 items to con- struct 3 scales (each containing 20 items), the items of each scale having similar messages but the scales themselves differing in their emotionality ratings. The procedure used for the construction of these three "parallel“ forms was described in the methods section (see page 13 and Table 4). Scheffe's paired comparisons test was performed on both the mean endorsement and mean emotionality ratings of the three forms (see Table 5). A significant difference (p<< .01) was found between the mean endorsement to the Low vs. Medium and Low vs. High emotionality levels. These differences were in the same direction as the majority of differences found in the within-triplet comparisons. All of the paired comparisons among emotionality ratings were significant (p«(.01) as was expected because of the manner in which the "parallel" forms were constructed. 23 TABLE 4.--Frequency distributions of the mean emotionality ratings of the items in the three "parallel" forms Mean item Forms emotionality rating LOW MEDIUM HIGH 4.4-4.3 3 4.2-4.1 2 4.0-3.9 4 3.8-3.7 4 3.6-3.5 4 3.4-3.3 3 3.2-3.1 4 2 3.0-2.9 2 4 2.8-2.7 5 3 2.6-2.5 2 3 2.4-2.3 5 3 l 2.2-2.1 4 2.0-1.9 l 1.8-1.7 l The variance of the endorsement scores on each of the three forms (reference is made to parenthesized values in the upper half of Table 5, which are in the form of the standard deviations) were compared through the use of a 't' test for variances computed from correlated scores (with 58 df.). The three comparisons (LOW-MEDIUM, MEDIUM- HIGH and LOW-HIGH) all yielded non-significant 't' ratios. 24 The results of these comparisons indicated that the forms did not significantly differ in variance. TABLE 5.--Means, standard deviations and significant differences of the three "parallel" forms Endorsement Component Forms Significance tests LOW MEDIUM HIGH 58.4 51.7 50.7 LOW-MEDIUM** (8.3)1 (8.2) (9.2) MEDIUM-HIGH LOW-HIGH** Emotionality Ratings Forms Significance tests LOW MEDIUM HIGH 48.8 57.7 74.7 LOW-MEDIUM** (10.2) (10.3) (10.6) MEDIUM-HIGH** LOW-HIGH** Parenthesized values are the standard deviations. **Significant at .01 level (Scheffe). The Endorsement by Emotionality Relationship Across Messages To ascertain the extent to which a general, across- message, relationship existed between an item's endorse- ment-and its emotionality of wording (regardless of the item's message), the following test.was performed. The mean endorsement and the mean emotionality ratings for each of the 60 items were computed. A correlation was 25 then computed between the item's mean endorsement and mean emotionality. The correlation obtained was r = -.21 (p<-10). The Shape of the Endorsement byiEmotionality Relationship (Question 2) Two questions concerning the shape of the endorse- ment by emotionality relationship can be asked: 1. To what extent is the shape of the endorsement by emotionality relationship consistent from message to message? If there is a predominant shape, what form does it take? 2. What shape best characterizes the relationship between endorsement by emotionality across messages? One method of attacking #1 above is to note the possible shapes (of the line connecting the three pairs of means in a triplet) that could occur and the number of occurrences.* If we assume that there is a main effect, i.e., the high emotional item is endorsed to a lesser degree than the low emotional item, then the maximum number of shapes is 3 (if no 2 points in a triplet are identical). *Since the mean emotionality ratings of the items in each triplet differed widely from triplet to triplet and the correlations between forms were not equivalent, it was not meaningful to characterize the shapes of endorsement by emotionality relationship using an AoV design. 26 We can assign either a l, 2 or 3 to each item in a triplet to indicate the position of that item's endorsement - emotionality mean relative to the endorsement - emotionality means of the other 2 items in the triplet. If we then ar- range these numbers by low, medium, and high emotionality, the shapes can be represented by the following sequences; (3,2,1), (3,1,2), (2,3,1). TABLE 6.--Representation of endorsement by emotionality relationship within triplets Emotionality Levels LOW MEDIUM HIGH # Triplet # 3 2 1 5 9,10,13,19,20 3 l 2 7 l,3,6,7,ll,12,l6 2 3 l 4 2,15,17,18 l Triplets 4, 5, 8, 14 were omitted from this Table since they did not demonstrate a main effect as defined in the text above. Table 6 does not expose any characteristic shape for the endorsement by emotionality relationship. The shape of the relationship appears to depend more upon the individual messages (and the peculiarities of the items with similar messages) than any general endorsement by emotionality effect. A method of characterizing the relationship between endorsement and emotionality across messages (Question 2 above) is to compare the r and r\(eta) calculated on the endorsement-emotionality means of the 60 items. The 27 obtained r = -.21 (p <:.10) and n = .54 (N.S. with 15 and 44 df.). A comparison of these two values yielded no sig- nificant difference. Thus, since no significant departure from linearity was found, the endorsement by emotionality relationship can most parsimoniously be expressed as linear (although this r only accounts for 4.82 per cent of the total variance). The Relationship betweeans' Response to Emotionality and TotaI Scale7SCOre (Question 3) To determine whether individual differences in susceptibility to the emotionality of item wording were related to the personality characteristic purportedly measured by the particular personality scale, the following test was performed: a correlation was computed for each subject as follows. (See Table B in Appendix D). This correlation was obtained by using a particular subject's endorsement of a particular item as one variable (X) and the total group's mean emotionality rating of that particular item as the second variable (Y). This procedure was fol- lowed for each subject across all of the 60 variables. A single correlation was then calculated between the above mentioned individual's correlation and the individual's total endorsement score on the scale (thus this was an across-subjects-correlation). The correlation coefficient arrived at through this procedure was .14 (N.S.). CHAPTER IV DISCUSSION Some Comments on Message Similarity Before discussing the findings of this study rela- tive to the three questions posed in the Introduction, there is one pervasive problem that the method employed in this study to explore the effects of emotionality of item wording led to. The problem was: "To what extent do the alternate items represent the same content as the original items?" Of the methods that were suggested for arriving at an index of similarity, none was felt to be apprOpriate. Therefore, although no statistical index of similarity was computed, every attempt was made by the investigator to equate for content. The correlations among the constructed items and the SFDS (see Table 2) lend some indirect support to the assump- tion of content similarity. The items are also presented in "content" triplets in Appendix E for inspection. In the present study the items were judged for simi- larity of message. In retrospect, however, it appeared to the E on inspection that several items (Triplets #1, ll, 28 29 14 and 20) within a triplet varied somewhat in message in- stead of remaining constant. The variation in message can be attributed to the difficulty of constructing items with a particular message while simultaneously manipulating the emotionality variable. In other words, the messages of certain items did not lend themselves to changes in emo- tionality without concomitant changes in item meaning. De- spite the c00peration of a number of colleagues in the wording of items and the help of the judges, this problem was not completely overcome. If the reader will go along with this admitted methodological weakness and assume that its effects were minimal, the three main findings may next be discussed. These three main findings answer the three questions raised in the Introduction: (1) degree of relationship between measured endorsement of item and emotionality ratings; (2) the mathematical form of this relationship and (3) relationship between emotionality of wording and score on the scale used in this study, the SFDS. The Relationship of Endorsement and Emotionality within Similar Messages (Question 1) Eight of the 20 triplets (#2, 4, 5, 8, 10, ll, 13 and 14) showed no significant differences in endorsement (within triplets differences). Some p953 Egg explanations for these 8 non-significant triplets are given below. In Triplet #2 the Low-Medium item comparison just fell short 30 of significance (p4<.05) with the difference being in the same direction as the significant differences. In Triplets #4, 5, and 11 the original SFDS item in that triplet was rated low in emotionality and, at the same time, these three messages were of such a nature as to elicit little or no endorsement in this sample (see Appendix E). There- fore, by making the wording on the new items more emotional, a "basement" effect on endorsement could have been en- countered. In Triplets #8 and 10 the range of the item's emotionality rating was small. In both triplets no signifi- cant difference in emotionality ratings was found between the items within the triplets. Therefore a significant difference between endorsement to the items in these trip- lets should not be expected. Triplet #13, like 4, 5 and 11, had a general low level of endorsement. While the items in Triplet #13 did not differ significantly in endorsement, the mean differ- ences were in the same direction as the significant differences that were found. In Triplet #14 both of the new items used the word “uncomfortable" as a substitute for the more emotional phrase "blood boils." Perhaps a substitution of the word "annoyed" or some similar term for "uncomfortable" would have been more in keeping with the meaning of the original SFDS item. Some such change might have resulted in a pat- tern of endorsements similar to those of the statistically significant triplets. 31 The majority of the triplets did, however, suggest the existence of a negative relation between an item's emotionality of wording and endorsement of the item. Thus, where items had similar messages (within triplets) an in- crease in emotionality of item wording, when endorsement was affected at all, was followed by a decrease in mean endorsement. If the inverse relationship between endorsement and emotionality is indicative of some stable socio-cultural response bias, then test administrators and personality theorists should exercise some caution in interpreting item endorsements. For example, items with low endorsement may not necessarily indicate that the p0pulation disagrees with the item's message, but may be related to the item's high level of emotionality. In opinion type research where ob- taining population base levels for certain items is the goal for the research, the introduction of emotionality as an extraneous variable may cause a distortion of these base rates or levels of responding. "Parallel“ Forms The analysis of the "parallel" forms (see Table 5) suggested that the degree of emotionality of wording could influence mean endorsement (to the scale) to a statistically significant degree. The direction of the relationship be- tween endorsement and emotionality was the same as that found in the within triplet analysis, i.e., inverse. 32. The effect of emotionality of wording should be kept in mind when com arin scores from " arallel" forms. 9 The Endorsement by Emotionality RelatiOnEhIp Across Message Although the 60 items spanned a large range of emo- tionality of wording, the across-message endorsement by emotionality relationship only accounted for approximately 5% of the total variance. The small size of the across- message correlation can be attributed to two factors (1) the endorsement to each message differed, even though the emotionality rating might have been the same and (2) the effects of emotionality Within a message were not Consistent. Both the above factors contributed to the with- in variance and consequently decreased the correlation. The Shape of the Endorsement by Emotionality Relationship (Question 2) The analysis within triplets did not expose any characteristic shape for the endorsement by emotionality relationship. The analysis across message indicated a linear trend (r = -.21, p