a. . - ...: . o O'.—¢...6.or ....4..... ........r... 0". .ol?\o .a .n,.. .o ...l...... . ...... .. . a. .'t~. I O .. .. . . .0 . o .. .. o. . . .. . . . .. 1. “an _ . . A . a. O p. . . c c o. 0 a . ......u...ao.- - ... ... .09 a.-io 1’. ouo,‘o.'.D¢QJHQ.O .5""-oo WTJB$F.P O . . . . o...........o.o’........o. ... . ’0 . .. .. .. .. .. ...........a......... . lili‘I _. . .u. . . o .. . . . . . . . I. . . 0.. .‘ _ .... . . _ .o .. .. .. .... ...r.. o. . .. “......n-Ihoo .....I . 1.0 ... . . . ~.u“.. . v o .. . . ... ... . .0. .7... o. a ..r ... _ . . .. .ao.............fnof-ofiodnwoh . ..t... ....oo...ov. .O.oao... v' . ..4'.o f. a co. .... . .. ... a _ ...... ....)v. o....... a..A .... ... ol.‘4r . . ... .. . .....A...... 0.... .A.....oa... .....o.. .4. ... . . . O ... ._ 4...... ... .. 1.....to... .. o .4 ... . . . ....... a........c. ....o... .......... . _ . A. . v..... .0... .fi..... .......o.....4..o.s.. .o ...... n . . . .. o . .vo . . 4 . . o v . . . . . .. . . .... . o a./.o0).. ......o........A.. ...: .... v.. .. . . . . . o . . - . _A.. .. . _ .o .. .. . .- o g . .. ... . . . . . _ . . .. .. _ . _ . . _. . . . .... ... v .a .. . ...... . o . .n .. n q. . 0.044 U. 131'. ‘ .. ... . . . . ... ... . .. .. .. _ . ... . . _. . .. .... ........,....... ..............p. . . . . . . . ... . . 4... . .... ...o).. A... ....c a¢0...o 1....1. n . .. . . . . . o _ . _ . . .... o. . ......no.v._ . .— .o a. .700. o.... .... o . . . u . . . . . _ . . g .. . .’. O ._ .a. . .. . o a: c O. a ...; .P- .¢.o.'....u.. . . O _ . . . . o . n o .. .n . ... .- .0 ,.o oo¢l ... ...’ . o n s . . . . .. . . . . . .. ... .. . -. .... . ..1... ...d..../o..o.... - n . a . . u . .. . m. .. . ... o .o c . . . . . . . . .. . . . . . .. 0.. A . ..... ...:o. .... . ..I....r.t..ocro 4... _. . .. . . .. . . . ... a .. . ... 4.. . . .V ......v.. IPOA.- o... .. .. . . _ . . .. .... . .. .. . ..on.. ..._. .. v. . . . . .. . . . . .. . . .... c/J ._ ....o .....l A .. v . c . . I . a . . p n c ._ o . ... u... «no. .DA..r'—o¢.‘ O... . . . a. . . . . 4 . .c u .o . . . . . _ .. . . ... . . . . . .. .... . ... r . ..l.'....l. or. . foul. a o T... Lt. " .. . .... . .. . . . .. .1 . p .. .1. .... ..avo..o.P.oc.o..oI.x . . . . . . ..o ... o . av ...-. .o . _ . . . . .. . .. . . .. . . ... ....I....o..n.. o..i. . .. . . . . . ... . .... . ...o. o. a. .... .. . . . A ... . .. ... a ....... 0.. . 4 . ... . 0.. .....c. . o o . .... oo - _ ...... a o. .. . . ...........O..’..pl..ov I 0.40.... . . . . . -. ... . . V._.. n . . . . . ._ . .. . . . . ....X.... ... ...... .. .... .. .....9......4..:»?.o.$..a9;..p.T 1c. . . . . . ... _ . . ... .. ... .. ., o.........,.;c.. ..r. -.. .. .. o o o . . . . . . . _ _ . .. ......o..,.. . . .. 0.14. .“...v.. ..40...m.rrno vl’... .033“ . .. . . _ .. . . . - . . . .. .4.. . 4 . v 0 ca. .... ... of..€..n.. . .l a~....n.....’.‘v4 . . n . . . . . A . . . .. - o . ... ... c 0. ...: o.ol.v 1.0 ....I.900- ..3‘...- 0t - o o . n U v _. n I . . o b . o . . c . . . _ . . o . . . . . . a . . . I u . I . .. _ O I o — . b, g c . .o 1.... .. .39 .o ... . 9. .. . — .ID4 . . . . . .. n . .. ....~ .0. . ¢ . .. 3 v4»: 3 .. . . .. .5 . .. . .. . ... .. ........lc.a... .. . ..flf . . .... . .A....o . .a.. . .. .. . .. . . ... O 0 . . I . n y u c . v . _. u. - i I _ _ . n— . . .. . O .. . . .. . o . . . . . o .- . o .. 0. c r . ..r .. v Ira. r” . 1.0.... .’fo.o. . ....ofo.c..r?.a ... . .o.. J.... .f.....l . . . v... t. . .o . . . _ . v . of. . a ..oicO ..cno? . . . . . o o . ... . . . . . . . . .. .o0..."..va)...uo.l...do..9.Jo. [1 ( . Mom‘wK 91.;m . .. . q . . .2 .. . .. . . . . . .. .... .. . ‘.../. .J.......!J«.V.J.. 3””. I . o .. I . o . . . . .. . . ... .oJof.4......oo/.o . .0 .. . . c . . . - ... . . . . _ A . .. . A .. . . . . . . 1.110.1P . .. . . . . . . .. A . . .... . . A . . . _ v”? .'.”V v ~nv—O «wan-y v-nyowntvvm', 091m _ . A ' 2 NM. WEN OF ’ . a. .. I . a . . . . . a . » p. ....Ivl..o.fro .. . u .. ..A..l ”u old..o".b .0 ... . . nu . _ .. . . . ..... h i . . . . no ..Ooa or. . . '1‘... u i . . . II . . . . .. ....... ...........,.»..6.r..:. u . . v i . . . .. J ...4...4 .oo-rooo.o..‘.ll.:.t . .. G . . ..p. .....J . ti o I .0. “fl-at... '... JNWP0WA¥ . . _ . . .I.A . .1..... a. . .u infirm... , -..- . . . o . p. .I .o ... .r.. 3..., o . A R . _ t. . ... ...... . ........’.-.- .......Il.o.qao. wlr.’o.!. . ...... T. D nu .. . _ . . . to . .n o . o..-.- o T ... .. . t . .... ..u no. vlda4.a . . . O ’- . _ T... U F. . 7 at no . u. E 3 _ W... i . _ . ...... . "...... 0 V. .9 . . .. .. x A . . .4 .a ‘A OJ . _ F G A t... . .. . . . gwwra... . I E R R . . . . .. . . . av. . v.4.fvt.r._ '..-P).n w . N U . . . of. . 1v“ . .- c v.0 o o ... "Jar. 0"], Inf... ... "...h'.’ _ . . .I.. . .... . .rIbA/l 3,9350. a .0 .. ”a. .. . I .. .. .. .. . .. ,. ... 4 . . .r!’0 __ L E III. E , .. .....c $.71... than... .IIWQHFFHQH. an“ .. A. D S I -. ... o . . a: .a a a ... u.... m ... fem” .Otd';\'.1. am. ..I'. .. . . .. . . . . ... ......r. ... ...........>J A. .... .. . . . . . . . . . . y . .f... #35:! .n 5 n c . o . o . . . u A o . o o. . . n . 0...... 0?..1 ' Jo. 0"..gu ".~! . . _. ... . . . .. . . ... . Y. . . ..v... {..JaaulrIIrJororv vs. .0. H . . K y o . I 0. . . I ' I .l'. o.’.';. '.. I . A’I . u a - t .. ., ...Iu_nuS____ v R _ n . a.’-. o ... .H‘. z. .. .w . R ..u .-... _ :1..- M I! . H . z . r .. . . o . I. n v O h. o .- -. _ R U TI TI 1 . . . .o . . . . ..mo . . 0 . .. . _ . ... ... Q R N . a. .. .. . . . _ . . q o. ‘ m N. D... F. 0 ...... _ .. . . ...... o . a . . -..]...r m. c . J . . _ o _ M...” _ ‘ x C O 7' u .. D . u. v . . a . . .. .r. . a m . . y .i o n a CO A. - ... I‘D O a . . . u. .. u . a . fi . .../I u. a u . , . .. m . . a . . . ~ . . . . . o u u . . . a . o O m o o u t Och Jr... “ . a . 0..-... ... .O.Ou.....) . . . . . ..,. ” . a. .1 gr I ..b o . 4 O . . . . ...: - . . .. - , . ~ . a. ’ . Ill "' .- .. . . ..I. M . I O. a \u .l .H a o w . v I. O .1 0,00 0. pt Q. o ' 3'. c . . do . .v . . a t v _ . . . . . . . H .- ... .... . .... .. ... ..v . . . . .n I.. . H. .5. 4.... ... ... ..... . ...-a.) in.” .. ... ......sd. .... o o .. _ I . o O L ..Pdo .. . o .. . . . .. ..u... . . ol . . . . . . . o . . ... .... u a... . . . .. .. 5.? o .I.._... ......... ......u. ax. . :0... 01. T. _ . A... o. ... v s... n .. . . . ..... .. . v.' i . .... .o . O _ I . ...v . .. a All. . déflo‘ ..J. o. a i an: . a ...a ..v. .... .. Id «42‘. Jouwwu oh . gar. K&¢~O§2f gird... ..ul . r F. v!p . . o o r . c . a 0.. o-.da..roo-.'—oho.ufin a.- ..Piu"... a. I‘ ‘ D ) u .0 . s u . u . . 4...‘ A ~x...v.~..oooho.o...bo¢,...... .+.o)ommbll;'cb.Oov-MO 04v o o . . . . o. U .9. to 5.. ....o.s.ch.u«o.-). . .f... n . c. . , . O I . .A . . .g . . .. .— .. .94 ..I: A)...J....O.l-n. Ono..‘.....1.o .0. u . n .l _ 0 . . o t . -..... ET... ...»....I. a. 29;. . ...-:3... ....v ._.. z... .-. .. no... ......_ .. . rt .o .001 l 22v fiat , K3" ,_ HDAG & sfiNS' BOUK BINDERY INC. | LlBRARY BINDERS ABSTRACT AN EMPIRICAL INVESTIGATION COMPARING THE EFFECTIVENESS OF FOUR SCORING STRATEGIES FOR THE KUDER OCCUPATIONAL INTEREST SURVEY FORM DD By Stephen Olejnik Objectives of the Inquiry Improving the discriminatory accuracy of interest surveys has been a major concern of measurement theorists for many years. As a result, several quantitative scoring procedures have been developed but empirical studies comparing techniques on a given instrument have been lacking. The purpose of the present study was to compare the effectiveness of several scoring strategies for the Kuder Occupational Interest Survey form DD. Specifically the techniques compared in- cluded: lambda coefficients, the procedure currently used; chi-square weights as developed by Porter (1965) and discriminant analysis using occupational scores generated by (a) lambda coefficients and (b) chi- square weights. Methods and/or Techniques In l958 Clemens had suggested that the relationship between an item and a criterion could be measured by a lambda coefficient which was defined as the ratio of the point biserial to the maximum Stephen Olejnik point biserial correlation. Several years later Kuder (1966) adopted the procedure of using the lambda coefficient as a measure of the re- lationship between an individual's responses to that of a specified criterion group on the Occupational Interest Survey form D. In com- puting the lambda ratio the selection or non-selection of a response pattern is considered the dichotomous variable while the continuous variable is the proportion of the criterion group selecting each of the possible response patterns. The individual is classified as be- longing to the criterion group in which he has the highest lambda coefficient. Although the author retained the original items, his revision of the scoring technique resulted in the instrument being renamed the Kuder Occupational Interest Survey form DD, which is currently in use. The second technique considered in the study was that sug- gested by Porter (l965) in which response weights are derived from the chi-square test statistic. Thus for each of the occupations considered, a fractional weight is calculated for each of the possible item re- sponses. The similarity of an individual's interest to that of a certain group is simply the sum of the chi-square weights for the lOO items. The occupation in which an individual's total score is highest is designated as the most compatible group. In a study comparing this procedure with Kuder's earlier scoring technique prior to its revision, Porter showed the technique to be superior. The third and fourth procedures considered, utilized the occupational scores generated by the lambda and chi-square procedures respectively and applied multiple discriminant analysis on each set Stephen Olejnik of data. Discriminant analysis has been shown (Rao, l948; Chappell, 1968) to be very successful in classifying both individuals and objects. The technique was therefore used here as an attempt to improve the accuracy of both the lambda and chi-square procedures. In order to compare the accuracy of the four scoring tech- niques described above, nine occupational groups were selected for study: pediatricians, veterinarians, physical therapists, x-ray technicians, optometrists, clinical psychologists, social workers, foresters and auto mechanics. The first five groups were designated as Set I and were considered as similar occupations, while the last five groups were considered as dissimilar and labeled Set II. One occupational group, optometrists, appeared in both sets of data thus making the two sets non-independent. In addition, each criterion group was randomly divided into two halves A and B; thus two independ- ent groups of data were available for each set. To obtain an estimate of the "true" effectiveness of each scoring procedure a double cross— validation technique as suggested by Moiser (l951) was followed. To analyze the results of this comparison, an analysis of variance procedure for mixed models was utilized. The fixed variables being sets (similar-dissimilar), measures (lambda--chi-square) and discriminant analysis (discriminant function--non-discriminant func- tion); all three of which were completely crossed. The random variable was occupations, having five levels and crossed with measures and dis- criminant analysis but nested within sets.. To solve the problem of non-independence only the cross-validation results of half A were used for optometrists in Set I and the cross-validation results of half B Stephen Olejnik were used with Set 11. Although occupations were not actually selected at random it was felt that using the Cornfield-Tukey bridge argument, the results of this study could be generalized to all similar occupa- tions. In addition, by assuming that occupations were randomly selected, the average percent correct identifications per group under each technique becomes the unit of analysis and the design is balanced. Data Sources The data used to develop and test the scoring keys in this study consisted of item responses made to the Kuder 015 by 3893 males from nine unequally sized occupational groups. These responses were originally collected by Kuder while developing the instrument and later obtained by Porter who identified the sets of similar and dis: similar occupations and randomly divided the groups into two subsets. Results and/or Conclusions The results of the study indicated that among similar occu- pations 56.37 percent of the individuals were correctly classified while among dissimilar occupations 68.41 percent correct classifica- tions were made. This difference was not statistically significant at a=.05. The test for a difference between the use of discriminant analysis procedures against the non-use of this technique indicated that while 60.62 percent correct classifications were made by the former and 66.66 percent correct classifications were made by the latter, the null hypothesis of no difference was not rejected at a=.05. A comparison of the measures, lambda vs. chi-square indicated Stephen Olejnik that an average of 61.29 percent correct classifications were made with the lambda technique while an average of 63.49 percent correct classifications were made with the chi-square procedure. This differ— ence between between measures was not found to be statistically sig- nificant at cx=.05. In addition 65.67 percent correct identifications were made with the discriminant analysis technique using the chi-square occupational scores; 55.56 percent of the individuals were correctly classified using the discriminant analysis technique with the lambda occupational scores; 61.32 percent correct classifications were made with the chi-square weights alone and 67.03 percent of the individuals were correctly classified using the lambda coefficients as the scor- ing technique. Although some differences seem to exist, the null hypothesis of no interaction between measures and discriminant analy- sis was not rejected at a=.05. In conclusion the results of this study showed that no one scoring procedure offers significantly greater accuracy than the other three procedures for classifying individuals into their appropriate occupational group. The study suggested other aspects of classifica- tion which should also be considered in choosing the "best" scoring strategy for the Kuder 015. One of these factors was the variability in the rate of correct classification across several occupations. Using Levene's test for equal variances it was pointed out that while no difference in variability of procedures was found with the homo- geneous occupations; statistically significant differences in pro- cedures were identified in the variability of correct classifications among heterogenous occupations. Post hoc tests indicated that of the Stephen Olejnik four scoring strategies studied, the least variable procedure was the use of lambda coefficients and the most variable was the chi-square technique. Based on the results of the study, it was concluded that of the four scoring strategies considered, the best technique for scoring and classifying individuals on the Kuder 015 was the use of the lambda coefficient. Although not having a statistically significant advan- tage in average accuracy over the other three strategies studied, the direction of difference in average accuracy favored the lambda tech- nique in both sets. Furthermore, the rate of correct classifications using the lambda technique was remarkably stable across several occu- pations, especially in the heterogeneous set. Finally further research suggestions were presented. AN EMPIRICAL INVESTIGATION COMPARING THE EFFECTIVENESS OF FOUR SCORING STRATEGIES FOR THE KUDER OCCUPATIONAL INTEREST SURVEY FORM DD By Stephen=Olejnik A THESIS Submitted to Michigan State University in partial fulfillment of the requirements for the degree of MASTER OF ARTS Department of Educational Psychology 1974 \Wahxawumw To Mom and Dad ii ACKNOWLEDGMENTS I am especially grateful to my committee chairman, Dr. Andrew Porter, for his guidance, time and encouragement through- out all phases of this research. I would also like to thank the members of my committee, Dr. William Mehrens and Susan Thrash for their assistance. Sincere appreciation is extended to Mr. George Heigho and Dr. William Loadman for their cooperation as well as to Albert Exner and Larry Isaacson for their assistance in the computer programming. Finally I wou1d like to thank the National Science Foundation whose support provided, in part, for the use of the computer facilities at Michigan State University. TABLE OF CONTENTS Page List of Tables ........................ vi List of Figures ....................... x CHAPTER I Introduction ........................ 1 Description of the Occupational Interest Survey ...... 3 CHAPTER II Development of Scoring Procedures ............. 7 Historical Review .................... 7 Kuder's Solution ..................... 9 Porter's Proposal .................... 15 Multiple Discriminant Analysis .............. 17 Pattern Analytic Approach ................ 19 Summary ......................... 21 CHAPTER III Method .......................... 23 Description of Data ................... 23 Cross-Validation Procedures ............... 25 Kuder's Scoring Keys ................... 25 Chi Square Scoring Keys ................. 26 Discriminant Analysis .................. 27 Statistical Analysis ................... 31 CHAPTER IV Results .......................... 34 CHAPTER V Discussion ......................... 57 Summary and Conclusions ................. 68 iv BIBLIOGRAPHY .......................... APPENDICES A B COMPUTER PROGRAM FOR CALCULATING LAMBDA COEFFICIENTS COMPUTER PROGRAM FOR CLASSIFYING INDIVIDUALS BASED ON THEIR LAMBDA COEFFICIENT TOTAL SCORES . . ...... COMPUTER PROGRAM FOR CALCULATING CHI-SQUARE WEIGHTS . . . . COMPUTER PROGRAM FOR CLASSIFYING INDIVIDUALS BASED ON THEIR CHI-SQUARE TOTAL SCORES ............ COMPUTER PROGRAM FOR COMPUTING THE SIMPLE d2 STATISTIC AND THE CLASSIFICATION OF INDIVIDUALS BASED ON DIS- CRIMINANT FUNCTION SCORES ................ DISCRIMINANT FUNCTION COEFFICIENTS BASED ON LAMBDA OCCUPATIONAL SCORES FOR SET I HALF A AND HALF B ..... DISCRIMINANT FUNCTION COEFFICIENTS BASED ON LAMBDA OCCUPATIONAL SCORES FOR SET II HALF A AND HALF B . . . . DISCRIMINANT FUNCTION COEFFICIENTS BASED ON CHI-SQUARE OCCUPATIONAL SCORES FOR SET I HALF A AND HALF B . . . . . . DISCRIMINANT FUNCTION COEFFICIENTS BASED ON THE CHI- SQUARE OCCUPATIONAL SCORES FOR SET II HALF A AND HALF B ......................... Page 75 79 81 83 84 85 88 89 9O 91 Table 1. LIST OF TABLES The percentage of males from each criterion group in Set I half A classified into each of the five occu- pational groups using the lambda scoring key derived on half B of Set I ................. . . The percentage of males from each criterion group in Set I half B classified into each of the five occu- pational groups using the lambda scoring keys derived on half A of Set I ................ . . . The percentage of males from each criterion group in Set I half A classified into each of the five occu- pational groups using the multiple discriminant analy- sisfprocedure based on lambda scores derived on half 8 0 Set I ....................... The percentage of males from each criterion group in Set I half 8 classified into each of the five occu- pational groups using the multiple discriminant analy- sis procedure based on lambda scores derived on half A of Set I ....................... The percentage of males from each criterion group in Set I half A classified into each of the five occu- pational groups using the chi-square scoring keys derived on half 8 of Set I ............... The percentage of males from each criterion group in Set I half 8 classified into each of the five occu- pational groups using the chi-square scoring keys derived on half A of Set I ............... The percentage of males from each criterion group in Set I half A classified into each of the five occu- pational groups using the multiple discriminant analy- sis procedure based on the chi-square scores derived on half B of Set I ................... vi Page 36 36 37 37 38 38 39 10. 11. 12. 13. 14. 15. 16. The percentage of males from each criterion group in Set I half 8 classified into each of the five occu- pational groups using the multiple discriminant analy- sis procedure based on the chi-square scores derived on half A of Set I ................... The percentage of males from each criterion group in Set II half A classified into each of the five occu- pational groups using the lambda scoring keys derived on half B of Set II .................. The percentage of males from each criterion group in Set II half B classified into each of the five occupational groups using the lambda scoring keys derived on half A of Set II .............. The percentage of males from each criterion group in Set 11 half A classified into each of the five occupational groups using the multiple discrimin- ant analysis procedure based on the lambda scores derived on half B of Set II .............. The percentage of males from each criterion group in Set II half 8 classified into each of the five occupational groups using the multiple discrimin- ant analysis procedure based on lambda scores derived on half A of Set II .............. The percentage of males from each criterion group in Set II half A classified into each of the five occupational groups using the chi-square scoring keys derived on half 8 of Set II ............ The percentage of males from each criterion group in Set II half 8 classified into each of the five occupational groups using the chi-square scoring keys derived on half A of Set II ............ The percentage of males from each criterion group in Set II half A classified into each of the five occupational groups using the multiple discimin- ant analysis procedure based on the chi-square scores derived on half B of Set II ........... The percentage of males from each criterion group in Set II half B classified into each of the five occupational groups using the multiple discrimin- ant analysis procedure based on the chi-square scores derived on half A of Set II ........... vii Page 39 4O 4O 41 41 42 42 43 43 17. 18. 19. 20. 21. 22. 23. 24. 25. The absolute value of differences between percent of individuals correctly identified as belonging to their actual occupational group in half A minus those correctly classified in half B for each of the four scoring strategies studied in sets I and II .................... Averages of percentages of males from each criterion group in Set I half A and Set I half 8 classified into each of the five occupational groups in Set I using the lambda weights .............. Averages of percentages of males from each criterion group in Set I half A and Set I half 8 classified into each of the five occupational groups in Set I using the multiple discriminant analysis procedure based on lambda occupational scores ......... Averages of percentages of males from each criterion group in Set I half A and Set I half 8 classified into each of the five occupational groups in Set I using the chi-square weights ............. Averages of percentages of males from each criterion group in Set I half A and Set I half B classified into each of the five occupational groups in Set I using the multiple discriminant analysis procedure based on chi-square occupational scores ....... Averages of percentages of males from each criterion group in Set II half A and Set II half 8 classified into each of the five occupational groups in Set II using the lambda scores ............... Averages of percentages of males from each criterion group in Set II half A and Set 11 half B classified into each of the five occupational groups in Set II using the multiple discriminant analysis procedure based on the lambda occupational scores ....... Averages of percentages of males from each criterion group in Set II half A and Set II half 8 classified into each of the five occupational groups in Set II using the chi-square weights ............. Averages of percentages of males from each criterion group in Set 11 half A and Set II half B classified into each of the five occupational groups in Set 11 using the multiple discriminant analysis procedure based on the chi-square occupational scores ..... viii Page 44 47 47 48 48 49 49 5O 5O 26. 27. 28. 29. 30. 31. 32. Page The average percent of individuals correctly classi- fied into their actual occupational group from half A and half 8 for Set I and Set II ......... 52 ANOVA table for mixed modes, analyzing the data from table 26 ........................ 53 The mean, variance, standard deviation and absolute error difference for each scoring strategy used in Levene's test for equality of variance across the four measures ..................... 56 The average rate of correct identifications for each occupational group in Set I using the cross— validated and quasi-cross- -va1idated ' . . The average rate of correct identifications for each occupational group in Set II using the cross- -validated and quasi- cross- -validated ............... 61 ANOVA tables for cross validated and quasi-cross- validated data ..................... 63 Averages of percent of individuals correctly identi- fied in the two sets under each of the four scoring strategies and considered ........... 66 ix LIST OF FIGURES Figure Page 1. Item format for the Kuder Occupational Interest Survey form DD ...................... 12 2. Response patterns utilized to represent selected item responses .................... . . 12 3. Example of a contingency table used in computing chi-square item response weights ............. 15 4. Data matrix for a mixed model analysis of variance ......................... 32 5. Data matrix for the test of equality of scoring consistency ....................... 45 CHAPTER I INTRODUCTION Individual interests have long been considered by psycholo- gists and educators as one of the prime factors in determining occu- pational success and satisfaction. This theory has been supported by several research findings which have indicated that the degree and direction of one's life accomplishments can be directly related to the individual's interests. Clark (1961) for example, has argued that occupational effectiveness is increased at least at the professional level when the person enters that field for which he is best fitted both intellectually and temperamentally. Moreover, extensive follow- up studies conducted by Lipsett and Wilson (1954) and McRae (1959) have indicated greater job dissatisfaction among individuals possess- ing the necessary mental ability but lacking in interest, than among individuals having the reverse characteristics. Thus interests, as well as ability seem to play a significant part in defining the level of job success and satisfaction. The need for an accurate assessment of one's interests is, therefore, obvious. Such a technique could be a valuable tool in identifying occupations suitable in terms of satisfaction for the individual and efficiency for the employer. As a result, the development of instruments meas- uring interests has become an area of major significance within the field of psychological testing. A major problem in the development of accurate interest inventories has been the evaluation of individual responses. These instruments consist primarily of questions related to personal feel- ings and attitudes which cannot be scored as either right or wrong. The effectiveness of such inventories in identifying suitable occupa- tions is, therefore, directly dependent upon the scoring procedure which is used. Several techniques have been suggested, each promising to improve the accuracy of interest surveys. The question as to which procedure is the "best" is a difficult one that has merited considerv able investigation. Kuder (1957) concluded that a number of factors were important for suggesting which approach would provide the great- est discriminatory power. He went on to suggest, "We need to build up an extensive background of experience and theory before it will be possible to make a good guess as to which technique will produce the best results in a specific situation." (p. 114) Recently Loadman (1971) compared several methods for scoring the Kuder Occupational Interest Survey form DD (this instrument will for the remainder of this paper be referred to as the Kuder 013) in- cluding the procedure currently used by the publisher referred to as lambda coefficients; chi-square weights based on the chi-square test statistic as suggested by Porter (1965); multiple discriminant analysis based on lambda occupational scores and a pattern analysis procedure based on a computer program developed by Clark (1969). The results of his study indicated that the procedure presently used to score the Kuder 018 (the lambda coefficient) offered the greatest differentia- tion among the occupations tested. In this study, however, two of the techniques, lambda and discriminant analysis, had non-cross-validated scoring keys, while the scoring keys developed using the chi-square and pattern analysis procedures were cross-validated. By comparing the effectiveness of the quasi-cross-validated techniques with the true cross-validated procedures, the author possibly gave an unfair advantage to the former, which had the opportunity to capitalize on chance factors. The results of his study are therefore questionable and a re-analysis of the scoring techniques was warranted. Further- more, Loadman did not consider the use of multiple discriminant analy- sis based on occupational scores obtained using the chi-square weights. It was the purpose of this investigation to compare the effectiveness of four scoring techniques including: lambda coeffi- cients, chi-square weights and multiple discriminant analysis using occupational scores based on (a) the lambda coefficients and (b) chi- square weights; when used with the Kuder Occupational Interest Survey form DD. Scoring weights for each procedure were developed and cross- validated, thus an unbiased test comparing scoring strategies was conducted. As a measure of effectiveness, the percent of individuals in the cross-validated group correctly identified as belonging to his actual occupation was used. The conclusions from this study should either provide support for establishing a new scoring procedure for the Kuder OIS or a new rationale for using the present technique. Description of the Occupational Interest Survey The Kuder Occupational Interest Survey, which was used in this study, was developed by Fredric Kuder over a period of years. During the early years of development, frequent changes in items and scoring techniques were made in attempts to improve the accuracy of the survey. The latest revision was made in 1966 when Kuder introduced a new scoring procedure and changed the name to the Kuder Occupational Interest Survey form DD. Except for scoring, the new survey is identi- cal to the previous instrument, i.e., the same format is used for presenting the same one hundred triadic items describing some common activities. Kuder designed this test to be used with junior and senior high school students along with college freshmen and adults seeking employment counseling. The author warns that the use of this instru- ment with a younger group may provide erroneous information. The reading vocabulary for the survey's directions and items is fixed at the sixth grade level, thus making it easily understood. From each group of three activities, representing an item, the subject is in- structed to select the activity he most prefers and that which he likes least. Testing time requires only about thirty minutes. The responses are scored using a complex procedure developed by the author for this instrument and will be described later. A person who has taken the Kuder OIS ultimately receives a report indicating those occupations and college majors that the in— dividual seems to be best suited for in terms of interests, but not necessarily ability. In total, there are 171 different scales avail- able, all of which would not be reported to any one particular in— dividual. Males do receive, however, scores on 77 occupations and 29 college majors while females are sent scores on 57 occupations and 27 college majors. A verification scale is also provided for each individual as a check on the confidence that can be placed in the subject's answers. Scoring is problematic since every response is a "correct" answer if the individual answered the item sincerely. One of the possible solutions which has been proposed deals with the assigning of weights to individual item responses for each occupation. The occupation on which a subject scores relatively high is indicated as a suitable occupation for the subject in so far as interests are con- cerned. This is the approach followed by both Kuder and Strong in the scoring of their respective interest inventories. This solution, however, provides another problem, that of determining how many points should be given for a particular item response for a particular occu- pational group. A number of solutions to this issue have been proposed over the years, but as Berdie and Campbell (Whitla 1967) have indicated the problem is still unresolved. Recent contributions by Kuder (1966) and Porter (1965) have stimulated new interest in this area. Still other proposals for the solution of the problem of interpreting responses are the use of pattern analysis and multiple discriminant analysis procedures. That is, for the former, investi- gators have looked at group responses to determine whether occupational groups display distinctive patterns in their answers. Patterns are then treated as items and are assigned weights. Multiple discriminant analysis on the other hand, is a technique used in studying the rela- tionship or classification of individuals among several groups. The procedure results in the reduction of multiple measurements to one or more weighted combinations having a maximum potential for discrimin- ating among members of the different groups. Both of these proposals have induced controversy and stimulated considerable research in their development. Returning to the problem at hand, however, the question still remains; which of the several possible scoring strategies is the most effective in discriminating individuals from several criterion groups? Following a review of the history and description of the development of several proposals for scoring, an attempt will be made to answer the question for the four earlier proposed strategies in relation to the Kuder OIS. CHAPTER II DEVELOPMENT OF SCORING PROCEDURES Historical Review According to Fryer (1931) the earliest investigators of interest theory often relied upon responses made to direct questions concerning selected jobs as the basis for predicting occupational in- terests. It was quickly discovered, however, that such replies were usually unrealistic, superficial and unreliable. Anastasi (1968) has offered insufficient information on the part of the subject and pre- vailing stereotypes attached to certain occupations as the reasons for these results. Fryer further suggested that family and peer pressure may also influence individual responses, especially among young subjects. As a consequence, researchers turned to more subtle methods for assessing interests. Inquiries were made into attitudes toward a variety of activities which were scored and used as a basis for determining occupational interests. Scoring of these early in- ventories was very subjective, assigning item weights solely on an estimate of an item's significance by a group of "experts". A review of the literature shows, however, that the early researchers of in- terest inventories were concerned with the development of more objec- tive scoring techniques. The publication of Yule's Introduction to the Theory of Statistics, in 1919 provided a major breakthrough in the development of objective scoring procedures for interest inventories, by supplying a framework for dealing with percentage differences. Based upon this work, M. J. Ream (1924) introduced the first objective weighting technique for interest surveys in a study of the interests of success— ful and unsuccessful salesmen, using the Carnegie Interest Inventory (the first standardized interest inventory). Ream's procedure was rather simple. After administering the inventory to two homogeneous groups, the researcher assigned a weight of i 1.0 when the difference between the groups was larger than one standard error; differences of lesser degree were not considered. This method was followed by sub- sequent investigators who modified and further developed the procedure (Freyd 1924, Kornhauser 1927). A second major development in the construction of objective scoring techniques came in 1929 when Cowdery suggested a method for weighting all items, adjusting for size of group differences by giving larger weights to bigger differences. Although considerably more complex than Ream's method, it took into consideration all responses made by a subject and was therefore logically argued to be a more discriminating procedure. Cowdery took each item on the Interest Report Blank (1921 edition) and divided the responses into a 2 x 2 matrix: like dislike occupation in question a b men in general c d where the letters (a, b, c, d) indicate the number of responses per cell for a particular item. The weight for the item was then calcu- lated using a formula devised by T. L. Kelley in 1923: b: ...—2— (1-¢z)o where b is the assigned weight, 4 is the coefficient of correlation ad-bc derived from: o = /, and o is the standard de- (a+C)(b+d)(a+b)(d+C) viation of the frequencies (a+b) and (b+d). The weight was given a positive or negative sign depending upon the direction of the percent- age difference of the responses between the two groups. Following a modified version of this scoring technique in which the method for calculating o and 0 were changed, Strong elimin- ated all decimals in the weights by rounding the scores to the nearest whole number and multiplying by a factor of 10. The result of such a procedure was to generate scoring weights ranging as high as i 30 which were undoubtedly difficult with which to work. Kuder's Solution As attempts to improve the accuracy of interest inventories continued, several researchers suggested and developed new procedures for the assigning of item weights. Among the leaders of this group of investigators was Fredric Kuder, who throughout the 50's and 60's published several papers on the weighting problem (1957, 1961, 1963, 1966). His interest in the scoring problem was necessitated by the development of the Preference Record-Occupational form 0 in 1956, when he was faced with the task of interpreting responses. Kuder realized 10 that the theoretically best method of evaluating all possible combina- tions of responses in all items was impractical. Alternatively, he sought a procedure which was relatively simple and straightforward yet accurately discriminated between occupational groups. Kuder's research to find such a technique, through a comparison of several scoring pro- cedures, resulted in the investigator concluding that "the effective- ness of any scoring key was contingent upon a number of variables including: the number of cases, the composition of the inventory, the content and type of items, the range of item validities, the homoge- neity of the groups and the extent to which the items can be considered to be uniformly distributed in the domain represented." (1957, p. 114) Kuder first used a technique which compared for each item the percentage of individuals from an occupation in question selecting a response to the analogus percentage of either a reference group of men in general or a contrasting occupational group. Utilizing Zubin's inverse arc sine transformation and a pre-established table, Kuder was able to assign item response weights ranging from i 10 to -10 for each occupation. The first solution to the scoring problem was not completely satisfactory,.and thus Kuder continued his investigation into alterna- tive scoring procedures. In 1963, he published a paper discussing the advantages and disadvantages of using Findley's (1956) formula for assigning occupational scores: 0 = EPA -ZPB, where D is the occupa- tional score, EPA is the sum of the proportion of subjects in group A who selected each of the preferences marked by the subject, and ZPB is the sum of the proportions of group B who marked each of the 11 preferences marked by the subject. Kuder applied Findley's technique to the data he had collected from the Preference Record, letting group B be men in general and group A be a particular occupation of interest. He found that such a procedure reduced the amount of overlapping be- tween groups substantially and, therefore, improved the discriminatory accuracy of the test. Kuder did not adopt Findley's procedure for scoring form D, but rather, he questioned the need for the reference group. He argued that instead of computing the difference in proportions between two groups, why not just consider the proportions of subjects which selec- ted the same preferences as the subject. The major problem with such a procedure was that the more homogeneous groups would tend to reflect higher scores and, therefore, a comparison between occupations would not be possible. Kuder's solution to the differential problem associ- ated with the homogeneity of groups was based on the work (Hi Clemens (1958) who had suggested that the relationship between an item and a criterion could be measured by a lambda coefficient which was defined as the ratio of the point biserial to the maximum point biserial cor- relations. The use of the lambda coefficient, as a measure of the relationship between an individual's responses and those of a particu- lar criterion group, represents the current procedure for scoring the Kuder 015. The development of the lambda coefficient can be described by considering the nature of a Kuder 015 test item. As noted earlier the instrument consists of 100 items having the format shown in Figure 1: 12 most least Activity 1 o (l) 0 (4) Activity 2 o (2) o (5) Activity 3 o (3) o (6) Figure 1. Item format for the Kuder Occupational Interest Survey form 00. A subject is instructed to indicate the activity which he likes most and the activity which he likes least. Thus for the one hundred items, each subject selects a subset of 200 responses from the 600 possible. Furthermore an individual's two responses for any one item may be represented by one of the response patterns shown in Figure 2. Response Pattern Responses most likes least likes 1 l and 5 2 1 and 6 3 2 and 4 4 2 and 6 5 3 and 4 6 3 and 5 Figure 2. Response patterns utilized to represent selected item responses. The subject's responses to the instrument may be represented by a total of 100 response patterns. To obtain a measure of the relation- ship between an individual's responses and those of a particular cri- terion group the lambda coefficient is computed. In calculating the point biserial correlation the dichotomous variable is the selection or non-selection of the 600 possible response patterns, while the continuous variable is the proportion of the criterion group selecting 13 each of the 600 response patterns. It might be noted that in calculat- ing the lambda coefficient, the responses made by one criterion group at a time are utilized. The point biserial formula used in computing M -M the lambda coefficient may be written as: r b' = —E——£- —E- where p. 15 0t N Mp is the average value of the continuous variable associated with the 100 P.. 100 response patterns selected by the individual. That is Mp= Z -—Ll i=1 100 where pij (the continuous variable) is the proportion of the criterion group selecting response pattern j for item i. Mt is the average value of the continuous variable across all 600 response patterns: 100 6 Z Z 'pij M = i=1 j=l = .667. o is the standard deviation of the con- t 600 ‘ t tinuous variable. Np is the number of response patterns selected by the individual; Np ='HMJ,while Nq is the number of response patterns not selected; Nq = 500. To compute the maximum point biserial cor- relation M is the average of the highest response pattern proportions 9 100 for each item across all 100 items; thus M9 = Z maxjpij where i=1 100 maxj pij is the highest response pattern proportion (j) for item i. An individual's lambda coefficient for a particular occu- pational group is the ratio of the point biserial and the maximum point biserial correlation: M -M 100 i—tYNP‘ g x = at L = i=1 5% -.667 Mg - Mt ch O 1' . .. ot Nq g0 maxJ p13 -.667 i=1 1OO Furthermore the lambda coefficient can be reduced to a subset of lambda weights: A. O 1.1 where Aij is the lambda weight associated with response pattern j for item i, such that the sum of 100 Aij gives the lambda coefficient. It might be noted that in calculating the lambda weights, the maximum point biserial correlation remains the same for all 600 weights for a particular criterion group but can vary across criterion groups. Two advantages are associated with the use of the lambda coefficient as a measure of the similarity of interests between an individual and a particular occupational group. First the lambda co- efficient is unaffected by the homogeneity of the criterion group and secondly it has an upper limit of 1.00. Kuder calculated the lambda coefficients for individuals from several occupational groups and compared the resultant discriminating accuracy against his then cur- rent method. The results indicated a sharp reduction in the degree of overlapping between occupations and therefore increased precision in occupational discrimination (Kuder, 1970). Kuder then revised all of his scoring keys using the technique for calculating the lambda coefficient and renamed his test the Occupational Interest Survey form 00, publishing it in 1966. 15 Porter's Proposal At the same time that Kuder was developing his lambda weights, Porter (1965) was taking a different approach to the develop- ment of a scoring key for the same test. His proposal was to use the chi-square test statistic for assigning weights to individual item response patterns. Defining response patterns identical to those shown in Figure 2 of the previous section, Porter constructed for each of the one hundred items on the test, a contingency table consisting of a simultaneous breakdown of subjects by occupations and by response patterns. An example is shown in Figure 3 below: Item 1 Occupational Pattern for Item Group 1 2 3 4 5 6 n1 Zij X1 n2 2 "3 Yx3, . 1 N1 XI N Y1 Y2 Y4 Y4 Y5 Y6 ZXi Figure 3. Example of a contingency table used in computing chi-square item response weights. For such a table the weights assigned per response pattern per occu- pation were calculated using the following formula: X. Y. 2 Ni. = (Z1. - -%§—l) x (sign of the unsquared J r J i numerator) XiY‘ 2X. 1 16 where i = l . . . . I, and j = 1 . . . . J. I denotes the number of groups, and J denotes the number of response patterns. Zij is the number of observed responses made by i-th group to the j-th response pattern. Xi is the total number of subjects in the i-th group, Yj is the total number of individuals selecting the j-th response pattern, and Ex, is the total number of subjects in the sample. Thus for each of the occupations considered, a fractional weight was calculated for each of the possible response patterns. An individual's score for an occupation was simply the sum of the chi— square weights associated with the individual's responses to the 100 items. The occupation in which an individual's total score was high- est was designated as the most compatible group. In testing this technique against the one used by Kuder in the Preference Record form 0 (note this was Kuder's scoring procedure prior to the development of lambda coefficient scoring technique), the results indicated a significant improvement in discriminating among similar occupations, but inconclusive results were obtained con- cerning heterogeneous occupations. It might be pointed out here, that in computing chi square weights for a given occupation, a set of other occupational groups are needed in order to compute the chi-square statistic. Furthermore, the numerical value assigned to the weights could vary with the number and type of occupations used in the computations. These characteristics are not present with the development of item weights using the lambda procedure. Thus there is a slight advantage to using the lambda 17 technique in terms of convenience, since item weights are computed with one occupational group at a time and numerical values of the weights are not dependent upon other occupational groups. Loadman (1971) compared Porter's procedure, to Kuder's new lambda weights and found the latter to be more discriminating. His lambda weights however were not cross-validated and the chi-square weights were. Loadman's results may therefore have been due to an unfair comparison of the two promising techniques. A re-analysis of the data was necessary in order to determine which of the two pro- cedures is the most discriminating for the Kuder Occupational Interest Survey form 00. Multiple Discriminant Analysis Still another approach to the scoring problem, and one that has been used with considerable success in other scoring problems, is the application of the linear discriminant function. The earliest work utilizing this procedure was conducted in the area of biometry in studies concerned with such issues as the classification of hair color. Psychologists quickly adopted the technique, however, for problems dealing with classification or selection of individuals based on a number of measurements. One of the first applications of multiple discriminant analysis was made by Barnard in 1935 when based on a suggestion by R. A. Fisher; the researcher used the procedure to classify a series of Egyptian skulls. Fisher himself later elaborated on the method (1936, 1938) proposing that "when two or more populations have been 18 measured in several characteristics n1, n2 . . . nx, special interest attaches to certain linear functions of that measurement by which pop— ulations are best discriminated." (Fisher, 1936 p. 179) Rao calcu- lated two discriminant functions and was able to achieve maximum classification of individuals into one of three Indian castes chosen for investigation. (Rao, 1948) Among the first psychological appli- cations of the procedure was one made by Rao and Slater in 1949 in a study attempting to discriminate five groups of neurotics from one group of normals by using thirteen personality variables. (Rao and Slater, 1949) Their results indicated that only three discriminant functions were necessary to account for the significant variation in the means of the six groups. In an unpublished doctoral dissertation, Bryan (1950) demon— strated a procedure which could identify all discriminant functions in a classification problem directly from the two matricies obtained from the original scores: the between groups deviation matrix and the within group deviation matrix. "The technique provides an exact determination of the characteristic equation and provides the latent vectors of matricies of a class to which those of discriminant analysis belong. Prior to this time lengthy iterative procedures have been required for these determinants." (Tiedeman 1951) As an exercise illustrating the above procedure, Tiedeman and Bryan (1954) analyzed the responses made to the Kuder Preference Record by a group of Harvard students from five different areas of study. The responses were scored and each subject received nine scores to correspond to the categories of interest identified by the 19 instrument. Hith these scores plus the information of the students' college major. the investigators computed the matrix of between groups sum of squares and cross products and the matrix of within groups sum of squares and cross products. Four discriminant functions were then identified but analysis showed that only two functions associated with the first and second latent roots were necessary to account for 91% of the total variance. The authors, however, did not cross-validate the computed discriminant functions and thus no measure of accuracy for predicting an individual's major was provided. Finally, a recent study conducted by Chappell (1967) at the University of Purdue utilizing the discriminant function showed the usefulness and power of the procedure. The investigator was interested in studying personality and interest differences between veterinarians, electrical engineers and guidance counselors all of which were graduate students. The results indicated that the suspected differences did exist, and that the discriminant function based on the scores from the Guilford-Zimmerman Temperment Survey and the Strong Vocational Interest Blank was capable of identifying these differences. The author con- cluded by urging a wider application of the procedure in educational and psychological research. Pattern Analytic Approach While some researchers continued the work begun by Cowdery and Strong by investigating and developing new procedures for assign- ing individual response weights, other investigators began looking at different approaches to the scoring problem. Some members of this 20 latter group suggested examining patterns in the responses to discrim~ inate between groups. Evidence to support such a procedure was pro- vided, at least at the theoretical level by Meehl (1950, 1954), who effectively argued that while two items taken separately may have predictive validity of zero, taken together the items could be perfect in prediction. A problem with the technique, however, has been in identify- ing procedures which could isolate the particular response patterns unique to each group. Throughout the 50's and 60's McQuitty proposed several methods in which this problem could be resolved. (1957a, 1957b, 1961a, 1961b, 1963, 1966) The procedures which he suggested were based on isolating types or categories of individuals through a technique called elementary linkage analysis or some modification of it. In a study assessing levels of mechanical experience McQuitty (1958) showed the technique to be superior to some item scoring pro- cedures when many subjects were used for each criterion group and when a number of criterion groups were to be discriminated. Clark (1968) developed a computer program, based on McQuitty's work and compared the discriminating power of the identified pattern responses to multiple regression analysis in predicting field dependency and U.N. row call voting behavior. The results indicated that Clark's pro- cedure was superior in cross-validated data. Based on the work of McQuitty and using the computer program developed by Clark, Loadman (1971) attempted to identify response patterns of several occupational groups to the Kuder 015. The results of this study, however, indicated that the identified patterns did no 21 better than chance in correctly discriminating individuals from the occupations studied. Thus while pattern analysis may provide a power- ful method for discrimination among groups, researchers have been unsuccessful in defining workable procedures for identifying response patterns for use with the Kuder 015. Summary The degree of job success and satisfaction has been shown to be highly related to an individual's interests. Since vocational counselors are interested in predicting job success and satisfaction, measurement practitioners have been concerned with the development of accurate instruments in this area. Early attempts, however, were often inaccurate and very subjective. Although vast improvements have been made, research continues in new attempts to achieve greater pre- cision in the measurement of interests. As Nunnally (1970) has cau- tioned, "Even though interests are very important to consider in choosing an occupation, it does not necessarily follow that the avail— able instruments are maximally effective measures of interests. As is true in most areas of testing, a great deal more research with interest inventories is needed." (p.48) One of the more fruitful areas of research in which advance- ments have been made, has been the development of scoring techniques for interest surveys. A review of the literature shows that at least three major techniques have been considered as possible procedures to be used in interpreting individual responses: (1) assigning item weights, (2) application of multiple discriminant analysis and (3) the 22 use of pattern analysis. While research findings with the first two techniques have indicated considerable success in classification prob— 1ems, the third technique, pattern analysis, has appeared considerably less promising, at least as presently operationalized. In particular Loadman's (1971) study suggested that one pattern approach was not useful for the Kuder 015. Thus further development in this latter procedure is needed before it can be considered as a possible procedure to follow. The solution to the scoring problem of the Kuder 015, at least for the present time, appears to be with item weighting tech— niques. It was the purpose of this investigation to determine which procedure offers the greatest accuracy in discriminating individuals among both similar and dissimilar occupations while using the Kuder 015. More specifically, two item weighting procedures and two appli- cations of multiple discriminant analysis were compared in an attempt to identify empirically which technique was best. The four scoring strategies which were considered include: (1) Kuder's adaption of the lambda coefficient--the procedure which is presently used, (2) chi- square weights as developed by Porter (1965) and discriminant analysis using occupational scores generated by (3) the lambda occupational scores and (4) the chi-square occupational scores. The most effective procedure was identified as that technique which correctly identified individuals to their corresponding occupation the greatest percentage of the time. CHAPTER III Method The empirical investigation of the accuracy of the four scoring procedures for the Kuder 015 required the use of several tech— niques. It was necessary to first develop scoring keys using a sample of individuals from several occupational backgrounds by each of the methods considered (lambda coefficients, chi-square weights, and multiple discriminant analysis based on (a) lambda weights and (b) chi-square weights). Then to insure that the keys were not based on chance responses or some idiosyncrasies of the particular sample group, the scoring keys were applied to a new independent sample of individ— uals. The efficiency of each of the procedures was estimated by the percentage of correctly identified subjects into their respective occupations on the cross validation sample. To determine whether a statistically significant difference actually existed among the pro- cedures considered, an analysis of variance for mixed models was computed. Description of Data The data used to develop and test the scoring keys consisted of responses made to the Kuder 015 by 3893 males from nine occupational groups. These responses were obtained from Porter, who had previously 23 24 acquired the data from Kuder, the originator of the 015. Since this comparative study required a large volume of data, without their assistance the investigation could not have been conducted. The nine criterion groups were divided into two classifica- tions: similar and dissimilar occupations, as suggested by Porter (1965) and were labeled Set I and Set II respectively. Each set then consisted of five occupational groups with one criterion group appear- ing in both sets. Set I consisted of 406 optometrists, 274 x-ray technicians, 455 pediatricians, 385 physical therapists and 396 veter- inarians. Set II on the other hand was composed of 500 clinical psy- chologists, 300 auto mechanics, 346 foresters, 400 optometrists and 451 social case workers. Although optometrists in Sets I and II were the same individuals, six subjects were randomly deleted from set II as a partial effort to reduce the total number of subjects in set II to meet the restrictions imposed by the computer facilities. In addition, it should be noted that since optometrists appeared among both similar and dissimilar occupational groups, the two sets were non-independent of each other. Within each set, the individual occu- pations were randomly divided into two parts: subset A and subset B. Thus each set consisted of ten independent groups ranging in size from a low of 136 individuals to a high of 250 individuals. The purpose of having two subsets was to provide one sample from each occupation to derive a scoring key for all four techniques considered, and at the same time have a second independent sample of each occu- pation to test the efficiency of the derived keys. 25 Since the data must be considered as old, being collected originally in the mid 1950's, the responses made by individuals in the criterion groups may not accurately reflect the attitudes and interests of individuals involved in these occupations today. Nevertheless these responses did provide valuable data for comparing the effectiveness of each scoring strategy which was the purpose of this study. Cross-Validation Procedures In order to estimate the "true" effectiveness of the scoring strategies being considered, a double cross-validation technique as suggested by Moiser (1951) was followed. Separate scoring keys were developed for subset A and B for each occupational group for both sets. The keys from one half of an occupational set then applied to the pthg[_half of the occupational set. For example, a scoring key derived half A for clinical psychologists was applied to half B, and the derived key from half B was applied to the data in half A. The estimate of the effectiveness of the scoring technique for an occupa- tion was computed by taking the average number of correct identifica- tions made by the two derived keys. Thus each of the four scoring techniques was applied to both halves of each of the five occupational groups in both sets. Kuder's Scoring Keys In developing the lambda weights as the fractional components of the lambda coefficient, associated with the individual response patterns, Kuder utilized the knowledge he had gained from his research 26 with scoring procedures. Although considerably more complex than the previous procedure used by Kuder, the increased discriminating accuracy of the method outweighed the inconvenience of its calculation. In addition the introduction of high speed computers made the task far less cumbersome. Thus in this study lambda weights were computed in the manner suggested by Kuder, on a sample of individuals from each of the occupational groups. Each individual from subset A received five occupational scores based on the lambda weights calculated for each of the criterion groups in subset B. By summing for each criterion group the lambda value associated with the selected response patterns across all 100 items, the lambda coefficient was obtained for each of the occupations in subset B. Similarly, each individual in subset B received five occupational scores based on the lambda weights computed on the cri- terion groups in subset A. Individuals were then identified as belong- ing to the occupational group for which he had the highest lambda coefficient. The efficiency of this technique was estimated by the average percent of correct identifications of individuals to their corresponding occupational group across the two subsets. Chi-Square Scoring Keys As was mentioned earlier, Porter (1965) had suggested that item weights be computed using the chi-square test statistic. He further demonstrated that this procedure was superior, at least in discriminating among similar occupations, to Kuder's scoring technique used prior to the development of the lambda weights. Thus to compare 27 the chi-square weights to Kuder's new scoring procedure, the former were calculated following the technique proposed by Porter and de- scribed earlier. The chi-square weights were assigned to each of the six response patterns per item for each occupational group. For each occupation in each subset, an individual's selected response patterns were matched with the chi-square weights calculated on the opposite subset and summed over the 100 items. The individual was then identi- fied as belonging to the occupational group which produced the highest sum of weights. As with the lambda technique, the efficiency of this procedure was estimated by the average percent of individuals correctly identified as belonging to his actual occupational group across both subsets. Discriminant Analysis Fisher (1936) had suggested using discriminant analysis procedures in classification problems when measures on two or more predictor variables were available. This technique attempts to identify one or more sets of coefficients (vector weights) for linear combinations of the variables which will maximize the variance between occupations relative to the variance within occupations. Thus several measures on an individual or object are combined to produce one com— posite score. Since discriminant analysis has been shown to be suc- cessful in previous studies (Rao, 1948; Tiedeman and Bryan, 1954; Chappell, 1967) for classifying individuals, it was considered in this study as a possible solution to the scoring problem for the Kuder OIS. 28 However, in order to use the multiple discriminant analysis procedure, it was necessary that the predictor data be at least at the ordinal level. Since responses made to the Kuder were on the nominal level it was necessary to transform the data to meet the above restriction. This was done in two ways: (1) lambda coefficients (2) chi-square occupational score, both of which were described earlier. Although either procedure could have been used, it was decided that for this study both weighting methods would be tried. Thus two mul- tiple discriminant analysis procedures were computed, one using the lambda occupation scores, and the other using the chi-square occupa- tion scores as the.predictor variables. Utilizing the procedure described by Overall and Klett (1972), the discriminant functions were identified. Each half of each set was used separately to derive the discriminant functions. These vector weights were then applied in a double cross—validation proce- dure, which was described earlier, to obtain the individual's composite score on which classification was based. To compute the vector weights, the total sums of products (SP) matrix was first calculated using the following equation: X'X - 7‘7; where X was a matrix consisting of five occupational scores on each individual in subset A; X' was the transpose of X. K was a matrix consisting of the five average occupational scores for each of the five criterion groups and 7" was the transpose of 71 The results of this equation produced a 5 by 5 matrix. The within group SP matrix was then calculated for each criterion group separately in a manner similar to that described above. The matricies SP1, SP2, . . . SPn 29 were then pooled together to form the within—group SP matrix for the entire set, i.e., SP(W) = SP + SP + SP + SP4 + SP5, where the sub- 1 2 3 script indicates the criteria group. The between-group SP matrix (B) was then calculated by taking the difference of within group SP matrix from the total SP matrix: SP(B) = SP(T) - SP(W). (Overall and Kleth, 1972, p. 45, ex. 2.29) The elements of the within—group SP matrix were then divided by the degrees of freedom, N - I where N was the total number of in— dividuals in a subset and I was the number of criterion groups in the subset. The new matrix W, was the within-group covariance matrix. The inverse of W was then computed by the square root method of matrix inversion. The results might be diagrammed as below: W ' I v' , v" where W is the within-group covariance matrix, I is the identity ma- trix, V' is the upper triangular factor of W such that VV' = W and V'] 1: is the lower triangular square root inverse such that V'V' I. The between-group SP matrix B was then pre and post multi- 1 1 plied by its transpose: V- BV" . The characteristic roots of this symmetric matrix were then identified, A], A2, . . . Av and the asso- 1, Z2 . . . Zn were obtained using the iterative ciated vectors Z method of identifying the principal components. The discriminant function coefficients were obtained by pre-multiplying each of the Z vectors by the triangular square root matrix V'. With the coefficients of the linear functions then identi- fied, it was possible to classify an individual into one of the five 30 criterion groups. For example if the following two linear functions were identified: ' Y1 ‘ C11x1 l C12X2 + C13X3 + C14x4 + C15X5’ Y2 = C21X1 + C22X2 l C23X3 l C24x4 l c25x5, where Yi is the composite score for the i-th function, C is the 13' linear coefficient computed for the i-th function and j-th measure, and Xj is the j-th occupational score derived from either the lambda weights or chi-square weights. Each individual receives two composite scores, Y1 and Y2, by summing the products associated with the indi- vidual's occupational scores and the corresponding vector weights. For example, a social worker will have an occupational score on the social worker measure plus a score on each of the four other occupa- tions in Set I. Each of these scores is multiplied by the appropriate coefficient and summed to give composite scores Y1 and Y2. In addi- tion, the average score on each function is computed for each of the criterion groups, i.e., uki where k indicates the criterion group and 1 indicates the function. To classify an individual, the simple (12 function was-uti- lized, i.e. the sum of the squared deviations of an individual's composite score from the mean composite score of each criterion group was computed. For example, if there were two functions identified and three criterion groups to discriminate then: 2 _ 2 2 d] ‘ (U11 ‘ Y1) + (U12 ' Y2) 9 D. N I 2 2 2 ‘ (921 ' Y1) I (“22 ‘ Y2) : Q. N I 2 2 3 ' (U31 ' Y1) + (U32 ‘ Y2) a where the subscript of d? indicates the associated criterion group. 31 The individual was classified as belonging to that criterion group corresponding to the smallest d2 value. Since the computations as described above would have been a monumental task if done by hand, the discriminant analysis procedure and classification of individuals was calculated by computer. The program by which the discriminant functions were calculated was written by Jeremy Finn, State University of New York at Buffalo, and modified for the computer facilities at Michigan State University by David Wright. The program to compute the d2 function and classification of individuals was written by the author and is included in appendix A. As mentioned earlier, discriminant functions on lambda weights and chi-square weights were both analyzed for similar and dissimilar cri- terion groups. Statistical Analysis To compare the results of the four Kuder OIS scoring pro- cedures, an analysis of variance for mixed models was utilized (see Figure 4)- The dependent variable in the study was the average (across halves A and B) percent of correctly identified individuals for an occupation. The problem of non-independence between sets was solved by using the cross-validation results of half A for optome— trists in Set I and the cross validation results of half 8 in Set II. Thus while the same occupational group, optometrists, appeared in both sets, a different group of individuals was used for respective cross Nalidation studies, making the two sets independent. The fixed inde- pendent variables were: sets of similar and dissimilar occupations, 32 S, lambda weights or chi-square weights, M, and discriminant analysis or not, 0. All three fixed independent variables were completely 'crossed with each other. Occupations was treated as a random independ- ent variable which was nested within S, with five levels per nest but crossed with the two scoring procedures 0 and M. 00000 01 LOCONO‘ 00000 10 , . Figure 4. Data matrix for a mixed model analysis of variance. Although occupations were not actually selected at random from a larger pool of occupations, it was felt that by using the Cornfield-Tukey bridge argument (Cornfield-Tukey, 1956) the results of this study could be generalized to all similar occupations. Had occupations been treated as fixed, greater power would have resulted in the analysis since the individual test respondent would have been 33 the unit of analysis rather than the occupation. The results of such a test, however, would have been limited to those occupations which were studied and thus would have had very little practical value. 0n the other hand, by treating occupations as random, some power was lost but the inferences which could be made were of greater interest. The hypotheses which were considered in this design included testing for differences in the discriminating accuracy: between sets of occupational groups, between the measures chi-square and lambda techniques and differences between using discriminant analysis or not. In addition interaction effects between measures and discriminant analysis as well as interaction effects with sets were also tested. Each hypothesis considered was tested for statistical significance ata= .05. CHAPTER IV RESULTS Scoring keys, using the four strategies discussed in the previous chapter, were developed on both half A and half 8 for each set. Since the two subsets, A and B, were independent of each other, data were available for cross-validation purposes. The keys developed on half A were used to score the responses of individuals in half 8, while the keys developed on half B were used to score the responses of individuals in half A. This procedure was followed with both sets of data and the effectiveness of each strategy was estimated from the cross-validated data. For the multiple discriminant analysis procedure, coeffi- cients for the best linear combination of the occupational scores were obtained by the procedure described in the previous chapter. Four latent roots were identified for each half of Set I and Set II, using the occupational scores based on lambda and on chi-square occupational scores separately. The associated eigenvectors were then utilized to compute the composite scores of individuals in the cross-validation sample of the data. Classification of these individual scores based on the simple d2 statistic was then made for each half of both Set I and Set II. 34 35 The results of each cross-validated half of each set based on the four scoring techniques being considered are presented in Tables 1 through 16. For each criterion group (row) the percentage of individuals classified as belonging to each of the occupational groups (columns), is shown. The main diagonal elements of Tables 1 through 16, indicate the percentage of the individuals correctly identified as belonging to their occupational group. The off-diagonal elements, however, indicate the percentage of the particular criterion group who were classified as belonging to one of the four other occu- pational groups and are considered as errors. These tables provide some interesting information on the stability of the four scoring procedures. Since half A and half B were obtained by randomly dividing each occupational group into two halves, the percentage of correct classifications for an occupation would be expected to be approximately equal across halves. Thus looking at the absolute values of the differences in the diagonals between halves gives some indication of the stability of each scoring strategy. The absolute values of the differences between subsets for the four scoring procedures are presented in Table 19 as well as the means, variances and standard deviations of the absolute values. Using these data as an estimate of the stability associated with each scoring strategy, an analysis of variance for mixed models was com- puted to test for differences between techniques, (see Figure 5). The dependent variable for the analysis was the absolute value of the differences between subsets. The fixed independent variable was the Table 1. The percentage of males from each criterion group (row) in Set I half A classified into each of the five occupational groups (columns) using the lambda scoring key derived on half 8 of Set I. Actual Occppation TEST INDICATED OCCUPATION Optome- X-ray Pediatri- Physical Veteri- trist Technician cian Therapist narian Optometrist 63.05 7.88 12.32 11.33 5.42 X-ray Technician 7.35 52.94 15.44 17.65 6.62 Pediatrician 3.21 8.72 71.56 11.47 5.05 Physical Therapist 11.73 26.26 12.85 45.81 3.35 Veterinarian 9.50 7.50 15.00 9.00 59.00 Table 2. The percentage of males from each criterion group (row) in Set I half 8 classified into each of the five occupational groups (columns) using the lambda scoring keys derived on half A of Set I. Actual Occupation TEST INDICATED OCCUPATION Optome- X-ray Pediatri- Physical Veteri- trist Technician cian Therapist narian Optometrist 62.56 9.36 11.33 11.82 4.93 X-ray Technician 3.62 55.80 7.97 27.54 5.07 Pediatrician 6.91 11.98 63.59 13.36 4.15 Physical Therapist 12.14 18.45 13.59 53.88 1.94 Veterinarian 8.67 10.20 12.76 7.14 61.22 Table 3. The percentage of males from each criterion group (row) in Set 1 half A Classified into each of the five occupational groups (columns) using the multiple discriminant analysis procedure based on lambda scores derived on half B of Set I. Actual Occupation TEST INDICATED OCCUPATION Optome- X-ray Pediatri- Physical Veteri- trist Technician cian Therapist narian Optometrist 54.19 6.40 9.85 9.36 20.20 X-ray Technician 8.09 23.53 3.68 33.09 31.62 Pediatrician 10.09 4.13 42.20 18.35 25.23 Physical Therapist 12.85 26.82 7.82 36.31 16.20 Veterinarian 8.50 8.50 8.00 3.50 71.50 Table 4. The percentage of males from each criterion group (row) in Set I half 8 classified into each of the five occupational groups (columns) using the multiple discriminant analysis procedure based on lambda scores derived on half A of Set I. Actual Occupation TEST INDICATED OCCUPATION Optome- X-ray Pediatri- Physical Veteri- trist Technician cian Therapist narian Optometrist 50.74 4.93 5.91 16.26 22.17 X-ray Technician 5.80 37.68 5.80 31.88 18.84 Pediatrician 5.99 5.99 46.08 21.20 20.74 Physical Therapist 14.56 13.59 9.71 48.06 14.08 Veterinarian 8.67 3.02 7.14 8.67 72.45 1 Table 5. The percentage of males from each criterion group (row) in Set I half A, classified into each of the five occupational groups (columns) using the chi-square scoring keys derived on half B of Set I. Actual Occupation TEST INDICATED OCCUPATION Optome- X-ray Pediatri- Physical Veteri- trist Technician cian Therapist narian Optometrist 60.10 4.43 18.23 6.90 10.34 X-ray Technician 10.29 44.12 16.18 11.76 17.65 Pediatrician 3.67 4.59 69.27 5.50 16.97 Physical Therapist 8.38 19.55 22.91 36.31 12.85 Veterinarian 3.00 1.00 7.50 1.50 87.00 Table 6. The percentage of males from each criterion group (row) in Set I half B, classified into each of the five occupational groups (columns) using the chi-square scoring keys derived on half A of Set I. Actual Occupation TEST INDICATED OCCUPATION Optome- X-ray Pediatri- Physical Veteri- trist Technician cian Therapist narian Optometrist 60.10 4.43 14.78 9.36 11.33 X-ray Technician 10.14 42.03 7.97 17.39 22.46 Pediatrician 4.15 4.61 61.29 6.91 23.04 Physical Therapist 19.42 7.77 16.99 39.81 16.02 Veterinarian 6.12 O 3.06 1.02 89.80 Table 7. The percentage of males from each criterion group (row) in Set I half A, classified into each of the five occupational groups (columns) using the multiple discriminant analysis procedure based on the chi-square scores derived on half 8 of Set I. Aetua' TEST INDICATED OCCUPATION Occupation Optome— X-ray Pediatri- Physical Veteri- trist Technician cian Therapist narian Optometrist 62.56 8.87 10.84 12.32 5.42 X-ray Technician 9.56 55.88 6.62 21.32 6.62 Pediatrician 5.05 9.17 57.80 15.60 12.39 Physical Therapist 7.26 23.46 10.06 54.75 4.47 Veterinarian 6.50 4.00 7.00 6.00 76.50 Table 8. The percentage of males from each criterion group (row) in Set I half 8, classified into each of the five occupational groups (columns) using the multiple discriminant analysis procedure based on the chi—square scores derived on half A of Set I. Aetua' TEST INDICATED OCCUPATION Occupation Optome- X-ray Pediatri- Physical Veteri- trist Technician cian Therapist narian Optometrist 67.00 5.91 9.36 9.36 8.37 X-ray Technician 13.04 50.00 9.42 15.94 11.59 Pediatrician 9.68 10.14 57.14 5.99 17.05 Physical Therapist 18.45 23.30 13.11 31.07 14.08 Veterinarian 8.16 3.06 6.12 2.04 80.61 40 Table 9. The percentage of males from each criterion group (row) in Set II half A, classified into each of the five occupational groups (columns) using the lambda scoring keys derived on half 8 of Set II. Actual Occupation TEST INDICATED OCCUPATION Clinical Auto Optome- Social T Psychologist Mechanic Forester trist Worker Clinical Psychologist 70.00 .40 4.00 6.40 19.20 Auto Mechanic .67 79.33 9.33 9.33 1.33 Forester 3.47 5.20 76.88 10.98 3.47 Optometrist 10.00 5.00 8.50 73.00 3.50 Social Worker 18.22 .44 2.67 12.44 66.22 Table 10. The percentage of males from each criterion group (row) in ‘ Set II half B, classified into each of the five occupational groups (columns) using the lambda scoring keys derived on half A of Set II. Actual Occupation TEST INDICATED OCCUPATION Clinical Auto Optome- Social Psychologist Mechanic Forester trist Worker Clinical Psychologist 72.40 0 2.40 10.40 14.80 Auto Mechanic .67 82.67 10.00 5.33 1.33 Forester 6.36 6.36 79.19 5.20 2.89 Optometrist 5.00 4.50 5.00 75.00 10.50 Social Worker 14.16 2.21 1.33 8.41 73.89 41 Table 11. The percentage of males from each criterion group (row) in Set II half A, classified into each of the five occupational groups (columns) using the multiple discriminant analysis procedure based on the lambda scores derived on half B of Set II. Actual Occupation TEST INDICATED OCCUPATION Clinical Auto Optome- Social Psychologist Mechanic Forester trist Worker Clinical Psychologist 46.80 .40 4.00 17.20 31.60 Auto Mechanic 0 86.00 8.00 4.00 2.00 Forester 5.20 10.40 76.30 5.20 2.89 Optometrist 15.50 6.00 4.00 70.00 4.50 Social Worker 21.25 3.98 1.77 6.64 66.37 Table 12. The percentage of males from each criterion group (row) in Set II half B, classified into each of the five occupational groups (columns) using the multiple discriminant analysis procedure based on lambda scores derived on half A of Set II. ACtua' TEST INDICATED OCCUPATION Occupation Clinical Auto Optome- Social Psychologist Mechanic Forester trist Worker Clinical PsychologiSt 68.27 1.20 3.21 4.02 23.29 Auto Mechanic .67 80.00 6.67 8.67 4.00 Forester 4.62 19.08 61.27 13.87 1.16 Optometrist 10.00 10.50 9.50 51.50 18.50 Social Worker 36.44 4.44 4.89 17.33 36.89 42 Table 13. The percentage of males from each criterion group (row) in Set II half A, classified into each of the five occupational groups (columns) using the chi-square scoring keys derived on half B of Set II. Actual Occupation TEST INDICATED OCCUPATION fl Clinical Auto Optome- Social Psychologist Mechanic Forester tristf Worker Clinical Psychologist 86.80 4.40 1.60 .40 7.20 Auto Mechanic .67 94.67 2.67 1.33 .67 Forester 9.83 43.35 42.77 3.47 .58 Optometrist 22.00 26.50 4.50 40.00 7.00 Social Worker 32.89 8.00 2.22 1.78 55.11 Table 14. The percentage of males from each criterion group (row) in Set II half B, classified into each of the five occupational groups (columns) using the chi-square scoring keys derived on half A of Set II. Actual Occupation TEST INDICATED OCCUPATION Clinical Auto Optome- Social Psychologist Mechanic Forester trist Worker Clinical ' Psychologist 86.80 4.00 2.00 1.60 5.60 Auto Mechanic .67 97.33 1.33 .67 O Forester 6.36 46.82 43.35 1.16 2.31 Optometrist 20.50 31.00 3.00 38.00 7.50 Social Worker 33.63 7.93 3.10 2.65 52.65 43 Table 15. The percentage of males from each criterion group (row) in Set II half A, classified into each of the five occupational groups (columns) using the multiple discriminant analysis procedure based on the chi-square scores, derived on half B of Set 11. Aetua' TEST INDICATED OCCUPATION Occupation ' Clinical Auto Optome- Social Psychologist Mechanic Forester trist . Worker Clinical Psychologist 66.80 .40 3.60 2.40 26.80 Auto Mechanic 0 82.00 6.67 7.33 4.00 Forester 1.73 8.67 76.30 8.67 4.62 Optometrist 2.50 5.00 7.50 69.50 15.50 Social Worker 34.67 1.33 3.56 11.11 49.33 Table 16. The percentage of males from each criterion group (row) in Set II half B, classified into each of the five occupational groups (columns) using the multiple discriminant analysis procedure based on the chi-square scores, derived on half A of Set 11. Aetua' TEST INDICATED OCCUPATION Occupation _ Clinical Auto Optome- Social Psychologist Mechanic Forester trist Worker Clinical Psychologist 76.00 .40 1.60 6.40 15.60 Auto Mechanic 0 84.67 8.00 4.67 2.67 Forester 2.89 10.40 76.30 7.51 2.89 Optometrist 3.50 4.50 4.00 78.00 10.00 Social Worker 26.11 2.65 2.65 11.50 57.08 44 Table 17. The absolute value of differences between percent of indi- viduals correctly identified as belonging to their actual occupational group in half A minus correctly classified in half B for each of the four scoring strategies studied in Sets I and II. Set I Discriminant Discriminant Chi Lambda With Chi Square With Lambda Square Optometrist 4.44 3.45 0 .49 X-ray Technician 5.88 14.15 2.09 2.89 Pediatrician .66 3.88 7.98 7.97 Physical Therapist 23.68 11.75 3.50 8.07 Veterinarian 4.11 .95 2.80 2.22 Average Absolute Value of Differences 7.75 6.84 3.27 4.32 Variance of Absolute Values 82.94 32.12 8.64 25.24 Standard Deviation 9.11 5.76 2.94 5.02 Set II Clinical Psychologist 9.20 21.47 0 2.40 Auto Mechanic 2.67 6.00 .34 3.34 Forester 0 15.03 .58 2.31 Optometrist 8.50 18.50 2.00 2.00 Social Worker 7.75 29.48 2.46 7.67 Average Absolute Value of Differences 5.62 18.10 1.08 3.54 Variance of Absolute Values 26.40 74.21 3.32 12.28 Standard Deviation 5.14 8.61 1.82 3.50 45 four scoring strategies studied M, while the random independent variable was the five occupational groups in a set, 0. N 0000 0143-0) Figure 5. Data matrix for the test of equality of scoring consistency. This model assumes no interaction between the random and fixed independ- ent variables as well as an equal pairwise correlation for levels of the fixed independent variable. The computed F ratios were .33 and 17.13 for Set I and Set II respectively. Thus while differences in the stability of scoring strategies were not found to be statistically sig— nificant among homogeneous occupational groups, statistically signifi- cant differences in the stability of the scoring procedures among heterogeneous occupations were identified at a = .05. Reviewing the mean absolute difference values for the scoring strategies in Set II. indicated that discriminant analysis with lambda occupational scores was the least stable scoring technique of the four procedures tested. Moiser (1951) suggested that the average of the two cross- validated halves be used as an estimate of the "true" effectiveness of a particular scoring strategy. Following his suggestion, the average percent of individuals identified as belonging to each of the 46 occupational groups was computed. The results of this computation are presented in Tables 18 through 25. The percent of individuals correctly identified using the lambda procedure ranged from 67.58 (pediatricians) to 49.85 (physical therapists) in Set I (see Table 18), and 81.00 (auto mechanics) to 70.06 (social workers) in Set II (see Table 22). Using the discriminant analysis technique based on the lambda scores, the percent correctly identified ranged from 71.98 (veterinarians) to 30.61 (x-ray technicians) in Set I (see Table 19) and 83.00 (auto mechanics) to 51.63 (social workers) in Set II (see Table 23). Based on the chi-square weights, how- ever, the percent correctly identified ranged from 88.40 (veteri~ narians) to 38.06 (physical therapists) in Set I (see Table 20) and 96.00 (auto mechanics) to 39.00 (optometrist) in Set II (see Table 24). And finally, with the discriminant analysis procedure based on the chi-square weights the precentage of correct identi- fications ranged from 78.56 (veterinarians) to 42.91 (physical theratpsts) in Set I (see Table 21) and for Set 11 correct identi- fications ranged from 83.24 percent (auto mechanics) to 53.21 per- cent (social workers), see Table 25. To facilitate a comparison of the effectiveness of the scor— ing strategies the elements on the main diagonals of Tables 18 through 25 were arranged together in one table. Since primary interest is with the number of correct identifications, the location of the main distractors for each technique is of little importance. Thus the Table 18. Averages of percentages of males from each criterion group (row) in Set I half A and Set I half B classified into each of the five occupational groups (columns) in Set I using the lambda weights. Actual Occupation TEST INDICATED OCCUPATION - Optome- X-ray Pediatri- Physical Veteri- trist Technician cian Therapist narian Optometrist 62.81 8.62 11.83 11.58 5.18 . X-ray Technician 5.49 54.37 11.71 22.60 5.85 Pediatrician 5.06 10.35 67.58 12.42 4.60 Physical Therapist 11.94 22.36 13.22 49.85 2.65 Veterinarian 9.08 8.85 13.88 8.07 60.11 Table 19. Averages of percentages of males from each criterion group (row) in Set I half A and Set I half B classified into each of the five occupational groups (columns) in Set I using the multiple discriminant analysis procedure based on lambda occupational scores. Actual Occupation TEST INDICATED OCCUPATION Optome- X-ray Pediatri- Physical . Veteri- trist Technician cian Therapist narian Optometrist 52.47 5.67 7.88 12.81 21.19 X-ray Technician 6.95 30.61 4.74 32.49 25.23 Pediatrician 8.04 5.06 44.14 19.85 22.99 Physical Therapist 13.71 20.21 8.77 42.19 15.14 Veterinarian 8.59 5.76 7.57 6.09 71.98 48 Table 20. Averages of percentages of males from each criterion group (row) in Set I half A and Set I half B classified into each of the five occupational groups (columns) in Set I using the chi-square weights. Actual Occupation TEST INDICATED OCCUPATION Optome- X-ray Pediatri- Physical Veteri- trist Technician cian Therapist jpnarian Optometrist 60.10 4.46 16.51 8.13 10.84 X-ray Technician 10.21 43.08 12.08 14.58 20.06 Pediatrician 3.91 4.60 65.25 6.21 20.01 Physical Therapist 13.90 13.66 19.95 38.06 14.44 Veterinarian 4.56 .50 5.28 1.26 88.40 Table 21. Averages of percentages of males from each criterion group (row) in Set I half A and Set I half B classified into each of the five occupational groups (columns) in Set I using the multiple discriminant analysis procedure based on chi- square occupational scores. Actual Occupation TEST INDICATED OCCUPATION Optome- X-ray Pediatri- Physical Veteri- trist Technician cian Therapist narian Optometrist 64.78 7.39 10.10 10.84 6.90 _eray Technician 11.30 52.94 8.02 18.63 9.11 Pediatrician 7.37 9.66 57.47 10.80 14.72 Physical Therapist 12.86 23.38 11.59 42.91 9.28 Veterinarian 7.33 3.53 6.56 4.02 78.55 49 Table 22. Averages of percentages of males from each criterion group (row) in Set II half A and Set II half 8 classified into each of the five occupational groups (columns) in Set II using the lambda weights. Actual Occupation TEST INDICATED OCCUPATION Clinical Auto Optome- Social Psychologist Mechanic Forester trist Worker Clinical Psychologist 71.20 .20 3.20 8.40 17.00 Auto Mechanic .67 81.00 9.67 7.33 1.33 Forester 4.92 5.18 78.04 8.09 3.18 Optometrist 7.50 4.75 6.75 74.00 7.00 Social Worker 16.19 1.33 2.00 10.43 70.06 Table 23. Averages of percentages of males from each criterion group (row) in Set II half A and Set II half B classified into each of the five occupational groups (columns) in Set II using the multiple discriminant analysis procedure based on the lambda occupational scores. ._' r? v—fi Actual Occupation TEST INDICATED OCCUPATION Clinical. Auto Optome- Social Psychologist Mechanic Forester trist Worker Clinical Psychologist 57.54 .80 3.61 10.61 27.45 Auto Mechanic .34 83.00 7.34 6.34 3.00 Forester 4.91 14.74 68.79 9.54 2.03 Optometrist 12.75 8.25 6.75 60.75 11.50 Social Worker 28.85 4.21 3.33 11.99 51.63 50 Table 24. Averages of percentages of males from each criterion group (row) 'hi Set II half A and Set II half B classified into each of the five occupational groups (columns) in Set II using the chi-square weights. Actual Occupation TEST INDICATED OCCUPATION Clinical Auto Optome- Social Psychologist Mechanic Forester trist Worker Clinical Psychologist 86.80 4.00 1.80 1.00 6.40 Auto Mechanic .67 96.00 2.00 1.00 .34 Forester 8.10 45.09 43.06 2.32 1.45 Optometrist 21.25 28.75 3.75 39.00 7.25 Social Worker 33.26 7.98 2.66 2.22 53.88 Table 25. Averages of percentages of males from each criterion group (row) in Set II half A and Set II half B classified into each of the five occupational groups (columns) in Set II using the multiple discriminant analysis procedure based on the chi-square occupational scores. Actual Occupation TEST INDICATED OCCUPATION Clinical Auto Optome- Social Psychologist Mechanic Forester trist Worker Clinical Psychologist 71.40 .40 2.60 4.40 21.20 Auto Mechanic 0 83.34 7.34 6.00 3.34 Forester 2.31 9.54 76.30 8.09 3.76 Optometrist 3.00 4.75 5.75 73.75 12.75 Social Worker 30.39 1.99 3.11 11.31 53.21 51 results in Table 26 summarize the average percent correct identifica- tions made by each of the four scoring strategies in each criterion group in both sets. To evaluate these results an analysis of variance for mixed models was utilized. It should be recalled, however, that since op- tometrists appeared in both similar and dissimilar occupational groups, the two sets were non—independent. To resolve this problem only the cross-validated results of half A for optometrists, using each of the scoring strategies, were considered in Set I; while only the cross- validated results of half B for optometrists were considered in Set II. Thus rather than presenting the average correct identifications as found in Tables 18 through 25, Table 26 contains for optometrists the cross validated results of half A for Set I and half B for II. The results Of the mixed model analysis of variance are presented in Table 27. Although the unweighted average percent of individuals correctly classified in Set I was 56.37; while Set II had 68.41 percent correct classifications, no statistically significant difference between sets was identified, p <.15. The difference in percent correct identifications between sets was however, in the pre- dicted direction. That is, a greater percentage of the individuals from the set of heterogeneous occupational groups were correctly clas- sified than individuals classified among the homogeneous occupations. The unweighted average percent of individuals correctly identified when multiple discriminant analysis procedures were utilized was 60.62; while non-use of this technique produced 66.66 percent correct classi- fications. This difference, however, was not found to be statistically 52 Table 26. The average percent of individuals correctly classified into their actual occupational group from half A and half B for Set I and Set II. Discriminant Non—Discriminant Analysis Analysis Chi-Square Lambda Chi-Square Lambda Weights Weights Weights Weights Set I ' Optometrist 62.54 54.19 60.10 63.05 X-Ray Technician 52.94 30.61 43.08 54.37 Pediatrician 57.47 44.14 65.28 67.58 Physical Therapist 42.91 42.19 38.06 49.85 Veterinarian 78.56 71.98 88.40 60.11 Set II Clinical Psychologist 71.40 57.54 86.80 71.20 Auto Mechanic 83.34 83.00 96.00 81.00 Forester 76.30 68.79 43.56 78.04 Optometrist 78.00 51.50 38.00 75.00 Social Worker 53.21 51.63 53.88 70.06 53 Table 27. ANOVA table for mixed models, analyzing the data from Table 14. - Sources Degrees of Means F P Freedom Squares S 1 1449.86 2.51 .15 0:8 8 578.07 0 1 126.59 1.47 .26 l 48.44 .48 .52 SD 1 28.06 .33 .59 SM 1 85.73 .85 .39 DM 1 625.84 3.55 .10 SDM 1 76.95 .44 .53 00:5 8 85.97 OM:S 8 100.90 ODM:S 8 176.51 54 significant, p <.26. The difference in percent correct identifications between use and non-use of the discriminant analysis technique was in the opposite direction from what had been expected. These results may have been due, however, to the lack of stability of discriminant func- tion in the cross-validated data as was presented earlier. A comparison of the measures, lambda vs. chi-square showed that for the former an unweighted average of 61.29 percent of the in— dividuals were correctly classified; while for the latter an average of 63.49 percent correct classifications were made. The null hypothesis of no difference between measures was not rejected, p <.57. Testing the interaction of discriminant analysis by measure indicated that an unweighted average of 65.67 percent correct classifications was made with the discriminant analysis based on chi-square scores; 55.56 per- cent correct classifications with discriminant analysis using lambda occupational scores, 61.32 percent correct classifications,with the chi-square scoring technique used alone and 67.03 percentcorrect classifications when the lambda coefficients were used alone. Again the analysis did not indicate statistically significant differences, p <.10. Finally no interaction effects with sets, S, were identified for various levels of p as indicated in Table 27. A further consideration in deciding which scoring technique was the most effective was consistency of accuracy with which the pr0* cedures correctly classified individuals across occupations. A technique which correctly classified individuals in one or two occupa- tions at a very high rate but classified individuals in other occupa- tional groups at low rates, may not, in the long run be as valuable 11 1 I I’ll" 55 as a procedure which consistently classified individuals at a moderately high rate over all occupations. Thus variability in the rate of cor- rect classifications might be an important aspect associated with a procedure when evaluating the scoring strategies. Using the data in Table 26, the equality of variance corresponding to each scoring pro- cedure was tested using Levene's test of homogeneity of variances for both Set I and Set II. The means, variances, standard deviations and means of the absolute values of deviations from the means associated with Levene's test, for both sets of data are presented in Table 28. The null hypothesis of equality of variance among the four measures was not rejected for Set I at a=.05. Using the same test with Set II, however, the null hypothesis of equal variances was rejected at a=.05. In addition Scheffe's post hoc technique (see Table 28) indicated that the chi-square procedure was more variable than the lambda technique and the discriminant procedure with chi-square weights at a=.05. Thus, at least for dissimilar criterion groups, the chi-square technique seems to correctly identify individuals to their actual occupational group less consistently than the other techniques. 56 Table 28. The mean, variance, standard deviation and absolute error difference for each scoring strategy used in Levene's test for equality of variance across the four measures. Set I Discriminant Discriminant Chi- Lambda With Ch1-Square With Lambda Square Average 58.89 48.62 58.98 58.99 Variance 173.29 240.62 319.40 57.84 Standard Deviation 13.16 15.51 17.87 7.61 Average 5' 9.33 11.57 14.73 5.51 Set II Average 72.45 62.49 63.65 75.06 Variance 133.88 175.89 547.89 21.02 Standard Deviation 11.57 13.26 23.40 4.58 Average 5' 8.12 10.72 22.20 3.57 Scheffé post hoc test statistic for comparing specific average error differences. “MW/(J ') FJ 1 (—J l)(I- 1) \I 5w {(3) (3.49) \[(35.03) (g) 1 (3.24) (3.74) m 1 12.13 :l—I CHAPTER V DISCUSSION The previous chapter presented the results of the development and cross-validation of several scoring keys for the Kuder 015. The analysis of variance test which was computed failed to reject the null hypothesis of no difference between the four scoring strategies in discriminating individuals among both similar and dissimilar occupa- tional groups. The lambda technique, however, had the highest per- centage of correct classifications for optometrists, x-ray technicians, pediatricians and physical therapists among the similar occupational groups and amongdissimilar occupations, the procedure had the highest rate of correct classification for foresters and social workers. On the other hand, the chi—square technique had the highest correct classification rate for veterinarians among the similar occupations and clinical psychologists and auto mechanics among dissimilar occu- pations. The discriminant analysis technique using the chi—square occupational scores had the highest rate of correct classification for optometrists among dissimilar occupations. The discriminant analysis technique using the lambda occupational scores had the lowest rate of correct classifications of the four techniques considered in four out of five occupations for the similar occupational groups and lowest rate of correct classifications in three out of five occupations 57 58 among dissimilar occupations. Thus the lambda weighting procedure correctly identified individuals as belonging to their actual occu- pational group at the highest rate in six of ten occupations, the chi- square technique in three of ten, the discriminant analysis technique with chi-square occupational scores in one of ten, and the discrimin— ant analysis technique with lambda occupational scores did not have the highest rate of correct classification in any of the occupations studied. Furthermore, Levene's test (for the equality of variance among j groups) indicated that while the null hypothesis of equal variances in percentage of correct classification was not rejected for I, there was a statistically significant difference in the vari- ability of correct classification among the four scoring procedures with Set 11. Using Scheffe's post hoc technique the lambda procedure was shown to be significantly less variable than the chi-square pro- cedure at a =.O5. Since the lambda and chi-square procedures did not differ on average accuracy, this greater uniformity across dissimilar occupations for the lambda technique is particularly important. The conclusion of no difference between scoring procedures is in disagreement with that made by Loadman (1971) when a similar comparison was made. It should be recalled, however, that Loadman did not generate his own set of scoring keys, using the lambda procedure, but rather had obtained the item weights from SRA. These weights were derived from the total population of respondents on whom the researcher had hoped to classify. Thus an independent set of data for cross-validation purposes was not available which may have resulted in an inflated percentage of correct identifications. In addition, 59 with twice as many individuals to build the scoring key another ad- vantage was given to the Kuder lambda procedure in Loadman's study. Therefore, an undetermined level of bias could have entered the data in favor of the lambda technique, thus making the results of his study questionable. An estimate of the bias which had entered Loadman's data is indicated in Tables 29 and 30. In these tables the average percent of individuals correctly classified in each occupational group, using the quasi-cross-validated scoring keys (Loadman 1971, p. 114) and the cross-validated scoring keys of the present study, are presented. The third column of each table shows the magnitude of the differences be- tween the quasi and true cross-validation procedures for each occupa- tion. The effect of quasi-cross-validation varied from one occupation to another, but in general the quasi-cross-validation percentages of correct classifications were considerably greater than the corres- ponding true cross-validation percentages. On two occasions however, the true cross-validated results produced considerably higher rates of correct classification than the quasi-cross-validated results: 1) Set I for physical therapists using the discriminant analysis tech— nique (see Table 29), and 2) Set II for social worker using the lambda technique without the discriminant function (see Table 30). To ex- plain these results, it might be suggested that since both of these occupational groups had a considerably lower rate of correct classi- fication than the other occupations within the same set and scoring technique, it is possible that some error existed in the scoring keys used by Loadman. Loadman had indicated several problems in obtaining 60 Table 29. The average rate of correct identifications for each occu- pational group in Set I using the cross-validated and quasi- cross-validated data. Quasi-Cross-Validated Cross-Validated Lambda Lambda Difference Optometrists 69.46 63.05 6.41 X-ray Technicians 66.78 54.37 12.41 Pediatricians 70.18 67.58 2.60 Physical Therapists 56.96 49.85 7.11 Veterinarians 59.75 60.11 -.36 Quasi-Cross-Validated Cross-Validated Discriminant Function Discriminant . With Lambda Scores Function With DIfference Lambda Scores Optometrists 67.98 52.47 15.51 X-ray Technicians 62.01 30.61 31.40 Pediatricians 66.98 44.14 22.84 Physical Therapists 20.99 42.19 -21.20 Veterinarians 79.75 71.98 7.77 61 Table 30. The average rate of correct identifications for each occu- pational group in Set II using the cross-validated and quasi-cross-validated data. f Ouasi-Cross-Validated Cross-Validated Lambda Lambda D'fference Clinical Psychologists 83.40 71.20 12.20 Auto Mechanics 80.98 81.00 -.02 Foresters 80.93 78.04 2.89 Optometrists 80.50 75.00 5.50 Social Worker 55.75 70.06 -14.21 Quasi-Cross-Validated Cross-Validated Discriminant Function Discriminant . With Lambda Scores Function With D1fference Lambda Scores Clinical Psychologists 80.60 57.54 23.06 Auto Mechanics 84.00 83.00 1.00 Foresters 81.50 68.75 12.71 Optometrists 75.50 51.50 24.00 Social Worker 64.38 51.63 12.75 62 the scoring keys from SRA, and these problems may not have been cleared up completely for his analysis. A further indication of the effect of using quasi-cross- validated data on the results of the study might be to compare ANOVA table for the cross-validated results with the ANOVA table for the quasi-cross-validated results. Such a comparison is made in Table 31. It should be noted that the only difference in data which were used to produce the ANOVA tables was that Loadman's quasi-cross-validated results form the data for the other ANOVA table. In both analyses no statistically significant effects were identified, but a review of the p values in both tables (column 3) indicates that the quasi-cross- validated data produced significant levels considerably lower than the cross-validated data. In particular, the significance level for measures (lambda vs chi-square) which was the effect of prime interest in this study, was much lower for the quasi-cross-validated data than for the cross-validated data. Thus although in this present study the Same conclusions would be drawn from either cross-validated or quasi- cross-validated data, the results of a comparison of the two ANOVA tables indicates that quasi-cross-validated data can inflate the actual differences which could, in some cases produce erroneous decisions. It should be noted that in obtaining these results, the analysis of variance procedure for mixed models was utilized which is in one sense an extremely conservative test of the research findings. For the analysis, occupations were treated as random, but if occupa- tions had been fixed the data would have been evaluated using 63 Table 31. ANOVA tables for cross-validated and quasi-cross-validated data. ANOVA for Cross-Validated Data Sources MS F P S 1449.86 2.51 .15‘ 0:3 578.07 0 126.59 1.47 .26 48.44 .48 .52 SD 28.06 .33 .59 SM 85.73 .85 .39 DM 625.84 3.55 .10 SDM 76.95 .41 .53 00:5 85.97 OM:S 100.90 ODM:S 176.51 ANOVA for Quasi-Cross—Validated Data S 1468.10 2.33 .17 0:8 630.82 0 24.07 .20 .67 397.09 2.70 .14 SD 151.44 1.27 .29 SM 90.21 .61 .46 DM 78.48 .46 .54 SDM 3.11 .02 .89 00:5 118.97 OM:S 146.83 ODM:S 172.34 64 individual respondents rather than occupations as the unit of analy- sis, resulting in a substantial increase in degrees of freedom. Fur- thermore, with occupations treated as random an additional source of variation is included in the E(MS) for both the numerator and denomin- ator of the F ratio. The result produces a reduced quotient and thus a conservative estimate of the effect of the factor being tested. Had occupations been treated as fixed, greater power would have re- sulted in the analysis, but the findings would have been limited to those occupations which were studied and thus would have had very little practical value. On the other hand, by treating occupations as random some power was lost, but inferences which could be made were of greater interest. It might be expected that individuals from similar occupa- tions would make similar responses to the instrument, thus making discrimination among similar groups more difficult than among dis- similar groups where interests are likely to be completely different. The average percent of individuals correctly classified in Set I, the similar occupations, was 56.37 while in Set II, the dissimilar occu- pations, had an average rate of 68.41 percent correct classifications. Thus although the results indicated a higher rate of correct classi- fications among dissimilar occupations, the difference was not found to be statistically significant. The statistical conclusion then was that accuracy in discriminating individuals among similar occupations was as good as discrimination made among dissimilar occupations. Although Loadman (1971) had cited a factor analytic study by Shutz and Baker (1962) to argue that the occupations selected were 65 of two types: a group of similar occupations and a group of dissim- ilar occupations, the study's findings may have been due to the fact that the group of similar and dissimilar occupations selected were not actually representative of the total population of similar and dissim- ilar sets. In other words, several other explanations for the results may be offered: the similar occupations studied were unusually easy to discriminate for similar occupations; the dissimilar occupations were exceptionally difficult to discriminate for dissimilar occupations or a combination of both. Thus some confounding between sets may have caused the results of the study. A better test of the discriminating accuracy of the techniques between sets might be a comparison of the accuracy of discrimination for a single occupation that appears in both sets. Such a comparison was possible with optometrists. Among the homogeneous occupations, Tables 1 through 8, optometrists were correctly identified an average of 60.04 percent of the time, while among heterogeneous occupations, Tables 9 through 16, 61.87 percent were correctly identified. This comparison suggested that for the techniques investigated, discrimination among similar occupations was, practically speaking, as accurate as among dissimilar groups. The use of multiple discriminant analysis was an attempt to improve upon the accuracy of both the lambda and chi—square techniques of scoring the Kuder 015. The analysis of variance test computed indicated that there was no statistically significant difference be- tween use or non-use of the discriminant function. Furthermore, the finding of no interaction between discriminant analysis or not and the measures, (lambda and chi-square), (DM), indicated that the effect 66 of using discriminant functions was the same for both chi-square and lambda techniques. When the non-rejection to these two null hypotheses were considered together, the results indicated that there was no dif- ference between the chi-square weights when used alone and when the chi-square weights are used with the discriminant functions. A similar conclusion can be drawn for the lambda weights. Although there was no statistically significant DM interaction, the means in Table 32 suggest some rather sizable differences. Virtually no difference was Table 32. Averages of percent of individuals correctly identified in the two sets under each of the four scoring strategies considered. Discriminant Function Non-Discriminant Function Chi-Sguare Lambda Chi-Square ‘ Lambda Set I 59.89 48.62 58.98 58.99 Set II 73.45 62.49 63.65 75.07 found in Set I between the average correct identifications made by the chi-square weights alone and the chi-square weights used in the discriminant function. In Set II, however, an apparent improvement was shown when the discriminant function was used over the chi-square weights alone. With the lambda technique, a decrease in accuracy resulted from using the discriminant function in both Set I and Set 11. These results seemed to indicate that among similar occupations the additional computations which are necessary when calculating the dis— criminant functions do not improve the accuracy of discrimination 67 over that obtained when chi-square weights are used alone. Consid- erable improvement in accuracy was shown among dissimilar occupations, however, when the discriminant was used. The lambda technique on the other hand, was more effective in discriminating individuals in both sets when used alone rather than when using the discriminant function. 68 Summary and Conclusions The measurement of interests has long been a major concern among both psychologists and educators. Although improvements have been made in the development and scoring of interest inventories, research continues in search of the optimal instrument. The quantita- tive scoring aspect of interest surveys has received a great deal of attention, but the best technique for a given instrument has yet to be identified. The present study compared the effectiveness of four scoring strategies for the Kuder Occupational Interest Survey form 00: Lambda coefficients, the procedure currently used by the test publisher; chi- square weights for item responses as suggested by Porter; and two applications of multiple discriminant analysis procedures utilizing occupational scores generated by (a) lambda coefficients and (b) chi- square weights. Scoring keys were developed on a set of similar and a set of dissimilar occupational groups following the procedures described by each technique. An independent set of data was used for cross- validation purposes. The accuracy of each scoring key was determined by the percent of the males in each criterion group that were correctly identified to their actual occupational group in the cross-validated sample. It should be pointed out that the data used in this study were old, (originally collected by Kuder for the development of the instrument in 1956) and the scoring keys developed may be meaningless for classifying individuals today. The data did provide the necessary information, however, for a valid test of the four scoring strategies considered. 69 A mixed model analysis of variance was used to test the hypothesis of no difference between scoring strategies on the depend- ent variable of percentage of correctly classified males. Independent variables were sets (similar-dissimilar), measures (lambda-Chi-square), and discriminant analysis (discriminant analysis or not) which were fixed and completely crossed, and occupations which was random and nested within sets, but crossed with measures and discriminant analysis. The results of the study indicated that among similar occu- pations 56.37 percent of the individuals were correctly classified while among dissimilar occupations 68.41 percent correct classifica- tions were made. This difference was not statistically significant at a =.05. The test for a difference between the use of discriminant analysis procedures against the non-use of this technique indicated that while 60.62 percent correct classifications were made by the former and 66.66 percent correct classifications were made by the latter, the null hypothesis of no difference was not rejected at a =.05. A comparison of the measures, lambda vs chi-square indicated that an average of 61.29 percent correct classifications were made with the lambda technique while an average of 63.49 percent correct classifications were made with the chi—square procedure. This dif- ference between measures was not found to be statistically significant at a = .05. In addition, 65.67 percent correct identifications were made with the discriminant analysis technique using the chi-square occupational scores; 55.56 percent of the individuals were correctly 7O classified using the discriminant analysis technique with the lambda occupational scores; 61.32 percent correct classifications were made with the chi-square weights alone and 67.03 percent of the individuals were correctly classified using the lambda coefficients as the scoring technique. Although some differences seem to exist, the null hypothe- sis of no interaction between measures and discriminant analysis was not rejected at a = .05. In conclusion the results of this study showed that no one scoring procedure offered significantly greater accuracy than the other three procedures for classifying individuals into their appro- priate occupational group. The study suggested other aspects of classification, however, which should also be considered in choosing the "best" scoring strategy for the Kuder 015. One of these factors was the variability in the rate of correct classification across several occupations. Using Levene's test for equal variances it was pointed out that while no difference in variability of procedures was found with the homogeneous occupations, statistically significant differences in procedures were identified in the variability of cor- rect classifications among heterogeneous occupations. Post hoc tests indicated that of the four scoring techniques studied, the least variable strategy was the use of lambda coefficients, and the most variable was the chi-square technique. An additional factor to be considered before choosing which strategy to use is related to difficulty of computation. This aspect of developing scoring strategies has its biggest effect on the use of multiple discriminant analysis. Although the task has been made 71 considerably easier with the introduction of high speed computers, the results of this study indicated that the extra effort is not worth— while. Finally, it was pointed out that a disadvantage of the chi- square technique is the fact that its scoring weights are dependent upon the group of occupations being considered. Thus it is possible to obtain different item response weights depending upon how many and which occupations are considered. For every set of occupations to be discriminated among a new set of item response weights must be gener- ated even when occupations are common across sets. Thus when all of these factors are taken into consideration in deciding which of the four techniques studied should be selected for scoring and classifying individuals on the Kuder 015, the best decision appears to be the lambda technique. Although not having a statistically significant advantage in average accuracy over the other three strategies studied, the direction of difference in average ac- curacy favored the lambda technique in both sets. Furthermore, in developing the scoring keys only one occupational group is considered at a time. Thus once developed the item weights remain the same re- gardless of which occupations are being compared. Finally the rate of correct classifications using the lambda technique was remarkably stable across several occupations, especially in the heterogeneous set. As far as future research is concerned it must be pointed out that the scoring problem has not yet been solved. Research should continue, attempting to improve the accuracy of occupational 72 discrimination for interest surveys. While for the moment, Kuder's lambda technique seems to be the "best" of the four scoring strategies considered here, other techniques should be tested, most notably pattern analytic procedures. Investigations developing and improving this approach to the scoring problem of interest surveys are needed and would be considered as an important contribution to this area of measurement. In addition to investigating different approaches to scor- ing the Kuder instrument, research should also turn to developing new more discriminating test items. Thus although the scoring procedure is an important factor in determining the effectiveness of the in- strument, the items themselves should not be neglected. An interesting study might be to look at the rate of correct classifications in a given occupation when it is considered with sets of similar and dissimilar occupations. The present study compared the rate of classification in Set I and Set II for optometrists but several more such comparisons are needed before the effect of similar and dissimilar sets can be determined. Finally, further research with the lambda coefficient should be encouraged. While Kuder and SRA have chosen one technique for utilizing the lambda ratio, based on the 100 responses selected by an individual, other techniques may be possible. Another approach might be to calculate lambda coefficients for each of the 600 possible re- sponse patterns with an individual's similarity to a particular group estimated by the sum of the 100 lambda ratios associated with the in- dividual's selected responses. 73 To calculate the lambda coefficient for each response pat— tern one criterion group at a time is considered. Each individual in the criterion group receives an agreement score based on the propor- tions of the total group selecting each of the response patterns. The individual's agreement score is simply the sum of the 100 proportions associated with his responses to the instrument. In computing the point biserial and maximum point biserial correlations, the continuous variable would be the agreement scores of the individuals in the cri- terion group. The dichotomous variable would be the selection or non- selection of a particular response pattern being considered. The lambda coefficient for a particular response pattern is calculated by computing the ratio of the point biserial to the maximum point biserial correlations. The point biserial correlation is calculated between the vector of N (the number of individuals in the criterion group) 1's and 0's, corresponding to selection or non-selection of the re- sponse pattern by the N individuals, and a vector of the N agreement scores. The maximum point biserial correlation is computed as if the n individuals (the number of individuals actually selecting the re- sponse pattern) having the highest or lowest agreement scores had selected the response pattern under consideration. The computational formula for the lambda coefficient asso- ciated with a particular response pattern can be presented as: Y -1(_ Am: _9 xt. ><| where xij is the computed lambda coefficient for item i and response pattern j, Y'is the overall average agreement score for the total 74 criterion group, Xg uals selecting response pattern j for item i, and ii is the average agreement score for the n individuals having the highest or lowest is the average agreement score for those n individ- agreement scores depending on whether the numerator is positive or negative respectively. The sign of the lambda coefficient would be that of the numerator positive if 75 is greater than Y'and negative if 75 is less than 71 Following this procedure then the weight for a particular response pattern for an item could range from +1.00 if those selecting the response pattern of interest had the highest agreement scores of the criterion group, and a lower limit of -l.OO if respondents selecting the response pattern had the lowest agreement scores of the criterion group. This procedure would then be used to calculate the lambda values for each of the 600 possible response patterns for the Kuder 015 and an individual's occupational score would be the sum of the 100 response pattern weights associated with the individual's selected responses. An individual could then be classified as belonging to that criterion group in which he had the highest occupational score. Although Kuder's procedure is capable of accurately dis- criminating individuals at a fairly high rate, further research on improving the discriminating accuracy through item weights should continue. One procedure which may accomplish this task might be the new procedure for scoring individual responses through lambda coef- ficients as described above. BIBLIOGRAPHY BIBLIOGRAPHY Anastasi, A. Psychological Testing 3rd. Ed. The Macmillan Company 1971. Barnard, M. M. 'The secular variations of skull characteristics on four series of Egyptian skulls." Annals of Eugenics, 1935, 6 (part IV), 352-371. Berdie and Campbell, 'Measurement of Interests!‘ In Handbook of Measurement and Assessment in Behavioral Sciences, Whitla D.K. Ed. Reading Mass., Addison-Wesley, 1968. Bryan, J. G. 'A Method for the Exact Determination of the Character- istic Equation and Latent Vectors of a Matrix With Applica- tion to the Discriminant Function for More Than Two Groups." Cambridge Mass., University Graduate School of Education 1950, Unpublished Doctoral Dissertation. Chappell, J. S. "Multivariate Discrimination Among Selected Occupa- tional Groups Utilizing Self Report Data." Unpublished Doctoral Dissertation, Purdue, 1967. Clark, J. A. "Criterion Pattern Analysis: A Method for Identifying Predictive Item Configurations." Unpublished Doctoral Dis- sertation, Michigan State University, 1968. Clark, K. E. Vocational Interests of Non-Professional Men. Minne- apolis: University of Minnesota Press, 1961. Clemens, W. V. "An Index of New Criterion Relationships.“ Educational and Psychological Measurement. 1958, 18, l. Cornfield, J. and Tukey, J. "Average Values of Means Squares in Factorials." Annals of Mathematical Statistics, 27, 1956, 907-949. ' Cowdery, K. M. "An Evaluation of the Expressed Attitudes of Members of Three Professions." Unpublished Doctoral Dissertation, Stanford University, 1925. Findley, W. "A Rationale for Evaluation of Item Discrimination." Educational and Psychological Measurement, 16, 175-180. 75 76 Fisher, R. A. "The Use of Multiple Measurements in Taxonomic Problems." Annals of Eugenics. 1936, 7, 179—188. Fisher, R. A. "The Statistical Utilization of Multiple Measurements." Annals of Eugenics. 1938, 13, 376—386. Freyd, M. "The Personalities of Socially and the Mechanically In- clined." Psychological Monograph. 33, 1924. Fryer, D. The Measurement of Interests. New York: Henry Holt and Co., 1931. Gaier, E. L. and Lee, M. C. "Pattern Analysis: The Configural Approach to Predictive Measurement." Psychological Bulletin. 1953, 50, 140-148. Kelley, T. L. Statistical Method. New York: Macmillan, 1923. Kornhauser, A. W. "Results From a Quantitative Questionnaire on Likes and Dislikes Used With a Group of College Freshmen." Journal of_Applied Psychology. 1927, 11, 85-94. Kuder, G. F. "A Comparative Study of Some Methods of Developing Occupational Keys." Educational and Psychological Measure- ment. 1957, 17, 105-114. . Kuder Preference Record Occupational, Form 0. Manual ChiCago: SRA. 1961. . "A Rationale for Evaluating Interests," Educational and Psychological Measurement. 1963, 23, 3-10. . "The Occupational Interest Survey." Personnel and Guidance Journal. S, 1966, 45, 72-77. . "A Note on the Comparability of Occupational Scores From Different Interest Inventories." Measurement and Evaluation Guidance, 2. (2) Su, 1969, 94-100. . Kuder Occupational Interest Survey Form 00. General Manual, Chicago: SRA. 1970. Lipsett, L. and Wilson, J. "00 Suitable Interests and Mental Ability Lead to Job Satisfaction." Educational and Psychological Measurement. 1954, 14, 373-380. Loadman, W. E. "A Comparison of Several Methods of Scoring the Kuder Occupational Interest Survey." Unpublished Doctoral Dis- sertation, Michigan State University, 1971. McRae, G. McQuitty, 77 G. "The Relationships of Job Satisfaction and Earlier Measured Interests." Unpublished Doctoral Dissertation, University of Florida, Gainesville, Florida, 1959. L. L. "Elementary Linkage Analysis for Isolating Orthogonal and Oblique Types and.Typal Relevancies." Educational and Psychological Measurement. 1957a, 17, 207-229. "Isolating Prediction Patterns Associated With Major Cri- terion Patterns." Educational and Psychological Measure- ment. 1957b, 17, 3-42. "Job Knowledge Scoring Keys by Item Versus Configural Analysis for Assessing Levels of Mechanical Experience." Educational and Psychological Measurement. 1958, 18, 661-680. "Differential Validity in Some Pattern Analytic Methods." . In Bass and Berg I.A. Objective Approaches to Personality Assessment. Princeton 0. Van Nostrand Co. Inc., 1959. ("Hierarchial Linkage Analysis for the Isolation of Types." . Educational and Psychological Measurement. 1960, 20, 55—67. "A Method for Selecting Patterns to Differentiate Cate- gories of People." Educational and Psychological Measure- ment. 1961a, 21, 85-94. "Item Selection for Configural Scoring." Educational . and Psychological Measurement. 1961b, 21, 925-928. Meehl, P. Mosier, C. Nunnally, "Single and Multiple Hierarchial Classification by Re- ciprocal Pairs and Rank Order Types." Educational and Psychological Measurement. 1966, 26, 253-265. E. "Configural Scoring." Journal of Consulting Psychology. 1950, 14, 165-171. Clinical vs. Statistical Prediction. Minneapolis: University of Minnesota Press, 1954. 1. "Problems and Design of Cross Validation." Educational and Psychological Measurement. 1951, 11, 5-11. J. C. Introduction to Psychological Measurement. New York: McGraw-Hill, 1972. Overall, J. E. and Klett, C. J. Applied Multivariate Analysis. New York: McGraw-Hill, 1972. Porter, A. Rao, C. R. Rao, C. R. Tatsuoka, Tiedeman, 78 C. "A Chi Square Approach to Discrimination Among Occu- pations.Using an Interest Inventory." Technical Report No. 24, University of Wisconsin Center for Cognitive Learning, Madison, Wisconsin, 1967. "Utilization of Multiple Measurements in Problems of Biological Classifications." Journal of Royal Statistical Society. 1948, 10, 159-193. and Slater, P. "Multivariate Analysis Applied to Differ- ences Between.Neurotic.Groups." British Journal of Psychology. (Statistical Section) 1949, 2, 17-29. M. M. Multivariate Analysis. New York: Wiley, 1971. and Tiedeman, D. V. "Discriminant Analysis." Review of Educational Research. 1954, 24, 402-420. D. V. and Roulon, P. J. and Bryan, J. G. "The Multiple Discriminant Function: A Symposium." Harvard Educational Review. 1951, 21, 71-95. ' and Bryan, J. G. "Prediction of College Field of Concen- Whitla, D. tration." Harvard Educational Review. 1954, 24, 122-139. K. (Ed) Handbook of Measurement and Assessment in Be- havioral Sciences. Reading, Mass., Addison-Wesley, 1968. APPENDICES . .—_——~ C -— C - -- COMPUTE- PDOPORTIONS- --- —— APPENDIX A _ _-. * ___.._____.._,.-_ . ... Computer program for calculating lambda coefficients . . _-___.___-- PROGRAM DR IVERI INPUT OOUTpUTQ TAPE 1 ‘_ TAPE2 o TAPE-:60: INPUT O-TAPE6130UTPUT _ +1 COMMON /TOTALS/TOTALSI1000695) DO 10JI=194 DO 5 =105 5 CALL LAMBDAIJ) "'WRITEII)TOTALS 1000 FORMATIioFBOS) .ENDFILE 1 1o CONTINUE ENDFILE 1 END , SUBROUTINE LAMBDAINGRDUP) PEAL IS! 1M. ITOT ”INTEGER RESP DIMENSION RPI1009610PII100061 DIMENSION ITOT(10006) DIMENSION IPESI100)9XNIZSOIOLI6 COMMON /TOTALS/ TOTALSIIOOO605) INITIALIZE ITOT ~‘DO"If1"I.-:‘I”'o“100 "‘ ’ _ ..-.— ‘.—..._ -.. -..— .000 A...__.-—. ....- ——-.- ...- )3HI6) DO 10 J=106 10 ITOTIIOJ1=OOO _ DEAD IN DATA. NpEOpLE- a '1” 12 NPEOPLE-NPEOPLE+1 --~13 CONTINUE ' C C _--—-—_n~--—--- ,-..-_-_._.____. ----.-- -..-N—_—--.—---_.‘_u_o.-o— READI2o1ooOIIDEs Iooo FORMATI6X064I1/6X036111 IFIEOFIp))7.Is ,“TOTAL-INFTHIS-CARDI$_DATA _______ ,u 18 CONTINUF DO 18 Jeioloo MJJ=1RFSIJT““”‘*“_'” C -C “—..—--..- —.__ -__--_..—_.—_ -.. --.._-—_————-—-——————————-_-—--_-_-——— .- ———_——_——.__—--~_.--— _—--_-.—_-_‘--——--————————_---————. F'SflJfl'mi’Zo TO 18 I 0 F06 16 I PRINT QQQONPEOPLEONGROUPOJCJU 999 FDRMATI%9**LAMBDS PERSON*I4* OF - ~-+*9I39*l= *0201 ~- 60 TO 13 IR CONTINUF' ‘DO 50 J: IVTOU'“ .1- ——._ ‘w' ... ------—-—.4—c—-———-‘-.-——-_-_~.—-- --—-———_—-_—._-—-_——.-~—_—.————---—-_——--— .-—-———. _.___—-—-—- GROUP*13* HAS BAD DATA. RESPONSE I _-——- ~...... .-pu-_———--.‘-_ .-_—-- :0 ITOTIJOIRESIJ11=ITOTIJOIRESIJ -.-—.--..‘.._.__---.----—--—~-—-_—-_—_.-~- c AND CALCULATE ppos SUM=OOO _—.—_..,..— .- P 1.11:4 TOT (-1... 11+1 ’----.—_.—_-—-———._-_-—--——-_--_ --.—..._-.-—----_—————'---_—-—- .jI/SUMIITDTIIJIHIJaLIEE __________________ “—“"““—DO“530“J=TTIOD IFIJ. GT.I)GO TO 505 - ISUM= Don _ DO 500 K-1 6 ' ' ""’ ‘ ISUM=IsuM+ITOTIJoKI - “500 CONTINU: , u _~- — ." ——-—-—_- -- -”_-_“_“_, u-" u-“-n 505 Do 510 K=196 — .-__—_-.-.--. _—._- --_.-_. K1=IITOTI ULJ.LLLJ;4L o I o 9 9 o o 0 U 06 60 T IFIPPIJ K1oGTo GO TO 570 O O m N O OLKD - 5m XMAX=RPIJ0K1"“‘“"‘”‘ —~ ~— 520 CONTINUE SUM: SUM+XMAX 53o CONTINUE .... COMPUTE LAMBDA VALUES DO 550 J=19100 DO 540 K=106 TOTALSIJOKO NGPOUP1-IPPIJ0K1’06671/(SUM‘66071 540 CONTINUF 550 CONTINU: RETURN END 1: 2*PIIJo11+PIIJo21+ _2*PIIJ021+PIIJ011+ 80 ___._ --....— -—-—-— —. J0K11/IISUM1 __u_ U, ' ..__-——..._-4-__ _._._-—.... (J (J 1+PIIJ041+ (J (J ———----—- ---—.....—-__~ pI PI PI 1+PIIJv21+pI 1+PIIJ031+PITU )+PI(J011+PIIJ . QI)_FOR EVERY ND SUM UP__qSUM:XT O 51%. ——————————————————————————————————— xMAx1Go TO 516 ———._-_-- —-——- ._-. ._—_.__—.-—_-_._ ,---,,-__ _‘......_...-. H.” “—.-—...— -..-..__.-«_- -..-..-.-o....._ APPENDIX B .—-————- Computer program for classifying individuals based on their Lamb a .-Coefficient total scores...“ I ..-.-5- ...--. .. ”am ”—..-—.w-“ ..-...- - -..- - —..._.._..~__ ._. PROGRAM DRIVERIINPUTQOUTPUTOTAPEIQTAPE2QTAPEBQTAPEéOlePUToTAPEél= +OUTPUT1 COMMON /TOTALS/TOTALSI10096951 1000 FORMATI1DF8.51 ._ REWIND P REWIND 1 ~ “ READI I ITOTALS ' " ‘ ’ '” ”’ ‘ "” " w“”‘"‘”““““‘"" ""“‘”“"““"‘“‘"“”'_'“-"“"“"' "“‘"‘”‘“ JK= 3 ‘ DO 60 I210 . -_ _L __,_W-__U_L ,, ‘2 _________ 60 CALL CLASIEYIIQ JKI REWIND .READII1DUMMY 4~ - . ,,,E,, 444-... --.- h_ .- _ I--qum-r READIIIDUMMY O=EOFII1 ' " READI I IDUMMY‘ ' """ T "“'“""""" " "’ ‘— “ ‘“ "““““' READII1DUMMY O=EOFI I) REAOI11TOTALS JK=4 DO 70 I=IQS 70 CALL CLASIFYIIOJK1 END QUBPOUTINE CLASIEYIIGPQJKI wmwm”"’"“‘“”””* ”W““"“”“”‘”“““”‘”“““m” COMMON /TOTALS/ TOTALSI10006951 DIMENSION MIIOO1OSCOPEI51 DIMENSION NNIPI1 DATA KT/I/ DATA NN/OQ + 406.678.1114. 1472. 1872. 2278. 2554. 2988. 3400.3792.4292. 4592. + 493845338. 5790. 6290.659o.6936. 7336.7788 / _ 1- o ,MHHM“_ 2__22_q"“_nra.l_h. KT: KT+1 KG=NNIKT1-NN(KT-11 _. _,_“_2_ u__ -J“_, _U_5 KKK:KT-1 PQINT 1.169 1 FORMAT(*IGROUD NO.*I?/*0INDIVIDUAL*T35*GROUP1*T44*GROUP2*T53*GROUP +3*T62*GDOUP4*T71*GROUPS*1 WDITEI3.41KG.KKK 4‘FORMATI16.141 - '7“ r-W ~“~- ~---~ J-~“T~T*~~~T~~-—”--—~—~-~ N1=0 N2=0 __ ,,W -__ .ln__ N3=O N4=0 N5=0 DO 10 DEADIz ' IFIMIS 100 FORMAT IFIEOF 20 DO 30 J SCOREIJ1=0 DO 40 K=1.loo .-n W ,, or .,, 1, 5_.WE SCOPEIJ)=$COREIJ1+TOTALSIK4MIK1 oJ1 40 CONTINUC 30 CONTINUE “'“ ‘ - *-~-"-""" -~TT~"T-~TT~~T7TT”T*T*”‘“""~“”7‘ IH=1 HIGH: SCOREI11 _ _ _ _ ,H_g___q DO 25 J: 2. 5 IFIHIGH, GFoSCOPEIJ11 GO TO 25 H163: SCOREIJ1 A a , , I E V 1H: co-ro mv ---—--~-- ~---» ~ —- —~ 5 ~- — ~—-—r~~-< — -—-~-5—~-- /6X936II1 020 ”v 1:1 910 0). (5X I21 :1 RI 1N0: -,,7_ 5 , W,., ”,5 a .__;._H E__-. _ rid- H.EO.11N1=N1+1 H.En.21N2= N2+1_ HQEO031N3= N3+1 . A , 4,, ,. ._ 1. . ____ HoEOod1N4=N4+I H.En.51N5=N5+1 ""“PRINT 2. 1.!GR.5CORE.IH*W*7* --***'*7‘ 7*“ '“ '7777'"“——"" ”_____ 2 FORMATI*91NDV.*14* OF GROUP NO.*13*. SCORES: *5F9.4' HIGHEST CORRES +DONDENCF wITH GROUP ¥**112o***1)'_ WRITEIB. 315CORF.I.IGR. JK ‘ ‘ 3 FORMATI=F9.4o5X.*INDIV.*14* GROUP*14* HALF*I21 10 CONTINUE , . . I” -H_-_II_M-M,75-__E CALL EXIT 600 1:1-1 '— PRINT 1500.! "'5' —L~-—"-"~ ~-~-"~w~*—-—-~———~-~—~— 1600 FORMATI*0*I4.* RESPONDENTS WERE FOUND*1 PRINT 1601. N1.N2. N3. N4. N5 1601 FORMATI*OGROUR1*0140* GROUP2*.I44* GROUP3*oIAo* GROUP4*9I44* GROUP +5*.141 END 1' I IH I I I .._. --_-- ..-—....“ -m —« APPENDIX C Computer program for Calculating Chi Square weights. -.-. ... _ , __-,. I - ..-..._.. ._ .... - .I,....,L ..-—..- _I__ .I -..—.... _. -—___._..—-—_.. —— PROGRAM DRIVEP(TAPE 33oINPUTOOUTPUTOPUNCHoTAPEéo-INPUToTAPEéI OUTP +UTQDISK. TAPE21=DISKI ‘ DIMENSIQN ANRI697OIOOI9 IDATA(IOO)0 NA(5I DO 999 NIM=104 < - I v . n -._ -- . 7,. .w- w H-L DO 60 K210100 DO 60 J:IO7 _ 00‘60 1:1 .6 " ' ‘ ““ ”W" W“ ‘ “”“"“”” ““‘"““’""“ “ ““““""““““"‘““ 50 ANRII.J.KI=OOD C PRESET ARRAY , ________ _ -_, _‘-_ C 1=RROFESSIONo J=PESPONREQ K=ITEM '- ”'fln' ’U‘H' 7"" DO 31 1:1 5 QEAD(33.1492INR n -7- - “Hue“,w A-“ ~«h '- - I- ._H 149? FORMATIIBI DO 32 M=Io NR ' "'READI33,18I(IDATA(KIUK=IrIOOI““'"m“M—"——“*“—““““"_""_.“”*““'““‘r 18 FORMAT(6X96411/6Xo36III DO P K=10100 _ J=IDATA(KI 2 ANR (IcJoKI=ANR(I9JoKI+IoO 3? CONTINUE 31 CONTINU: DEADI33.149?IDuMMY -“‘ " IFIEOFtfl3)9106601067"‘ ""‘““*”‘“*“**“h“““*“"“"“"“'“““'“"“““"*" I067FMINT1068 1058 FORMAT(* NO EOF FOUND*I STOP ’ ’ ”" " "“ I066 CONTINUE C FOUND OBSERVED VALUES 71 DO 5 K=10100 DO 5 J '“DOES ~_—..-_..-.._-_._..._.. ..-, ANRIOQJ J.K)+ANR(I.J K) 0 , a n p 2 D m =ANP(IO7OKI+ANRIIQJQKI _ CHI F OBSEQVED VALUES. ROW TOTALSQCOLUMN TOTALS . -.—- .-.. . _--- ...... __.- ...I..._ .-q— M-M.-.—~_.-_._.H_ ”—...—... -“—.‘.—-._ n :23 u 3 ID —-|-o OOHKD0L~K0~ LR quu~quuuguu II I! II -{o r-O—bx. ”—...... ...... C COMPUTE DO I? NRgIO7oKII/AN9(6979KI I - I.-- I 8. 909 ANQ( 1‘s). ANR( 19‘). GO TO I? ANR(I¢J. K CONTINuc c~ COMPUTED cHI SQUARE wEIGHTs-wITH-SIGN-—*nw~u~u-u«u»m“w~-» C PUT THEM RACK IN ANR 00 I7 K=10100 . PRINT ?n 1 .K ' ‘ "“‘ “’“' "“' ' '“ P01 FORMATIIHDoéH ITEM 0I3I ‘ DO 16 1:1. I. J KKI-EXP)**2/EXP * ' ' - »‘ .-. ._MM OJ 0 KI*SIGN unto- V“ 00m PRINT 51OIOIANPIIOJOKIOJ=106I 53FDRWUW134PRWTBSHNIOIUQXngoaI - “ WRITEI210301I(ANR(IOJOKIOJ=106IOIONIM 301 FORMATI6F9040?OX01193X 011) 16 CONTINUE 17 CONTINUE coo CONTINUE . END ' ‘ "wm— APPENDIX D . _,..,_ . ..- _ ..-, . ... _ -... . 7--.. . ‘ -——— . .__ o. _.--.-—. _-—-- -.. ...—..-m-" -—.—— , .kw-p—yu—_ —-— Computer program for classifying individuals based on their Chi Square ~total scores. ....— __.. _._.-,_..,-- —‘_.—.-.-. - wH-c—u-p..--—_ ._ ---—_‘-— .-fi- .— pROGRAM SCOREIINPUTOOUTPUTOPUNCHQTAPE 330TAPEGO: -INPUTQ TAPE6130UTPU +ToTAPEép=PUNCH5 C TOTAL SCODES FOP SET P HALF B USING WEIGHTS FROM HALF A ' DIMENSION ANRHI596e10030IDATA(lDO)QTSA(5)0NA(5) DO 2 K=19100 DO 2 I=195 * p PEAD 72. (ANPB(I0J9K10J=1'¢6)‘ --- ...-..-..-_-__,.-..M ' ———- ~~—---—— ~~--—— '7? FORMATIfiFgoaI C WEIGHTS ARE IN __ _ _ ______ PRINT 8 8 FORMAT(12H PROFESSION 0.3X3HONE06X03HTWOQ SXQSHTHREEQ4X04HFOUR +=XQ4HFIVE) OO 31 L: 105 PFAD(33. BOINQ 30 FORMAT(13) ' " ' ' ‘ "m "“e ' “""w "--*-~--~-~ --..-- ~— -- u—“w-v *—- -‘----~«—7 ~—~—-- PRINT SnIoL 301 FORMAT {/13H PROFESSION oIl) KOUNT= O DO 32 leoNR DO 11 Jelos 11 TSAIJ)=OOO READ(33.1)(IDATAIK’0K=19100) “*' 1 FORMAT (6X964I1910X/6X936I1) DO 6 1:105 DO 6 K=IOIOO _ 7 . rm _ 7 _ _r‘w_d J=IDATA(K) 6 TSA¢I>=TSAIII+ANRB(IIJQK) KOUNT ‘5 KOUNT+I HIGH=-909990 DO ?‘5 1:105 ‘IF(TIGH.GEOTSA(I))GO‘TO“2S”"*““”“M“'“‘““*““"“"*“~“*qu*"~-'W-“‘—~ IH‘ ’ HIGH: TSAII) 7_ 7 H _ 2S CONTINUF ' ‘ ' ‘ '” ‘ "" " " “ '"h “ PRINT 7. KOUNTOLQTSAQIH 7 FORMAT(§QINDIVIDUAL*14* OF GROUP NO.*I3*0 _SCORES 3*5F904* HIGHEST» +CORRESPONDFNCF WITH GROUP*5XQI2) PUNCH 9. (TSA(I)9I=105)QKOUNTOL " O FORMAT(=F9o4(SXQ*INDIV0*I4* GROUP*Ia)~~--~-Mm ~—~WN---~“-~w~ _—--7 3? CONTINU: 31 CONTINUF’ V q _ _, -.- 7 __ END ...-_~--.-r--_ -i n--_-.—... 84 APPENDIX E . «nu-.....- .— ”-..—.....h M.» . ...-.. ._ . duh, ; a -z.fli . . I Computer program for computing the Simple d statlstlc and the classification of individuals based on discriminant-functionscores. -—-~—-—-———--— -- .-..._.____,__.,__ ._ _—....i .—-..——-.—-r. EYEEQEI$N52Q§9VEIE.4I.Y1(250.5).Y2(250.5).Y3<250o5)9Y4(2Eo.5).MU(5' +o4)oS(5)oKT(5) - PEAL MU u u -. - -- -_ --- ._H a m --U COMPUTE EACH PEPSONOS scope FOR ALL 4 FUNCTIONS. THEN FIND MU FOP FACH FUNCYION WHEPE MY IS AVEPAGE OF ALL PERSONS IN THE GROUP. THIS SURPOUTINP DOES 1 DATA“SET“(Ech'2-AI*AT—A-TIME'*“~-~*“-"w»~- ~~~ PEAD 10C (N100 DO 100 169:1;5 ‘ ' “ ”** '”‘“"T ~ v-~ -~w-wiu— ........... ---i_uiu_n 1 FORMATI:F9.4I DO 200 INDIV=Iolooo PEAD lgq ’ ‘ "“ ~ ‘~“-v~--—--A - -H_____” a-”_. IFIEOFIAO))150¢2 P'YIIINDIVOIGRI=C(I013*SIT7+C<21T3*512)+C€3fifl*593fl+C+fiV+4*S(4)+C(Sr*~ +1)*S(S) -Y2*S<3>+C<492?f5i4)+C£5J._ ...—.——. “-— +?)*S(5) Y3IINDIVOIGR)=C(103)*S(1)+C(203)*S(2)+C(303)*S(3)+C(493)*S(4)+C(So +3)*S(‘331 - -—- -----~~~- - ~-----—-‘ AM----—-~--~- -. - YAIINDIVQIGR)=C(194)*SI1)+C(294)*S(2)+C(304)*S(3)+C(404)*S(4)+C(Ro +1)*S(5) “ " 200 CONTINUF‘”"" “"""""‘ “‘ ““””““”“"""'““"“ "‘”““’““ 150 CONTINUF INDIV=IMDIV~I , , _-_ - _ _ MUCIGPo1)=MU(I69025=MUIIGRQBI=MUII69043=O ' ” ”‘“ “ KT(IGR)=INDIV DO 160 [=19INDIV» ~ -~- --- - —---———__—____-.----.----...-- .i -i---_i--i..-----_-______-_ MUIIGR01)=MU(IGR¢1)+Y1(10IGR) MUIIGQO?)=MU(I6997)+Y2(IOIGR) * M“ - 'MU(IGR¢11=MU(IGPQBI+YBIIoIGQ)*”“"“”“‘*“‘*“"“““ MUIIGRod)=MU(IGQQ4)+Y4(IOIGR) 160 CONTINUE _ , _ I- -m- -_i___,-_”-_‘”i__"_u__ _i- - -_ MU(IGR01)=MU(IGRQII/INDIV MU